Issue with Spark 1.5.1: es-hadoop "Connection error (check network and/or proxy settings)- all nodes failed"

Hi,
I compiled Spark 1.5.1 with Hive and SparkR with the following command:
mvn -Pyarn -Phive -Phive-thriftserver -PsparkR -DskipTests -X clean package

After its installation, the file "hive-site.xml" has been added in Spark's conf directory (this is not an hard copy, it's a symbolic link). I also created a new directory in Spark which includes the elasticsearch-hadoop.jar connector ($SPARK_HOME/elastic/jar/elasticsearch-hadoop-2.1.1.jar).
In order to start the spark-shell with the connector, I'm using the following command:
spark-shell -v --jars /opt/application/Spark/current/elastic/jar/elasticsearch-hadoop-2.1.1.jar --name elasticsearch-hadoop --master yarn-client --conf spark.es.net.ssl=true --conf spark.es.net.http.auth.user=username --conf spark.es.net.http.auth.pass=password --conf spark.es.nodes=loadbalancer.fr --conf spark.es.port=9200 --conf spark.es.field.read.empty.as.null=true --conf spark.es.resource=index/type

Once Spark fully loaded and the SparkContext properly created, I can successfuly do:
scala> import org.elasticsearch.spark._
import org.elasticsearch.spark._

scala> val RDD = sc.esRDD("index/type/", "?q=Ge*")
RDD: org.apache.spark.rdd.RDD[(String, scala.collection.Map[String,AnyRef])] = ScalaEsRDD[0] at RDD at AbstractEsRDD.scala:17

scala> RDD.count
res1: Long = 39

The result returned by "RDD.count" is  correct and is the same as the one I can get when I do a curl request
curl -XGET -k "https://loadbalancer.fr:9200/orange/bank/_search?q=Ge*&pretty" -u username:password

Nevertheless, if I try to get the document sources (with the es-hadoop connector), I'm getting the following errors:
scala> RDD.collect().foreach(println)
[Stage 0:>                                                          (0 + 2) / 5]15/11/04 09:43:17 ERROR TaskSetManager: Task 1 in stage 0.0 failed 4 times; aborting job
org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 0.0 failed 4 times, most recent failure: Lost task 1.3 in stage 0.0 (TID 9, node07.fr): org.apache.spark.util.TaskCompletionListenerException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[node01.fr:9200, loadbalancer.fr:9200, node02.fr:9200, node03.fr:9200, node04.fr:9200, node05.fr:9200, node06.fr:9200, node07.fr:9200]]
        at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:90)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1283)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1271)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1270)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1270)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697)
        at scala.Option.foreach(Option.scala:236)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:697)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1496)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1458)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1447)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:567)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1822)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1835)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1848)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1919)
        at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:905)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
        at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
        at org.apache.spark.rdd.RDD.collect(RDD.scala:904)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:21)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:26)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:28)
        at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:30)
        at $iwC$$iwC$$iwC$$iwC.<init>(<console>:32)
        at $iwC$$iwC$$iwC.<init>(<console>:34)
        at $iwC$$iwC.<init>(<console>:36)
        at $iwC.<init>(<console>:38)
        at <init>(<console>:40)
        at .<init>(<console>:44)
        at .<clinit>(<console>)
        at .<init>(<console>:7)
        at .<clinit>(<console>)
        at $print(<console>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
        at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340)
        at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
        at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
        at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
        at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
        at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
        at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
        at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
        at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
        at org.apache.spark.repl.Main$.main(Main.scala:31)
        at org.apache.spark.repl.Main.main(Main.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.spark.util.TaskCompletionListenerException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[node01.fr:9200, loadbalancer.fr:9200, node02.fr:9200, node03.fr:9200, node04.fr:9200, node05.fr:9200, node06.fr:9200, node07.fr:9200]]
        at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:90)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

This issue also happens when the "hive-site.xml" is deleted. I can reproduce the issue at 100%. The only way to get the document sources is to deactivate SSL and the authentication through Searchguard. You'll understand that this solution is an unacceptable option.
Also important, there is nothing in the logs related to this error, neither in the loadbalancer nor in the nodes.
Note: I also tried to get the sources with the last beta connector and the result is the same.
Here are the specs of my environment:

Red Hat ES 6.7 x86_64

Spark 1.5.1, Scala 2.10.4, Java 1.7.0_85, Hive 1.2.1

Global authentication done through Kerberos

Elasticsearch 1.6.2, Lucene 4.10.4

SSL activated and authentication done through searchguard

Me too, using hive 1.2.1 loading data from hive to ES,
Hadoop job throws error, and stopped:
2015-11-06 15:40:55,230 ERROR [main] org.elasticsearch.hadoop.rest.NetworkClient: Node [Read timed out] failed (<IP_ADDRESS>:9200); no other nodes left - aborting...
2015-11-06 15:40:55,259 FATAL [main] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"user_id":"492923825","is_register":null,"register_time":null,"phone_number":null,"nickname":null,"gender":null,"us_first_time":null,"us_last_time":null,"us_first_order_time":"2015-07-26 17:12:49","us_last_order_time":"2015-11-03 22:19:09","us_total_order_count":10,"us_success_order_count":5,"us_cash_order_count":5,"us_success_order_price":30000.0,"us_cash_order_price":30000.0,"us_user_count":2,"us_user_phone_count":1,"us_user_address_count":2,"us_ship_order_count":5,"us_avg_ship_hours":16.4,"us_tp50_ship_hours":13.0,"us_avg_order_price":6000.0,"us_tp50_order_price":6000.0,"us_max_order_price":10000.0,"us_avg_cash_order_price":6000.0,"us_order_day_duration":101,"us_order_day_count":5,"us_is_spam":0,"us_spam_score":1.0,"us_spam_score_detail":"1.0 _ 0.0:0.05 , 0.0:0.4 , 0.0:6000 , 0.0:6000 , 1.0:10000.0 , 0.0:13"}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:518)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:163)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[<IP_ADDRESS>:9200, es.op.koudai.com:9200, <IP_ADDRESS>:9200, <IP_ADDRESS>:9200, <IP_ADDRESS>:9200, <IP_ADDRESS>:9200]]
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:142)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:317)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:313)
at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:150)
at org.elasticsearch.hadoop.rest.RestRepository.tryFlush(RestRepository.java:209)
at org.elasticsearch.hadoop.rest.RestRepository.flush(RestRepository.java:232)
at org.elasticsearch.hadoop.rest.RestRepository.doWriteToIndex(RestRepository.java:185)
at org.elasticsearch.hadoop.rest.RestRepository.writeProcessedToIndex(RestRepository.java:164)
at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$EsHiveRecordWriter.write(EsHiveOutputFormat.java:63)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:753)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
... 9 more
2015-11-06 15:40:55,259 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: 3 finished. closing...
2015-11-06 15:40:55,259 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: RECORDS_IN:879999
2015-11-06 15:40:55,259 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0

@vanjor, for my part the ES connector is working on Hive. I only have the issue through Spark.
Here is what I did to make it work with Hive:

create a symbolic link in the Hive conf directory to "spark-default.conf" (eg: spark-defaults.conf -> /opt/application/Spark/current/conf/spark-defaults.conf)

create a directory in Hive which will have the ES connector jar (eg: /opt/application/Hive/current//elastic/jar/elasticsearch-hadoop-2.1.1.jar)

start hive

import the ES connector jar as below:

hive (yolo)> add jar /opt/application/Hive/current/elastic/jar/elasticsearch-hadoop-2.1.1.jar;

create a Hive table wich will do the connection between Hive and ES (its very close to the Hive/HBase connection). Caution, you must set the same table parameters as the mapping used in the index. For example:
Hive parameter1 = account_number BIGINT
Hive parameter3 = firstname STRING

ES mapping1 = account LONG
ES mapping1 = firstname STRING
hive (yolo) > CREATE EXTERNAL TABLE elastic_table_1(account_number BIGINT, balance BIGINT, firstname STRING, lastname STRING, age BIGINT, gender STRING, address STRING, employer STRING, email STRING, city STRING, state STRING) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES ("es.net.ssl" = "true", "es.net.http.auth.user" = "username", "es.net.http.auth.pass" = "password", "es.nodes" = "node01.fr", "es.resource" = "index/type", "es.index.auto.create" = "false", "es.query" = "?q=*");

then you will be able to load you data

hive (yolo) > SELECT * FROM elastic_table_1;

I hope this will help you.

@ArcTheMaster thanks for your help, I was succeeded to load billions of data into ES by using hive, but when I try to update the data into ES, the job will failed after several hours.
Briefly, It okay to loading data into an empty index by using hive, but it will failed in the midway when updating those large scale ES data by using hive.

@ArcTheMaster First off, thanks for the detailed ticket - appreciated.
Interesting to see that count works (that's because it's a straight call to ES) while iterating does not. Can you try 2.1.2 or potentially 2.2.0.beta1? If the issue still occurs please enable logging on the REST package (to TRACE) and upload the gist somewhere.
Likely this has something to do with the SSL/load balancer. By using a load balancer (not sure why you need that but that's a different discussion), you require that all clients have to go through that 'check point'. ES-Hadoop makes direct connections to each shard based on the number of running tasks so in this case, so likely after discovering the shards, the connector tries to access the them but fails (as they are not exposed).
This scenario is supported in 2.2 only through the wan feature as explained here. Do note that this will impact performance significantly.
@vanjor  You are experiencing a typical case of cluster overload. You define a retry of 10K with a wait of 10s in between - no wonder your jobs take hours. Increasing retries simply postpones the job failing if the cluster lags behind - and there are several reasons why this can happen from too many map/reduce jobs to too little resources on the ES side.
There's a dedicated chapter on the topic.
Last but not least, please open a different issue or better yet a topic on the forum if you'd like to continue the conversation. I'd like to keep this ticket on @ArcTheMaster's topic.
@vanjor @ArcTheMaster Guys, I'm glad to see folks helping each other out yet at the same ticket, it looks like two different tickets are being discussed in the same thread. Would you mind opening a new ticket or potentially discussing over the forums?  Reduces confusion for everyone involved.

@costin Thank you for your reply and all the input you provided.
I did a lot of tests today, with multiple ES-Hadoop versions. I enabled logging on the REST package as you said.
I also disabled SSL but the authentication through SearchGuard was maintained. I wanted to check the network traces, particularly the authentication process between the driver and the executors. With SSL activated, this won't be possible.
Here are my global results for each ES-Hadoop connector:

elasticsearch-hadoop-2.1.1.jar (ES-Hadoop 2.1.1)

val RDD2 = sc.esRDD("my_index/my_type", "?q=Ge*")
RDD2.count ===> functional
RDD2.collect().foreach(println) ===> not functional

elasticsearch-hadoop-2.1.2.jar (ES-Hadoop 2.1.2)

val RDD2 = sc.esRDD("my_index/my_type", "?q=Ge*")
RDD2.count ===> not functional
RDD2.collect().foreach(println) ===> not functional

elasticsearch-hadoop-2.2.0-beta1.jar (ES-Hadoop 2.2.0 Beta)

val RDD2 = sc.esRDD("my_index/my_type", "?q=Ge*")
RDD2.count ===> not functional
RDD2.collect().foreach(println) ===> not functional
As you can see, I have a different behaviour between version 2.1.1 and 2.1.2 or 2.2.0. The count was functional before which is not the case now.
In all cases, when the request is not functional I have the following logs:
scala> import org.elasticsearch.spark._
import org.elasticsearch.spark._

scala> val RDD2 = sc.esRDD("my_index/my_type", "?q=Ge*")
RDD2: org.apache.spark.rdd.RDD[(String, scala.collection.Map[String,AnyRef])] = ScalaEsRDD[0] at RDD at AbstractEsRDD.scala:17

scala> RDD2.collect().foreach(println)
15/11/10 15:24:13 INFO CommonsHttpTransport: Using detected HTTP Auth credentials...
15/11/10 15:24:13 TRACE CommonsHttpTransport: Opening HTTP transport to loadbalancer01.fr:9200
15/11/10 15:24:13 TRACE CommonsHttpTransport: Tx [GET]@[loadbalancer01.fr:9200][] w/ payload [null]
15/11/10 15:24:14 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{
  "status" : 200,
  "name" : "loadbalancer01.fr",
  "cluster_name" : "MYCLUSTER.PEI",
  "version" : {
    "number" : "1.6.2",
    "build_hash" : "622039121e53e5f520b5ff8720fdbd3d0cb5326b",
    "build_timestamp" : "2015-07-29T09:24:47Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}
]
15/11/10 15:24:14 TRACE CommonsHttpTransport: Closing HTTP transport to loadbalancer01.fr:9200
15/11/10 15:24:14 INFO CommonsHttpTransport: Using detected HTTP Auth credentials...
15/11/10 15:24:14 TRACE CommonsHttpTransport: Opening HTTP transport to loadbalancer01.fr:9200
15/11/10 15:24:14 TRACE CommonsHttpTransport: Tx [GET]@[loadbalancer01.fr:9200][_nodes/transport] w/ payload [null]
15/11/10 15:24:15 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{"cluster_name":"ELASTIC.PEI","nodes":{"innll3d8QMuo_iYqQCBm_Q":{"name":"loadbalancer01.fr","transport_address":"inet[loadbalancer01.fr/<IP_ADDRESS>:9300]","host":"loadbalancer01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[loadbalancer01.fr/<IP_ADDRESS>:9200]","attributes":{"data":"false","master":"false"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[loadbalancer01.fr/<IP_ADDRESS>:9300]","profiles":{}}},"HuZ-Vpy0QwSTzRkclkMpRg":{"name":"node05.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node05.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node05.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[/<IP_ADDRESS>:9300]","profiles":{}}},"qOaZ0CXsRL-80-LnbEHk1Q":{"name":"uabigname01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"uabigname01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","attributes":{"data":"false","master":"true"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[/<IP_ADDRESS>:9300]","profiles":{}}},"1RKHMmjcQgGyl9n_6VsDuw":{"name":"node06.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node06.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node06.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[/<IP_ADDRESS>:9300]","profiles":{}}},"VQEqVFm4RUGUVtJufmJdAQ":{"name":"node01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node01.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[/<IP_ADDRESS>:9300]","profiles":{}}},"MwZk6zsfRX65xkc7vnNj-A":{"name":"node04.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node04.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node04.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[/<IP_ADDRESS>:9300]","profiles":{}}},"ujZ7vRpZTyKcWyBsrI72vg":{"name":"node03.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node03.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node03.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"transport":{"bound_address":"inet[/<IP_ADDRESS>:9300]","publish_address":"inet[/<IP_ADDRESS>:9300]","profiles":{}}}}}]
15/11/10 15:24:15 TRACE CommonsHttpTransport: Closing HTTP transport to loadbalancer01.fr:9200
15/11/10 15:24:15 INFO CommonsHttpTransport: Using detected HTTP Auth credentials...
15/11/10 15:24:15 TRACE CommonsHttpTransport: Opening HTTP transport to <IP_ADDRESS>:9200
15/11/10 15:24:15 TRACE CommonsHttpTransport: Tx [GET]@[<IP_ADDRESS>:9200][_nodes/http] w/ payload [null]
15/11/10 15:24:16 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{"cluster_name":"ELASTIC.PEI","nodes":{"innll3d8QMuo_iYqQCBm_Q":{"name":"loadbalancer01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"loadbalancer01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[loadbalancer01.fr/<IP_ADDRESS>:9200]","attributes":{"data":"false","master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"HuZ-Vpy0QwSTzRkclkMpRg":{"name":"node05.fr","transport_address":"inet[node05.fr/<IP_ADDRESS>:9300]","host":"node05.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node05.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[node05.fr/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"qOaZ0CXsRL-80-LnbEHk1Q":{"name":"uabigname01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"uabigname01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","attributes":{"data":"false","master":"true"}},"1RKHMmjcQgGyl9n_6VsDuw":{"name":"node06.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node06.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node06.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"VQEqVFm4RUGUVtJufmJdAQ":{"name":"node01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node01.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"MwZk6zsfRX65xkc7vnNj-A":{"name":"node04.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node04.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node04.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"ujZ7vRpZTyKcWyBsrI72vg":{"name":"node03.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node03.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node03.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}}}}]
15/11/10 15:24:16 TRACE CommonsHttpTransport: Closing HTTP transport to <IP_ADDRESS>:9200
15/11/10 15:24:16 INFO CommonsHttpTransport: Using detected HTTP Auth credentials...
15/11/10 15:24:16 TRACE CommonsHttpTransport: Opening HTTP transport to <IP_ADDRESS>:9200
15/11/10 15:24:16 TRACE CommonsHttpTransport: Tx [HEAD]@[<IP_ADDRESS>:9200][my_index/my_type] w/ payload [null]
15/11/10 15:24:17 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [null]
15/11/10 15:24:17 TRACE CommonsHttpTransport: Tx [GET]@[<IP_ADDRESS>:9200][my_index/_search_shards] w/ payload [null]
15/11/10 15:24:17 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{"nodes":{"MwZk6zsfRX65xkc7vnNj-A":{"name":"node04.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","attributes":{"master":"false"}},"1RKHMmjcQgGyl9n_6VsDuw":{"name":"node06.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","attributes":{"master":"false"}},"ujZ7vRpZTyKcWyBsrI72vg":{"name":"node03.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","attributes":{"master":"false"}},"HuZ-Vpy0QwSTzRkclkMpRg":{"name":"node05.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","attributes":{"master":"false"}},"VQEqVFm4RUGUVtJufmJdAQ":{"name":"node01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","attributes":{"master":"false"}}},"shards":[[{"state":"STARTED","primary":false,"node":"HuZ-Vpy0QwSTzRkclkMpRg","relocating_node":null,"shard":0,"index":"my_index"},{"state":"STARTED","primary":true,"node":"1RKHMmjcQgGyl9n_6VsDuw","relocating_node":null,"shard":0,"index":"my_index"}],[{"state":"STARTED","primary":false,"node":"VQEqVFm4RUGUVtJufmJdAQ","relocating_node":null,"shard":1,"index":"my_index"},{"state":"STARTED","primary":true,"node":"ujZ7vRpZTyKcWyBsrI72vg","relocating_node":null,"shard":1,"index":"my_index"}],[{"state":"STARTED","primary":false,"node":"VQEqVFm4RUGUVtJufmJdAQ","relocating_node":null,"shard":2,"index":"my_index"},{"state":"STARTED","primary":true,"node":"MwZk6zsfRX65xkc7vnNj-A","relocating_node":null,"shard":2,"index":"my_index"}],[{"state":"STARTED","primary":false,"node":"HuZ-Vpy0QwSTzRkclkMpRg","relocating_node":null,"shard":3,"index":"my_index"},{"state":"STARTED","primary":true,"node":"ujZ7vRpZTyKcWyBsrI72vg","relocating_node":null,"shard":3,"index":"my_index"}],[{"state":"STARTED","primary":false,"node":"1RKHMmjcQgGyl9n_6VsDuw","relocating_node":null,"shard":4,"index":"my_index"},{"state":"STARTED","primary":true,"node":"MwZk6zsfRX65xkc7vnNj-A","relocating_node":null,"shard":4,"index":"my_index"}]]}]
15/11/10 15:24:17 TRACE CommonsHttpTransport: Tx [GET]@[<IP_ADDRESS>:9200][_nodes/http] w/ payload [null]
15/11/10 15:24:18 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{"cluster_name":"ELASTIC.PEI","nodes":{"innll3d8QMuo_iYqQCBm_Q":{"name":"loadbalancer01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"loadbalancer01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[loadbalancer01.fr/<IP_ADDRESS>:9200]","attributes":{"data":"false","master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"HuZ-Vpy0QwSTzRkclkMpRg":{"name":"node05.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node05.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node05.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"qOaZ0CXsRL-80-LnbEHk1Q":{"name":"uabigname01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"uabigname01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","attributes":{"data":"false","master":"true"}},"1RKHMmjcQgGyl9n_6VsDuw":{"name":"node06.fr","transport_address":"inet[node06.fr/<IP_ADDRESS>:9300]","host":"node06.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node06.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[node06.fr/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"VQEqVFm4RUGUVtJufmJdAQ":{"name":"node01.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node01.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node01.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"MwZk6zsfRX65xkc7vnNj-A":{"name":"node04.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node04.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node04.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}},"ujZ7vRpZTyKcWyBsrI72vg":{"name":"node03.fr","transport_address":"inet[/<IP_ADDRESS>:9300]","host":"node03.fr","ip":"<IP_ADDRESS>","version":"1.6.2","build":"6220391","http_address":"inet[node03.fr/<IP_ADDRESS>:9200]","attributes":{"master":"false"},"http":{"bound_address":"inet[/<IP_ADDRESS>:9200]","publish_address":"inet[/<IP_ADDRESS>:9200]","max_content_length_in_bytes":104857600}}}}]
15/11/10 15:24:18 TRACE CommonsHttpTransport: Tx [GET]@[<IP_ADDRESS>:9200][my_index/_aliases] w/ payload [null]
15/11/10 15:24:18 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{"my_index":{"aliases":{}}}]
15/11/10 15:24:18 TRACE CommonsHttpTransport: Tx [GET]@[<IP_ADDRESS>:9200][my_index/my_type/_mapping] w/ payload [null]
15/11/10 15:24:18 TRACE CommonsHttpTransport: Rx @[<IP_ADDRESS>] [200-OK] [{"my_index":{"mappings":{"my_type":{"properties":{"account_number":{"type":"long"},"address":{"type":"string"},"age":{"type":"long"},"balance":{"type":"long"},"city":{"type":"string"},"email":{"type":"string"},"employer":{"type":"string"},"firstname":{"type":"string"},"gender":{"type":"string"},"lastname":{"type":"string"},"state":{"type":"string"}}}}}}]
15/11/10 15:24:18 DEBUG RestRepository: Closing repository and connection to Elasticsearch ...
15/11/10 15:24:18 DEBUG RestRepository: Sending batch of [0] bytes/[0] entries
15/11/10 15:24:18 TRACE CommonsHttpTransport: Closing HTTP transport to <IP_ADDRESS>:9200
[Stage 0:>                                                          (0 + 2) / 5]15/11/10 15:24:20 ERROR TaskSetManager: Task 0 in stage 0.0 failed 4 times; aborting job
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 7, node03.fr): org.apache.spark.util.TaskCompletionListenerException: [POST] on [my_index/my_type/_search?search_type=scan&scroll=5&size=50&preference=_shards:0;_only_node:HuZ-Vpy0QwSTzRkclkMpRg] failed; server[<IP_ADDRESS>:9200] returned [401|Unauthorized:]
        at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:90)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
        at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1283)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1271)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1270)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1270)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697)
        at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697)
        at scala.Option.foreach(Option.scala:236)
        at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:697)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1496)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1458)
        at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1447)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
        at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:567)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1822)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1835)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1848)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:1919)
        at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:905)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
        at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
        at org.apache.spark.rdd.RDD.collect(RDD.scala:904)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:27)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:32)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:34)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36)
        at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:38)
        at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40)
        at $iwC$$iwC$$iwC$$iwC.<init>(<console>:42)
        at $iwC$$iwC$$iwC.<init>(<console>:44)
        at $iwC$$iwC.<init>(<console>:46)
        at $iwC.<init>(<console>:48)
        at <init>(<console>:50)
        at .<init>(<console>:54)
        at .<clinit>(<console>)
        at .<init>(<console>:7)
        at .<clinit>(<console>)
        at $print(<console>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
        at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340)
        at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
        at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
        at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
        at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
        at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
        at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
        at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
        at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
        at org.apache.spark.repl.Main$.main(Main.scala:31)
        at org.apache.spark.repl.Main.main(Main.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.spark.util.TaskCompletionListenerException: [POST] on [my_index/my_type/_search?search_type=scan&scroll=5&size=50&preference=_shards:0;_only_node:HuZ-Vpy0QwSTzRkclkMpRg] failed; server[<IP_ADDRESS>:9200] returned [401|Unauthorized:]
        at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:90)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

The error is not the same as the one I posted at the beginning (maybe there was a mistake with an old "export http_proxy" in memory). Now this is a 401 unauthorized error.
I checked the network traces and I saw the following:

The spark client machine (which is the Spark Driver) uses the basic auth given when spark-shell was started (--conf spark.es.net.http.auth.user=username --conf spark.es.net.http.auth.pass=password)
The Driver is talking with the client node (I called it loadbalancer in my first post). The basic auth is used during this step.

REQUEST
GET / HTTP/1.1
Authorization: Basic c2ltbzEyMzQ6QWRtaW4xMjMt
User-Agent: Jakarta Commons-HttpClient/3.1
Host: loadbalancer01.fr:9200

RESPONSE
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 359

Three HTTP GET are visible. All of these requests are accepted with a 200 OK.

GET HTTP /
GET HTTP /_nodes/_transport
GET HTTP /_nodes/http

The Driver is talking with a data node (node04.fr). The basic auth is also used here.

REQUEST
HEAD /my_index/my_type HTTP/1.1
Authorization: Basic c2ltbzEyMzQ6QWRtaW4xMjMt
User-Agent: Jakarta Commons-HttpClient/3.1
Host: node04.fr:9200

RESPONSE
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Content-Length: 0

Five HTTP requests are visible. All of these requests are accepted with a 200 OK.

 For example, the node07 is talking with the node06. In that part the basic auth is not done.

Several HTTP POST are done repeatedly. All of these requests are refused with a 401 unauthorized.
REQUEST
POST /my_index/my_type/_search?search_type=scan&scroll=5&size=50&preference=_shards:0;_only_node:1RKHMmjcQgGyl9n_6VsDuw HTTP/1.1
User-Agent: Jakarta Commons-HttpClient/3.1
Host: <IP_ADDRESS>:9200
Content-Length: 44
Content-Type: application/json; charset=UTF-8

RESPONSE
HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="Search Guard"
Content-Type: text/plain; charset=UTF-8
Content-Length: 0

In think this is the origin of the issue. The ES-Hadoop connector doesn't seem to transmit the credentials to the executors and that's why I get the 401.

@ArcTheMaster Thanks again for the detailed response and for your time. I'll check why the configuration is not passed to the workers. I'm guessing you are specifying the configuration initially on the SparkConf/Context and that's it, right?
This might be caused by another issue that was reported a while back about the driver configuration not being properly sent to the drivers.
A quick workaround (albeit a verbose one) would be to specify the config as a Map and pass that to the ES-Hadoop/Spark methods.
By the way, your assesment and diagnostic (again, really appreciated) is correct - during the discovery (on the driver) the correct configuration is used and the cluster state is evaluated but on the workers, the config seems to be lost somehow. Need to investigate why.

@costin Thanks for your feedback. You're right, I'm specifying the configuration on the SparkConf/Context. I'm not using spark-submit in this use case, only the spark-shell. I specify the configuration as follow:
spark-shell -v --jars /opt/application/Spark/current/elastic/jar/elasticsearch-hadoop-2.1.1.jar --name elasticsearch-hadoop --master yarn-client --conf spark.es.net.ssl=true --conf spark.es.net.http.auth.user=username --conf spark.es.net.http.auth.pass=password --conf spark.es.nodes=loadbalancer.fr --conf spark.es.port=9200 --conf spark.es.field.read.empty.as.null=true --conf spark.es.resource=index/type

I don't stop the SparkContext once spark-shell has started. I talked with the Spark JIRA support and they told me that stopping the SparkContext in the shell is not supported (it can work, but it can also fail). That's why I'm specifiying the ES connector configuration at startup.
About the workaround, I won't be able to test it now. When I find some time, I will give it a try.
Please let me know when the issue is corrected. Don't hesitate to ask me if you need further input.

@ArcTheMaster I've committed a fix in master that should fix the issue. I've triggered the nightly build so hopefully in 20' max, one should be available with the fix it. Can you please try it out - it's the dev build in this case 2.2.0.BUILD-SNAPSHOT?

@costin, wow I thank you for your reactivity on this subject. It's really appreciated.
I tested the dev build 2.2.0.BUILD-SNAPSHOT (that one elasticsearch-hadoop-2.2.0.BUILD-20151119.024605-154.zip to be precise) this morning and I wasn't able to reproduce the error during my tests. I think we can say it's corrected.
Will you put this update in the next stable release of the ES connector?
Once again, I thank you for your support on this topic.

Thank for confirming. Yes, the fix will be available in the next version (see the issue tags in the right column).
Thanks for reporting (even more so in so great detail).
Cheers,

Ok thanks for the last info. I'm going to use the BETA version of the connector for now and I will use the stable version when it will be released.
Can I close the issue or do I need to keep it open?
Cheers

I am also trying to run the same code, but inside a container. So my spark and elasticsearch are running on different conatiners in the same box. I am getting below error. It seems it is trying to connect to local host instead of <IP_ADDRESS>(another container where ES is running).
I am running below command to start spark shell:
 ./bin/spark-shell --master local[4] --packages "org.elasticsearch:elasticsearch-spark_2.10:2.1.0.Beta2" --name elasticsearch-hadoop  --conf spark.es.net.ssl=true --conf spark.es.nodes=<IP_ADDRESS> --conf spark.es.port=9200 --conf spark.es.field.read.empty.as.null=true --conf spark.es.resource=index/type
scala> val RDD = sc.esRDD("index/type/", "?q=Ge*")
RDD: org.elasticsearch.spark.rdd.ScalaEsRDD = ScalaEsRDD[0] at RDD at AbstractEsRDD.scala:17
scala> RDD.count

16/11/21 21:26:06 INFO httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
16/11/21 21:26:06 INFO httpclient.HttpMethodDirector: Retrying request
16/11/21 21:26:06 INFO httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
16/11/21 21:26:06 INFO httpclient.HttpMethodDirector: Retrying request
16/11/21 21:26:06 INFO httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
16/11/21 21:26:06 INFO httpclient.HttpMethodDirector: Retrying request
16/11/21 21:26:06 ERROR rest.NetworkClient: Node [Connection refused (Connection refused)] failed (localhost:9200); no other nodes left - aborting...
org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[localhost:9200]]
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:123)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:303)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:287)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:291)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:118)
at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:100)
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:57)
at org.elasticsearch.hadoop.rest.RestService.findPartitions(RestService.java:220)
at org.elasticsearch.spark.rdd.AbstractEsRDD.esPartitions$lzycompute(AbstractEsRDD.scala:49)
at org.elasticsearch.spark.rdd.AbstractEsRDD.esPartitions(AbstractEsRDD.scala:48)
at org.elasticsearch.spark.rdd.AbstractEsRDD.getPartitions(AbstractEsRDD.scala:25)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
at org.apache.spark.rdd.RDD.count(RDD.scala:1157)

@yogeshdarji99 Please upgrade to a more recent version of ES-Hadoop than 2.1.0.Beta2. The ability to recognize spark command line options was added after that version. I would recommend using the most recent versions of ES-Hadoop (5.0.1 or 2.4.0 if you really need to).

@jbaiera I also tried Elasticsearch-spark with 2.4.0 and 5.0.0-alph2(since 5.0.0 is nota available). It is not working. Still trying to connect to localhost instead of <IP_ADDRESS>:9200. Is it still in development phase?

Hi @jbaiera / @yogeshdarji99 , I am also facing same issue while using it in container. I have simple application which uses sparkSQL to process the data and store it in Elasticsearch.
Elasticsearch version: 2.4.1
Maven dependency added in project :

org.elasticsearch
elasticsearch-spark_2.10
2.4.0

project properties :
ENV cassandra.nodelist <IP_ADDRESS>
ENV elasticsearch.port 31921
Error:
<IP_ADDRESS>:31921]  | 2016-11-21 23:01:18.989  INFO 1 --- [launch worker-0] o.e.spark.sql.EsDataFrameWriter          : Writing to [testAnalyticsApp/proxy_data]  | 2016-11-21 23:03:26.258  INFO 1 --- [launch worker-0] o.a.c.httpclient.HttpMethodDirector      : I/O exception (java.net.ConnectException) caught when processing request: Connection timed out  | 2016-11-21 23:03:26.259  INFO 1 --- [launch worker-0] o.a.c.httpclient.HttpMethodDirector      : Retrying request  | 2016-11-21 23:05:33.618  INFO 1 --- [launch worker-0] o.a.c.httpclient.HttpMethodDirector      : I/O exception (java.net.ConnectException) caught when processing request: Connection timed out   | 2016-11-21 23:05:33.619  INFO 1 --- [launch worker-0] o.a.c.httpclient.HttpMethodDirector      : Retrying request   | 2016-11-21 23:07:40.978  INFO 1 --- [launch worker-0] o.a.c.httpclient.HttpMethodDirector      : I/O exception (java.net.ConnectException) caught when processing request: Connection timed out  | 2016-11-21 23:07:40.979  INFO 1 --- [launch worker-0] o.a.c.httpclient.HttpMethodDirector      : Retrying request  | 2016-11-21 23:09:48.338 ERROR 1 --- [launch worker-0] o.e.hadoop.rest.NetworkClient            : Node [<IP_ADDRESS>:9200] failed (Connection timed out); no other nodes left - aborting...   | 2016-11-21 23:09:48.345 ERROR 1 --- [launch worker-0] org.apache.spark.executor.Executor       : Exception in task 0.0 in stage 0.0 (TID 0)   |    | org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[<IP_ADDRESS>:9200]]   | 	at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:142) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:434) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]   | 	at org.elasticsearch.hadoop.rest.RestClient.executeNotFoundAllowed(RestClient.java:442) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]   | 	at org.elasticsearch.hadoop.rest.RestClient.exists(RestClient.java:518) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]   | 	at org.elasticsearch.hadoop.rest.RestClient.touch(RestClient.java:524) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.elasticsearch.hadoop.rest.RestRepository.touch(RestRepository.java:491) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.elasticsearch.hadoop.rest.RestService.initSingleIndex(RestService.java:412) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:400) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:40) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]   | 	at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:59) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:59) ~[elasticsearch-spark_2.10-2.4.0.jar!/:2.4.0]  | 	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) ~[spark-core_2.10-1.6.1.jar!/:1.6.1]  | 	at org.apache.spark.scheduler.Task.run(Task.scala:89) ~[spark-core_2.10-1.6.1.jar!/:1.6.1]   | 	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) ~[spark-core_2.10-1.6.1.jar!/:1.6.1]  | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_102]   | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_102]  | 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]

@jbaiera  @yogeshdarji99 , I found out the problem. I had not specified the property
" es.nodes.wan.only = true " my bad. Specifying this property solved my problem


Enhancement: Allow setting arbitrary properties on web

The spo web set property should allow users to update the value of any property supported by the Web class in SPO.
At this moment, the spo web set command supports only a handful of options that correspond to the properties of the Web class. This is similar to the Set-PnPWeb cmdlet. However, because PnP PowerShell works with CSOM, this limitation is often circumvented, by retrieving the web CSOM object, updating its properties and calling $web.ExecuteQuery() to persist the changes outside of PnP cmdlets. Because the CLI doesn't use CSOM, this isn't possible and users are limited to using whatever options are exposed by the CLI commands.
We could address this limitation by allowing users to set the value of any property exposed on the Web object. This would work only for properties that have values of simple types like bool, string or int, because complex values likely can't be serialized to string without additional logic.
We would offer rudimentary logic of checking if the specified properties are valid, by examining the latest version of the SPO CSOM assemblies and checking which properties on the Web class have setters and values of the supported types and thus can be used.
By looking at the telemetry, we can track the usage of the different properties and gradually implement native support for the most frequently used properties.
Once the approach is validated, we could implement it on other types like site, list, listitem, field, file, etc.

Here is a proof of concept for the implementation based on spo web set: https://github.com/waldekmastykarz/office365-cli/commit/df54ed2945e5a4f1a8c107bd059a114e19292c21


Paula Bernal: Added homework and hackathon for week 1

Great job on building a random world generator and counting the land on your world! I like your use of random.choice. It would be cool if you could print the world out to the user so they can see their new random world map.
The next step for your solution would be to choose a land coordinate in the world and start counting the land tiles connected to it to see how large a continent you are standing on.
You’ve made great use of lists and loops in your code. You’ve definitely learnt a lot in your first week and I look forward to seeing your solution for the next challenge!

Added hackathon week 2


I want to translate github docs for education purpose. Which one do you think is better?

I want to translate github docs for education purpose. Which one do you think is better?

Fork and translate \content\ directly
Create pull request on this repo, but I don't know where to put my translation scripts

Update : I found \translation\ folder :)
Originally posted by @sownt in https://github.com/github/docs/discussions/11795

England
On Sat, 5 Feb 2022, 13:51 welcome[bot], @.***> wrote:

Thanks for opening this issue. A GitHub docs team member should be by to
give feedback soon. In the meantime, please check out the contributing
guidelines https://github.com/github/docs/blob/main/CONTRIBUTING.md.
—
Reply to this email directly, view it on GitHub
https://github.com/github/docs/issues/15116#issuecomment-1030628495, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AXID4ISBVOD7U2PKBPF7SBTUZUTMRANCNFSM5NT7I3GA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

The world is now not limited to all. Knowledge and freedom to know are the beauty of earth.
Its a better idea


[SR-3994] Add round-trip tests for break-statement

Previous ID
SR-3994

Radar
None

Original Reporter
@bitjammer

Type
Bug

Additional Detail from JIRA

Votes
0

Component/s
Compiler

Labels
Bug, Syntax

Assignee
@bitjammer

Priority
Medium

md5: ea545fa584530a822fe477b30efb1a05

Issue Description:
Include semicolons.

Closing because libSyntax is no more.


allow metagenomic - per sample fastq ebi submissions

AFAIK this is possible but only via CLI, we should open via GUI

Make sure option is only with MG submissions
On Thu, Sep 20, 2018 at 12:55 PM, Antonio Gonzalez<EMAIL_ADDRESS>
wrote:

AFAIK this is possible but only via CLI, we should open via GUI
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/biocore/qiita/issues/2677, or mute the thread
https://github.com/notifications/unsubscribe-auth/AB69gOsbMDDg-hJ3fizaCiKUrzm0Ap1Uks5uc_KXgaJpZM4Wy1D-
.

--
Gail Ackermann
Knight Lab
UCSD
<EMAIL_ADDRESS><EMAIL_ADDRESS>

... and add reverse reads, perhaps?

Agree - we can add the reverse reads in a single template or a separate
template depending what we think is best
On Wed, Sep 26, 2018 at 9:59 AM, Antonio Gonzalez<EMAIL_ADDRESS>wrote:

... and add reverse reads, perhaps?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/biocore/qiita/issues/2677#issuecomment-424792766, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB69gPc_flyBXVvH5Dml0Q08uxTbyN3hks5ue7JVgaJpZM4Wy1D-
.

--
Gail Ackermann
Knight Lab
UCSD
<EMAIL_ADDRESS><EMAIL_ADDRESS>


Remove Radar code as it's no longer used

@gtanzillo @chessbyte Please review.

I think this also means we can remove sqlite-devel from the RPMs.  See https://github.com/ManageIQ/manageiq-appliance-build/pull/260 and walk the chain.
/Users/jfrey/dev/manageiq-rpm_build/Dockerfile:41:      sqlite-devel \
/Users/jfrey/dev/manageiq-rpm_build/rpm_spec/subpackages/manageiq-gemset:15:Requires: sqlite

test failure is due to a sporadic test failure.  I created #21053 to fix that.

@Fryguy A conflict occurred during the backport of this pull request to morphy.
If this pull request is based on another pull request that has not been marked for backport, add the appropriate labels to the other pull request. Otherwise, please create a new pull request direct to the morphy branch in order to resolve this.
Conflict details:

@Fryguy A conflict occurred during the backport of this pull request to morphy.
If this pull request is based on another pull request that has not been marked for backport, add the appropriate labels to the other pull request. Otherwise, please create a new pull request direct to the morphy branch in order to resolve this.
Conflict details:

Seems this is already in morphy and lasker.

Seems this is already in morphy and lasker.

@Fryguy wonder if the backport script should check that condition and auto-comment appropriately

Yeah that should be possible - I'm kind of surprised git doesn't handle it internally automatically.

Skipping backport to morphy, because it is already in the branch.

Skipping backport to morphy, because it is already in the branch.


Feature Request: Cleaner interfaces

Hi,
all Infomodel Object interfaces seem to share common functions, such as getId(), toRdf, setProperties and getProperties.

I would like to propose to segregate the "fat" IDS interfaces and rather compose the IDS interfaces by using single purpose interfaces. E.g.:

// Maybe even make key and values generic?
public interface DynamicProperties<K, V> {
    void setProperty(K key, V value);
    Map<K, V> getProperties();
}

By defining such it would be possible to operate on IDS objects in a generic way. E.g. call toRdf on all IDS objects regardless if they are ResourceCatalogs or Artifacts (which is currently not possible, see definition of Catalog and Artifact).

Further i would like to propose to remove duplications in interface definitions.
e.g. When looking at the definition of a ResourceCatalog it definies functions like getId, toRdf. But those are already definied in the parent interface Catalog.

It gets even more complicated when looking at classes like Representation that inherit multiple interfaces, but all are defining those functions. This makes interfaces larger then they need to be and it obscures how a e.g. ResourceCatalog differs from a Catalog.

Maybe this could be fixed by defining a common shared base interface? Something like IdsObject? This IdsObject would inherit those single purpose interface (defined in 1) and every IDS Object would inherit from IdsObject. This would also allow to narrow the scope of generic functions and classes because it would allow the usage of <? extends IdsObject>. At the moment such scoping is not possible since the only common interface is the java base class Object.

Hi Brian,
almost as in #1, we have started working on this suggestion and created some generalizing interfaces. The first improvements, covering most of what you suggest here, should be ready in roughly three weeks from today.
Thanks for the great suggestions!


Coworker appreciation app template

Docs Build status updates of commit 2ffd36d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 2ffd36d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 7cdd41e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 7cdd41e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit f8fc2fe:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit f8fc2fe:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 6a7030f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 6a7030f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 97e514a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 97e514a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 0048165:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 0048165:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/samples/app-templates.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


IPsec crashing

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
IPsec / strongswan suddenly crashes as reported in the German forum:
https://forum.opnsense.org/index.php?topic=31857.0
The ipsec service becomes unresponsive (in web gui and via ssh "ipsec" command), all tunnels fail, the log just stops.
No hints for a root cause on my side so far - but other reports state that it is a problem with the "charon vici queue" (see links below).
What helps is to stop and start ipsec via SSH or reboot the server.
Other similar reports I found:
https://forum.opnsense.org/index.php?topic=22224.0
https://forum.netgate.com/topic/172075/my-ipsec-service-hangs/68
https://forum.netgate.com/topic/165661/charon-becoming-unresponsive/33
https://redmine.pfsense.org/issues/13014
My system is actually running as a CARP failover cluster in Hyper-V and the IPsec part was stable for about 4 months (sept 22 - jan 23) after I set it up. Back then I started with 1 SA and added more over the next months, currently there are 21. When I added the last 3 SAs in the beginning of January, the first crash occured. Maybe that was kind of a threshold for my system regarding to IPsec / charon / vici.
To Reproduce
This is quite difficult and possible the reason why there is no solution after about 1.5 years (referring to the oldest reports from pfSense). As far as I know there is yet no reliable way to reproduce.
Maybe have a system with atleast 10-20 SAs, maybe some inactive tunnels but with keep-alive enabled and/or "interesting traffic", maybe a lot of manual trying to start inactive tunnels.
For example I have some monitoring scripts in place, which look at "ipsec status" or "ipsec statusall" and depending on the tunnel, they will try to restart phase 1 / phase 2 connections with "ipsec up $con_id" or "swanctl --initiate --child $con_id".
Expected behavior
The ipsec service should be rock stable and even if some connections may fail from time to time for different reasons, the daemon state itself should never get in such a broken state that it fully stops working and even possible block the web gui from loading.
Describe alternatives you considered
Updated from 22.7.3 to 22.7.10
Considered to split the cluster into multiple independent servers to lower the crash chance by equally distributing all SAs
Environment
OPNsense 22.7.10_2-amd64
FreeBSD 13.1-RELEASE-p5
OpenSSL 1.1.1s 1 Nov 2022

Forgot to mention related strongswan issues:
https://github.com/strongswan/strongswan/issues/268
https://github.com/strongswan/strongswan/issues/566

Sorry, I'm not fully aware of everything inside of IPsec / strongswan.
I know there is a migration happening from ipsec.conf to swanctl.conf.
Does this include the change from using stroke to using swanctl as the control facility (as in: the command to start/stop tunnels)?
Maybe this is part of the problem as I'm using both commands in some of my "repair" scripts?
For example:
ipsec start con2
swanctl --initiate --child con4

In general I would recommend to try a newer released version before reporting an issue that might already be fixed. I've made the same comment in the forum thread.
Cheers,
Franco

Will do it tonight

The server is now running with OPNsense 23.1_6-amd64 :-)
I did not yet migrate the tunnels (ipsec.conf) to connections (swanctl.conf) though.
I will report back if another crash occurs.

The migration to swanctl.conf is automatic for old tunnels so underneath they work like the new connections.
Cheers,
Franco

Nice, good to know

We just had another crash (after about 14 hours with 20 active SAs). All tunnels down, "Status Overview" in webgui empty, ipsec service not running.
netstat -Lan | grep charon.vici said:

unix  5/0/3                            /var/run/charon.vici

ipsec stop said:

Stopping strongSwan IPsec failed: starter is not running

ipsec start said:

no files found matching '/usr/local/etc/strongswan.opnsense.d/*.conf'
Starting weakSwan 5.9.9 IPsec [starter]...
charon is already running (/var/run/charon.pid exists) -- skipping daemon start
no files found matching '/usr/local/etc/ipsec.conf'
failed to open config file '/usr/local/etc/ipsec.conf'
unable to start strongSwan -- fatal errors in config

In top I saw the charon process with sigwait state:

64269 root         17  52    0    82M    17M sigwai   3   5:24   0.00% charon

I killed the process and tried to start ipsec once again, of course it did not work.
So I had to restart the server to fix the problem.

you need swanctl  (https://docs.strongswan.org/docs/5.9/swanctl/swanctl.html)to control the deamon nowadays, stop/start from the gui should forcefully stop the daemon, if that doesn't work, you might also be looking at a kernel/driver/hardware issue by the way.

Stop/start from the gui did not work.
What can I do to further investigate the issue?
Do you need me to add more detail information or try specific commands or change settings to get a clue?

when stalled, I would certainly try to manually kill the processes with a kill -9 and see if that stops them, if not, driver/hardware is starting to sounds more and more logical. What type of equipment is this?

It's a virtual machine running on a Hyper-V-Failovercluster (Microsoft Server 2012R2)

Another crash today...
netstat -Lan | grep charon.vici said

unix  1/0/3                            /var/run/charon.vici

I was able to kill the process

daemon: /usr/local/libexec/ipsec/charon[88628] (daemon)

but for

/usr/local/libexec/ipsec/charon --use-syslog

I had to use kill -9.
After that I was able to login via web gui again and start the ipsec service there.

There is a possible fix from the honorable Tobias Brunner (StrongSwan) which atleast seems to fix the issue on pfSense.
Any chance we could get this in one of the next OPNsense patches? 🙏  ❤
https://github.com/strongswan/strongswan/commit/f33cf9376e90f371c9eaa1571f37bd106cbf3ee4
https://github.com/strongswan/strongswan/issues/566

Frankly, all we need is for StrongSwan to push this fix into a release.

Hi, just for contribute.
I have a PFSense running with about 25 site-to-site IPSec tunnels, and 56 SAs. I´m facing exactl the same issue.
Version: PFSense community edition - version 2.6.0.
Have found a script to run inside cron to check the connections status, and send a service stop and then start. This helped in some way, but the problem persists.
Will continue waiting for a solution.

I added said patch to a snapshot build for a customer... but it appears it's not helpful for them so I wouldn't get my hopes up.
# opnsense-revert -z strongswan

Commit in question is https://github.com/opnsense/ports/commit/73f93c86d2
Cheers,
Franco

Have found a script to run inside cron to check the connections status, and send a service stop and then start. This helped in some way, but the problem persists. Will continue waiting for a solution.

The script helped me too as a workaround, I forgot to mention. Before I realized that the crashes are depending on the total amount of (phase 2) tunnels, I had to split up the tunnels from my OPNsense cluster (CARP) to currently 5 distinct servers. But one of them still has 19 connections which can't be separated, so here the cron script repairs the once in a while occuring crash.
I have adapted the script from https://forum.netgate.com/topic/172075/my-ipsec-service-hangs/73 for OPNsense as follows.

#!/usr/local/bin/bash
log="/var/log/ipsec_restart.log"
queue=$(netstat -Lan | grep charon.vici | tr -s ' ' | cut -d' ' -f2)
queue_length=$(echo $queue | cut -d'/' -f1)
if [ $queue_length -gt 0 ]
then
echo "$(date +%F\ %H:%M:%S) charon crash (vici queue: $queue)" >> $log
/usr/bin/killall -9 charon
sleep 5
/usr/local/etc/rc.d/strongswan onestop; /usr/local/sbin/pluginctl -c ipsec
sleep 5
/usr/local/etc/rc.d/strongswan onestop; /usr/local/sbin/pluginctl -c ipsec
fi

@Charburner maybe you want to test the snapshot build instead

@fichtner I've already scheduled server maintenance for next weekend :-)

Instructions have been posted. Trying to patch the ports tree is a bit ineffective and was suggested nowhere never before.

Oh yeah, I missed that part and instead searched google for how to install a patch/commit on opnsense... (https://forum.opnsense.org/index.php?topic=7537.0)

opnsense-revert -z strongswan

Patch successfully installed, reboot is scheduled for tonight. It might need several weeks to see if another crash occurs.

Almost lost it because of this bug...
The Dashboard become unresponsive because of the IPSec widget. (Async calls for widgets to mitigate bugs like this maybe?)
I'm going to test this aswel.

You guys are funny. This has been shipped to stable weeks ago.

Ok so the reported commit was never the issue. Unlikely that it’s been fixed on 23.7.
Cheers,
Franco

Patch successfully installed, reboot is scheduled for tonight. It might need several weeks to see if another crash occurs.

Hey guys,
last charon crash: 2023-06-15
patch installed: 2023-06-20
no crashes for seven weeks (on this system with 19 active phase 2 connections) - looks good to me
of course I have no way to really be sure like reproducing a crash or testing actively

ok closing as per positive result from @Charburner who opened this ticket. If other issues still exist please make a separate ticket.

just to add more contextual information:
before applying the patch, crashes occured multiple times per month

2023-04-02 18:49:01 charon crash (vici queue: 1/0/3
2023-04-03 13:06:05 charon crash (vici queue: 2/0/3)
2023-04-06 23:51:55 charon crash (vici queue: 1/0/3)
2023-04-10 14:57:06 charon crash (vici queue: 1/0/3)
2023-04-12 08:51:15 charon crash (vici queue: 2/0/3)
2023-04-16 19:05:00 charon crash (vici queue: 1/0/3)
2023-04-16 23:56:00 charon crash (vici queue: 1/0/3)
2023-04-20 11:12:00 charon crash (vici queue: 1/0/3)
2023-04-23 14:01:00 charon crash (vici queue: 1/0/3)
2023-05-06 03:31:00 charon crash (vici queue: 1/0/3)
2023-05-13 05:02:00 charon crash (vici queue: 1/0/3)
2023-06-04 06:13:00 charon crash (vici queue: 1/0/3)
2023-06-09 18:56:00 charon crash (vici queue: 1/0/3)
2023-06-15 19:33:00 charon crash (vici queue: 1/0/3)

Hello we have the exact same problem on OPNsense 24.4.1-amd64 should it fixed there?

Have found a script to run inside cron to check the connections status, and send a service stop and then start. This helped in some way, but the problem persists. Will continue waiting for a solution.

The script helped me too as a workaround, I forgot to mention. Before I realized that the crashes are depending on the total amount of (phase 2) tunnels, I had to split up the tunnels from my OPNsense cluster (CARP) to currently 5 distinct servers. But one of them still has 19 connections which can't be separated, so here the cron script repairs the once in a while occuring crash. I have adapted the script from https://forum.netgate.com/topic/172075/my-ipsec-service-hangs/73 for OPNsense as follows.

#!/usr/local/bin/bash
log="/var/log/ipsec_restart.log"
queue=$(netstat -Lan | grep charon.vici | tr -s ' ' | cut -d' ' -f2)
queue_length=$(echo $queue | cut -d'/' -f1)
if [ $queue_length -gt 0 ]
then
echo "$(date +%F\ %H:%M:%S) charon crash (vici queue: $queue)" >> $log
/usr/bin/killall -9 charon
sleep 5
/usr/local/etc/rc.d/strongswan onestop; /usr/local/sbin/pluginctl -c ipsec
sleep 5
/usr/local/etc/rc.d/strongswan onestop; /usr/local/sbin/pluginctl -c ipsec
fi

thanks!  your script fixed my issue on the newest version of opnsense.  no matter what I did in the gui, it would never re-enable.


Godot messes up log files when running multiple intances

Godot version
4.1.1-stable
System information
Arch Linux
Issue description
When running multiple instances (from CLI with commands like godot4 -d --path x/y/z) of a game that has heavy logging (maybe every frame, every 1 sec, etc.), you'll notice logs will be inconsistent in comparison to running one single instance alone. In fact it looks like the first instance will still output logs to godot.log after the second instance has renamed it to something like godot2023-07-26T09.57.09.log and created a new godot.log file, which is very dangerous if you're unexperienced and will most likely interpret invalid log information.
A way to at least workaround this bug would be to add a --log-file argument to CLI where I could give fixed names for files. (e.g. --log-file=server.log vs --log-file=client.log)
Steps to reproduce

Create a project
Somehow print a lot of information (with print())
Run two instances from CLI (with godot4 -d maybe)
Notice that the logs are messed up

Minimal reproduction project
It's intermitent so very hard to catch:
Screenshot:

MRP:
MRP.zip

A way to at least workaround this bug would be to add a --log-file argument to CLI where I could give fixed names for files. (e.g. --log-file=server.log vs --log-file=client.log)

This is a good idea in general (not just for multiple instances). In this case, the multiple instance launcher in the editor can pass the correct CLI argument that follows the project setting, but with the instance ID appended before the .log extension.

A way to workaround that on Linux is to pipe the output to the appropriate file like this:
# For the server
godot4 -d | tee logs/server.log

# For the client
godot4 -d | tee logs/client.log

However the downside is that you'll lose the colors.

Adding a command line for a log file path is a relatively simple task. We can just override the project's setting debug/file_logging/log_path in such case.
And probably it makes sense to enable debug/file_logging/enable_file_logging as well if log file path is specified in the command line.
I can take care of implementing this (with the documentation), if no one minds.


gradle setup on README

Thank you for your contribution.
I feel I should clarify that the reason I did not include a Gradle example in the readme, is because it is pretty easy (in this case, at least) to mechanically convert the Maven config to a Gradle script. But seeing as you've done the work of adding this, and it's nice to have, I'll merge it.


Images to avif, with fallback

For faster loading.
Fallback:
<picture>
  <source srcset='screenshot.avif' type='image/avif'>
  <img src='screenshot.jpg alt='screenshot'>
</picture>

Generate smaller images in webp and avif formats at the site generation time is a good idea.

Generate smaller images in webp and avif formats at the site generation time is a good idea.

@biodranik This might be useful, but Zola still doesn't support conversion to AVIF, HEIC, and JPEG2000.
I commonly use HEIC and JPEG2000 for Safari versions which don't support AVIF and WEBP, respectively. The size of a ImageMagick-converted transparent AVIF image is usually AVIF < HEIC < WEBP < JPEG2000 (slight savings) < PNG.

Why jpeg2000 instead of jpeg?

JPEG2000 supports alpha transparency like those in PNG.
JPEG2000 is useful for targeting Safari on iOS 13 and macOS Catalina and below because they do not natively support WEBP.
Also apparently you can achive a 50% compression rate with JPEG2000 by switching to lossy format (loseless is default in ImageMagick) with -quality 50.


Simplify peak picking and range picking

We should combine manual and automatic peak and range picking.
The idea is that is you press 'p' for peak picking you are in the manual peak picking mode and you can start to manually select peaks but at the same time the top bar change to the automatic peak picking

Same if true for range peak picking when you press 'r'
This will reduce the number of buttons while keeping all the functionalities.

@lpatiny
Only when press p or do you mean we have only one button in the toolbar for peak picking and you can perform both manual and auto.

Not only when you press p. Only one button in the toolbar for peak picking.


First section - sentence flaw

you can must delete and re-create the variable as un-encrypted.
Can I? or Must I?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fbcf48d3-ba48-75e5-092d-40317111b389
Version Independent ID: 055126f7-51a5-37b3-a447-829b75c694b8
Content: Variable assets in Azure Automation
Content Source: articles/automation/shared-resources/variables.md
Service: automation
Sub-service: shared-capabilities
GitHub Login: @MGoedtel
Microsoft Alias: magoedte

@perkops Thanks for the comment. We are actively investigating on the issue and will get back to you shortly with an update.


1.6.0  Input Click problem

Expected behavior
Datepicker does not pop upon input click, works on 1.5.0
Actual behavior
Steps to reproduce
<DatePicker          customInput={TimepickerInput}          minDate={moment.tz()}          selected={this.state.scheduledAt}          onChange={(e) =>this.setScheduleTime(e)}          showTimeSelect          timeIntervals={5}          dateFormat="LLL"          isClearable={false}          placeholderText="Click to select a date"          preventOpenOnFocus          readOnly          todayButton={${tmz}}           />

You have readOnly property which changed behavior in https://github.com/Hacker0x01/react-datepicker/pull/1419 . Don't ask me why it works like that right now ;) Or ... does not work ... to be more exact :/

If you want your users to only use the calendar control and not the input field, you need to use an alternate method for blocking text input - read-only has a defined meaning, per W3C - it specifies the value can not be edited by the user. see
https://www.w3.org/TR/html401/interact/forms.html#adef-readonly


feat(codemod): replace responsive entry codemod

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.679%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
8123

Relevant Lines:
9224

💛  - Coveralls


How to disable writing openCL binaries to file?

When running MACE on GPU, the openCL binaries get dumped to file in the device, is there any way to prevent this to happen? For example I don't have permission to access device storage in Android. @lu229

@gasgallo If you did not invoke the method GPUContextBuilder::SetStoragePath, MACE will not dump the bin to a file, but on that occasion, MACE will run very slowly in the initialization and the first run.

@lu229 thanks, that works. Do you know why this solution runs more slowly during initialization and warmup?

@gasgallo If you refer to this document and set the tuning bin file, the performance will run fast, but after the OpenCL Driver update, the bin file will be invalid, on this occasion, MACE will run and dump the bin file and store it on the device, in the next time MACE will load the dumped bin file and MACE will fun fast, too.
If you did not invoke the method 'GPUContextBuilder::SetStoragePath', MACE will not store the bin file, the next time MACE will can not load the bin file and MACE will run more slowly.
The bin file is the binary from the OpenCL source code's compilation, which can accelerate the performance.


docs: updated contribution guidelines for website

Fixes #3558

Added the section for contributing to the website.
Mentioned useful directories.

Thanks for doing this and so quickly!  However, the content should be in https://github.com/keptn/lifecycle-toolkit/issues/2947 with a link from CONTRIBUTING.md.  We did that because we have too much contributing information to be manageable in CONTRIBUTING. md.  I might suggest a separate page/file called "website", that is published just before the "blogs" page.   This means that you need to add a file to the mkdocs.yml file.  See https://keptn.sh/stable/docs/contribute/docs/source-file-structure/#specifying-the-doc-structure for instructions and holler if you need help.

Might want to add a note about the CSS as well.  If you don't have better info than what @mowies provided, just add that and put a note inviting anyone with better information to contribute it.  That's one of the cool tricks you can do in open source documentation ;-)

I have added the file in the docs directory and added the useful notes in the file as well !

More details about what is required:

Create a new file, website/index.md, in the lifecycle-toolkit/docs/content/en/contribute directory.
Add this file to the lifecycle-toolkit/mkdocs.yml file either just before or after the blog/index.md
Add content to this file.  Not sure how much is required, but definitely:

Location of files used to modify website
Point to any particularly interesting ones, like website.yml, with a note about its use
Info about how to test/verify changes before submitting.  I think changes are reflected in the readthedocs preview for the PR but need to verify that

@StackScribe
I have made the specified changes but i couldnt figure out how did you want the mkdocs.yml to be modified.
So, please check last 2 commits on mkdocs.yaml and let me know if one of these are the modifications you wanted.

check this new commit as well

There was constant error appearing in mkdocs.yml and i think its resolved now.

Hi @BalaadityaPatanjali thanks for the contribution 🚀 ! The ticket is still under need-discussion, we will discuss about it tomorrow during the community meeting and come back at you with a review 😃

Hi @BalaadityaPatanjali thanks for the contribution 🚀 ! The ticket is still under need-discussion, we will discuss about it tomorrow during the community meeting and come back at you with a review 😃

Thankyou for the update ! Looking forward to the review.

You need to fix the markdown problems.  Running make markdownlint-fix on your local branch then pushing another commit with those results will fix most of the problems -- I'm not sure if it will fix the sentence that doesn't start on a new line but the output will tell you if there is more to fix.  Someone else will have to advise about fixing the YAML checks.

You need to fix the markdown problems. Running make markdownlint-fix on your local branch then pushing another commit with those results will fix most of the problems -- I'm not sure if it will fix the sentence that doesn't start on a new line but the output will tell you if there is more to fix. Someone else will have to advise about fixing the YAML checks.

I have resolved the yaml and markdown issues. I was not aware that it would dismiss the review. Please re -review when you get a chance.


grype: 0.34.3 -> 0.34.4

Description of changes

Bump grype to 0.34.4
It also needs go 1.18
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[X] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[X] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 165242 run on x86_64-linux 1

1 package built:

grype


Code editor tooltip gets cut off by a panel to the side.

Godot version: 3.0.5 Stable Official

OS version: Windows 10

Issue description:

If the tooltip is big enough, it will get cut off by the panel to it's side (the inspector panel in attached screenshot).
Steps to reproduce:
Write a line long enough for it's tooltip to reach the edge of the neighbouring panel.

I'm pretty sure this is a duplicate, but I can't find the earlier issue(s) about it...

Looks like #16225 is the same bug.

Duplicate of #16225. (While the tooltip is cut at the bottom in the linked issue, the underlying bug is the same.)


[Impeller] optimize drawAtlas with blending

Currently drawAtlas with a blend mode will create a subpass that is as large as the final area needed. For example, if two sprites 50x50 drawn one at 0,0 and one at 1000,1000, then we end up with a 1050x1050 roughly subpass texture.
This is much less efficient than it could be. Since the subpass is only used to create blended texture, we could instead collapse this to a 50x100 texture and then position the sprites when compositing back onto the parent pass. We could even collapse cases where the per sprite color blend and sample is identical.

This would more or less require a rect packer. I'm not sure if we can/should use Skias. We could also create a simple one for ourselves.

Will do https://github.com/flutter/flutter/issues/131345 instead


Hide some irrelevant inputs if the form is not court related: Issue #316

Set "none of the above" to False for interview.allowed_courts and interview.categories because they are addressed by court_related=False.

There are a few places that the "Letter" form type is already used that this hides--if it's OK I'll make a commit that just moves the "Not court related choice" down into the list of form types  but keeps the rest of the behavior of hiding the irrelevant inputs. Is that OK?

@purplesky2016 I can take over incorporating Bryce's suggestions since you're not working today

FYI--there is definitely a bug with the next_steps document when you select "other_form" as the form type which this change makes more prominent. I will work on that next--there's a refactor I'm planning for the next steps integration already so I will address it there. See #371

Test is failing but this commit didn't touch unmap_suffixes.docx. I think the number of fields just changed in that file in a different commit w/o updating the test.

Something went very wrong in https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/c7d3822f925cc3193bffa838d2832eba0dc7aea2, it reverted https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/99f00cecb46c89f7ce522ad11b6d5fe54c26eafd, https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/76112d47aeaa22107c7b123ceb1e8896c6ae663d, and https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/c64da3eb37bc7e44470d7f5dfa5fbf2e03c4ae7f, at minimum. https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/pull/396 has a sus commit log, so I think something went wrong there. I'll take a look at fixing this.

Uggh, thanks for fixing.
On Tue, May 18, 2021, 4:43 PM Bryce Willey @.***> wrote:

Something went very wrong in c7d3822
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/c7d3822f925cc3193bffa838d2832eba0dc7aea2,
it reverted 99f00ce
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/99f00cecb46c89f7ce522ad11b6d5fe54c26eafd,
76112d4
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/76112d47aeaa22107c7b123ceb1e8896c6ae663d,
and c64da3e
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/c64da3eb37bc7e44470d7f5dfa5fbf2e03c4ae7f,
at minimum. #396
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/pull/396
has a sus commit log, so I think something went wrong there. I'll take a
look at fixing this.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/pull/387#issuecomment-843547016,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB2KTSLY4HJXE6T7GZAMEC3TOLGQTANCNFSM44ZG6SVA
.

I think https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/691f3c5ed309dd5ca49321678ead1ef2ce4484f7 and https://github.com/SuffolkLITLab/docassemble-assemblylinewizard/commit/e73be42403b5d7c19dc3d57ad024391c52b9128c should resolve things. Let me know if something from this PR didn't make it back in though, I had to edit some parts by hand, as reverting merges is notoriously difficult.


font request: OpenWRT

I wish I could be more helpful, but I'm not really even sure what kind of logo would work well for OpenWRT. Their current logo seems to be horizontal text:

I'm using these on the WireGuard install page, and OpenWRT is the only one missing a good logo.

I can't just invent an icon for a distribution, so unless they come up with something that works well in an icon font (near-square dimensions, recognisable) one day, I'm afraid this is a wontfix.


Support turn costs in 2024 default planner

As previously discussed, the new default planner needs to account for turn costs. The required changes are at two levels:

Expander needs to updated, to consider turn actions
State description needs to be updated, to add agent direction
Heuristic tables need to be extended, from (x, y) to (x, y, \vec{v}) where \vec{v} is a unit vector describing the agent's orientation.

update in branch 2024develop-pibt-turn

comments on 2024-9-6
Test the turning implementation

with first come first server task assignemnt
with minimal makespan task assignment.

What's the status of this one? I think we implemented (1) and (2) but not (3), is that right? @YueZhang-studyuse

I think this is done, sans heuristic. Seems to work well enough, even though the estimates are less accurate.