an error on Mesos

Good afternoon.
Thank you for CaffeOnSpark. May I ask a question?
I tried to run the MNIST example on CaffeOnSpark. In the local mode it worked fine. But in the Mesos cluster mode, there was an error. Would you help me? I attached the Spark log below.
 I copied lenet_memory_train_test.prototxt.mesos to each worker computer at the same path.
Then there was an error:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-examples-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
16/06/16 15:46:49 INFO SparkContext: Running Spark version 1.6.0
16/06/16 15:46:49 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/06/16 15:46:49 INFO SecurityManager: Changing view acls to: hadoop
16/06/16 15:46:49 INFO SecurityManager: Changing modify acls to: hadoop
16/06/16 15:46:49 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); users with modify permissions: Set(hadoop)
16/06/16 15:46:49 INFO Utils: Successfully started service 'sparkDriver' on port 33611.
16/06/16 15:46:49 INFO Slf4jLogger: Slf4jLogger started
16/06/16 15:46:49 INFO Remoting: Starting remoting
16/06/16 15:46:50 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@<IP_ADDRESS>:53685]
16/06/16 15:46:50 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 53685.
16/06/16 15:46:50 INFO SparkEnv: Registering MapOutputTracker
16/06/16 15:46:50 INFO SparkEnv: Registering BlockManagerMaster
16/06/16 15:46:50 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-7c2b7f3a-9a1a-4624-ace3-51bcdfa38d2e
16/06/16 15:46:50 INFO MemoryStore: MemoryStore started with capacity 511.1 MB
16/06/16 15:46:50 INFO SparkEnv: Registering OutputCommitCoordinator
16/06/16 15:46:50 INFO Utils: Successfully started service 'SparkUI' on port 4040.
16/06/16 15:46:50 INFO SparkUI: Started SparkUI at http://<IP_ADDRESS>:4040
16/06/16 15:46:50 INFO HttpFileServer: HTTP File server directory is /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/httpd-a725ec82-a8f4-4e18-8577-b7dfcbb93a46
16/06/16 15:46:50 INFO HttpServer: Starting HTTP Server
16/06/16 15:46:50 INFO Utils: Successfully started service 'HTTP file server' on port 46356.
16/06/16 15:46:50 INFO SparkContext: Added JAR file:/home/hadoop/jaewoo.song/CaffeOnSpark/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar at http://<IP_ADDRESS>:46356/jars/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar with timestamp<PHONE_NUMBER>288
16/06/16 15:46:50 INFO Utils: Copying /home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_solver.prototxt.mesos to /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/userFiles-443ed678-92ee-404d-bf06-5fddafd23d08/lenet_memory_solver.prototxt.mesos
16/06/16 15:46:50 INFO SparkContext: Added file file:/home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_solver.prototxt.mesos at http://<IP_ADDRESS>:46356/files/lenet_memory_solver.prototxt.mesos with timestamp<PHONE_NUMBER>375
16/06/16 15:46:50 INFO Utils: Copying /home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_train_test.prototxt.mesos to /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/userFiles-443ed678-92ee-404d-bf06-5fddafd23d08/lenet_memory_train_test.prototxt.mesos
16/06/16 15:46:50 INFO SparkContext: Added file file:/home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_train_test.prototxt.mesos at http://<IP_ADDRESS>:46356/files/lenet_memory_train_test.prototxt.mesos with timestamp<PHONE_NUMBER>383
I0616 15:46:50.541450  3442 sched.cpp:222] Version: 0.28.0
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@716: Client environment:host.name=hkcloud-node1
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.10.1.el7.x86_64
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Tue Feb 16 17:03:50 UTC 2016
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@733: Client environment:user.name=hadoop
2016-06-16 15:46:50,544:3351(0x7fc3593e4700):ZOO_INFO@log_env@741: Client environment:user.home=/home/hadoop
2016-06-16 15:46:50,544:3351(0x7fc3593e4700):ZOO_INFO@log_env@753: Client environment:user.dir=/home/hadoop/jaewoo.song/CaffeOnSpark
2016-06-16 15:46:50,544:3351(0x7fc3593e4700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=hkcloud-m:2181,hkcloud-node1:2181,hkcloud-node2:2181 sessionTimeout=10000 watcher=0x7fc365896280 sessionId=0 sessionPasswd=<null> context=0x7fc398004680 flags=0
2016-06-16 15:46:50,547:3351(0x7fc3572df700):ZOO_INFO@check_events@1703: initiated connection to server [<IP_ADDRESS>:2181]
2016-06-16 15:46:50,556:3351(0x7fc3572df700):ZOO_INFO@check_events@1750: session establishment complete on server [<IP_ADDRESS>:2181], sessionId=0x25515b774600033, negotiated timeout=10000
I0616 15:46:50.556560  3437 group.cpp:349] Group process (group(1)@<IP_ADDRESS>:48907) connected to ZooKeeper
I0616 15:46:50.556592  3437 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I0616 15:46:50.556603  3437 group.cpp:427] Trying to create path '/mesos' in ZooKeeper
I0616 15:46:50.557202  3437 detector.cpp:152] Detected a new leader: (id='86')
I0616 15:46:50.557274  3437 group.cpp:700] Trying to get '/mesos/json.info_0000000086' in ZooKeeper
I0616 15:46:50.557571  3437 detector.cpp:479] A new leading master (UPID=master@<IP_ADDRESS>:5050) is detected
I0616 15:46:50.557610  3437 sched.cpp:326] New master detected at master@<IP_ADDRESS>:5050
I0616 15:46:50.557930  3437 sched.cpp:336] No credentials provided. Attempting to register without authentication
I0616 15:46:50.564281  3437 sched.cpp:703] Framework registered with a179a27d-cd33-4e1d-a996-5afec0bc1c4b-0035
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Registered as framework ID a179a27d-cd33-4e1d-a996-5afec0bc1c4b-0035
16/06/16 15:46:50 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 35384.
16/06/16 15:46:50 INFO NettyBlockTransferService: Server created on 35384
16/06/16 15:46:50 INFO BlockManagerMaster: Trying to register BlockManager
16/06/16 15:46:50 INFO BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:35384 with 511.1 MB RAM, BlockManagerId(driver, <IP_ADDRESS>, 35384)
16/06/16 15:46:50 INFO BlockManagerMaster: Registered BlockManager
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Mesos task 0 is now TASK_RUNNING
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Mesos task 1 is now TASK_RUNNING
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Mesos task 2 is now TASK_RUNNING
16/06/16 15:46:52 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node2:49157) with ID f73f53d0-4f46-4691-ad01-5fed545d76e5-S0
16/06/16 15:46:52 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node1:60208) with ID 753a7274-173a-41bf-b800-0345adc29be0-S0
16/06/16 15:46:52 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node2:52796 with 511.1 MB RAM, BlockManagerId(f73f53d0-4f46-4691-ad01-5fed545d76e5-S0, hkcloud-node2, 52796)
16/06/16 15:46:52 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node1:55612 with 511.1 MB RAM, BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 55612)
16/06/16 15:46:52 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-m:60051) with ID 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3
16/06/16 15:46:52 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-m:41380 with 511.1 MB RAM, BlockManagerId(114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3, hkcloud-m, 41380)
16/06/16 15:47:20 INFO CoarseMesosSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000(ms)
16/06/16 15:47:21 INFO DataSource$: Source data layer:0
16/06/16 15:47:21 INFO LMDB: Batch size:1
16/06/16 15:47:21 INFO SparkContext: Starting job: collect at CaffeOnSpark.scala:127
16/06/16 15:47:21 INFO DAGScheduler: Got job 0 (collect at CaffeOnSpark.scala:127) with 1 output partitions
16/06/16 15:47:21 INFO DAGScheduler: Final stage: ResultStage 0 (collect at CaffeOnSpark.scala:127)
16/06/16 15:47:21 INFO DAGScheduler: Parents of final stage: List()
16/06/16 15:47:21 INFO DAGScheduler: Missing parents: List()
16/06/16 15:47:21 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[3] at map at CaffeOnSpark.scala:116), which has no missing parents
16/06/16 15:47:21 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.3 KB, free 3.3 KB)
16/06/16 15:47:21 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 2.1 KB, free 5.3 KB)
16/06/16 15:47:21 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:35384 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:21 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1006
16/06/16 15:47:21 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at map at CaffeOnSpark.scala:116)
16/06/16 15:47:21 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
16/06/16 15:47:21 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, hkcloud-node1, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:22 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-node1:55612 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:23 ERROR TaskSchedulerImpl: Lost executor 753a7274-173a-41bf-b800-0345adc29be0-S0 on hkcloud-node1: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:23 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, hkcloud-node1): ExecutorLostFailure (executor 753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:23 INFO DAGScheduler: Executor lost: 753a7274-173a-41bf-b800-0345adc29be0-S0 (epoch 0)
16/06/16 15:47:23 INFO BlockManagerMasterEndpoint: Trying to remove executor 753a7274-173a-41bf-b800-0345adc29be0-S0 from BlockManagerMaster.
16/06/16 15:47:23 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 55612)
16/06/16 15:47:23 INFO TaskSetManager: Starting task 0.1 in stage 0.0 (TID 1, hkcloud-node2, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:23 INFO BlockManagerMaster: Removed 753a7274-173a-41bf-b800-0345adc29be0-S0 successfully in removeExecutor
16/06/16 15:47:23 INFO CoarseMesosSchedulerBackend: Mesos task 2 is now TASK_FAILED
16/06/16 15:47:23 INFO CoarseMesosSchedulerBackend: Mesos task 3 is now TASK_RUNNING
16/06/16 15:47:24 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-node2:52796 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:24 ERROR TaskSchedulerImpl: Lost executor f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 on hkcloud-node2: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:24 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 1, hkcloud-node2): ExecutorLostFailure (executor f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:24 INFO DAGScheduler: Executor lost: f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 (epoch 1)
16/06/16 15:47:24 INFO BlockManagerMasterEndpoint: Trying to remove executor f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 from BlockManagerMaster.
16/06/16 15:47:24 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(f73f53d0-4f46-4691-ad01-5fed545d76e5-S0, hkcloud-node2, 52796)
16/06/16 15:47:24 INFO BlockManagerMaster: Removed f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 successfully in removeExecutor
16/06/16 15:47:24 INFO TaskSetManager: Starting task 0.2 in stage 0.0 (TID 2, hkcloud-m, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:24 INFO CoarseMesosSchedulerBackend: Mesos task 0 is now TASK_FAILED
16/06/16 15:47:24 INFO CoarseMesosSchedulerBackend: Mesos task 4 is now TASK_RUNNING
16/06/16 15:47:25 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-m:41380 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:25 ERROR TaskSchedulerImpl: Lost executor 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 on hkcloud-m: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:25 WARN TaskSetManager: Lost task 0.2 in stage 0.0 (TID 2, hkcloud-m): ExecutorLostFailure (executor 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:25 INFO DAGScheduler: Executor lost: 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 (epoch 2)
16/06/16 15:47:25 INFO BlockManagerMasterEndpoint: Trying to remove executor 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 from BlockManagerMaster.
16/06/16 15:47:25 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3, hkcloud-m, 41380)
16/06/16 15:47:25 INFO BlockManagerMaster: Removed 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 successfully in removeExecutor
16/06/16 15:47:25 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node1:60216) with ID 753a7274-173a-41bf-b800-0345adc29be0-S0
16/06/16 15:47:25 INFO DAGScheduler: Host added was in lost list earlier: hkcloud-node1
16/06/16 15:47:25 INFO TaskSetManager: Starting task 0.3 in stage 0.0 (TID 3, hkcloud-node1, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:25 INFO CoarseMesosSchedulerBackend: Mesos task 1 is now TASK_FAILED
16/06/16 15:47:25 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node1:56506 with 511.1 MB RAM, BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 56506)
16/06/16 15:47:25 INFO CoarseMesosSchedulerBackend: Mesos task 5 is now TASK_RUNNING
16/06/16 15:47:26 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-node1:56506 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node2:49171) with ID f73f53d0-4f46-4691-ad01-5fed545d76e5-S0
16/06/16 15:47:26 INFO DAGScheduler: Host added was in lost list earlier: hkcloud-node2
16/06/16 15:47:26 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node2:36823 with 511.1 MB RAM, BlockManagerId(f73f53d0-4f46-4691-ad01-5fed545d76e5-S0, hkcloud-node2, 36823)
16/06/16 15:47:26 ERROR TaskSchedulerImpl: Lost executor 753a7274-173a-41bf-b800-0345adc29be0-S0 on hkcloud-node1: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:26 WARN TaskSetManager: Lost task 0.3 in stage 0.0 (TID 3, hkcloud-node1): ExecutorLostFailure (executor 753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:26 ERROR TaskSetManager: Task 0 in stage 0.0 failed 4 times; aborting job
16/06/16 15:47:26 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/06/16 15:47:26 INFO TaskSchedulerImpl: Cancelling stage 0
16/06/16 15:47:26 INFO DAGScheduler: ResultStage 0 (collect at CaffeOnSpark.scala:127) failed in 4.689 s
16/06/16 15:47:26 INFO DAGScheduler: Executor lost: 753a7274-173a-41bf-b800-0345adc29be0-S0 (epoch 3)
16/06/16 15:47:26 INFO DAGScheduler: Job 0 failed: collect at CaffeOnSpark.scala:127, took 4.909061 s
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, hkcloud-node1): ExecutorLostFailure (executor 753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
	at scala.Option.foreach(Option.scala:236)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
	at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:927)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
	at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
	at org.apache.spark.rdd.RDD.collect(RDD.scala:926)
	at com.yahoo.ml.caffe.CaffeOnSpark.train(CaffeOnSpark.scala:127)
	at com.yahoo.ml.caffe.CaffeOnSpark$.main(CaffeOnSpark.scala:40)
	at com.yahoo.ml.caffe.CaffeOnSpark.main(CaffeOnSpark.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/06/16 15:47:26 INFO SparkContext: Invoking stop() from shutdown hook
16/06/16 15:47:26 INFO BlockManagerMasterEndpoint: Trying to remove executor 753a7274-173a-41bf-b800-0345adc29be0-S0 from BlockManagerMaster.
16/06/16 15:47:26 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 56506)
16/06/16 15:47:26 INFO BlockManagerMaster: Removed 753a7274-173a-41bf-b800-0345adc29be0-S0 successfully in removeExecutor
16/06/16 15:47:26 INFO SparkUI: Stopped Spark web UI at http://<IP_ADDRESS>:4040
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Mesos task 3 is now TASK_FAILED
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Blacklisting Mesos slave 753a7274-173a-41bf-b800-0345adc29be0-S0 due to too many failures; is Spark installed on it?
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Shutting down all executors
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Asking each executor to shut down
I0616 15:47:26.619835  3641 sched.cpp:1903] Asked to stop the driver
I0616 15:47:26.622957  3433 sched.cpp:1143] Stopping framework 'a179a27d-cd33-4e1d-a996-5afec0bc1c4b-0035'
16/06/16 15:47:26 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: driver.run() returned with code DRIVER_STOPPED
16/06/16 15:47:26 INFO MemoryStore: MemoryStore cleared
16/06/16 15:47:26 INFO BlockManager: BlockManager stopped
16/06/16 15:47:26 INFO BlockManagerMaster: BlockManagerMaster stopped
16/06/16 15:47:26 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/06/16 15:47:26 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/06/16 15:47:26 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/06/16 15:47:26 INFO SparkContext: Successfully stopped SparkContext
16/06/16 15:47:26 INFO ShutdownHookManager: Shutdown hook called
16/06/16 15:47:26 INFO ShutdownHookManager: Deleting directory /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/httpd-a725ec82-a8f4-4e18-8577-b7dfcbb93a46
16/06/16 15:47:26 INFO ShutdownHookManager: Deleting directory /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235

I had changed the batch size to 1 to avoid any not-enough-memory problem. Meanwhile, a default example Spark program in spark-directory/example/src/main/python/pi.py ran fine in the setting. Also, the CaffeOnSpark MNIST example ran okay in a local mode.
Thank you for your time and concern.
Regards,
Jaewoo

Jaewoo:
I am not familiar with Spark on Mesos.
Your log indicates that one executor failed.

16/06/16 15:47:23 WARN TaskSetManager: Lost task 0.0 in stage 0.0
(TID 0, hkcloud-node1): ExecutorLostFailure (executor
753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the
running tasks) Reason: Remote RPC client disassociated. Likely due to
containers exceeding thresholds, or network issues. Check driver logs
for WARN messages.
Please check out your executor log.
How many executors do you use? What's the value for ${DEVICES}?
Andy
Andy
On Thu, Jun 16, 2016 at 1:25 AM, jsongcse<EMAIL_ADDRESS>wrote:

Good afternoon.
Thank you for CaffeOnSpark. May I ask a question?
I tried to run the MNIST example on CaffeOnSpark. In the local mode it
worked fine. But in the Mesos cluster mode, there was an error. Would you
help me? I attached the Spark log below.
First, my command was:
/opt/spark-1.6.0-bin-hadoop2.6/bin/spark-submit
--files ${CAFFE_ON_SPARK}/data/lenet_memory_solver.prototxt.mesos,${CAFFE_ON_SPARK}/data/lenet_memory_train_test.prototxt.mesos
--conf spark.driver.extraLibraryPath="${LD_LIBRARY_PATH}"
--conf spark.executorEnv.LD_LIBRARY_PATH="${LD_LIBRARY_PATH}"
--class com.yahoo.ml.caffe.CaffeOnSpark
${CAFFE_ON_SPARK}/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar
-train
-features accuracy,loss -label label
-conf ${CAFFE_ON_SPARK}/data/lenet_memory_solver.prototxt.mesos
-devices ${DEVICES}
-connection ethernet
-model hdfs:///mnist.model
-output hdfs:///mnist_features_result
I modified ${CAFFE_ON_SPARK}/data/lenet_memory_solver.prototxt.mesos so
that it would look for
${CAFFE_ON_SPARK}/data/lenet_memory_train_test.prototxt.mesos in each
worker computer. I copied lenet_memory_train_test.prototxt.mesos to each
worker computer at the same path.
Then there was an error:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/spark-1.6.0-bin-hadoop2.6/lib/spark-examples-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
16/06/16 15:46:49 INFO SparkContext: Running Spark version 1.6.0
16/06/16 15:46:49 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/06/16 15:46:49 INFO SecurityManager: Changing view acls to: hadoop
16/06/16 15:46:49 INFO SecurityManager: Changing modify acls to: hadoop
16/06/16 15:46:49 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); users with modify permissions: Set(hadoop)
16/06/16 15:46:49 INFO Utils: Successfully started service 'sparkDriver' on port 33611.
16/06/16 15:46:49 INFO Slf4jLogger: Slf4jLogger started
16/06/16 15:46:49 INFO Remoting: Starting remoting
16/06/16 15:46:50 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@<IP_ADDRESS>:53685]
16/06/16 15:46:50 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 53685.
16/06/16 15:46:50 INFO SparkEnv: Registering MapOutputTracker
16/06/16 15:46:50 INFO SparkEnv: Registering BlockManagerMaster
16/06/16 15:46:50 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-7c2b7f3a-9a1a-4624-ace3-51bcdfa38d2e
16/06/16 15:46:50 INFO MemoryStore: MemoryStore started with capacity 511.1 MB
16/06/16 15:46:50 INFO SparkEnv: Registering OutputCommitCoordinator
16/06/16 15:46:50 INFO Utils: Successfully started service 'SparkUI' on port 4040.
16/06/16 15:46:50 INFO SparkUI: Started SparkUI at http://<IP_ADDRESS>:4040
16/06/16 15:46:50 INFO HttpFileServer: HTTP File server directory is /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/httpd-a725ec82-a8f4-4e18-8577-b7dfcbb93a46
16/06/16 15:46:50 INFO HttpServer: Starting HTTP Server
16/06/16 15:46:50 INFO Utils: Successfully started service 'HTTP file server' on port 46356.
16/06/16 15:46:50 INFO SparkContext: Added JAR file:/home/hadoop/jaewoo.song/CaffeOnSpark/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar at http://<IP_ADDRESS>:46356/jars/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar with timestamp<PHONE_NUMBER>288
16/06/16 15:46:50 INFO Utils: Copying /home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_solver.prototxt.mesos to /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/userFiles-443ed678-92ee-404d-bf06-5fddafd23d08/lenet_memory_solver.prototxt.mesos
16/06/16 15:46:50 INFO SparkContext: Added file file:/home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_solver.prototxt.mesos at http://<IP_ADDRESS>:46356/files/lenet_memory_solver.prototxt.mesos with timestamp<PHONE_NUMBER>375
16/06/16 15:46:50 INFO Utils: Copying /home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_train_test.prototxt.mesos to /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/userFiles-443ed678-92ee-404d-bf06-5fddafd23d08/lenet_memory_train_test.prototxt.mesos
16/06/16 15:46:50 INFO SparkContext: Added file file:/home/hadoop/jaewoo.song/CaffeOnSpark/data/lenet_memory_train_test.prototxt.mesos at http://<IP_ADDRESS>:46356/files/lenet_memory_train_test.prototxt.mesos with timestamp<PHONE_NUMBER>383
I0616 15:46:50.541450  3442 sched.cpp:222] Version: 0.28.0
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@716: Client environment:host.name=hkcloud-node1
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.10.1.el7.x86_64
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Tue Feb 16 17:03:50 UTC 2016
2016-06-16 15:46:50,543:3351(0x7fc3593e4700):ZOO_INFO@log_env@733: Client environment:user.name=hadoop
2016-06-16 15:46:50,544:3351(0x7fc3593e4700):ZOO_INFO@log_env@741: Client environment:user.home=/home/hadoop
2016-06-16 15:46:50,544:3351(0x7fc3593e4700):ZOO_INFO@log_env@753: Client environment:user.dir=/home/hadoop/jaewoo.song/CaffeOnSpark
2016-06-16 15:46:50,544:3351(0x7fc3593e4700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=hkcloud-m:2181,hkcloud-node1:2181,hkcloud-node2:2181 sessionTimeout=10000 watcher=0x7fc365896280 sessionId=0 sessionPasswd= context=0x7fc398004680 flags=0
2016-06-16 15:46:50,547:3351(0x7fc3572df700):ZOO_INFO@check_events@1703: initiated connection to server [<IP_ADDRESS>:2181]
2016-06-16 15:46:50,556:3351(0x7fc3572df700):ZOO_INFO@check_events@1750: session establishment complete on server [<IP_ADDRESS>:2181], sessionId=0x25515b774600033, negotiated timeout=10000
I0616 15:46:50.556560  3437 group.cpp:349] Group process (group(1)@<IP_ADDRESS>:48907) connected to ZooKeeper
I0616 15:46:50.556592  3437 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I0616 15:46:50.556603  3437 group.cpp:427] Trying to create path '/mesos' in ZooKeeper
I0616 15:46:50.557202  3437 detector.cpp:152] Detected a new leader: (id='86')
I0616 15:46:50.557274  3437 group.cpp:700] Trying to get '/mesos/json.info_0000000086' in ZooKeeper
I0616 15:46:50.557571  3437 detector.cpp:479] A new leading master (UPID=master@<IP_ADDRESS>:5050) is detected
I0616 15:46:50.557610  3437 sched.cpp:326] New master detected at master@<IP_ADDRESS>:5050
I0616 15:46:50.557930  3437 sched.cpp:336] No credentials provided. Attempting to register without authentication
I0616 15:46:50.564281  3437 sched.cpp:703] Framework registered with a179a27d-cd33-4e1d-a996-5afec0bc1c4b-0035
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Registered as framework ID a179a27d-cd33-4e1d-a996-5afec0bc1c4b-0035
16/06/16 15:46:50 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 35384.
16/06/16 15:46:50 INFO NettyBlockTransferService: Server created on 35384
16/06/16 15:46:50 INFO BlockManagerMaster: Trying to register BlockManager
16/06/16 15:46:50 INFO BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:35384 with 511.1 MB RAM, BlockManagerId(driver, <IP_ADDRESS>, 35384)
16/06/16 15:46:50 INFO BlockManagerMaster: Registered BlockManager
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Mesos task 0 is now TASK_RUNNING
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Mesos task 1 is now TASK_RUNNING
16/06/16 15:46:50 INFO CoarseMesosSchedulerBackend: Mesos task 2 is now TASK_RUNNING
16/06/16 15:46:52 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node2:49157) with ID f73f53d0-4f46-4691-ad01-5fed545d76e5-S0
16/06/16 15:46:52 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node1:60208) with ID 753a7274-173a-41bf-b800-0345adc29be0-S0
16/06/16 15:46:52 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node2:52796 with 511.1 MB RAM, BlockManagerId(f73f53d0-4f46-4691-ad01-5fed545d76e5-S0, hkcloud-node2, 52796)
16/06/16 15:46:52 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node1:55612 with 511.1 MB RAM, BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 55612)
16/06/16 15:46:52 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-m:60051) with ID 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3
16/06/16 15:46:52 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-m:41380 with 511.1 MB RAM, BlockManagerId(114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3, hkcloud-m, 41380)
16/06/16 15:47:20 INFO CoarseMesosSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000(ms)
16/06/16 15:47:21 INFO DataSource$: Source data layer:0
16/06/16 15:47:21 INFO LMDB: Batch size:1
16/06/16 15:47:21 INFO SparkContext: Starting job: collect at CaffeOnSpark.scala:127
16/06/16 15:47:21 INFO DAGScheduler: Got job 0 (collect at CaffeOnSpark.scala:127) with 1 output partitions
16/06/16 15:47:21 INFO DAGScheduler: Final stage: ResultStage 0 (collect at CaffeOnSpark.scala:127)
16/06/16 15:47:21 INFO DAGScheduler: Parents of final stage: List()
16/06/16 15:47:21 INFO DAGScheduler: Missing parents: List()
16/06/16 15:47:21 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[3] at map at CaffeOnSpark.scala:116), which has no missing parents
16/06/16 15:47:21 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 3.3 KB, free 3.3 KB)
16/06/16 15:47:21 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 2.1 KB, free 5.3 KB)
16/06/16 15:47:21 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:35384 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:21 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1006
16/06/16 15:47:21 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at map at CaffeOnSpark.scala:116)
16/06/16 15:47:21 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
16/06/16 15:47:21 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, hkcloud-node1, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:22 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-node1:55612 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:23 ERROR TaskSchedulerImpl: Lost executor 753a7274-173a-41bf-b800-0345adc29be0-S0 on hkcloud-node1: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:23 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, hkcloud-node1): ExecutorLostFailure (executor 753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:23 INFO DAGScheduler: Executor lost: 753a7274-173a-41bf-b800-0345adc29be0-S0 (epoch 0)
16/06/16 15:47:23 INFO BlockManagerMasterEndpoint: Trying to remove executor 753a7274-173a-41bf-b800-0345adc29be0-S0 from BlockManagerMaster.
16/06/16 15:47:23 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 55612)
16/06/16 15:47:23 INFO TaskSetManager: Starting task 0.1 in stage 0.0 (TID 1, hkcloud-node2, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:23 INFO BlockManagerMaster: Removed 753a7274-173a-41bf-b800-0345adc29be0-S0 successfully in removeExecutor
16/06/16 15:47:23 INFO CoarseMesosSchedulerBackend: Mesos task 2 is now TASK_FAILED
16/06/16 15:47:23 INFO CoarseMesosSchedulerBackend: Mesos task 3 is now TASK_RUNNING
16/06/16 15:47:24 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-node2:52796 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:24 ERROR TaskSchedulerImpl: Lost executor f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 on hkcloud-node2: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:24 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 1, hkcloud-node2): ExecutorLostFailure (executor f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:24 INFO DAGScheduler: Executor lost: f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 (epoch 1)
16/06/16 15:47:24 INFO BlockManagerMasterEndpoint: Trying to remove executor f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 from BlockManagerMaster.
16/06/16 15:47:24 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(f73f53d0-4f46-4691-ad01-5fed545d76e5-S0, hkcloud-node2, 52796)
16/06/16 15:47:24 INFO BlockManagerMaster: Removed f73f53d0-4f46-4691-ad01-5fed545d76e5-S0 successfully in removeExecutor
16/06/16 15:47:24 INFO TaskSetManager: Starting task 0.2 in stage 0.0 (TID 2, hkcloud-m, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:24 INFO CoarseMesosSchedulerBackend: Mesos task 0 is now TASK_FAILED
16/06/16 15:47:24 INFO CoarseMesosSchedulerBackend: Mesos task 4 is now TASK_RUNNING
16/06/16 15:47:25 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-m:41380 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:25 ERROR TaskSchedulerImpl: Lost executor 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 on hkcloud-m: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:25 WARN TaskSetManager: Lost task 0.2 in stage 0.0 (TID 2, hkcloud-m): ExecutorLostFailure (executor 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:25 INFO DAGScheduler: Executor lost: 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 (epoch 2)
16/06/16 15:47:25 INFO BlockManagerMasterEndpoint: Trying to remove executor 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 from BlockManagerMaster.
16/06/16 15:47:25 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3, hkcloud-m, 41380)
16/06/16 15:47:25 INFO BlockManagerMaster: Removed 114d8677-dbe9-4b2a-bf79-dc4a34d2cede-S3 successfully in removeExecutor
16/06/16 15:47:25 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node1:60216) with ID 753a7274-173a-41bf-b800-0345adc29be0-S0
16/06/16 15:47:25 INFO DAGScheduler: Host added was in lost list earlier: hkcloud-node1
16/06/16 15:47:25 INFO TaskSetManager: Starting task 0.3 in stage 0.0 (TID 3, hkcloud-node1, partition 0,PROCESS_LOCAL, 2209 bytes)
16/06/16 15:47:25 INFO CoarseMesosSchedulerBackend: Mesos task 1 is now TASK_FAILED
16/06/16 15:47:25 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node1:56506 with 511.1 MB RAM, BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 56506)
16/06/16 15:47:25 INFO CoarseMesosSchedulerBackend: Mesos task 5 is now TASK_RUNNING
16/06/16 15:47:26 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on hkcloud-node1:56506 (size: 2.1 KB, free: 511.1 MB)
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (hkcloud-node2:49171) with ID f73f53d0-4f46-4691-ad01-5fed545d76e5-S0
16/06/16 15:47:26 INFO DAGScheduler: Host added was in lost list earlier: hkcloud-node2
16/06/16 15:47:26 INFO BlockManagerMasterEndpoint: Registering block manager hkcloud-node2:36823 with 511.1 MB RAM, BlockManagerId(f73f53d0-4f46-4691-ad01-5fed545d76e5-S0, hkcloud-node2, 36823)
16/06/16 15:47:26 ERROR TaskSchedulerImpl: Lost executor 753a7274-173a-41bf-b800-0345adc29be0-S0 on hkcloud-node1: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:26 WARN TaskSetManager: Lost task 0.3 in stage 0.0 (TID 3, hkcloud-node1): ExecutorLostFailure (executor 753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/16 15:47:26 ERROR TaskSetManager: Task 0 in stage 0.0 failed 4 times; aborting job
16/06/16 15:47:26 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/06/16 15:47:26 INFO TaskSchedulerImpl: Cancelling stage 0
16/06/16 15:47:26 INFO DAGScheduler: ResultStage 0 (collect at CaffeOnSpark.scala:127) failed in 4.689 s
16/06/16 15:47:26 INFO DAGScheduler: Executor lost: 753a7274-173a-41bf-b800-0345adc29be0-S0 (epoch 3)
16/06/16 15:47:26 INFO DAGScheduler: Job 0 failed: collect at CaffeOnSpark.scala:127, took 4.909061 s
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, hkcloud-node1): ExecutorLostFailure (executor 753a7274-173a-41bf-b800-0345adc29be0-S0 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:927)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.collect(RDD.scala:926)
at com.yahoo.ml.caffe.CaffeOnSpark.train(CaffeOnSpark.scala:127)
at com.yahoo.ml.caffe.CaffeOnSpark$.main(CaffeOnSpark.scala:40)
at com.yahoo.ml.caffe.CaffeOnSpark.main(CaffeOnSpark.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/06/16 15:47:26 INFO SparkContext: Invoking stop() from shutdown hook
16/06/16 15:47:26 INFO BlockManagerMasterEndpoint: Trying to remove executor 753a7274-173a-41bf-b800-0345adc29be0-S0 from BlockManagerMaster.
16/06/16 15:47:26 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(753a7274-173a-41bf-b800-0345adc29be0-S0, hkcloud-node1, 56506)
16/06/16 15:47:26 INFO BlockManagerMaster: Removed 753a7274-173a-41bf-b800-0345adc29be0-S0 successfully in removeExecutor
16/06/16 15:47:26 INFO SparkUI: Stopped Spark web UI at http://<IP_ADDRESS>:4040
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Mesos task 3 is now TASK_FAILED
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Blacklisting Mesos slave 753a7274-173a-41bf-b800-0345adc29be0-S0 due to too many failures; is Spark installed on it?
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Shutting down all executors
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: Asking each executor to shut down
I0616 15:47:26.619835  3641 sched.cpp:1903] Asked to stop the driver
I0616 15:47:26.622957  3433 sched.cpp:1143] Stopping framework 'a179a27d-cd33-4e1d-a996-5afec0bc1c4b-0035'
16/06/16 15:47:26 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/06/16 15:47:26 INFO CoarseMesosSchedulerBackend: driver.run() returned with code DRIVER_STOPPED
16/06/16 15:47:26 INFO MemoryStore: MemoryStore cleared
16/06/16 15:47:26 INFO BlockManager: BlockManager stopped
16/06/16 15:47:26 INFO BlockManagerMaster: BlockManagerMaster stopped
16/06/16 15:47:26 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/06/16 15:47:26 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/06/16 15:47:26 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/06/16 15:47:26 INFO SparkContext: Successfully stopped SparkContext
16/06/16 15:47:26 INFO ShutdownHookManager: Shutdown hook called
16/06/16 15:47:26 INFO ShutdownHookManager: Deleting directory /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235/httpd-a725ec82-a8f4-4e18-8577-b7dfcbb93a46
16/06/16 15:47:26 INFO ShutdownHookManager: Deleting directory /tmp/spark-e28ec4b7-cffb-474d-a920-e65632853235
I had changed the batch size to 1 to avoid any not-enough-memory problem.
Meanwhile, a default example Spark program in
spark-directory/example/src/main/python/pi.py ran fine in the setting.
Also, the CaffeOnSpark MNIST example ran okay in a local mode.
Thank you for your time and concern.
Regards,
Jaewoo
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yahoo/CaffeOnSpark/issues/86, or mute the thread
https://github.com/notifications/unsubscribe/AClTeNpd6svlkz47DRyzOQYL8SwWcZxgks5qMQh3gaJpZM4I3I9X
.

Dear Andy,
Good afternoon. Thank you for your reply.
Below is what I found in the log directory of spark master node. It seems there is not much information. (I tried to find logs of spark worker nodes but I couldn't.)
{"Event":"SparkListenerLogStart","Spark Version":"1.6.0"}
{"Event":"SparkListenerBlockManagerAdded","Block Manager ID":{"Executor ID":"driver","Host":"<IP_ADDRESS>","Port":60428},"Maximum Memory":1307836416,"Timestamp":1466481232878}

Three computers are used for Apache Spark, and the cluster manager is Apache Mesos. I chose 2 for ${DEVICES}.
Thank you for your help.
Regards,
Jaewoo

It was a problem about -clusterSize and -devices options. Apache Mesos also needs -clusterSize as same as YARN. Thank you for your help and encourage.

@jsongcse Please close this if it works. Would be great if you could send the doc for mesos setup (like the way we have a doc for yarn setup: https://github.com/yahoo/CaffeOnSpark/wiki/GetStarted_yarn)

Thank you for your help.


Add s3_session_token to Parquet S3 guide

This was inspired by a comment here: https://github.com/duckdb/duckdb/pull/2234

Thanks!


Invalid status code for player api response: 400

Basic Troubleshooting

[X] I have checked for similar issues (open AND closed), and my issue is not a duplicate. (MANDATORY)
[X] I have checked for pull requests that may already address my issue. (MANDATORY)
[X] I am using (and my issue is reproducible on) the latest version of youtube-source.
[X] My issue is reproducible with IPv6 rotation.

Version of youtube-source
1.8.3
The search query/queries, URL(s), playlist ID(s)/URL(s) or video ID(s)/URL(s) that triggered the issue
ytsearch
Code Example
N/A
Exception and Stacktrace
2024-11-15T19:09:31.409+07:00 ERROR 6792 --- [Lavalink] [lava-daemon-pool-playback-1-thread-1] c.s.d.l.t.p.LocalAudioTrackExecutor      : Error in playback of 3jwxsav_Ezk

com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Something broke when playing the track.
	at com.sedmelluq.discord.lavaplayer.tools.ExceptionTools.wrapUnfriendlyExceptions(ExceptionTools.java:45) ~[lavaplayer-2.2.1.jar!/:na]
	at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:121) ~[lavaplayer-2.2.1.jar!/:na]
	at com.sedmelluq.discord.lavaplayer.player.DefaultAudioPlayerManager.lambda$executeTrack$2(DefaultAudioPlayerManager.java:339) ~[lavaplayer-2.2.1.jar!/:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:1575) ~[na:na]
Caused by: java.io.IOException: Invalid status code for player api response: 400
	at com.sedmelluq.discord.lavaplayer.tools.io.HttpClientTools.assertSuccessWithContent(HttpClientTools.java:154) ~[lavaplayer-2.2.1.jar!/:na]
	at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadJsonResponse(NonMusicClient.java:67) ~[jar:file:C:\Users\Administrator\Desktop\Music Port\.\plugins\youtube-plugin-1.8.3.jar!/:na]
	at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:107) ~[jar:file:C:\Users\Administrator\Desktop\Music Port\.\plugins\youtube-plugin-1.8.3.jar!/:na]
	at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35) ~[jar:file:C:\Users\Administrator\Desktop\Music Port\.\plugins\youtube-plugin-1.8.3.jar!/:na]
	at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:193) ~[jar:file:C:\Users\Administrator\Desktop\Music Port\.\plugins\youtube-plugin-1.8.3.jar!/:na]
	at dev.lavalink.youtube.track.YoutubeAudioTrack.processWithClient(YoutubeAudioTrack.java:119) ~[jar:file:C:\Users\Administrator\Desktop\Music Port\.\plugins\youtube-plugin-1.8.3.jar!/:na]
	at dev.lavalink.youtube.track.YoutubeAudioTrack.process(YoutubeAudioTrack.java:72) ~[jar:file:C:\Users\Administrator\Desktop\Music Port\.\plugins\youtube-plugin-1.8.3.jar!/:na]
	at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:109) ~[lavaplayer-2.2.1.jar!/:na]
	... 4 common frames omitted

What is your client configuration?
server: # REST and WS server   port: 25581 # The port that the server listens on   address: <IP_ADDRESS>   http2:     enabled: false # Whether to enable HTTP/2 support plugins:   dunctebot:     ttsLanguage: 'en-US' # language of the TTS engine     sources:       # true = source enabled, false = source disabled       getyarn: true # www.getyarn.io       clypit: true # www.clyp.it       tts: true # speak:Words to speak       pornhub: true # should be self-explanatory       reddit: true # should be self-explanatory       ocremix: true # www.ocremix.org       tiktok: true # tiktok.com       mixcloud: true # mixcloud.com       soundgasm: true # soundgasm.net       pixeldrain: true # pixeldrain.com   jiosaavn:     apiURL: "https://jiosaavn-plugin-api.deno.dev" # JioSaavn API URL     playlistTrackLimit: 50 # The maximum number of tracks to return from given playlist (default 50 tracks)     recommendationsTrackLimit: 10 # The maximum number of track to return from recommendations (default 10 tracks)   lavasrc:     providers: # Custom providers for track loading. This is the default       # - "dzisrc:%ISRC%" # Deezer ISRC provider       # - "dzsearch:%QUERY%" # Deezer search provider       - "ytsearch:\"%ISRC%\"" # Will be ignored if track does not have an ISRC. See https://en.wikipedia.org/wiki/International_Standard_Recording_Code       - "ytsearch:%QUERY%" # Will be used if track has no ISRC or no track could be found for the ISRC       #  you can add multiple other fallback sources here     sources:       spotify: true # Enable Spotify source       applemusic: false # Enable Apple Music source       deezer: false # Enable Deezer source       yandexmusic: false # Enable Yandex Music source       flowerytts: false # Enable Flowery TTS source       youtube: true # Enable YouTube search source (https://github.com/topi314/LavaSearch)       vkmusic: false # Enable Vk Music source     lyrics-sources:       spotify: true # Enable Spotify lyrics source       deezer: false # Enable Deezer lyrics source       youtube: true # Enable YouTube lyrics source       yandexmusic: false # Enable Yandex Music lyrics source       vkmusic: false # Enable Vk Music lyrics source     spotify:       clientId: "df09524006714bbdaae89715df577981"       clientSecret: "09fcf59d3cff4df78248f6781694e1e1"       # spDc: "your sp dc cookie" # the sp dc cookie used for accessing the spotify lyrics api       countryCode: "US" # the country code you want to use for filtering the artists top tracks. See https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2       playlistLoadLimit: 6 # The number of pages at 100 tracks each       albumLoadLimit: 6 # The number of pages at 50 tracks each       resolveArtistsInSearch: true # Whether to resolve artists in track search results (can be slow)       localFiles: false # Enable local files support with Spotify playlists. Please note `uri` & `isrc` will be `null` & `identifier` will be `"local"`     applemusic:       countryCode: "US" # the country code you want to use for filtering the artists top tracks and language. See https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2       mediaAPIToken: "your apple music api token" # apple music api token       # or specify an apple music key       keyID: "your key id"       teamID: "your team id"       musicKitKey: |         -----BEGIN PRIVATE KEY-----         your key         -----END PRIVATE KEY-----             playlistLoadLimit: 6 # The number of pages at 300 tracks each       albumLoadLimit: 6 # The number of pages at 300 tracks each     deezer:       masterDecryptionKey: "your master decryption key" # the master key used for decrypting the deezer tracks. (yes this is not here you need to get it from somewhere else)       # arl: "your deezer arl" # the arl cookie used for accessing the deezer api this is optional but required for formats above MP3_128       formats: [ "FLAC", "MP3_320", "MP3_256", "MP3_128", "MP3_64", "AAC_64" ] # the formats you want to use for the deezer tracks. "FLAC", "MP3_320", "MP3_256" & "AAC_64" are only available for premium users and require a valid arl     yandexmusic:       accessToken: "your access token" # the token used for accessing the yandex music api. See https://github.com/TopiSenpai/LavaSrc#yandex-music       playlistLoadLimit: 1 # The number of pages at 100 tracks each       albumLoadLimit: 1 # The number of pages at 50 tracks each       artistLoadLimit: 1 # The number of pages at 10 tracks each     flowerytts:       voice: "default voice" # (case-sensitive) get default voice from here https://api.flowery.pw/v1/tts/voices       translate: false # whether to translate the text to the native language of voice       silence: 0 # the silence parameter is in milliseconds. Range is 0 to 10000. The default is 0.       speed: 1.0 # the speed parameter is a float between 0.5 and 10. The default is 1.0. (0.5 is half speed, 2.0 is double speed, etc.)       audioFormat: "mp3" # supported formats are: mp3, ogg_opus, ogg_vorbis, aac, wav, and flac. Default format is mp3     youtube:       countryCode: "US" # the country code you want to use for searching lyrics via ISRC. See https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2     vkmusic:       userToken: "your user token" # This token is needed for authorization in the api. Guide: https://github.com/topi314/LavaSrc#vk-music       playlistLoadLimit: 1 # The number of pages at 50 tracks each       artistLoadLimit: 1 # The number of pages at 10 tracks each       recommendationsLoadLimit: 10 # Number of tracks   youtube:     enabled: true # Whether this source can be used.     allowSearch: true # Whether "ytsearch:" and "ytmsearch:" can be used.     allowDirectVideoIds: true # Whether just video IDs can match. If false, only complete URLs will be loaded.     allowDirectPlaylistIds: true # Whether just playlist IDs can match. If false, only complete URLs will be loaded.     # The clients to use for track loading. See below for a list of valid clients.     # Clients are queried in the order they are given (so the first client is queried first and so on...)     clients:       - ANDROID_MUSIC       - MUSIC       - WEB       - WEBEMBEDDED       - TVHTML5EMBEDDED       # This secure your lavalink is working and playing music if you used this two things below.     oauth:       enabled: true # IF YOU RUN YOUR LAVALINK CHECK YOUR CONSOLE AND CLICK THE GOOGLE.COM/DEVICES LINK AND THERE IS A CODE THAT YOU NEED TO PUT IN TO THE GOOGLE AND USE A BURNER ACCOUNT OR ALT ACCOUNT THERE IS A POSSIBLE CHANGE YOU CAN GET BANNED FROM GOOGLE OR YOUTUBE SO JUST USE A BURNER ACCOUNT IN CASE.     #pot:    #   token: "" # THIS REQUIRE INSTALLING https://github.com/iv-org/youtube-trusted-session-generator THIS IS A PYTHON FILE INSTALL THE REQUIREMENTS AND RUN PYTON INDEX FILE AND AFTER RUNNING YOU WILL RECIVE AND TOKEN AND VISTOR DATA AND PASTE IT BELOW.    #   visitorData: "" lavalink:   plugins:     - dependency: "com.github.appujet:jiosaavn-plugin:1.0.2"       repository: "https://jitpack.io"     - dependency: "com.dunctebot:skybot-lavalink-plugin:1.7.0"       snapshot: false # set to true if you want to use snapshot builds     - dependency: "com.github.topi314.lavasearch:lavasearch-plugin:1.0.0"       snapshot: false # set to true if you want to use snapshot builds     - dependency: "com.github.topi314.lavasrc:lavasrc-plugin:4.3.0"       snapshot: false # set to true if you want to use snapshot builds     - dependency: "com.github.topi314.sponsorblock:sponsorblock-plugin:3.0.1"       snapshot: false # set to true if you want to use snapshot builds     - dependency: "dev.lavalink.youtube:youtube-plugin:1.8.3"       snapshot: false # set to true if you want to use snapshot builds   pluginsDir: './plugins'   server:     password: "localhost"     sources:       # The default Youtube source is now deprecated and won't receive further updates. Please use https://github.com/lavalink-devs/youtube-source#plugin instead.       youtube: false       bandcamp: true       soundcloud: true       twitch: true       vimeo: true       mixer: true       nico: true       http: true # warning: keeping HTTP enabled without a proxy configured could expose your server's IP address.       local: false     filters: # All filters are enabled by default       volume: true       equalizer: true       karaoke: true       timescale: true       tremolo: true       vibrato: true       distortion: true       rotation: true       channelMix: true       lowPass: true     bufferDurationMs: 400 # The duration of the NAS buffer. Higher values fare better against longer GC pauses. Duration <= 0 to disable JDA-NAS. Minimum of 40ms, lower values may introduce pauses.     frameBufferDurationMs: 5000 # How many milliseconds of audio to keep buffered     opusEncodingQuality: 10 # Opus encoder quality. Valid values range from 0 to 10, where 10 is best quality but is the most expensive on the CPU.     resamplingQuality: MEDIUM # Quality of resampling operations. Valid values are LOW, MEDIUM and HIGH, where HIGH uses the most CPU.     trackStuckThresholdMs: 10000 # The threshold for how long a track can be stuck. A track is stuck if does not return any audio data.     useSeekGhosting: true # Seek ghosting is the effect where whilst a seek is in progress, the audio buffer is read from until empty, or until seek is ready.     youtubePlaylistLoadLimit: 10 # Number of pages at 100 each     playerUpdateInterval: 5 # How frequently to send player updates to clients, in seconds     youtubeSearchEnabled: true     soundcloudSearchEnabled: true     gc-warnings: true     #ratelimit:       #ipBlocks: ["<IP_ADDRESS>/8", "..."] # list of ip blocks       #excludedIps: ["...", "..."] # ips which should be explicit excluded from usage by lavalink       #strategy: "RotateOnBan" # RotateOnBan | LoadBalance | NanoSwitch | RotatingNanoSwitch       #searchTriggersFail: true # Whether a search 429 should trigger marking the ip as failing       #retryLimit: -1 # -1 = use default lavaplayer value | 0 = infinity | >0 = retry will happen this numbers times     #youtubeConfig: # Required for avoiding all age restrictions by YouTube, some restricted videos still can be played without.       #email: "your account mail" # Email of Google account       #password: "your account password" # Password of Google account     #httpConfig: # Useful for blocking bad-actors from ip-grabbing your music node and attacking it, this way only the http proxy will be attacked       #proxyHost: "localhost" # Hostname of the proxy, (ip or domain)       #proxyPort: 3128 # Proxy port, 3128 is the default for squidProxy       #proxyUser: "" # Optional user for basic authentication fields, leave blank if you don't use basic auth       #proxyPassword: "" # Password for basic authentication  metrics:   prometheus:     enabled: false     endpoint: /metrics  sentry:   dsn: ""   environment: "" #  tags: #    some_key: some_value #    another_key: another_value  logging:   file:     path: ./logs/    level:     # Set this to DEBUG to enable more detailed logging. Please note that this will likely spam your console.     root: INFO     # Set this to DEBUG to enable more detailed logging from Lavalink.     lavalink: INFO    request:     enabled: true     includeClientInfo: true     includeHeaders: false     includeQueryString: true     includePayload: true     maxPayloadLength: 10000    logback:     rollingpolicy:       max-file-size: 1GB       max-history: 30

Simillar issue using but using dev.lavalink.youtube version 1.8.2 in jagrosh/MusicBot
Nov 15 14:58:18 Pingel.pl java[954637]: [14:58:18] [ERROR] [LocalAudioTrackExecutor]: Error in playback of 6hmPxjuDytI
Nov 15 14:58:18 Pingel.pl java[954637]: com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Something broke when playing the track.
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.tools.ExceptionTools.wrapUnfriendlyExceptions(ExceptionTools.java:45)
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:121)
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.player.DefaultAudioPlayerManager.lambda$executeTrack$2(DefaultAudioPlayerManager.java:339)
Nov 15 14:58:18 Pingel.pl java[954637]:         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Nov 15 14:58:18 Pingel.pl java[954637]:         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Nov 15 14:58:18 Pingel.pl java[954637]:         at java.base/java.lang.Thread.run(Thread.java:829)
Nov 15 14:58:18 Pingel.pl java[954637]: Caused by: java.io.IOException: Invalid status code for player api response: 400
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.tools.io.HttpClientTools.assertSuccessWithContent(HttpClientTools.java:154)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadJsonResponse(NonMusicClient.java:67)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:107)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:193)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.track.YoutubeAudioTrack.processWithClient(YoutubeAudioTrack.java:119)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.track.YoutubeAudioTrack.process(YoutubeAudioTrack.java:72)
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:109)
Nov 15 14:58:18 Pingel.pl java[954637]:         ... 4 common frames omitted
Nov 15 14:58:18 Pingel.pl java[954637]: [14:58:18] [ERROR] [AudioHandler]: Track 6hmPxjuDytI has failed to play
Nov 15 14:58:18 Pingel.pl java[954637]: com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Something broke when playing the track.
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.tools.ExceptionTools.wrapUnfriendlyExceptions(ExceptionTools.java:45)
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:121)
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.player.DefaultAudioPlayerManager.lambda$executeTrack$2(DefaultAudioPlayerManager.java:339)
Nov 15 14:58:18 Pingel.pl java[954637]:         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Nov 15 14:58:18 Pingel.pl java[954637]:         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Nov 15 14:58:18 Pingel.pl java[954637]:         at java.base/java.lang.Thread.run(Thread.java:829)
Nov 15 14:58:18 Pingel.pl java[954637]: Caused by: java.io.IOException: Invalid status code for player api response: 400
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.tools.io.HttpClientTools.assertSuccessWithContent(HttpClientTools.java:154)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadJsonResponse(NonMusicClient.java:67)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:107)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:193)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.track.YoutubeAudioTrack.processWithClient(YoutubeAudioTrack.java:119)
Nov 15 14:58:18 Pingel.pl java[954637]:         at dev.lavalink.youtube.track.YoutubeAudioTrack.process(YoutubeAudioTrack.java:72)
Nov 15 14:58:18 Pingel.pl java[954637]:         at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:109)
Nov 15 14:58:18 Pingel.pl java[954637]:         ... 4 common frames omitted

Same thing happens for me

Have the same issue

same for me

Please stop with the "me too" posts. They do not offer anything of value. If you have information that might be useful for the resolution of the problem then please post it, otherwise, refrain from adding a comment otherwise this issue will be locked.

youtube patched oauth, try using something else
https://github.com/yt-dlp/yt-dlp/pull/11558
https://github.com/yt-dlp/yt-dlp/issues/11462

Updated to 1.9.0 - still having the same issue as the one who originally posted this

Some clients like ANDROID_TESTSUITE cause Invalid status code for player api response: 400 so try using a different client to perform playback.

Still having the same issue, correct me if I am wrong, is there a chance youtube banned the account?
YoutubeAudioSourceManager youtubeAudioSourceManager = new YoutubeAudioSourceManager(/*allowSearch:*/ true, new Client[] { new Music(), new Web() });
Error message if its useful
lava-daemon-pool-playback-1-thread-1] ERROR com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor - Error in playback of 60ItHLz5WEA com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Something broke when playing the track. 	at com.sedmelluq.discord.lavaplayer.tools.ExceptionTools.wrapUnfriendlyExceptions(ExceptionTools.java:45) 	at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:121) 	at com.sedmelluq.discord.lavaplayer.player.DefaultAudioPlayerManager.lambda$executeTrack$2(DefaultAudioPlayerManager.java:339) 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) 	at java.base/java.lang.Thread.run(Thread.java:840) Caused by: java.io.IOException: Invalid status code for player api response: 400 	at com.sedmelluq.discord.lavaplayer.tools.io.HttpClientTools.assertSuccessWithContent(HttpClientTools.java:154) 	at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadJsonResponse(NonMusicClient.java:67) 	at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:107) 	at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35) 	at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:193) 	at dev.lavalink.youtube.track.YoutubeAudioTrack.processWithClient(YoutubeAudioTrack.java:119) 	at dev.lavalink.youtube.track.YoutubeAudioTrack.process(YoutubeAudioTrack.java:72) 	at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:109) 	... 4 more

Still encountering the same issue:
What I've done:

Updated youtube plugin to 1.9.0
tested multiple clients, and all produced similar error message [MUSIC, WEB, WEBEMBEDDED, ANDROID_TESTSUITE, ANDROID_VR, TVHTML5EMBEDDED]
created a new google account for auth

However, these all produced the Invalid status code for player api response: 400 error
Full:
2024-11-30 17:38:53.195 ERROR 2836573 --- [back-1-thread-1] c.s.d.l.t.p.LocalAudioTrackExecutor      : Error in playback of yhS9LnDoo_w

com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Something broke when playing the track.
        at com.sedmelluq.discord.lavaplayer.tools.ExceptionTools.wrapUnfriendlyExceptions(ExceptionTools.java:45) ~[lavaplayer-1.5.4.jar!/:na]
        at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:113) ~[lavaplayer-1.5.4.jar!/:na]
        at com.sedmelluq.discord.lavaplayer.player.DefaultAudioPlayerManager.lambda$executeTrack$1(DefaultAudioPlayerManager.java:324) ~[lavaplayer-1.5.4.jar!/:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
        Suppressed: com.sedmelluq.discord.lavaplayer.tools.exception.EnvironmentInformation:
  lavaplayer.version: 1.5.4
  os.arch: amd64
  os.name: Linux
  os.version: 6.1.0-25-amd64
  java.vendor: Azul Systems, Inc.
  java.version: 17.0.12
  java.runtime.version: 17.0.12+7-LTS
  java.vm.version: 17.0.12+7-LTS
Caused by: java.io.IOException: Invalid status code for player api response: 400
        at com.sedmelluq.discord.lavaplayer.tools.io.HttpClientTools.assertSuccessWithContent(HttpClientTools.java:154) ~[lavaplayer-1.5.4.jar!/:na]
        at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadJsonResponse(NonMusicClient.java:67) ~[youtube-plugin-1.9.0.jar!/:na]
        at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:107) ~[youtube-plugin-1.9.0.jar!/:na]
        at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35) ~[youtube-plugin-1.9.0.jar!/:na]
        at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:193) ~[youtube-plugin-1.9.0.jar!/:na]
        at dev.lavalink.youtube.track.YoutubeAudioTrack.processWithClient(YoutubeAudioTrack.java:119) ~[youtube-plugin-1.9.0.jar!/:na]
        at dev.lavalink.youtube.track.YoutubeAudioTrack.process(YoutubeAudioTrack.java:72) ~[youtube-plugin-1.9.0.jar!/:na]
        at com.sedmelluq.discord.lavaplayer.track.playback.LocalAudioTrackExecutor.execute(LocalAudioTrackExecutor.java:101) ~[lavaplayer-1.5.4.jar!/:na]
        ... 4 common frames omitted

These clients + clientOptions work for me:
Clients:

MUSIC
ANDROID_MUSIC
WEB
WEBEMBEDDED
TVHTML5EMBEDDED

ClientOptions:

ANDROID_MUSIC:
playback: false
playlistLoading: false
searching: false
videoLoading: true
MUSIC:
playback: false
playlistLoading: false
searching: true
videoLoading: false
TVHTML5EMBEDDED:
playback: true
playlistLoading: false
searching: false
videoLoading: true
WEB:
playback: false
playlistLoading: true
searching: true
videoLoading: false
WEBEMBEDDED:
playback: false
playlistLoading: false
searching: false
videoLoading: false

Thanks to @ shreyasinghrathore (discrord) for sharing this publicy on the Lavalink discord.

@Vexify4103 hi, could u share your application.yml? I still got the same error after setting the client.
plugins:
  youtube:
    enabled: true
    allowSearch: true
    allowDirectVideoIds: true
    allowDirectPlaylistIds: true
    clients:
      - ANDROID_MUSIC
      - MUSIC
      - TVHTML5EMBEDDED
      - WEB
      - WEBEMBEDDED
    oauth:
      enabled: true
      skipInitialization: true
      refreshToken: "my_token"
    ANDROID_MUSIC:
      playback: false
      playlistLoading: false
      searching: false
      videoLoading: true
    MUSIC:
      playback: false
      playlistLoading: false
      searching: true
      videoLoading: false
    TVHTML5EMBEDDED:
      playback: true
      playlistLoading: false
      searching: false
      videoLoading: true
    WEB:
      playback: false
      playlistLoading: true
      searching: true
      videoLoading: false
    WEBEMBEDDED:
      playback: false
      playlistLoading: false
      searching: false
      videoLoading: false

Can confirm @Vexify4103 's clientoption works!
Here's my config for others to just copy-paste lol
plugins:
  youtube:
    enabled: true
    allowSearch: true
    allowDirectVideoIds: true
    allowDirectPlaylistIds: true
    clients:
      - ANDROID_MUSIC
      - MUSIC
      - TVHTML5EMBEDDED
      - WEB
      - WEBEMBEDDED
    oauth:
      enabled: true
      skipInitialization: true
      refreshToken: "your_token"
    clientOptions:
      ANDROID_MUSIC:
        playback: false
        playlistLoading: false
        searching: false
        videoLoading: true
      MUSIC:
        playback: false
        playlistLoading: false
        searching: true
        videoLoading: false
      TVHTML5EMBEDDED:
        playback: true
        playlistLoading: false
        searching: false
        videoLoading: true
      WEB:
        playback: false
        playlistLoading: true
        searching: true
        videoLoading: false
      WEBEMBEDDED:
        playback: false
        playlistLoading: false
        searching: false
        videoLoading: false

have you tried POT?

have you tried POT?

tried, didn't work, just like @RainzDev said.

Should be fixed in 8742c341b0e51d222dc9c2a25a85e24bef189b20


I can't change Current Theme

I add new theme resources/views/layouts
From admin panel settings tab i don't see new theme.

How did you add a new theme? Is there a theme.json file in it and the name is different?

I just copy the default them folder and rename the folder.

I've changed the name in theme.json but still the same.


Need match score(similarity percentage) for classified images

It would be nice to have a classification method which also provides similarity score for compared images.

You can use func SquaredEuclideanDistance(descriptor1, descriptor2) in face.go


setup.sh exits after VOSK install

the setup.sh script dated 13 FEB exits after the VOSK setup. I am unable to complete the install due to missing files as no config file is created and setup.sh is unable to run option 6. As result setup.sh scp  < /path/to/key> also fails due to the missing config file.
this worked in the previous version

cannot onboard OSKR Vector as "sudo ./setup.sh scp vectorip /path/to/key" fails with error "server_config.json file missing. You need to generate this file with ./setup.sh's 6th option" If I then run setup.sh again it exits after setting up VOSK with message "wire-pod has been set up successfully!".

Pull the latest commit and follow the new wiki instructions, OSKR/dev-bot setup is now done via the web interface

I did.
If you have an OSKR or dev bot, you can set them up with wire-pod via the web configuration interface. In the "Bot Setup" section, look for "Set up OSKR/dev bot", enter the bot's IP address, upload the key, then press set up
I assume that via the web configuration interface you mean the web server in wire-pod. When I select "Bot Setup”. I get instructions to pair with the robot. This never works. I have tried with Safari and with Chrome. Running MacOS, wire-pod installed on Raspberry Pi - only thing running there. Looking at the code it appears that you call some javascript for the Bot Setup.
This javascript does not work - I get a button that appears that says “Disconnect”. Sometimes I get a button labeled with the robot name but clicking on the button does nothing. https://keriganc.com/vector-epod-setup/html/main.html works just fine for pairing.
Should I post this to git as an issue?

On Feb 14, 2023, at 18:35 , Wire @.***> wrote:
Pull the latest commit and follow the new wiki instructions, OSKR/dev-bot setup is now done via the web interface
—
Reply to this email directly, view it on GitHub https://github.com/kercre123/wire-pod/issues/88#issuecomment-1430668072, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABID5PBIXFOXFHHFGCSGLYLWXQ6GTANCNFSM6AAAAAAU2YV5HU.
You are receiving this because you authored the thread.

Are you sure you are on the latest commit? I pushed some code yesterday. There should be a section under the bot authentication which is specifically for OSKR/dev-bots (should be two sections overall in Bot Setup).  Also make sure to do CTRL+SHIFT+R (or CMD on macOS) on the configuration page in order to redownload all resources.

I will start all over again. I did the pull yesterday so it probably matters what time.

On Feb 15, 2023, at 19:18 , Wire @.***> wrote:
Are you sure you are on the latest commit? I pushed some code yesterday. There should be a section under the bot authentication which is specifically for OSKR/dev-bots (should be two sections overall in Bot Setup). Also make sure to do CTRL+SHIFT+R (or CMD on macOS) on the configuration page in order to redownload all resources.
—
Reply to this email directly, view it on GitHub https://github.com/kercre123/wire-pod/issues/88#issuecomment-1432437371, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABID5PD4ONH2YAGBOSUET2DWXWL6TANCNFSM6AAAAAAU2YV5HU.
You are receiving this because you authored the thread.

Closing due to inactivity.


Waymo Dataset converter.py

First of all, thank you for the great work. I used the converter as instructed to convert the Waymo dataset. Afterward, I counted the number of images and calibrations in the Waymo validation_org set and found that there were 39,987 of each, but only 39,047 labels. When I applied the setup_split, I discovered that some data had not been converted due to the lack of labels. As a result, I only have 51,257 training samples and 38,960 validation samples, while your paper states that there are 52,386 training and 39,848 validation samples. How can I obtain the same number of samples as described in the paper (52,386, 39,848)?

Hi @mingyuShin
Thank you for your interest in DEVIANT. I quickly checked my waymo validation_org and validation folders. My validation contains 39,848 images we report in the paper. Here is a screenshot of the same:

Your number of validation_org labels is wrong, which results in lower images for the validation set after setup_split.py. Could you please let me know which images of this ImageSet are absent during conversion?
I could paste those image labels here to confirm if they are empty images.

empty_val.txt
empty_trian.txt
Thank you for your help. I don't have these images's label...

Not all of the training are empty files and therefore, I upload the training and validation labels of the waymo split at this  drive link.
Unzip the above file and place them as follows:
./code
├── data
│      └── waymo
│             ├── ImageSets
│             ├── training
│             │     ├── calib
│             │     └── label
│             │
│             └── validation
│                   ├── calib
│                   └── label

Then consider copying the corresponding images in the image sub-folder of training and validation folders to complete the folder structure:
./code
├── data
│      └── waymo
│             ├── ImageSets
│             ├── training
│             │     ├── calib
│             │     ├── image
│             │     └── label
│             │
│             └── validation
│                   ├── calib
│                   ├── image
│                   └── label

Let me know if that helps

You are a really kind person. Thank you so much for the helpful explanation. The issue has been resolved.


For release 0.3

Codecov Report
Base: 48.31% // Head: 97.95% // Increases project coverage by +49.64% :tada:

Coverage data is based on head (a2358ff) compared to base (b7acf87).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #175       +/-   ##
===========================================
+ Coverage   48.31%   97.95%   +49.64%
===========================================
  Files           6        4        -2
  Lines         592      293      -299
===========================================
+ Hits          286      287        +1
+ Misses        306        6      -300

Impacted Files
Coverage Δ

src/tmle_inputs/from_param_files.jl
98.21% <100.00%> (ø)

src/tmle_inputs/tmle_inputs.jl
96.96% <0.00%> (+1.01%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Can't use atsign-micropython custom built releases in old Linux versions due to a package version mismatch

It is possible to manually compile the coverage variant of micropython in older Linux versions (such as Debian 10). It will execute and allow the user to run files. However, it wont do it properly:

astt@as2019:~/Escritorio/at_pico_w/src$ micropython
MicroPython v1.20.0-161-gdb5444f68 on 2023-06-06; linux [GCC 8.3.0] version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>>
astt@as2019:~/Escritorio/at_pico_w/src$ micropython main.py
linux
Traceback (most recent call last):
  File "main.py", line 9, in <module>
  File "./lib/atclient.py", line 8, in <module>
  File "./lib/third_party/rsa/__init__.py", line 27, in <module>
  File "./lib/third_party/rsa/key.py", line 572, in <module>
AttributeError: 'module' object has no attribute 'rsa'
astt@as2019:~/Escritorio/at_pico_w/src$

This probably happens due to a mismatch in the libffi library (atsign-micropython is compiled using libffi7, while some older Linux versions will compile it with libffi6).
This is what happens when I try to run atsign-micropython in Debian 10:
astt@as2019:~/Escritorio/at_pico_w/src$ ~/Escritorio/test-upy/micropython_1.19.1_63a59ee4e/micropython main.py
/home/astt/Escritorio/test-upy/micropython_1.19.1_63a59ee4e/micropython: error while loading shared libraries: libffi.so.7: cannot open shared object file: No such file or directory

And here we can see the libffi version:
astt@as2019:~/Escritorio/at_pico_w/src$ apt-cache search libffi
libffindex0 - library for simple index/database for huge amounts of small files
libffindex0-dev - library for simple index/database for huge amounts of small files (development)
libjffi-java - Java Foreign Function Interface
libjffi-jni - Java Foreign Function Interface (JNI library)
libalt-alien-ffi-system-perl - simplified alternative to Alien::FFI that uses system libffi
libffi-dev - Foreign Function Interface library (development files)
libffi6 - Foreign Function Interface library runtime
libffi6-dbg - Foreign Function Interface library runtime (debug symbols)
libffi-checklib-perl - module to check availability of a library for FFI
libffi-platypus-perl - module to create Perl bindings to non-Perl libraries with FFI

Same code works fine when executed using atsign-micropython in a more recent Linux (Ubuntu 20.04.3 LTS):

realvarx@rs-2022:~/Desktop/at_pico_w/src$ micropython main.py
Welcome! What would you like to do?
	1) Set recipient atSign (presently cpswan)
	2) Connect to arrogantcheetah
	3) Send at atTalk message to cpswan
	4) Exit
>

Fixed by https://github.com/atsign-foundation/at_pico_w/pull/18/commits/100f77649280362b94369ed2dcf2bc8bfc28eba5


Attribute values > to_phlex_attribute_value, to_str, to_s

Fixes #569
Attribute values can be any object that responds to to_phlex_attribute_value or to_str, falling back to to_s if neither is defined.

This looks good to me. I’m happy with to_str coming first.

Oh, good point re tests. I’ll open an issue for that since I merged this already.


Add delegation tests

This PR resolves #367 by adding tests for transferring and burning tokens when delegated to another account.

Side note: that random test failure on token-swap was pretty interesting! #3334

Looks like this just needs a cargo fmt to pass CI. I'll investigate and fix the token-swap error, sorry about that.

Whoops yes I've now adjusted my IDE settings so this shouldn't happen again 😅

Closing / opening to redo all CI


'Flutter/Flutter.h' file not found   in FLUTTER IOS moudle mode .

flutter doctor

:
pod versio

n
podfil

e
error:

i trid
flutter clean  =>flutter build ios => pod install =>build (xcode ) more times ,but not working nomal !!!!
but it is nomal at flutter 1.22.6 ,

Same here

@369235109
First of all, please refrain from providing details in screenshots as it becomes difficult to read or use any contents from it.
Please provide the details in properly formatted text along with ios/podfile.

@369235109
First of all, please refrain from providing details in screenshots as it becomes difficult to read or use any contents from it.
Please provide the details in properly formatted text along with ios/podfile and [flutter run -.md](https://github.com/flutter/flutter/files/7207100/flutter.run.-.md). The verbose log if huge, put in a file and share it here.

podflie:
Uncomment the next line to define a global platform for your project
platform :ios, '11.0'
target 'jinpan' do
Comment the next line if you don't want to use dynamic frameworks
use_frameworks!
flutter_application_path = '../kinpan_app/'
load File.join(flutter_application_path, '.ios', 'Flutter', 'podhelper.rb')
install_all_flutter_pods(flutter_application_path)
Pods for jinpan
#pod 'UMCCommon'
#pod 'UMCCommonLog'
pod 'BaiduMobStatCodeless'
pod 'XHLaunchAd'
pod 'XinstallSDK'
pod 'LEGOAppleSignInSDK'
end
#post_install do |installer|
installer.pods_project.targets.each do |target|
target.build_configurations.each do |config|
config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = '9.0'
end
end
#end

files of flutter run -version
flutter run -.md

@369235109
The log you shared doesn't contain the actual error flutter.h found, but shows [  +65 ms] No supported devices connected. because you might have run without connecting any of the devices.
Please provide us the log which contains flutter.h not found entries in it.

hello darshankawar
actually，it is normal at android ,and i used the follow code at XCODE not androidstudio.
and i found it is normal when i connect at Real machine ,just error on virtual machine

so ,it is hard to share the error log to you

@369235109
You need to have flutter_additional_ios_build_settings in your podfile and it should look like below:
https://github.com/flutter/flutter/blob/e605b7c2250e803f46017b1d544fc8c297007546/packages/flutter_tools/templates/cocoapods/Podfile-ios-swift
Make above changes in your podfile and see if it works for you.


Fallback Creds

When doing the Python Quickstart for Google Sheets, on Windows 10, it gives an error:
[WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions
It's caused by creds = flow.run_local_server()
This PR adds a fallback to it shows a different auth method, as per the docs: "_ If this fails, copy the URL from the console and manually open it in your browser._"

#86 was just fixed by using an ephemeral port instead of the default 8080. Try pulling latest version of code and see if that fixes the issue instead.

We just published a new change that fixes a similar issue (#86) that we think will work better in the long run. Thanks for the contribution though.

Looks like our replies crossed, oops! Keeping this open until you confirm Steve's fix works.


Dev into master

GitHub Issue
None
Changes
Merge dev into master
Packages Added
Brief list of any packages added, and if folks need to rerun yarn install to run locally
Checks
Before making your PR, please check the following:

[ ] Critical lint errors are resolved
[ ] App runs locally
[ ] App builds locally (run the build command for any impacted package and check for any errors before the PR)

Thanks for doing this -- do you think we're at a point where we want to get into a cadence of doing this at the end of every 2nd week? Before Retro maybe? @alexkeating @skuhlmann

Thanks for doing this -- do you think we're at a point where we want to get into a cadence of doing this at the end of every 2nd week? Before Retro maybe? @alexkeating @skuhlmann

Sounds good to me. Once used to it we can figure out of we need a quicker cadence or whatever.

Yes, the main reason I made this was to deploy the summon and hub apps


lower required Python version?

pyproject.toml mentions support for Python 3.7+ (via classifiers) but also has python = "^3.10" which translates into Requires-Python: >=3.10,<4.0 in the distributed wheel.
Unfortunately, there are some environments where I can not use Python 3.10+. Would you accept PRs to lower the required version of Python, maybe even down to Python 3.6?

The lowest version of python acceptable would be python3.8, since both 3.6 and 3.7 have reached EOL. 3.8 is due for EOL in a couple of months as well.

The lowest version of python acceptable would be python3.8, since both 3.6 and 3.7 have reached EOL. 3.8 is due for EOL in a couple of months as well.

Hmm, from my point of view that definition is too strict. For example some Linux distributions still support Python 2.7 (until mid 2024) and Python 3.6 (until 2028 or so) with security updates. Also there is pypy. Just looking at CPython's support cycles does not really fit the complexities of Python deployments. IMHO the python_requires should be of technical necessity (language syntax, stdlib API).
But of course I can accept a different view, just saying :-)

I can be very strict about EOL support. Regardless of other python interpreter implementations, CPython is the default, base implementation, and sticking to old, unsupported versions for a few users is not in my plans.
Lastly, poetry, this package's dependancy manager requires python >= 3.8 to function, which is yet another reason to not drop the requirements lower.
Out of curiosity, which distros still support python 3.6 ? None of the very popular distros have their latest LTS still supporting that old a version

Out of curiosity, which distros still support python 3.6 ? None of the very popular distros have their latest LTS still supporting that old a version

That is correct if you just look for the "latest LTS". However I'm still seeing new versions being deployed to RHEL 7 and 8. Actually I am looking into requests_har to add some diagnostics capabilities to a Python application running on RHEL 7.
The poetry requirement is of course a good argument for requiring Python 3.8 (besides me having a strong dislike for poetry). I guess the only thing then is to remove the old versions from the classifiers.
I guess we'll just fork requests_har internally and deploy that version (without poetry). That approach just requires some shepherding - organizational laziness means fixes/improvements might be added just to the forked version which never makes it upstream then.

python version has been lowered to 3.8, as discussed I'm not willing to go lower (made it a bit hellish to type properly as well).
I'm closing the issue, the package (1.1.0) has been published on pypi. Feel free to fork and remove typing if you need to


Release 0.6.0

Ok I tagged the release! 🎉  @gbanjac @goulart-paul please approve the review to merge it to master


GraphQL with Context

Updated GraphQLOptions with abstract class GraphQLContext.
This abstract class have Context from http request or websocket context.

I don't really know what this does @7agustibm, but it looks fine. Is it ready for merge?

This improvement is to add authenticated. For example, add JWT token in headers and run auth handler before run graphql. Yes, It's ready for merge.

Thanks!

Out as part of 3.9.1 now! https://javalin.io/news/2020/06/21/javalin-3.9.1-released.html


Module not found Ether

Sorry for bothering you but I am getting this error
Error: Cannot find module 'ethers'
Require stack:

/Users/mehmetuyar/Documents/testnet/testnet-deposit-main/deposit.js
at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
at Module._load (node:internal/modules/cjs/loader:985:27)
at Module.require (node:internal/modules/cjs/loader:1235:19)
at require (node:internal/modules/helpers:176:18)
at Object. (/Users/mehmetuyar/Documents/testnet/testnet-deposit-main/deposit.js:11:16)
at Module._compile (node:internal/modules/cjs/loader:1376:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
at Module.load (node:internal/modules/cjs/loader:1207:32)
at Module._load (node:internal/modules/cjs/loader:1023:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'/Users/mehmetuyar/Documents/testnet/testnet-deposit-main/deposit.js'
]
}

Install ethers package with this command: npm install ethers.

Thank you @kargaa for raising this, I also faced the same error. And thank you @hkey0 for the solution, it works, but now I face another error, which is JsonRpcProvider failed to detect network and cannot start up; retry in 1s (perhaps the URL is wrong or the node is not started)


Release new version of VS Code extension

:tada: This PR is included in version 0.103.0 :tada:
The release is available on:

v0.103.0
GitHub release

Your semantic-release bot :package::rocket:


Tus for project dataset

Motivation and context

Continue TUS integration to provide chunk upload for large dataset files in projects
How has this been tested?

Checklist

[x] I submit my changes into the develop branch
[x] I have added a description of my changes into CHANGELOG file
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[x] I have linked related issues (read github docs)
[x] I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

[x] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
[x] I have updated the license header for each file (see an example below)

# Copyright (C) 2022 Intel Corporation
#
# SPDX-License-Identifier: MIT

Generally patch works for me except of 2 moments:

When I first time tried to import dataset, I get error with CSRF token (403), see gif above. I re-loginned and was not able to reproduce it anymore. So, maybe it was not related with this patch. Let's see if somebody meet the same issue.
When error occurs, it does not show any errors, probably related with the error I commented in the code (generatedError is not raised)


[PyTorch] Fix MultiheadAttention docstring

Fixes the docstring for the MultiheadAttention.__init__. It currently states that the default value for input_layernorm is True, but it should be False.

@timmoon10 Signed!