Tap to dismiss doesn't cancel previously scheduled auto hide DispatchWorkItem

I've implemented a queue of toasts, using PopupView for presentation. The issue I'm running into is that when a toast is dismissed via tap and I present the next toast in queue, it gets auto-hidden early. It looks like this is the code with the bug.
if let autohideIn = autohideIn {
    dispatchWorkHolder.work?.cancel()
    dispatchWorkHolder.work = DispatchWorkItem(block: {
        self.isPresented = false
    })
    if isPresented, let work = dispatchWorkHolder.work {
        DispatchQueue.main.asyncAfter(deadline: .now() + autohideIn, execute: work)
    }
}

It seems that since dispatchWorkHolder is a class, it's work property is being passed by reference. Therefore, when I present a second toast, the work property is set to a new DispatchWorkItem that has not been cancelled. So at the time the previously scheduled asyncAfter executes, it is not a cancelled item.

Hey @JLZ , thank you for your feedback. Could you please try modifying PopupView's code like this:
if self.closeOnTap {         self.dispatchWorkHolder.work?.cancel()         self.isPresented = false     }
If that doesn't fix your issue, please attach some code for me to reproduce your issue. If it does, I'll make new pod release with this change. Have a nice day!

@f3dm76 good call. This fixed the issue. I posted a PR with the fix here.
Thanks!

@JLZ All done, version 0.0.8, you are most welcome to check it out, have a nice day!


Fork, Commit, Merge - Easy Issue (Mojo)

Fork, Commit, Merge - Easy Issue (Mojo)
Create a Simple "Hello, Mojo!" Web Application
Note: You don't have ask permission to start solving the issue or get assigned, since these issues are supposed to be always open for new contributors. The actions-user bot will reset the file back to previous state for the next contributor after your commit is merged. So you can just simply start working with the issue right away!
Objective:
Create a basic Mojo web application that returns "Hello, Mojo!" when accessed at the root path (/). This application should be a single-file app.
How to get started:
Open up the hello_mojo.pl file inside of tasks/mojo/easy directory and start implementing your solution!
To run the script:
Make sure the script is executable:
chmod +x hello_mojo.pl

Run the application:
./hello_mojo.pl daemon

You can now access the web application at http://localhost:3000. It should display "Hello, Mojo!" in your web browser.

To work with this issue, you need to have Mojo installed to your local machine.
Check out README.md for more instructions of installing Mojo and how to make a pull request.
Feel free to ask any questions here if you have some problems!
Also, kindly give this project a star to enhance its visibility for new developers!

Can you assign it for me?

Can you assign it for me?

Hi!
As it says on the description:
Note: You don't have ask permission to start solving the issue or get assigned, since these issues are supposed to be always open for new contributors. The actions-user bot will reset the file back to previous state for the next contributor after your commit is merged. So you can just simply start working with the issue right away!


Ignore small regressions for JavaIDEModelPerformanceTest

The test is very flaky, no matter how much we rebaseline it. Let's ignore any regressions below 5% for it.
This PR also ignores regressions below 10ms median difference and always detects regressions above 20% median difference, no matter the confidence.

@bot-gradle test and merge

OK, I've already triggered a build for you.


Feature/119 3rd party lib credits

About and Contribution screen updates.
Fixes #119

Current coverage is 0.00% (diff: 100%)

Merging #122 into develop will not change coverage

@@           develop   #122   diff @@
=====================================
  Files            1      1
  Lines           34     34
  Methods          0      0
  Messages         0      0
  Branches         0      0
=====================================
  Hits             0      0
  Misses          34     34
  Partials         0      0

Powered by Codecov. Last update f6b8a1d...5457904


Better package repr

@ResidentMario do we want to include any other data? Right now it's just node names

If the max length is 20 lines, and a package has 25 top-level directories, is it correct to show no grandchildren, 19 child directories and the ..., or should we show some grandchildren? What is the correct behavior to cut the line count down?

In that case I think that 19 child directories and the ... would be best. If the user wants to look at individual files in some directory, they can key into the package directory they're interested in, get the subpackage repr, and see those files then. But if we don't show them the directory they have access to right in the repr, they won't know where to go to get those files, so their ability to proceed would be hampered (they would have to print out pkg.keys() and look through that). So I think showing as many top-level folders as possible is higher priority than showing many files.

@ResidentMario Try out this new version. Also, should we make max_lines higher than 20?

20 is about how many will fit in a terminal window (at least on my machine) so it seems like a good starting point. The output of this implementation looks fine. One problem spot: printing an empty Package results in an empty string. For now that's fine, but we might want to tweak that later.


ticdc(test): add more ut for kafka sink v2

What problem does this PR solve?
Issue Number: ref #8241
What is changed and how it works?
Add more ut for kafka sink v2, increase the coverage rate of factory.go from 20% to 37.1%
Check List
Tests

[x] Unit test

Release note
`None`

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/merge

/run-all-tests

/test kafka-integration-test

/run-kafka-integration-test

/run-all-tests

/run-integration-test

/merge

/run-engine-integration-test

/merge

/run-engine-integration-test


sdkman-init.sh: various optimizations and cleanup

In my tests, this reduces execution time by about 50% with bash,
and 40% with zsh. Most of the speedup is from removing calls to
external commands like basename and cat.
To raise a new Pull Request, the following prerequisites need to be met. Please tick before proceeding:

[x] a conversation was held in the appropriate Gitter Room.
[x] a Github Issue was opened for this feature / bug.
[ ] test coverage was added (Cucumber or Spock as appropriate).

This addresses Issue #624.

@jjlin Thanks for taking the time and contributing! I'll be releasing your change to the stable environment within a day or two. Thanks again!

Thanks!


Welcome Screen shows  only one time

Hi
at first i thank you for your good library...
i use it with the default configuration you explain at introduction page of Github...
but when i install my app in mobile i see welcome screen one time, and on another use this page is invisible...
i want to see it every time my application is started not only in first use...
best thanks...

That is the behaviour when using .show(), as usually onboarding/intro screens are only shown on the first app launch. You can use .forceShow() to always show the welcome screen.
However, be careful, I just realized there's no protection against starting overlapping welcome screens like with .show() when the activity is recreated (try rotating the device). You may need to store something in the savedInstanceState and check that to make sure it's only started once. You might also be able to start it from an Application subclass instead, I can't check that at the moment though.


Selected event for CheckBoxGroup

I have set of four CheckBoxes inside CheckBoxGroup to mimic the behavior of RadioButtonGroup.
There are no bindable properties inside CheckBox class which I can use with MvvmCross (or maybe I just didn't find them?). Thus I wanted to subscribe on DidTapCheckBox or AnimationDidStopForCheckBox events and Execute a command from ViewModel on the events.
Events are invoked perfectly without CheckBoxGroup. But if I add them to CheckBoxGroup then no handlers are called at all, nor DidTapCheckBox nor AnimationDidStopForCheckBox. It seems like CheckBoxGroup consumes a user's tap, changes its own CheckBoxGroup.SelectedCheckBox and then suppress the tap propagation, so no DidTapCheckBox event is fired on any single CheckBox.
Actually, it's reasonable behavior for a group of controls but the problem is that CheckBoxGroup does not expose external event I can subscribe on to know when CheckBoxGroup.SelectedCheckBox was changed. Or maybe I should use another mechanism for it?
var dev  = new BEMCheckBox();
var test = new BEMCheckBox();
var demo = new BEMCheckBox();
var live = new BEMCheckBox();

dev .DidTapCheckBox += (_,__) => ViewModel.Dev .ExecuteAsync();
test.DidTapCheckBox += (_,__) => ViewModel.Test.ExecuteAsync();
demo.DidTapCheckBox += (_,__) => ViewModel.Demo.ExecuteAsync();
live.DidTapCheckBox += (_,__) => ViewModel.Live.ExecuteAsync();

var boxGroup = new BEMCheckBoxGroup()
{
    //SelectedCheckBox = dev,
    //MustHaveSelection = true
};
boxGroup.AddCheckBoxToGroup(dev);
boxGroup.AddCheckBoxToGroup(test);
boxGroup.AddCheckBoxToGroup(demo);
boxGroup.AddCheckBoxToGroup(live);

Maybe the problem is in line:
View.AddSubviews(dev,test,demo,live);

However, I can't write:
View.AddSubview(boxGroup);

Because
Error	CS1503	Argument 1: cannot convert from 'SaturdayMP.XPlugins.iOS.BEMCheckBoxGroup'
to 'UIKit.UIView'

Sorry, but it's working now. Probably the reason of wrong behavior was me.

@Brains Thank you for submitting the issue and also solving it.  Do you think you can provide more details on how you solved it in-case someone else has the same problem you did?
Happy holidays and all the best in 2019.

I really don't know.
Today I just checked out the same commit and it worked, no changes were performed with code.
Maybe restart helped (Windows, Visual Studio, Mac Virtual Machine), it's the only thing I can suppose of.

Happy holidays for you too.


Fix wgs_cnv_calling default config paths

Issue
Inaccessible path defined in DEFAULT_CONFIG from workflow wgs_cnv_calling:
DEFAULT_CONFIG = r"""
# Default configuration wgs_cnv_calling
step_config:
  wgs_cnv_calling:
    ...
    sv2:
      ...
      path_sv2_resource: /fast/work/users/holtgrem_c/cubit_incoming/SV2/v<IP_ADDRESS>   # REQUIRED
      path_sv2_models: /fast/work/users/holtgrem_c/cubit_incoming/SV2/v<IP_ADDRESS>/training_sets   # REQUIRED
"""

Fix according to changes in cubit - https://github.com/bihealth/cubit/issues/246

Decision: close as we are dropping erds and consequently erds_sv2 step.


ERROR: No recipes available for: ..../meta-updater-raspberrypi/recipes-bsp/u-boot/libubootenv_%.bbappend

I'm trying to add support for ostree to out boot2qt yocto warrior build for raspberry pi4. I've added meta-updater & meta-updater-raspberrypi to the build and when I start bitbake I get the following error:
ERROR: No recipes available for:
.../sources/meta-updater-raspberrypi/recipes-bsp/u-boot/libubootenv_%.bbappend
I've tried downloading the HereOtaConnect sample project but it doesn't have the libubootenv recipe.
I researched where libubootenv is from and it is part of the sbabic swupddate system. Does this mean that I need to install meta-swupdate to get the libuboot? Does libubootenv automatically disable the fw* utilities from uboot?
From our bblayers.conf:
${BSPDIR}/sources/meta-openembedded/meta-filesystems
${BSPDIR}/sources/meta-updater
${BSPDIR}/sources/meta-updater-raspberrypi \
From our local.conf:
first, need to use u-boot to boot to Linux
RPI_USE_U_BOOT = "1"
then, need to use initramfs
KERNEL_INITRAMFS = "-initramfs"
INITRAMFS_IMAGE = "initramfs-ostree-image.bb"
INITRAMFS_IMAGE_BUNDLE = "1"
BOOT_SPACE = "1073741"
INITRAMFS_MAXSIZE = "315400"
IMAGE_FSTYPES_pn-${INITRAMFS_IMAGE} = "${INITRAMFS_FSTYPES}"
make sure to inherit OTA functionality from the meta-updater* layers
INHERIT += " sota"
DISTRO_FEATURES += " usrmerge"
Is there something else that I'm missing?
Regards,
Greg Wilson-Lindberg

It seems that my copying of code in to the message has triggered some formatting, the bold text is supposed to be comments preceded by a hashtag sign.

the bold text is supposed to be comments preceded by a hashtag sign.

Fixed by inserting the triple backticks. :)

when I start bitbake I get the following error:
ERROR: No recipes available for:
  .../sources/meta-updater-raspberrypi/recipes-bsp/u-boot/libubootenv_%.bbappend

I've tried downloading the HereOtaConnect sample project but it doesn't have the libubootenv recipe.
I researched where libubootenv is from and it is part of the sbabic swupddate system. Does this mean that I need to install meta-swupdate to get the libuboot? Does libubootenv automatically disable the fw* utilities from uboot?

I can't find a reference to libubootenv anywhere in meta-updater-raspberrypi, now or in the past. Something must be wrong with your build environment. Our layers certainly do not depend on meta-swupdate. Looking at your bblayers.conf, though, it looks like you might be using an older version of the meta-updater-raspberrypi or meta-updater layers. meta-updater-raspberrypi also explicitly depends on meta-raspberrypi, and meta-updater also explicitly depends on meta-python.
Can you share the output of repo manifest -r? Or if you aren't using repo, I just want to know what revisions of our layers you are using.

@patrickvacek actually we have introduced the libubootenv bbappend after a recent change on poky master: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=cfdaad287b8bfe6909acc0257d120c58abe8ae5e.
@GWLindberg since you said you're using warrior layers, I think the mistake was to use the master branch of meta-updater-raspberrypi (maybe meta-updater as well?) and the fix is to switch to the corresponding master branches in our repos.

I did a git clone from the [github.com/advancedtelematic/meta-updater.git] and [github.com/advancedtelematic/meta-updater-raspberrypi.git] repo’s to download the directories that I’m using. And my bblayers.conf has meta-raspberrypi and meta-openembedded/meta-python in it.
Is there a different repo I should be using or a different branch in the repo that I have? Or is there a different way to get the libubootenv tools?
Regards,
Greg
From: lbonn<EMAIL_ADDRESS>Sent: Tuesday, February 25, 2020 2:22 AM
To: advancedtelematic/meta-updater-raspberrypi<EMAIL_ADDRESS>Cc: Greg Wilson-Lindberg<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [advancedtelematic/meta-updater-raspberrypi] ERROR: No recipes available for: ..../meta-updater-raspberrypi/recipes-bsp/u-boot/libubootenv_%.bbappend (#79)
@patrickvacekhttps://github.com/patrickvacek actually we have introduced the libubootenv bbappend after a recent change on poky master: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=cfdaad287b8bfe6909acc0257d120c58abe8ae5e.
@GWLindberghttps://github.com/GWLindberg since you said you're using warrior layers, I think the mistake was to use the master branch of meta-updater-raspberrypi (maybe meta-updater as well?) and the fix is to switch to the corresponding master branches in our repos.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/advancedtelematic/meta-updater-raspberrypi/issues/79?email_source=notifications&email_token=AE7XRT34AHW2GOMLMNE3433RETWKZA5CNFSM4K2UIKZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM3MWJI#issuecomment-590793509, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE7XRT6N3XKNQVZRGOZIFLLRETWKZANCNFSM4K2UIKZA.

I did a git clone from the [github.com/advancedtelematic/meta-updater.git] and [github.com/advancedtelematic/meta-updater-raspberrypi.git] repo’s to download the directories that I’m using. And my bblayers.conf has meta-raspberrypi and meta-openembedded/meta-python in it. Is there a different repo I should be using or a different branch in the repo that I have? Or is there a different way to get the libubootenv tools?

The problem appears to simply be that you are using mismatched branches of your layers. It sounds like you may be trying to use the master branch of our repos. You should git checkout warrior on both of them, and you should probably verify that the other layers you are using are indeed also on compatible branches based on warrior.


Resource / Task names

the framework doesn't like names like: 'backend'. start and end cause runtime issues.

this is due to the direct passing of task/resource names as variable names to the underlying solver package pulp, which doesnt like certain variable names. There should be some alias replacement, will be done.


Errors when activating: include missing

Hello,
I'm getting errors when activating the objects:

At first I thought it was a question of activation order because before I had it and solved it by manually activating the objects in the right order. But now it seems it is missing one include:

I am importing from a ZIP into which I exported the whole package including this missing include.
Please advise.
Regards,
Nuno

I guess this is a function group? The function groups have a few problems, and the issue for it is not yet closed, https://github.com/larshp/abapGit/issues/15
Is the missing include part of the zip file?
Can you try creating a dummy function group to see if the issue can be reproduced? And provide steps to reproduce.

abapGit uses function module RS_GET_ALL_INCLUDES to find all relevant includes for the function group, can you try executing it in your system to see if it finds the missing include?
https://github.com/larshp/abapGit/blob/master/src/zabapgit_object.prog.abap#L1782

is the function group generated via the table maintenance generator?

Yes it was. The function group was not created manually. It was generated automatically through SE11 when generating the maintenance views for the 1st of the 3 tables it currently supports. No other custom function modules there. Just generated code.
Do you think manually creating the function group and only then using it to generate the maintenance views would make a difference? I could try if you think it's worth it.

It is more the fact that the code was created by the maintenance generator.
I've reproduced the issue, take any database table and generate a table maintenance. Export/push the code. Include LZASDFT00(in this case) is not exported,

Hmm, it looks like the T00 include is a "reserved name", also it is not possible to edit the include. Could fetch all includes from database table RSEUINC, and serialize it, however I feel like it will make issues creating a locked include.
Another approach(earlier discussed with @sbcgua ): dont serialize all the abap code for function groups only containing generated table maintenance views, instead just serialize the meta data, and at installation call the SAP standard code to generate the function group.
hmm

Another approach(earlier discussed with @sbcgua ): dont serialize all the abap code for function groups only containing generated table maintenance views, instead just serialize the meta data, and at installation call the SAP standard code to generate the function group.

That seems to be an excellent idea! I just wonder how complex it may be... But since table maintenance generation function groups are so common, it would be great to find a solution for this particular issue.
My friend Nuno Morais (@WebworkApp) has worked on a dependencies check tool and has some experience with these SAP functions. Maybe he has an idea? Let's see what he says.
Thanks!
Nuno

Hello Lars,
Did you find any solution for this one? It's a quite annoying one.
To bypass this problem I currently I've been keeping all the table maintenance function groups in a temporary isolated package and have to manually generate the table maintenance views on each target system.
Thanks,
Nuno

no, have not had time to look into it :(

Sure. You already do so much! Anyway I was not pushing, just curious to know :)
Thanks!

I am not familiar with this generation of standard code on which you're proficient :) but if I find a way to do it I'll try to fix it myself.

I'm not proficient :smile: will also have to look into how the standard generation works

I could not find any APIs to generate views. The package where generation is done is SVIM.
Following the ideas above: Serialize the metadata from transaction SE55 and use CALL TRANSACTION with Batch input to manually generate/delete the "generated function groups".
Still open: How to know if a Function Group is generated or not?

API: is it possible to do a series of PERFORM IN PROGRAM?
Regenerate function group: one approach could be always regenerate the function group in case the corresponding table has been changed today. This is easy, it will be difficult to make a perfect solution without a lot of refactoring.

It may be possible, but the forms there weren't very...friendly.

now merged into master, make sure to add the T00 include, and the function group should activate without errrors
try again

closing, open new issue if the include is still causing issues


the job stop and don't move

like i stopped and can't do  nothing，the executors added and it run.
foreach at partition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:53(job 0)
collect at pythonrdd：scala:453(job1)
and then it stop and do nothing.
I want to know what's wrong and I'm appreciated if you can help me.
By the time I will look around the solved questions,mybe I can find a same situation

I will be appreciated if someone can help me...

Same issue here

Please look in your executor logs (Spark UI -> Executors -> stderr) for any errors.

Hi, this is the output I get in stderr:
log:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/11/07 15:11:46 INFO CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
17/11/07 15:11:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/07 15:11:47 INFO SecurityManager: Changing view acls to: david
17/11/07 15:11:47 INFO SecurityManager: Changing modify acls to: david
17/11/07 15:11:47 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(david); users with modify permissions: Set(david)
17/11/07 15:11:49 INFO SecurityManager: Changing view acls to: david
17/11/07 15:11:49 INFO SecurityManager: Changing modify acls to: david
17/11/07 15:11:49 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(david); users with modify permissions: Set(david)
17/11/07 15:11:50 INFO Slf4jLogger: Slf4jLogger started
17/11/07 15:11:50 INFO Remoting: Starting remoting
17/11/07 15:11:50 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutorActorSystem@<IP_ADDRESS>:41052]
17/11/07 15:11:50 INFO Utils: Successfully started service 'sparkExecutorActorSystem' on port 41052.
17/11/07 15:11:51 INFO DiskBlockManager: Created local directory at /tmp/spark-20a25310-733b-4be9-9f7d-de6cc752629b/executor-13d0a6fe-2375-443a-98ea-9911e3876119/blockmgr-e019b4fd-7710-4d6d-aba9-62a24d102f2e
17/11/07 15:11:51 INFO MemoryStore: MemoryStore started with capacity 511.1 MB
17/11/07 15:11:51 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@<IP_ADDRESS>:40023
17/11/07 15:11:51 INFO WorkerWatcher: Connecting to worker spark://Worker@<IP_ADDRESS>:42430
17/11/07 15:11:51 INFO CoarseGrainedExecutorBackend: Successfully registered with driver
17/11/07 15:11:51 INFO Executor: Starting executor ID 1 on host <IP_ADDRESS>
17/11/07 15:11:51 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 46498.
17/11/07 15:11:51 INFO NettyBlockTransferService: Server created on 46498
17/11/07 15:11:51 INFO BlockManagerMaster: Trying to register BlockManager
17/11/07 15:11:51 INFO BlockManagerMaster: Registered BlockManager
17/11/07 15:11:51 INFO CoarseGrainedExecutorBackend: Got assigned task 1
17/11/07 15:11:51 INFO Executor: Running task 1.0 in stage 0.0 (TID 1)
17/11/07 15:11:52 INFO Executor: Fetching http://<IP_ADDRESS>:46615/files/mnist_dist.py with timestamp<PHONE_NUMBER>389
17/11/07 15:11:52 INFO Utils: Fetching http://<IP_ADDRESS>:46615/files/mnist_dist.py to /tmp/spark-20a25310-733b-4be9-9f7d-de6cc752629b/executor-13d0a6fe-2375-443a-98ea-9911e3876119/spark-8f006f53-955d-495c-898b-12bddd021c6b/fetchFileTemp4241939507592543270.tmp
17/11/07 15:11:52 INFO Utils: Copying /tmp/spark-20a25310-733b-4be9-9f7d-de6cc752629b/executor-13d0a6fe-2375-443a-98ea-9911e3876119/spark-8f006f53-955d-495c-898b-12bddd021c6b/17577459481510085504389_cache to /home/david/TensorFlowOnSpark/spark-1.6.3-bin-hadoop2.6/work/app-20171107151144-0002/1/./mnist_dist.py
17/11/07 15:11:52 INFO Executor: Fetching http://<IP_ADDRESS>:46615/files/mnist_spark.py with timestamp<PHONE_NUMBER>322
17/11/07 15:11:52 INFO Utils: Fetching http://<IP_ADDRESS>:46615/files/mnist_spark.py to /tmp/spark-20a25310-733b-4be9-9f7d-de6cc752629b/executor-13d0a6fe-2375-443a-98ea-9911e3876119/spark-8f006f53-955d-495c-898b-12bddd021c6b/fetchFileTemp2774939994065177720.tmp
17/11/07 15:11:52 INFO Utils: Copying /tmp/spark-20a25310-733b-4be9-9f7d-de6cc752629b/executor-13d0a6fe-2375-443a-98ea-9911e3876119/spark-8f006f53-955d-495c-898b-12bddd021c6b/4300411671510085504322_cache to /home/david/TensorFlowOnSpark/spark-1.6.3-bin-hadoop2.6/work/app-20171107151144-0002/1/./mnist_spark.py
17/11/07 15:11:53 INFO TorrentBroadcast: Started reading broadcast variable 2
17/11/07 15:11:53 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 7.4 KB, free 511.1 MB)
17/11/07 15:11:53 INFO TorrentBroadcast: Reading broadcast variable 2 took 215 ms
17/11/07 15:11:53 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 10.6 KB, free 511.1 MB)
2017-11-07 15:11:55,424 INFO (MainThread-13772) connected to server at ('<IP_ADDRESS>', 46874)
2017-11-07 15:11:55,426 INFO (MainThread-13772) TFSparkNode.reserve: {'authkey': '\xc2\xc95\xd5\xa0iF\xe2\x83\xa4\xd3\xf7\x02\xd4v\x9a', 'worker_num': 1, 'host': '<IP_ADDRESS>', 'tb_port': 0, 'addr': '/tmp/pymp-YSHIXZ/listener-kjEt3q', 'ppid': 13755, 'task_index': 0, 'job_name': 'worker', 'tb_pid': 0, 'port': 43182}
2017-11-07 15:11:57,432 INFO (MainThread-13772) node: {'addr': ('<IP_ADDRESS>', 35517), 'task_index': 0, 'job_name': 'ps', 'authkey': '\xdf\x1d\xd2\x12\x01NN\t\x98\x1d\xd7\xb0\x96\xe5$1', 'worker_num': 0, 'host': '<IP_ADDRESS>', 'ppid': 13753, 'port': 46742, 'tb_pid': 0, 'tb_port': 0}
2017-11-07 15:11:57,432 INFO (MainThread-13772) node: {'addr': '/tmp/pymp-YSHIXZ/listener-kjEt3q', 'task_index': 0, 'job_name': 'worker', 'authkey': '\xc2\xc95\xd5\xa0iF\xe2\x83\xa4\xd3\xf7\x02\xd4v\x9a', 'worker_num': 1, 'host': '<IP_ADDRESS>', 'ppid': 13755, 'port': 43182, 'tb_pid': 0, 'tb_port': 0}
2017-11-07 15:11:57,445 INFO (MainThread-13772) Starting TensorFlow worker:0 on cluster node 1 on background process
17/11/07 15:11:57 INFO PythonRunner: Times: total = 3605, boot = 1442, init = 78, finish = 2085
17/11/07 15:11:57 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 999 bytes result sent to driver
17/11/07 15:11:57 INFO CoarseGrainedExecutorBackend: Got assigned task 2
17/11/07 15:11:57 INFO Executor: Running task 0.0 in stage 1.0 (TID 2)
17/11/07 15:11:57 INFO TorrentBroadcast: Started reading broadcast variable 3
17/11/07 15:11:57 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 6.6 KB, free 511.1 MB)
17/11/07 15:11:57 INFO TorrentBroadcast: Reading broadcast variable 3 took 26 ms
17/11/07 15:11:57 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 12.5 KB, free 511.1 MB)
17/11/07 15:11:57 INFO HadoopRDD: Input split: file:/home/david/TensorFlowOnSpark/examples/mnist/csv/train/images/part-00005:0+11173834
17/11/07 15:11:57 INFO TorrentBroadcast: Started reading broadcast variable 0
17/11/07 15:11:57 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 19.3 KB, free 511.1 MB)
17/11/07 15:11:57 INFO TorrentBroadcast: Reading broadcast variable 0 took 35 ms
17/11/07 15:11:58 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 280.8 KB, free 510.8 MB)
17/11/07 15:11:58 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
17/11/07 15:11:58 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
17/11/07 15:11:58 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
17/11/07 15:11:58 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
17/11/07 15:11:58 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
2017-11-07 15:11:58,595 INFO (MainThread-13801) 1: ======== worker:0 ========
2017-11-07 15:11:58,595 INFO (MainThread-13801) 1: Cluster spec: {'ps': ['<IP_ADDRESS>:46742'], 'worker': ['<IP_ADDRESS>:43182']}
2017-11-07 15:11:58,595 INFO (MainThread-13801) 1: Using CPU
E1107 15:11:58.700764039   13801 ev_epoll1_linux.c:1051]     grpc epoll fd: 12
2017-11-07 15:11:58.706813: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> <IP_ADDRESS>:46742}
2017-11-07 15:11:58.706860: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> localhost:43182}
2017-11-07 15:11:58.707517: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:324] Started server with target: grpc://localhost:43182
17/11/07 15:11:58 INFO HadoopRDD: Input split: file:/home/david/TensorFlowOnSpark/examples/mnist/csv/train/labels/part-00005:0+245760
17/11/07 15:11:58 INFO TorrentBroadcast: Started reading broadcast variable 1
17/11/07 15:11:58 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 19.3 KB, free 510.8 MB)
17/11/07 15:11:58 INFO TorrentBroadcast: Reading broadcast variable 1 took 33 ms
17/11/07 15:11:59 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 280.8 KB, free 510.5 MB)
2017-11-07 15:11:59,303 INFO (MainThread-13821) Connected to TFSparkNode.mgr on <IP_ADDRESS>, ppid=13755, state='running'
tensorflow model path: file:///home/david/TensorFlowOnSpark/mnist_model
2017-11-07 15:11:59,327 INFO (MainThread-13821) mgr.state='running'
2017-11-07 15:11:59,328 INFO (MainThread-13821) Feeding partition <generator object load_stream at 0x7f06d4689500> into input queue <multiprocessing.queues.JoinableQueue object at 0x7f06d463d790>
2017-11-07 15:11:59.511686: E tensorflow/core/distributed_runtime/master.cc:257] Master init: Unavailable: Trying to connect an http1.x server
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.UnavailableError'>, Trying to connect an http1.x server
2017-11-07 15:11:59,514 INFO (MainThread-13801) Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.UnavailableError'>, Trying to connect an http1.x server
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "mnist_dist.py", line 133, in map_fun
with sv.managed_session(server.target) as sess:
File "/usr/lib/python2.7/contextlib.py", line 17, in enter
return self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 964, in managed_session
self.stop(close_summary_writer=close_summary_writer)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 792, in stop
stop_grace_period_secs=self._stop_grace_secs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 953, in managed_session
start_standard_services=start_standard_services)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 708, in prepare_or_wait_for_session
init_feed_dict=self._init_feed_dict, init_fn=self._init_fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/session_manager.py", line 279, in prepare_session
sess.run(init_op, feed_dict=init_feed_dict)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1118, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1315, in _do_run
options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
raise type(e)(node_def, op, message)
UnavailableError: Trying to connect an http1.x server

Are you using TF 1.4?  I see this stack overflow which looks similar.  (Can you try TF 1.3?)

@leewyang i use tensorflow 1.3.0 tensorflowonspark 1.0.6

@zerodarkzone have you already solved？

[root@master tensorflowonspark]# /home/gzs/spark/bin/spark-submit --master spark://master:7077 --py-files tfspark.zip,examples/mnist/spark/mnist_dist.py --conf spark.cores.max=12 --conf spark.task.cpus=4 --conf spark.executorEnv.JAVA_HOME="$JAVA_HOME" examples/mnist/spark/mnist_spark.py --cluster_size 3 --images hdfs://master:9000/user/root/examples/mnist/csv/train/images --labels hdfs://master:9000/user/root/examples/mnist/csv/train/labels --format csv --mode train --model mnist_model
17/11/09 14:01:45 INFO spark.SparkContext: Running Spark version 2.1.1
17/11/09 14:01:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/09 14:01:46 WARN spark.SparkConf:
SPARK_WORKER_INSTANCES was detected (set to '3').
This is deprecated in Spark 1.0+.
Please instead use:

./spark-submit with --num-executors to specify the number of executors
Or set SPARK_EXECUTOR_INSTANCES
spark.executor.instances to configure the number of instances in the spark config.

17/11/09 14:01:46 INFO spark.SecurityManager: Changing view acls to: root
17/11/09 14:01:46 INFO spark.SecurityManager: Changing modify acls to: root
17/11/09 14:01:46 INFO spark.SecurityManager: Changing view acls groups to:
17/11/09 14:01:46 INFO spark.SecurityManager: Changing modify acls groups to:
17/11/09 14:01:46 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
17/11/09 14:01:46 INFO util.Utils: Successfully started service 'sparkDriver' on port 40865.
17/11/09 14:01:46 INFO spark.SparkEnv: Registering MapOutputTracker
17/11/09 14:01:46 INFO spark.SparkEnv: Registering BlockManagerMaster
17/11/09 14:01:46 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/11/09 14:01:46 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/11/09 14:01:46 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-5fd9a47f-bd07-40d6-8730-b5bd0091958c
17/11/09 14:01:46 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
17/11/09 14:01:46 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/11/09 14:01:46 INFO util.log: Logging initialized @2106ms
17/11/09 14:01:46 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@51e018c1{/jobs,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1aedaaea{/jobs/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@14592990{/jobs/job,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4c3f562b{/jobs/job/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1dc8697d{/stages,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6a053ad4{/stages/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@374d9bad{/stages/stage,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1750d61{/stages/stage/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5f280fb3{/stages/pool,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1c385d18{/stages/pool/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@47733dc1{/storage,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6ec43e4b{/storage/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@710ef1b7{/storage/rdd,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2f00a213{/storage/rdd/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@71c30f07{/environment,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3eba5112{/environment/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@38bf9c3b{/executors,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1285a105{/executors/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b573f65{/executors/threadDump,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1789d3dc{/executors/threadDump/json,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5393cb52{/static,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6ca1c5ac{/,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2e4f634a{/api,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7701526c{/jobs/job/kill,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3a05de85{/stages/stage/kill,null,AVAILABLE,@Spark}
17/11/09 14:01:46 INFO server.ServerConnector: Started Spark@7dfd5c5e{HTTP/1.1}{<IP_ADDRESS>:4040}
17/11/09 14:01:46 INFO server.Server: Started @2215ms
17/11/09 14:01:46 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/11/09 14:01:46 INFO ui.SparkUI: Bound SparkUI to <IP_ADDRESS>, and started at http://<IP_ADDRESS>:4040
17/11/09 14:01:47 INFO spark.SparkContext: Added file file:/home/gzs/tensorflowonspark/examples/mnist/spark/mnist_spark.py at spark://<IP_ADDRESS>:40865/files/mnist_spark.py with timestamp<PHONE_NUMBER>137
17/11/09 14:01:47 INFO util.Utils: Copying /home/gzs/tensorflowonspark/examples/mnist/spark/mnist_spark.py to /tmp/spark-e19a6cb7-92b4-4bee-9eba-ef2e4437d175/userFiles-8aae0d81-4e51-4eb4-8114-1afcf1378d42/mnist_spark.py
17/11/09 14:01:47 INFO spark.SparkContext: Added file file:/home/gzs/tensorflowonspark/tfspark.zip at spark://<IP_ADDRESS>:40865/files/tfspark.zip with timestamp<PHONE_NUMBER>148
17/11/09 14:01:47 INFO util.Utils: Copying /home/gzs/tensorflowonspark/tfspark.zip to /tmp/spark-e19a6cb7-92b4-4bee-9eba-ef2e4437d175/userFiles-8aae0d81-4e51-4eb4-8114-1afcf1378d42/tfspark.zip
17/11/09 14:01:47 INFO spark.SparkContext: Added file file:/home/gzs/tensorflowonspark/examples/mnist/spark/mnist_dist.py at spark://<IP_ADDRESS>:40865/files/mnist_dist.py with timestamp<PHONE_NUMBER>168
17/11/09 14:01:47 INFO util.Utils: Copying /home/gzs/tensorflowonspark/examples/mnist/spark/mnist_dist.py to /tmp/spark-e19a6cb7-92b4-4bee-9eba-ef2e4437d175/userFiles-8aae0d81-4e51-4eb4-8114-1afcf1378d42/mnist_dist.py
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Connecting to master spark://master:7077...
17/11/09 14:01:47 INFO client.TransportClientFactory: Successfully created connection to master/<IP_ADDRESS>:7077 after 22 ms (0 ms spent in bootstraps)
17/11/09 14:01:47 INFO cluster.StandaloneSchedulerBackend: Connected to Spark cluster with app ID app-20171109140147-0006
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Executor added: app-20171109140147-0006/0 on worker-20171107204122-<IP_ADDRESS>-35030 (<IP_ADDRESS>:35030) with 4 cores
17/11/09 14:01:47 INFO cluster.StandaloneSchedulerBackend: Granted executor ID app-20171109140147-0006/0 on hostPort <IP_ADDRESS>:35030 with 4 cores, 1024.0 MB RAM
17/11/09 14:01:47 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 36632.
17/11/09 14:01:47 INFO netty.NettyBlockTransferService: Server created on <IP_ADDRESS>:36632
17/11/09 14:01:47 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Executor added: app-20171109140147-0006/1 on worker-20171107204144-<IP_ADDRESS>-33358 (<IP_ADDRESS>:33358) with 4 cores
17/11/09 14:01:47 INFO cluster.StandaloneSchedulerBackend: Granted executor ID app-20171109140147-0006/1 on hostPort <IP_ADDRESS>:33358 with 4 cores, 1024.0 MB RAM
17/11/09 14:01:47 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, <IP_ADDRESS>, 36632, None)
17/11/09 14:01:47 INFO storage.BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:36632 with 366.3 MB RAM, BlockManagerId(driver, <IP_ADDRESS>, 36632, None)
17/11/09 14:01:47 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, <IP_ADDRESS>, 36632, None)
17/11/09 14:01:47 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, <IP_ADDRESS>, 36632, None)
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Executor added: app-20171109140147-0006/2 on worker-20171107204251-<IP_ADDRESS>-43884 (<IP_ADDRESS>:43884) with 4 cores
17/11/09 14:01:47 INFO cluster.StandaloneSchedulerBackend: Granted executor ID app-20171109140147-0006/2 on hostPort <IP_ADDRESS>:43884 with 4 cores, 1024.0 MB RAM
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Executor updated: app-20171109140147-0006/0 is now RUNNING
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Executor updated: app-20171109140147-0006/2 is now RUNNING
17/11/09 14:01:47 INFO client.StandaloneAppClient$ClientEndpoint: Executor updated: app-20171109140147-0006/1 is now RUNNING
17/11/09 14:01:47 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@350d2ef{/metrics/json,null,AVAILABLE,@Spark}
17/11/09 14:01:47 INFO cluster.StandaloneSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
args: Namespace(batch_size=100, cluster_size=3, epochs=1, format='csv', images='hdfs://master:9000/user/root/examples/mnist/csv/train/images', labels='hdfs://master:9000/user/root/examples/mnist/csv/train/labels', mode='train', model='mnist_model', output='predictions', rdma=False, readers=1, steps=1000, tensorboard=False)
2017-11-09T14:01:47.575404 ===== Start
17/11/09 14:01:47 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 238.8 KB, free 366.1 MB)
17/11/09 14:01:48 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 23.1 KB, free 366.0 MB)
17/11/09 14:01:48 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:36632 (size: 23.1 KB, free: 366.3 MB)
17/11/09 14:01:48 INFO spark.SparkContext: Created broadcast 0 from textFile at NativeMethodAccessorImpl.java:0
17/11/09 14:01:48 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 238.8 KB, free 365.8 MB)
17/11/09 14:01:48 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 23.1 KB, free 365.8 MB)
17/11/09 14:01:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on <IP_ADDRESS>:36632 (size: 23.1 KB, free: 366.3 MB)
17/11/09 14:01:48 INFO spark.SparkContext: Created broadcast 1 from textFile at NativeMethodAccessorImpl.java:0
zipping images and labels
17/11/09 14:01:48 INFO mapred.FileInputFormat: Total input paths to process : 10
17/11/09 14:01:48 INFO mapred.FileInputFormat: Total input paths to process : 10
2017-11-09 14:01:48,740 INFO (MainThread-6282) Reserving TFSparkNodes
2017-11-09 14:01:48,743 INFO (MainThread-6282) listening for reservations at ('<IP_ADDRESS>', 32904)
2017-11-09 14:01:48,744 INFO (MainThread-6282) Starting TensorFlow on executors
2017-11-09 14:01:48,766 INFO (MainThread-6282) Waiting for TFSparkNodes to start
2017-11-09 14:01:48,766 INFO (MainThread-6282) waiting for 3 reservations
17/11/09 14:01:48 INFO spark.SparkContext: Starting job: foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253
17/11/09 14:01:48 INFO scheduler.DAGScheduler: Got job 0 (foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253) with 3 output partitions
17/11/09 14:01:48 INFO scheduler.DAGScheduler: Final stage: ResultStage 0 (foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253)
17/11/09 14:01:48 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/11/09 14:01:48 INFO scheduler.DAGScheduler: Missing parents: List()
17/11/09 14:01:48 INFO scheduler.DAGScheduler: Submitting ResultStage 0 (PythonRDD[8] at foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253), which has no missing parents
17/11/09 14:01:48 INFO cluster.CoarseGrainedSchedulerBackend$DriverEndpoint: Registered executor NettyRpcEndpointRef(null) (<IP_ADDRESS>:44374) with ID 1
17/11/09 14:01:48 INFO cluster.CoarseGrainedSchedulerBackend$DriverEndpoint: Registered executor NettyRpcEndpointRef(null) (<IP_ADDRESS>:39942) with ID 0
17/11/09 14:01:48 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 10.7 KB, free 365.8 MB)
17/11/09 14:01:48 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 7.6 KB, free 365.8 MB)
17/11/09 14:01:48 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on <IP_ADDRESS>:36632 (size: 7.6 KB, free: 366.2 MB)
17/11/09 14:01:48 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:996
17/11/09 14:01:48 INFO scheduler.DAGScheduler: Submitting 3 missing tasks from ResultStage 0 (PythonRDD[8] at foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253)
17/11/09 14:01:48 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 3 tasks
17/11/09 14:01:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:44337 with 366.3 MB RAM, BlockManagerId(1, <IP_ADDRESS>, 44337, None)
17/11/09 14:01:48 INFO storage.BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:33516 with 366.3 MB RAM, BlockManagerId(0, <IP_ADDRESS>, 33516, None)
17/11/09 14:01:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, <IP_ADDRESS>, executor 0, partition 0, PROCESS_LOCAL, 6234 bytes)
17/11/09 14:01:48 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, <IP_ADDRESS>, executor 1, partition 1, PROCESS_LOCAL, 6234 bytes)
17/11/09 14:01:49 INFO cluster.CoarseGrainedSchedulerBackend$DriverEndpoint: Registered executor NettyRpcEndpointRef(null) (<IP_ADDRESS>:57552) with ID 2
17/11/09 14:01:49 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, <IP_ADDRESS>, executor 2, partition 2, PROCESS_LOCAL, 6234 bytes)
17/11/09 14:01:49 INFO storage.BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:37930 with 366.3 MB RAM, BlockManagerId(2, <IP_ADDRESS>, 37930, None)
2017-11-09 14:01:49,768 INFO (MainThread-6282) waiting for 3 reservations
2017-11-09 14:01:50,769 INFO (MainThread-6282) waiting for 3 reservations
2017-11-09 14:01:51,770 INFO (MainThread-6282) waiting for 3 reservations
2017-11-09 14:01:52,772 INFO (MainThread-6282) waiting for 3 reservations
2017-11-09 14:01:53,773 INFO (MainThread-6282) waiting for 3 reservations
17/11/09 14:01:54 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on <IP_ADDRESS>:37930 (size: 7.6 KB, free: 366.3 MB)
2017-11-09 14:01:54,775 INFO (MainThread-6282) waiting for 3 reservations
17/11/09 14:01:55 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on <IP_ADDRESS>:44337 (size: 7.6 KB, free: 366.3 MB)
17/11/09 14:01:55 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on <IP_ADDRESS>:33516 (size: 7.6 KB, free: 366.3 MB)
2017-11-09 14:01:55,776 INFO (MainThread-6282) waiting for 2 reservations
2017-11-09 14:01:56,777 INFO (MainThread-6282) all reservations completed
2017-11-09 14:01:56,777 INFO (MainThread-6282) All TFSparkNodes started
2017-11-09 14:01:56,778 INFO (MainThread-6282) {'addr': '/tmp/pymp-RXc1Cr/listener-mBn82J', 'task_index': 1, 'port': 40030, 'authkey': '\xcc\xe4R\xa8\x9a\xd2FM\xae{kn\x1c?\xaf\xe3', 'worker_num': 2, 'host': '<IP_ADDRESS>', 'ppid': 9941, 'job_name': 'worker', 'tb_pid': 0, 'tb_port': 0}
2017-11-09 14:01:56,778 INFO (MainThread-6282) {'addr': '/tmp/pymp-VrPdJp/listener-vkMbcA', 'task_index': 0, 'port': 40454, 'authkey': '\xc9\x85s\xf5\xd4\xefL\xc0\x88\xe2w\xf9\x9b\x88\xd6\xa1', 'worker_num': 1, 'host': '<IP_ADDRESS>', 'ppid': 23707, 'job_name': 'worker', 'tb_pid': 0, 'tb_port': 0}
2017-11-09 14:01:56,778 INFO (MainThread-6282) {'addr': ('<IP_ADDRESS>', 39708), 'task_index': 0, 'port': 36314, 'authkey': 'G\xa7\x1d\xe2Q\xcbJ\xdd\x84\x02\x81\xa6\xc1\xcb\xb8\xe6', 'worker_num': 0, 'host': '<IP_ADDRESS>', 'ppid': 12725, 'job_name': 'ps', 'tb_pid': 0, 'tb_port': 0}
2017-11-09 14:01:56,778 INFO (MainThread-6282) Feeding training data
17/11/09 14:01:56 INFO spark.SparkContext: Starting job: collect at PythonRDD.scala:453
17/11/09 14:01:56 INFO scheduler.DAGScheduler: Got job 1 (collect at PythonRDD.scala:453) with 10 output partitions
17/11/09 14:01:56 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (collect at PythonRDD.scala:453)
17/11/09 14:01:56 INFO scheduler.DAGScheduler: Parents of final stage: List()
17/11/09 14:01:56 INFO scheduler.DAGScheduler: Missing parents: List()
17/11/09 14:01:56 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (PythonRDD[10] at RDD at PythonRDD.scala:48), which has no missing parents
17/11/09 14:01:56 INFO memory.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 12.8 KB, free 365.8 MB)
17/11/09 14:01:56 INFO memory.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 6.9 KB, free 365.8 MB)
17/11/09 14:01:56 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on <IP_ADDRESS>:36632 (size: 6.9 KB, free: 366.2 MB)
17/11/09 14:01:56 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:996
17/11/09 14:01:56 INFO scheduler.DAGScheduler: Submitting 10 missing tasks from ResultStage 1 (PythonRDD[10] at RDD at PythonRDD.scala:48)
17/11/09 14:01:56 INFO scheduler.TaskSchedulerImpl: Adding task set 1.0 with 10 tasks
17/11/09 14:01:57 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 3, <IP_ADDRESS>, executor 1, partition 0, ANY, 6785 bytes)
17/11/09 14:01:57 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 8861 ms on <IP_ADDRESS> (executor 1) (1/3)
17/11/09 14:01:57 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on <IP_ADDRESS>:44337 (size: 6.9 KB, free: 366.3 MB)
17/11/09 14:01:57 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:44337 (size: 23.1 KB, free: 366.3 MB)
17/11/09 14:01:57 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 4, <IP_ADDRESS>, executor 2, partition 1, ANY, 6785 bytes)
17/11/09 14:01:57 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 8774 ms on <IP_ADDRESS> (executor 2) (2/3)
17/11/09 14:01:57 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on <IP_ADDRESS>:37930 (size: 6.9 KB, free: 366.3 MB)
17/11/09 14:01:58 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:37930 (size: 23.1 KB, free: 366.3 MB)
17/11/09 14:01:58 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on <IP_ADDRESS>:44337 (size: 23.1 KB, free: 366.2 MB)
17/11/09 14:01:59 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on <IP_ADDRESS>:37930 (size: 23.1 KB, free: 366.2 MB)
(kill)
foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253 +details
org.apache.spark.rdd.RDD.collect(RDD.scala:935)
org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:453)
org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:280)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)
py4j.GatewayConnection.run(GatewayConnection.java:214)
java.lang.Thread.run(Thread.java:748)
collect at PythonRDD.scala:453 +details
org.apache.spark.rdd.RDD.collect(RDD.scala:935)
org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:453)
org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:280)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)
py4j.GatewayConnection.run(GatewayConnection.java:214)
java.lang.Thread.run(Thread.java:748)

the job stop and did't show nothing more.
i go to the webui and copy the info for details.

and the stderr on executer 0
17/11/09 14:01:31 INFO executor.CoarseGrainedExecutorBackend: Started daemon with process name: 12671@slave1
17/11/09 14:01:31 INFO util.SignalUtils: Registered signal handler for TERM
17/11/09 14:01:31 INFO util.SignalUtils: Registered signal handler for HUP
17/11/09 14:01:31 INFO util.SignalUtils: Registered signal handler for INT
17/11/09 14:01:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/09 14:01:32 INFO spark.SecurityManager: Changing view acls to: root
17/11/09 14:01:32 INFO spark.SecurityManager: Changing modify acls to: root
17/11/09 14:01:32 INFO spark.SecurityManager: Changing view acls groups to:
17/11/09 14:01:32 INFO spark.SecurityManager: Changing modify acls groups to:
17/11/09 14:01:32 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
17/11/09 14:01:32 INFO client.TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:40865 after 44 ms (0 ms spent in bootstraps)
17/11/09 14:01:32 INFO spark.SecurityManager: Changing view acls to: root
17/11/09 14:01:32 INFO spark.SecurityManager: Changing modify acls to: root
17/11/09 14:01:32 INFO spark.SecurityManager: Changing view acls groups to:
17/11/09 14:01:32 INFO spark.SecurityManager: Changing modify acls groups to:
17/11/09 14:01:32 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
17/11/09 14:01:32 INFO client.TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:40865 after 1 ms (0 ms spent in bootstraps)
17/11/09 14:01:32 INFO storage.DiskBlockManager: Created local directory at /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/blockmgr-df8c5cae-1a94-47c7-823c-b70a800d9f78
17/11/09 14:01:32 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
17/11/09 14:01:32 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@<IP_ADDRESS>:40865
17/11/09 14:01:32 INFO worker.WorkerWatcher: Connecting to worker spark://Worker@<IP_ADDRESS>:35030
17/11/09 14:01:32 INFO client.TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:35030 after 0 ms (0 ms spent in bootstraps)
17/11/09 14:01:32 INFO worker.WorkerWatcher: Successfully connected to spark://Worker@<IP_ADDRESS>:35030
17/11/09 14:01:32 INFO executor.CoarseGrainedExecutorBackend: Successfully registered with driver
17/11/09 14:01:32 INFO executor.Executor: Starting executor ID 0 on host <IP_ADDRESS>
17/11/09 14:01:32 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 33516.
17/11/09 14:01:32 INFO netty.NettyBlockTransferService: Server created on <IP_ADDRESS>:33516
17/11/09 14:01:32 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/11/09 14:01:32 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(0, <IP_ADDRESS>, 33516, None)
17/11/09 14:01:32 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(0, <IP_ADDRESS>, 33516, None)
17/11/09 14:01:32 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(0, <IP_ADDRESS>, 33516, None)
17/11/09 14:01:33 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 0
17/11/09 14:01:33 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0)
17/11/09 14:01:33 INFO executor.Executor: Fetching spark://<IP_ADDRESS>:40865/files/mnist_spark.py with timestamp<PHONE_NUMBER>137
17/11/09 14:01:33 INFO client.TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:40865 after 1 ms (0 ms spent in bootstraps)
17/11/09 14:01:33 INFO util.Utils: Fetching spark://<IP_ADDRESS>:40865/files/mnist_spark.py to /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/spark-d3c9059d-be5a-4a65-aebb-6c3397f3922b/fetchFileTemp4946111245642308510.tmp
17/11/09 14:01:33 INFO util.Utils: Copying /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/spark-d3c9059d-be5a-4a65-aebb-6c3397f3922b/-17510682901510207307137_cache to /home/gzs/spark/work/app-20171109140147-0006/0/./mnist_spark.py
17/11/09 14:01:33 INFO executor.Executor: Fetching spark://<IP_ADDRESS>:40865/files/tfspark.zip with timestamp<PHONE_NUMBER>148
17/11/09 14:01:33 INFO util.Utils: Fetching spark://<IP_ADDRESS>:40865/files/tfspark.zip to /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/spark-d3c9059d-be5a-4a65-aebb-6c3397f3922b/fetchFileTemp492752251712140703.tmp
17/11/09 14:01:39 INFO util.Utils: Copying /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/spark-d3c9059d-be5a-4a65-aebb-6c3397f3922b/13007049121510207307148_cache to /home/gzs/spark/work/app-20171109140147-0006/0/./tfspark.zip
17/11/09 14:01:39 INFO executor.Executor: Fetching spark://<IP_ADDRESS>:40865/files/mnist_dist.py with timestamp<PHONE_NUMBER>168
17/11/09 14:01:39 INFO util.Utils: Fetching spark://<IP_ADDRESS>:40865/files/mnist_dist.py to /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/spark-d3c9059d-be5a-4a65-aebb-6c3397f3922b/fetchFileTemp7652626049412260218.tmp
17/11/09 14:01:39 INFO util.Utils: Copying /tmp/spark-3065f81e-2a0d-4bb6-b8fc-5e03aaec3e33/executor-1a54ffb7-d4c7-4237-9afc-8a78316c82ab/spark-d3c9059d-be5a-4a65-aebb-6c3397f3922b/1633669251510207307168_cache to /home/gzs/spark/work/app-20171109140147-0006/0/./mnist_dist.py
17/11/09 14:01:39 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 2
17/11/09 14:01:39 INFO client.TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:44337 after 1 ms (0 ms spent in bootstraps)
17/11/09 14:01:39 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 7.6 KB, free 366.3 MB)
17/11/09 14:01:39 INFO broadcast.TorrentBroadcast: Reading broadcast variable 2 took 149 ms
17/11/09 14:01:39 INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 10.7 KB, free 366.3 MB)
2017-11-09 14:01:40,185 INFO (MainThread-12731) connected to server at ('<IP_ADDRESS>', 32904)
2017-11-09 14:01:40,188 INFO (MainThread-12731) TFSparkNode.reserve: {'authkey': 'G\xa7\x1d\xe2Q\xcbJ\xdd\x84\x02\x81\xa6\xc1\xcb\xb8\xe6', 'worker_num': 0, 'host': '<IP_ADDRESS>', 'tb_port': 0, 'addr': ('<IP_ADDRESS>', 39708), 'ppid': 12725, 'task_index': 0, 'job_name': 'ps', 'tb_pid': 0, 'port': 36314}
2017-11-09 14:01:41,192 INFO (MainThread-12731) node: {'addr': ('<IP_ADDRESS>', 39708), 'task_index': 0, 'job_name': 'ps', 'authkey': 'G\xa7\x1d\xe2Q\xcbJ\xdd\x84\x02\x81\xa6\xc1\xcb\xb8\xe6', 'worker_num': 0, 'host': '<IP_ADDRESS>', 'ppid': 12725, 'port': 36314, 'tb_pid': 0, 'tb_port': 0}
2017-11-09 14:01:41,192 INFO (MainThread-12731) node: {'addr': '/tmp/pymp-VrPdJp/listener-vkMbcA', 'task_index': 0, 'job_name': 'worker', 'authkey': '\xc9\x85s\xf5\xd4\xefL\xc0\x88\xe2w\xf9\x9b\x88\xd6\xa1', 'worker_num': 1, 'host': '<IP_ADDRESS>', 'ppid': 23707, 'port': 40454, 'tb_pid': 0, 'tb_port': 0}
2017-11-09 14:01:41,192 INFO (MainThread-12731) node: {'addr': '/tmp/pymp-RXc1Cr/listener-mBn82J', 'task_index': 1, 'job_name': 'worker', 'authkey': '\xcc\xe4R\xa8\x9a\xd2FM\xae{kn\x1c?\xaf\xe3', 'worker_num': 2, 'host': '<IP_ADDRESS>', 'ppid': 9941, 'port': 40030, 'tb_pid': 0, 'tb_port': 0}
2017-11-09 14:01:41,198 INFO (MainThread-12731) Starting TensorFlow ps:0 on cluster node 0 on background process
2017-11-09 14:01:46,776 INFO (MainThread-12745) 0: ======== ps:0 ========
2017-11-09 14:01:46,776 INFO (MainThread-12745) 0: Cluster spec: {'ps': ['<IP_ADDRESS>:36314'], 'worker': ['<IP_ADDRESS>:40454', '<IP_ADDRESS>:40030']}
2017-11-09 14:01:46,776 INFO (MainThread-12745) 0: Using CPU
2017-11-09 14:01:46.777485: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-09 14:01:46.777520: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-09 14:01:46.777536: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-11-09 14:01:46.788955: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> localhost:36314}
2017-11-09 14:01:46.789021: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> <IP_ADDRESS>:40454, 1 -> <IP_ADDRESS>:40030}
2017-11-09 14:01:46.789588: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:36314

It works for me on Tensorflow 1.3 but it fails with 1.4.
@lzzzzzzzzz  could you show the stderr from your other executors?,  the error I have only shows in one of them, the rest seem fine.

@leewyang where can i find mnist_model ,beacause the job is waiting for model to be ready but it doesn't exists.

@zerodarkzone i found mnist_model doesn't exists ,i have to upload on hdfs://master:9000/user/root/mnist_model

@zerodarkzone i view the stderr and find one of the  job is waiting for model to be ready and the model path is'hdfs://master:9000/user/root/mnist_model' so it stop forever!

I have a similar issue with HDP 2.6.3, the training session doesn't finish, here's what I did:
[root@I77700 TensorFlowOnSpark-master]# spark-submit \

--master yarn
--deploy-mode cluster
--queue default
--num-executors 1
--executor-memory 3G
--archives /root/mnistdata/mnist.zip#mnist
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA
--driver-library-path=$LIB_CUDA
/root/Downloads/TensorFlowOnSpark-master/examples/mnist/mnist_data_setup.py
--output mnist/csv
--format csv
Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set
Spark1 will be picked by default
.
.
17/11/10 15:23:01 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: <IP_ADDRESS>
ApplicationMaster RPC port: 0
queue: default
start time:<PHONE_NUMBER>043
final status: SUCCEEDED
tracking URL: http://I77700:8088/proxy/application_1510355100393_0008/
user: root
17/11/10 15:23:01 INFO ShutdownHookManager: Shutdown hook called
17/11/10 15:23:01 INFO ShutdownHookManager: Deleting directory /tmp/spark-f9bc91c3-250a-45d9-8edc-23d4194d8eb6
[root@I77700 TensorFlowOnSpark-master]# hdfs dfs -ls /user/root
Found 2 items
drwxr-xr-x   - root root          0 2017-11-10 15:23 /user/root/.sparkStaging
drwxr-xr-x   - root root          0 2017-11-10 15:21 /user/root/mnist

[root@I77700 TensorFlowOnSpark-master]# hdfs dfs -ls  /user/root/mnist/csv/test/images
Found 11 items
-rw-r--r--   1 root root          0 2017-11-10 15:23 /user/root/mnist/csv/test/images/_SUCCESS
-rw-r--r--   1 root root    1810248 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00000
-rw-r--r--   1 root root    1806102 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00001
-rw-r--r--   1 root root    1811128 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00002
-rw-r--r--   1 root root    1812952 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00003
-rw-r--r--   1 root root    1810946 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00004
-rw-r--r--   1 root root    1835497 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00005
-rw-r--r--   1 root root    1845261 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00006
-rw-r--r--   1 root root    1850655 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00007
-rw-r--r--   1 root root    1852712 2017-11-10 15:22 /user/root/mnist/csv/test/images/part-00008
-rw-r--r--   1 root root    1833942 2017-11-10 15:23 /user/root/mnist/csv/test/images/part-00009
[root@I77700 TensorFlowOnSpark-master]# hdfs dfs -ls  /user/root/mnist/csv/train/images
Found 11 items
-rw-r--r--   1 root root          0 2017-11-10 15:22 /user/root/mnist/csv/train/images/_SUCCESS
-rw-r--r--   1 root root    9338236 2017-11-10 15:21 /user/root/mnist/csv/train/images/part-00000
-rw-r--r--   1 root root   11231804 2017-11-10 15:21 /user/root/mnist/csv/train/images/part-00001
-rw-r--r--   1 root root   11214784 2017-11-10 15:21 /user/root/mnist/csv/train/images/part-00002
-rw-r--r--   1 root root   11226100 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00003
-rw-r--r--   1 root root   11212767 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00004
-rw-r--r--   1 root root   11173834 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00005
-rw-r--r--   1 root root   11214285 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00006
-rw-r--r--   1 root root   11201024 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00007
-rw-r--r--   1 root root   11194141 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00008
-rw-r--r--   1 root root   10449019 2017-11-10 15:22 /user/root/mnist/csv/train/images/part-00009
Here's the training script:
[root@I77700 TensorFlowOnSpark-master]# spark-submit \

--master yarn
--deploy-mode cluster
--queue default
--num-executors 1
--executor-memory 3G
--driver-cores 4
--py-files /root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.yarn.maxAppAttempts=1
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA
--conf spark.executorEnv.HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
--driver-library-path=$LIB_CUDA
/root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_spark.py
--images mnist/csv/train/images
--labels mnist/csv/train/labels
--mode train
--model mnist_model

And stderr:
2017-11-10 15:27:44,654 INFO (MainThread-29907) Feeding partition <generator object load_stream at 0x2321780> into input queue <multiprocessing.queues.JoinableQueue object at 0x21d8ed0>
17/11/10 15:27:46 INFO PythonRunner: Times: total = 1599, boot = -547, init = 612, finish = 1534
17/11/10 15:27:46 INFO PythonRunner: Times: total = 61, boot = 2, init = 7, finish = 52
2017-11-10 15:27:46,253 INFO (MainThread-29887) 0: ======== worker:0 ========
2017-11-10 15:27:46,253 INFO (MainThread-29887) 0: Cluster spec: {'worker': ['<IP_ADDRESS>:40096']}
2017-11-10 15:27:46,253 INFO (MainThread-29887) 0: Using CPU
2017-11-10 15:27:46.254213: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-10 15:27:46.254223: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-10 15:27:46.254226: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-11-10 15:27:46.254229: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-10 15:27:46.254232: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-11-10 15:27:46.258064: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> localhost:40096}
2017-11-10 15:27:46.258653: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:316] Started server with target: grpc://localhost:40096
tensorflow model path: hdfs://i77700:8020/user/root/mnist_model
.
.
17/11/10 15:27:43 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on <IP_ADDRESS>:43212 (size: 6.7 KB, free: 457.8 MB)
17/11/10 15:27:43 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1008
17/11/10 15:27:43 INFO DAGScheduler: Submitting 10 missing tasks from ResultStage 1 (PythonRDD[10] at RDD at PythonRDD.scala:43)
17/11/10 15:27:43 INFO YarnClusterScheduler: Adding task set 1.0 with 10 tasks
17/11/10 15:27:43 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, I77700, partition 0,NODE_LOCAL, 2620 bytes)
17/11/10 15:27:43 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 1575 ms on I77700 (1/1)
17/11/10 15:27:43 INFO YarnClusterScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/11/10 15:27:43 INFO DAGScheduler: ResultStage 0 (foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253) finished in 1.658 s
17/11/10 15:27:43 INFO DAGScheduler: Job 0 finished: foreachPartition at /usr/lib/python2.7/site-packages/tensorflowonspark/TFCluster.py:253, took 1.687859 s
17/11/10 15:27:43 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on I77700:44197 (size: 6.7 KB, free: 1983.0 MB)
17/11/10 15:27:43 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on I77700:44197 (size: 29.8 KB, free: 1983.0 MB)
17/11/10 15:27:44 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on I77700:44197 (size: 29.8 KB, free: 1982.9 MB)
17/11/10 15:57:39 INFO BlockManagerInfo: Removed broadcast_2_piece0 on <IP_ADDRESS>:43212 in memory (size: 7.6 KB, free: 457.8 MB)
17/11/10 15:57:39 INFO BlockManagerInfo: Removed broadcast_2_piece0 on I77700:44197 in memory (size: 7.6 KB, free: 1982.9 MB)
17/11/10 15:57:39 INFO ContextCleaner: Cleaned accumulator 2
Then the job stops progressing Trying to attach two html thread dumps, one for the master one for the worker:
threaddump1.txt
threaddump2.txt
Thanks.

Same situation with Spark2:

@lzzzzzzzzz @djygithub That symptom is usually associated with the fact that you don't have the libhdfs.so in your LD_LIBRARY_PATH.  TensorFlow needs this in order to save/create the mnist_model on HDFS.

Changed the spark log level from INFO to ALL:

Invoked training:
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_HOME=/usr/hdp/current/hadoop-client/
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_PREFIX=/usr/hdp/current/hadoop-client/
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_CONF_DIR=/etc/hadoop/conf
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_COMMON_HOME=/usr/hdp/current/hadoop-client/
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_MAPRED_HOME=/usr/hdp/current/hadoop-client/
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_LIBEXEC_DIR=/usr/hdp/current/hadoop-client/libexec
[root@I77700 TensorFlowOnSpark-master]# export PYTHONPATH=/usr/bin/python
[root@I77700 TensorFlowOnSpark-master]# export HADOOP_PREFIX=/usr/hdp/current/hadoop-client
[root@I77700 TensorFlowOnSpark-master]# export LIB_HDFS=/usr/hdp/<IP_ADDRESS>-8/usr/lib:/usr/hdp/current/hadoop-client/lib/native:
[root@I77700 TensorFlowOnSpark-master]# export PYTHON_ROOT=/usr
[root@I77700 TensorFlowOnSpark-master]# export LD_LIBRARY_PATH=${PATH}
[root@I77700 TensorFlowOnSpark-master]# export PYSPARK_PYTHON=${PYTHON_ROOT}/bin/python
[root@I77700 TensorFlowOnSpark-master]# export SPARK_YARN_USER_ENV="PYSPARK_PYTHON=/usr/bin/python"
[root@I77700 TensorFlowOnSpark-master]# export PATH=${PYTHON_ROOT}/bin/:$PATH
[root@I77700 TensorFlowOnSpark-master]# export QUEUE=default
[root@I77700 TensorFlowOnSpark-master]# export LIB_JVM=/usr/jdk64/jdk1.8.0_112/jre/lib/amd64
[root@I77700 TensorFlowOnSpark-master]# export SPARK_HOME=/usr/hdp/current/spark1-client
[root@I77700 TensorFlowOnSpark-master]# export LIB_CUDA=/usr/local/cuda-8.0/targets/x86_64-linux/lib/
[root@I77700 TensorFlowOnSpark-master]# export CUDA_HOME=/root/Downloads/cuda
[root@I77700 TensorFlowOnSpark-master]# export CUDA_HOME="/root/Downloads/cuda"
[root@I77700 TensorFlowOnSpark-master]# export LIB_CUDA=/usr/local/cuda-8.0/targets/x86_64-linux/lib/
[root@I77700 TensorFlowOnSpark-master]# spark-submit \

--master yarn
--deploy-mode cluster
--queue default
--num-executors 1
--executor-memory 3G
--driver-cores 1
--py-files /root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.yarn.maxAppAttempts=1
--conf spark.executorEnv.HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA:$LIB_JVM:$LIB_HDFS
--driver-library-path=$LIB_CUDA
/root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_spark.py
--images mnist/csv/train/images
--labels mnist/csv/train/labels
--mode train
--model mnist_model

Seems to be looping around this message, i tried to increase the yarn max contatiner size it just keeps asking for more:
17/11/11 09:34:39 INFO Client: Application report for application_1510418026588_0010 (state: RUNNING)
17/11/11 09:34:39 DEBUG Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: <IP_ADDRESS>
ApplicationMaster RPC port: 0
queue: default
start time:<PHONE_NUMBER>744
final status: UNDEFINED
tracking URL: http://I77700:8088/proxy/application_1510418026588_0010/
user: root
17/11/11 09:34:40 TRACE ProtobufRpcEngine: 1: Call -> i77700/<IP_ADDRESS>:8050: getApplicationReport {application_id { id: 10 cluster_timestamp:<PHONE_NUMBER>588 }}
17/11/11 09:34:40 DEBUG Client: IPC Client (669284403) connection to i77700/<IP_ADDRESS>:8050 from root sending #144
17/11/11 09:34:40 DEBUG Client: IPC Client (669284403) connection to i77700/<IP_ADDRESS>:8050 from root got value #144
17/11/11 09:34:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
17/11/11 09:34:40 TRACE ProtobufRpcEngine: 1: Response <- i77700/<IP_ADDRESS>:8050: getApplicationReport {application_report { applicationId { id: 10 cluster_timestamp:<PHONE_NUMBER>588 } user: "root" queue: "default" name: "mnist_spark.py" host: "<IP_ADDRESS>" rpc_port: 0 yarn_application_state: RUNNING trackingUrl: "http://I77700:8088/proxy/application_1510418026588_0010/" diagnostics: "" startTime:<PHONE_NUMBER>744 finishTime: 0 final_application_status: APP_UNDEFINED app_resource_Usage { num_used_containers: 2 num_reserved_containers: 0 used_resources { memory: 6144 virtual_cores: 2 } reserved_resources { memory: 0 virtual_cores: 0 } needed_resources { memory: 6144 virtual_cores: 2 } memory_seconds: 455313 vcore_seconds: 150 queue_usage_percentage: 25.0 cluster_usage_percentage: 25.0 } originalTrackingUrl: "http://<IP_ADDRESS>:39700" currentApplicationAttemptId { application_id { id: 10 cluster_timestamp:<PHONE_NUMBER>588 } attemptId: 1 } progress: 0.1 applicationType: "SPARK" log_aggregation_status: LOG_NOT_START unmanaged_application: false appNodeLabelExpression: "" amNodeLabelExpression: "<DEFAULT_PARTITION>" }}
17/11/11 09:34:40 INFO Client: Application report for application_1510418026588_0010 (state: RUNNING)
I've attached the console output, thanks again.
tfoshdfs.traceall.txt

Tried adding the full path for the mnist_model as folows, the results are the same:
11/12/2017 two
export HADOOP_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_PREFIX=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_HDFS_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_CONF_DIR=/usr/hdp/<IP_ADDRESS>-8/hadoop/etc/hadoop
export HADOOP_COMMON_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_MAPRED_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_LIBEXEC_DIR=/usr/hdp/<IP_ADDRESS>-8/hadoop/libexec
export PYTHONPATH=/usr/bin/python
export LIB_HDFS=/usr/hdp/<IP_ADDRESS>-8/usr/lib:/usr/hdp/current/hadoop-client/lib/native:
export PYTHON_ROOT=/usr
export LD_LIBRARY_PATH=${PATH}
export PYSPARK_PYTHON=${PYTHON_ROOT}/bin/python
export SPARK_YARN_USER_ENV="PYSPARK_PYTHON=/usr/bin/python"
export PATH=${PYTHON_ROOT}/bin/:$PATH
export QUEUE=default
export LIB_JVM=/usr/jdk64/jdk1.8.0_112/bin/
export SPARK_HOME=/usr/hdp/current/spark1-client
export LIB_CUDA=/usr/local/cuda-8.0/targets/x86_64-linux/lib/
export CUDA_HOME=/root/Downloads/cuda
export JAVA_HOME=/usr/jdk64/jdk1.8.0_112/bin/
spark-submit
--master yarn
--deploy-mode cluster
--queue default
--num-executors 1
--executor-memory 2G
--py-files /root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.yarn.maxAppAttempts=1
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA:$LIB_JVM:$LIB_HDFS
--driver-library-path=$LIB_CUDA
--conf spark.executorEnv.HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
/root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_spark.py
--images mnist/csv/train/images
--labels mnist/csv/train/labels
--mode train
--model hdfs://I77700:8020/user/root/mnist_model
Thanks.

Hello, just like you said, libhdfs.so is the culprit.  Copied libhdfs.so into

Thanks a ton!
export HADOOP_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_PREFIX=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_HDFS_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_CONF_DIR=/usr/hdp/<IP_ADDRESS>-8/hadoop/etc/hadoop
export HADOOP_COMMON_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_MAPRED_HOME=/usr/hdp/<IP_ADDRESS>-8/hadoop
export HADOOP_LIBEXEC_DIR=/usr/hdp/<IP_ADDRESS>-8/hadoop/libexec
export PYTHONPATH=/usr/bin/python
export LIB_HDFS=/usr/hdp/<IP_ADDRESS>-8/usr/lib:/usr/hdp/current/hadoop-client/lib/native:
export PYTHON_ROOT=/usr
export LD_LIBRARY_PATH=${PATH}
export PYSPARK_PYTHON=${PYTHON_ROOT}/bin/python
export SPARK_YARN_USER_ENV="PYSPARK_PYTHON=/usr/bin/python"
export PATH=${PYTHON_ROOT}/bin/:$PATH
export QUEUE=default
export LIB_JVM=/usr/jdk64/jdk1.8.0_112/jre/lib/amd64/server/
export SPARK_HOME=/usr/hdp/current/spark1-client
export LIB_CUDA=/usr/local/cuda-8.0/targets/x86_64-linux/lib/
export CUDA_HOME=/root/Downloads/cuda
export JAVA_HOME=/usr/jdk64/jdk1.8.0_112/bin/
spark-submit
--master yarn
--deploy-mode cluster
--queue default
--num-executors 1
--executor-memory 2G
--py-files /root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.yarn.maxAppAttempts=1
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA:$LIB_JVM:$LIB_HDFS
--driver-library-path=$LIB_CUDA
--conf spark.executorEnv.HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
/root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_spark.py
--images mnist/csv/train/images
--labels mnist/csv/train/labels
--mode train
--model hdfs://I77700:8020/user/root/mnist_model
17/11/13 06:04:42 INFO Client: Application report for application_1510578512329_0001 (state: FINISHED)
17/11/13 06:04:42 DEBUG Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: <IP_ADDRESS>
ApplicationMaster RPC port: 0
queue: default
start time:<PHONE_NUMBER>004
final status: SUCCEEDED
tracking URL: http://I77700:8088/proxy/application_1510578512329_0001/
user: root
17/11/13 06:04:42 TRACE ProtobufRpcEngine: 1: Call -> I77700/<IP_ADDRESS>:8020: getFileInfo {src: "/user/root/.sparkStaging/application_1510578512329_0001"}
17/11/13 06:04:42 DEBUG Client: The ping interval is 60000 ms.
17/11/13 06:04:42 DEBUG Client: Connecting to I77700/<IP_ADDRESS>:8020
17/11/13 06:04:42 DEBUG Client: IPC Client<PHONE_NUMBER>) connection to I77700/<IP_ADDRESS>:8020 from root: starting, having connections 2
17/11/13 06:04:42 DEBUG Client: IPC Client<PHONE_NUMBER>) connection to I77700/<IP_ADDRESS>:8020 from root sending #105
17/11/13 06:04:42 DEBUG Client: IPC Client<PHONE_NUMBER>) connection to I77700/<IP_ADDRESS>:8020 from root got value #105
17/11/13 06:04:42 DEBUG ProtobufRpcEngine: Call: getFileInfo took 3ms
17/11/13 06:04:42 TRACE ProtobufRpcEngine: 1: Response <- I77700/<IP_ADDRESS>:8020: getFileInfo {}
17/11/13 06:04:42 INFO ShutdownHookManager: Shutdown hook called
17/11/13 06:04:42 INFO ShutdownHookManager: Deleting directory /tmp/spark-a9fe2931-5e40-480e-b1d9-3ca7834bdc43
17/11/13 06:04:42 DEBUG Client: stopping client from cache: org.apache.hadoop.ipc.Client@4b7dc788
17/11/13 06:04:42 DEBUG ShutdownHookManager: ShutdownHookManger complete shutdown.
[root@I77700 ~]#
[root@I77700 ~]# hdfs dfs -ls /user/root
Found 3 items
drwxrwxrwx   - root root          0 2017-11-13 06:04 /user/root/.sparkStaging
drwxrwxrwx   - root root          0 2017-11-10 15:21 /user/root/mnist
drwxr-xr-x   - yarn root          0 2017-11-13 06:04 /user/root/mnist_model
[root@I77700 ~]#

@djygithub you can look issue #33.

@leewyang
first time I  try to export LD_LIBRARY_PATH=/home/gzs/hadoop/lib/native
i have libhadoop.so in this path,and the warn can't load native is disappeared.
but still stuck out.
then I try to run commd with --conf LD_LIBRARY_PATH=/home/gzs/hadoop/lib/native
but still stuck out.
I look up issue #33,the I try to change file.
mnist_dist.py
line 115 logdir=none
line 124 logdir=none
It doesn't work.
At last,I'm not sure if it's my problem.
And tomorrow i will upload a detailed log report for you.
Thank you for help!

@leewyang  @djygithub I think the answer is correct.I will try again tomorrow.Thank you.

The other shoe (inference)
spark-submit
--master yarn
--deploy-mode cluster
--queue default
--num-executors 1
--executor-memory 2G
--py-files /root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.yarn.maxAppAttempts=1
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA:$LIB_JVM:$LIB_HDFS
--driver-library-path=$LIB_CUDA
--conf spark.executorEnv.HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
/root/Downloads/TensorFlowOnSpark-master/examples/mnist/spark/mnist_spark.py
--images mnist/csv/test/images
--labels mnist/csv/test/labels
--mode inference
--model hdfs://I77700:8020/user/root/mnist_model
--output predictions

@lzzzzzzzzz you will need to use --conf spark.executorEnv.LD_LIBRARY_PATH=/home/gzs/hadoop/lib/native.  This sets the LD_LIBRARY_PATH on the executors (not just the driver).  Note that the libhdfs.so file must be available at that location on the executors.

executor0.txt
executor1.txt
executor2.txt
master.txt
@leewyang sometimes I will get stderr like this

executor0.txt
executor1.txt
executor2.txt
master.txt
mosttime the stderr like this.
throw I copy the libhdfs.so to all slaves in the same path.

@leewyang Does the task require hardware?
Each computer have 2.6G memory and 4 cores (cpu) and one of them does't have a gpu.
And the job just take 1G memory.

@lzzzzzzzzz for the KeyError: 'input' issue, that happens if:

you're configured for more than one task per executor, or
you have mismatched cluster_size vs num_executors.

Please see the FAQ for more details.
As for hardware, the MNIST example is fairly small, and should run on CPU nodes in under 1-2GB, and should complete in a couple minutes max.

finally.txt
@leewyang thank you for all the time.
I export LD_LIBRARY=usr/java/jdk/jre/amd64/server (And I put the libhdfs.so in the path.)
and the whloe log I have already copyed in a txt.
Everything is difficult at first
Thank you！

Glad you got it working!


Center quotes & source in same line

So quotes can be really long and short, so it would be nice to have the option to center the text, vertically and horizontally.
For me I would maybe do this as the default behavior.
Also it would also be nice then to have the option that the source is in the same line:
[Quote] - [Source]

Imagine something like that without the background and the text in the center of the image.

Hi - I've released 4.3.0 (it might take a day or so for it to appear in Google Play / F-Droid): it contains a center option for the Quotation & Source lines.
I wasn't able to combine the Quotation & Source into a single line. As far as I can tell it's not possible to separate a "tap" (two are required: one for the Configuration screen + one for any possible Wikipedia link) between different areas on a block of text that all reside in the same line.
Anyhow, thanks for raising this issue, I think it improves things.
I'll close this issue, but if you spot any related defect please raise a new issue :-)

Thank you, it looks definitely better for me now :heart:
The only thing that I would change in addition is to also be able to center the text vertically.
So you don't have so much space between the text and the toolbar. :thinking:

A other idea would be to put the author in the next line but I can create seperate issue for that. :)


704 | BinarySearch| c++

BinarySearch -
**added the main class, header file and the given vector will be sorted before search because in binary search its important that the given elements are in sorted order ** -
Type of Issue -
Please add/delete options that are not relevant.

[x] Adding New Code
[x] Improving Code

Programming Language
Please add/delete options that are not relevant.

[x] C++

Please assign this to me! I can do this!

Could you please assign this to me?


Is 'instanceof' allowed for both projects?

Description
For both projects (Discrete Event Simulator and Infinite List), is the instanceof keyword allowed?
i.e. Will CodeCrunch accept instanceof?

From my experience, instanceof is not allowed for Project(shop one).

Nope i don't think so, but you can have a method such as:
String getType() {
    return "Serve Event";
}

in your "child" classes. Then to check the type of class in the main class, do something like this:
if (class.getType().equals("Serve Event") {
   // Do this
}

Not recommended but it is a way to get around.

Nope i don't think so, but you can have a method such as:
String getType() {
    return "Serve Event";
}

in your "child" classes. Then to check the type of class in the main class, do something like this:
if (class.getType().equals("Serve Event") {
   // Do this
}

Not recommended but it is a way to get around.

Why not let the event have a boolean method to check if 2 strings are same (input string and attribute string)

I think you can use instanceof to check if the class type is the same as the current class type. For example, in the Logger lab, if (value instanceof Logger) { throw new IllegalArgumentException("already a Logger");
But you cannot check other class types, like checking if (event instanceof Arrive) in Simulator.java

Nope instanceof can not be used. What u can do is to create methods like isSelfCheckServer(), isWait(), isServe() etc in your code to get pass of it

nope instanceof, null and Optional .get()/isPresent()/isEmpty() are all nt allowed

I think instanceof is allowed in the mini-project. (update me if I am wrong because I tried this some time ago)
all java reflections, including getName(), getClass(), instanceof,  are not allowed in the project.

I assume there's a better way, but here's my take with regards to event type checking:
Assign each concrete event type a static 'priority' or 'identifier' integer i.e. Arrival events could be 1, Wait events could be 2, etc.
I have found two uses for doing so:
The first is that you can use a switch block to determine which event type is currently being evaluated, and accordingly deal with each event type in the cases.
The second is that if, for any reason, you need to order two events that occur at the same time, this integer score could provide an additional tiebreaker to determine which should go first.
Again, probably better ways to implement this, but just my two cents.


PinJoint2D: fix inertia tensor, taking center of mass into account

The same inertia tensor calculation is actually repeated in the k_tensor function in the same file, used by the GrooveJoint2D. In that function, the center of mass of each body was already taken into account:
https://github.com/godotengine/godot/blob/d3f983262fbc272c66c20d03f82b6dc658e37f4b/servers/physics_2d/godot_joints_2d.cpp#L239-L240
This PR takes the center of mass into account in the inertia tensor calculation for PinJoint2D in exactly the same way.
Fixes https://github.com/godotengine/godot/issues/58802.
The two calculations of the inertia tensor could be refactored into one function in the future.

It might be worth checking whether this issue occurs in 3D as well (with GodotPhysics).

It might be worth checking whether this issue occurs in 3D as well (with GodotPhysics).

It's probably not an issue there:
https://github.com/godotengine/godot/blob/d3f983262fbc272c66c20d03f82b6dc658e37f4b/servers/physics_3d/joints/godot_pin_joint_3d.cpp#L71-L72

Thanks!

Cherry-picked for 3.5.


Keyboard navigation

Is your feature request related to a problem? Please describe.
Currently there is no way to navigate in the tree using the keyboard.
Describe the solution you'd like
Allow navigating in the tree using the keyboard similar to the explorer panel . Up / down moves the focus from file to file. Left / right also opens inner folders
Additional context
I believe that if you'd use the built-in TreeViewProvider you'd get that out of the box as long as more vscode natively experience :)
Thanks

@moshfeu  keyboard navigation support is added to this fork. Check it out if you are interested.
marketplace link - https://marketplace.visualstudio.com/items?itemName=HabeebArul.react-component-tree
fork link - https://github.com/hb1998/react-component-tree


Date selection issue

For Twitter, the last working month is June 2023. If I select a previous month (say May) and then try to select June again, it does not work.

I fixed the bug and updated to the live version. Thanks!


Enforce StrType.

Parse methods should always be given a string, so the second part is redundant.

pushed a delta with just the norm() check and a test for it


Add more SPO attributes to service review payload to AirTable

Service reviews (smiley faces) received from the app are stored in AirTable, from where we want to pass the reviews on to the relevant SPO to give them feedback on their performance from their users.
Currently this process is hampered by the fact that the data stored in AirTable contains only the SPO id. This means we have to look up the service record to find additional needed information.
Expected behaviour is that the AirTable records created when a review is submitted also contains the service organization name, and contact email address so that is becomes easier to pass on the information to somebody at the organization.
Acceptance Criteria

[ ]  AirTable records of service reviews contain service provider organization name (PublicName)
[ ]  AirTable records of service reviews contain contact email address (MainContactEmailAddress)
[ ] Stretch goal: Also the name of the person the contact email address belongs to (MainContactName)

instead of slack, I want a gmail message that has an attachment with a record

i am waiting for someone to give feedback in the app
to QA this issue


Repackage as cordova-plugin-secure-storage-echo

Thanks for repackaging and for your work to keep this plugin alive! In a future change would you mind adding the 5.0.0 release to the changelog?
Thanks again!


About face traking code

Hi, thanks for your work.
I don't understand how the proj_pts method handles data.
line 60 in data_uitls/face_tracker/utils.py
def proj_pts(rott_geo, focal_length, cxy):
    cx, cy = cxy[0], cxy[1]
    X = rott_geo[:, :, 0]
    Y = rott_geo[:, :, 1]
    Z = rott_geo[:, :, 2]
    fxX = focal_length*X
    fyY = focal_length*Y
    proj_x = -fxX/Z + cx
    proj_y = fyY/Z + cy
    return torch.cat((proj_x[:, :, None], proj_y[:, :, None], Z[:, :, None]), 2)

Are there any references to repo for this part?

Hi, thanks for your work. I don't understand how the proj_pts method handles data. line 60 in data_uitls/face_tracker/utils.py
def proj_pts(rott_geo, focal_length, cxy):
    cx, cy = cxy[0], cxy[1]
    X = rott_geo[:, :, 0]
    Y = rott_geo[:, :, 1]
    Z = rott_geo[:, :, 2]
    fxX = focal_length*X
    fyY = focal_length*Y
    proj_x = -fxX/Z + cx
    proj_y = fyY/Z + cy
    return torch.cat((proj_x[:, :, None], proj_y[:, :, None], Z[:, :, None]), 2)

Are there any references to repo for this part?

I got it，proj_pts method converts camera coordinate system to pixel coordinate system. reference https://blog.csdn.net/level_code/article/details/134437269


Use same elixir version for all apps

Elixir versions are not the same through all the apps, we'll stick to 1.4 cc @carlo-ev

Closed via #12


Android screenshots are out of date and Windows Phone screenshots need to be removed/replaced w/UWP

The latest control templates and stylings of stock android 8/9 look very different from the screenshots here - it might be time to update them (note: I'm not sure if the iOS screenshots are similarly out of date). Also the Windows Phone screenshots need to be removed and preferably replaced with UWP versions.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7ad16e14-5abd-7115-a64a-69db1178d58a
Version Independent ID: 1602a6ea-3968-006b-9d63-63c25051b5b4
Content: Xamarin.Forms TableView - Xamarin
Content Source: docs/xamarin-forms/user-interface/tableview.md
Product: xamarin
GitHub Login: @davidbritch
Microsoft Alias: dabritch

Hi @JeroMiya
Agreed. The screenshots will gradually be updated, but it's a big task.


Is it possible to specify the primary language of a project?

I'm getting a lot of stars for HTML, presumably because of the large amount of HTML documentation files in my most active repo, even though it's a javascript project. (link)

Unfortunately no, this is discussed in the Github FAQ : https://help.github.com/articles/my-repository-is-marked-as-the-wrong-language/
Have a look at this issue for more details : https://github.com/vdaubry/github-awards/issues/6
Hope that helps, I'm closing this issue since it's a Github related problem.


Q: Handling large number of pages?

I have a case where the number of records is relatively large, so it produces e.g. 50+ pages. This cases a bit of an issue with UI since those pages don't fit the page nicely (break into multiple rows etc.).
For kaminari for example, this is configurable (even has default behavior) via various window related params. So, is there a way to achieve the same for scrivener?
If not, would there be interest in adding this to scrivener or do you prefer to leave it to the consumers?
Thank you for your time 😄
P.S. If there'd be interest in adopting this for scrivener, I have to solve this for a project, might ask the owner to devote some time to add this to scrivener so I might help.

@elvanja scrivener remains agnostic about how you choose to display the page information in a UI. If you're interested in a project focused on that, please see scrivener_list.

Excellent. Thank you @drewolson!

Just a small update, there is the https://github.com/mgwidmann/scrivener_html project that does exactly this.