vFile volume driver is not fully initialized yet.

Hi,
I've installed vDVS as described in the documentation. This is working correctly.
docker volume create --driver=vsphere --name=SharedVol -o size=10gb

docker volume ls
DRIVER              VOLUME NAME
vsphere:latest      SharedVol@MSA_DS1

When I try to create a volume with vFile instead of vSphere I get an error:
docker volume create --driver=vfile --name=SharedVol1 -o size=10gb
Error response from daemon: create SharedVol1: VolumeDriver.Create: vFile volume driver is not fully initialized yet.

Both plugins are installed and enabled:
docker plugin ls
ID                  NAME                DESCRIPTION                           ENABLED
821d4601d65a        vsphere:latest      VMWare vSphere Docker Volume plugin   true
7aae40b225e5        vfile:latest        VMWare vFile Docker Volume plugin     true

Looks like I forgot to initialize the vFile driver. I've searched the documentation but I don't find anything about this error or how to correctly configure the plugin. Anybody who can help?
Thanks!
Bart

Are your docker in swarm mode ? this is an issue to me, how can i use it without swarm ?

@thiagolealassis
Hi and yes swarm is a necessary requirement for vFile plugin.
Is your environment restricted to a particular container orchestrator?

@bartlenaerts
Hi Bart,
Thanks for reporting this issue.
As suggested by @thiagolealassis , could you please verify if your docker is in swarm mode?
If yes, can you paste the log at /var/log/vfile.log?

@luomiao
Hi,
thanks for looking in to this.
I'm running in swarm mode. We have a setup of 3 managers and 6 workers. All hosts have the vSphere and vFile plugins installed.
manager1
2017-10-30 15:49:43.46669919 +0000 UTC [INFO] Starting plugin driver=vfile log_level=info config="/etc/vfile.conf"
2017-10-30 15:49:43.468224536 +0000 UTC [INFO] Getting volume data from unix:///var/run/docker.sock
2017-10-30 15:49:45.468959877 +0000 UTC [INFO] Can't connect to unix:///var/run/docker.sock due to (An error occurred trying to connect: context deadline exceeded), skipping discovery
2017-10-30 15:49:45.469025026 +0000 UTC [INFO] Refcounting failed: (An error occurred trying to connect: context deadline exceeded).
2017-10-30 15:49:45.469078075 +0000 UTC [INFO] vFile plugin started version="vFile Volume Driver v0.2"
2017-10-30 15:49:45.469342679 +0000 UTC [INFO] Scheduling again after 2 seconds
2017-10-30 15:49:45.469470521 +0000 UTC [INFO] Started loading file server image
2017-10-30 15:49:45.470365338 +0000 UTC [INFO] Going into ServeUnix - Listening on Unix socket address="/run/docker/plugins/vfile.sock"
2017-10-30 15:49:47.469658453 +0000 UTC [INFO] Getting volume data from unix:///var/run/docker.sock
2017-10-30 15:49:49.470912421 +0000 UTC [INFO] Can't connect to unix:///var/run/docker.sock due to (An error occurred trying to connect: context deadline exceeded), skipping discovery
2017-10-30 15:49:49.471002649 +0000 UTC [INFO] Refcounting failed: (An error occurred trying to connect: context deadline exceeded). Attempts left: 20
2017-10-30 15:49:49.471024175 +0000 UTC [INFO] Scheduling again after 4 seconds
2017-10-30 15:49:53.472447961 +0000 UTC [INFO] Getting volume data from unix:///var/run/docker.sock
2017-10-30 15:49:53.567183475 +0000 UTC [INFO] Found 0 running or paused containers
2017-10-30 15:49:53.569472649 +0000 UTC [INFO] Discovered 0 volumes that may be in use.
2017-10-30 15:49:53.569532447 +0000 UTC [INFO] Refcounting successfully completed
2017-10-30 15:53:44.06466798 +0000 UTC [INFO] VolumeDriver Get: 83b6a0bcae897689e588998ba7cd25c86a698c0069b7efa8298efa874cd9ef42
2017-10-30 15:54:11.935853374 +0000 UTC [INFO] VolumeDriver Get: ee23d46de20bf7030d702207f2563f5b9c806867914c5751b0e605684046e1b5
2017-10-30 15:58:12.629964145 +0000 UTC [INFO] VolumeDriver Get: e7c3c19045d0a71a0695ae42f227e84ac1b1fcaba6da18aabfbdc5986c72eca6
2017-10-31 09:06:24.626061738 +0000 UTC [INFO] VolumeDriver Get: ed48485793f1f5621dbff92463b99ea5ad822c26fafbf205f3ff173bf3157e4d

manager2
2017-10-31 09:00:16.15312973 +0000 UTC [INFO] VolumeDriver Get: c6b9821c5e02c00de7885427fa5593cff5f94fcad41f80a4aaf8a4dd0c00630b
2017-10-31 09:00:43.88508694 +0000 UTC [INFO] VolumeDriver Get: 2583f6f93875845030f20b1a6fe2612e5800352d4d736ff5abef8cdb65d06678
2017-10-31 09:01:45.457576671 +0000 UTC [INFO] VolumeDriver Get: ae0c26458fcfdc4de3ed18f73f39fc3c293b347ea5d2b9e24dab14b18a34f306
2017-10-31 09:02:47.892570486 +0000 UTC [INFO] VolumeDriver Get: 1b0a3b97349b9dcfaf028e32cab8da2e27897c7a09f7d60dee2803492210236d
2017-10-31 09:03:48.60524559 +0000 UTC [INFO] VolumeDriver Get: 2caea1b5c8c31874e8d8826d6905fd2494f4b729e12bae662abe618be05da27c
2017-10-31 09:05:20.956716195 +0000 UTC [INFO] VolumeDriver Get: 41f10a305a34d2ba55e97ae2ae36f8860135bf1da28baca9702589529dab219e
2017-10-31 09:05:48.397540093 +0000 UTC [INFO] VolumeDriver Get: 88dde33159dd7ea6eb978fea03a03cbaae41583934e74fd3838d547f8a70d5b1
2017-10-31 09:06:15.854202428 +0000 UTC [INFO] VolumeDriver Get: bd8a08dfa8014cbb0b51a1d238d29d03f0251b33344677f97cc04c25f3476f7e

manager3
2017-10-31 09:11:54.329242199 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 09:11:54.329416049 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 09:11:59.581617309 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:05.624456251 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:10.750564573 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:16.217355943 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:23.70115703 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:28.826429604 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 09:12:28.826588979 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 09:12:39.142116964 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:44.275405478 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:49.725257798 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:12:54.97203895 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 09:12:54.972169615 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 09:13:03.851710096 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 09:13:03.851870749 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 09:13:09.537374433 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 09:13:14.752017914 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 09:13:14.752153911 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 09:13:20.134451492 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 09:13:20.134620496 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"

At the moment, manager3 is the Leader of the swarm.
Regards,
Bart

Hi @bartlenaerts
Thanks for posting the logs.
Can you post a little bit more for the beginning part of the log on manager3? I'd like to check when the ETCD is going wrong.
Also, from the log of manager1, it seems docker service just got restarted. Has the node been restarted after the plugin installed?
We are currently working on the beta release of vFile which will have more flexibility about the stability and install order of the plugin. But for now, you may want strictly follow the steps below:

Please make sure docker version is larger than 17.06
Please make sure ETCD ports (2379, 2380) are opened on your manager VMs
Install vsphere plugins first on all the nodes, before install the vFile plugins
Disable and remove all previous existing vFile plugins from your VMs
Install vFile plugin on swarm leader node first and on all the other nodes (this should not be a requirement but let's follow this one to debug)
After the vFile plugin installation on each node, please check the log to find the following line:

2017-10-25 16:54:24.987122121 -0700 PDT [INFO] Going into ServeUnix - Listening on Unix socket address="/run/docker/plugins/vfile.sock"

Hi @luomiao,
I followed your steps and now it works! So the steps I did were:

Checked Docker version: 17.09
Opened ETCD ports on all 3 managers.
Removed the vSphere and vFile plugins on all nodes.
Restarted Docker on all nodes (just to be sure).
Installed vSphere on all nodes first.
Installed vFile on leader manager only.
Checked the log (see below).
Installed vFile on all other nodes.
I was then able to create a vFile volume. docker volume create --driver=vfile --name=SharedVol -o size=10gb

vfile.log
2017-10-31 10:02:58.34807531 +0000 UTC [INFO] Starting plugin driver=vfile log_level=info config="/etc/vfile.conf"
2017-10-31 10:02:58.348457389 +0000 UTC [INFO] Getting volume data from unix:///var/run/docker.sock
2017-10-31 10:02:58.381721215 +0000 UTC [INFO] Found 0 running or paused containers
2017-10-31 10:02:58.382469762 +0000 UTC [INFO] Discovered 0 volumes that may be in use.
2017-10-31 10:02:58.382499895 +0000 UTC [INFO] Refcounting successfully completed
2017-10-31 10:02:58.382542227 +0000 UTC [INFO] vFile plugin started version="vFile Volume Driver v0.2"
2017-10-31 10:02:58.382620002 +0000 UTC [INFO] Started loading file server image
2017-10-31 10:02:58.384038204 +0000 UTC [INFO] Going into ServeUnix - Listening on Unix socket address="/run/docker/plugins/vfile.sock"
2017-10-31 10:02:58.41946116 +0000 UTC [INFO] Swarm node role: leader, start ETCD cluster nodeID=i5c6sievyy4rggvpzqecsmu7c
2017-10-31 10:02:59.420076507 +0000 UTC [INFO] Checking ETCD client is started
2017-10-31 10:03:09.452428503 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:14.901041414 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:19.971432827 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 10:03:19.971590919 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:03:25.005591279 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:30.079088397 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 10:03:30.07923584 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:03:35.110973839 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:40.143142735 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:45.531294864 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:50.581942072 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:03:55.975909884 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:01.008991757 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:06.04353414 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 10:04:06.043698527 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:04:11.148425895 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix prefix="SVOLS_stat_" error="context deadline exceeded"
2017-10-31 10:04:11.148575098 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:04:16.182878652 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:21.217242235 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:26.252000231 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 10:04:26.252117244 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:04:31.291695968 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:36.328717021 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:41.366399264 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 10:04:41.366510315 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:04:46.400956082 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:51.827967533 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:04:56.868578217 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:05:02.26486748 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:05:07.303951319 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:05:12.340111352 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix error="context deadline exceeded" prefix="SVOLS_stat_"
2017-10-31 10:05:12.34027219 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:05:17.429504872 +0000 UTC [WARNING] Failed to get volume states from ETCD due to error context deadline exceeded.
2017-10-31 10:05:22.467149029 +0000 UTC [ERROR] Failed to call ETCD Get for listing all keys with prefix prefix="SVOLS_stat_" error="context deadline exceeded"
2017-10-31 10:05:22.467267836 +0000 UTC [ERROR] Failed to get volume list error="context deadline exceeded"
2017-10-31 10:22:04.283225357 +0000 UTC [INFO] VolumeDriver Get: SharedVol
2017-10-31 10:22:04.318059744 +0000 UTC [INFO] Volume not found: SharedVol
2017-10-31 10:22:04.318226541 +0000 UTC [ERROR] Failed to get volume meta-data name=SharedVol error="No such volume"
2017-10-31 10:22:04.319159223 +0000 UTC [INFO] VolumeDriver Create: SharedVol
2017-10-31 10:22:04.319339815 +0000 UTC [INFO] Attempting to write initial metadata entry for SharedVol
2017-10-31 10:22:04.354541407 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-10-31 10:22:04.3557077 +0000 UTC [INFO] Attempting to create internal volume for SharedVol
2017-10-31 10:22:10.707840759 +0000 UTC [INFO] Attempting to update volume state to ready for volume: SharedVol
2017-10-31 10:22:10.744852291 +0000 UTC [INFO] Successfully created volume: SharedVol

The problem could be one of two:

The ETCD ports were not added to firewalld on the managers.
Because the hosts were created with Terraforms, the vSphere and vFile plugins were automatically installed on the host, before other hosts were available. So the Leader had already those 2 plugins, without there being already extra managers and workers.

Thanks for your help!
Regards,
Bart

@bartlenaerts
Great to know it works!
The first one might be the reason. We will update the user guide to highlight this requirement.
The second one should not be a problem though. And from the log on leader, it seems the leader might not be able to talk to itself's ETCD port? It may be able to talk to other managers though.

@luomiao swarm is an restriction for my company, are you planning to delivery an orchestration technology independent version ?

@thiagolealassis
We don't have an orchestrator independent plan but we are planning a vFile plugin/provider for Kubernetes in Q1 2018.

Hi @luomiao,
not still there yet. Sorry...
I can create a vFile volume but I still have problems mounting it to a Docker container:
VolumeDriver.Mount: Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete.

Any ideas?
Regards,
Bart

@bartlenaerts
There is a recently change in the SMB server which is utilized by vFile, which increased the starting time of a vFile volume to around 1 minute. This could result in the timeout error you saw...
Would you please try to increase the VFILE_TIMEOUT_IN_SECOND when you install the plugin:
docker plugin install --grant-all-permissions --alias vfile vmware/vfile:latest VFILE_TIMEOUT_IN_SECOND=90
We are currently working on the solve this long start time: https://github.com/vmware/docker-volume-vsphere/issues/1954

Hi @luomiao
sorry to keep bothering you, but I've got some new error now...
I created the vFile volume and mounted it to my container. I see a vFileServerassets container running. But it comes and goes and changes continuously from ContainerID. Therefore the referencing container can't start and gives these errors:
"Post http://%2Frun%2Fdocker%2Fplugins%2F24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8%2Fvfile.sock/VolumeDriver.Mount: dial unix /run/docker/plugins/24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8/vfile.sock: connect: no such file or directory"

"task: non-zero exit (1)"

The first error probably because the vFileServer changed it's container ID, the second error while the vFileServer is not available at all.
vfile.log
2017-11-02 13:35:51.92403881 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-02 13:35:52.031943042 +0000 UTC [INFO] Volume not found: jobwalkr_assets
2017-11-02 13:35:52.031986049 +0000 UTC [ERROR] Failed to get volume meta-data name="jobwalkr_assets" error="No such volume"
2017-11-02 13:35:52.036334682 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-02 13:35:52.093771792 +0000 UTC [INFO] Volume not found: jobwalkr_assets
2017-11-02 13:35:52.093815283 +0000 UTC [ERROR] Failed to get volume meta-data name="jobwalkr_assets" error="No such volume"
2017-11-02 13:35:52.094233375 +0000 UTC [INFO] VolumeDriver Create: jobwalkr_assets
2017-11-02 13:35:52.094356322 +0000 UTC [INFO] Attempting to write initial metadata entry for jobwalkr_assets
2017-11-02 13:35:52.145466086 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:35:52.145727574 +0000 UTC [INFO] Attempting to create internal volume for jobwalkr_assets
2017-11-02 13:36:18.400656062 +0000 UTC [INFO] Attempting to update volume state to ready for volume: jobwalkr_assets
2017-11-02 13:36:18.470265516 +0000 UTC [INFO] Successfully created volume: jobwalkr_assets
2017-11-02 13:37:42.50291006 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:37:43.503513485 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:37:43.592737218 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:38:59.099365506 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:39:00.102141058 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:39:01.099632248 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:39:02.099705788 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:39:03.099571491 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:39:04.100088114 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:39:04.168884487 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:39:04.16893153 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:39:05.169707581 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:39:05.293969382 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:39:17.59621115 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-02 13:39:17.807093978 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-02 13:39:17.808360544 +0000 UTC [INFO] Directory doesn't exist, creating it path="/mnt/vfile/jobwalkr_assets/"
2017-11-02 13:39:17.808515762 +0000 UTC [INFO] Before AtomicIncr
2017-11-02 13:39:19.025947836 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:39:19.114016528 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-02 13:40:14.163994607 +0000 UTC [INFO] Add VM[<IP_ADDRESS>] with ID[i5c6sievyy4rggvpzqecsmu7c] to ClientList for volume[jobwalkr_assets]
2017-11-02 13:40:14.228384113 +0000 UTC [INFO] Volume state mounted, prepare to mounting locally
2017-11-02 13:40:14.228451221 +0000 UTC [INFO] Get info for mounting name="jobwalkr_assets" Port=30000 ServiceName="vFileServerjobwalkr_assets"
2017-11-02 13:40:14.228483431 +0000 UTC [INFO] Mounting volume with options volume name="jobwalkr_assets" arguments=[-t cifs -o username=root,password=badpass,port=30000,vers=3.0 //<IP_ADDRESS>/share1 /mnt/vfile/jobwalkr_assets/]
2017-11-02 13:40:14.228517356 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-02 13:40:15.458150885 +0000 UTC [INFO] Unmounting Volume name="jobwalkr_assets"
2017-11-02 13:40:15.465774951 +0000 UTC [INFO] Before AtomicDecr
2017-11-02 13:40:16.658644751 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:40:16.738970698 +0000 UTC [INFO] Remove VM[<IP_ADDRESS>] with ID[i5c6sievyy4rggvpzqecsmu7c] from ClientList for volume[jobwalkr_assets]
2017-11-02 13:40:17.007109858 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-02 13:40:17.008335355 +0000 UTC [INFO] Before AtomicIncr
2017-11-02 13:40:18.140162498 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:40:18.199360455 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-02 13:40:19.252764519 +0000 UTC [INFO] Add VM[<IP_ADDRESS>] with ID[i5c6sievyy4rggvpzqecsmu7c] to ClientList for volume[jobwalkr_assets]
2017-11-02 13:40:19.331676249 +0000 UTC [INFO] Volume state mounted, prepare to mounting locally
2017-11-02 13:40:19.331735085 +0000 UTC [INFO] Get info for mounting name="jobwalkr_assets" Port=30000 ServiceName="vFileServerjobwalkr_assets"
2017-11-02 13:40:19.331763975 +0000 UTC [INFO] Mounting volume with options volume name="jobwalkr_assets" arguments=[-t cifs -o username=root,password=badpass,port=30000,vers=3.0 //<IP_ADDRESS>/share1 /mnt/vfile/jobwalkr_assets/]
2017-11-02 13:40:19.331802896 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-02 13:40:20.021045254 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:40:24.789026722 +0000 UTC [INFO] Unmounting Volume name="jobwalkr_assets"
2017-11-02 13:40:24.811283565 +0000 UTC [INFO] Before AtomicDecr
2017-11-02 13:40:26.075377759 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:40:26.15533851 +0000 UTC [INFO] Remove VM[<IP_ADDRESS>] with ID[i5c6sievyy4rggvpzqecsmu7c] from ClientList for volume[jobwalkr_assets]
2017-11-02 13:40:27.075666681 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:40:28.07568919 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:40:29.075833466 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:40:30.075741729 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:40:31.075648386 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:40:31.145977948 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:41:07.123652691 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:41:08.12391323 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:41:08.339085714 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:41:18.921287077 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:42:20.43542045 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:42:20.72238295 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:42:21.722691798 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:42:22.722697724 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:42:23.722805505 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:42:24.722806464 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:42:25.722770251 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:42:25.822887719 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:42:25.822963757 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:42:26.823220615 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:42:26.920156237 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:42:26.920252071 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:43:41.450977382 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:43:43.176891405 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:43:44.177813775 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:43:45.177211784 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:43:46.177277774 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:43:47.177215746 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:43:48.177288179 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:43:48.267026903 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:43:48.267096818 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:43:49.26734785 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:43:49.358526825 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:45:00.17391791 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:45:01.174255888 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:45:02.174297397 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:45:03.174409292 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:45:04.174746174 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:45:05.174284417 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:45:05.264105386 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:45:05.264175806 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:45:06.264463974 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:45:06.424592 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:45:10.788458674 +0000 UTC [INFO] VolumeDriver Get: 8c0991d45fcb7901dd012529f0cfa3e132195a80643197ca29dfac30fde40231
2017-11-02 13:45:10.83618306 +0000 UTC [INFO] Volume not found: 8c0991d45fcb7901dd012529f0cfa3e132195a80643197ca29dfac30fde40231
2017-11-02 13:45:10.836231719 +0000 UTC [ERROR] Failed to get volume meta-data name=8c0991d45fcb7901dd012529f0cfa3e132195a80643197ca29dfac30fde40231 error="No such volume"
2017-11-02 13:45:53.789808896 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:46:21.267028877 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:46:22.763232793 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:46:24.618890025 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:46:30.30886352 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:46:31.309104588 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:46:32.309188834 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:46:33.309231832 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:46:34.309162756 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:46:35.309209949 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:46:35.397882227 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:46:35.397952838 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:46:36.399738385 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:46:36.507795265 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:46:41.700434194 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:47:48.100001694 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:47:48.426759456 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:47:49.427097766 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:47:50.427111969 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:47:51.427104071 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:47:52.427167677 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:47:53.4280003 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:47:53.487290112 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:47:53.48733546 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:47:54.487671041 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:47:54.630392502 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:47:54.630519018 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:49:06.247108545 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:49:08.032145704 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:49:09.032471138 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:49:10.03245913 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:49:11.032728282 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:49:12.032522635 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:49:13.032516805 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-02 13:49:13.137063916 +0000 UTC [INFO] Volume not in proper state for the operation: Ready
2017-11-02 13:49:13.137161852 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:49:14.137418039 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-02 13:49:14.226225668 +0000 UTC [INFO] Volume not in proper state for the operation: Mounting
2017-11-02 13:49:22.188268228 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-02 13:49:22.366973981 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-02 13:49:22.370727446 +0000 UTC [INFO] Before AtomicIncr
2017-11-02 13:49:23.550579102 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-02 13:49:23.630558523 +0000 UTC [INFO] Service start timeoutvalue=90

Any help would be greatly appreciated!
Regards,
Bart

@bartlenaerts
vFile is still an experimental feature and we are working to make it stable, so you're very welcome to submit those problems you have met during the deployment! So we can solve them for the beta release :)
Regarding to this error, I think the order might be different from what you thought about.
From the log, it looks like the reason why vFile server keeps changing its container ID, is because the user container somehow keeps restarting. vFile plugin is able to mount/umount successfully. However user container hit an error with the returned mount point between the following two log messages:
2017-11-02 13:40:14.228483431 +0000 UTC [INFO] Mounting volume with options volume name="jobwalkr_assets" arguments=[-t cifs -o username=root,password=badpass,port=30000,vers=3.0 //<IP_ADDRESS>/share1 /mnt/vfile/jobwalkr_assets/]
2017-11-02 13:40:14.228517356 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-02 13:40:15.458150885 +0000 UTC [INFO] Unmounting Volume name="jobwalkr_assets"

And due to the unmount, vFile plugin needs to stop its server, which results in the change of container ID that you have observed.
And the error why user container keeps restarting might be the one you posted:
"Post http://%2Frun%2Fdocker%2Fplugins%2F24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8%2Fvfile.sock/VolumeDriver.Mount: dial unix /run/docker/plugins/24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8/vfile.sock: connect: no such file or directory"
May I ask which docker version are you using? And also how did you start your container? Are you starting it by docker service, or docker run?
Another question is, are you using any special permission for your container, i.e, some user permission other than root?
Thanks!

I did some tests with other docker plugins, and found out this is an issue not only for us, but for all the plugins if they are installed by "docker plugin install" way.
That is, if the plugin is re-installed after a volume is created with this plugin, this volume won't be able to function correctly. This volume is bound to the specific plugin that was removed.
@bartlenaerts would you please check when the error happens, does the ID inside the /run/docker/plugins/[ID] error message is matching with the plugin ID you saw through docker plugin ls command?
More important, can you verify if the vFile plugin got re-installed after the volume is created?

@luomiao
i followed your suggest added timeout to 90s. still got same error:
Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete.
below is my test lab environment:
ESXi 6.0.0 build 2494585
Docker 17.06-CE

2017-11-06 09:41:27.244955838 +0000 UTC [INFO] VolumeDriver Get: SharedVol
2017-11-06 09:41:27.569263652 +0000 UTC [INFO] Mounting volume name=SharedVol
2017-11-06 09:41:27.570079456 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 09:41:28.922490942 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-06 09:41:29.105063978 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 09:41:29.922852498 +0000 UTC [INFO] Attempting to change volume state to Mounting
2017-11-06 09:41:30.133822207 +0000 UTC [INFO] Volume not in proper state for the operation: Error
2017-11-06 09:43:10.370235779 +0000 UTC [ERROR] error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete." name=SharedVol
2017-11-06 09:43:10.370442619 +0000 UTC [ERROR] Failed to mount error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete." name=SharedVol
2017-11-06 09:43:10.37044535 +0000 UTC [INFO] Watcher on global refcount returns event type=PUT
2017-11-06 09:43:10.370503621 +0000 UTC [INFO] Detaching SharedVol - it is not used anymore
2017-11-06 09:43:11.370916302 +0000 UTC [INFO] Attempting to change volume state to Unmounting
2017-11-06 09:43:11.612027185 +0000 UTC [INFO] Volume not in proper state for the operation: Error

Hi @luomiao,
because of the weekend, I left it running/trying for the last 4 days, and now it is up and running by itself. Quite strange, I don't know why. Patient is really a virtue.
But to answer your questions:

I installed the service with docker service.
The output of docker plugin ls and the /run/docker/plugins folder is:

drwx------. 2 root root 60 Nov  1 13:20 195cedec9fbc6aac7b80b5453a8db62de26e3d76ee71ef127396556ae9fe97bc
drwx------. 2 root root 60 Nov  2 09:59 327267222b5332e5ca49eff2ee0d8d06949cb3220aee5b9c8d7cd7607d2e7f1e

ID                  NAME                DESCRIPTION                           ENABLED
327267222b53        vfile:latest        VMWare vFile Docker Volume plugin     true
195cedec9fbc        vsphere:latest      VMWare vSphere Docker Volume plugin   true

Looks like both ID's matches. I've checked all 9 workers and managers, and they all have the same ID's. So nothing strange there.

I'm sure that the plugin wasn't re-installed after the volume was created.

To sum up, it is working now, but looks like 10 hours ago their still was an error:
[root@manager1 plugins]# docker service ps k4w --no-trunc
ID                          NAME                               IMAGE               NODE                    DESIRED STATE       CURRENT STATE           ERROR                                                                                PORTS
mjw4593oejsiifqk3jm2vykva   vFileServerjobwalkr_assets.1       dperson/samba       worker6.jobwalkr.com    Running             Running 10 hours ago
fh3hompok1jrgnqkk4g114fyf    \_ vFileServerjobwalkr_assets.1   dperson/samba       worker6.jobwalkr.com    Shutdown            Failed 10 hours ago     "starting container failed: error while mounting volume '': VolumeDriver.Mount: Failed to add disk 'scsi0:1'. disk /vmfs/volumes/VSA_DS1/dockvols/_DEFAULT/_vF_jobwalkr_assets.vmdk already attached to VM=5005cd4c-cf85-1dee-8b3b-f8f05268dc7c"
5qjdva8p09zk97tdc8h5jdrkn    \_ vFileServerjobwalkr_assets.1   dperson/samba       worker5.jobwalkr.com    Shutdown            Failed 10 hours ago     "starting container failed: error while mounting volume '/mnt/vmdk/_vF_jobwalkr_assets/': VolumeDriver.Mount: Failed to add disk 'scsi0:1'. disk /vmfs/volumes/VSA_DS1/dockvols/_DEFAULT/_vF_jobwalkr_assets.vmdk already attached to VM=5005cd4c-cf85-1dee-8b3b-f8f05268dc7c"
fwiy4ly630iz6zae1z8c30r9q    \_ vFileServerjobwalkr_assets.1   dperson/samba       manager2.jobwalkr.com   Shutdown            Shutdown 10 hours ago

[root@manager1 plugins]# docker service ps 7qp --no-trunc
ID                          NAME                    IMAGE                                                                                NODE                    DESIRED STATE       CURRENT STATE           ERROR                                                                                PORTS
p4gi0o2flhmp1oqfq6o72lslm   jobwalkr_assets.1       registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   worker2.jobwalkr.com    Running             Running 10 hours ago
znllkdlcbbti3gypza9qozw7f    \_ jobwalkr_assets.1   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   manager3.jobwalkr.com   Shutdown            Rejected 11 hours ago   "Post http://%2Frun%2Fdocker%2Fplugins%2F24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8%2Fvfile.sock/VolumeDriver.Mount: dial unix /run/docker/plugins/24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8/vfile.sock: connect: no such file or directory"
zzzj874dkegyb4ivvt6d6icgq    \_ jobwalkr_assets.1   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   worker3.jobwalkr.com    Shutdown            Rejected 2 days ago     "VolumeDriver.Mount: Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
zz30oqswdppow73ki4ptlb5sj    \_ jobwalkr_assets.1   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   worker3.jobwalkr.com    Shutdown            Rejected 2 days ago     "VolumeDriver.Mount: Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
zysowrqx7mzo44pypikgzpqut    \_ jobwalkr_assets.1   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   manager3.jobwalkr.com   Shutdown            Rejected 3 days ago     "Post http://%2Frun%2Fdocker%2Fplugins%2F24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8%2Fvfile.sock/VolumeDriver.Mount: dial unix /run/docker/plugins/24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8/vfile.sock: connect: no such file or directory"
nocvvb9fvm0n1yub3fopoc5cj   jobwalkr_assets.2       registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   worker3.jobwalkr.com    Running             Running 10 hours ago
5aze09yanlqf1nlomoq7t9hh7    \_ jobwalkr_assets.2   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   manager3.jobwalkr.com   Shutdown            Shutdown 10 hours ago
svw8qh4g6lbgmcpc0702p4a4h    \_ jobwalkr_assets.2   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   manager3.jobwalkr.com   Shutdown            Shutdown 10 hours ago
0yj8jilv7e8dqq0gsm4s96bkk    \_ jobwalkr_assets.2   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   manager3.jobwalkr.com   Shutdown            Shutdown 10 hours ago
zzt4b8avuvsx7g8wguxdz30qj    \_ jobwalkr_assets.2   registry.gitlab.com/bamboo3.jobwalkr/bamboo3.jobwalkr.assets:latest@sha256:db2504795b99ba451106d1e03a1853027bd5d708e59af92c4d9f513876069dcf   worker6.jobwalkr.com    Shutdown            Rejected 2 days ago     "VolumeDriver.Mount: Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."

This is the vfile.log on worker2.jobwalkr.com:
2017-11-03 17:25:04.186734565 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-03 17:25:04.698565343 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-03 17:25:04.699166541 +0000 UTC [INFO] Directory doesn't exist, creating it path="/mnt/vfile/jobwalkr_assets/"
2017-11-03 17:25:04.699274603 +0000 UTC [INFO] Before AtomicIncr
2017-11-03 17:25:06.058027556 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-03 17:26:19.196644801 +0000 UTC [INFO] Add VM[<IP_ADDRESS>] with ID[0m4raijxcehioq6yov27fauu7] to ClientList for volume[jobwalkr_assets]
2017-11-03 17:26:19.331361115 +0000 UTC [INFO] Volume state mounted, prepare to mounting locally
2017-11-03 17:26:19.331470891 +0000 UTC [INFO] Get info for mounting name="jobwalkr_assets" Port=30000 ServiceName="vFileServerjobwalkr_assets"
2017-11-03 17:26:19.331506565 +0000 UTC [INFO] Mounting volume with options volume name="jobwalkr_assets" arguments=[-t cifs -o username=root,password=badpass,port=30000,vers=3.0 //<IP_ADDRESS>/share1 /mnt/vfile/jobwalkr_assets/]
2017-11-03 17:26:19.331550771 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-03 17:26:20.651574113 +0000 UTC [INFO] Unmounting Volume name="jobwalkr_assets"
2017-11-03 17:26:20.667908388 +0000 UTC [INFO] Before AtomicDecr
2017-11-03 17:26:22.089856726 +0000 UTC [INFO] Remove VM[<IP_ADDRESS>] with ID[0m4raijxcehioq6yov27fauu7] from ClientList for volume[jobwalkr_assets]
2017-11-03 17:26:22.509861826 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-03 17:26:22.510411346 +0000 UTC [INFO] Before AtomicIncr
2017-11-03 17:26:23.874220336 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-03 17:26:24.9488618 +0000 UTC [INFO] Add VM[<IP_ADDRESS>] with ID[0m4raijxcehioq6yov27fauu7] to ClientList for volume[jobwalkr_assets]
2017-11-03 17:26:25.030445419 +0000 UTC [INFO] Volume state mounted, prepare to mounting locally
2017-11-03 17:26:25.030502655 +0000 UTC [INFO] Get info for mounting name="jobwalkr_assets" Port=30000 ServiceName="vFileServerjobwalkr_assets"
2017-11-03 17:26:25.030545702 +0000 UTC [INFO] Mounting volume with options arguments=[-t cifs -o username=root,password=badpass,port=30000,vers=3.0 //<IP_ADDRESS>/share1 /mnt/vfile/jobwalkr_assets/] volume name="jobwalkr_assets"
2017-11-03 17:26:25.030569989 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-03 17:26:36.673009232 +0000 UTC [INFO] Unmounting Volume name="jobwalkr_assets"
2017-11-03 17:26:36.690330221 +0000 UTC [INFO] Before AtomicDecr
2017-11-03 17:26:38.0675721 +0000 UTC [INFO] Remove VM[<IP_ADDRESS>] with ID[0m4raijxcehioq6yov27fauu7] from ClientList for volume[jobwalkr_assets]
2017-11-06 05:15:17.629489491 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:15:19.869340218 +0000 UTC [WARNING] Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:15:19.869553049 +0000 UTC [WARNING] Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:15:19.8695957 +0000 UTC [ERROR] Failed to get volume meta-data name="jobwalkr_assets" error="Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Swar$
2017-11-06 05:15:20.636203793 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:15:22.86066109 +0000 UTC [WARNING] Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:15:22.860831554 +0000 UTC [WARNING] Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:15:22.860883742 +0000 UTC [ERROR] Failed to get volume meta-data name="jobwalkr_assets" error="Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Sw$

vfile.log worker3.jobwalkr.com:
2017-11-05 15:18:13.717954638 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 04:22:01.103111451 +0000 UTC [INFO] VolumeDriver Get: 8256f80592eb00512ee30c7dff0755e8ac0c42f4b7eb24dc8983a4154f66297e
2017-11-06 04:22:01.534985875 +0000 UTC [INFO] Volume not found: 8256f80592eb00512ee30c7dff0755e8ac0c42f4b7eb24dc8983a4154f66297e
2017-11-06 04:22:01.535043188 +0000 UTC [ERROR] Failed to get volume meta-data name=8256f80592eb00512ee30c7dff0755e8ac0c42f4b7eb24dc8983a4154f66297e error="No such volume"
2017-11-06 04:35:56.088914427 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 04:35:56.586154029 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 04:35:56.64786963 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 04:35:56.6518307 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 04:35:58.201107861 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 04:37:39.884242468 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 04:37:39.884348016 +0000 UTC [ERROR] Failed to mount error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete." name="jobwalkr_assets"
2017-11-06 04:37:39.884463786 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:06:30.846500667 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:06:31.353015513 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:06:31.365055099 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:06:31.366050955 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:06:33.246069777 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:08:14.998695306 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:08:14.998858585 +0000 UTC [ERROR] Failed to mount error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete." name="jobwalkr_assets"
2017-11-06 05:08:14.998925399 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:08:15.413549092 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:08:15.794082745 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:08:15.822464104 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:08:15.823282247 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:08:17.530279536 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:09:59.093448824 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:09:59.093635368 +0000 UTC [ERROR] Failed to mount name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:09:59.093698402 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:10:40.977900957 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:10:41.583115473 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:10:41.620409472 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:10:41.620855147 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:10:43.373549271 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:12:25.011539027 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:12:25.011684317 +0000 UTC [ERROR] Failed to mount name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:12:25.011731325 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:12:25.462510444 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:12:25.933857928 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:12:25.946157591 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:12:25.946933716 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:12:27.648751252 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:14:09.149162863 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:14:09.149223041 +0000 UTC [ERROR] Failed to mount name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:14:09.14924629 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:14:09.651124988 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:14:10.206183491 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:14:10.218752236 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:14:10.219576335 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:14:11.974259547 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:14:24.974870418 +0000 UTC [ERROR] Failed to compare and get from ETCD error="context deadline exceeded" key="SVOLS_stat_jobwalkr_assets" value=Mounted new key="SVOLS_info_jobwalkr_assets"
2017-11-06 05:14:28.201925421 +0000 UTC [ERROR] Failed to Get key-value from ETCD key="SVOLS_gref_jobwalkr_assets" error="context deadline exceeded"
2017-11-06 05:14:28.202092152 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: context deadline exceeded. Also failed to decrease global refcount. Error: context deadline exceeded."
2017-11-06 05:14:28.20213657 +0000 UTC [ERROR] Failed to mount name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: context deadline exceeded. Also failed to decrease global refcount. Error: context deadline exce$
2017-11-06 05:14:28.202170672 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:14:46.861775301 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:14:49.067818778 +0000 UTC [WARNING] Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:14:49.067976012 +0000 UTC [WARNING] Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:14:49.068008503 +0000 UTC [ERROR] Failed to get volume meta-data name="jobwalkr_assets" error="Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Sw$
2017-11-06 05:14:49.339188432 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:14:51.525956223 +0000 UTC [WARNING] Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:14:51.526097009 +0000 UTC [WARNING] Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy
2017-11-06 05:14:51.52616641 +0000 UTC [ERROR] Failed to get volume meta-data name="jobwalkr_assets" error="Failed to read metadata for volume jobwalkr_assets from KV store. Transactional metadata read failed: context deadline exceeded.Swarm cluster maybe unhealthy"

vfile.log worker6.jobwalkr.com:
2017-11-06 05:03:55.907482864 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:03:56.419162813 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:03:56.42897166 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:03:56.42943645 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:03:58.080514435 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:05:39.667608081 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:05:39.667821596 +0000 UTC [ERROR] Failed to mount name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:05:39.668076077 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:07:23.210689729 +0000 UTC [INFO] VolumeDriver Get: jobwalkr_assets
2017-11-06 05:07:23.629129868 +0000 UTC [INFO] GetClientList key=SVOLS_client_jobwalkr_assets_i5c6sievyy4rggvpzqecsmu7c IP=<IP_ADDRESS>
2017-11-06 05:07:23.639076698 +0000 UTC [INFO] Mounting volume name="jobwalkr_assets"
2017-11-06 05:07:23.639430395 +0000 UTC [INFO] Before AtomicIncr
2017-11-06 05:07:25.102044469 +0000 UTC [INFO] Service start timeoutvalue=90
2017-11-06 05:09:06.693201812 +0000 UTC [ERROR] name="jobwalkr_assets" error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete."
2017-11-06 05:09:06.693279418 +0000 UTC [ERROR] Failed to mount error="Failed to blocking wait for Mounted state. Error: Timeout reached; BlockingWait is not complete." name="jobwalkr_assets"
2017-11-06 05:09:06.693305351 +0000 UTC [INFO] Detaching jobwalkr_assets - it is not used anymore
2017-11-06 05:45:14.916418219 +0000 UTC [INFO] VolumeDriver Get: 1f8d27a94f176efadcabf29c5dd1ff723149da46a5bc7046ac32fef06bfa470b
2017-11-06 05:45:17.457772465 +0000 UTC [INFO] Volume not found: 1f8d27a94f176efadcabf29c5dd1ff723149da46a5bc7046ac32fef06bfa470b
2017-11-06 05:45:17.457871058 +0000 UTC [ERROR] Failed to get volume meta-data error="No such volume" name=1f8d27a94f176efadcabf29c5dd1ff723149da46a5bc7046ac32fef06bfa470b
2017-11-06 05:46:08.659363845 +0000 UTC [INFO] VolumeDriver Get: b111c63cace7db055ece80c2d2d1537d0b5002317f929d5b6a62ff45b6ce0ec0
2017-11-06 05:46:08.993500778 +0000 UTC [INFO] Volume not found: b111c63cace7db055ece80c2d2d1537d0b5002317f929d5b6a62ff45b6ce0ec0
2017-11-06 05:46:08.993570843 +0000 UTC [ERROR] Failed to get volume meta-data name=b111c63cace7db055ece80c2d2d1537d0b5002317f929d5b6a62ff45b6ce0ec0 error="No such volume"

I'll follow it up the next days, and let you know if anything changes. If I still can provide you with any input, just let me know and I'll post it.
Regards,
Bart

@victormakmo
would you please check if ETCD ports (2379, 2380) are opened on your manager VMs?
Also may I ask if you have your VMs residing on the same ESX or there are multiple ESXs involved?
The third thing you may want to try is to increase the timeout value to 3 minutes and try...
I know this is a too large timeout, but due to the known slow-starting issue of the SMB server, we need this large timeout to identify if this is due to slowness or due to a problem in the code.
Thanks!

@bartlenaerts
Thank you for the log!
In fact the output of the path under /var/run/docker/plugins will always match with your current output of docker plugin ls.
However, the error you saw previously has a different ID with it:
"Post http://%2Frun%2Fdocker%2Fplugins%2F24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8%2Fvfile.sock/VolumeDriver.Mount: dial unix /run/docker/plugins/24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8/vfile.sock: connect: no such file or directory"
vs.
drwx------. 2 root root 60 Nov  2 09:59 327267222b5332e5ca49eff2ee0d8d06949cb3220aee5b9c8d7cd7607d2e7f1e
The error message indicates that there was a vFile plugin with ID 24ff4b45da04d67a56fd3c3e09ec19cae8657692b37675c129f398e2fe1ea7b8 installed on this node before, and the volume is created with that plugin. But now the plugin has a new ID 327267222b5332e5ca49eff2ee0d8d06949cb3220aee5b9c8d7cd7607d2e7f1e.
So just to double check, is this docker plugin ls output from your manager3?

Hi @luomiao,
This it the output for docker plugin ls from manager3:
ID                  NAME                DESCRIPTION                           ENABLED
ac7712db2e71        vfile:latest        VMWare vFile Docker Volume plugin     true
fc58f64a4ff6        vsphere:latest      VMWare vSphere Docker Volume plugin   true

and the list in /run/docker/plugins
drwx------. 2 root root 60 Nov  7 10:18 ac7712db2e71d01515fbec64a750c577ce4999b7243d18b8c3c204aaa30ced64
drwx------. 2 root root 60 Nov  7 10:18 fc58f64a4ff69408c721965dabc6ee4f6d69630f129ef500ac82338b64b40e76

Something probably went bad because of all the testing I did. Removing plugins, re-installing them...
I'll try again and let you know how it went!
Regards,
Bart

@victormakmo
From the log, it seems the servers are up correctly.
But somehow the node where the container is running now cannot detect the server is up.
This usually can be caused by two reasons:

ETCD port is not open. Even firewall is disabled, sometimes it doesn't mean the ports are all open, especially if the VM is a photon os...
Swarm routing mesh is not working: to verify this, do a simple test by running command: docker service create --replicas 1 -p 8080:80 --name web nginx on one of the manager nodes. Then running  curl <IP_ADDRESS>:8080  on every nodes to verify if all the nodes can get response correctly.

If you still see the problem after the two checks above, we can set up a conference call to help debugging the problem :)

Hi @luomiao,
I cleaned up everything yesterday, so removed all services, all volumes, all images and containers... Today I reinstalled my first service again, and it worked like a charm right out of the box!
Conclusion to me is, make sure the ETCD ports are open, and that you install the vFile driver with the option VFILE_TIMEOUT_IN_SECOND=90. And of course have a clean Docker swarm with no dangling issues.
Thanks for all your help!
Regards,
Bart

hi @luomiao,

For ETCD connection, i have checked on all managers use netstat, all the managers have established connection with port 2379, 2380. i assume the ETCD is up and running

Swarm routing also no problem. all managers can get response on curl.

If you have time, we may set up a conference call. :-)

Hi @victormakmo
sorry I didn't find your email address from github. Would you please send an email to me at<EMAIL_ADDRESS>so we can set up the conference call?
@bartlenaerts Glad to know things are working! You are welcome to try out the Beta release which will be available at the beginning of December. If everything is good for now, would you please close this issue? We will open another one for @victormakmo since it seems a different issue. Thank you!


Add Total transactions counter

Draft This needs some more consideration
Option 1: Simply stop resetting total every week, use that
This is simple but somewhat hacky - we have incoming, outgoing and total - it's redundant
So rather than having to migrate data, we can just use total for all time total, and reset incoming and outgoing weekly
Problem: No nice way of restoring the old values. Which for the record we never kept track of - our counters were always supposed to reset weekly, only we didn't do it.
Option 2 - keep track in history contract

[ ] Add stats table in history contract - account name and total transactions - I don't want to redeploy the token contract

account name

incoming_tx, outgoing_tx

timestamp time last updated

[ ] add migration method that counts all existing tx and updates the number

[ ] On new entry, update the number for bot

[ ] change transactions check in makecitizen and makeresident in accounts contract to use that value, update the value first

Maybe we could keep (and recalculate) total_transactions, incoming_tx, outgoing_tx and add new fields weekly_total, weekly_incoming, weekly_outgoing and reset the latter weekly instead of the former?

Fixed the counter issue - app can make calls to scope and get the count of transactions with a normal HTTP call
REQUEST:
---------------------
POST /v1/chain/get_table_by_scope HTTP/1.0
Host: node.hypha.earth
content-length: 143
Accept: /
Connection: close

{
  "code": "histry.seeds",
  "table": "transactions",
  "lower_bound": "vasylmarchuk",
  "upper_bound": "",
  "limit": 1,
  "reverse": false
}
---------------------
{
  "rows": [{
      "code": "histry.seeds",
      "scope": "vasylmarchuk",
      "table": "transactions",
      "payer": "histry.seeds",
      "count": 4
    }
  ],
  "more": "vasylmarchuk"
}

@VasylMarchuk closing this


Implement Block Data propagation

Summary
We need to implement block propagation during consensus differently: validators need all block data and fullnodes only a part of the data.
Context
Previously, we thought of replacing the tendermint block propagation with a variant that uses erasure-coding instead. While we still want that in some way, the design shifted slightly with the decision that validators need all data. The concrete design will be captured in #389.  The original thinking is kept below for historical context:

Currently, block propagation works roughly as follows, the block is split into equal sized chunks and then gossiped to peers:
https://github.com/lazyledger/lazyledger-core/blob/541b6e7f4e1c2941c7467ca7e0f89a1d35d18854/consensus/reactor.go#L193
https://github.com/lazyledger/lazyledger-core/blob/541b6e7f4e1c2941c7467ca7e0f89a1d35d18854/consensus/reactor.go#L504-L513
Also described here: https://github.com/tendermint/spec/issues/78#issue-562267268
As we tackle supporting validating blocks via data availability proofs (#65), we could as well consider changing block propagation to erasure-coded chunks.
ref: https://github.com/tendermint/spec/issues/78#issuecomment-583918641
This could be interesting for the tendermint core team as well.

The way we (roughly) want to approach this for a very first version, is the following:
The block proposer, will, after erasure coding the block into a square and compute the row and column roots:

We need to write the nodes of these trees into an IPFS DAG. To make this possible, we first need to collect the nodes of the Merkle trees - the current NMT implementation returns only the root of a tree and throws away the inner nodes (see: https://github.com/lazyledger/nmt/issues/9).
Then, we need to implement a IPLD plugin, similar to the one used in our experiments to properly store the nodes into the ipfs DAG.
To be useful this requires running an IPFS node along with a tendermint node.
There are two options how we could write into the DAG:

Use dag put: This requires the nodes to be encoded into a format the plugin can parse prior storing them.
Directly batch add the nodes: dag put internally  decodes and then batch adds the decoded nodes via the DAG api / a NodeAdder. We probably can directly use this adder instead.

(1.) has the advantage that we can also use this via the command-line and we know that it works. The only downside is that it requires a, in this context, superfluous encoding and decoding step (roughly Nodes->Encode(Nodes)->Decode(Nodes)->Nodes->AddBatch(Nodes))
(2.) does directly what we want. We could even pass a "store" into the NMT, that adds the nodes directly to the batch. (roughly Nodes->AddBatch(Nodes)). Downside (not really) is that this might require some more digging into how to setup a environment.
As a first step this  block propagation mechanism will be added to the current gossip-based mechanism, later we will remove the current mechanism.

We decided to stick with Tendermint's block propagation for now. closing.


md-radio-group and md-radio-button not getting enabled.

Bug
It could be a bug.
What is the expected behavior?
Radio button must be selected and ripple effect should occur as displayed in mater2 demo.
What is the current behavior?
The MD2 radio button not working as expected.  The button is always in a disabled state or unable to enable it and ripple effect not shown.
What are the steps to reproduce?
Providing a Plunker (or similar) is the best way to get the team to see your issue.
Plunker template: https://goo.gl/DlHd6U
With plunker the md-radio-group and md-radio-button works fine with the template given systemjs.config.ts
What is the use-case or motivation for changing an existing behavior?
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
{
"name": "ajs",
"version": "0.0.0",
"license": "MIT",
"scripts": {
"ng": "ng",
"start": "ng serve",
"build": "ng build",
"test": "ng test",
"lint": "ng lint",
"e2e": "ng e2e",
"build:aot": "ngc -p src/tsconfig-aot.json && rollup -c src/rollup-config.ts"
},
"private": true,
"dependencies": {
"@angular/animations": "^4.3.0",
"@angular/cdk": "^2.0.0-beta.8",
"@angular/common": "^4.3.0",
"@angular/compiler": "^4.3.0",
"@angular/core": "^4.3.0",
"@angular/forms": "^4.3.0",
"@angular/http": "^4.3.0",
"@angular/material": "^2.0.0-beta.8",
"@angular/platform-browser": "^4.3.0",
"@angular/platform-browser-dynamic": "^4.3.0",
"@angular/platform-server": "^4.3.0",
"@angular/router": "^4.3.0",
"@ngui/map": "^0.18.3",
"@types/googlemaps": "^3.26.16",
"angularfire2": "^4.0.0-rc.1",
"core-js": "^2.4.1",
"firebase": "^4.1.3",
"geofire": "^4.1.2",
"google-maps-angular2": "^1.1.2",
"hammerjs": "^2.0.8",
"ng2-flex-layout": "^0.1.5",
"ng2-validation": "^4.2.0",
"rxjs": "^5.4.2",
"system": "^1.3.0",
"systemjs": "^0.20.15",
"typescript": "^2.4.1",
"zone.js": "^0.8.13"
},
"devDependencies": {
"@angular/cli": "^1.2.1",
"@angular/compiler-cli": "^4.3.0",
"@types/jasmine": "2.5.53",
"@types/node": "~8.0.13",
"codelyzer": "~3.1.2",
"jasmine-core": "~2.6.4",
"jasmine-spec-reporter": "~4.1.1",
"karma": "~1.7.0",
"karma-chrome-launcher": "~2.2.0",
"karma-cli": "~1.0.1",
"karma-coverage-istanbul-reporter": "^1.3.0",
"karma-jasmine": "~1.1.0",
"karma-jasmine-html-reporter": "^0.2.2",
"protractor": "~5.1.2",
"rollup": "^0.45.2",
"rollup-plugin-commonjs": "^8.0.2",
"rollup-plugin-node-resolve": "^3.0.0",
"rollup-plugin-uglify": "^2.0.1",
"ts-node": "~3.2.0",
"tslint": "~5.5.0",
"webpack": "^3.3.0"
}
}
Is there anything else we should know?
/** Type declaration for ambient System. */
declare const System: any;
// Apply the CLI SystemJS configuration.
System.config({
transpiler: 'typescript',
typescriptOptions: {
emitDecoratorMetadata: true
},
paths: {
'node:': 'node_modules/',
},
'geofire': {
main: 'dist/geofire.js',
defaultExtension: 'js',
meta: {'dist/geofire.js': {format: 'global'}}
},
'system': {
main:'system/system.js'
},
map: {
'app': './src',
'rxjs': 'node:rxjs',
'firebase':'node:firebase',
'main': 'main.js',
'geofire': 'node:geofire',
'google-maps-angular2': 'node:google-maps-angular2/dist',
'@ngui/map' <EMAIL_ADDRESS>// Angular specific mappings.
'@angular/core': <EMAIL_ADDRESS>'@angular/common': <EMAIL_ADDRESS>'@angular/compiler': <EMAIL_ADDRESS>'@angular/http': <EMAIL_ADDRESS>'@angular/forms': <EMAIL_ADDRESS>'@angular/router': <EMAIL_ADDRESS>'@angular/animations': <EMAIL_ADDRESS>'@angular/animations/browser': <EMAIL_ADDRESS>'@angular/platform-browser': <EMAIL_ADDRESS>'@angular/platform-browser/animations': <EMAIL_ADDRESS>'@angular/platform-browser-dynamic': <EMAIL_ADDRESS>'@angular/material': <EMAIL_ADDRESS>'@angular/cdk': <EMAIL_ADDRESS>},
packages: {
'firebase':                   { main: 'firebase.js', defaultExtension: 'js' },
'geofire':                    { main: 'dist/geofire.js', defaultExtension: 'js' },
rxjs: {
main: 'Rx.js',
defaultExtension: 'js'
},
'google-maps-angular2': {
main: 'index.js',
defaultExtension: 'js'
},
'.': {
defaultExtension: 'ts'
},
'vendor': {
defaultExtension: 'js'
}
}
});

Please resubmit this issue and provide a Plunker that shows your issue. Thanks

As I mentioned before, the html template works in the plunker with the default template but it's not working in my project with the above given configuration.

@ravivit9 This question will probably get better results being asked in Stack Overflow.

I have a strong suspicion that material design 2 library is not being loaded properly or missing some dependency library.

Github issues is for issue/bug tracking, it's not a help forum.
That being said, why is your systemJS import for @angular/cdk an external url? It should be<EMAIL_ADDRESS>Also is flex-layout a dependency that needs to be loaded? I'm not sure. Finally, you should consider setting the @angular/cdk and @angular/material package.json to an exact version (remove the ^)


added manifest to web to prompt app install in chrome

Details

In order to make chrome mobile to prompt the user to install our app we need to add a manifest.json to our mWeb page. I am not really sure if having a manifest file has any other side effect.
I had to change the webpack script exactly to keep the manifest.json in the web build.
Good SO post about how to implement and test this.
Actual documentation which is confusing but maybe we can implement the banner in a better way later.
Fixed Issues

$ https://github.com/Expensify/Expensify/issues/325315
Tests

Enabled this flag in chrome mobile chrome://flags/#bypass-app-banner-engagement-checks
Ran ND over https using ngrok
Opened up the mWeb page in a device with Google Play installed
Verified that the prompts to install the app appear
Verified that the prompts to install the app actually take the user to the Google Play app's page
Opened up chrome desktop and verified that it still works and that there is no prompt

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

This is a little bit tricky to test, so feel free to DM me.
Make sure you are in an Android device with Google Play installed
And if you are running this locally make sure to be serving it over https.

Enable this flag in your Android device in chrome mobile chrome://flags/#bypass-app-banner-engagement-checks
Open the mWeb page and wait a few seconds
Verify that the banner is shown
Verify that following it gets you to the Google Play store

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

Use the links below to test this build in android and iOS. Happy testing!

:test_tube::test_tube: Use the links below to test this build in android and iOS. Happy testing! :test_tube::test_tube:

android :robot:
iOS :apple:

https://ad-hoc-expensify-cash.s3.amazonaws.com/android/29491/index.html
https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/29491/index.html

desktop :computer:
web :spider_web:

https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/29491/NewExpensify.dmg
https://29491.pr-testing.expensify.com

:test_tube::test_tube: Use the links below to test this build in android and iOS. Happy testing! :test_tube::test_tube:

android :robot:
iOS :apple:

https://ad-hoc-expensify-cash.s3.amazonaws.com/android/29491/index.html
https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/29491/index.html

desktop :computer:
web :spider_web:

https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/29491/NewExpensify.dmg
https://29491.pr-testing.expensify.com

Going to review this tomorrow

@rlinoz I tested https://29491.pr-testing.expensify.com/ which is https but prompt not showing. Also splash logo is cut
https://github.com/Expensify/App/assets/108292595/f63e4991-dd14-4639-842b-eacdfcc3cc8f

Is the New Expensify app not installed in the device? Also, did you enabled the flag chrome://flags/#bypass-app-banner-engagement-checks?

Both are done already

Also, just to be sure the device has Play Store installed right?

Right.
I will try again. Btw please update logo icon and generate build again

:test_tube::test_tube: Use the links below to test this build in android and iOS. Happy testing! :test_tube::test_tube:

android :robot:
iOS :apple:

https://ad-hoc-expensify-cash.s3.amazonaws.com/android/29491/index.html
https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/29491/index.html

desktop :computer:
web :spider_web:

https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/29491/NewExpensify.dmg
https://29491.pr-testing.expensify.com

Still logo is cut
https://github.com/Expensify/App/assets/108292595/f66488ad-35c6-4450-8431-62b828f74806
Happening only on QR build. Fine in localhost

That is a problem with mWeb for adhoc builds, it is not related to this PR.
Other PRs with the same issue: https://29991.pr-testing.expensify.com https://29569.pr-testing.expensify.com

ok then out of scope.
Btw, were you able to see prompt with https://29491.pr-testing.expensify.com/?

Hey @situchan, I was just able to see the banner. But only when I opened the website from a link in WhatsApp...
Can you review the code, and I'd be happy to approve too.

Unassigning @jasperhuangg as I've already reviewed

@situchan in order for this to work I had to do the following:

Uninstall the NewDot Android app
Close all the mWeb NewDot tabs
Kill Chrome app
Paste the URL into WhatsApp
Click the link

Oh and also paste this flag into chrome: chrome://flags/#bypass-app-banner-engagement-checks, then enable it

@Julesssss That worked. Thanks for the detailed step. Hope it won't be complex for real users when they visit new.expensify.com

Once I open it, no longer prompt shows. Is it expected?

Hope it won't be complex for real users when they visit new.expensify.com

I mostly just care about enabling it. If Google make the conditions difficult, that seems fine to me.

Once I open it, no longer prompt shows again. Is it expected?

Yeah.

Great, so @situchan would you mind just filling out the checklist (obviously you don't need to test other platforms) and I'll merge?

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/Julesssss in version: 1.3.90-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@rlinoz will you do the Internal QA for this one?

Thanks !

🚀 Deployed to production by https://github.com/marcaaron in version: 1.3.90-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

Hey @Julesssss @jasperhuangg @situchan I was assigned this issue for payment, but I'm unsure about the amount and if the urgency bonus is applied. Am I correct to think that @situchan should be paid $500 for the C+ review? Does this issue qualify for the urgency bonus?
Also, it looks like the PR was merged 2 days ago, so should I wait 7 days to make sure there are no regressions before I make the payment? Thanks!

No bonus. $500 if no regressions until Nov 1

Yep, as @situchan mentioned. I don't believe we include the urgency bonus on internal PRs

🚀 Deployed to production by https://github.com/marcaaron in version: 1.3.91-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[FEATURE]: Adding more columns to Supabase auth.users table schema

What version of drizzle-orm are you using?
0.36.0
What version of drizzle-kit are you using?
0.27.0
Describe the Bug
The current native implementation of the Supabase auth.users table schema is missing many of the columns that the table actually includes in Supabase.
Expected behavior
I expect at least some of the missing columns, such as email, phone, and created_at, to be added to the auth.users table schema, so we can avoid creating duplicate or redundant columns in a custom, public table we make ourselves.
This would also allow us to create more relations on the auth.users table.
Environment & setup
No specific environment.

Was added in<EMAIL_ADDRESS>


Audio can not be played (CORS error, can't get the media file)

Hi, I found an issue about web application when playing audio. I used both chrome and edge for testing, the results are the same.
Here is the flow:
(1) After setting up source url to AudioPlayer(), I tried to play the audio but nothing happen.
(2) I take a look for audioPlayer.getDuration(), it shows 0 duration.
(3) I press F12 to enter dev mode of chrome, press "Network" and saw that one row about audio file
Name: xxx.mp3
Status: CORS error
Type: media
Initiator: Other
Size: 0B
The URL is end with .mp3, it is accessible and not a redirect.
final audioPlayer = AudioPlayer();
await audioPlayer.setSourceUrl(url);
print(await audioPlayer.getDuration()); // 0:00:00.000000

Tested version: v1.1.1 or above.

Did you find #693? You must enable Cors on your server, or disable web security e.g. in your chrome browser.

This will help https://www.youtube.com/watch?v=IMQdSTlTXjA


”Dorolea”, ”Dumbrava”, ”Valea Poenii” are listed in the drop-down menu

Description:
When user writes ”Dorolea”, ”Dumbrava”, ”Valea Poenii” the locations appear in the drop-down menu next to the comune and the county witch is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Dorolea"

Expected results
The location appears in the drop-down menu as "Dorolea, Bistrița-Năsăud (Livezile)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Dumbrava"

Expected results
The location appears in the drop-down menu as "Dumbrava, Bistrița-Năsăud (Livezile)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 5
Write in the search bar "Valea Poenii"

Expected results
The location appears in the drop-down menu as "Valea Poenii, Bistrița-Năsăud (Livezile)".


Distribute TypeScript typings

It would be good to have the typings for chalk distributed with the npm package:

Easier to use with TypeScript (no external dependencies needed)
Easier to ensure that the typings are in sync with the library
Easier for users to be more confident that the typings are in sync

I usually don't do this, for reasons outlined in https://github.com/sindresorhus/ama/issues/439, but since Chalk is one of the most popular packages on npm, I'm gonna accept it if you or someone else can commit to maintain it.

Since I write quite a bit of TypeScript, I could write a typing for Chalk and maintain it... That wouldn't be an issue.

@basedjux That would be awesome. Pull request welcome :)

@sindresorhus cool!
I'll get working on that 😄

Never mind @sindresorhus, looks like someone beat me to it writing a definition:
https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/chalk

That one is very outdated.

Ah yeah it is, 3 months.
Okay I'll write the updated one.

It's based on a more than 2 year old Chalk version.

Yeesh yeah that's scary outdated.

@basedjux @sindresorhus Is this going to be distributed with the npm package? I think that's the best way to create a seamless experience.

Yes

@basedjux We just bumped to v2 by the way :)

@Qix- Yeah, I saw the tweet!
I'll be looking at those API changes.

This was added in v2.2.0.


Error on each use

I've got the plugin all configured and connected to my account. However, no matter what page I am on, when I click the rM button, it says 50%... 75%... and then err.
I am unable to find a place where the error is detailed.

Thanks for reporting this! There are a few places you can find a more detailed error, by digging into the extension logs from the extension menu. However, I already know what the cause is. reMarkable updated their cloud API which broke this extension. Unfortunately the API isn't public, and the people whom I know figured it out before have moved on. I'm going to try and figure it out and will update this when I have.
In the meantime you have two options:

continue to use this by just downloading the epubs and uploading them using the app. Kind of a pain, but you should still get the images.
switch back to the normal extension without images until I get this fixed.

Sorry for the inconvenience!

I'm having the same problem.  Thanks for the suggested work arounds... going to try #1 for now.  And thanks again for the effort that you put into this extension.. this is way better than the official version.

Okay, I have a work around that's not ideal, but works for now. As of v4.0.0 in the store, which should release shortly, you should be able to upload again. Unfortunately, the newer API doesn't allow modifying things like the font or the margin. It's not ideal, but this is what I have for now.
Since the extensions is more or less working, I'm going to close this. However, I'm going to be on the search for better apis to bring back some of the old functionality.


[MRG] Adding bash to Dockerfile to fix git-credential-env

Re: #736
Since alpine images  don't include bash by default, this breaks the usage private repos with git-credential-env which requires bash.....small PR to just add bash to the Dockerfile (image size increased ~5MB)

Is bash necessary, or would changing https://github.com/jupyter/repo2docker/blob/master/docker/git-credential-env#L1 to /bin/sh also work?

@manics that was my initial thought at first too, but because of https://github.com/jupyter/repo2docker/pull/389 I wanted to keep git-credential-env consistent by using bash

LGTM! Thanks again for the excellent bug report, tracking it down and fixing it. The open-source trifecta 🏆!

@betatim no problem! glad i could contribute