VM doesn't boot correctly after restoring its boot disk

Steps to reproduce:

Copy vdisk from a template (vd2)
Start a VM using qemu with nbd, tlog servers
echo a file for in the VM after it boots
Stop the VM (and stopping nbd, tlog servers)
restore the vdisk /bin/zeroctl restore vdisk -f vd2 --config <IP_ADDRESS>:2379 --end-timestamp<PHONE_NUMBER>000000000 --data-shards 1 -v --parity-shards 1
Start the vm again with nbd, tlog servers, it halts at this

config in etcd
vd2:vdisk:conf:static
blockSize: 4096
readOnly: false
size: 4
type: boot

vd2:vdisk:conf:storage:nbd
slaveStorageClusterID: ''
storageClusterID: myblockcluster
templateStorageClusterID: 87ec688e821f5f0006619b7207d656bd
tlogServerClusterID: tlogserver_vm0_525400123456

vd2:vdisk:conf:storage:tlog
zeroStorClusterID: myobjectcluster

myblockcluster:cluster:conf:storage
dataStorage:
- address: <IP_ADDRESS>:2000
- address: <IP_ADDRESS>:2000
label: myblockcluster
metadataStorage:
  address: <IP_ADDRESS>:2001
nodes:
- '525400123456'
- '525400123459'
status: ready

myobjectcluster:cluster:conf:zerostor
iyo:
  clientID: <clientID>
  namespace: test
  org: zero-khaled
  secret: <secret>
metadataServers:
- address: <IP_ADDRESS>:2379
servers:
- address: <IP_ADDRESS>:2002
- address: <IP_ADDRESS>:2002

vm0:nbdserver:conf:vdisks
vdisks:
- vd2

tlogserver_vm0_525400123456:cluster:conf:tlog
servers:
- <IP_ADDRESS>:11211

Did you see something in the logs of nbdserver and tlogserver?

Nothing that raises red flags, just success negotiations....etc

@khaledkbadr need logs of these:

nbd & tlog before restore : normal logs is enough
restoring : verbose log
nbd & tlog after restore : verbose log

Logs of nbdserver before restore
t=2017-08-30T06:06:50+0000 lvl=info msg="Starting listening on unix:/server.socket.vm0" module=nbdserver caller=listener.go:151
t=2017-08-30T06:06:51+0000 lvl=info msg="Connect to unix:/server.socket.vm0 from @" module=nbdserver caller=listener.go:165
t=2017-08-30T06:06:51+0000 lvl=info msg="Getting vdisk \"vd2\"" module=global caller=exportcontroller.go:52
t=2017-08-30T06:06:52+0000 lvl=info msg="Negotiation succeeded with @/vd2" module=nbdserver caller=connection.go:609
t=2017-08-30T06:11:13+0000 lvl=info msg="vdisk 'vd2' received `Stop` command from vdisk completion" module=global caller=backend.go:232
t=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot read data to write: unexpected EOF" module=nbdserver caller=connection.go:398
t=2017-08-30T06:11:28+0000 lvl=info msg="Receiver exiting for @/vd2" module=nbdserver caller=connection.go:224
t=2017-08-30T06:11:28+0000 lvl=info msg="Worker forced close for @/vd2" module=nbdserver caller=connection.go:634
t=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot write reply: write unix /server.socket.vm0->@: write: broken pipe" module=nbdserver caller=connection.go:205
t=2017-08-30T06:11:28+0000 lvl=info msg="Replyer exiting for @/vd2" module=nbdserver caller=connection.go:188
t=2017-08-30T06:11:28+0000 lvl=info msg="GoBackground exited for vdisk: vd2" module=global caller=tlog.go:294
t=2017-08-30T06:11:28+0000 lvl=info msg="error while closing tlog client: close tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=tlog.go:297
t=2017-08-30T06:11:28+0000 lvl=info msg="tlogclient : reconnect from read because of : read tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:334
t=2017-08-30T06:11:28+0000 lvl=info msg="tlogclient: reconnect because `read tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection` failed: client already closed" module=global caller=client.go:340
t=2017-08-30T06:11:33+0000 lvl=eror msg="tlogclient failed to send: write tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:543
t=2017-08-30T06:11:33+0000 lvl=info msg="tlog client : reconnect from send attemp(1) failed:client already closed" module=global caller=client.go:534
t=2017-08-30T06:11:38+0000 lvl=eror msg="tlogclient failed to send: write tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:543
t=2017-08-30T06:11:38+0000 lvl=info msg="tlog client : reconnect from send attemp(1) failed:client already closed" module=global caller=client.go:534
t=2017-08-30T06:11:43+0000 lvl=eror msg="tlogclient failed to send: write tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:543
t=2017-08-30T06:11:43+0000 lvl=info msg="tlog client : reconnect from send attemp(1) failed:client already closed" module=global caller=client.go:534
t=2017-08-30T06:11:43+0000 lvl=info msg="vdisk 'vd2' finished the flush under SIGTERM handler" module=global caller=backend.go:245
t=2017-08-30T06:11:43+0000 lvl=info msg="Backend background thread exiting for @/vd2" module=nbdserver caller=connection.go:616
t=2017-08-30T06:11:43+0000 lvl=info msg="Closed connection from @/vd2" module=nbdserver caller=connection.go:596
flush timeout

Logs of tlog before restore (The second log due to shutting down of the nbdserver I think)
t=2017-08-30T06:11:28+0000 lvl=info msg="failed to send resp to :vd2, err:writev tcp <IP_ADDRESS>:11211-><IP_ADDRESS>:34906: writev: connection reset by peer" module=global caller=server.go:302

zeroctl restore logs (this is a short version of the same logs basically)
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93
flushing persistent storage" module=global caller=storage.go:93

nbd logs after restore
t=2017-08-30T06:29:07+0000 lvl=dbug msg="flags parsed: memorystorage=false tlsonly=false profileaddress=\"\" protocol=\"unix\" address=\"/server.socket.vm0\" config=\"<IP_ADDRESS>:2379\" lbacachelimit=20971520 logfile=\"\" id=\"vm0\"" module=global caller=main.go:73
t=2017-08-30T06:29:07+0000 lvl=info msg="Starting listening on unix:/server.socket.vm0" module=nbdserver caller=listener.go:151
t=2017-08-30T06:29:07+0000 lvl=dbug msg="started ExportController background thread (config hot reloading)" module=global caller=exportcontroller.go:102
t=2017-08-30T06:29:07+0000 lvl=dbug msg="watch goroutine for etcd key 'vm0:nbdserver:conf:vdisks' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:07+0000 lvl=dbug msg="WatchNBDVdisksConfig for server vm0 started" module=global caller=config_api.go:187
t=2017-08-30T06:29:07+0000 lvl=info msg="Connect to unix:/server.socket.vm0 from @" module=nbdserver caller=listener.go:165
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Start negotation with@" module=nbdserver caller=connection.go:599
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Sending newstyle header to@" module=nbdserver caller=connection.go:644
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Receiving client flags from@" module=nbdserver caller=connection.go:661
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Receiving options from@" module=nbdserver caller=connection.go:670
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Received option 3 from @" module=nbdserver caller=connection.go:686
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Received option 1 from @" module=nbdserver caller=connection.go:686
t=2017-08-30T06:29:07+0000 lvl=dbug msg="getting exportConfig for: vd2" module=nbdserver caller=connection.go:751
t=2017-08-30T06:29:07+0000 lvl=info msg="Getting vdisk \"vd2\"" module=global caller=exportcontroller.go:52
t=2017-08-30T06:29:07+0000 lvl=dbug msg="generating export using driver ardb for @" module=nbdserver caller=connection.go:1065
t=2017-08-30T06:29:07+0000 lvl=dbug msg="create nbd config listener for vd2" module=global caller=provider.go:183
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) loading primary storage cluster: myblockcluster" module=global caller=composition_api.go:724
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watchVdiskNBDConfig for vdisk vd2 started" module=global caller=config_api.go:253
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key 'vd2:vdisk:conf:storage:nbd' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) loading template storage cluster: 87ec688e821f5f0006619b7207d656bd" module=global caller=composition_api.go:732
t=2017-08-30T06:29:08+0000 lvl=dbug msg="WatchStorageClusterConfig for cluster myblockcluster started" module=global caller=config_api.go:385
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key 'myblockcluster:cluster:conf:storage' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) loading slave storage cluster: " module=global caller=composition_api.go:741
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) already watches slave cluster: " module=global caller=composition_api.go:864
t=2017-08-30T06:29:08+0000 lvl=dbug msg="spawn redisProvider listener goroutine for vd2" module=global caller=provider.go:194
t=2017-08-30T06:29:08+0000 lvl=dbug msg="creating LBA for vdisk vd2 with 64 bucket(s)" module=global caller=lba.go:47
t=2017-08-30T06:29:08+0000 lvl=dbug msg="WatchStorageClusterConfig for cluster 87ec688e821f5f0006619b7207d656bd started" module=global caller=config_api.go:385
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key '87ec688e821f5f0006619b7207d656bd:cluster:conf:storage' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="creating tlogStorage for backend vd2 (boot)" module=global caller=backend_factory.go:114
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key 'tlogserver_vm0_525400123456:cluster:conf:tlog' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="WatchTlogClusterConfig for cluster tlogserver_vm0_525400123456 started" module=global caller=config_api.go:516
t=2017-08-30T06:29:08+0000 lvl=dbug msg="generating backend ardb, using 333 flags, for @" module=nbdserver caller=connection.go:1120
t=2017-08-30T06:29:08+0000 lvl=info msg="Negotiation succeeded with @/vd2" module=nbdserver caller=connection.go:609
t=2017-08-30T06:29:08+0000 lvl=dbug msg="Start transmition phase with@/vd2" module=nbdserver caller=connection.go:624
t=2017-08-30T06:29:08+0000 lvl=dbug msg="background goroutine spawned for tlogstorage vd2" module=global caller=tlog.go:291
t=2017-08-30T06:29:08+0000 lvl=dbug msg="starting background thread for vdisk vd2's backend" module=global caller=backend.go:218

tlog logs after restore
t=2017-08-30T06:29:07+0000 lvl=dbug msg="flags parsed: memorystorage=false tlsonly=false profileaddress=\"\" protocol=\"unix\" address=\"/server.socket.vm0\" config=\"<IP_ADDRESS>:2379\" lbacachelimit=20971520 logfile=\"\" id=\"vm0\"" module=global caller=main.go:73
t=2017-08-30T06:29:07+0000 lvl=info msg="Starting listening on unix:/server.socket.vm0" module=nbdserver caller=listener.go:151
t=2017-08-30T06:29:07+0000 lvl=dbug msg="started ExportController background thread (config hot reloading)" module=global caller=exportcontroller.go:102
t=2017-08-30T06:29:07+0000 lvl=dbug msg="watch goroutine for etcd key 'vm0:nbdserver:conf:vdisks' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:07+0000 lvl=dbug msg="WatchNBDVdisksConfig for server vm0 started" module=global caller=config_api.go:187
t=2017-08-30T06:29:07+0000 lvl=info msg="Connect to unix:/server.socket.vm0 from @" module=nbdserver caller=listener.go:165
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Start negotation with@" module=nbdserver caller=connection.go:599
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Sending newstyle header to@" module=nbdserver caller=connection.go:644
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Receiving client flags from@" module=nbdserver caller=connection.go:661
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Receiving options from@" module=nbdserver caller=connection.go:670
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Received option 3 from @" module=nbdserver caller=connection.go:686
t=2017-08-30T06:29:07+0000 lvl=dbug msg="Received option 1 from @" module=nbdserver caller=connection.go:686
t=2017-08-30T06:29:07+0000 lvl=dbug msg="getting exportConfig for: vd2" module=nbdserver caller=connection.go:751
t=2017-08-30T06:29:07+0000 lvl=info msg="Getting vdisk \"vd2\"" module=global caller=exportcontroller.go:52
t=2017-08-30T06:29:07+0000 lvl=dbug msg="generating export using driver ardb for @" module=nbdserver caller=connection.go:1065
t=2017-08-30T06:29:07+0000 lvl=dbug msg="create nbd config listener for vd2" module=global caller=provider.go:183
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) loading primary storage cluster: myblockcluster" module=global caller=composition_api.go:724
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watchVdiskNBDConfig for vdisk vd2 started" module=global caller=config_api.go:253
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key 'vd2:vdisk:conf:storage:nbd' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) loading template storage cluster: 87ec688e821f5f0006619b7207d656bd" module=global caller=composition_api.go:732
t=2017-08-30T06:29:08+0000 lvl=dbug msg="WatchStorageClusterConfig for cluster myblockcluster started" module=global caller=config_api.go:385
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key 'myblockcluster:cluster:conf:storage' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) loading slave storage cluster: " module=global caller=composition_api.go:741
t=2017-08-30T06:29:08+0000 lvl=dbug msg="nbdStorageConfigWatcher (vd2) already watches slave cluster: " module=global caller=composition_api.go:864
t=2017-08-30T06:29:08+0000 lvl=dbug msg="spawn redisProvider listener goroutine for vd2" module=global caller=provider.go:194
t=2017-08-30T06:29:08+0000 lvl=dbug msg="creating LBA for vdisk vd2 with 64 bucket(s)" module=global caller=lba.go:47
t=2017-08-30T06:29:08+0000 lvl=dbug msg="WatchStorageClusterConfig for cluster 87ec688e821f5f0006619b7207d656bd started" module=global caller=config_api.go:385
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key '87ec688e821f5f0006619b7207d656bd:cluster:conf:storage' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="creating tlogStorage for backend vd2 (boot)" module=global caller=backend_factory.go:114
t=2017-08-30T06:29:08+0000 lvl=dbug msg="watch goroutine for etcd key 'tlogserver_vm0_525400123456:cluster:conf:tlog' started" module=global caller=source_etcd.go:86
t=2017-08-30T06:29:08+0000 lvl=dbug msg="WatchTlogClusterConfig for cluster tlogserver_vm0_525400123456 started" module=global caller=config_api.go:516
t=2017-08-30T06:29:08+0000 lvl=dbug msg="generating backend ardb, using 333 flags, for @" module=nbdserver caller=connection.go:1120
t=2017-08-30T06:29:08+0000 lvl=info msg="Negotiation succeeded with @/vd2" module=nbdserver caller=connection.go:609
t=2017-08-30T06:29:08+0000 lvl=dbug msg="Start transmition phase with@/vd2" module=nbdserver caller=connection.go:624
t=2017-08-30T06:29:08+0000 lvl=dbug msg="background goroutine spawned for tlogstorage vd2" module=global caller=tlog.go:291
t=2017-08-30T06:29:08+0000 lvl=dbug msg="starting background thread for vdisk vd2's backend" module=global caller=backend.go:218

t=2017-08-30T06:11:28+0000 lvl=info msg="GoBackground exited for vdisk: vd2" module=global caller=tlog.go:294
t=2017-08-30T06:11:28+0000 lvl=info msg="error while closing tlog client: close tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=tlog.go:297
t=2017-08-30T06:11:28+0000 lvl=info msg="tlogclient : reconnect from read because of : read tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:334
t=2017-08-30T06:11:28+0000 lvl=info msg="tlogclient: reconnect because read tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection failed: client already closed" module=global caller=client.go:340
t=2017-08-30T06:11:33+0000 lvl=eror msg="tlogclient failed to send: write tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:543
t=2017-08-30T06:11:33+0000 lvl=info msg="tlog client : reconnect from send attemp(1) failed:client already closed" module=global caller=client.go:534
t=2017-08-30T06:11:38+0000 lvl=eror msg="tlogclient failed to send: write tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:543
t=2017-08-30T06:11:38+0000 lvl=info msg="tlog client : reconnect from send attemp(1) failed:client already closed" module=global caller=client.go:534
t=2017-08-30T06:11:43+0000 lvl=eror msg="tlogclient failed to send: write tcp <IP_ADDRESS>:34906-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:543
t=2017-08-30T06:11:43+0000 lvl=info msg="tlog client : reconnect from send attemp(1) failed:client already closed" module=global caller=client.go:534
t=2017-08-30T06:11:43+0000 lvl=info msg="vdisk 'vd2' finished the flush under SIGTERM handler" module=global caller=backend.go:245
t=2017-08-30T06:11:43+0000 lvl=info msg="Backend background thread exiting for @/vd2" module=nbdserver caller=connection.go:616

Thanks @khaledkbadr , there is no problem from the logs. It is tlogclient that didn't disconnect in gracefull way, addressed at #426

there is no problem from the logs. It is tlogclient that didn't disconnect in gracefull way

Is still simply something you observe from the logs, or does this also explain @khaledkbadr his issue?

Is still simply something you observe from the logs, or does this also explain @khaledkbadr his issue?

khaled told me in the chat that he see problem in logs before restore, i told him that it is not a problem, and will  explain to  him here :)

Ah ok. So that means the issue origin is still unknown :(

tlog logs after restore

It seems you provide nbd instead of tlog logs

I did, but edited. refresh :P

@GlenDC do you see some problems there?

No, nbdserver logs doesn't seem to be verbose-enabled though. But I can't see anything wrong.
I do know that the restore tlog code could use more verbose logs, because right now the zeroctl restore vdisk -v really doesn't give any verbose logging at all, besides the flush messages you see in the logs above. Perhaps that could help with debugging this issue.
@khaledkbadr is this issue only with a specific image, or is it with any image? if it is with a specific, you could perhaps give it to @iwanbk, as that might help him with reproducing it on his dev machine.

We use the ubuntu-1604 cloud image (hacked version of it so we can login), You can find it in hub.gig.com under template:ubuntu-1604

@khaledkbadr hmm weird, I use a very similar image, among other images I test with. Although the ubuntu-1604 cloud image I use is the original one as downloaded. Yet I can restore with it... At least when I tried last.

hmmmmm, All I know is that the issue appeared after we integrated 0-stor with tlogserver. And it seems to be persistent issue not just now and then kinda issue.

hmmmmm, All I know is that the issue appeared after we integrated 0-stor with tlogserver. And it seems to be persistent issue not just now and then kinda issue.

Confusing, i just redo the test yesterday:

start vm
destroy nbd's ardb
restore
start vm again

And all was good.
Maybe i need different procedures...

we don't destroy the ardb, just stop the nbdserver, tlogserver

we don't destroy the ardb, just stop the nbdserver, tlogserver

Why you need to restore when you still have the ardb?

we use the restore for rollback of vdisk not to restore after delete.

@iwanbk restore command is also used to do rollbacks. In which case the vdisk gets deleted first, and than the vdisk gets rebuild using the restore command.

we use the restore for rollback of vdisk not to restore after delete.

Could you also try the restore after delete, not  restore for rollback?

@iwanbk restore command is also used to do rollbacks.

Yeah, i know. I was expecting rollback action to use rollback word instead of restore.

We use the -f flag, that does the same thing. no?

We use the -f flag, that does the same thing. no?

Yeah, it should. I'll test

-f flag  worked for me,

Yes, defining the -f flag in the zeroctl restore vdisk command should at all times be the same logic as a combined zeroctl delete vdisk && zeroctl restore vdisk command. If that's not the case that would be a bug. In the current version I'm sure they are the same, but mentioning it for future encounters, should you ever have suspicions.

t=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot read data to write: unexpected EOF" module=nbdserver caller=connection.go:398
t=2017-08-30T06:11:28+0000 lvl=info msg="Receiver exiting for @/vd2" module=nbdserver caller=connection.go:224
t=2017-08-30T06:11:28+0000 lvl=info msg="Worker forced close for @/vd2" module=nbdserver caller=connection.go:634
t=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot write reply: write unix /server.socket.vm0->@: write: broken pipe" module=nbdserver caller=connection.go:205
t=2017-08-30T06:11:28+0000 lvl=info msg="Replyer exiting for @/vd2" module=nbdserver caller=connection.go:188

I was too focused with tlog so didn't notice these lines.
There is some error in nbdserver module.
The error seems to be between nbd client & server.

Stop the VM (and stopping nbd, tlog servers)

@khaledkbadr can you make sure that you stop nbd & tlog first before stop the VM?

@khaledkbadr can you make sure that you stop nbd & tlog first before stop the VM?

My bad, i forgot that qemu will automatically wait for all data to be flushed, right?

Which means the VM wait for nbd & tlog to do flush & stop.
I guess not I test on the stopping the vm not shutting it down, cause we need the rollback most in stopping the vm non-gracefully since data loss may happen due to it. That's why we asked for a flush on SIGTERM signal for the nbdserver.

I guess not I test on the stopping the vm not shutting it down, cause we need the rollback most in stopping the vm non-gracefully since data loss may happen due to it. That's why we asked for a flush on SIGTERM signal for the nbdserver.

by stopping, you mean the same case as #288, right?
It actually wait for some reasonable timeout, then destroy the VM.
But yes, it doesn't do graceful shutdown, only waiting for reasonable timeout
See https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainDestroy
From the ndb logs, SIGTERM still handled properly.
From the gondbserver error logs above (especially the line below), it seems that the flush still not finished after the reasonable timeout and then qemu simply close the nbd connection.
And there might be some data that not sent yet to nbd server.
=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot read data to write: unexpected EOF" module=nbdserver caller=connection.go:398

You said that it started to happen after we were moving to 0-stor.
Until yesterday, i was pretty sure that 0-stor is way slower (which might causing unfinished flush).
But yesterday 0-stor server got quite performance improvement (https://github.com/zero-os/0-stor/pull/212).
Please make sure you have that improvement and redo the test.

Copy vdisk from a template (vd2)
I also see some possible error here.

@GlenDC does the zeroctl copy vdisk also copy tlog data? I guess not, because it doesn't implemented yet, right? see #280
@khaledkbadr did you also provide tlog related config when copying the  disk?

@khaledkbadr did you also provide tlog related config when copying the disk?

This all I got in my etcd cluster + templatecluster config (which I forgot to add) but I guess it doesn't matter right?

Until yesterday, i was pretty sure that 0-stor is way slower (which might causing unfinished flush).

Sure, I will try again after checking if my 0-stor flist is at the latest or not. But that may raise problems in the testing/production env since it uses much networking unlike my env which is totally local and still get this problem.

This all I got in my etcd cluster + templatecluster config (which I forgot to add) but I guess it doesn't matter right?

I think it does matter, not sure, i don't really know what current zeroctl copy does. I'll check it.
So, if zeroctl copy doesn't write to tlog, then there is no way for tlog to restore it. Because tlog doesn't have the data it needs to restore.
If possible, please do the test without zeroctl copy step. I assume that #288 is without zeroctl copy, no?

But that may raise problems in the testing/production env since it uses much networking unlike my env which is totally local and still get this problem.

Yeah, we are working to improve the 0-stor performance. https://github.com/zero-os/0-stor/pull/212 is one of it.

the zeroctl copy vdisk step is only done at the very first when trying to copy template data to a disk ie vd2. Either than that it's not used at all. Actually if I skipped it the vd2 would not have any data and it will not boot at all, so it can't be skipped

OK, i just checked the zeroctl copy command. I think it is not copying tlog data, as expected because we still have #280 open.
My conclusion so far, the problem could be caused by:

slow 0-stor server, as mentioned above, some improvement at https://github.com/zero-os/0-stor/pull/212

zeroctl copy doesn't copy tlog data. So, tlog doesn't have data which needed to restore the base ubuntu image. All tlog has is the transaction after VM booted up. In @khaledkbadr test is only single echo to a file
But i'm still not sure if it is the case, because from the logs above, the flush need 40 seconds to finish, it must be a lot of transaction, not only single echo to a file.
@GlenDC might have some explanation about this.

@iwanbk stop focussing on zeroctl copy. @khaledkbadr is using that command right at the start, before anything else, to copy the deduped metadata from a template vdisk, so that when he starts vd2 he can actually fetch that vdisk's data on the fly.
It's obviously true that zeroctl copy doesn't copy tlog (meta)data, hence why there is an issue open for it. And why we even talked about it yesterday or so.
So far the issue seems to be that the tlogserver shuts down before it is finished. Which is not unsurprising that it can happen. This is the main reason https://github.com/zero-os/0-Disk/issues/322 would be really useful. Because just waiting "reasonable" time from the nbdserver is flawed and not reasonable at all. Better it would be to have tlogserver also handle SIGTERM so that it can shutdown only when it is finished with its work. (I suppose you could put like a very big upper limit of 5 minutes (really not sure what a reasonable upper limit is here) or more, in case there is a bug in the tlogserver that causes it to not finish ever).
It's our best lead so far, as an unfinished tlogserver would definitely explain why the restored vdisk is not working as expected.

So far the issue seems to be that the tlogserver shuts down before it is finished.

No, the log clearly mentioned that flush has been finished.

Because just waiting "reasonable" time from the nbdserver is flawed and not reasonable at all

It is qemu that doing this, not tlog
You really missed the flow here

Better it would be to have tlogserver also handle SIGTERM so that it can shutdown only when it is finished with its work

It is already finished it's work.
I need to repeat it again :

nbd flush handle sigterm
nbd flush only finished after tlog finished it's work. In this point, tlog already finished it's work
tlog get sigterm, no problem : it already has nothing to do.

slow 0-stor server, as mentioned above, some improvement at zero-os/0-stor#212

It couldn't be the reason.
If this is the reason, then ndb will log 'flush timed out' instead of finished the flush under SIGTERM handler

Ok if you're so confident about that flow @iwanbk, than I'm out of suggestions for now, yet this bug is still there...

@iwanbk I've just tested with the latest of 0-stor and I still got the same problem

@khaledkbadr you also have the latest of 0-Disk's tlogserver (master)?

Nope, I didn't update that. anything changed there?

I don't know at which version you were building before, so hard to tell for me if something changed. All changes can be found @ https://github.com/zero-os/0-Disk/commits/master, should give you the answer you're looking for. Just best to give all relevant build commits hashes each time you open an issue, and when you give comments where you updated one of them. This prevents confusion related to the version you might be on.

I just tried with both 0-stor and 0-disk master branches and still got the same problem.
0-disk commit: ac3c10967ee69349581eee8cbf536e7add62798c
0-stor commit: ca6146cfada48dfb1e436fd5e3260912047001a2

t=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot read data to write: unexpected EOF" module=nbdserver caller=connection.go:398
t=2017-08-30T06:11:28+0000 lvl=info msg="Receiver exiting for @/vd2" module=nbdserver caller=connection.go:224
t=2017-08-30T06:11:28+0000 lvl=info msg="Worker forced close for @/vd2" module=nbdserver caller=connection.go:634
t=2017-08-30T06:11:28+0000 lvl=info msg="Client @/vd2 cannot write reply: write unix /server.socket.vm0->@: write: broken pipe" module=nbdserver caller=connection.go:205
t=2017-08-30T06:11:28+0000 lvl=info msg="Replyer exiting for @/vd2" module=nbdserver caller=connection.go:188

@GlenDC any idea what happens here?

Seems to be just the end of a connection. It only says it's unexpected EOF because ReadFull (as done in gonbdserver) probably reports that like that.
However that info message is only returned if it can't read data from the open connection, after it has already received the CMD_WRITE command, so that does seem suspicious. But perhaps it's normal, I'm not too sure. Even so, that is an error that was logged before the restore process, and I think at that point the vdisk was still working as expected, so even it it were a problem, it wouldn't explain this (issue's) problem I think.
But it might also be an expected thing, as around that gonbdserver code there is also a check to see if it's not an error that we ourselves send, so it seems it might be expected that at that point a connection ends. But agian, I'm not too sure of this.

Seems to be just the end of a connection. It only says it's unexpected EOF because ReadFull (as done in gonbdserver) probably reports that like that.

I didn't get this in my test.
I worry that it is because the qemu ungraceful shutdown

Your worries could be very quickly resolved in case @khaledkbadr could confirm that after Logs of nbdserver before restore, he can still boot up the vdisk or not.

Your worries could be very quickly resolved in case @khaledkbadr could confirm that after Logs of nbdserver before restore, he can still boot up the vdisk or not.

Ah, yes, you're right. i think @khaledkbadr already did this and it worked

@khaledkbadr could you test with these branchs of 0-disk?

nbd-eol-log : it won't fix the issue, i only added some log there
nbd-eol-experiment : (forked from nbd-eol-log) it might fix your issue or reveal other error(s). Even it fix your issue. it is not the proper solution, it is only a quick dirty hack

I need you to do these because:

it is the part in nbd that i don't fully understand
this part was changed

Thx

nbd-eol-log

t=2017-09-05T08:08:16+0000 lvl=info msg="Starting listening on unix:/server.socket.vm0" module=nbdserver caller=listener.go:151
t=2017-09-05T08:08:16+0000 lvl=info msg="Connect to unix:/server.socket.vm0 from @" module=nbdserver caller=listener.go:165
t=2017-09-05T08:08:16+0000 lvl=info msg="Getting vdisk \"vd2\"" module=global caller=exportcontroller.go:52
t=2017-09-05T08:08:17+0000 lvl=info msg="Negotiation succeeded with @/vd2" module=nbdserver caller=connection.go:609
t=2017-09-05T08:08:19+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:08:19+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:06+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:06+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:06+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:06+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:06+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:06+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:06+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:06+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:11+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:11+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:13+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:13+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:17+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:17+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211
t=2017-09-05T08:09:17+0000 lvl=info msg="BACKEND FLUSH" module=global caller=backend.go:179
t=2017-09-05T08:09:17+0000 lvl=info msg="tlogStorage Flush()" module=global caller=tlog.go:211

i just tested with nbd-eol-experiment branch and still same behaviour after restore

t=2017-09-05T08:18:47+0000 lvl=info msg="vdisk 'vd2' finished the flush under SIGTERM handler. err = flush timeout" module=global caller=backend.go:247

Thx, we can see that we got flush timeout here.
Current timeout in nbd is set to 20 seconds, which i think is too low but worked for ardb based tlog.

@khaledkbadr could you please try these branches:

nbd-fix-close : it should fixes your issue, at least it fixes the issue on my machine. I could finally reproduce it.
nbd-inc-tlog-timeout : not sure  it will fix your issue. I could reproduce it, but not with exact same flow as yours. It works on my machine but maybe not on yours

tested with nbd-fix-close

t=2017-09-05T10:01:43+0000 lvl=info msg="Starting listening on unix:/server.socket.vm0" module=nbdserver caller=listener.go:151
t=2017-09-05T10:01:43+0000 lvl=info msg="Connect to unix:/server.socket.vm0 from @" module=nbdserver caller=listener.go:165
t=2017-09-05T10:01:43+0000 lvl=info msg="Getting vdisk \"vd2\"" module=global caller=exportcontroller.go:52
t=2017-09-05T10:01:44+0000 lvl=info msg="Negotiation succeeded with @/vd2" module=nbdserver caller=connection.go:609
t=2017-09-05T10:05:50+0000 lvl=info msg="Client @/vd2 waiting for inflight requests prior to disconnect" module=nbdserver caller=connection.go:553
t=2017-09-05T10:05:50+0000 lvl=info msg="Client @/vd2 requested disconnect\n" module=nbdserver caller=connection.go:511
t=2017-09-05T10:05:50+0000 lvl=info msg="Receiver exiting for @/vd2" module=nbdserver caller=connection.go:224
t=2017-09-05T10:05:50+0000 lvl=info msg="Worker forced close for @/vd2" module=nbdserver caller=connection.go:634
t=2017-09-05T10:05:50+0000 lvl=info msg="tlogStorage close vdisk `vd2" module=global caller=tlog.go:257
t=2017-09-05T10:05:50+0000 lvl=info msg="GoBackground exited for vdisk: vd2" module=global caller=tlog.go:291
t=2017-09-05T10:05:50+0000 lvl=info msg="error while closing tlog client: close tcp <IP_ADDRESS>:39180-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=tlog.go:294
t=2017-09-05T10:05:50+0000 lvl=info msg="tlogclient : reconnect from read because of : read tcp <IP_ADDRESS>:39180-><IP_ADDRESS>:11211: use of closed network connection" module=global caller=client.go:337
t=2017-09-05T10:05:50+0000 lvl=info msg="tlogclient: reconnect because `read tcp <IP_ADDRESS>:39180-><IP_ADDRESS>:11211: use of closed network connection` failed: client already closed" module=global caller=client.go:343
t=2017-09-05T10:05:50+0000 lvl=info msg="Replyer exiting for @/vd2" module=nbdserver caller=connection.go:188
t=2017-09-05T10:05:50+0000 lvl=info msg="Backend background thread exiting for @/vd2" module=nbdserver caller=connection.go:616
t=2017-09-05T10:05:50+0000 lvl=info msg="Closed connection from @/vd2" module=nbdserver caller=connection.go:596

t=2017-09-05T10:08:26+0000 lvl=info msg="Starting listening on unix:/server.socket.vm0" module=nbdserver caller=listener.go:151
t=2017-09-05T10:08:27+0000 lvl=info msg="Connect to unix:/server.socket.vm0 from @" module=nbdserver caller=listener.go:165
t=2017-09-05T10:08:27+0000 lvl=info msg="Getting vdisk \"vd2\"" module=global caller=exportcontroller.go:52
t=2017-09-05T10:08:27+0000 lvl=info msg="Negotiation succeeded with @/vd2" module=nbdserver caller=connection.go:60

tested with nbd-fix-close

The weird thing here is that i can't see any SIGTERM handler log.
That branch already gave us the expected logs, i increased the timeout now.
(really sorry for this, i really need to reproduce this test on my own with 0-orch).

I still can't see the SIGTERM handler now too

I still can't see the SIGTERM handler now too

i'm afraid that you did different test procedures, as all the update do is increase the timeout value

That's really weird, I just tried again and made sure I used the nbd-eol-experiment branch and got nearly the same logs

That's really weird, I just tried again and made sure I used the nbd-eol-experiment branch and got nearly the same logs

from tlogStorage Close() line in the log, you indeed seems to use the right branch.
But the nbd log must have this line to indicate that SIGTERM handler was working
"vdisk 'vd2' received `Stop` command from vdisk completion" module=global caller=backend.go:232

Since I use the Orchestrator which had the same code for all testing, I believe we still still send the SIGTERM signal. But I don't know why it doesn't react on it

Once this issue is resolved make sure to open a PR with the solution using the 1.1.0-alpha-8 branch as the base branch. Master is from now on meant for beta code.

@khaledkbadr
If you are sure that SIGTERM already delivered, then it seems that nbd's vdisk worker indeed already stopped at the moment SIGTERM came (as we can see in the log, there is tlogStorage Close() there), which could be caused by:

unexpected Close : Close() event that unexpected by nbd's tlogStorage to happen. The Close() handler doesn't have Flush code there, which might be needed in our case. Still checking it.

expected Close : in this case, there is indeed no data to flush anymore, in other words tlog already fully synced with nbd. If this is what happened, the restore should succeed. please confirm that you already tried to restore and failed

I believe it's an expected Close, and I just confirmed that the restore results in the same behaviour.

I believe it's an expected Close

Why you believe this?
the unexpected close didn't necessarily caused by testing procedures or testing code. It could be caused by the interaction between qemu & our nbd server.

Aha, I thought you meant the tlogserver closed, and even that the tlogStorage Close() was there before I stopped the vm (nbdserver, tlogserver). My bad :)

@khaledkbadr i updated nbd-eol-experiment branch, added some handler in case unexpected close happens. Could you please try it again?

Same behavior

I tested with last version of the nbd-eol-experiment where Iwan added more logs
nbd logs before restore

zeroctl restore logs

nbd logs after restore

Also @iwanbk noted that the number of blocks is low since I do 4k block size with 331M ubuntu image

I tested with last version of the nbd-eol-experiment where Iwan added more logs

From the logs above:

nbd closed with last sequence = 17693 and cache empty (or blocks already flushed)
zeroctl restore also restore until 17693 blocks.

From above facts, i think tlog client, tlog server, and tlog replay (restore)  already do it's job.

Also @iwanbk noted that the number of blocks is low since I do 4k block size with 331M ubuntu image

Yeah, i use smaller lede images and have 67K blocks.
The difference between my image and khaled's is that his image is copied image using zeroctl copy while mine is from qemu-img convert
My plan is checking how nbd's tlogStorage work with copied image.

My plan is checking how nbd's tlogStorage work with copied image.

After checking, i am pretty sure that the problem lies in the copied image because it only copied the metadata.
It is true that when reading a disk, nbd also try write to primary cluster if it is not exist yet, which result in tlog transaction being sent to tlog server.
The problem is, booting a VM doesn't mean read the entire disk.Which make the tlog doesn't have all data it's need.
I'm working to provide quick fix as proof of concept to fix this issue.

I'm working to provide quick fix as proof of concept to fix this issue.

The idea is, we also generate tlog transaction when copying vdisk.
We don't copy the tlog's data because not all template have it.
But we can improve it in the future.

@GlenDC
I found that vdisk copy use raw redis connection instead of backend/storage object.
Which mean my plan won't work because it needs to get the block in order to generate tlog transaction.
Any idea for quick fix so we don't block the 0-orch team?

Are you 100% sure this is the issue? As generating tlog transactions during copy doesn't seem like a good idea to me anyhow.

Are you 100% sure this is the issue? As generating tlog transactions during copy doesn't seem like a good idea to me anyhow.

Yes, please read more in telegram chat.
Oh man, we shouldn't discuss in telegram in the first place.
It is the best idea i have now to unblock 0-orch team

@Dinaamagdy tested the zeroctl copy vdisk from the nbd-eol-experiment on packet and got this error
t=2017-09-10T08:03:37+0000 lvl=info msg="generating tlog data for target vdisk `vdisktest`" module=global caller=vdisk.go:104
Error: failed to generate tlog data for vdisk `vdisktest` : ListBlockIndices failed for vdisk `template:ubuntu-1604`: ERR unknown command:evalsha
Usage:
  zeroctl copy vdisk source_vdiskid target_vdiskid [target_clusterid] [flags]

Flags:
      --config SourceConfig   config resource: dialstrings (etcd cluster) or path (yaml file) (default config.yml)
      --data-shards int       data shards (K) variable of erasure encoding (default 4)
  -h, --help                  help for vdisk
      --parity-shards int     parity shards (M) variable of erasure encoding (default 2)
      --priv-key string       private key (default "12345678901234567890123456789012")
      --same                  enable flag to force copy within the same nbd servers

Global Flags:
  -v, --verbose   log available information

failed to generate tlog data for vdisk `vdisktest` : ListBlockIndices failed for vdisk `template:ubuntu-1604`: ERR unknown command:evalsha```

When using hub.gig.tech as templatecluster we get the above error, this is because hub.gig.tech blocks all commands that can do a potential write operation.

this is because hub.gig.tech blocks all commands that can do a potential write operation.

As far as i can see, there is no write operation in ListBlockIndices func.
But we are using Lua script there. I guess this is where the issue is, right?

When using hub.gig.tech as templatecluster we get the above error, this is because hub.gig.tech blocks all commands that can do a potential write operation.

I get that @grimpy, but can that not be something blocked on a more fine-grained level. I mean in the end a Lua script is still only calling redis commands. So is there no way to block this more properly, it would really suck (for you) if we cannot use Lua scripts, which is what the evalsha command is for.

@maxux would it be possible to open this? I mean withouth jeperdazing security, when allowing this command would the lua scripts be able to do write operation?

I'm not asking you to allow write operations @grimpy, as we do not need that ever. No 0-Disk process will ever write to the ETCD Config Cluster or template storage cluster.
I'm asking you to allow lua scripts which do not do any write operations. I think you understand that already, but I want to be clear as your request to @maxux is a bit ambiguous/confusing.

From what I see, ´evalsha` allows to run a "script" server side.
This is okay for me, as soon as this script cannot do more commands than a regular client (aka, no write operation). Can someone test that if a command is not allowed via a regular call, it's denied via the script too ?

How do you do it right now @maxux, I hope you are not manually keeping a witelist of commands you accept. Is there no configuration in the server (that you are using) that allows you to only accept readonly connections?

rename-command FLUSHDB ""
rename-command FLUSHALL ""
rename-command CONFIG ""
rename-command SLAVEOF ""
rename-command DEBUG ""
rename-command CLIENT ""
rename-command EVAL ""
rename-command EVALSHA ""
rename-command IMPORT ""

We deny explictly evalsha, so I guess it's "potentially" dangerous :/

It isn't. I mean it is just calling commands itself. You really should have some way that you accept read-only connections, which allows for any command and blocks write operations (ran directly or indirectly).

In theory, write are denied as soon as SLAVEOF is executed (which is the case).
Can somebody confirm that write/set/any command renamed are not available as well from the script ? I can't test right now :/

Can somebody confirm...

Which server are you guys using? If you link me to the repo of that server technology I don't mind searching the answer for you in its codebase.

Upstream ardb with commands above on the config file

as in https://github.com/yinqiwen/ardb?

Yes

Which server are you guys using? If you link me to the repo of that server technology I don't mind searching the answer for you in its codebase.

@GlenDC any update about this? Do we need to convert the Lua script to Go?

In theory, write are denied as soon as SLAVEOF is executed (which is the case).
Can somebody confirm that write/set/any command renamed are not available as well from the script ? I can't test right now :/

Just tested it.
If my below tests is the right procedure, slaveof will also prevent write operation from the script.
start two ardb servers at port 16386 and 16387
set the 16387 as slave of 16386
<IP_ADDRESS>:16387> slaveof <IP_ADDRESS> 16386
OK

try set operation to 16387, it failed as expected
<IP_ADDRESS>:16387> set a 1
(error) READONLY You can't write against a read only slave.

try set operation using lua script, it failed
<IP_ADDRESS>:16387> eval "return redis.call('set','foo','bar')" 0
(error) ERR @user_script: 1: -READONLY You can't write against a read only slave.

disable slave
<IP_ADDRESS>:16387> slaveof no one
[9045] 09-11 16:51:10,794 INFO [Slave]Replication connection closed at state:INVALID_STATE
[ardb] 0:redis-cli*Z                                                                      [9045] 09-11 16:51:10,794 WARN Remove snapshot file:/tmp/tlog1/backup/sync-lmdb-backup.0.0.1505123164              "siputih" 16:51 11-Sep-17
                                                                                [9044] 09-11 16:51:10,794 WARN Slave ipOK<IP_ADDRESS>,port=16387 closed.

set using lua script
<IP_ADDRESS>:16387> eval "return redis.call('set','foo','bar')" 0
OK

Okay, it's fine to open EVALSHA on the hub for me :)

@maxux Can you notify me when EVALSHA is open in the hub so we can test?

@iwanbk you don't seem to test if a read-only command from a lua script does work on a read-only slave.

As QA, @Dinaamagdy, just verified that the rollback finally worked on packet with a reasonable time for the create and the rollback.

Fixed by merging PR #453 into the 1.1.0-alpha-8 branch, which will be used to make the official alpha-8 release as soon as the beta-1 milestone has come to an end.


Missing tsconfig.json

Since project is using typescript, what's reasoning to exclude tsconfig.json from git? One of the issues is that test for useOnClickOutside doesn't pass because TS is not configured to handle JSX by default.
Sample error when running tests without compilerOptions.jsx = "react-jsx"
src/__tests__/useOnClickOutside.spec.tsx:9:9 - error TS17004: Cannot use JSX unless the '--jsx' flag is provided.

added to git


Requested features preset properties.

Requested features preset properties. Similarly, as is done in a Blender Game Engine.

You mean http://docs.godotengine.org/en/latest/reference/gdscript.html#exports ?

Thanks!


Private repos are not showing up

Hi private repositories are not showing up on the drop down list and users cannot be selected.

Fixed on master!


Add missing bower.json

Adding the missing bower.json would be very nice to work with bower. It can download the project but it's not able to know which file to include, which is the exact purpose of the "main" property of the bower.json.
For more infos on how to do it, refer to the following issue: https://github.com/CreateJS/SoundJS/issues/76

Had the same problem with bower.json's "main" property. Our scripts rely on that property to build the project - similar with what @metaman described.
Unfortunately we had to rely on big.js but the development is hindered due to it's reduced feature set.
Would love to see an update on this.

@nicolaes Okay, I'll add a bower.json tomorrow.

Thanks @MikeMcl .
As bower automatic dependencies are almost always used with build scripts that include minification I suggest going with "main": "bignumber.js" - as you did in the big.js library :)
If there is need I can also lend a hand with the development around, by the way.

bower.json added in v2.0.6


Rework data setup

Move neo4j Docker to named volume

add less and other tools to that container to allow log inspection

Current data/ become's app's data/, so just data/import instead of data/app/import.

Did this in #23.


Implementation of index from Navigator

This PR moves implementation of ResourceIndex from Navigator app.

Looks like we need to move this class too:
https://github.com/ARK-Builders/ARK-Navigator/blob/main/app/src/main/java/space/taran/arknavigator/mvp/model/repo/index/ResourcesIndexRepo.kt

Superseded by https://github.com/ARK-Builders/ARK-Retouch/pull/20


Add more specificity to script syntax errors message.

Had a guy in the IRC today asking why he no longer got a specific error message when his script contains syntax errors. I pinpointed the change to commit 212f0170ff20deaa9cca984a30e89f5873421937 . I suggest passing the error message from /ErrorStdOut into the extra parameter of Util_Error.

So, essentially a more specific and elaborate error message. Good idea! :+1:


NIFI-9430: Create initial C2 structure, add c2-protocol-api module

Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Adds a top-level "c2" module and populates the c2-protocol-api submodule with API classes. Also updated MiNiFi to use the same version of Swagger that NiFi does. Unit tests are not included as the API classes are basically beans.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically main)?

[x] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[x] Have you verified that the full build is successful on JDK 8?
[ ] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

Yeah that makes more sense, let's divide it up by functionality, maybe an agent one too and a single common one they both inherit from? Then users of the libraries can decide which dependencies they need without including classes they don't want.

Re lombok...
Just my two-cents, but I would probably lean against using it for classes that we consider API classes. I would rather us maintain control over the structure, rather than depending on code generation.
In other cases I would say I am neutral, I think it comes down to a lot of personal preference in terms of how much "magic" you like. I tend to prefer just using the IDE to generate getters/setters/equals/toString/etc, but then having the full class available like normal.
Re nifi-api...
I view nifi-api as the highest level API we have for the overall nifi Java ecosystem. It has the interfaces for the extensions that are run in NiFi and MiNiFi Java (Processor, ControllerService, ReportingTask), as well as the VersionedComponent data model for registry/flow definitions.
Part of me almost suggested that the component/extension manifest API could become part of nifi-api since it is essentially a model that can be produced from extensions/components that implement the interfaces in nifi-api, but I didn't want to complicate it too much. So I think depending on it should be ok.

I'm going to change the base branch to main, that way we can get the API into the NiFi codebase for community developers to start working with earlier. The feature branch NIFI-9428 (once rebased) can then be used for work on the server and client and such.

Seems like we have reached consensus here with several approvals, going to merge this, thanks @mattyb149 !!!


Redirect_uri should be required, if the "redirect_uri" param was included in the authorization request

In authorization_code grant, redirect_uri is required, if the "redirect_uri" param was used in the authorization request.
From https://tools.ietf.org/html/rfc6749#section-4.1.3
redirect_uri is required,if the "redirect_uri" parameter was included in the
authorization request as described in Section 4.1.1, and their values MUST be identical.

I think this is how the grant currently works: https://github.com/thomseddon/node-oauth2-server/blob/master/lib/authCodeGrant.js#L92-L96 ?
Closing as it's taken me so long to reply, please feel free to re-open if needed :)


[bug]: The BatchOpenChannel command has stopped working on some servers (cli and API)

Background
The BatchOpenChannel command has stopped working on some servers (cli and API). The lncli batchopenchannel is just hanging, there is no mention of work in the logs. The API also hangs and calls are no longer shown in the logs.
On other servers, the lncli works with the same version of LND.
Your environment

version of lnd 0.17.0-beta
version of btcd, bitcoind, or other backend - bitcoind v23.0

Steps to reproduce
Now I have a suspicion what could have caused the problem.
In the last two days, I have been massively opening channels through the BatchOpenChannel API (in LND v0.16.4-beta!). But not so long ago I noticed strange messages in the logs. They said that an open channel was not found, although transaction confirmations had more than 9 confirmations. For example:
2023-10-03 23:58:32.153 [ERR] NANN: Unable to retrieve chan status for Channel(be9c2008ad15ab4dbfe34aab8f9829410f7a342479416d2715d1eba5feb08145:7): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(43cb005583d44ce8e62ec8f570e5e8b48609ff166a5d642ad88493a0009c4736:2): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(4cbde5a3c3b171d78417615093c9a9e1fdc5d18104640539ab03c69afcdc0d84:4): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(be9c2008ad15ab4dbfe34aab8f9829410f7a342479416d2715d1eba5feb08145:9): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(7219175edcc8d63018d094e6c3e45c5edfb0bb889207ede2306514878cb2e968:8): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(6b374e693fcc44e54167b958a70568c16dc6a7c01d4f3e03de58ed5caf622fb5:1): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(4cbde5a3c3b171d78417615093c9a9e1fdc5d18104640539ab03c69afcdc0d84:6): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(c62ac55a1ecc1b5b9a36fabd6824d1c83cbe4191aae5b8581bfc225bf11be4e8:1): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(e5cadc0a20427063e245df9354188f908eb338b8dfedead187cad30741117be9:4): edge not found
2023-10-03 23:58:32.154 [ERR] NANN: Unable to retrieve chan status for Channel(7219175edcc8d63018d094e6c3e45c5edfb0bb889207ede2306514878cb2e968:0): edge not found

and some times these lines:
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(d24d2ebc5d6076ef211749ee8fc8377c35f395bd11764f6087bf4235b59ab88f:1) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(330c9be65a9352b84d67c36271e0b4801ff0d14328b5dfe33a59e5f30280dc5c:0) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(abb7dcbdf515266a788b6acfd636cf18b88bf847de190e9b966334ce6f3cc241:8) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(deea0d713701e99bb0d1b42663d74db67c8b3ecc0ca4ecdc1fc6c30b35836fce:1) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(08598d122f0011532b317d0c1f2813a76c4249cf00e9b77736f7dce086ecab15:1) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(ef890c00273efbf5e5788c01b1ff2f3ea1bc88eceb5ad93c337309c6e066bd8b:1) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(242180d40ead79447a2cbd9766e1f1804a067591a1d041ce9826f083bed021e5:1) w/ newer version
2023-10-03 23:06:00.312 [WRN] CHBU: Replacing disk backup for ChannelPoint(ae079b910e45403c5cfab3f0e20c3d870a119b3e967a2b5097c3c8372ad9c486:0) w/ newer version

and so on again and again...
All channels were opened via BatchOpenChannel without errors, but LND stubbornly considered them inactive. It was like this all the time until I restarted LND (v0.16.4-beta).
But the restart did not go without problems. LND wrote that it was finishing work, but the process was not completed and logs were being written endlessly. At that time, I even found an issue about a similar problem.
In logs at that time:
2023-10-04 00:33:33.324 [ERR] DISC: Update edge for short_chan_id(887563169359200257) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(885802851401728001) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(883983159864262657) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(864076501498068993) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(878707702715252737) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(890017279411355648) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(888167900829581313) got: router shutting down
2023-10-04 00:33:34.626 [ERR] DISC: Update edge for short_chan_id(864199646836948993) got: router shutting down
2023-10-04 00:33:34.722 [ERR] DISC: router shutting down
2023-10-04 00:33:34.749 [ERR] DISC: Update edge for short_chan_id(889978796506939392) got: router shutting down
2023-10-04 00:33:34.749 [ERR] DISC: Update edge for short_chan_id(854406296714608640) got: router shutting down
2023-10-04 00:33:34.756 [ERR] DISC: router shutting down
2023-10-04 00:33:34.992 [ERR] DISC: Update edge for short_chan_id(740458409203924992) got: router shutting down
2023-10-04 00:33:37.523 [ERR] DISC: router shutting down
2023-10-04 00:33:38.098 [ERR] DISC: Update edge for short_chan_id(888638491870887937) got: router shutting down
2023-10-04 00:33:38.224 [ERR] DISC: Update edge for short_chan_id(888591212870762497) got: router shutting down
2023-10-04 00:33:38.224 [ERR] DISC: Update edge for short_chan_id(889716013129203713) got: router shutting down
2023-10-04 00:33:38.225 [ERR] DISC: Update edge for short_chan_id(883732470891872257) got: router shutting down
2023-10-04 00:33:38.225 [ERR] DISC: Update edge for short_chan_id(883983159864262657) got: router shutting down
2023-10-04 00:33:38.225 [ERR] DISC: Update edge for short_chan_id(885184925730930689) got: router shutting down
2023-10-04 00:33:38.225 [ERR] DISC: Update edge for short_chan_id(890465880186093569) got: router shutting down
2023-10-04 00:33:38.480 [ERR] DISC: router shutting down
2023-10-04 00:33:39.612 [ERR] DISC: Update edge for short_chan_id(874909989597806593) got: router shutting down
2023-10-04 00:33:39.612 [ERR] DISC: Update edge for short_chan_id(840313856276627457) got: router shutting down
2023-10-04 00:33:39.612 [ERR] DISC: Update edge for short_chan_id(888969444749279240) got: router shutting down
2023-10-04 00:33:39.613 [ERR] DISC: Update edge for short_chan_id(862184241978998785) got: router shutting down
2023-10-04 00:33:39.613 [ERR] DISC: Update edge for short_chan_id(890998043747287049) got: router shutting down

I completed the lnd process only after the kill -TERM pid command
After restarting, all those channels that were inactive and about which it wrote Unable to retrieve chan status for Channel(channel_point:N): edge not found - they became active. So everything worked until today, after which I made an update to v0.17.0-beta.
After the update, I started running BatchOpenChannel commands again and they hang. It's as if some locks have been installed somewhere in the database itself.
The same servers where I did not open channels so massively (maybe the API call speed was lower) - there was no such thing in the logs and everything is working under version v0.17.0-beta there now

I have now carefully studied the logs of LND v0.16.4-beta, when I opened channels via BatchOpenChannel yesterday and found that the problem began to manifest itself in version 0.16.4-beta.
Yesterday the last API call log was:
2023-10-03 22:58:11.225 [ERR] RPCS: [/lnrpc.Lightning/BatchOpenChannel]: batch funding failed: error batch opening channel, initial negotiation failed: remote canceled funding, possibly timed out: received funding error from 0227d47539ebbfbb3548abdb31a243063bf2ff2dd77da2af04404da61ab8264560: chan_id=b1d87abd81b8a4a2>
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=03f3643570433b67f22ef1c2e7660e33c9b9d53f786cb1779544983aa5c8f286e3, pending_chan_id=c6b5d948606fd4e5c0f056cc4c79006e7ff307b6db041507b889c054b58eff3d: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.225 [INF] FNDG: Cancelling funding reservation for node_key=03f3643570433b67f22ef1c2e7660e33c9b9d53f786cb1779544983aa5c8f286e3, chan_id=c6b5d948606fd4e5c0f056cc4c79006e7ff307b6db041507b889c054b58eff3d
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=03808b1a698906bbf0457922aec66168a8edbfe98b1379249ace270b41ae0dde48, pending_chan_id=2e34146484aa6420f6bd0b0efe3ba05d2fb79100a33a81ae80e80bbc78153a46: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.225 [INF] FNDG: Cancelling funding reservation for node_key=03808b1a698906bbf0457922aec66168a8edbfe98b1379249ace270b41ae0dde48, chan_id=2e34146484aa6420f6bd0b0efe3ba05d2fb79100a33a81ae80e80bbc78153a46
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=03dd09855c3634daa5c7de0146f4d54eb395b51ae422bc86ee1132a0d7ed50e720, pending_chan_id=ee0272150e58339b143f9aa3efeaef18ef0f39ca933fd11ed4f4e69fb4a3fc8c: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=03dd09855c3634daa5c7de0146f4d54eb395b51ae422bc86ee1132a0d7ed50e720, chan_id=ee0272150e58339b143f9aa3efeaef18ef0f39ca933fd11ed4f4e69fb4a3fc8c
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=02abee897f5ab347fb917b30bf184c4b3ee33bd81d568958193f0cf3b7f7b41b44, pending_chan_id=f3002a81908bdcb25caa319859e6fc89e46657d7b780ab5c9c5bc18883d4b9a5: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=02abee897f5ab347fb917b30bf184c4b3ee33bd81d568958193f0cf3b7f7b41b44, chan_id=f3002a81908bdcb25caa319859e6fc89e46657d7b780ab5c9c5bc18883d4b9a5
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=0288037d3f0bdcfb240402b43b80cdc32e41528b3e2ebe05884aff507d71fca71a, pending_chan_id=221c7998b3c37b2e65f200cd4f8e96e89b62b91085124986a1669cb597038b90: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=0288037d3f0bdcfb240402b43b80cdc32e41528b3e2ebe05884aff507d71fca71a, chan_id=221c7998b3c37b2e65f200cd4f8e96e89b62b91085124986a1669cb597038b90
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=022eb09a7993a0edde69537b420f4119c0de833e0ee47651753b46bf884db75235, pending_chan_id=8520f0bf1d2b443bdb1f20496d49190a74631aea6a3e1568bef1a884a4cfc36d: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=022eb09a7993a0edde69537b420f4119c0de833e0ee47651753b46bf884db75235, chan_id=8520f0bf1d2b443bdb1f20496d49190a74631aea6a3e1568bef1a884a4cfc36d
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=03a09f56bba3d2c200cc55eda2f1f069564a97c1fb74345e1560e2868a8ab3d7d0, pending_chan_id=5b79f5527b6717556da404a66305389c5de1008e23c88bcef36668db0abfacdd: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=03a09f56bba3d2c200cc55eda2f1f069564a97c1fb74345e1560e2868a8ab3d7d0, chan_id=5b79f5527b6717556da404a66305389c5de1008e23c88bcef36668db0abfacdd
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=038ba8f67ba8ff5c48764cdd3251c33598d55b203546d08a8f0ec9dcd9f27e3637, pending_chan_id=d56cc1313ab78b0a79105a743d8153c531866e10ac2a2770fecd0e88a7175a05: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=038ba8f67ba8ff5c48764cdd3251c33598d55b203546d08a8f0ec9dcd9f27e3637, chan_id=d56cc1313ab78b0a79105a743d8153c531866e10ac2a2770fecd0e88a7175a05
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=0223d7aa491e26a307b4203916b8d60ded8cfe832806b432d6cfc278e96374f9aa, pending_chan_id=3cbe47cf151c6a1bbbe447e0c3d0e8bf90bfa84fc3bd5e5f4264a2e07a8e9c8d: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=0223d7aa491e26a307b4203916b8d60ded8cfe832806b432d6cfc278e96374f9aa, chan_id=3cbe47cf151c6a1bbbe447e0c3d0e8bf90bfa84fc3bd5e5f4264a2e07a8e9c8d
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=02a98e8c590a1b5602049d6b21d8f4c8861970aa310762f42eae1b2be88372e924, pending_chan_id=64b0ec4d8acb63a976097a66e8ce6bbbcb86756f5d114a7fe08b163439cefe89: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=02a98e8c590a1b5602049d6b21d8f4c8861970aa310762f42eae1b2be88372e924, chan_id=64b0ec4d8acb63a976097a66e8ce6bbbcb86756f5d114a7fe08b163439cefe89
2023-10-03 22:58:11.225 [ERR] FNDG: Unable to handle funding accept message for peer_key=020f7c502e19ccb375d4abc689f2c1feb6816961d4a1e2dcfa8813f11c3bc9a5e1, pending_chan_id=409d790795774ba70f7e6e4939242ef4e5f84bcf1a0a46797902bc12709b2fd0: aborting PSBT flow: user canceled funding
2023-10-03 22:58:11.226 [INF] FNDG: Cancelling funding reservation for node_key=020f7c502e19ccb375d4abc689f2c1feb6816961d4a1e2dcfa8813f11c3bc9a5e1, chan_id=409d790795774ba70f7e6e4939242ef4e5f84bcf1a0a46797902bc12709b2fd0
2023-10-03 22:58:11.238 [INF] FNDG: Initiating fundingRequest(local_amt=0.16777215 BTC (subtract_fees=false), push_amt=0 mSAT, chain_hash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, peer=02abee897f5ab347fb917b30bf184c4b3ee33bd81d568958193f0cf3b7f7b41b44, min_confs=1)
2023-10-03 22:58:11.273 [INF] FNDG: Target commit tx sat/kw for pendingID(cd4ceebc3ab38463dfbba15e1f06448072b1706422d223ed419b25098b89c6a1): 2500
2023-10-03 22:58:11.273 [INF] FNDG: Dust limit for pendingID(cd4ceebc3ab38463dfbba15e1f06448072b1706422d223ed419b25098b89c6a1): 0.00000354 BTC
2023-10-03 22:58:11.273 [INF] FNDG: Starting funding workflow with <IP_ADDRESS>:13923 for pending_id(cd4ceebc3ab38463dfbba15e1f06448072b1706422d223ed419b25098b89c6a1), committype=anchors-zero-fee-second-level
2023-10-03 22:58:11.273 [INF] FNDG: Initiating fundingRequest(local_amt=0.16777215 BTC (subtract_fees=false), push_amt=0 mSAT, chain_hash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, peer=0227d47539ebbfbb3548abdb31a243063bf2ff2dd77da2af04404da61ab8264560, min_confs=1)
2023-10-03 22:58:11.318 [INF] FNDG: Target commit tx sat/kw for pendingID(cbfbb8fe731eb6d7e43454116b82270719b22f7f9fddab7561f24f822686068c): 2500
2023-10-03 22:58:11.318 [INF] FNDG: Dust limit for pendingID(cbfbb8fe731eb6d7e43454116b82270719b22f7f9fddab7561f24f822686068c): 0.00000354 BTC
2023-10-03 22:58:11.318 [INF] FNDG: Starting funding workflow with qg6fwphvo5n2i345bkyhwwvd3sh5vd2ekvdwpjtbgsl7f7z4sgltwgyd.onion:9735 for pending_id(cbfbb8fe731eb6d7e43454116b82270719b22f7f9fddab7561f24f822686068c), committype=anchors-zero-fee-second-level
2023-10-03 22:58:11.318 [INF] FNDG: Initiating fundingRequest(local_amt=0.16777215 BTC (subtract_fees=false), push_amt=0 mSAT, chain_hash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, peer=03f3643570433b67f22ef1c2e7660e33c9b9d53f786cb1779544983aa5c8f286e3, min_confs=1)
2023-10-03 22:58:11.363 [INF] FNDG: Target commit tx sat/kw for pendingID(f67b6c51745f8e3d71ab92941dd3b4e3b9a3da4fa389cd78353a2d23a8c81d5c): 2500

That is, in the logs there is already a response from BatchOpenChannel (2023-10-03 22:58:11.225 [ERR] RPCS: [/lnrpc.Lightning/BatchOpenChannel]: batch funding failed: error batch opening channel, initial negotiation failed: remote canceled...)
Then my scripts called this function 7-8 more times later, but there is no mention of it in the LND's logs anymore. Since my scripts have a timeout of 2 minutes, after which I forcibly end waiting for the synchronous output of the BatchOpenChannel, then I came out of waiting and called a new call with another channel batch opening. Then there was a timeout again and I tried a new package and so on. It turns out that RPC stopped working with this call already at that time (when I had v0.16.4-beta).
By the time of the last call, which is recorded in the LND logs, I had run out of confirmed UTXO (I open now channels only from confirmed funds), which can be seen in the logs above: aborting PSBT flow: user canceled funding...
It is possible that the end of the confirmed funds, and then subsequent calls of the next attempts to open channels led to some internal locks that remained and have remained until now, even after updating the version to 0.17.0-beta

Important addtition now:
After above mentioned line:
2023-10-03 22:58:11.225 [ERR] RPCS: [/lnrpc.Lightning/BatchOpenChannel]: batch funding failed: error batch opening channel, initial negotiation failed: remote canceled funding, possibly timed out: received funding error from 0227d47539ebbfbb3548abdb31a243063bf2ff2dd77da2af04404da61ab8264560: chan_id=b1d87abd81b8a4a2>

my script parsed that problem was with the peer 0227d47539ebbfbb3548abdb31a243063bf2ff2dd77da2af04404da61ab8264560 there was error and at the same moment, it excluded this peer 0227...264560 from the package, formed a new package from the same nodes, with the exception of the problematic one, but at the same time added one new peer and immediately made a call BatchOpenChannel. But, as I see from the logs, the returned error code does not complete the BatchOpenChannel transaction (Cancelling funding reservation for node_key=03f3643570... like this one), and the command itself continued to make cancellations with nodes. But at that moment a new opening command came with the same peers. It seems to me that a deadlock or something similar could have taken place.

I restarted the LND v0.17.0-beta and now the BatchOpenChannel works there!
But may be there is bug....

If after a restart the batch open suddenly works, that is just more indication that there is a deadlock somewhere that holds the main server mutex (which apparently the batch channel open also waits for). So as mentioned in https://github.com/lightningnetwork/lnd/issues/7928, getting a goroutine dump (using /debug/pprof/goroutine?debug=2 instead of ?debug=1 as mentioned here) when the lockup occurs would be extremely helpful.

Closing this issue, as there has been a few fixes made to address concurrency issues for openchannel rpc. If the issue reoccurs with the latest release, feel free to reopen. thanks.


What is a textfile

I'd really like this page extending to define exactly what is a textfile, how is it formatted etc.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 84f987fd-0a70-4781-0810-20443b81fd63
Version Independent ID: b6121095-795a-83cd-9d9f-f4b4c1ea4abc
Content: How to: Read Text from Files with a StreamReader (Visual Basic)
Content Source: docs/visual-basic/developing-apps/programming/drives-directories-files/how-to-read-text-from-files-with-a-streamreader.md
Product: dotnet-visualbasic
GitHub Login: @KathleenDollard
Microsoft Alias: kdollard

Closing due to the lack of response.


Handling Dates

Feature Request
Hi everyone !
While trying to generate OpenAPI/Swagger docs using classes with native JS Date attributes, Nestia converts it to empty objects in the Swagger output. I don't understand well how Nestia understands types yet, but there must be a way to convert Date attributes to Swagger's standard date or date-time.
Currently, Date attributes in classes/interfaces give this result in the Swagger output
"Date": {
    "type": "object",
    "properties": {},
    "nullable": false,
    "description": "Enables basic storage and retrieval of dates and times."
},

In the OpenAPI specs, the standard way to handle dates is the following
"Date": {
    "type": "string",
    "format": "date-time", // date | date-time
    "nullable": false,
    "description": "Enables basic storage and retrieval of dates and times."
},

You can find more details in Swagger's documentation
https://swagger.io/docs/specification/data-models/data-types/#string
Nestia Version used : 3.0.16

Hello everyone!
Commenting here because I have a related issue with Date handling when generating the swagger.json file.
We use NestJS payload object transformation (https://docs.nestjs.com/techniques/validation#transform-payload-objects). For example, a DTO would look like this:
import { IsNotEmpty } from 'class-validator';

export class MyObjectCreateDTO {
   /**
   * Start date
   * @example 21/12/2022
   */
  @IsNotEmpty() dateStart!: Date;
}

The Date is handled automatically by NestJS which is transforming the string received in the payload to a Date object. I tried to adding @format date-time but then I get the following error:
Error on typia.MetadataTagFactory.generate(): format requires string type, but no "MyObjectCreateDTO.dateStart"
Is there any way to override this validation and force the swagger generation to ignore the Date object type?
Thank you very much!

Now no problem through update of typia.


Mercy DMG Boost

Created by: DpsDoomMain
Mercy’s damage boost should be at 30% currently it’s at 25%

moved from ow1_emulator_backup#655

moved to ow1_emulator#79

Created by: DpsDoomMain
Question can you see who she has healing beam on?

Created by: MaxwellJung
Not sure if this is possible. There's currently no way to check who's getting damage boosted by mercy in workshop.


Updated .openpublishing.redirection file

Updated redirects in .openpublishing.redirection.json.

Docs Build status updates of commit 2bfb318:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 2bfb318:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

#sign-off and merged.


Fixing some Test issues.

CampPlatform was added.
Fixed an error on testDslInChatApplication.

It is added in #19


Why export button is not working in Video Annotation  example?

Hey, thank you for testing and contributing to wavesurfer.js!
Please make sure you can check all of these points below before opening an issue:
(You don't have to post this section)

[x] I have checked the FAQ and it doesn't solve my problem.
[x] I have checked the documentation and it doesn't solve my problem
[x] I have searched for already open issues which desribe my problem.
[x] The issue I'm having is related to and caused by wavesurfer.js, not by other software (which maybe packages and uses wavesurfer incorrectly) – In that case you should open the issue on the respective project pages.

Please make sure you provide the following information (if applicable):
Wavesurfer.js version(s):
Browser and operating system version(s):
Code needed to reproduce the issue:
(Please reduce your code as much as possible and only post the minimum code needed to reproduce the issue. A Code pen is an excellent way to share such code)
Use behaviour needed to reproduce the issue:

I want to export or download a particular section of audio.How to do it ?


could not load platform certs

FYI, I performed gdrive account import on remote for account added locally
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-rustls-0.23.2/src/config.rs:31:62:
could not load platform certs: Os { code: 2, kind: NotFound, message: "No such file or directory" }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Hi @ysharma1126 Did you manage to resolve the issue and if so how? Thx.

thread 'main' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-rustls-0.23.2/src/config.rs:31:62:
could not load platform certs: Custom { kind: Other, error: Error { code: -25262, message: "The Trust Settings Record was corrupted." } }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace


Add remove flags for service update

Following up on #23701
Updating array and mapping types by starting with an empty collection is fine when there are a small number of items, but it becomes a large burden for users if they have many ports, labels, or mounts, and would like to just add or remove a single one. The case of just adding or removing a single item should be a lot more common than removing everything.
This PR adds --remove-x flags and restores the old merge behaviour for these types.
Fixes #24064
cc @icecrime

Does it make sense to replace the value if the mount target or exposed port matches?

I think so, but I suspect that is done by the daemon somewhere already.

Design LGTM

Sorry but i find this really confusing for users. I'd much rather have update --label +foo=bar and update --label -foo

Sorry but i find this really confusing for users. I'd much rather have update --label +foo=bar and update --label -foo

Would it be sufficient to support --label -foo=bar to be the equivalent of --label -foo, or make the latter remove only the specified instance? For example, --label -foo would remove foo=bar or foo=baz, but --label -foo=bar would only remove foo=bar, but not foo=baz?

I think the "accept key" or "accept full value" is a much smaller part of the issue. Supporting both would be fine with me.
I think the larger issue is to not wipe out the entire structure when you want to add or remove a single value from it, and how to do that with command line flags.
My feeling is that using - and + prefixes, which it seems simple, is not easy to discover. It also restricts the values we can use in the future because we now have these restrictions for updates.
Using --remove-x is obvious (you can see it in --help), and should be explicit (it's clear what you're try do).

Changed the update/add flags to be --x-add and the remove flags to --x-rm

My feeling is that using - and + prefixes, which it seems simple, is not easy to discover. It also restricts the values we can use in the future because we now have these restrictions for updates.

I agree.

Changed the update/add flags to be --x-add and the remove flags to --x-rm

I'm happy with this solution.
Bikeshedding a little, but why this over --add-x / --remove-x? Putting the verb at the end feels backwards. I feel like I would type --add-x a lot and have to correct myself.
It probably doesn't matter much though.

Putting the verb at the end feels backwards.

The primary argument for this (based on some discussion) was that it makes --help display a lot better. The add and remove flags will be next to each other.
At first the prefix form felt more natural to me, but after implementing it this way, I think I'm happy with it as a suffix.

This has been updated and is ready for another review

I've removed the short flags, and rebased

LGTM

Janky hitting #23401

@dnephin needs a rebase, but overall looks good

/ping @dnephin 👼

This change doesn't allow multiple labels update within single --label flag or --label-add flag. But it allows removal of multiple labels with one --label-rm flag. Is this expected behavior?
Ping @dnephin , @thaJeztah.
dchen@vm2:~/docker-1.12$ docker service create --name test1 --label k1=v1,k2=v2 redis
11hma3k87jszuk9kv0b4lfqoo
dchen@vm2:~/docker-1.12$ docker service inspect test1 | grep -i label -A 2
            "Labels": {
                "k1": "v1,k2=v2"
            },
dchen@vm2:~/docker-1.12$ docker service update --label-add k3=v3 --label-add k4=v4 test1
test1
dchen@vm2:~/docker-1.12$ docker service inspect test1 | grep -i label -A 4
            "Labels": {
                "k1": "v1,k2=v2",
                "k3": "v3",
                "k4": "v4"
            },
dchen@vm2:~/docker-1.12$ docker service update --label-rm k1,k3 test1
test1
dchen@vm2:~/docker-1.12$ docker service inspect test1 | grep -i label -A 2
            "Labels": {
                "k4": "v4"
            },

dchen@vm2:~/docker-1.12$  docker version
Client:
 Version:      1.12.0-dev
 API version:  1.25
 Go version:   go1.6.2
 Git commit:   e7f699b
 Built:        Tue Jul 12 01:39:09 2016
 OS/Arch:      linux/amd64

dchen@vm2:~/go/src/github.com/docker/docker$ git log -n 3 --oneline
e7f699b Change the add/update flags to include 'add'
64ba8c6 PR feedback
364de29 Add remove flags for service update

@dongluochen I agree, it's not consistent; left a comment here; https://github.com/docker/docker/pull/24397#issuecomment-232114842
ping @tiborvass for UX ^^

temporarily moving back to design review, pending a decision on the comma-separated notation

The same Env parameter can be added multiple times. And it goes out of range at env-rm.
dchen@vm2:~/go/src/github.com/docker/docker$ docker service create --name test1 --env PATH redis
6ciwaqa71m1reis5uhe3aeafw

dchen@vm2:~/go/src/github.com/docker/docker$ docker service inspect test1 | grep -i env -A 4
                    "Env": [
                        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/go/bin:/home/dchen/go/bin:/home/dchen/go/src/github.com/docker//swarm-v2/bin"
                    ]
                },

dchen@vm2:~/go/src/github.com/docker/docker$ docker service update --env-add SHELL --env-add PATH test1
test1

dchen@vm2:~/go/src/github.com/docker/docker$ docker service inspect test1 | grep -i env -A 5
                    "Env": [
                        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/go/bin:/home/dchen/go/bin:/home/dchen/go/src/github.com/docker//swarm-v2/bin",
                        "SHELL=/bin/bash",
                        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/go/bin:/home/dchen/go/bin:/home/dchen/go/src/github.com/docker//swarm-v2/bin"
                    ]
                },

dchen@vm2:~/go/src/github.com/docker/docker$ docker service update --env-rm PATH test1
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0xc37a00, 0xc820010090)
	/usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/docker/docker/api/client/service.updateEnvironment(0xc82007ea20, 0xc8200d2428)
	/go/src/github.com/docker/docker/api/client/service/update.go:270 +0x5bb
github.com/docker/docker/api/client/service.updateService(0xc82007ea20, 0xc8200d23c8, 0x0, 0x0)
	/go/src/github.com/docker/docker/api/client/service/update.go:152 +0x30f
github.com/docker/docker/api/client/service.runUpdate(0xc820296b80, 0xc82007ea20, 0x7ffef5bb57bf, 0x5, 0x0, 0x0)
	/go/src/github.com/docker/docker/api/client/service/update.go:63 +0x221
github.com/docker/docker/api/client/service.newUpdateCommand.func1(0xc820306000, 0xc820326b70, 0x1, 0x3, 0x0, 0x0)
	/go/src/github.com/docker/docker/api/client/service/update.go:28 +0x6e
github.com/spf13/cobra.(*Command).execute(0xc820306000, 0xc820326ae0, 0x3, 0x3, 0x0, 0x0)
	/go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:593 +0x6fe
github.com/spf13/cobra.(*Command).ExecuteC(0xc8200818c0, 0xc820306000, 0x0, 0x0)
	/go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:683 +0x55c
github.com/spf13/cobra.(*Command).Execute(0xc8200818c0, 0x0, 0x0)
	/go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:642 +0x2d
github.com/docker/docker/cli/cobraadaptor.CobraAdaptor.run(0xc8200818c0, 0xc820296b80, 0x7ffef5bb57a2, 0x7, 0xc82000a0e0, 0x4, 0x4, 0x0, 0x0)
	/go/src/github.com/docker/docker/cli/cobraadaptor/adaptor.go:119 +0x25f
github.com/docker/docker/cli/cobraadaptor.CobraAdaptor.Command.func1(0xc82000a0e0, 0x4, 0x4, 0x0, 0x0)
	/go/src/github.com/docker/docker/cli/cobraadaptor/adaptor.go:127 +0x8c
github.com/docker/docker/cli.(*Cli).Run(0xc820326570, 0xc82000a0d0, 0x5, 0x5, 0x0, 0x0)
	/go/src/github.com/docker/docker/cli/cli.go:81 +0x34b
main.main()
	/go/src/github.com/docker/docker/cmd/docker/docker.go:72 +0x599

For UX, let's not complicate and not implement --label-rm k1,k2 and have only --label-rm k1 --label-rm k2 for now. OK?

The comma separated values are the default. Not accepting them makes it more complicated.
You can always do --label-rm k1 --label-rm k2 . StringSliceVar() supports both by default.

@dnephin but --label does not support comma-separated values;
docker run -dit --label foo=bar,bar=baz --name foobar alpine sh
docker inspect --format "{{json .Config.Labels }}" foobar

{"foo":"bar,bar=baz"}

@dnephin do we have something else than StringSliceVar? I really don't want --label-rm k1,k2.

Sure, we can use ListOpt

@dnephin perfect, let's do that then :)

Now using ListOpts for rm flags

The same Env parameter can be added multiple times. And it goes out of range at

Thanks, fixed in the last commit.

LGTM

LGTM 🐸

I'll make a follow-up for docs update (that way we don't run the CI again)


Change 'make serve' target in ImageProc to create a https server

Image processing demo in ImageProc directory currently creates a SimpleHTTPServer to test the compiled NaCl executable.  As of Chrome 47 getUserMedia is only supported from secure origins: https URLs or localhost.
For ease of testing it would be good if the 'make serve' Makefile target created a local https server - currently if I want to test the executable on my Chromebook I need to deploy the .nexe file to my remote webserver.

I used the web server extension for chrome and it worked great.
Just go to the extension url: https://chrome.google.com/webstore/detail/web-server-for-chrome/ofhbbkphhbklhfoeikjpcbhemlocgigb
Add the extension, enable it, and point it to the ImageProc folder, and it should work fine.


Improve discoverability of Contribution docs

This project, as many open-source projects that don't have the muscle of some investment firm backing it, has to rely on the community.
Community helps with feedback, stimulates organic growth, creates and enforces the rules in the social network, and finally, it helps with building a product.
So those who are willing to help in various capacities should be exposed to information on how they could contribute as soon as they join the Bastyon.
There are a coulpe of ways that come to mind:

A message with links, as soon as a user joins, similar to what people receive today about the secure messenger.
A new user is automatically enrolled to follow one of the Bastyon's help team accounts. Bastyon's help team periodically circulates contribution materials through posts.
It would be useful if the Bastyon team writes some how-tos or shared various resources, using the long article format and reposting them, periodically.

The bottom line - the discoverability requirement has to be met.
Types of contributions
The initial document should include how volunteers could help Bastyon, e.g. how they can help promote it.
Some examples for help docs on how-to:

How to file a bug/complain/feature request, etc.
How they can contribute to the development of the project. There would be a link to the contributions.md.
Promotion of Bastyon with links to documents with how-tos and best practices.
Art, Writing, Localization, Author migration from other platforms, etc.

https://opensource.guide


chore(main): release v2.0.6

:robot: I have created a release beep boop
2.0.6 (2024-06-10)
Bug Fixes

Tabulate still not appearing in image. (#56) (a3a9332)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/actuarysailor/gha-repo-manager/releases/tag/v2.0.6 :sunflower:


Configure the Selinux type tag when the AppArmor is disabled regardless of EnableSelinux flag

Configure the Selinux type tag when the AppArmor is disabled regardless of EnableSelinux flag
Use the proper init container ID to avoid confusion

What type of PR is this?

/kind bug
What this PR does / why we need it:
This introduced a regression on Flatcar Linux. The SELinux type tag needs to be configured independent of EnableSelinux flag, because the SELinux can be active on the node regardless if the SELinux feature is enabled or not in the operator.
For instance, on Flatcar Linux SELinux type tag needs to be set to 'unconfined_t' instead of 'spc_t'
even though SELinux is disabled in order to get the containers to start.
Which issue(s) this PR fixes:

Does this PR have test?

Special notes for your reviewer:
Does this PR introduce a user-facing change?

Configure the Selinux type tag when the AppArmor is disabled regardless of EnableSelinux flag.

@pjbgf please could you also have a look since was introduced in https://github.com/kubernetes-sigs/security-profiles-operator/commit/025c4d1722a4069531909cb3fdbdbbda41b285b9.

The failure is orthogonal, and is highlighting we need to bump the ubi-image.

I am overriding the ubi-image failure and merging this. We need to handle the ubi-image fix as a separate PR.


time-1.5 compatibility

The time package now exposes its own TimeLocale.

Nice fix!
The upper limit on time needs to be removed in the cabal file though.

@nikomi as far as I can tell there are not bounds on time (which is itself a bit problematic). Are you seeing something I'm not?

In hslogger.cabal of version 1.2.6 on Hackage I see the following:
if flag(small_base)
    build-depends: base >= 4 && < 5, containers, directory, process,
        time < 1.5, old-locale
else

I believe this < 1.5 needs to be removed when applying this pull request. The else part describes dependencies for base < 3 - I don't think that path needs to be changed.
Since the pull request commit does not seem to mention an update to hslogger.cabal I thought I'd better mention it.

@nikomi ahh, right. Unfortunately the latest Hackage release hasn't been pushed to the repository so there's really no way to fix this at the moment.

No prob - it should just be remembered when the pull request is merged.


Required page not found

Hi, while studying the Stacks CLI, I was going through the documentation, under the checking balance, there was written that "To receive the testnet STX tokens, please use the faucet".

But when I click the faucet link it shows me that, "The page is missing".

Kindly do resolve this issue for future references. Thanks

This is appearing on https://docs.stacks.co/understand-stacks/command-line-interface#checking-balance
Note the double mention of "faucet faucet" as well.


Start/Stop All Services buttons not showing up

version: v0.21.2
In the old UI, there used to be two buttons for "Start Services" and "Stop Services" that would start ALL services or stop ALL services whenever a service was added. They no longer show up.

Added the project state and actions to the top-right like the other screens, so it's in there now.  And removed export config button since that is too.

v0.21.3
Verified fixed