Example pipeline does not work.

$ pachctl create-pipeline -f grep-example/grep-example-pipeline.json
Error reading from stdin: unknown field "input_repo" in pps.CreatePipelineRequest

After changing example pipeline to the json below[0], the pachctl create-pipeline -f grep-example/grep-example-pipeline.json command runs without any output but it fails silently [1]
[0]
{
  "pipeline": {
    "name": "grep-example"
  },
  "transform": {
    "image": "pachyderm/grep-example",
    "cmd": [
      "/bin/grep-example.sh"
    ]
  },
  "shards": "1",
  "inputs": [
    {
      "repo": {
        "name": "data"
      }
    }
  ]
}

[1]
2016-01-12T16:25:28Z INFO  protorpclog.Call {"service":"pachyderm.pps.persist.API","method":"CreatePipelineInfo","request":"pipeline_name:\"grep-example\" transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\004data\" \u003e created_at:\u003cseconds:1452615928 nanos:345096425 \u003e ","response":"pipeline_name:\"grep-example\" transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\004data\" \u003e created_at:\u003cseconds:1452615928 nanos:345096425 \u003e ","error":"","duration":{"seconds":"0","nanos":6094069}}
2016-01-12T16:25:28Z INFO  protorpclog.Call {"service":"pachyderm.pps.PipelineAPI","method":"CreatePipeline","request":"pipeline:\u003cname:\"grep-example\" \u003e transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\004data\" \u003e ","response":"","error":"","duration":{"seconds":"0","nanos":10328074}}
2016-01-12T16:25:28Z INFO  pipeline errored: rpc error: code = 2 desc = "repo \n\x04data not found"

Oops, that's an error that's not being propagated correctly.
Pipelines expect their input repos to exist before they're created so just do pachctl create-repo data before you create the pipeline.
I'll get a fix for the error not propagating merged soon.

I had the repo created in /tmp/pfs which was mounted using pachctl mount -p /tmp/pfs.
I'm working right now to setup a vagrant file with everything needed for working with pfs. Is this something I can contribute ?
There were a few problems:

I can't mount by default to /pfs without beeing root
If running on osx I need to install osx-fuse from homebrew which looks buggy.
Port forwarding from docker-machine does not work well: In the quickstart it is recommented to port forward host port 650 to port 30650 running inside the vm. I can't bind to this port using ssh without beeing root. If I run it as root, ssh will not work since I don't have the public/private key set up.
After binding on another port and setting PFS_ADDRESS and PPS_ADDRESS to localhost:30650 and 30651, ssh port forward breaks gRPC connection becouse it is tcp over tcp. I had this problem while port forwarding other gRPC services too, for example youtube vitess or my own services.
if I set PFS_ADDRESS=<IP_ADDRESS>:30650 and export PPS_ADDRESS=<IP_ADDRESS>:30651 in my zshrc it works and I can succesfully mount the fuse filesystem, but it does not correctly get replicated. I lost my energy at this point :disappointed:

I'm thinking of the following solution which will help other people to get a quick start using pfs:

Create a ubuntu machine with docker running using device-mapper
Start kubernetes cluster automatically using upstart so it can start on machine boot
Persist etcd, so after vm restart kubernetes starts the pods again. Currently this pods need to be created every time if cluster is running under docker-machine.
Mount /pfs automatically: here I need to find a way to delay this mount on machine boot since kubernetes pods take a while to start

What do you think ?
PS. Also, after created the pipeline which silently failed, pachctl list-pipelines was throwing a go panic, and pachctl delete-pipeline was exiting cleanely but the pipeline was not deleted. Here I suspect another silent error.

First, thanks so much for taking the time to write this up for us :D. It's a huge help.
Send me an email at<EMAIL_ADDRESS>with your address and tshirt size and I'd love to send you some swag.

I'm working right now to setup a vagrant file with everything needed for working with pfs. Is this something I can contribute ?

Ideally I'd like to keep developing using docker-machine, I've found that flow to work really well. That being said it looks like there are some annoyances that need to be dealt with here.

I can't mount by default to /pfs without beeing root

Not super attached to this default, I think it's good for inside containers since it saves you some typing but on user machines it's cumbersome. What would be a good default for this?

If running on osx I need to install osx-fuse from homebrew which looks buggy.

https://osxfuse.github.io/ is the best bet for this, adding it to the README.

Port forwarding from docker-machine does not work well: In the quickstart it is recommented to port forward host port 650 to port 30650 running inside the vm. I can't bind to this port using ssh without beeing root. If I run it as root, ssh will not work since I don't have the public/private key set up.
After binding on another port and setting PFS_ADDRESS and PPS_ADDRESS to localhost:30650 and 30651, ssh port forward breaks gRPC connection becouse it is tcp over tcp. I had this problem while port forwarding other gRPC services too, for example youtube vitess or my own services.
if I set PFS_ADDRESS=<IP_ADDRESS>:30650 and export PPS_ADDRESS=<IP_ADDRESS>:30651 in my zshrc it works and I can succesfully mount the fuse filesystem, but it does not correctly get replicated. I lost my energy at this point :disappointed:

Ugh, this sounds really annoying. Really sorry you ran into this. One easy thing we can do here is make it so that mount looks for pfs on port 30650 by default, then you can bind w/o sudo.
I'm not sure I totally understand the TCP over TCP problem what goes wrong with that?
For the replication problem are you seeing an issue where you create a repo and the results of ls /pfs don't change? If so that's an issue with the OS caching results. On OSX there's a flag you can pass that prevents it but last I checked it was obfuscated by the library. We're working with the maintainer to get it punched through so we can use it which should fix the problem.

In general I'm in favor of having a number of having different ways to deploy, but I have a pretty strong preference that our recommended deploy be possible with just Docker, no VMs needed.

Regarding the panic, pb definitions are still changing kind of fast, so it's possible just rebuilding will fix it. If it happens again definitely grab the back trace for me. I can't find a way to reproduce right now.
And regarding delete-pipelines that one's still unimplemented :(

Yes, I'd prefer using docker-machine too, I couldn't make docker work with devicemapper in vagrant without any [bugs](https://github.com/docker/docker/issues/5684].
It looks very convenient to mount to /pfs when running inside docker and I think you can let like this by default but maybe there should be a message if unable to mount to this path:
$ pachctl mount
mkdir /pfs: permission denied

$ # proposed:
$ pachctl mount
mkdir /pfs: permission denied
If you are not running as root you can use pachctl mount -p /path/to/mount/pfs to mount it to a different directory.

Currently as far as I understand, pachctl works by sending gRPC requests to pachctl mount daemon. Somehow, writing multiple times to a file inside a commit duplicates the file content:
$ ls /tmp/pfs      # ignore this files since I'm creating a new repo:
data                  example               pipeline-grep-example
$ pachctl create-repo testrepo
$ pachctl list-repo
NAME                    CREATED             SIZE
example                 46 years ago        0 B
testrepo                46 years ago        0 B
data                    46 years ago        0 B
pipeline-grep-example   46 years ago        0 B
$ ls /tmp/pfs
data                  example               pipeline-grep-example testrepo
$ commit=$(pachctl start-commit testrepo)
$ echo $commit
7c06c56e3a544a968060415679b6da5d
$ echo foo > /tmp/pfs/testrepo/$commit/file
$ echo bar > /tmp/pfs/testrepo/$commit/file
$ echo baz > /tmp/pfs/testrepo/$commit/file
$ pachctl finish-commit testrepo $commit
$ cat /tmp/pfs/testrepo/$commit/file # bar gets duplicated
foo
bar
baz
bar
$ pachctl create-pipeline -f grep-example/grep-example-pipeline.json # changed pipeline name and repo

ppsd log:
2016-01-13T06:57:44Z ERROR protorpclog.Call {"service":"pachyderm.pps.persist.API","method":"CreatePipelineInfo","request":"pipeline_name:\"grep-example\" transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\010testrepo\" \u003e created_at:\u003cseconds:1452668264 nanos:279957078 \u003e ","response":"\u003cnil\u003e","error":"Duplicate primary key `PipelineName`:\n{\n\t\"CreatedAt\":\t{\n\t\t\"Nanos\":\t179209495,\n\t\t\"Seconds\":\t1452666254\n\t},\n\t\"InputRepo\":\t[\n\t\t{\n\t\t\t\"Name\":\t\"\\n\\u0004data\"\n\t\t}\n\t],\n\t\"PipelineName\":\t\"grep-example\",\n\t\"Shards\":\t1,\n\t\"Transform\":\t{\n\t\t\"Cmd\":\t[\n\t\t\t\"/bin/grep-example.sh\"\n\t\t],\n\t\t\"Image\":\t\"pachyderm/grep-example\",\n\t\t\"Stdin\":\t\"\"\n\t}\n}\n{\n\t\"CreatedAt\":\t{\n\t\t\"Nanos\":\t279957078,\n\t\t\"Seconds\":\t1452668264\n\t},\n\t\"InputRepo\":\t[\n\t\t{\n\t\t\t\"Name\":\t\"\\n\\btestrepo\"\n\t\t}\n\t],\n\t\"PipelineName\":\t\"grep-example\",\n\t\"Shards\":\t1,\n\t\"Transform\":\t{\n\t\t\"Cmd\":\t[\n\t\t\t\"/bin/grep-example.sh\"\n\t\t],\n\t\t\"Image\":\t\"pachyderm/grep-example\",\n\t\t\"Stdin\":\t\"\"\n\t}\n}","duration":{"seconds":"0","nanos":8113960}}
2016-01-13T06:57:44Z ERROR protorpclog.Call {"service":"pachyderm.pps.PipelineAPI","method":"CreatePipeline","request":"pipeline:\u003cname:\"grep-example\" \u003e transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\010testrepo\" \u003e ","response":"\u003cnil\u003e","error":"Duplicate primary key `PipelineName`:\n{\n\t\"CreatedAt\":\t{\n\t\t\"Nanos\":\t179209495,\n\t\t\"Seconds\":\t1452666254\n\t},\n\t\"InputRepo\":\t[\n\t\t{\n\t\t\t\"Name\":\t\"\\n\\u0004data\"\n\t\t}\n\t],\n\t\"PipelineName\":\t\"grep-example\",\n\t\"Shards\":\t1,\n\t\"Transform\":\t{\n\t\t\"Cmd\":\t[\n\t\t\t\"/bin/grep-example.sh\"\n\t\t],\n\t\t\"Image\":\t\"pachyderm/grep-example\",\n\t\t\"Stdin\":\t\"\"\n\t}\n}\n{\n\t\"CreatedAt\":\t{\n\t\t\"Nanos\":\t279957078,\n\t\t\"Seconds\":\t1452668264\n\t},\n\t\"InputRepo\":\t[\n\t\t{\n\t\t\t\"Name\":\t\"\\n\\btestrepo\"\n\t\t}\n\t],\n\t\"PipelineName\":\t\"grep-example\",\n\t\"Shards\":\t1,\n\t\"Transform\":\t{\n\t\t\"Cmd\":\t[\n\t\t\t\"/bin/grep-example.sh\"\n\t\t],\n\t\t\"Image\":\t\"pachyderm/grep-example\",\n\t\t\"Stdin\":\t\"\"\n\t}\n}","duration":{"seconds":"0","nanos":8414681}}
2016-01-13T06:57:58Z INFO  protorpclog.Call {"service":"pachyderm.pps.persist.API","method":"CreatePipelineInfo","request":"pipeline_name:\"grep-example2\" transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\010testrepo\" \u003e created_at:\u003cseconds:1452668278 nanos:719612052 \u003e ","response":"pipeline_name:\"grep-example2\" transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\010testrepo\" \u003e created_at:\u003cseconds:1452668278 nanos:719612052 \u003e ","error":"","duration":{"seconds":"0","nanos":3852347}}
2016-01-13T06:57:58Z INFO  protorpclog.Call {"service":"pachyderm.pps.PipelineAPI","method":"CreatePipeline","request":"pipeline:\u003cname:\"grep-example2\" \u003e transform:\u003cimage:\"pachyderm/grep-example\" cmd:\"/bin/grep-example.sh\" \u003e shards:1 input_repo:\u003cname:\"\\n\\010testrepo\" \u003e ","response":"","error":"","duration":{"seconds":"0","nanos":11903499}}
2016-01-13T06:57:58Z INFO  pipeline errored: rpc error: code = 2 desc = "repo \n\btestrepo not found"

pfsd log
2016-01-13T06:56:59Z INFO  protorpclog.Call {"service":"pachyderm.pfs.InternalAPI","method":"FinishCommit","request":"commit:\u003crepo:\u003cname:\"testrepo\" \u003e id:\"b123fa7293f84173adc92b9ca4fd7177\" \u003e finished:\u003cseconds:1452668219 nanos:596205987 \u003e ","response":"","error":"","duration":{"seconds":"0","nanos":32633490}}
2016-01-13T06:56:59Z INFO  protorpclog.Call {"service":"pachyderm.pfs.API","method":"FinishCommit","request":"commit:\u003crepo:\u003cname:\"testrepo\" \u003e id:\"b123fa7293f84173adc92b9ca4fd7177\" \u003e finished:\u003cseconds:1452668219 nanos:596205987 \u003e ","response":"","error":"","duration":{"seconds":"0","nanos":34046873}}
2016/01/13 06:57:14 Appends: map[.:children:<key:"file" value:true >  file:block_refs:<block:<hash:"DPkYCnZKuoY6Z7bXLwkYvBMcZ3JkLLLc5aNPCnAvlHDdwr8SXBIZixmVwjPDS0r9NGxUojNMNQqUilG26LTmtg==" > range:<upper:4 > > block_refs:<block:<hash:"zAaAjLvuBRAzGql5dBMujcKWrreVviKdBkuueEsKh6XPQoHYLoyZJxt12yFI8IoCbBpg7Zyr24ysbSQkLaxAYw==" > range:<upper:4 > > block_refs:<block:<hash:"yoCXzzisVHEYV9lrMk2vhW5Y240E9X1X_rppOvdIIn-_-u2rWqzp96YQ2XOJS9C_hLtaljbbDNhrSFzsfoE7Kw==" > range:<upper:8 > > ]
2016/01/13 06:57:14 using: block_refs:<block:<hash:"DPkYCnZKuoY6Z7bXLwkYvBMcZ3JkLLLc5aNPCnAvlHDdwr8SXBIZixmVwjPDS0r9NGxUojNMNQqUilG26LTmtg==" > range:<upper:4 > > block_refs:<block:<hash:"zAaAjLvuBRAzGql5dBMujcKWrreVviKdBkuueEsKh6XPQoHYLoyZJxt12yFI8IoCbBpg7Zyr24ysbSQkLaxAYw==" > range:<upper:4 > > block_refs:<block:<hash:"yoCXzzisVHEYV9lrMk2vhW5Y240E9X1X_rppOvdIIn-_-u2rWqzp96YQ2XOJS9C_hLtaljbbDNhrSFzsfoE7Kw==" > range:<upper:8 > >
2016/01/13 06:57:14 returning: file:<commit:<repo:<name:"testrepo" > id:"b123fa7293f84173adc92b9ca4fd7177" > path:"file" > file_type:FILE_TYPE_REGULAR size_bytes:16 modified:<seconds:1452668219 nanos:596205987 > commit_modified:<repo:<name:"testrepo" > id:"b123fa7293f84173adc92b9ca4fd7177" >
2016-01-13T06:57:14Z INFO  protorpclog.Call {"service":"pachyderm.pfs.InternalAPI","method":"GetFile","request":"file:\u003ccommit:\u003crepo:\u003cname:\"testrepo\" \u003e id:\"b123fa7293f84173adc92b9ca4fd7177\" \u003e path:\"file\" \u003e size_bytes:16 ","response":"","error":"","duration":{"seconds":"0","nanos":7933837}}
2016-01-13T06:57:14Z INFO  protorpclog.Call {"service":"pachyderm.pfs.API","method":"GetFile","request":"file:\u003ccommit:\u003crepo:\u003cname:\"testrepo\" \u003e id:\"b123fa7293f84173adc92b9ca4fd7177\" \u003e path:\"file\" \u003e size_bytes:16 ","response":"","error":"","duration":{"seconds":"0","nanos":8808770}}
2016-01-13T06:57:58Z INFO  protorpclog.Call {"service":"pachyderm.pfs.InternalAPI","method":"CreateRepo","request":"repo:\u003cname:\"pipeline-grep-example2\" \u003e ","response":"","error":"","duration":{"seconds":"0","nanos":5120}}
2016-01-13T06:57:58Z INFO  protorpclog.Call {"service":"pachyderm.pfs.API","method":"CreateRepo","request":"repo:\u003cname:\"pipeline-grep-example2\" \u003e ","response":"","error":"","duration":{"seconds":"0","nanos":1989055}}
2016-01-13T06:57:58Z ERROR protorpclog.Call {"service":"pachyderm.pfs.InternalAPI","method":"ListCommit","request":"repo:\u003cname:\"\\n\\010testrepo\" \u003e commit_type:COMMIT_TYPE_READ block:true ","response":"\u003cnil\u003e","error":"repo \n\u0008testrepo not found","duration":{"seconds":"0","nanos":59402}}
2016-01-13T06:57:58Z ERROR protorpclog.Call {"service":"pachyderm.pfs.API","method":"ListCommit","request":"repo:\u003cname:\"\\n\\010testrepo\" \u003e commit_type:COMMIT_TYPE_READ block:true ","response":"\u003cnil\u003e","error":"rpc error: code = 2 desc = \"repo \\n\\btestrepo not found\"","duration":{"seconds":"0","nanos":2517764}}

Regarding tcp over tcp, I tried again now and it looks to work for pachctl mount and maybe I was missing something last night, I can't reproduce anymore. Nevermind about tcp over tcp, I think this is happening only if I tunnel the traffic over a ssh socks socket.

pachctl list-pipeline is returning a panic:
pachctl list-pipeline
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x15f5d5]

goroutine 1 [running]:
github.com/pachyderm/pachyderm/src/pps/pretty.PrintPipelineInfo(0x2043028, 0xc82010e800, 0xc82017b400)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/src/pps/pretty/pretty.go:38 +0x435
github.com/pachyderm/pachyderm/src/pps/cmds.Cmds.func6(0x10f5cd0, 0x0, 0x0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/src/pps/cmds/cmds.go:226 +0x3ea
github.com/pachyderm/pachyderm/vendor/go.pedge.io/pkg/cobra.RunBoundedArgs.func1(0xc820249000, 0x10f5cd0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/go.pedge.io/pkg/cobra/pkgcobra.go:25 +0xb0
github.com/pachyderm/pachyderm/vendor/github.com/spf13/cobra.(*Command).execute(0xc820249000, 0x10f5cd0, 0x0, 0x0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/github.com/spf13/cobra/command.go:572 +0x869
github.com/pachyderm/pachyderm/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc820126400, 0xc820249000, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/github.com/spf13/cobra/command.go:662 +0x54e
github.com/pachyderm/pachyderm/vendor/github.com/spf13/cobra.(*Command).Execute(0xc820126400, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/github.com/spf13/cobra/command.go:618 +0x2d
main.do(0x7aa3a0, 0xc82015c5a0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/src/cmd/pachctl/main.go:118 +0x7a7
github.com/pachyderm/pachyderm/vendor/go.pedge.io/env.Main(0xd42750, 0x7aa3a0, 0xc82015c5a0, 0x0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/go.pedge.io/env/env.go:46 +0x1ff
main.main()
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/src/cmd/pachctl/main.go:37 +0x74

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 19 [chan receive]:
github.com/pachyderm/pachyderm/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x10cc300)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/github.com/golang/glog/glog.go:882 +0x67
created by github.com/pachyderm/pachyderm/vendor/github.com/golang/glog.init.1
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/github.com/golang/glog/glog.go:410 +0x297

goroutine 22 [select]:
github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc.(*Conn).transportMonitor(0xc820252000)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/clientconn.go:452 +0x417
github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc.NewConn.func1(0xc820252000)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/clientconn.go:291 +0x1b5
created by github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc.NewConn
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/clientconn.go:292 +0x4ab

goroutine 24 [select]:
github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport.(*http2Client).controller(0xc8202383c0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport/http2_client.go:812 +0x5d8
created by github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport.newHTTP2Client
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport/http2_client.go:193 +0x1536

goroutine 25 [IO wait]:
net.runtime_pollWait(0x2042be0, 0x72, 0xc820012210)
	/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82011ad80, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82011ad80, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82011ad20, 0xc8201794f4, 0x9, 0x9, 0x0, 0x15a0050, 0xc820012210)
	/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8200fc2f8, 0xc8201794f4, 0x9, 0x9, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:172 +0xe4
io.ReadAtLeast(0x2042cc8, 0xc8200fc2f8, 0xc8201794f4, 0x9, 0x9, 0x9, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xe6
io.ReadFull(0x2042cc8, 0xc8200fc2f8, 0xc8201794f4, 0x9, 0x9, 0xc82023f2c8, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x62
github.com/pachyderm/pachyderm/vendor/golang.org/x/net/http2.readFrameHeader(0xc8201794f4, 0x9, 0x9, 0x2042cc8, 0xc8200fc2f8, 0x20000000, 0x0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/golang.org/x/net/http2/frame.go:227 +0xa5
github.com/pachyderm/pachyderm/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc8201794d0, 0x0, 0x0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/golang.org/x/net/http2/frame.go:372 +0xe6
github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport.(*framer).readFrame(0xc82023f230, 0x0, 0x0, 0x0, 0x0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport/http_util.go:450 +0x3d
github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport.(*http2Client).reader(0xc8202383c0)
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport/http2_client.go:742 +0x115
created by github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport.newHTTP2Client
	/Users/toni/code/gopath/src/github.com/pachyderm/pachyderm/vendor/google.golang.org/grpc/transport/http2_client.go:199 +0x1595

We now have new examples: https://github.com/pachyderm/pachyderm/tree/master/examples/fruit_stand


handle case where link text contains a footnote

link text can't contain a link (by definition), but it could contain a footnote.
I should check for that. I bet it's wrong right now.

[ ] check for md output
[ ] check for json output
[ ] check for footnotes that contain other footnotes

- AAA: [footnote [^1] in a link](https://example.com)
- BBB: footnote countains foonote[^2]
- CCC: footnote contains link[^3]

[^1]: the link's footnote text
[^2]: this footnote contains[^a] a footnote
[^3]: this footnote contains a [link][3a]
[^a]: this is the footnote's footnote

[3a]: https://example.com/3a

AAA: footnote [^1] in a link
BBB: footnote countains foonote[^2]
CCC: footnote contains link[^3]

[^1]: the link's footnote text
[^2]: this footnote contains[^a] a footnote
[^3]: this footnote contains a link
[^a]: this is the footnote's footnote

Oh actually, footnotes-within-links don't work. I just hadn't noticed that it's rendered wrong, but it is.
mdast handles these the same as GH does: see https://gist.github.com/yshavit/6af0a784e338dc32e66717aa6f495ffe

Can you also have footnotes that reference themselves? [^1]
[^1]: let's see if you can[^1]

(answer: kinda, but GitHub gets confused!)


minrelaytxfee=0 LND 0.5 - Opened a Direct Channel - TX does not exist after 4 days (mempool empty), on-chain funds locked?

Background
My funds got locked after trying to open 2 direct channels through Zap. On-chain transactions do not appear to exist, but my on-chain funds transferred into Zap are not shown and are locked somewhere. One thing I recently applied is minrelaytxfee=0
Your environment

version of lnd:0.5-beta
which operating system (uname -a on *Nix)
version of btcd, bitcoind, or other backend -0.16.3
any other relevant environment details

Steps to reproduce
Earlier my node was having some problems, (https://lightningcommunity.slack.com/archives/C6AFCN3KL/p1537906528000100?thread_ts=1537905613.000100&cid=C6AFCN3KL)
So I set minrelaytxfee=0.  This was happening on 0.42 so I applied minrelaytxfee=0 and updated to 0.5.
Afer I tried to pay invoice to a node that was new on the network, I had no direct path, so I tried to open a new direct channel with a node<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd@<IP_ADDRESS>:9735
I transferred my funds via on-chain to Zap and opened the channel with the node in question and one more (issue appears for both nodes , but I'll just use one in this case to avoid spamming logs)
I tried opening 2 direct channels via Zap wallet and after 3 days the transactions still don’t show in any blockexplorer.
TX1 786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e
TX2 60fd48550bcb9f83650f07aca467824c72d027850e1bb7f0feb87e320c3ea88b
These very same transactions show in my LND logs, so this is not a Zap related issue.
Zap shows these channels as “Pending, but the transactions on-chain do not seem to exist.
So it looks like the money is “locked” somewhere since the on-chain balance was reduced in Zap, but the transactions simply do not exist.
Here are logs for one of the nodes (sorry I'm not good at logs, so I'm pasting everything related.
2018-10-11 09:16:21.116 [INF] FNDG: Initiating fundingRequest(localAmt=0.04 BTC, remoteAmt=0 mSAT, capacity=0.04 BTC, chainhash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, peer=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd, dustLimit=0.00000573 BTC, min_confs=0)
2018-10-11 09:16:21.121 [INF] LNWL: Performing funding tx coin selection using 533 sat/kw as fee rate
2018-10-11 09:16:21.286 [INF] FNDG: Target commit tx sat/kw for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5): 1119
2018-10-11 09:16:21.286 [INF] FNDG: Starting funding workflow with <IP_ADDRESS>:9735 for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:21.495 [INF] FNDG: Recv'd fundingResponse for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:21.528 [INF] FNDG: pendingChan(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5): remote party proposes num_confs=3, csv_delay=144
2018-10-11 09:16:21.529 [INF] FNDG: Generated ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1) for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
……
2018-10-11 09:16:22.241 [INF] FNDG: Finalizing pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5) over ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1), waiting for channel open on-chain
2018-10-11 09:16:22.241 [INF] FNDG: Waiting for funding tx (786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e) to reach 3 confirmations
2018-10-11 09:16:22.241 [INF] CNCT: Close observer for ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1) active

2018-10-11 09:16:20.865 [INF] SRVR: Established connection to: <IP_ADDRESS>:9735
2018-10-11 09:16:20.866 [INF] SRVR: Finalizing connection to<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd, inbound=false
2018-10-11 09:16:20.951 [INF] SRVR: Pruning peer<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd from persistent connections, number of open channels is now zero
2018-10-11 09:16:20.952 [INF] SRVR: Negotiated chan series queries with<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd
2018-10-11 09:16:20.952 [INF] DISC: Creating new gossipSyncer for peer=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd
2018-10-11 09:16:20.952 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): requesting new chans from height=545135 and<PHONE_NUMBER> blocks after
2018-10-11 09:16:21.116 [INF] FNDG: Initiating fundingRequest(localAmt=0.04 BTC, remoteAmt=0 mSAT, capacity=0.04 BTC, chainhash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, peer=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd, dustLimit=0.00000573 BTC, min_confs=0)
2018-10-11 09:16:21.121 [INF] LNWL: Performing funding tx coin selection using 533 sat/kw as fee rate
2018-10-11 09:16:21.286 [INF] FNDG: Target commit tx sat/kw for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5): 1119
2018-10-11 09:16:21.286 [INF] FNDG: Starting funding workflow with <IP_ADDRESS>:9735 for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:21.495 [INF] FNDG: Recv'd fundingResponse for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:21.528 [INF] FNDG: pendingChan(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5): remote party proposes num_confs=3, csv_delay=144
2018-10-11 09:16:21.529 [INF] FNDG: Generated ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1) for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:22.044 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): applying new update horizon: start=1970-01-01 00:00:00 +0000 UTC, end=2106-02-07 06:28:15 +0000 UTC, backlog_size=28516
2018-10-11 09:16:22.045 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): buffering chan range reply of size=0
2018-10-11 09:16:22.045 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): filtering through 0 chans
2018-10-11 09:16:22.046 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): remote peer has no new chans
2018-10-11 09:16:22.046 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): applying gossipFilter(start=2018-10-11 08:16:22.046318396 +0000 UTC m=+51626.381696084)
2018-10-11 09:16:22.147 [INF] FNDG: Broadcasting funding tx for ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1): (*wire.MsgTx)(0xc003232ac0)({
 Version: (int32) 1,
 TxIn: ([]*wire.TxIn) (len=3 cap=15) {
  (*wire.TxIn)(0xc0034bc180)({
   PreviousOutPoint: (wire.OutPoint) 3f3476d883325619dba2bba9ade9447bd2da9fcc429e377b77cd1856c933f147:0,
   SignatureScript: ([]uint8) <nil>,
   Witness: (wire.TxWitness) (len=2 cap=2) {
    ([]uint8) (len=71 cap=144) {
     00000000  30 44 02 20 4e 76 70 bb  27 ed 98 12 cc fa 42 6a  |0D. Nvp.'.....Bj|
     00000010  58 10 7c ad 54 ef 31 1d  31 04 f5 ea d3 5f 04 24  |X.|.T.1.1...._.$|
     00000020  6f 6d 2e 8d 02 20 56 13  a3 97 c0 e9 a0 c7 ae d4  |om... V.........|
     00000030  31 13 53 39 88 b5 89 4f  47 be 63 9a e4 5e 85 6c  |1.S9...OG.c..^.l|
     00000040  22 ae f1 ac 5a 97 01                              |"...Z..|
    },
    ([]uint8) (len=33 cap=33) {
     00000000  02 c3 2c c0 cd 76 4f 4e  6c a5 de 9c 20 37 8e 35  |..,..vONl... 7.5|
     00000010  9c df fc 2b 3a d7 65 51  65 04 a4 77 fc 0c 13 a8  |...+:.eQe..w....|
     00000020  ad                                                |.|
    }
   },
   Sequence: (uint32)<PHONE_NUMBER>
  }),
  (*wire.TxIn)(0xc0034bc120)({
   PreviousOutPoint: (wire.OutPoint) 4557716f33fd5765a0cb18d65afca3dd80f643342fa01cd627bac451fa352fe0:0,
   SignatureScript: ([]uint8) <nil>,
   Witness: (wire.TxWitness) (len=2 cap=2) {
    ([]uint8) (len=71 cap=144) {
     00000000  30 44 02 20 31 ad 02 35  f3 fb 3c d9 a2 dd 1b 13  |0D. 1..5..<.....|
     00000010  a3 65 2c 44 57 e7 ba 93  67 b9 31 8d 51 02 09 ee  |.e,DW...g.1.Q...|
     00000020  cf 04 64 91 02 20 0f c6  4e 8e 97 78 40 6c 4c fc  |..d<EMAIL_ADDRESS>     00000030  2c 9f 17 5a 6e 18 b4 d0  74 77 39 0f 1b 3c 49 06  |,..Zn...tw9..<I.|
     00000040  46 03 f1 63 c7 ec 01                              |F..c...|
    },
    ([]uint8) (len=33 cap=33) {
     00000000  03 a4 5e 89 aa ba 2b 40  58 ab 0d f8 4a fe 7c b0<EMAIL_ADDRESS>     00000010  34 dc 5d b4 6b 7e de b7  21 9d 70 96 29 79 60 bd  |4.].k~..!.p.)y`.|
     00000020  38                                                |8|
    }
   },
   Sequence: (uint32)<PHONE_NUMBER>
  }),
  (*wire.TxIn)(0xc0034bc0c0)({
   PreviousOutPoint: (wire.OutPoint) 60fd48550bcb9f83650f07aca467824c72d027850e1bb7f0feb87e320c3ea88b:1,
   SignatureScript: ([]uint8) <nil>,
   Witness: (wire.TxWitness) (len=2 cap=2) {
    ([]uint8) (len=72 cap=144) {
     00000000  30 45 02 21 00 b1 6e a1  79 3c fc af 5c f6 db 4e  |0E.!..n.y<..\..N|
     00000010  eb bd cb a3 85 4a f0 70  dd a5 f2 14 2b 89 83 52  |.....J.p....+..R|
     00000020  5f 54 d3 30 de 02 20 30  8f cd 2a ab a1 ba ae 14  |_T.0.. 0..*.....|
     00000030  23 53 85 6f e2 10 fc 8d  49 cd 77 e3 8b 66 5d 5d  |#S.o....I.w..f]]|
     00000040  5b 11 e3 63 a2 d6 4a 01                           |[..c..J.|
    },
    ([]uint8) (len=33 cap=33) {
     00000000  02 54 64 8e d4 f1 c8 b4  46 3f 43 99 c0 e9 58 c1  |.Td.....F?C...X.|
     00000010  de 09 9a 02 2c a4 d7 a8  72 a3 2f 54 07 53 87 ef  |....,...r./T.S..|
     00000020  8b                                                |.|
    }
   },
   Sequence: (uint32)<PHONE_NUMBER>
  })
 },
 TxOut: ([]*wire.TxOut) (len=2 cap=15) {
  (*wire.TxOut)(0xc000f24fa0)({
   Value: (int64) 63496,
   PkScript: ([]uint8) (len=22 cap=500) {
    00000000  00 14 e7 b7 8e 9d 7d 93  62 51 86 46 a6 71 e3 3c  |......}.bQ.F.q.<|
    00000010  84 e4 47 90 1e 9e                                 |..G...|
   }
  }),
  (*wire.TxOut)(0xc0008b0820)({
   Value: (int64) 4000000,
   PkScript: ([]uint8) (len=34 cap=500) {
    00000000  00 20 db bd f8 c2 67 e0  1a 18 c6 dc 2b 1e ed 86  |. ....g.....+...|
    00000010  05 16 07 53 73 06 ca be  1d 5e b9 3f 8b 34 04 e5  |...Ss....^.?.4..|
    00000020  bd 45                                             |.E|
   }
  })
 },
 LockTime: (uint32) 0
})

2018-10-12 05:14:36.047 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): buffering chan range reply of size=132
2018-10-12 05:14:36.047 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): filtering through 132 chans
2018-10-12 05:14:36.047 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): starting query for 1 new chans
2018-10-12 05:14:36.047 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): querying for 1 new channels
2018-10-12 05:15:25.314 [INF] DISC: Broadcasting batch of 11 new announcements
2018-10-12 05:15:34.766 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): no more chans to query
2018-10-12 05:15:34.767 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): applying gossipFilter(start=2018-10-12 04:15:34.767013791 +0000 UTC m=+123579.102391509)
2018-10-12 05:15:35.270 [INF] CRTR: Updated vertex data for node=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd

Some more logs
2018-10-15 12:59:49.436 [INF] SRVR: Established connection to: <IP_ADDRESS>:9735
2018-10-15 12:59:49.437 [INF] SRVR: Finalizing connection to 02425af7b5f27f324bd3312bc19b58ef9dd40a6a7cac581b70e0cb2f3e31ae77a6, inbound=false
2018-10-15 12:59:49.482 [INF] SRVR: Established connection to: <IP_ADDRESS>:9735
2018-10-15 12:59:49.483 [INF] SRVR: Finalizing connection to 03087d5a72090d460313065e077d24294021c44f91778ae8893a927d923664b58a, inbound=false
2018-10-15 12:59:49.620 [WRN] PEER: Unable to find our forwarding policy for channel 786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1, using default values
2018-10-15 12:59:49.834 [INF] PEER: NodeKey(02425af7b5f27f324bd3312bc19b58ef9dd40a6a7cac581b70e0cb2f3e31ae77a6) loading ChannelPoint(c2e19fbb274eef40f4384457257ba8120ce4bdc9e9dee5d28d6918da32b304f4:1)
2018-10-15 12:59:49.838 [INF] PEER: NodeKey(03087d5a72090d460313065e077d24294021c44f91778ae8893a927d923664b58a) loading ChannelPoint(ec7d8ca5fc32b5e27c9bdfb6c1c312db728b17237be71f845ca73da2bd560504:0)
2018-10-15 12:59:49.838 [INF] HSWC: Removing channel link with ChannelID(5ee800c69f083f7436fa8c231311b9db392166c698a09ffc5ec9197bbb606879)
2018-10-15 12:59:49.838 [INF] HSWC: ChannelLink(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1) is starting
2018-10-15 12:59:49.844 [INF] HSWC: Adding pending link chan_id=5ee800c69f083f7436fa8c231311b9db392166c698a09ffc5ec9197bbb606879, short_chan_id=0:0:0
2018-10-15 12:59:49.845 [INF] HSWC: HTLC manager for ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1) started, bandwidth=3959190000 mSAT
2018-10-15 12:59:49.845 [INF] HSWC: Attempting to re-resynchronize ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1)
2018-10-15 12:59:49.845 [ERR] PEER: unable to send out active channel update: edge not found
2018-10-15 12:59:49.845 [INF] SRVR: Negotiated chan series queries with<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd
2018-10-15 12:59:49.846 [INF] DISC: Creating new gossipSyncer for peer=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd
2018-10-15 12:59:49.846 [INF] HSWC: Received re-establishment message from remote side for channel(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1)
2018-10-15 12:59:49.846 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): requesting new chans from height=545708 and 42944215

Expected behaviour
The direct channels should be opened and the on-chain funds transferred there.
Actual behaviour
Channels are pending and the on-chain funds are locked. The on-chain transactions do not exist.

Here's a bit more logs from when transaction was created

2018-10-11 09:16:20.866 [INF] SRVR: Finalizing connection to<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd, inbound=false
2018-10-11 09:16:20.951 [INF] SRVR: Pruning peer<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd from persistent connections, number of open channels is now zero
2018-10-11 09:16:20.952 [INF] SRVR: Negotiated chan series queries with<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd
2018-10-11 09:16:20.952 [INF] DISC: Creating new gossipSyncer for peer=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd
2018-10-11 09:16:20.952 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): requesting new chans from height=545135 and<PHONE_NUMBER> blocks after
2018-10-11 09:16:21.116 [INF] FNDG: Initiating fundingRequest(localAmt=0.04 BTC, remoteAmt=0 mSAT, capacity=0.04 BTC, chainhash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, peer=0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd, dustLimit=0.00000573 BTC, min_confs=0)
2018-10-11 09:16:21.121 [INF] LNWL: Performing funding tx coin selection using 533 sat/kw as fee rate
2018-10-11 09:16:21.286 [INF] FNDG: Target commit tx sat/kw for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5): 1119
2018-10-11 09:16:21.286 [INF] FNDG: Starting funding workflow with <IP_ADDRESS>:9735 for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:21.495 [INF] FNDG: Recv'd fundingResponse for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:21.528 [INF] FNDG: pendingChan(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5): remote party proposes num_confs=3, csv_delay=144
2018-10-11 09:16:21.529 [INF] FNDG: Generated ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1) for pendingID(1501d9a42a350a912b721bf123c439b0a73433f331ba9a6c21cae7e3c6c517f5)
2018-10-11 09:16:22.044 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): applying new update horizon: start=1970-01-01 00:00:00 +0000 UTC, end=2106-02-07 06:28:15 +0000 UTC, backlog_size=28516
2018-10-11 09:16:22.045 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): buffering chan range reply of size=0
2018-10-11 09:16:22.045 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): filtering through 0 chans
2018-10-11 09:16:22.046 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): remote peer has no new chans
2018-10-11 09:16:22.046 [INF] DISC: gossipSyncer(0236648105f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd): applying gossipFilter(start=2018-10-11 08:16:22.046318396 +0000 UTC m=+51626.381696084)
2018-10-11 09:16:22.147 [INF] FNDG: Broadcasting funding tx for ChannelPoint(786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e:1): (*wire.MsgTx)(0xc003232ac0)({

The strange thing is
 2018-10-11 16:18:23.540 [INF] CRTR: **New channel discovered!** Link connects<PHONE_NUMBER>f51c815caa98cd914af7236b2bca3071a124c9f63450d6ae961a27fd and 02529db69fd2ebd3126fb66fafa234fc3544477a23d509fe93ed229bb0e92e4fb8 with ChannelPoint(85d3e5997727eaa0f740c121ec332a4cb486aed416e2a4a580ed9b5d2d5abcc6:1): chan_id=599602173546266625, capacity=0.020003 BTC

So for some reason the 786860bb7b19c95efc9fa098c6662139dbb91113238cfa36743f089fc600e85e funding tx was not properly propagated on the network. I see it spending 60fd48550bcb9f83650f07aca467824c72d027850e1bb7f0feb87e320c3ea88b which also failed to propagate.
Can you provide the logs from lnd at startup? Preferably with debuglevl=debug set in lnd.conf. The fundingmanager should try to re-broadcast the tx at this point, and we'll get an indication why it fails.

The issue stemmed from a sweep with an insufficient fee. minrelaytxfee=0 allowed the transaction to  be broadcasted to the backend node and added the UTXO to the wallet. This same UTXO was then unintentionally used for funding transactions later on (this will be fixed by #2059), causing them to be stuck until the sweep transaction confirms on-chain.

I have a similiar problem here: https://github.com/lightningnetwork/lnd/issues/2099

Closing per https://github.com/lightningnetwork/lnd/issues/2052#issuecomment-430441433.


UCF_CC_50 data prepare

Dear svishwa, I don't know how to devided train and test dataset of UCF_CC_50, could you please provide me the UCF_CC_50 data prepare codes, if available?

The pre-processing code provided for ShanghaiTech dataset can be modified for other datasets. Please follow the cross-validation split as discussed in the UCF_CC_50 dataset paper to perform the experiments.


  Sorry about that.

https://github.com/husobee/vestigo/pull/44 should correct, let me know if this works for you and i will get it merged in.

If all tests pass, that works for me. You should updated your travis to run tests on all these version of Go.
go:
  - 1.0
  - 1.1
  - 1.2
  - 1.3
  - 1.4
  - 1.5
  - 1.6
  - 1.7
  - tip


scrollTo in ViewActions cannot be applied to (org.hamcrest.Matcher<android.view.View>

Line: 124 - Getting an error that scrollTo doesn't work with a Matcher as a parameter
https://github.com/googlecodelabs/android-testing/blob/master/app/src/androidTest/java/com/example/android/testing/notes/notes/NotesScreenTest.java

Hey thanks for reporting this. Would be great to get a bit more info. Can please paste the logcat output here? Also would be great to know which device/emulator you're running on.

My bad, auto imports imported from the wrong package

Had run into the same issue as well. There were unreferenced imports that were automagically being cleaned up by Android Studio.
For future developers go to:

And uncheck Optimize imports on the fly.


No textures

Hi! I have a problem: Item models are displayed in the resource pack, but textures are not! What could be the problem? I play through the official launcher, version 1.20.1, I have attached all the mods present.

Most times the problem is to do with the pack not being updated and textures not being in the correct folders for the atlas to pick them up. I'd need to see the pack to verify. (also, logs are much more informative than a picture of the mods folder if you open issues in the future)


Update config.template.toml

Added OpenAI o1-preview model

Hi @Soyel-Akter-Habib , thanks a lot for the contribution! But I don't think we necessarily need to add every model to config.template.toml, I think people should be able to figure out how to add new models based on the current example. I'm going to close this PR, but we'd appreciate any contributions in the future!


Initial Client ID middleware

Are we storing the client IDs in S3? Any thoughts how to maintain them?

They are maintained and deployed by https://dev.azure.com/skfdc/REP-SW/_git/enlight-client-ids


import error cv2 error

remote access not avail  stock RP14 8G fresh buster install...explicit directions followed can't start anything

Hi @ptegler, can you describe the steps you performed that caused this error?


Pull abbreviated messages as transition triggers into edges of "contract negotiation state machine"

Might be reviewed together with https://github.com/International-Data-Spaces-Association/ids-specification/pull/43

@SebastianOpriel could you add the full name of the messages and events back into the diagram and update it to the latest version of @juliapampus including the transitions to terminated? It might become too complex, but maybe we can use it for a better explanation of typical flows.


Add HttpRequest.split()

So that we don't have to check if the instance is an HttpData or HttpHeaders(trailers) in the subscriber.
Currently, we only have HttpResponse.split().

May I try on this issue???

Sure, it's all yours. :-)


Use mimalloc and target-cpu=native in rust tonic_st and grpcio

I ran the CI benchmark before/after the change (Only for rust, ruby and java because of https://github.com/LesnyRumcajs/grpc_bench/issues/126)
before: https://github.com/nico-abram/grpc_bench/issues/6
after: https://github.com/nico-abram/grpc_bench/issues/5
I am on a windows box without docker or wsl so I couldn't run the benchmarks locally. I don't see any significant changes in the CI benchmark but I have a feeling using mimalloc could significantly improve the multithreaded benchmark

@LifeIsStrange I'm inclined to go with one of those allocators indeed, as long as we don't create a hydra like cpp_jemalloc_st, cpp_mimalloc_st, cpp_tcmalloc_st, cpp_mimalloc_st_standard, cpp_jemalloc_mt.... I could imagine having a nice script to somehow manage that, but for now I'd be happy to have locally tested "best allocator". :)


WIP Hide before_all, after_all, and error lanes

Small fix for #597, it hides before_all, after_all, and error lanes in Project Settings.
Update: I noticed that this also affects "create new project", I'm working on unifying the filter.

Cool, thanks, yeah make sure to always use the WIP if it's not ready for review + merge yet, thanks 👍

Oh while you're at this, could you also remove the no_platform prefix if there is no platform? We'd just not render a platform if there is no platform 👍

Just to clarify, @KrauseFx , do you still want to set no_platform as platform to project or we set nil also there?

I made some changes:

no_platform won't be rendered but will still be set in the platform property of the project
private lanes are also filtered out
filter is applied in the visualisation of the lanes and creation of the project too
clicking in the title button of the lane visualisation now correctly select the lane (previously an alert was shown mentioning we didn't have this feature)
fixed the lint 🎆

@KrauseFx this should be good now, let me know if having other comments :) I'm going to remove the WIP.


Generalized RowDisplayPubmedPublicationStatusDateArrivedInPubmed and …

…RowDisplayPubmedPublicationStatusDateArrivedInPubmed to work for BiblioEditor and BiblioDisplay, since they're displayOnly fields.  Generalized displaying a pair of data in one row, as Valerio suggested.  Added display of RowDisplayBooleanDisplayOnly for prepublication_pipeline, generalized to allow any future boolean field that should display its tag as the value, when the value it true

Thanks @valearna !


Add tests

Pull Request Test Coverage Report for Build 350

1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+1.05%) to 87.928%

Totals

Change from base Build 348:
1.05%

Covered Lines:
2185

Relevant Lines:
2485

💛  - Coveralls


Cleanup all hackweek references.

[x] remove logo
[x] remove text from intro page
[x] change copyright
[x] fix Code of Conduct

#47 and #42 address those.


Add a press facet

ACCEPTANCE

[x] The platform-wide catalog page has a facet that lets me limit results to a single press.

I can meet the AC by adding a line to controllers/catalog_controlller.rb:
config.add_facet_field solr_name('press', :facetable), label: "Press", helper_method: :press_name_for_subdomain
But it relies on a helper (in helpers/application_helpers.rb) to get Press.name from Press.subdomain since Press.name isn't in indexed in solr. A better idea would be to index Press.name is solr similar to Press.subdomain?

Instead of using a helper to pull the press.name from the database to create the facet, press.name is now indexed in solr via monograph_indexer.rb

This was PR 149


feat: add block <-> views helper methods

Codecov Report

Merging #173 (3baba6e) into main (895306f) will decrease coverage by 0.17%.
The diff coverage is 25.31%.

@@            Coverage Diff             @@
##             main     #173      +/-   ##
==========================================
- Coverage   62.86%   62.70%   -0.17%
==========================================
  Files          99       99
  Lines       21299    21366      +67
==========================================
+ Hits        13390    13397       +7
- Misses       7909     7969      +60

Impacted Files
Coverage Δ

taos-query/src/common/raw/views/json_view.rs
71.85% <0.00%> (-8.15%)
:arrow_down:

taos-query/src/common/raw/views/n_char_view.rs
78.89% <0.00%> (-7.11%)
:arrow_down:

taos-query/src/common/raw/views/var_char_view.rs
62.43% <0.00%> (-6.18%)
:arrow_down:

taos-query/src/helpers/database.rs
72.04% <ø> (-0.18%)
:arrow_down:

taos-query/src/helpers/describe.rs
68.44% <0.00%> (-3.60%)
:arrow_down:

taos-query/src/lib.rs
45.66% <ø> (-0.11%)
:arrow_down:

taos-query/src/prelude.rs
55.76% <ø> (ø)

taos-query/src/tmq/mod.rs
38.70% <ø> (ø)

taos-sys/src/ffi.rs
0.00% <ø> (ø)

taos-sys/src/lib.rs
54.75% <0.00%> (+0.11%)
:arrow_up:

... and 15 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


sbt-coursier sometimes gets stuck resolving deps with 404 response for .sha1 files

I'm not sure if this is a coursier's issue, but it looks like this:
// a bunch of successful downloads

Downloading https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1
https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

// and it continues...

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

// and never stops (at least I haven't waited for longer than few minutes)
^C

 curl -I https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1
HTTP/1.1 404 Not Found
Date: Wed, 23 Aug 2017 14:12:39 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d0329880c502d3b7751b1a1e6a6b0d6e11503497559; expires=Thu, 23-Aug-18 14:12:39 GMT; path=/; domain=.flywaydb.org; HttpOnly
Access-Control-Allow-Origin: *
X-GitHub-Request-Id: 36B1:22ADC:529409:720CE8:599D8D57
Server: cloudflare-nginx
CF-RAY: 392eab039f4c8b88-KBP

And after restart it works fine again.
I have this:
addSbtPlugin("io.get-coursier" % "sbt-coursier" % "1.0.0-RC8")

Today I got this again with RC11 enabled.
Downloading https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1
https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1

https://flywaydb.org/repo/org/flywaydb/flyway-sbt_2.10_0.13/4.2.0/flyway-sbt-4.2.0.pom.sha1
...
// few minutes of lines like this
^C

Still can't figure out if this is output from coursier or something built-in into sbt :)

The output is from coursier (the non-progress bar loggers prints the state of the downloads if they take too much time). I'll try to have a look at that…

Closing for now. Please, re-open if you have more details (repro, …).


Remove TLS record expansion API in Mbed TLS 3.0

Context: The SSL API mbedtls_ssl_get_max_record_expansion() provides an upper bound for the difference between record plaintext and ciphertext sizes, that is, the amount of overhead that record protection adds on the wire.
I can't see a reason why this would be of interest to users. If the goal is to calculate approximations of the maximum plaintext sizes from knowledge of the I/O buffer sizes, this isn't the way to do it as there's dedicated API for that purpose (see #4381).
Suggestion: Remove mbedtls_ssl_get_max_record_expansion() from the public API for Mbed TLS 3.0.
Work items for Mbed TLS 3.0:

Move the declaration of mbedtls_ssl_get_max_record_expansion() to ssl_misc.h.

@hanno-arm , I cannot find the mbedtls_ssl_get_max_record_expansion() function in the current code. The only similar function I see is mbedtls_ssl_get_max_record_payload(). Is this a typo or the function has been deleted in the meantime?

@TRodziewicz I think we've just solved this, but forgot to tie the PR to this issue. #4382

As @hanno-arm said the issue has been resolved in #4382, so I'm closing this issue.

Thus just to be sure, @hanno-arm is it the case that #4382 fixed both #4381 and this issue ?


RPi GPIO monitoring

Spawned from issue #3: Monitor mode only listens to kegboards, but we've had some long-standing requests for RPI GPIO support.  Add a trivial flow meter daemon for these pins.

I'm a bit confused as to if Raspberry Pi GPIO is now supported. Can you clarify the status?

Also want to know an update here. Surely after almost 2 years something has come out of this. :)

Any updates on this?

Was any progress ever made on this?


Add GIF select to composer

Adds a simple way to add GIFs to your post, via the GIPHY API. This also updates the photo/camera icons in the composer to ALF icons
iOS (got rate limited while recording lol):
https://github.com/bluesky-social/social-app/assets/10959775/f0e417f1-db6d-4364-8091-ad19ce110922
Android:
https://github.com/bluesky-social/social-app/assets/10959775/6d9cc6fb-db90-480a-a9f1-b39d3c81eb74
Web:
https://github.com/bluesky-social/social-app/assets/10959775/450f549b-94df-4f53-b925-2d3622d91d26
Consent flow:
https://github.com/bluesky-social/social-app/assets/10959775/0dbc370e-dcad-4932-86fc-b61b5ddf0232
Test plan

Check that you can add a GIF
Check that search works
Check that the consent flow works
3a. Confirm that the copy is good
Check error/loading states

Seems a bit confusing that you can still press GIF or Photo if you already have a gif. Not super obvious that this would replace it. Maybe that's OK, but then it's confusing that the button does get disabled in the opposite scenario (see last example).
https://github.com/bluesky-social/social-app/assets/810438/a844f0b4-44cf-4189-ac67-7b5b4e1f2e9e

Add a GIF
Add some photos (they currently replace the GIF)
Delete photos

The GIF pops back up as if it's still there. But actually if I didn't delete the photos, it wouldn't be there.

Play button being unclickable in the composer feels very confusing. I presume the new embed will fix it?

Pressing Escape on the web should dismiss the dialog.

Play button being unclickable in the composer feels very confusing. I presume the new embed will fix it?

I think it would? Maybe it should allow taps

Is this legit?

When I choose a gif, I see a few loading states:
https://github.com/bluesky-social/social-app/assets/810438/65fe1fec-9d21-4608-9d5f-15b3383e4aea
First:

Second:

Third:

Since we already know the aspect ratio (do we?), can we jump straight to the third one? Or, if we have to keep the spinner, can we specify the aspect ratio so that it doesn't jump multiple times?
Or would this be resolved by Hailey's changes?

Is this legit?

Missing type on the .web

Play button being unclickable in the composer feels very confusing. I presume the new embed will fix it?

Yea. I don't think it's so bad to show it but 1. should follow the autoplay setting and 2. just needs to look better.

What's up with our text input that allows this?

I don't know if this is just us, but it used to happen in our old login screens before we used our new text input API. So if it is us it's something deeper than just that API. (This could be that predictive text junk too, although I thought that was only enabled in a multiline text input. Looks similar though)


Not working

I download your sample and does not work for me, I am with the same behavior in my app.
o check my aasas validation and is all good!

in my app I have at my Entitlement.plist

I check the iOS logs and I get the message

I am testing with Rocksim:

but still not working, I am testing only on emulator.
And I also check the associated domains in developer portal and generate a new profile for release the app.
but, not working....

I have a similar experience with the Universal link is valid, but I can't even get log to fire in ContinueUserActivity.


Add the ast::fold::Fold trait, replacing MapAst

cc @pca006132 - you'd still be able to do what you wanted the custom field for, with this instead of MapAst, right?

cc @pca006132 - you'd still be able to do what you wanted the custom field for, with this instead of MapAst, right?

Looks good to me, but haven't tried yet. I would try this later today.

Looks good to me now.

Hi, just wondering when would the new version of rustpython-parser be released to crates.io? Not sure if my project should depend on the github version or the crates.io version for now.


Session invalid bug

Over load balancer to zurich

@tgdriph3 This is the prober problem. Needs to be fixed there. We need somebody who can track the problem on the TCP level.

This should be fixed due to the global sessions.

still a problem in current master.


feat: Add support for window rotation OS-17652

Description of Change

Checklist

[ ] PR description included and stakeholders cc'd
[ ] npm test passes
[ ] tests are changed or added
[ ] relevant documentation, tutorials, templates and examples are changed or added
[ ] PR release notes describe the change in a way relevant to app developers, and are capitalized, punctuated, and past tense.

Release Notes
Notes:

I think native_window_views.cc is used for all linux targets. Did you have a look why doesn't work on desktop linux? To me, it looks like this should work in the same way on desktop linux.
The reason I am asking is, we can add a test case for this. This is a quite big patch, which might get broken with Electron updates. Although series4 isn't that important, we will probably end up using this on series5 too, and we will have to keep this patch around.

I have been wanting to do this. I am not sure why its not working on the desktop linux build, but you speculated its due to software rendered version of skin not supporting rotation. When I run it with a rotation I can see the compositor resolution is changed but it's not rotated. I will try looking into this more.

Can you try on Brightsign with --disable-gpu flag and see if it behaves the same?

Can you try on Brightsign with --disable-gpu flag and see if it behaves the same?

Yes, it behaves the same way with GPU disabled on my ls5.

Can you try implementing these to methods in
electron/src/components/viz/service/display_embedder/software_output_surface.h
  void SetDisplayTransformHint(gfx::OverlayTransform transform) override {}
  gfx::OverlayTransform GetDisplayTransform() override;

Can you try implementing these to methods in electron/src/components/viz/service/display_embedder/software_output_surface.h
  void SetDisplayTransformHint(gfx::OverlayTransform transform) override {}
  gfx::OverlayTransform GetDisplayTransform() override;

Thanks, that worked. I will now look into making an Electron test case.


Show POH PerHeapHistoryData in Raw XML view

This change allows us to view the POH data in the raw XML view.
@brianrob
@Maoni0

other than what I mentioned in this comment it LGTM!

@cshung & @brianrob
This PR performed the following breaking change:

We have code in our codebase like this:
        public TraceGCEx(TraceGC gc)
        {
            Number = gc.Number;
            RelativeTimestamp = gc.StartRelativeMSec;
            Generation = gc.Generation;
            Reason = gc.Reason.ToString();
            Type = gc.Type.ToString();
            PauseTime = gc.PauseDurationMSec;
            PinnedObjectsSize = (double)gc.GetPinnedObjectSizes() / (1024 * 1024);
            PinnedObjectPercent = gc.GetPinnedObjectPercentage() >= 0 ? (double)gc.GetPinnedObjectPercentage() : 0;
            SuspensionTime = gc.SuspendDurationMSec;
            Issues = new Dictionary<string, double>();

            // per heap calculations
            HeapSize = HeapFragmentation = 0;
            SurvivalPercent = new Dictionary<string, double>();

            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);
                if (!Double.IsNaN(surv))
                {
                    SurvivalPercent.Add(((Gens)gen).ToString(), surv);
                }
            }

What is the recommended way forward??

@cshung & @brianrob
This PR performed the following breaking change:
We have code in our codebase like this:
        public TraceGCEx(TraceGC gc)
        {
            Number = gc.Number;
            RelativeTimestamp = gc.StartRelativeMSec;
            Generation = gc.Generation;
            Reason = gc.Reason.ToString();
            Type = gc.Type.ToString();
            PauseTime = gc.PauseDurationMSec;
            PinnedObjectsSize = (double)gc.GetPinnedObjectSizes() / (1024 * 1024);
            PinnedObjectPercent = gc.GetPinnedObjectPercentage() >= 0 ? (double)gc.GetPinnedObjectPercentage() : 0;
            SuspensionTime = gc.SuspendDurationMSec;
            Issues = new Dictionary<string, double>();

            // per heap calculations
            HeapSize = HeapFragmentation = 0;
            SurvivalPercent = new Dictionary<string, double>();

            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);
                if (!Double.IsNaN(surv))
                {
                    SurvivalPercent.Add(((Gens)gen).ToString(), surv);
                }
            }

What is the recommended way forward? If I change to using .MaxGenCount we get Index OOBs on our traces when processing and choosing .GenPinObj just because it happens to be the same int value as what we were using previously seems awfully dirty.

Fundamentally, the runtime introduced a new "generation" (technically, the pinned object heap is not a new generation, but it is convenient to represent it that way) in 5.0, so that bound needs to be dynamically adjusted depending on whether or not the trace was captured before 5.0 or not.
To make this task easier, there is a new property named NumGenerations in this PR, can you use that property instead of hard coding?

Thanks for that explanation, @cshung

To make this task easier, there is a new property named NumGenerations in this PR, can you use that property instead of hard coding?

I considered that, however even after updating to v3.0.4 the object we're using doesn't have that property :(

Any other ideas? I'm inheriting this code so not quite sure what the right thing is ;)

Any other ideas? I'm inheriting this code so not quite sure what the right thing is ;)

I tried to look into this, and it appears to me there are some disconnects.
The new property NumGenerations is added on the GCPerHeapHistoryTraceData. That class seems to be mostly concerned with parsing trace data in binary format. The new property is used mostly because the trace format changed to have variable size content because we have a variable number of generations, and therefore the parsing need to deal with that.
But it appears to me all you have is a TraceGC object instead. I assumed TraceGC is being built as part of the parsing process, but I am unfamiliar with those structures, in particular, I am not sure how you can reach into the corresponding GCPerHeapHistoryTraceData object that was used during the parsing process to get access to that. This will need more study.
You mentioned that there is an index out of bound error when you were experimenting with the code. Do you happen to know which array are you accessing? Can you reach into that array and read the length of that array? Or surface the length of that array as a property of the TraceGC class?

Yeah the blow-up happens in the loop I posted:
            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);

Do you need me to execute it and get details on what gc looks like? It doesn't appear these objects/methods are proper arrays but rather methods which, in turn, index into an array.

Yeah the blow-up happens in the loop I posted:
            for (int gen = (int)Gens.Gen0; gen < (int)Gens.Gen0After; gen++)
            {
                HeapSize += gc.HeapStats != null ? gc.GenSizeAfterMB((Gens)gen) : 0;
                HeapFragmentation += gc.GenFragmentationMB((Gens)gen);
                var surv = gc.SurvivalPercent((Gens)gen);

Do you need me to execute it and get details on what gc looks like? It doesn't appear these objects/methods are proper arrays but rather methods which, in turn, index into an array.

Yes, a stack trace with the TraceEvent bits included would be nice.

does that shed any light on things?

It looks like you have a GenSizeBeforeMB array that you could access, and the dimension is 4, which is probably the number you are looking for. What would happen if you replace Gen.Gen0After by the length of that array?

Yeah, no good.

any other ideas?

This is not going to be very productive to chat here once a day or two, would you mind finding me on teams?


fix(content): unsubscribe from viewCtrl observables after content ins…

…tance destroyed
Short description of what this resolves:
Resolves error Uncaught TypeError: Cannot read property 'ev' of null when user switches views too quickly
Changes proposed in this pull request:
The subscribers on the viewCtrl observables can resolve after the Content instance has been destroyed. This results in the _scroll property being undefined at the time the subscriber resolves, causing this error. This may also be related to #9593
Ionic Version: 2.x
Fixes: #10045

I also submitted this PR:
https://github.com/driftyco/ionic/pull/10060/files
Now I have noticed my PR does not fix the issue, but it might help. We are subscribing to the event before this.scroll is actually instanced.
I think it is safer to initialize this._scroll before.

Both changes added

Merged! thanks a lot for the PR!


Add support for "3D Scans"

This is an initial partially working version. I thought this would be a good point to ask if what I have done so far is reasonable. I can run 3x100 and Nx1 scans manually through the web GUI with no problems. But when we tested it with GDA, it got very upset (41 point step scan, 0.5 second exposure, ~1 second per run):
removed

This, or something similar, happens everytime, but half the time the scan does actually finish as expected. The terminal continues partially logging if you hit enter, but the prompt is gone and if I try to run any more scans it doesn't do anything.
Have I done something insane? I guess the most obvious thing is the try catch I have added in the timeout loop. Or, maybe it is unrelated.

It looks like there is some overlap between this and #290. Do we need both?

It looks like there is some overlap between this and #290. Do we need both?

I have take out my hacky try except timeout and used event_timeout instead and the scan works without the horrible messages and breaking the prompt, through GDA.
There is still an issue with the completion status, which I think confuses GDA:

I have take out my hacky try except timeout and used event_timeout instead and the scan works without the horrible messages and breaking the prompt, through GDA.
There is still an issue with the completion status, which I think confuses GDA:

Web GUI fixed. Apparently GDA might be confused with progress for other reasons.

Web GUI fixed. Apparently GDA might be confused with progress for other reasons.

I have updated all the tests except the one that is failing. This seems like an unrelated issue. Does anyone know what the problem might be here?
__________________ TestXmap3DetectorDriverPart.test_configure __________________

Traceback (most recent call last):
  File "/home/travis/build/dls-controls/pymalcolm/tests/test_modules/test_xmap/test_xmapdriverpart.py", line 74, in test_configure
    assert self.child.handled_requests.mock_calls == [
AssertionError: assert [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.put('attributesFile', 'Z:\\mri-attributes.xml'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None)] == [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None),\n call.put('attributesFile', 'Z:\\mri-attributes.xml')]

  At index 11 diff: call.put('attributesFile', 'Z:\\mri-attributes.xml') != call.post('start')

  Full diff:
    [
     call.put('arrayCallbacks', True),
     call.put('arrayCounter', 0),
     call.put('autoPixelsPerBuffer', 'Manual'),
     call.put('binsInSpectrum', 2048),
     call.put('collectMode', 'MCA mapping'),
     call.put('ignoreGate', 'No'),
     call.put('inputLogicPolarity', 'Normal'),
     call.put('pixelAdvanceMode', 'Gate'),
     call.put('pixelsPerBuffer', 1),
     call.put('pixelsPerRun', 456),
     call.put('presetMode', 'No preset'),
  +  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
     call.post('start'),
     call.when_value_matches('acquiring', True, None),
  -  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
    ]

Next I will add some new tests.

I have updated all the tests except the one that is failing. This seems like an unrelated issue. Does anyone know what the problem might be here?
__________________ TestXmap3DetectorDriverPart.test_configure __________________

Traceback (most recent call last):
  File "/home/travis/build/dls-controls/pymalcolm/tests/test_modules/test_xmap/test_xmapdriverpart.py", line 74, in test_configure
    assert self.child.handled_requests.mock_calls == [
AssertionError: assert [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.put('attributesFile', 'Z:\\mri-attributes.xml'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None)] == [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None),\n call.put('attributesFile', 'Z:\\mri-attributes.xml')]

  At index 11 diff: call.put('attributesFile', 'Z:\\mri-attributes.xml') != call.post('start')

  Full diff:
    [
     call.put('arrayCallbacks', True),
     call.put('arrayCounter', 0),
     call.put('autoPixelsPerBuffer', 'Manual'),
     call.put('binsInSpectrum', 2048),
     call.put('collectMode', 'MCA mapping'),
     call.put('ignoreGate', 'No'),
     call.put('inputLogicPolarity', 'Normal'),
     call.put('pixelAdvanceMode', 'Gate'),
     call.put('pixelsPerBuffer', 1),
     call.put('pixelsPerRun', 456),
     call.put('presetMode', 'No preset'),
  +  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
     call.post('start'),
     call.when_value_matches('acquiring', True, None),
  -  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
    ]

Next I will add some new tests.

I added a commit to fix mypy, but I noticed I didn't have to do the same elsewhere. It is because in PositionLabellerPart done_when_reaches is defined as None, but in DetectorDriverPart it is defined as 0. Should this be made consistent @thomascobb ?

I added a commit to fix mypy, but I noticed I didn't have to do the same elsewhere. It is because in PositionLabellerPart done_when_reaches is defined as None, but in DetectorDriverPart it is defined as 0. Should this be made consistent @thomascobb ?

I have updated all the tests except the one that is failing. This seems like an unrelated issue. Does anyone know what the problem might be here?
__________________ TestXmap3DetectorDriverPart.test_configure __________________

Traceback (most recent call last):
  File "/home/travis/build/dls-controls/pymalcolm/tests/test_modules/test_xmap/test_xmapdriverpart.py", line 74, in test_configure
    assert self.child.handled_requests.mock_calls == [
AssertionError: assert [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.put('attributesFile', 'Z:\\mri-attributes.xml'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None)] == [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None),\n call.put('attributesFile', 'Z:\\mri-attributes.xml')]

  At index 11 diff: call.put('attributesFile', 'Z:\\mri-attributes.xml') != call.post('start')

  Full diff:
    [
     call.put('arrayCallbacks', True),
     call.put('arrayCounter', 0),
     call.put('autoPixelsPerBuffer', 'Manual'),
     call.put('binsInSpectrum', 2048),
     call.put('collectMode', 'MCA mapping'),
     call.put('ignoreGate', 'No'),
     call.put('inputLogicPolarity', 'Normal'),
     call.put('pixelAdvanceMode', 'Gate'),
     call.put('pixelsPerBuffer', 1),
     call.put('pixelsPerRun', 456),
     call.put('presetMode', 'No preset'),
  +  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
     call.post('start'),
     call.when_value_matches('acquiring', True, None),
  -  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
    ]

Next I will add some new tests.

I think this is caused by the move of arm_detector() in the DetectorDriverPart's on_configure() method, so that now it is after setting the attributes up - so the order of calls is now slightly different is all.

I have updated all the tests except the one that is failing. This seems like an unrelated issue. Does anyone know what the problem might be here?
__________________ TestXmap3DetectorDriverPart.test_configure __________________

Traceback (most recent call last):
  File "/home/travis/build/dls-controls/pymalcolm/tests/test_modules/test_xmap/test_xmapdriverpart.py", line 74, in test_configure
    assert self.child.handled_requests.mock_calls == [
AssertionError: assert [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.put('attributesFile', 'Z:\\mri-attributes.xml'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None)] == [call.put('arrayCallbacks', True),\n call.put('arrayCounter', 0),\n call.put('autoPixelsPerBuffer', 'Manual'),\n call.put('binsInSpectrum', 2048),\n call.put('collectMode', 'MCA mapping'),\n call.put('ignoreGate', 'No'),\n call.put('inputLogicPolarity', 'Normal'),\n call.put('pixelAdvanceMode', 'Gate'),\n call.put('pixelsPerBuffer', 1),\n call.put('pixelsPerRun', 456),\n call.put('presetMode', 'No preset'),\n call.post('start'),\n call.when_value_matches('acquiring', True, None),\n call.put('attributesFile', 'Z:\\mri-attributes.xml')]

  At index 11 diff: call.put('attributesFile', 'Z:\\mri-attributes.xml') != call.post('start')

  Full diff:
    [
     call.put('arrayCallbacks', True),
     call.put('arrayCounter', 0),
     call.put('autoPixelsPerBuffer', 'Manual'),
     call.put('binsInSpectrum', 2048),
     call.put('collectMode', 'MCA mapping'),
     call.put('ignoreGate', 'No'),
     call.put('inputLogicPolarity', 'Normal'),
     call.put('pixelAdvanceMode', 'Gate'),
     call.put('pixelsPerBuffer', 1),
     call.put('pixelsPerRun', 456),
     call.put('presetMode', 'No preset'),
  +  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
     call.post('start'),
     call.when_value_matches('acquiring', True, None),
  -  call.put('attributesFile', 'Z:\\mri-attributes.xml'),
    ]

Next I will add some new tests.

I think this is caused by the move of arm_detector() in the DetectorDriverPart's on_configure() method, so that now it is after setting the attributes up - so the order of calls is now slightly different is all.

@GDYendell mentioned this in #321

@GDYendell mentioned this in #321

the move of arm_detector() in the DetectorDriverPart's on_configure()
Of course. Thanks @BenBradnick

the move of arm_detector() in the DetectorDriverPart's on_configure()
Of course. Thanks @BenBradnick

I added a commit to fix mypy, but I noticed I didn't have to do the same elsewhere. It is because in PositionLabellerPart done_when_reaches is defined as None, but in DetectorDriverPart it is defined as 0. Should this be made consistent?

I added a commit to fix mypy, but I noticed I didn't have to do the same elsewhere. It is because in PositionLabellerPart done_when_reaches is defined as None, but in DetectorDriverPart it is defined as 0. Should this be made consistent?

I added a commit to fix mypy, but I noticed I didn't have to do the same elsewhere. It is because in PositionLabellerPart done_when_reaches is defined as None, but in DetectorDriverPart it is defined as 0. Should this be made consistent?

Yes, we should make it consistent. In this case they can both be defined as 0, as we always set it in configure.

I added a commit to fix mypy, but I noticed I didn't have to do the same elsewhere. It is because in PositionLabellerPart done_when_reaches is defined as None, but in DetectorDriverPart it is defined as 0. Should this be made consistent?

Yes, we should make it consistent. In this case they can both be defined as 0, as we always set it in configure.

I have added some tests for the functionality this adds, so I think this is ready to look at again @thomascobb .
Initially I thought they would be tests where I call configure, run, run etc and check behaviour based on what the frame counter does, but it does not look like this is done in the existing tests. I don't know if there are any examples of system-ish tests like that I could look at to add something similar here.
I also have some local black failures for lines I didn't change (imalcolm.py and tetrammdriverpart.py), but the travis build seems happy, so I am not sure whether to add them here.

I have added some tests for the functionality this adds, so I think this is ready to look at again @thomascobb .
Initially I thought they would be tests where I call configure, run, run etc and check behaviour based on what the frame counter does, but it does not look like this is done in the existing tests. I don't know if there are any examples of system-ish tests like that I could look at to add something similar here.
I also have some local black failures for lines I didn't change (imalcolm.py and tetrammdriverpart.py), but the travis build seems happy, so I am not sure whether to add them here.

re black: I noticed that recent versions have changed some subtle rules (I've already forgotten what) this may explain your experience?

re black: I noticed that recent versions have changed some subtle rules (I've already forgotten what) this may explain your experience?

Codecov Report

Merging #321 (03239ce) into master (1b005ec) will decrease coverage by 0.00%.
The diff coverage is 91.42%.

@@            Coverage Diff             @@
##           master     #321      +/-   ##
==========================================
- Coverage   92.60%   92.59%   -0.01%
==========================================
  Files         214      214
  Lines       11325    11349      +24
==========================================
+ Hits        10487    10509      +22
- Misses        838      840       +2

Impacted Files
Coverage Δ

malcolm/imalcolm.py
20.00% <0.00%> (-0.16%)
:arrow_down:

malcolm/modules/ADAndor/parts/andordriverpart.py
100.00% <ø> (ø)

malcolm/modules/adUtil/parts/reframepluginpart.py
100.00% <ø> (ø)

malcolm/modules/xmap/parts/xmapdriverpart.py
100.00% <ø> (ø)

malcolm/modules/ADOdin/parts/odinwriterpart.py
92.02% <80.00%> (-0.43%)
:arrow_down:

malcolm/modules/ADCore/parts/detectordriverpart.py
87.97% <93.33%> (+0.81%)
:arrow_up:

malcolm/modules/ADCore/parts/hdfwriterpart.py
96.31% <100.00%> (+0.06%)
:arrow_up:

...lcolm/modules/ADCore/parts/positionlabellerpart.py
92.68% <100.00%> (+0.27%)
:arrow_up:

...modules/scanning/controllers/runnablecontroller.py
92.69% <100.00%> (ø)

malcolm/modules/scanning/hooks.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b005ec...03239ce. Read the comment docs.

Codecov Report

Merging #321 (03239ce) into master (1b005ec) will decrease coverage by 0.00%.
The diff coverage is 91.42%.

@@            Coverage Diff             @@
##           master     #321      +/-   ##
==========================================
- Coverage   92.60%   92.59%   -0.01%
==========================================
  Files         214      214
  Lines       11325    11349      +24
==========================================
+ Hits        10487    10509      +22
- Misses        838      840       +2

Impacted Files
Coverage Δ

malcolm/imalcolm.py
20.00% <0.00%> (-0.16%)
:arrow_down:

malcolm/modules/ADAndor/parts/andordriverpart.py
100.00% <ø> (ø)

malcolm/modules/adUtil/parts/reframepluginpart.py
100.00% <ø> (ø)

malcolm/modules/xmap/parts/xmapdriverpart.py
100.00% <ø> (ø)

malcolm/modules/ADOdin/parts/odinwriterpart.py
92.02% <80.00%> (-0.43%)
:arrow_down:

malcolm/modules/ADCore/parts/detectordriverpart.py
87.97% <93.33%> (+0.81%)
:arrow_up:

malcolm/modules/ADCore/parts/hdfwriterpart.py
96.31% <100.00%> (+0.06%)
:arrow_up:

...lcolm/modules/ADCore/parts/positionlabellerpart.py
92.68% <100.00%> (+0.27%)
:arrow_up:

...modules/scanning/controllers/runnablecontroller.py
92.69% <100.00%> (ø)

malcolm/modules/scanning/hooks.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b005ec...03239ce. Read the comment docs.

re black: I noticed that recent versions have changed some subtle rules (I've already forgotten what) this may explain your experience?

Yeah the current builds of Travis are using black 20.x, which has some breaking changes for formatting rules - I had the same issue

re black: I noticed that recent versions have changed some subtle rules (I've already forgotten what) this may explain your experience?

Yeah the current builds of Travis are using black 20.x, which has some breaking changes for formatting rules - I had the same issue

I think this is ready to look at again @thomascobb

Rebased again to resolve conflicts. @thomascobb I think this is ready to look at again. Note the comment above in reference to renaming steps_until_done, which is now redundant.


Unknown column type.

First of all, this looks amazing. I cannot wait until the custom keybinds come out, moving from left to right with arrow keys seems a bit unintuitive.
I'm getting this error that it doesn't know the column type binary, making me not able to use this table. Is there a way to either include this column type or at least show the data without that specific column?

Same for TEXT type with sqlite


Can we add a function to download a cvs or xlsx report on local machine?

Can we add a function to download a cvs or xlsx report on local machine?

You can save the result as a CSV to your local machine:
import salesforce_reporting
import csv

sf = salesforce_reporting.Connection(
    username='xxxxxxx',
    password='xxxxxxx',
    security_token='xxxxxxxxxx')
report = sf.get_report(report_id, details=True)
parser = salesforce_reporting.ReportParser(report)
items = parser.records()
labels = parser._get_field_labels()
headers = [labels[i] for i in labels]
# include headers
items.insert(0, headers)

with open('csvfile.csv', 'w') as csvfile:
    f = csv.writer(csvfile)
    f.writerows(items)


it seems  hard to reproduce your results in paper.

I tried to grid search the params in generate program, but the results can't achieve which in your paper. May I ask if it possible to share your SOTA CKPTS ?

The best score on DSTC8-AVSD so far (LM + VASM)
CIDEr: 1.220
Bleu_4: 0.434
Bleu_3: 0.517
Bleu_2: 0.617
Bleu_1: 0.738
ROUGE_L: 0.589
METEOR: 0.284

thanks for your reply, the parameters of generate model are: ckpt:114888 beam_size: 4 min_len: 1 penalty: 0.3. I test it on dstc8avsd test set. training procedure is same with your released code.


CDPAM-1403 | Add logrotate conf to enable log rotation for ccmv2 agent logs

In this PR I have added a logrotate configuration file for ccmv2-inverting-proxy-agent's log to rotate the log every 6 days or when the size is 128MB. The init.sls script is updated to trigger this logrotate policy as well.
The configuration follows the rules for https://github.com/hortonworks/cloudbreak/blob/master/freeipa/src/main/resources/freeipa-salt/salt/logrotate/conf/ipabackup.

@lacikaaa I triggered the job http://build.eng.hortonworks.com:8080/job/cloudbreak-image-burn-salt-v3/16176/ which succeeded. Could the changes be merged now?


Segmentation fault when running tensorflow.python.framework.kernels.get_registered_kernels_for_op

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
No
Source
binary
TensorFlow version
2.11.0
Custom code
Yes
OS platform and distribution
22.04
Mobile device
No response
Python version
3.9
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
nvidia-cudnn-cu11==<IP_ADDRESS>, cudatoolkit=11.8.0
GPU model and memory
No response
Current behavior?
Probably due to feeding None argument
Standalone code to reproduce the issue
import tensorflow as tf
import os
import numpy as np
from tensorflow.python.framework import kernels
try:
  arg_0 = None
  out = kernels.get_registered_kernels_for_op(arg_0,)
except Exception as e:
  print("Error:"+str(e))

### Relevant log output

```shell
2023-08-12 13:41:10.388491: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Segmentation fault

@dmc1778 ,
I have replicated the behaviour in tf-nightly and got segmentation fault. Attaching logs below for reference.
(tf2.13) suryanarayanay@surya-ubuntu20:~$ python 61531_nightly.py
2023-08-14 07:07:51.716502: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9498] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-08-14 07:07:51.716678: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-08-14 07:07:51.716781: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-08-14 07:07:51.725885: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-14 07:07:52.645760: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
WARNING:tensorflow:From /home/suryanarayanay/miniconda3/envs/tf2.13/lib/python3.11/site-packages/tensorflow/python/ops/distributions/distribution.py:259: ReparameterizationType.__init__ (from tensorflow.python.ops.distributions.distribution) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use `tfp.distributions` instead of `tf.distributions`.
WARNING:tensorflow:From /home/suryanarayanay/miniconda3/envs/tf2.13/lib/python3.11/site-packages/tensorflow/python/ops/distributions/bernoulli.py:165: RegisterKL.__init__ (from tensorflow.python.ops.distributions.kullback_leibler) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use `tfp.distributions` instead of `tf.distributions`.
2.15.0-dev20230813
Segmentation fault (core dumped)

Since Segmentation faults can be considered as potential security vulnerabilities please read SECURITY.md and report them through proper channel as mentioned in SECURITY.md.
Thanks!

@SuryanarayanaY bug exists on 2.13.0