issue with containers communicating with each other on same host

I have experimental docker (Version : 1.9.0-dev) installed and I deployed ip-vlan-docker-plugin driver on my host after building it myself.
I start my driver as follows
./main --gateway=<IP_ADDRESS> --ipvlan-subnet=<IP_ADDRESS>/22 --host-interface=em1 --mode=l2
it starts up fine. Then I launch 2 containers on this host both running busybox. It starts up fine and gets IP addresses assigned and I can see the routes are added to the route table in the container. I am pasting the output of "route -n" below from one of the containers. Both look same.
route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
<IP_ADDRESS>         <IP_ADDRESS>     <IP_ADDRESS>         UG    0      0        0 eth0
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS>   U     0      0        0 eth0
What I am finding is that I am unable to ping one container from the other container on the same host. However if I start another container on a remote host in another subnet, I am able to reach it without any issues. I know from a container I cannot reach host interface. However I expected that containers within the same host should be able to communicate with each other.
I decided to test this outside the docker ecosystem as follows
// create first name space and provision it
ip netns add ns0
ip link add link em1 ipvl0 type ipvlan mode l2
ip link set dev ipvl0 netns ns0
ip netns exec ns0 bash
ip link set dev ipvl0 up
ip -4 addr add <IP_ADDRESS>/22 dev ipvl0
ip -4 route add default via <IP_ADDRESS> dev ipvl0
root@ubuntu:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
<IP_ADDRESS>         <IP_ADDRESS>     <IP_ADDRESS>         UG    0      0        0 ipvl0
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS>   U     0      0        0 ipvl0
// create a second name space
ip netns add ns1
ip link add link em1 ipvl1 type ipvlan mode l2
ip link set dev ipvl1 netns ns1
ip netns exec ns1 bash
ip link set dev ipvl1 up
ip -4 addr add <IP_ADDRESS>/22 dev ipvl1
ip -4 route add default via <IP_ADDRESS> dev ipvl1
Both these namespaces are on the same host and I have no problem pinging the interfaces in each of these namespaces from the bash shell in the other namespace.
Problem seems to be with only the docker container. Has anybody else experienced similar problems?

I have narrowed this down to the fact that the container stack does not respond to arp requests. ARP seems to be disabled on the IPVLAN interface in the container. This cannot be an issue with the IPVLAN kernel driver as it works correctly in the name space example I posted. There is something with docker and IPVLAN that is not setup correctly.

after disabling arp on the interface in the namespace by issuing "ifconfig ipvl0 -arp" I was able to reproduce this issue in name space too. So docker seems to be disabling arp on the eth0 (IPVLAN) interface assigned to the container. I would like to know how to make docker not disable arp.

Hey @shmurthy62 ! Sorry for the delay, I will dig into it tonight. I remember seeing that issue also with L2 mode. Im wondering if its a hairpin issue or how the iface is setup. Thanks a bunch for verifying it should work with iproute2 commands. Thats awesome!

It's definitely an ARP issue. BTW I could not get L3 mode to work at all. I will first make it work with iproute2 and then debug the issue with ipvlanplugin driver. Important thing is to understand if docker shuts off arp on the interface. So far I have scanned through libnetwork code and I have not come across code that turns off arp so far. May be I missed something.

Hi shmurthy62, I am a huge fan of L3 mode because it doesn't need any responders for ARP. I was scratching my head at first w/ L3 mode because I wasn't able to ping the ipvlan interface from the master interface (eth) and had them on the same subnet, different subnet, etc. I finally poked thru the netlink patch and saw everything from the head end of the interface pair (default ns eth0 for example) is filtered. So the cool thing is the underlying host is isolated from the namespace iface IP which makes sense for providers wanting a transparent underlying infra. So by design a namespace shouldn't be able to talk to the default namespace IP and ARP/Multicast is also filtered out.
The trick with IPVlan is you need a route to the namespace that the IPVlan tail is attached to from the default namespace that ingress traffic would come into the master interface. Those should be separate subnets between default namespace and container namespace, thus L3 mode. The plugin should include that route, but since it is single host it will not add the route to other namespaces on other hosts. Something would need to distribute that. Could be a k/v store or any other distribution mechanism. I played around with using Git since there is an interesting logging/accounting property to bootstrapping or for example something I am very interested in using a routing protocol to distribute route/nexthop since those are well known and understood with to massive scale. Here is an example of distributing the routes if interested. /32 routes with BGP + IPVlan is probably the cleanest and least amount of things that can break with no ARP and a solid BGP FSM for packet forwarding. Lemme know if you get stuck and I will gist the output to be clearer on the routes for multihost.

Hi Brent,
Thanks for looking in to this for me. From what you are telling me L2 mode is not usable at all. How do you explain what I did with iproute2 and namespaces. In that case ARP is enabled on the interface in the name space with IPVLAN l2 mode. Is the GO netlink library disabling this. The L2 mode is quite simple if ARP is enabled. If we could optionally get (ip link set ${interface name} arp on) supported in the netlink package - we can set this up through the ipvlan plugin driver.
If I understand you correctly L3 mode will require a BGP instance on each host - very similar to Calico solution. I will follow your suggestion and pursue L3. If we have to run BGP instance on each host and if the number of nodes is in 1000's it would be a nightmare to manage BGP and triage issues. In any case I will test this out.

Hey shmurthy62, L2 should be usable, sorry was just talking about L3. I went to test it and ran into the latest GetCapabilities patch PR #516. Opened issue #7 to fix it. So I can't test it atm but will tomorrow. On the IPVlan there shouldn't be any filtering above netlink so Im betting its just something setup wrong in the L2 mode here.
L3 mode would just require something to distribute network route and nexthop. L2 mode does this discovery with flooding and learning of mac addresses, L3 mode cares not for ethernet addresses since that is hidden away and the namespace is an altogether separate subnet from the default namespace. So distributing those routes would have to occur in some form or fashion, whether its etcd, bgp, zookeeper etc. If the environment is completely static then just declare it one time, burn it into the infra pointers and forget it but that takes a good crystal ball lol.
Good point on the clustering node count btw lol. In that case is a completely declarative model of a /23 or something per host that is static and routed out of a ToR make more sense? Or alternatively munging network state into orchestration clustering for packet forwarding? Something about keeping network state separate feels right to me.
Will update soon as I get issue #7 patched and verify L2 mode buddy. Thanks for the help in getting this proofed out.

@shmurthy62 I updated with the libnetwork API changes and verified L2 ipvlan mode. Pasting in a screenshot that shows two containers on the same docker host pinging one another. Lemme know if that doesn't work in your environment and I will try and replicate. I tested on two different hosts so it seems g2g. Cheers.

Does not work for me - I am trying to ping container on same host.
ubuntu@ubuntu-1:~$ sudo docker run -i -t --rm ubuntu /bin/bash
[sudo] password for ubuntu:
root@027397c45225:/# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:26:55:82:4f:1c
inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
inet6 addr: fe80::226:55ff:fe82:4f1c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:100 (100.0 B)  TX bytes:258 (258.0 B)
lo        Link encap:Local Loopback
inet addr:<IP_ADDRESS>  Mask:<IP_ADDRESS>
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING  MTU:65536  Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
root@027397c45225:/# ping <IP_ADDRESS>  // pinging another docker container on same host
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
From <IP_ADDRESS> icmp_seq=1 Destination Host Unreachable
From <IP_ADDRESS> icmp_seq=2 Destination Host Unreachable
From <IP_ADDRESS> icmp_seq=3 Destination Host Unreachable
From <IP_ADDRESS> icmp_seq=4 Destination Host Unreachable
^C
--- <IP_ADDRESS> ping statistics ---
12 packets transmitted, 0 received, +9 errors, 100% packet loss, time 11063ms
Now I ping my gw and can reach it
root@027397c45225:/# ping <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=0.389 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=0.235 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=64 time=0.230 ms
^C
Now going across the router to remote host - no issues
root@027397c45225:/# ping <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=63 time=0.185 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=63 time=0.183 ms
remote host to this container - works because I pinged this host from container
ubuntu@ubuntu-3:~$ ping <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=63 time=0.311 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=63 time=0.219 ms
^C
--- <IP_ADDRESS> ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.219/0.265/0.311/0.046 ms
However if I ping the other container which is on same host as container <IP_ADDRESS> it does not work because of arp not responding.
ubuntu@ubuntu-3:~$ ping <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
From <IP_ADDRESS> icmp_seq=1 Destination Host Unreachable
From <IP_ADDRESS> icmp_seq=2 Destination Host Unreachable
From <IP_ADDRESS> icmp_seq=3 Destination Host Unreachable
Not sure what is the issue - the ifconfig eth0 in the docker container shows arp is enabled but still it does
not respond.
if I ping remote host <IP_ADDRESS> from container <IP_ADDRESS> then the router learns its mac - after that I can ping from either end.
Not sure why it works for you and not me. I only have trouble with docker container and not with iproute2 and namespaces
I am using Ubuntu 14.04 and have deployed linux kernel 4.2
ubuntu@ubuntu-3:~$ uname -a
Linux ubuntu-3 4.2.0-040200-generic #201508301530 SMP Sun Aug 30 19:31:40 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
can you tell me your version.
Did something get fixed in netlink library in the last 2-3 weeks. My code ipvlan driver plugin code is that old.

Hey! Thanks for the info, super helpful. I think I had mentioned I had run into something similar and I was able to reproduce it. Definitely agree you are following the exactly right thing with checking with iproute commands like you did. Your logs sure do look like a hairpin issue on the localhost.
Have you setup macvlan links on the host at all recently? Ive noticed that if a macvlan link is defined pointing to the same master interface as an ipvlan interface it won't create a link. Thats not relevant to your situation since the interface was created but I figured I would throw it out there.
Try this if you don't mind:  Check /var/run/docker/netns/ for any orphaned but still mounted interfaces. Simply unmount and delete them. If you see anywhere Device or resource busy when doing netlink operations I've found that just means it is still mounted.
(='o'=)[root@ub-148:~$]# ls /var/run/docker/netns/
38fc8c67f5b1  9497f5d26b60  ae74a60c0c58  bb93e56b138f  bf54688c9f24
(='o'=)[root@ub-148:~$]# umount /var/run/docker/netns/*
(='o'=)[root@ub-148:~$]# rm /var/run/docker/netns/*

Btw, what is the container output for ip route? Here is mine:
root@02b2713c9a97:/# ip route
default via <IP_ADDRESS> dev eth0
<IP_ADDRESS>/24 dev eth0  proto kernel  scope link  src <IP_ADDRESS>

I put in the world's longest error message but it can really be a pita because all netlink says is the resource is busy.
log.Error("Ensure there are no existing [ ipvlan ] type links and remove with 'ip link del <link_name>'," +
			" also check `/var/run/docker/netns/` for orphaned links to unmount and delete, then restart the plugin")

My host is just a bit newer but I was using the same kernel as you until y-day:
$ uname -a
Linux ub-148 4.3.0-040300rc1-generic #201509160642 SMP Wed Sep 16 10:44:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=15.04
DISTRIB_CODENAME=vivid

Apprec the digging in, we will get support upstream packaged natively relatively soon but getting the early issues knocked out is going to be helpful so thanks again. If still hitting issues we can irc it out Monday too if up for it realtime. Lastly I updated and versioned the repo with new API updates that will work with the latest docker experimental in case you update one, just update the other.

HI Brent, really appreciate you spending so much time with me.
I did try out macvlan plugin driver before this on the same host - that worked great! I upgraded kernel to 3.19 first and then 4.2 to test IPVLAN. I rebooted the node before I started IPVLAN test and have rebooted the node several times after that.
/var/run/docker/netns/ is clean.
As far as my interface em1 going to promiscuous mode its because of running tcpdump atleast in my case.
Here's the routes in one of my container
root@18d9dd6a1343:/# ip route
default via <IP_ADDRESS> dev eth0
<IP_ADDRESS>/22 dev eth0  proto kernel  scope link  src <IP_ADDRESS>
routes in another container
root@eca6f0e96791:/# ip route
default via <IP_ADDRESS> dev eth0
<IP_ADDRESS>/22 dev eth0  proto kernel  scope link  src <IP_ADDRESS>
BTW the interesting thing is if I ping a namespace (setup outside of docker) from within any of these containers I get a response mainly because the namespace ipvlan interface responds to arp request.
// Here if you notice no arp reply for container to container pings
08:41:52.154021 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:41:53.172414 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:41:53.172412 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:41:54.170021 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:41:54.170020 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:41:55.170024 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:41:55.170022 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
// now when I ping a namespace @ <IP_ADDRESS>
08:42:32.368291 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:42:32.368290 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
// now when I ping a namespace @ <IP_ADDRESS>
08:42:56.472324 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
08:42:56.472322 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
Thanks again for your help.
Sharad

Hey Sharad, its my pleasure and very helpful so its me that owes you the thanks. The routes look spot on. I was feeling your pain so installed a 14.10 VM and the one thing I ran into of significance was the Docker experimental version. The wget -qO- https://experimental.docker.com/ | sh doesn't have a manifest for v1.9.0-dev only up to v1.8.
Unfortunately none of that clues me into the issue you are having. What getting the latest experimental fixed was provisioning the link with the latest libnetwork API changes. Good thing is this is going to motivate me to get a Vagrantfile together to share with you. I will work on it tomorrow evening and Im guilty of never actually doing one cause it seems every project I ever work on @dave-tucker always did the vagrantfile lol. So hopefully tomorrow'ish I will push one. Thanks again buddy and check out the docker version for 14.10 in the readme updates if you have a second tomorrow.

Hi Brent,
I have some good news. I cleaned out /var/lib/docker/ from my nodes and after that everything works in l2 mode. I saw a few files under /var/lib/docker/containers/ hanging around from before. What exactly goes into this folder?
I will continue testing to make sure there are no false positives. However I see one more thing which has to do with stability. Once the libnetwork driver hits an issue like I give it a wrong subnet and it reports an error like network not found, I see that I have to restart docker also along with the plugin driver. I can create a separate issue for this if you like.
In the mean time I will start evaluating l3 mode. Did you by any chance try to setup an arp proxy on the host?

Hi Brent,
I now have the latest ipvlan driver code deployed. L2 checked out fine with this code. However L3 does not work at all. This is a docker issue. L3 mode works fine with Linux name spaces and hence not a driver issue. BTW I have BGP installed on all hosts now to create a L3 mesh.
Here are the logs from docker and ipvlan driver
WARN[0000] Running experimental build
WARN[0000] Usage of loopback devices is strongly discouraged for production use. Please use --storage-opt dm.thinpooldev or use man docker to refer to dm.thinpooldev section.
INFO[0006] Option DefaultDriver: ipvlan
INFO[0006] Option DefaultNetwork: foo
INFO[0006] Firewalld running: false
WARN[0006] Your kernel does not support swap memory limit.
INFO[0006] Loading containers: start.
INFO[0006] Loading containers: done.
INFO[0006] Daemon has completed initialization
INFO[0006] Docker daemon                                 commit=a3b04de execdriver=native-0.2 graphdriver=devicemapper version=1.9.0-dev
INFO[0006] Listening for HTTP on unix (/var/run/docker.sock)
INFO[0613] POST /v1.21/containers/create
ERRO[0613] Handler for POST /v1.21/containers/create returned error: No such image: ubuntu:latest
ERRO[0613] HTTP Error                                    err=No such image: ubuntu:latest statusCode=404
INFO[0613] POST /v1.21/images/create?fromImage=ubuntu&tag=latest
INFO[0615] Image manifest for ubuntu:latest has been verified
INFO[0624] POST /v1.21/containers/create
INFO[0624] POST /v1.21/containers/015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30/attach?stderr=1&stdin=1&stdout=1&stream=1
INFO[0624] POST /v1.21/containers/015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30/start
INFO[0624] No non-localhost DNS nameservers are left in resolv.conf. Using default external servers : [nameserver <IP_ADDRESS> nameserver <IP_ADDRESS>]
INFO[0624] IPv6 enabled; Adding default IPv6 external servers : [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]
WARN[0625] exit status 2
ERRO[0625] failed to umount /var/lib/docker/containers/015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30/shm: invalid argument
ERRO[0625] failed to umount /var/lib/docker/containers/015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30/mqueue: invalid argument
ERRO[0625] 015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30: Failed to umount ipc filesystems: failed to cleanup ipc mounts:
invalid argument
invalid argument
ERRO[0625] Error unmounting device 015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30: UnmountDevice: device not-mounted id 015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30
ERRO[0625] Handler for POST /v1.21/containers/015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30/start returned error: Cannot start container 015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30: [8] System error: failed to set gateway while updating gateway: file exists
ERRO[0625] HTTP Error                                    err=Cannot start container 015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30: [8] System error: failed to set gateway while updating gateway: file exists statusCode=500
INFO[0625] DELETE /v1.21/containers/015f1b6bbacc4164c2379bb874e8f40f9269435419199005137ca3d768584b30?v=1
ipvlan driver logs
DEBU[0000] The plugin absolute path and handle is [ /run/docker/plugins/ipvlan.sock ]
INFO[0000] Plugin configuration options are:
container subnet: [<IP_ADDRESS>/24],
container gateway: [],
host interface: [eth0],
mmtu: [1500],
ipvlan mode: [l3]
INFO[0000] IPVlan network driver initialized successfully
DEBU[0039] Handshake completed
DEBU[0039] Capabilities exchange complete
DEBU[0039] /sbin/iptables, [--wait -C POSTROUTING -t nat -s <IP_ADDRESS>/24 -j MASQUERADE]
DEBU[0039] /sbin/iptables, [--wait -I POSTROUTING -t nat -s <IP_ADDRESS>/24 -j MASQUERADE]
DEBU[0039] Adding route for the local ipvlan subnet [ <IP_ADDRESS>/24 ] in the default namespace using the specified host interface [ eth0]
DEBU[0039] The container subnet for this context is [ <IP_ADDRESS>/24 ]
INFO[0039] Allocated container IP: [ <IP_ADDRESS>/24 ]
DEBU[0039] Create endpoint response: &{Interface:{Address:<IP_ADDRESS>/24 AddressIPv6: MacAddress:7a:42:0a:01:02:02}}
DEBU[0039] Create endpoint a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8 &{Interface:{Address:<IP_ADDRESS>/24 AddressIPv6: MacAddress:7a:42:0a:01:02:02}}
DEBU[0039] Endpoint info request: &{NetworkID:f5f6c109686af2e4caedfad7592b3049eedcabae6e5c0e90c8ea68fe8f9ae7d4 EndpointID:a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8}
DEBU[0039] Endpoint info a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8
DEBU[0039] Join request: &{NetworkID:f5f6c109686af2e4caedfad7592b3049eedcabae6e5c0e90c8ea68fe8f9ae7d4 EndpointID:a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8 SandboxKey:/var/run/docker/netns/06971ada2598 Options:map[com.docker.network.generic:map[ChildEndpoints: ParentEndpoints:]]}
INFO[0039] Created ipvlan link: [ a1541 ] with a mode: [ l3 ]
DEBU[0039] Join response: &{Gateway: InterfaceName:{SrcName:a1541 DstName: DstPrefix:eth} StaticRoutes:[0xc82010d260]}
DEBU[0039] Join endpoint f5f6c109686af2e4caedfad7592b3049eedcabae6e5c0e90c8ea68fe8f9ae7d4:a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8 to /var/run/docker/netns/06971ada2598
DEBU[0039] Leave request: &{NetworkID:f5f6c109686af2e4caedfad7592b3049eedcabae6e5c0e90c8ea68fe8f9ae7d4 EndpointID:a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8 Options:map[]}
DEBU[0039] Leave f5f6c109686af2e4caedfad7592b3049eedcabae6e5c0e90c8ea68fe8f9ae7d4:a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8
DEBU[0039] Delete endpoint request: &{NetworkID:f5f6c109686af2e4caedfad7592b3049eedcabae6e5c0e90c8ea68fe8f9ae7d4 EndpointID:a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8}
DEBU[0039] Delete endpoint a1541d7e9cc3011f3db3a290bebe3475220cb2b2040da8429dd08f1648a080e8
DEBU[0039] The requested interface to delete [ a1541 ] was not found on the host: route ip+net: no such network interface
ERRO[0039] The requested interface to delete [ a1541 ] was not found on the host.
FYI I am running Linux L6430-1 4.2.1-040201-generic #201509211431 SMP Mon Sep 21 18:34:44 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
one of the things missing in this kernel version is aufs module - I still went ahead with testing - don't see issues in L2 mode. I know docker needs it but for now I need to get libnetwork working. I cannot understand why docker is trying to set gateway for L3 mode.
FYI in a namespace I add a default route as follows "ip route add <IP_ADDRESS>/0 dev eth0". I only need this to get going. When I add the ip address for the ipvlan interface, I don't provide a mask with the address like in L2 mode and things work perfectly.
Can you please look into this.
Sharad

Hey @shmurthy62 ! Apologies for the delay. Last week was crazy and have been working on a native driver for ipvlan/macvlan. Thanks in part to your great feedback I think we have enough of a PoC with the caveats to pursue pushing it upstream. The IPAM APIs are changing in experimental this week as it comes out of experimental, along with some minor others so I had to get that all rectified. I will ping you with the local branch I will probably just push here so we can collaborate with everyone easier as soon as Madhu's docker integration branch gets pushed into docker/docker feedback on the UX and functionality if you are up to it. Thanks for the help on getting this proofed out Sharad!

Hi Brent,
I hope by "native driver" you mean it will not require external libnetwork driver plugin as it is today - we would prefer this model. The current model is too brittle and has reliability issues. After my experience with current network plugin architecture we are not comfortable with the solution and are looking at using docker with --net=none option and implementing our own solution which I am against. This would be an interim solve till we get something concrete from docker. However I would strongly recommend revisiting the current network plugin model - the plugin should not impact the operation of of docker and vice versa.
I would be happy to test out your native driver and provide feedback.
I am looking for a functionality where we want to call a pre-processor in the name space before our application is executed in the docker container.
Something like this
ip netns exec  preProcessor
ip netns exec  myapp
So 2 images need to be fetched one after another and executed in the same order specified within the container.
Alternatively docker would take a fully qualified path to an application through a command line option and execute it in namespace before executing  application.
We want to hide this from the application developer who would write the docker file. Is there a way to achieve this today?
Once I start testing I would recommend we move the conversation to irc or skype so its more realtime.

Hey @shmurthy62 ! Having an in-tree driver is the goal. This was to shake out any early blockers which you have been super helpful with. The v1.9 API changes are merged for the most part so they will solidify.
The API that is now out of experimental will leverage the docker network command. The -d lets you use multiple drivers since they are bundled into libnetwork without having to run a separate process or container.
Define a network and run some containers:
# Macvlan Bridge Mode
docker network  create  -d macvlan  --subnet=<IP_ADDRESS>/24 --gateway=<IP_ADDRESS> macnet1
docker  run --net=macnet1 -it --rm ubuntu
# IPVlan L2 Mode:
xdock network  create  -d ipvlan  --subnet=<IP_ADDRESS>/24 --gateway=<IP_ADDRESS> ipvnet1
xdock  run --net=ipvnet1 -it --rm ubuntu

Networks associated to different drivers simultaneously.
$ docker network ls
NETWORK ID          NAME                DRIVER
251bd9251032        bridge              bridge
42e5991a44de        macnet1          macvlan
51f32322a2c3         ipvnet1            ipvlan
96a466f03110        none                null
3535f2a67719        host                host

I agree the hard part of having an external driver are the failure scenarios and float in state. Having the generic port creations upstream will take care of the single host operations. From there we can plug in whatever we want externally for state distribution. Something like IPVlan L3 mode doesn't really have a single host implementation since it requires routes to be added to other nodes needing to reach the subnet residing only in the namespace. It could be Etcd, Consul, whatever someone wants if they prefer a distributed datastore approach, or another one I am excited about is the GoBGP implementation that could propagate host routes or a summary for a pod etc. Will likely do the L3 mode experimentation here if there aren't any objections since the ipvlan functionality will be out of the box without a remote driver.
The generic L2 use cases are super simple which is very attractive and something as simple as a ToR could route the L2 network getting plumbed up from a phy eth iface or a (m)LAG bonding.
Will catch up with you, curious what your thoughts on L3 state distribution is too. The OSRG folks are super guys and I have been a fan of creating a Quagga replacement :)

Thanks for considering making ipvlan a native driver. I would recommend sourcing config from a file - default could be /etc/docker/docker.conf. CLI options could override that. In production environment file is preferable.
Re l2 mode:
One of the things I have done in our lab is to figure out default gw by parsing output of ip route. That way you are not forced to provision it. This might be a preferred way in nodes where we have only 1 nic. So if the default gw is not specified in config file you should figure it out. As far as IP address allocation is concerned you should delegate it to an IPAM. You can have a default IPAM within docker so people can run it standalone. In conf file, you provide a way to specify IPAM URL. However in a production environment, it should call out to an IPAM service to fetch an IP for the container and subnet mask. This should be a rest interface and the api could take a host IP and fetch an IP range or ip and subnet mask. This way both options are deployable. If you ask for a range you should maintain state (binding of IP to contianer ID) so you know what is allocated - this way you can reuse an IP once the container to which it was allocated has been removed. I have to admit that my thoughts are evolving in this area - might take a little time to firm up - but I would like to brainstorm with you guys on this.
I agree L2 is super simple and will definitely work for us. It however will not allow IP mobility as the IP addresses will have to fall in the host's subnet.
Re l3 mode:
The beauty of this solve is it gives IP mobility. For us IP mobility is a want not a must. I have setup a BGP mesh using quagga in our lab. My current setup bypasses docker networking but I still use dockers. I think all docker needs to do is to setup a default route <IP_ADDRESS>/0 via the ipvlan interface. Then ask IPAM for an IP address. Leave it to IPAM to figure out what IP address to dole out. In this mode the containers can have IP addresses outside the host subnet. This will provide IP mobility. I have a BGP instance per host peering with a BGP RR (Quagga Route server is even better). The BGP RR is at the leaf and will peer with a BGP at the spine level.
In my current lab setup I am actually assigning a subnet to each host for docker containers and advertising it through BGP. However what I want to test is to inject a route per container IP (/32 address) in to the linux kernel and let BGP advertise it. I picked Quagga mainly because I can make it accept routes from the kernel and advertise it. I do not know how to do this with GoBGP (let me know if you know how to do it). Ideally docker should insert and remove the routes from the kernel based on the container lifecycle. If you can do that you leave it to the system integrator then to figure out how to advertise that route.
I am currently prototyping this experience and will provide feedback once my thoughts are more concrete. L3 is a very complex setup and I want to make sure all failure modes are covered.
I would want you to consider providing a REST API to expose config, state and stats. This way controllers can expose this to an operator. This will be very useful in large setups.

Brent,
I have experienced the same issue mentioned above by Sharad in the L3 case. I found that the remote driver is making assumptions without delegating all the controls to the remote plugin.
Our use case is to have pure routed network without bridging. The current implementation of L3 fails to create the default route with :
Error response from daemon: Cannot start container 1c5b403b5e7e1f1627f1d33598fb9d7a5b045b6ccaacaa737a6a07580121df57: [8] System error: failed to set gateway while updating gateway: file exists

if the content of the join response is :
    respIface := &netApi.InterfaceName{
        SrcName: tempName,
        DstPrefix: "eth",
    }
    sandboxRoute := netApi.StaticRoute{
        Destination: "<IP_ADDRESS>/0",
        RouteType:     1, // CONNECTED
        NextHop:    "",
    }
    resp := &netApi.JoinResponse{
        StaticRoutes: []netApi.StaticRoute{sandboxRoute},
        InterfaceName: respIface,
    }

The daemon takes additional liberties if you just return:
    respIface := &netApi.InterfaceName{
        SrcName: tempName,
        DstPrefix: "eth",
    }
    resp := &netApi.JoinResponse{
        InterfaceName: respIface,
    }

in which case it creates a bridged interface anyway on the default docker network ...
I would be interested to understand the future of the plugin model, and how to make this work.

I found the origin of the problem and filed https://github.com/docker/libnetwork/issues/775
I've done limited testing for my use case and it seems to solve this problem.

@jc-m @nerdalert
I am still seeing this issue when trying to use ipvlan in L3 mode. I have added disable default gw setting in docker network create command as well.
docker: Error response from daemon: Cannot start container 2dcb35966fcce550050fbe0e781c7328e94a6cd68953536e68cc1a9c375c253f: [9] System error: failed to set gateway while updating gateway: file exists.
Can you please help me, I am not sure what am I missing.
Steps:
**Step 1 **- Run ipvlan binary:
./ipvlan-docker-plugin-0.2-Linux-x86_64 -d --mode="l3" --ipvlan-subnet="192.169.57/24" --host-interface="enp0s3" --gateway="<IP_ADDRESS>"&
Step 2 - Create Docker network:
docker network  create  -d ipvlan  --subnet=<IP_ADDRESS>/24 --gateway=<IP_ADDRESS> -o host_iface=enp0s3 -o mode=l3  -o com.docker.network.driver.disable_default_gateway net1
DEBU[0015] Network Create Called: [ {NetworkID:5a6c6ff0a97e9cf712165a6670445b55995c232e7528e521ae0a2f44f3ebaf48 Options:map[com.docker.network.generic:map[com.docker.network.driver.disable_default_gateway: host_iface:enp0s3 mode:l3]] IpV4Data:[{AddressSpace: Pool:<IP_ADDRESS>/24 Gateway:<IP_ADDRESS>/24 AuxAddresses:map[]}] ipV6Data:[]} ]
DEBU[0015] Libnetwork Opts Sent: [ com.docker.network.driver.disable_default_gateway ] Value: [  ]
DEBU[0015] Libnetwork Opts Sent: [ host_iface ] Value: [ enp0s3 ]
DEBU[0015] Libnetwork Opts Sent: [ mode ] Value: [ l3 ]
DEBU[0015] Ipvlan mode is L3 [ l3 ]
DEBU[0015] Adding route for the local ipvlan subnet [ <IP_ADDRESS>/24 ] in the default namespace using the specified host interface [ enp0s3]
Step 3: Start the contaiener, this is where it fails -
docker run --net=net1 -it --rm myubuntu bash DEBU[0019] The container subnet for this context is [ <IP_ADDRESS>/24 ]
INFO[0019] Allocated container IP: [ <IP_ADDRESS>/24 ]
DEBU[0019] Create endpoint response: &{Interface:{Address: AddressIPv6: MacAddress:7a:42:00:00:00:00}}
DEBU[0019] Create endpoint 1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5 &{Interface:{Address: AddressIPv6: MacAddress:7a:42:00:00:00:00}}
DEBU[0019] Join request: &{NetworkID:5a6c6ff0a97e9cf712165a6670445b55995c232e7528e521ae0a2f44f3ebaf48 EndpointID:1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5 SandboxKey:/var/run/docker/netns/0add445bda28 Options:map[]}
INFO[0019] Created ipvlan link: [ 1410c ] with a mode: [ l3 ]
DEBU[0019] Join response: &{Gateway: InterfaceName:{SrcName:1410c DstName: DstPrefix:eth} StaticRoutes:[0xc8201ab830] DisableGatewayService:false}
DEBU[0019] Join endpoint 5a6c6ff0a97e9cf712165a6670445b55995c232e7528e521ae0a2f44f3ebaf48:1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5 to /var/run/docker/netns/0add445bda28
DEBU[0020] Leave request: &{NetworkID:5a6c6ff0a97e9cf712165a6670445b55995c232e7528e521ae0a2f44f3ebaf48 EndpointID:1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5 Options:map[]}
DEBU[0020] Leave 5a6c6ff0a97e9cf712165a6670445b55995c232e7528e521ae0a2f44f3ebaf48:1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5
DEBU[0020] Delete endpoint request: &{NetworkID:5a6c6ff0a97e9cf712165a6670445b55995c232e7528e521ae0a2f44f3ebaf48 EndpointID:1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5}
DEBU[0020] Delete endpoint 1410ccd52f0b32401bebf404caedb68a3753613d302d917d48886f9b624f6cb5
DEBU[0020] The requested interface to delete [ 1410c ] was not found on the host: route ip+net: no such network interface
ERRO[0020] The requested interface to delete [ 1410c ] was not found on the host.
docker: Error response from daemon: Cannot start container 2dcb35966fcce550050fbe0e781c7328e94a6cd68953536e68cc1a9c375c253f: [9] System error: failed to set gateway while updating gateway: file exists.
Thanks

Hey @shmurthy62  @jc-m ! Wanted to send this PR to you guys since you have been so helpful and patient on ipvlan issues. We are close to getting it into libnetwork natively. There is a link in the PR notes if you want to download the binary or compile it and give it a go. If its good for you a :+1: is appreciated. Thanks again, will be happy to get this in natively and then we can start doing fun state distribution and services! Thanks again for your help and patience guys!


broker: Persistent storage for redis

This PR adds persistent storage where data stored in redis is persisted even when the pod is deleted as a result of creating or deleting a binding.

LFAD.

Merged on the commandline.


[Feature suggestion] Controller configuration data support in base Module class

\yii\base\Controller accepts configuration data array as constructor param, but \yii\base\Module::createController() nor \yii\base\Module:: createControllerByID() don't. I suggest to add param $config = [] to both of these methods and pass it further to created controller.

Can't be done in 2.0.x because of backwards compatibility.

'controllerMap'=>[
   'my'=>[
        'class' => 'app\controllers\MyController',
        'param1' => 'something',
        'param2' => 'another',
    ]
]

@mikolaj-milewski You could configure the controllers via Dependency Injection
http://www.yiiframework.com/doc-2.0/guide-concept-di-container.html


Add missing examples to list of examples

added several examples to the list of examples
Example I.1 now has the caption "Example I.1"
caption "Examples 1.1. " was changes to "Example 2.1. " (because it is in chapter/section 2 and not in chapter/section 1)
changed "5.13." to "5.14." because a new examples 5.13 was inserted
appended a space to caption "Table D.1." => "Table D.1. "
added captions Table I.1.  and Table I.1.

See issue #418 for discussion of these changes.
Release checklist

[ ] Authors updated in cf-conventions.adoc? (no reason to do so)
[ ] Next version in cf-conventions.adoc up to date? Versioning inspired by SemVer. (no reason to do so)
[x] history.adoc up to date?
[ ] Conformance document up-to-date? (no reason to do so)

For maintainers
After the merge remember to delete the source branch.
Tags are set at the conclusion of the annual meeting; until then master always is a draft for the next version.

Replaced by #450


Added borderValue parameter to rotate_bound().

Hi!
This is such a useful package. I have been rotating images and wanted to have some control over the "extra space" revealed when the image is rotated. This is catered for by the borderValue parameter of the warpAffine() function.
This allows you to specify the "background colour" which is revealed when the image is rotated. By default this is black. Specifying borderValue=(255, 255, 255) will make this new portion of the image white.
Thanks,
Andrew.

Applied same change for rotate().

this is indeed useful. @jrosebr1 , could you mind merging the pull request?

I find this still a useful addition :)

Yes, this would proof useful


(Minor) auth.ts uses deprecated http.ServerRequest

Fixes #477

[ ] npm run test succeeds.
[x] npm run lint succeeds.
[ ] Appropriate changes to README are included in PR.

'npm run test` had errors but they do not seem related to the changes in PR
No need to update README.md

I have a CLA now...

https://github.com/google/clasp/pull/478#issue-242228177
In markdown you can check by entering x between [and] .
- [ ] no checked
- [x] checked

@takanakahiko I fixed it. Thanks.

Thanks!


[BUG] - Deployment on minikube fails: minio and postgresql charts not found

OS system and architecture in which you are running QHub
Linux 5.18.1-arch1-1 x86_64
Expected behavior
Following the minikube guide, I expect to be able to deploy QHub locally.
Actual behavior
Upon doing python -m qhub deploy --config qhub-config.yaml --disable-prompt, the deployment fails. All the preceeding steps work execute successfully.
It looks like minio 6.7.4 and postgresql 10.13.12 charts aren't present in https://charts.bitnami.com/bitnami.
How to Reproduce the problem?
pip install qhub
minikube start --cpus 6 --memory 8192 --driver=docker
python qhub/tests/scripts/minikube-loadbalancer-ip.py
minikube addons enable metallb
mkdir qhub-local-deployment
cd qhub-local-deployment
mkdir data
cd data
qhub init local --project=localdeployment --domain github-actions.qhub.dev --auth-provider=password --terraform-state=local
python -m qhub deploy --config qhub-config.yaml --disable-prompt

Command output
...
[terraform]: ╷
[terraform]: │ Error: chart "minio" version "6.7.4" not found in https://charts.bitnami.com/bitnami repository
[terraform]: │
[terraform]: │   with module.kubernetes-conda-store-server.module.minio.helm_release.minio,
[terraform]: │   on modules/kubernetes/services/minio/main.tf line 7, in resource "helm_release" "minio":
[terraform]: │    7: resource "helm_release" "minio" {
[terraform]: │
[terraform]: ╵
[terraform]: ╷
[terraform]: │ Error: chart "postgresql" version "10.13.12" not found in https://charts.bitnami.com/bitnami repository
[terraform]: │
[terraform]: │   with module.kubernetes-conda-store-server.module.postgresql.helm_release.postgresql,
[terraform]: │   on modules/kubernetes/services/postgresql/main.tf line 7, in resource "helm_release" "postgresql":
[terraform]: │    7: resource "helm_release" "postgresql" {
[terraform]: │
[terraform]: ╵

Problem encountered: Terraform error

Versions and dependencies used.
[~]$ kubectl version --output=yaml
clientVersion:
  buildDate: "2022-05-27T18:33:09Z"
  compiler: gc
  gitCommit: 3ddd0f45aa91e2f30c70734b175631bec5b5825a
  gitTreeState: archive
  gitVersion: v1.24.1
  goVersion: go1.18.2
  major: "1"
  minor: "24"
  platform: linux/amd64
kustomizeVersion: v4.5.4
serverVersion:
  buildDate: "2022-01-25T21:19:12Z"
  compiler: gc
  gitCommit: 816c97ab8cff8a1c72eccca1026f7820e93e0d25
  gitTreeState: clean
  gitVersion: v1.23.3
  goVersion: go1.17.6
  major: "1"
  minor: "23"
  platform: linux/amd64

Compute environment
Tried with QHub 0.4.1 (from pip install qhub) as well as the latest commit (2687f13f54e3833234fc4282587be16521f0c84a).
minikube
Integrations
No response
Anything else?
No response

Hey @peytondmurray thanks for raising this! I will try to reproduce this error and report back 👍

It looks like those versions of the minio and postgresql charts have been deprecated in some way as they are no longer hosted on https://charts.bitnami.com/bitnami (as mentioned above).
A simple solution would be to pin these charts to a newer versions. That said there might have been a reason @costrouc pinned minio in the first place. Here is the comment right above the minio version: # last release that was Apache-2.0

Agreed, I saw that comment. Sounds like it might be best to wait for comment from @costrouc about supporting newer versions.

This bitnami issue seems to suggest that a recent update that bitnami made to their index.yaml is causing trouble for many other projects as well.
I get a 404 error when I try to access their outdated index.yaml:
https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami

solved by #1308


Query execution failed

Table:
CREATE TABLE `virtual_users` (
`id` INT NOT NULL AUTO_INCREMENT,
`domain_id` INT NOT NULL,
`password` VARCHAR(106) NOT NULL,
`email` VARCHAR(120) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Query:
INSERT INTO `virtual_users` (`id`, `domain_id`, `password` , `email`)
VALUES('1', '1', ENCRYPT('gppai', CONCAT('$6$', SHA2(RANDOM_BYTES(64), '256'))), 'adao.duque@example.com.br');

Message error:

But, run the query directly on the Mysql command line, this works perfectly.
What is going on ?
Thank you.

Sorry for my mistake.
This is not a bug in DBeaver

Query in mysql command line works perfectly.

This is because encrypt functions returns NULL in your case. Maybe you are testing this on Windows machine (while ENCRYPT works ok only if MySQL server is runnign on some Linux machine).
Check this: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_encrypt

Thank you for response.


Docker updates /etc/hosts for all containers, I don't want it to

I swear this is recent - I have never seen it before 1.8 (but maybe I am nuts).  It used to be that if you used docker links, the /etc/hosts file would be updated.  Now it seems that ALL containers (even those without links) have their /etc/hosts files updated to know about all containers on the local machine.
Sniffing around this look like it might be libnetwork related, but I am not 100% sure.
How do I turn this off?  Container names are not what we want to be exposing to users.
@mavenugo

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

@thockin this was introduced as part of the inbuilt service discovery functionality in libnetwork. we have plans for making service discovery as a pluggable component with which the default battery can be replaced by other service discovery plugin.
Having said that, we are tracking this issue & i just opened https://github.com/docker/libnetwork/issues/567 in the libnetwork project. It will be addressed as part of 1.9.

So no way to turn that off?
On Sun, Sep 27, 2015 at 10:03 PM, Madhu Venugopal<EMAIL_ADDRESS>wrote:

@thockin https://github.com/thockin this was introduced as part of the
inbuilt service discovery functionality in libnetwork. we have plans for
making service discovery as a pluggable component with which the default
battery can be replaced by other service discovery plugin.
Having said that, we are tracking this issue & i just opened
docker/libnetwork#567 https://github.com/docker/libnetwork/issues/567
in the libnetwork project. It will be addressed as part of 1.9.
—
Reply to this email directly or view it on GitHub
https://github.com/docker/docker/issues/16619#issuecomment-143641789.

@thockin you mean in 1.8 ? unfortunately not.

It looks like the workaround was also broken: #10316

And the workaround was apparently fixed:  #14613.

A change that doesn't help us: #17230

Related: https://github.com/docker/libnetwork/issues/519

What's worse, if you use the --link option as before, there now exist two entries for the same container name in /etc/hosts:
<IP_ADDRESS>	server 59cef7f41bbc
<IP_ADDRESS>	server

This would be fine, since they're not inconsistent -- except that when the server container is subsequently stopped, the /etc/hosts entry remains for the stopped container:
<IP_ADDRESS>	server 59cef7f41bbc

Then, if you remove the server container, and create a new one also called server, the /etc/hosts file in the linked container then becomes inconsistent!
<IP_ADDRESS>	server 59cef7f41bbc
<IP_ADDRESS>	server

See this issue https://github.com/docker/docker/issues/17190#issuecomment-150743021 for more information


Use FPGA manager for Zynq-7000 designs

Looks like xdevcfg has disappeared but luckily the ZU+ code download works

Hi Peter,
You already might know that configuration in MPSoC is handled by CSU unit. There is also an FPGA configuration library, at least for standalone OS, that provides simple API to transfer bitstream.
Regards,
Parimal
From: Peter Ogden<EMAIL_ADDRESS>Sent: Wednesday, September 5, 2018 4:09 AM
To: Xilinx/PYNQ<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [Xilinx/PYNQ] Use FPGA manager for Zynq-7000 designs (#716)
EXTERNAL EMAIL
Looks like xdevcfg has disappeared but luckily the ZU+ code download works

You can view, comment on, or merge this pull request online at:
https://github.com/Xilinx/PYNQ/pull/716
Commit Summary

Use FPGA manager for Zynq-7000 designs

File Changes

M pynq/pl.pyhttps://github.com/Xilinx/PYNQ/pull/716/files#diff-0 (59)

Patch Links:

https://github.com/Xilinx/PYNQ/pull/716.patch
https://github.com/Xilinx/PYNQ/pull/716.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/Xilinx/PYNQ/pull/716, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AI1H1-VB9VbXqPK84VVCXJaxd9nF57Cwks5uX7DXgaJpZM4Wars1.

i guess this PR will conflict with #715. Both of these 2 PRs modify pl.py. Need to fix.

Doesn't conflict - I've tried the merge


Improved UI Design

Title and Issue number

**Title: Improving UI design of contribution page
**Issue No.: #348
Close #398
Screenshots/Video (mandatory)
before

after

before

after

before

after

before

after

I added margin to the top and bottom of the feedback page for better spacing.
I already added hover-effect on social media.
Checklist:

[ x] I have mentioned the issue number in my Pull Request.
[ x] I have commented my code, particularly in hard-to-understand areas.
[x ] I have gone through the contributing.md file before contributing.

Are you contributing under any Open-source programme?

[ x] I'm a GSSOC-EXT contributor
[ x] I'm a HACKTOBERFEST contributor

@Anuj3553 here show again conflict plz enable the resolve conflict button so that i can resolve it

@Anuj3553 here show again conflict plz enable the resolve conflict button so that i can resolve it

There is no option to enable the button -

@Anuj3553 then this work will not show my contribution

@Anuj3553 hey what about my PR

@Anuj3553 hey what about my PR

Unfortunately the event is ended and you have to clear your conflicts but there are still conflict and I'm also do nothing for this. Request the event management for this issue.


Removed the check style plugin dependencies

Removed the check style plugin dependencies

I guess if we do this we'd better ask Emma to create stories for restoring CheckStyle without security and build issues?


Enable different handling of cumulative data for test runs

Don't know yet how we do this, but we need to ensure that test run outputs don't contaminate real cumulative data, even if test runs are done in parallel with real runs (which means just dumping the cumulative data folders at the end of the run is not sufficient).

in exitlog branch, by @AmandaBirmingham


Device Client v1.9 release

Motivation
New release for Device Client.
Modifications
Change summary

Updated version to 1.9x
Create new v1.9 tag
Added Change log with issues resolved and changes made since last release.

Testing
N/A
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Note: The CI test for AmazonLinux are failing because of a known issue. The issue/bug is in the Docker image used for testing the code change and not in the Device Client code itself. Team is working on fixing the docker base images.


Make sure that relative URLs actually work when filtering

It's possible for Url::parse("xxx") == Err(url::ParseError::RelativeUrlWithoutBase) to evaluate to true,
but for the eventual url.join("xxx") to still fail with a parse erro, depending on what "xxx" actually is.
To detect this, we check that joining succeeds in addition to the normal checks for relative URLs.
This PR also runs "cargo fmt".
Fixes #136

bors r+


Updated readme with instruction to users to use an updated version of go version

What type of PR is this?
documentation
What this PR does / Why we need it?
It instructs the user to use an updated go version so that there won't be any kube config files left after testing
Which Jira/Github issue(s) does this PR fix?
Resolves #OSD-17636
Pre-checks (if applicable)

[ ] Ran unit tests locally
[ ] Validated the changes in a cluster
[X] Included documentation changes with PR

Codecov Report

Merging #170 (a730b4b) into main (fd84937) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #170   +/-   ##
=======================================
  Coverage   49.59%   49.59%
=======================================
  Files          51       51
  Lines        3353     3353
=======================================
  Hits         1663     1663
  Misses       1411     1411
  Partials      279      279

/lgtm


Lots of FP from apple imessages

Hi there,
apple devices with iMessages using:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0
Will need to disable this plugin and/or modify lua/rule on my own to take care of these requests! (lots of)
Thanks for your work!

Hi @azurit
this is the user-agent being registered for some requests at nginx logs, and this user-agent is triggering the last version of fakebot plugin configured and installed with modsecurity v4, so rule 9504110 is denying because of the user agent.
If you research a bit, you may see this user agent is commonly used by some apple applications (search for imessages too).
As it contains words reserved for affected robots, lua is performing its checks.  But honestly, the Apple product seems to be giving out wrong information about it's identity and this plugin was designed to detect such anomalies.
The fact that Apple is affected does not make the false claim any better.
Did you open an issue with Apple?
If we get a response from Apple with a technical reason why it's OK to submit this UA in this situation, then we're happy to reconsider. But until then I do not see us changing the plugin.
What you can do it creating a rule exclusion for this situation. Do you need help with that.

Apple sucks, nothing new under the sun.. hehehe
Well, I am just a sysadmin using your excellent plugin, but facing this problem with legit requests... Will just create an exclusion for this user-agent better than disabling.... you are certain ;)
Thanks!

@feanorknd Thank you, closing this issue!

Sounds like a plan. And sorry for the inconvenience.


[CBR-503] Refactor script-runner testcases; fix testHFM

Description
I had repeated myself in testHardForkMechanism, by copying the
body of test4. Due to (what seems like) a botched rebase, my
CBR-503 PR was broken, and because of the duplication, I didn't
catch it in review. This commit factors out the shared logic, and
does checking (confSetBvAndSv) to make sure proposals are sane.
I also fix testHFM by making the BlockVersions and SoftwareVersions
agree.
Linked issue
https://iohk.myjetbrains.com/youtrack/issue/CBR-503
Type of change

[x] 🐞 Bug fix (non-breaking change which fixes an issue)
[~] 🛠 New feature (non-breaking change which adds functionality)
[~] ⚠ Breaking change (fix or feature that would cause existing functionality to change)
[x] 🏭 Refactoring that does not change existing functionality but does improve things like code readability, structure etc
[x] 🔨 New or improved tests for existing code
[~] ⛑ git-flow chore (backport, hotfix, etc)

Developer checklist

[x] I have read the style guide document, and my code follows the code style of this project.
[x] If my code deals with exceptions, it follows the guidelines.
[x] I have updated any documentation accordingly, if needed. Documentation changes can be reflected in opening a PR on cardanodocs.com, amending the inline Haddock comments, any relevant README file or one of the document listed in the docs directory.
[x] CHANGELOG entry has been added and is linked to the correct PR on GitHub.

Testing checklist

[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

QA Steps
See script-runner/README.md
How to merge
Send the message bors r+ to merge this PR. For more information, see docs/how-to/bors.md.

bors r+

bors r+

bors r+


Initial version of ReduxFX sample

This is the initial version of the ReduxFX sample. It will change in the future as ReduxFX is still evolving.

Many thanks for the PR. Really cool to see redux with JavaFX.
Simply send new Pull Requests when you like to change something or update to a newer version of ReduxFX


Update pool info periodically

Closed by https://github.com/foxypool/chia-blockchain/commit/b6620ee156d8eea6db50b4bed760bdaeba2b53b8


IONIC 4.2 PRO COMPILING VERY SLOW

Bug Report
Ionic Info
Run ionic info from a terminal/cmd prompt and paste the output below.
ionic (Ionic CLI)          : 4.2.0 (/home/jacques/.npm-global/lib/node_modules/ionic)
Ionic Framework            : @ionic/angular 4.0.0-beta.12
@angular-devkit/core       : 0.7.5
@angular-devkit/schematics : 0.7.5
@angular/cli               : 6.1.5
@ionic/ng-toolkit          : 1.1.0
@ionic/schematics-angular  : 1.0.7
Cordova:
cordova (Cordova CLI) : 8.1.1<EMAIL_ADDRESS>Cordova Platforms     : not available
Cordova Plugins       : not available
System:
Android SDK Tools : 26.1.1 (/home/jacques/DevTools)
NodeJS            : v9.11.2 (/usr/bin/node)
npm               : 5.6.0
OS                : Linux 4.15
insert the output from ionic info here
no adverse output
Describe the Bug
The system is very slow at compiling even for small changes.
Steps to Reproduce
Edit a project and serve it.
Expected Behavior
Fast compile
Additional Context
List any other information that is relevant to your issue. Stack traces, related issues, suggestions on how to fix, Stack Overflow links, forum links, screenshots, OS if applicable, etc.

Updating today our app to last beta.12 and ionic-cli 4.1.2 makes the app compiling incredible slow.
If we run a clean starter template we have to wait about 1 minutes before app is compiled at all and first page is loaded. If we run it on our very big app the compiling is never finished, at least not for the time we are waiting for
We have also a Linux machine, like the issue owner
ionic (Ionic CLI)          : 4.1.2 (/usr/lib/node_modules/ionic)
   Ionic Framework            : @ionic/angular 4.0.0-beta.12
   @angular-devkit/core       : 0.7.5
   @angular-devkit/schematics : 0.7.5
   @angular/cli               : 6.1.5
   @ionic/ng-toolkit          : 1.1.0
   @ionic/schematics-angular  : 1.0.7

Cordova:
   cordova (Cordova CLI) : 8.1.1<EMAIL_ADDRESS>   Cordova Platforms     : android 7.1.1
   Cordova Plugins       : cordova-plugin-ionic-keyboard 2.1.3, cordova-plugin-ionic-webview 2.2.0, (and 4 other plugins)

System:
   Android SDK Tools : 26.1.1 (/home/abc/software/android/sdk)
   NodeJS            : v8.11.1 (/usr/bin/node)
   npm               : 6.4.1
   OS                : Linux 4.15

Hi, I need to upgrade to Node v10 to fix this issue.
see: https://github.com/ionic-team/ionic/issues/15844

Thank you for the issue. I am going to close this as a duplicate of https://github.com/ionic-team/ionic/issues/15844


Border smoothness is lost on Android devices

Type
Cosmetic
Bug description
On Android devices, the borders are jagged as when using the Vulkan renderer on the desktop (like https://github.com/ppy/osu/discussions/27659#discussioncomment-9106738).
Screenshots or videos
Android (OpenGL)

Linux (OpenGL)

Version
2024.412.1
Logs
logs.zip

Perhaps it's that this is a mediump:
https://github.com/ppy/osu-framework/blob/d73af6ccc9a197e63139711d2cb1df964f82fd0d/osu.Framework/Resources/Shaders/Internal/sh_GlobalUniforms.h#L30


How to connect hub to persistent storage?

Our group wants to use the hub to collaborate on our ICESat-2 research project. We have several people across multiple institutions who are already using Jupyter notebooks for analysis and there's a lot of overlap in the data that everyone is using, including data products that are stored on DAACs as well as "intermediate" data products that are being derived from the DAAC datasets. Ideally, what we need is common cloud storage that we can share so that we all have access to these shared datasets.
Is there a way to connect the hub to persistent cloud storage? For example, could we set up an S3 bucket for our team to use that can be directly accessed by the hub?
Right now, we do a lot of file transfer back and forth between team members and we're storing the same data in multiple (local) places. Consolidating this data would make things a lot more efficient for us!

So, hubs maintained by 2i2c for research generally have two object storage env vars set: https://infrastructure.2i2c.org/en/latest/topic/features.html#scratch-buckets-on-object-storage
SCRATCH_BUCKET is a s3 bucket with read/write permissions to all users, but lifecycle set to delete after every 7 days. This helps with intermediate products, and makes sharing easy - while SCRATCH_BUCKET env var for each user will have their username interpolated into it, users can access each others'. So you can write into your scratch bucket and someone else can read from it. It's also possible to arbitrarily create project specific ones - there's basically no access control at the path level :)
There's also PERSISTENT_BUCKET which works exactly the same as SCRATCH_BUCKET but without the 7 day wipe rule. This allows for longer term sharing of intermediate products. HOWEVER, this can also baloon costs so someone needs to keep an eye on it.
I see this is currently not enabled on the cryo hub. Would you like us to enable it?

Thanks for everyone's responses so far. To answer @weiji14, I'll gather some better estimates from my group but we would be sharing data files that are mostly on the order of 100 MB, with some that are as large as 10 GB. It's possible we'd have some that are larger than that but not many. Again, I can get a better estimate next week. And we're looking at about 6-8 users who would be reading this data directly from the S3 storage.
As far as syncthing, I don't think it's the way for us to go for two reasons. One is that, as @weiji14 said, we'd be storing duplicate copies of data on everyone's home directories on the DaskHub (if I understand how it works correctly). The second is that I'd prefer not to make everyone sync every time a dataset was updated, although we can certainly live with this. The third is that I think we'd fill up everyone's home directory storage pretty quickly with the data.
We would definitely want this data to persist for the life of the project so PERSISTENT_BUCKET would be what's needed. @yuvipanda - I totally understand the issue of balooning cost. Maybe the next step is for me to get a better total estimate and we can see how much that would cost?

Hi all - I wanted to give an update on this. My team decided to purchase AWS S3 storage that's dedicated to our project (funded by NASA ROSES). I created a new bucket in us-west-2 and created access keys for team members to be able to access the bucket. I'm in the process of writing a new entry for the CryoInTheCloud JupyterBook to describe the process of setting this up.

@tsnow03 - my team has been able to set up a project-specific AWS S3 bucket, and set the user policy so that the team can access it from CryoHub. I can write up a short JupyterBook entry for how to do this but I'm still having trouble setting up the conda environment. I'm using conda env create -f conda/environment.yml and trying on my laptop and on the CyoHub.
On my laptop, I get this error:
Solving environment: failed

ResolvePackageNotFound:
  - qgis==3.18.3
  - syncthing==1.22.1
  - websockify[version='>=0.10']

On the CryoHub, it times out while Solving environment. Any tips for getting the environment replicated?
Side note - I also tried the hub-image environment.yml and getting a similar error on my laptop:
Solving environment: failed

ResolvePackageNotFound:
  - qgis==3.28.2
  - syncthing==1.22.1
  - websockify[version='>=0.10']

Yes! I'm so sorry this hasn't been resolved yet. I'm working on this right now. I'll let you know once I have it updated!

No worries at all!

@Denis Felikson I’ve played with the environment.yml a bit. Short story is that you need to have osx>=10.15 to make this work on a local computer. Please let me know if this is a problem for you and you absolutely do not want to upgrade your osx. I can continue working to solve our environment for osx<10.15!

@tsnow03 - do you mean osx 12.15 (not 10.15)? I have 12.6 installed. I'll upgrade to 13.X eventually but NASA isn't letting us upgrade yet (they're still doing testing of that OS).
I'll try to see if I can get at least part of the environment configured so that I can get the JupyterBook up and running locally. That's all that I want to try and do right now.
Thanks for your work on this! And there's not a huge rush - this can wait until I upgrade.

That is odd. It should be 10.15 (better than Mojave) but running with mamba would tell you for sure. It probably makes more sense to just create a markdown file and commit/create a pull request and then I can test it during the review.

Thanks to @tsnow03 for pointing out that I should try mamba. I was actually able to get the environment configured on my laptop without error. Here's what I did:

conda create --name mamba
conda activate mamba
conda install mamba -c conda-forge
mamba env create --name cryocloud --file conda/environment.yml
conda activate cryocloud
jupyter-book build book

At this point, I got an error: Could not import jupytext.reads (needed for MyST-NB nb_custom_formats: .py) (exception: No module named 'jupytext')
So, I did:

conda install jupytext -c conda-forge
jupyter-book build book

And that seemed to work! I was able to open book/_build/html/index.html and it looked good in my browser.
I suggest to add jupytext to conda/environment.yml.

Great! I'm glad you got it working. I'll checkout the issue with jupytext and look into adding it. I'm accepting your Jupyter Book input. Are we ready to close this issue or is there something still outstanding?

Let's go ahead and close it. Thanks, @tsnow03!


central_bank_of_republic_turkey is not working

Swap::latest('USD/TRY')
The chain resulted in 1 exception(s): ErrorException: Undefined offset: 0
../vendor/florianv/exchanger/src/Service/Chain.php
Line 66
May be central_bank_of_republic_turkey changed service ?
https://www.tcmb.gov.tr/kurlar/kurlar_tr.html

The Central Bank of the Republic of Turkey (TCMB) no longer allows HTTP requests, redirecting to HTTPS. This causes the file not to be found; hence, throws the error.
Made a PR for this => https://github.com/florianv/exchanger/pull/116

Thanks @msamdan for reporting and @uguurozkan for fixint it 👍
It has been released in version 2.5.1 of exchanger, so you'll have to do a composer update.


SQL 错误 [1105] [HY000]: Unknown system variable 'transaction_read_only'

System information:

Operating system (distribution) and version: MacBook Pro M1
DBeaver version: <IP_ADDRESS>112040536

Connection specification:

Database name and version: mycat （mysql proxy software）
Driver name: mysql
Do you use tunnels or proxies: SSH

Describe the problem you're observing:

Steps to reproduce, if exist:
I hope this error can be ignored to facilitate SQL execution and get the response results.

Hi @tianmingxing
Can you please tell me more about your issue? When did this issue appear? Also, a full error log will be very helpful.

Hi @tianmingxing
Can you please tell me more about your issue? When did this issue appear? Also, a full error log will be very helpful.

Please give me some guidelines on how to collect the complete log!
There is another situation to report. When I connect to the database with Navicat, it works normally, and it seems to do better.

@tianmingxing Can you try creating a connection with MySQL 5 driver?

@tianmingxing Can you try creating a connection with MySQL 5 driver?

SQL 错误 [S1000]: Could not retrieve transaction read-only status from server
Could not retrieve transaction read-only status from server
Could not retrieve transaction read-only status from server
Unknown system variable 'transaction_read_only'
Unknown system variable 'transaction_read_only'

@tianmingxing I meant using this driver when creating a connection

Or if you are using a custom copy, to copy it from this one.

@Matvey16 @LonwoLonwo  The solution you mentioned has no effect。

If only unrecognized variables appear, can this situation be ignored? Do not interrupt the current operation, or the problem may be solved.

@tianmingxing For now it can't be ignored, but the last possible workaround is to use MySQL (Old) driver to create a connection. It works as a generic type, so would probably ignore this error. Please tell if it worked, otherwise we will need to dig deeper.

use MySQL (Old) drive success! tks!


Add a README section to my Prework Study Guide

I want to add a README section into my prework study guide. I want to include it's own box and have a link setup to it so I can use it for future uses.

Finished


cannot install packages tried everything

Is it duplicate? hell yes
did i find a solution ? No
did someone is the discord channel managed to help? no
BUG REPORT
1.8.7
3.5.2
8.10.0
cannot bower install see attached
https://drive.google.com/file/d/1dzrjIxfag3p-zFWVrasmO1OcDaHbYSj2/view?usp=sharing

Additional environment details (proxy, private registry, etc.):
Steps to reproduce the issue:

Describe the results you received:
stuck
Describe the results you expected:
finish
Additional information:
never happened to me before the last 5 years

This issue is now stale, closing


trash-cli miss bash/zsh completion script

Describe the bug
according to https://github.com/andreafrancia/trash-cli#install-shell-completions, trash-cli has supported to generate shell completion script by itself. And the zsh completion script in zsh-completions has been removed because upstream had supported.
Steps To Reproduce
Steps to reproduce the behavior:
Expected behavior
Screenshots
Additional context
Add any other context about the problem here.
Notify maintainers
@symphorien

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

Oh, it is repeated by #202165.


Add possibilities to mock url with all ids

I call 2 methods when saving data:
1st  - POST /job - this method return job id(jobId).
2nd - POST /job/{jobId}/generate  - this method create task for generate job.
I don't know which jobId I going to get. I want to mock 2nd method for all jobIds. For example /job/*/generate
Can I do it now? If not, could you please implement it?

Hi @ledeneveugene,
try using the /job/:jobId/generate, this should work for all jobId if it doesn't, please let me know!


Extend nodes stats metrics.

Added:
indices.indexing: is_throttled, throttle_time .
indices.search: scroll_total, scroll_time.
indices.merges: total_throttled_time.
breakers: overhead.
Fixed:
indices.query_cache_miss_count, indices.request_cache_miss_count

Hi @leveldorado,
looks good, thanks for your contribution


Bluetooth 4.0 (non-BLE)

Is it possible to use this as a "server", where my phone will connect and pair, and than send data like JSON objects etc? The phone can use cordova plugin, or nativescript native APIs.. If its not, do you guys know any module that can help me build normal non-BLE bluetooth server/client?

I think you're after bleno. That allows device to act like a peripheral, so it can be connected to.

@gfwilliams Isnt bleno for BLE only ? I need module for the normal high bandwidth bluetooth 4.0

Yes, it is. I missed that bit.

@denkomanceski I'm not aware of any non-BLE libraries.


Tuple syntax with 1 value needs better errors

let foo = (1); prints as let foo = 1; Is there a way we could provide better warnings for this? Or not worth the trouble?

don't think that's worth the trouble? tuples literally can't have 1 value. Also, putting parentheses around an expression is totally valid...
what I really want is for atom-reason to show me all(most?) inferred types all the time :D so I don't have to guess / mouseover

Yeah I guess most languages have them, so this is fine. It's hard to give good error messages anyway since (1) can legit mean 1.


delete the redundant func "waitExit"

Signed-off-by: Yanqiang Miao<EMAIL_ADDRESS>

Codecov Report

Merging #918 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #918   +/-   ##
=======================================
  Coverage   59.55%   59.55%
=======================================
  Files           5        5
  Lines         759      759
=======================================
  Hits          452      452
  Misses        199      199
  Partials      108      108

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 199544e...e4e80fb. Read the comment docs.

LGTM


Mvn clean install on spock-example provokes a TestNG error message

Hi all,
I cloned the spock-example project and got this message when running mvn clean install:

[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[TestNG] [ERROR] No test suite found. Nothing to run
Usage:  [options] The XML suite files to run
Options:
-configfailurepolicy
Configuration failure policy (skip or continue)
-d
Output directory
-dataproviderthreadcount
Number of threads to use when running data providers
...

After this message, the Spock tests run as expected.
Kind regards,
Rodrigo

I did some investigation on the issue, it seems the cause is the underlying provider maven-surefire is using, in this case TestNG.
As per surefire documentation:

Surefire normally automatically selects which test-framework provider to use based on the version of TestNG/JUnit present in your project's classpath.

I managed to solve the problem for my local install by specifying junit as the provider:
      <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.0.0-M3</version>
        <configuration>
          <useFile>false</useFile>
          <includes>
            <include>**/*Test.java</include>
            <include>**/*Spec.java</include>
          </includes>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>3.0.0-M3</version>
          </dependency>
        </dependencies>
      </plugin>
This was the result:

Sharing this in case it is helpful.
Kind regards

This should be fixed in the current master, furthermore don't use groovy-all as surefire gets confused by groovy-testng.


consolidate Draw Ire failure and critical failure effects

Success has a different duration, so I left that alone.
Part of #4144

We try not to delete effects linked from spells without a backward compatibility solution.

I restored the deleted effects. Please review them when you get a moment.


Added docker-cakephp repo to Development Environment section

Added docker-cakephp repo based on a email request.

👍

@stefanvangastel Can you please fix the order so we can merge?

@dereuromark Yeah, sorry. Will do asap.

Changed alph order


Add unit tests for AzureFileHandler and refactor HTTP file downloads

We have seen a number of errors when downloading changesets and briefcases, and the common thread seems to be a lack  transient fault handling in AzureFileHandler.
On further inspection, it looks like AzureFileHandler's HTTP/HTTPS file downloading has evolved into a bit of a buggy mess, so I decided it was finally time to write some unit tests for this and refactor to use got, a thirdparty HTTP client that supports basic retry functionality out of the box.

@wgoehrig you'll need to run extract-api I assume.


Make postinstall-postinstall work with npm

I found two related issues to this #2, #3. Currently in a project with postinstall-postinstall, when there's no yarn available on the machine, the installation will break.
Shall we make postinstall-postinstall gracefully switch to npm when yarn is unavailable?

@ds300 can we cut a release for this patch?


Account for DID WG removing VMs from context

See https://github.com/trustbloc/sidetree-core-go/issues/570

We did this, context has an injected vocab to prevent other JSON-LD issues.


contributor/author: add Benjamen Keroack

Add Benjamen Keroack as contributor per request in go-interpreter/wagon#43

needed by go-interpreter/wagon#45.


Bing infinite scrolling on mobile

When I scroll down to the bottom of bing results on mobile, it auto-loads the next page. Results there are not block, nor there is a link to block them.

I'm trying to reproduce this issue in order to fix it, but for some reason I'm not able to on Desktop.
If I use an actual phone, then I get the "Load more search results" button, but with Browser Developer Tools (mobile emulation), I get a "Next" button instead.

@iorate are you able to reproduce this issue?
I've tested on Firefox, Brave and Chromium.

@aug-dev Sorry for my delayed response.
I also couldn’t reproduce this issue. It might be region-specific, similar to #384.

Sorry for my delayed response.

No problem, I've also been short on time lately.
I'll see what I can do.


Incorrect usage of x_axis, y_axis and z_axis in 3d plot layout

Hello plotly-rs maintainers,
I tried to create a 3d scatter plot using the library in Rust version 0.8.3. I faced difficulty in customizing the axes labels for the plot and was consulting the example function customized_scatter3d_plot() available in the official repository.
After going through the code of the layout setup in the example function, I found that x_axis, y_axis, and z_axis were used to set the axis labels for a 3d plot. However, I figured out that these functions do not set the axis labels for a 3d plot. Instead, the Scene element has to be implemented to set the axis labels and other 3d properties.
The output of the function results in the following plot without any axis titles:

I noticed that there was a pull request submitted a while ago which aimed to implement the Scene element for this functionality, but it was rejected for wrong formalities. I am interested in contributing to the project and wanted to know if a new pull request for implementing the Scene element would be welcomed.
Thank you!

I would certainly be happy to review a PR as you describe.

Ok, great, then I'll see if I can finish and test what I've coded so far.


[BUG] Multi-stage Recsys  deployment notebook  is broken

Bug description
I get the following error from the 02-Deploying-multi-stage-RecSys-with-Merlin-Systems.ipynb notebook, when I execute the ensemble = Ensemble(ordering, request_schema)  command.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [20], in <cell line: 4>()
      1 # define the path where all the models and config files exported to
      2 export_path = os.path.join(BASE_DIR + 'poc_ensemble')
----> 4 ensemble = Ensemble(ordering, request_schema)
      5 ens_config, node_configs = ensemble.export(export_path)

File /usr/local/lib/python3.8/dist-packages/merlin/systems/dag/ensemble.py:51, in Ensemble.__init__(self, ops, schema, name, label_columns)
     37 """_summary_
     38
     39 Parameters
   (...)
     48     List of strings representing label columns, by default None
     49 """
     50 self.graph = Graph(ops)
---> 51 self.graph.construct_schema(schema)
     52 self.name = name
     53 self.label_columns = label_columns or []

File /usr/local/lib/python3.8/dist-packages/merlin/dag/graph.py:73, in Graph.construct_schema(self, root_schema, preserve_dtypes)
     70 def construct_schema(self, root_schema: Schema, preserve_dtypes=False) -> "Graph":
     71     nodes = list(postorder_iter_nodes(self.output_node))
---> 73     self._compute_node_schemas(root_schema, nodes, preserve_dtypes)
     74     self._validate_node_schemas(root_schema, nodes, preserve_dtypes)
     76     return self

File /usr/local/lib/python3.8/dist-packages/merlin/dag/graph.py:80, in Graph._compute_node_schemas(self, root_schema, nodes, preserve_dtypes)
     78 def _compute_node_schemas(self, root_schema, nodes, preserve_dtypes=False):
     79     for node in nodes:
---> 80         node.compute_schemas(root_schema, preserve_dtypes=preserve_dtypes)

File /usr/local/lib/python3.8/dist-packages/merlin/dag/node.py:188, in Node.compute_schemas(self, root_schema, preserve_dtypes)
    183 self.selector = self.op.compute_selector(
    184     self.input_schema, self.selector, parents_selector, dependencies_selector
    185 )
    187 prev_output_schema = self.output_schema if preserve_dtypes else None
--> 188 self.output_schema = self.op.compute_output_schema(
    189     self.input_schema, self.selector, prev_output_schema
    190 )

File /usr/local/lib/python3.8/dist-packages/merlin/systems/dag/ops/softmax_sampling.py:106, in SoftmaxSampling.compute_output_schema(self, input_schema, col_selector, prev_output_schema)
     88 def compute_output_schema(
     89     self, input_schema: Schema, col_selector: ColumnSelector, prev_output_schema: Schema = None
     90 ) -> Schema:
     91     """
     92     Describes the operator's outputs, which are computed from the input schema.
     93
   (...)
    103         Not used.
    104     """
    105     col_schemas = sorted(
--> 106         [
    107             schema
    108             for name, schema in input_schema.column_schemas.items()
    109             if name in col_selector.names
    110         ],
    111         key=lambda cs: cs.name,
    112     )
    113     return Schema(col_schemas)

File /usr/local/lib/python3.8/dist-packages/merlin/systems/dag/ops/softmax_sampling.py:109, in <listcomp>(.0)
     88 def compute_output_schema(
     89     self, input_schema: Schema, col_selector: ColumnSelector, prev_output_schema: Schema = None
     90 ) -> Schema:
     91     """
     92     Describes the operator's outputs, which are computed from the input schema.
     93
   (...)
    103         Not used.
    104     """
    105     col_schemas = sorted(
    106         [
    107             schema
    108             for name, schema in input_schema.column_schemas.items()
--> 109             if name in col_selector.names
    110         ],
    111         key=lambda cs: cs.name,
    112     )
    113     return Schema(col_schemas)

AttributeError: 'NoneType' object has no attribute 'names'

Environment details

Merlin version:
Platform:
Python version:
PyTorch version (GPU?):
Tensorflow version (GPU?):

merlin-tensorflow:22.07 the latest main branches are pulled.

There seems to be a general issue with SoftmaxSampling since these two PRs were merged:

https://github.com/NVIDIA-Merlin/systems/pull/155
https://github.com/NVIDIA-Merlin/systems/pull/154

We should fix or revert those changes before 22.08 is released.

note that this test is skipped during CI builds in order to get docker images to publish, but the bug still exists.

This should be resolved with https://github.com/NVIDIA-Merlin/systems/pull/172 and https://github.com/NVIDIA-Merlin/systems/pull/171. Waiting for verification.


[Flake] Failure in google/cloud/internal/background_threads_impl_test.

Looks like another timing problem, last seen in a Windows+Bazel build.
INFO: From Testing //google/cloud:internal_background_threads_impl_test:
==================== Test output for //google/cloud:internal_background_threads_impl_test:
Running main() from gmock_main.cc
[==========] Running 3 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 2 tests from CustomerSuppliedBackgroundThreads
[ RUN      ] CustomerSuppliedBackgroundThreads.LifecycleNoShutdown
[       OK ] CustomerSuppliedBackgroundThreads.LifecycleNoShutdown (15 ms)
[ RUN      ] CustomerSuppliedBackgroundThreads.SharesCompletionQueue
[       OK ] CustomerSuppliedBackgroundThreads.SharesCompletionQueue (3 ms)
[----------] 2 tests from CustomerSuppliedBackgroundThreads (18 ms total)

[----------] 1 test from AutomaticallyCreatedBackgroundThreads
[ RUN      ] AutomaticallyCreatedBackgroundThreads.IsActive
google/cloud/internal/background_threads_impl_test.cc(79): error: Expected equality of these values:
  std::future_status::ready
    Which is: 4-byte object <00-00 00-00>
  expired.wait_for(ms(100))
    Which is: 4-byte object <01-00 00-00>
[  FAILED  ] AutomaticallyCreatedBackgroundThreads.IsActive (127 ms)
[----------] 1 test from AutomaticallyCreatedBackgroundThreads (127 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 2 test suites ran. (146 ms total)
[  PASSED  ] 2 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] AutomaticallyCreatedBackgroundThreads.IsActive

 1 FAILED TEST
================================================================================

https://source.cloud.google.com/results/invocations/60a6b8d9-0028-46b9-b439-9ed0bb6e5786

A very similar one:
https://source.cloud.google.com/results/invocations/3f741ca2-c9b8-4a7a-a083-adc8b8340076

I changed the title because it seems we have a lot of tests that do something in one thread and then expect something to happen in another thread within 100ms. That seems too short. If my memory serves me, we had to fix the future<T> tests to use 500ms timeouts (see #2047).
Ideally we should avoid these timeouts altogether, and write the tests without them. If we cannot, the 500ms timeouts seem to be Okay in practice (though there is no guarantee, of course).


How to get barcode value to enter in mobile client

When I was trying to run mobile client, it asking me to enter bar code value, Please let me know ,what value I have to give

@Rasmi91 When you launch the app on your mobile device it will give you a QR code to register.  Reading the associated blog should clarify.  https://blogs.bing.com/maps/2018-01/Announcing-the-Bing-Maps-Fleet-Tracker-Solution

Register devices for tracking by shooting a QR code or manually entering device information in the web application. The mobile companion app features client control of when location tracking is on or off, giving the mobile worker complete control of their location information


Add editor setting for display mode on file dialog and filesystem dock

Add a editor settings for choosing the display mode of EditorFileDialog, similar to the hidden folder setting.
Closes #2179.

nice, make it work for the filesystem dock too if you can :D
On Tue, Dec 15, 2015 at 1:43 PM, Franklin Sobrinho<EMAIL_ADDRESS>
wrote:

Add a editor settings for choosing the display mode of EditorFileDialog,
similar to the hidden folder setting.
Closes #2179 https://github.com/godotengine/godot/issues/2179.
You can view, comment on, or merge this pull request online at:
https://github.com/godotengine/godot/pull/3083
Commit Summary

Add editor setting for file dialog display mode

File Changes

M tools/editor/editor_file_dialog.cpp
https://github.com/godotengine/godot/pull/3083/files#diff-0 (19)
M tools/editor/editor_file_dialog.h
https://github.com/godotengine/godot/pull/3083/files#diff-1 (2)
M tools/editor/editor_node.cpp
https://github.com/godotengine/godot/pull/3083/files#diff-2 (1)
M tools/editor/editor_settings.cpp
https://github.com/godotengine/godot/pull/3083/files#diff-3 (2)

Patch Links:

https://github.com/godotengine/godot/pull/3083.patch
https://github.com/godotengine/godot/pull/3083.diff

—
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/pull/3083.

Ping @neikeq as he was also working on this very bug IINM :)

I didn't work on it because I still had doubts about what we talked on irc (project specific settings).

@reduz, done, now it will work on FileSystem dock too.

It doesn't seem to work for me in the FileSystem dock. It always load with the "expanded tree" view, even though I quit the editor with the "list" view (not tree, just list of what is in the current directory).

Nevermind, I misunderstood the feature, I thought it would be saved automatically, but it has to be set in the editor settings :) Which is totally fine IMO.


Intermittent 401 (invalid oauth token) from Chrome API endpoint

We're seeing intermittent 401 responses from the Chrome API endpoint, when the chrome devices send a heartbeat.
Traceback (most recent call last):
  File "/base/data/home/apps/f~ncau-tech-grab-n-go-prod/chrome:mcshaneg-20181104.413743242539803060/external/endpoints_archive/endpoints/users_id_token.py", line 369, in _set_bearer_user_vars
    authorized_scopes = oauth.get_authorized_scopes(sorted(all_scopes))
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/1/google/appengine/api/oauth/oauth_api.py", line 171, in get_authorized_scopes
    _maybe_call_get_oauth_user(scope)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/1/google/appengine/api/oauth/oauth_api.py", line 220, in _maybe_call_get_oauth_user
    _maybe_raise_exception()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/3b44e98ed7fbb86b/python27/python27_lib/versions/1/google/appengine/api/oauth/oauth_api.py", line 239, in _maybe_raise_exception
    raise InvalidOAuthTokenError(error_detail)
InvalidOAuthTokenError

Doesn't seem to be any specific trigger, and sometimes an initial heartbeat will be succesfull causing the device to be assigned to a user, but around 20 seconds later it'll 401 so the rest of the onboarding flow fails to appear.
I've tried using a different network, un-enrolling and re-enrolling chromebooks , using different chromebooks, using different users (both superadmin and non-superadmin), wiping the chromebooks, redeploying the web app, redeploying the chrome app, creating a new OAuth Client ID and redploying the system - nothing seems to stabilise it.
Any help with the right direction to start troubleshooting this would be ace.

we recently pushed an update to the master branch, can you please pull those changes down and deploy another version of the application? Please let me know how that goes.


tests: Fix failure on 5.11 kernel

Signed-off-by: Stéphane Graber<EMAIL_ADDRESS>

Testsuite passed


Prevent user from naming pipeline job with internal name like git

If I use git as a pipeline job, the pipeline execution hangs after executing the internal git command.

@anbraten can you add a demo pipeline config?

@anbraten tested:
pipeline:
  build:
    image: debian
    pull: true
    commands:
      - echo "Build test"
    when:
      event: [push, tag]

  git:
    image: debian
    pull: true
    commands:
      - echo "zap"

# The following statement causes the issue:
branches: main

and it worked just fine!

haven't noticed it again


My Relays aren't saving

My relay settings aren't saving.. for long. I'm using Primal website with the Alby extension and Android Beta. The website confirms and connects to my chosen relays. I noticed on the Primal website that my relays are restored to the defaults after I open the Android app using the same npub. The bug doesn't reoccur if I keep the Android app closed.

This has been fixed.


Specific cursed, recursive, inscription not showing in the wallet

It is this inscription: 81433a25e784366a7db917af98393be6eba232a03646e1fc072904fc3b8803b2i2 I see they have different negative inscription numbers between the Hiro explorer and the ord.io explorer. But since it is available in the Hiro ordinals explorer I would expect the wallet to at least display an empty placeholder for the inscription. Support for cursed inscriptions was added here.
https://ordinals.hiro.so/inscription/81433a25e784366a7db917af98393be6eba232a03646e1fc072904fc3b8803b2i2

https://www.ord.io/81433a25e784366a7db917af98393be6eba232a03646e1fc072904fc3b8803b2i2

Environment: Leather Chrome extension v6.13.0.
I have seen cases where some (recursive) inscriptions will only display as a blank or black placeholder inscription but in this case, nothing is shown. Although the user says another cursed inscription in a similar range is displayed normally.

False alarm it turns out there is a placeholder being displayed by Leather but it was missed because the short inscription number is different on ord.io.

I think this should be handled with #4077 in that case.


Rainbow + warn method exception in jRuby

Rubocop throws error when executed on jRuby environment.
$ rubocop
wrong argument type Rainbow::NullPresenter (expected String)
org/jruby/common/RubyWarnings.java:224:in `warn'
org/jruby/RubyKernel.java:1247:in `warn'
org/jruby/RubyKernel.java:1234:in `warn'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/config_loader.rb:154:in `warn_on_pending_cops'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/config_loader.rb:111:in `block in configuration_from_file'
org/jruby/RubyKernel.java:1871:in `tap'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/config_loader.rb:110:in `configuration_from_file'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/config_store.rb:58:in `for_dir'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/config_store.rb:37:in `for_pwd'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/cli.rb:124:in `apply_default_formatter'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/lib/rubocop/cli.rb:40:in `run'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/exe/rubocop:13:in `block in <main>'
C:/jruby-<IP_ADDRESS>/lib/ruby/stdlib/benchmark.rb:308:in `realtime'
C:/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/rubocop-0.90.0/exe/rubocop:12:in `<main>'
org/jruby/RubyKernel.java:1008:in `load'
C:\jruby-<IP_ADDRESS>\bin\rubocop:23:in `<main>'

Expected behavior
Rubocop should inspect files
Actual behavior
Rubocop fails with exception trace
Steps to reproduce the problem
Execute rubocop from terminal on jRuby environment.
RuboCop version
$ rubocop -V
0.90.0 (using Parser <IP_ADDRESS>, rubocop-ast 0.3.0, running on jruby 2.5.3 java)

jRuby version
$ jruby -v
jruby <IP_ADDRESS> (2.5.3) 2019-02-11 15ba00b Java HotSpot(TM) 64-Bit Server VM 25.251-b08 on 1.8.0_251-b08 +jit [mswin32-x86_64]

Debugging information
Following is the code that breaks
warn Rainbow(PENDING_BANNER).yellow

My analysis is, warn method is not able to handle Rainbow::NullPresenter instance which is returned by Rainbow(PENDING_BANNER).yellow
I read one comment here where @koic applied to_s to handle this error.
Similar code is present in many places in Rubocop. Should I go ahead and fix those places using to_s and give a pull-request? Or please suggest some elegant way to handle it?

Has an issue been opened with JRuby?

Has an issue been opened with JRuby?

Nope.
I will do it too.

I'm going to close this issue because JRuby 9.2 is no longer runtime supported.


add a short timeout to API calls if chat is offline CORE-4890

The point of this is if we are on a high packet loss network, and chat has taken itself online, that we raise the bar on API server response times. Requests to the API servers in these conditions will likely fail, but might take awhile since the connection might be nominally alive.

cc @patrickxb


Removing old dependency packages

I'm very glad I found this script. It's beautiful and neat, and very helpful for people that are far too used to apt's syntax.
I believe there may be a small issue with the "remove" command, though. It appears to run "pacman -R", which does not remove automatically installed dependencies. Would it not be better to replace this with "pacman -Rs", which does remove dependencies?
Alternatively, or additionally, an "autoremove" command could be added. This would be consistent with apt and apt-get, which would align with the goal of this script. I believe in this case it could run "pacman -Qdtq | pacman -Rs -" or "pacman -Rs $(pacman -Qqtd)" to do the job.

Great suggestions, I will absolutely integrate these changes.


[Java] Implement/use ServiceProvider for discovering drivers

https://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html

I noticed that the current FlightSqlDriver and JdbcDriver classes register just the name of the package holding the Driver class as the "driver name" presently. Should it be the fully-qualified class name of the driver (like in JDBC) @lidavidm ? This would make sense but break existing code.

Yeah, the full package would make more sense. I'm not too concerned about breakage here so long as the core interfaces remain stable.

The ServiceLoader requires that any AdbcDriver implementations provide a public parameter-less constructor. This is a bit of a problem since we currently require the BufferAllocator parameter to be supplied to the AdbcDriver.
Perhaps we make the BufferAllocator supplied to the AdbcDatabase or AdbcConnection instead?