Bugs affecting setting up LDAP replication

I'm trying to set up two containers with replication between them. The process is failing in a couple of places.
The first is in this script:
https://github.com/osixia/docker-openldap/blob/2a03b392b019adbf810f11bd64a2fed753d77b9d/image/service/slapd/startup.sh#L259-L261
This doesn't declare ldaps:// as a protocol which causes problems if LDAP_REPLICATION_HOSTS does reference ldaps://. That is, though, fairly simple to fix in that script.
A bigger problem - and one I cannot find the answer to - is host names and domain names.
If I set hostname to be the fully-qualified name and leave domainname unset then /etc/hosts get correctly set but the environment variable HOSTNAME in the container gets changed to just the leafname. This then causes slapd to just listen to the leafname and not the fully-qualified domain name. That results in read_config: no serverID / URL match found. Check slapd -h arguments.
If I set hostname to just be the hostname and set domainname to the domain then no change is noticed - /etc/hosts is set correctly and HOSTNAME is still set to the leafname.
If I follow the comments in docker-compose.yml and set hostname and domainname to the same value then /etc/hosts is set incorrectly - the entry is a concatenation of both hostname and domainname but HOSTNAME is left alone as the fully qualified name! That results in daemon: bind(6) failed errno=99 (Cannot assign requested address).
I cannot find a way to make sure that /etc/hosts is set correctly and HOSTNAME is left as the fully-qualified domain name.
A potential workaround would be to have HOSTNAME truncated to the leafname but then modify the references to slapd so that they concatenate the domainname value but that doesn't seem to make it into the container's environment variables.

"ldaps://" is deprecated in favour of STARTTLS so I'm now experimenting with removing all references to HOSTNAME where slapd is run. Instead, just using ldap:/// ldapi:///.

@pcolmer Hi, I'm running into the same issues trying to setup replication.
Did you have any luck getting it to work so far?

@guptamp Yes, I have. I keep meaning to raise a PR and forgetting :). I'll get it done this morning.

@guptamp https://github.com/osixia/docker-openldap/pull/420
Fairly small changes. If you want to test them yourself, you can put the two modified scripts into a folder called "patches" and then build a modified container with this Dockerfile:
FROM osixia/openldap:1.3.0
COPY --chown=openldap:openldap patches/startup.sh /container/service/slapd/
COPY --chown=openldap:openldap patches/process.sh /container/service/slapd/
RUN chmod a+x /container/service/slapd/*.sh

@guptamp Please note the changes to docker-compose.yml though - it is very important to configure hostname and domainname correctly. They must not be the same value as per the current recommendations.

@pcolmer I tried your fix, but somehow it's still complaining about the -h parameter for slapd...
`Start OpenLDAP...
Waiting for OpenLDAP to start...
5eac17f7 @(#) $OpenLDAP: slapd  (Jul 30 2019 16:24:19) $
Debian OpenLDAP Maintainers<EMAIL_ADDRESS>5eac17f7 read_config: no serverID / URL match found. Check slapd -h arguments.
5eac17f7 slapd stopped.
5eac17f7 connections_destroy: nothing to destroy.
`
How can i see the exact command line which is used to start it?

@guptamp

How can i see the exact command line which is used to start it?

What I did was add this echo statement to process.sh:
echo "/usr/sbin/slapd -h $HOST_PARAM ldapi:/// -u openldap -g openldap -d $LDAP_LOG_LEVEL"
just before the exec line. Also, if you add:
command: ["--loglevel", "debug"]
to your docker-compose.yml file, you'll get more output from the container.
Hope that helps.

@pcolmer It was helpful, thank you.
I think the issue was that i didn't explicitly set a hostname for the container in docker. so when you query hostname --fqdn in process.sh it returns the random docker id that was generated for container hostname.

Personally I would find it more logical if you would use $HOSTNAME + $LDAP_DOMAIN here.

@guptamp

Personally I would find it more logical if you would use $HOSTNAME + $LDAP_DOMAIN here.

There is, unfortunately, a drawback to that approach. The LDAP domain does not need to be the same as the domain that the host is sitting in, so that approach would be more restrictive. I accept that there will be plenty of occasions when it is the same, but just as many where it isn't.
Glad you got it working though.

@pcolmer You are of course right, I didn't consider that scenario.
However I just can get the second server to start, seems various reasons, one being that in the startup.sh the docker-openldap-was-started-with-replication gets parsed and there only the hostname is written. So essentially it fails already before going into process.sh.
I think I spend already too much time and either need to find another container or fall back to VM's...
But many thanks for your help!
I hope that soon your changes get merged and maybe I can have another try.

@pcolmer You are of course right, I didn't consider that scenario.
However I just can get the second server to start, seems various reasons, one being that in the startup.sh the docker-openldap-was-started-with-replication gets parsed and there only the hostname is written. So essentially it fails already before going into process.sh.
I think I spend already too much time and either need to find another container or fall back to VM's...
But many thanks for your help!
I hope that soon your changes get merged and maybe I can have another try.

@guptamp I am not sure that going back to VM or find another container (I do not know any other nor found one which does this and is still active) will solve your case.
The great advantage of containers is that you can easily and effortlessly clone the repo and integrate the fix to produce your own version (which I do quite often) and this also avoids to wait for official release. Just suggesting ....

@obourdon @pcolmer You are of course right and after sleeping over it and giving the basic sample a try again i realized something.
I believe the issue might be coming from me not manually updating the /etc/hosts file with the fqdn of BOTH hosts.
I was expecting the containers to resolve the dns through my dns server or am i wrong in thinking that?
The second possible issue is that one of the ldap servers is on a none standard port (1389). I put in the correct URI in the Replication_Hosts, but maybe this needs to be modified somewhere else too?
Would it be acceptable to build the images with a modified hosts file if needed? Kind of feels wrong but if it helps...

@guptamp are you not using FQDNs that are externally resolvable? In other words, if I tried to resolve the hostnames you are using, would I be able to?
If you are using public FQDNs, you shouldn't need to update the hosts file. The containers will perform DNS queries and resolve them normally. The only reason for /etc/hosts to be updated with the name of the server itself is so that OpenLDAP knows what it is binding as.
Why are you using a non-standard port? Is it because you are running both containers on the same host?

Would it be acceptable to build the images with a modified hosts file if needed? Kind of feels wrong but if it helps...

What you could do, if you really feel that is the correct solution for you, is modify startup.sh around line 510 so that you can update the hosts file. However, my concern would be how you then figure out the correct IP addresses to put in there.
If you are still stuck, perhaps you could share a bit more info about your network setup.

I got it working!
In conclusion i think i had 2 issues.

There seems to be a problem if you specify the port in the URI. Once I managed to free up the port on my NAS so i could use it for LDAP things almost worked.
There was still a problem with StartTLS and an error -11 which i tracked down to be related to certificate signing. The following LDIF needs to be added to make it work:

``
dn: cn=config
changetype: modify
replace: olcTLSCipherSuite
olcTLSCipherSuite: SECURE256:SECURE128:-VERS-SSL3.0:-VERS-TLS-ALL:+VERS-TLS1.2:+VERS-TLS1.1
``
Thanks @obourdon & @pcolmer for sticking with me!

@guptamp no pb at all and congrats for keeping up trying and succeeding using the container the way you wanted

Just small update for others...
I used the docker-compose from the example folder.
In there the TLSCipherSuite is explicitly set to "SECURE256:-VERS-SSL3.0" and this is what's causing the failure for TLS. I didn't pay enough attention to the other Env vars in there.

This thread was really helpful for me, as i was setting replication for the first time for my docker environment. @pcolmer I have modifed my process.sh and startup.sh as mentioned in this thread and I am getting the hostnames correctly ie in /etc/hosts as well environment variable hostname.   Im facing a small problem with replication , Node 1 doesnt add the hostname+ipaddress details of node 2 in /etc/hosts file due to which replication fails. If i manaually add the hostname and ipaddress details of node 2 in node 1 (vice versa needs to be done on node2 as well ie add the hostname and ipaddress of node 1)then replication works perfectly fine . How can i fix this?
Thanks in advance

@fabiandsouza88 in my environment, I'm using fully-qualified domain names that are resolved via DNS. That is why I didn't need to add node 2's details into node 1's /etc/hosts file.
I don't know if that helps you figure out a solution?

@guptamp #420
Fairly small changes. If you want to test them yourself, you can put the two modified scripts into a folder called "patches" and then build a modified container with this Dockerfile:
FROM osixia/openldap:1.3.0
COPY --chown=openldap:openldap patches/startup.sh /container/service/slapd/
COPY --chown=openldap:openldap patches/process.sh /container/service/slapd/
RUN chmod a+x /container/service/slapd/*.sh

Thanks for this. I had to use it to get the darn thing to start with replication. I spent super long trying to figure this out.

@pcolmer , thanks for digging into the codes.
Issue
I'm facing similar issue which leads to "no serverID / URL match found" error after upgrading image to v1.4.0. The root cause cames with the hostname and FQDN just as your description.
However, my ldap servers are deployed as containers over kubernetes network, via statefulset manifest file, with multi-master replication enabled. In kubernetes context, each pod that wraps the container are assigned a DNS resolvable FQDN, which is formatted as 'pod-name.service-name.namespace.svc.cluster.local'. As kubernetes restricts pod's hostname to leafname only, I haven't found any way to personalize the FQDN.
This leads to that 'hostname --fqdn' instruction returns 'ldap-x.my-service.my-namespace.svc.cluster.local'. So I added these domain names to LDAP_REPLICATION_HOSTS. But with no luck, the slapd service had been not initializing. Container persist printing following logs.

*** Running runit daemon...
*** runit daemon started as PID 212
hostname: Name or service not known
Sep  7 10:48:28 ldap-0 syslog-ng[217]: syslog-ng starting up; version='3.19.1'
hostname: Name or service not known
hostname: Name or service not known
hostname: Name or service not known
...
...
hostname: Name or service not known
hostname: Name or service not known
hostname: Name or service not known

But when I exec the into one of the containers, all fqdns are resolvable.
[/ldap]$ kubectl exec -it -n ldap ldap-0 -c openldap bash
root@ldap-0:/# python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>import socket
>>>socket.gethostbyname('ldap-0.my-service.my-namespace.svc.cluster.local')
'<IP_ADDRESS>'
>>>socket.gethostbyname('ldap-1.my-service.my-namespace.svc.cluster.local')
'<IP_ADDRESS>'
>>>socket.gethostbyname('ldap-2.my-service.my-namespace.svc.cluster.local')
'<IP_ADDRESS>'

I have no idea, what's going wrong.
Alternative solution
Since putting fqdn in replication list not working. I'm wondering wether it is possible to let user define the hostname via HOSTNAME variable.
https://github.com/osixia/docker-openldap/blob/7ce084bab1f9016830548558241fdefb60862482/image/service/slapd/process.sh#L12-L16
at line #16 , change

exec /usr/sbin/slapd -h "$HOST_PARAM ldapi:///" -u openldap -g openldap -d "$LDAP_LOG_LEVEL"bash
to
exec /usr/sbin/slapd -h "$HOSTNAME ldapi:///" -u openldap -g openldap -d "$LDAP_LOG_LEVEL"bash

Since I don't have a docker environment. So please correct me if I'm wrong.
As result of the PR #420, the HOSTNAME variable, /etc/host, leafname and fqdn have been set correctly. So docker users could use the combination of --hostname and $HOSTNAME to control the slapd listening port, while kubernetes users could set $HOSTNAME to kubernetes format.

@yhamcit
Your suggested change to line 16 won't quite work because HOST_PARAM (which you are replacing with HOSTNAME) is more than just the fully qualified domain name. It is the LDAP protocol and port as well.
You said:

However, my ldap servers are deployed as containers over kubernetes network, via statefulset manifest file, with multi-master replication enabled. In kubernetes context, each pod that wraps the container are assigned a DNS resolvable FQDN, which is formatted as 'pod-name.service-name.namespace.svc.cluster.local'. As kubernetes restricts pod's hostname to leafname only, I haven't found any way to personalize the FQDN.

The hostname should be the leafname only. If you look at the example docker-compose.yml file, you'll see that you can set domainname to the rest of the server's FQDN, e.g.:
domainname: my-service.my-namespace.svc.cluster.local

Does that help?

@pcolmer. Thanks for quick reply. There is a mistake in my previous post. I'm going to correct it soon.
This is the original solution:

exec /usr/sbin/slapd -h "$HOSTNAME ldapi:///" -u openldap -g openldap -d "$LDAP_LOG_LEVEL"bash

This is the correct solution:

HOST_PARAM="ldap://$HOSTNAME:$LDAP_PORT ldaps://$HOSTNAME:$LDAPS_PORT"
exec /usr/sbin/slapd -h "$HOST_PARAM ldapi:///" -u openldap -g openldap -d "$LDAP_LOG_LEVEL"bash

Regarding to set domainname:
Unfortunately, unlike docker-compose.yaml. Kubernetes does not allow for changing the FQDN. Here is the description from Kubernetes' doc.

Pod's hostname and subdomain fields
Currently when a pod is created, its hostname is the Pod's metadata.name value.
The Pod spec has an optional hostname field, which can be used to specify the Pod's hostname. When specified, it takes precedence over the Pod's name to be the hostname of the pod. For example, given a Pod with hostname set to "my-host", the Pod will have its hostname set to "my-host".
The Pod spec also has an optional subdomain field which can be used to specify its subdomain. For example, a Pod with hostname set to "foo", and subdomain set to "bar", in namespace "my-namespace", will have the fully qualified domain name (FQDN) "foo.bar.my-namespace.svc.cluster-domain.example".

The point of my suggeting:
hostname: from hostname field in docker-compose.yml or set by kubernetes, for example 'my-ldap-0'
FQDN:  either from domainname (docker) or fixed value (kubernetes) , for example: 'my-ldap-0.my-service.my-namespace...'
/etc/hosts:   ip-address    FQDN    HOSTNAME-ALIAS(LEAFNAME)
HOSTNAME environment variable: let user make decision and be used in slapd -h argument, so that user could control what is used to launch slapd service. (This might provide some kind of flexibility)
Would it work?

@yhamcit I'm getting a bit confused ...

*** HOSTNAME = ldap-0.my-service.my-namespace
*** POD_NAME = ldap-0

Are both of these being set by Kubernetes or are you setting HOSTNAME? The reason I ask is because that value for HOSTNAME is not just a leafname.
Also, are you trying to use these servers from outside the cluster or within it? In other words, are the clients of your LDAP servers also within the "cluster.local" namespace?

Both POD_NAME & HOSTNAME are set by myself in accord with kubernetes. Meanwhile, kubernetes will set hostname and hostname fqdn automatically. Here is the hostname command output:
root@ldap-0:/# hostname
ldap-0
root@ldap-0:/# hostname --fqdn
ldap-0.coredbs.my-service.my-namespace.cluster.local
root@ldap-0:/# echo $HOSTNAME
ldap-0.my-service.my-namespace
root@ldap-0:/# echo $POD_NAME
ldap-0

"ldap-0.my-service.my-namespace" is the short name that kubernetes could resolve and could be used by pods to communicate each other. I use this shortened string because in original image v1.4.0, when FQDN is used to lauch "slapd -h", the container prints error:

hostname: Name or service not known

I can not finger out what's wrong and have no idea to fix it. So I'm trying to find a way to bypass.
In my cluster, ldap and ldap clients stays in different namespace, but, still in same cluster. So they could both find each other via the domainname 'pod-name.service-name.namespace'. When I use
slapd -h "ldap://ldap-0.my-service.my-namespace:389 ldaps://ldap-0.my-service.my-namespace:636 ldapi:///" ...
to launch the service, and set REPLICATION-HOST-LIST as:

LDAP_REPLICATION_HOSTS:
- ldap://ldap-0.my-service.my-namespace
- ldap://ldap-1.my-service.my-namespace
- ldap://ldap-2.my-service.my-namespace

The ldap pods and replicatin would work.

In my cluster, ldap and ldap clients stay in different namespace, but, still in same cluster. So they could both find each other via the domainname 'pod-name.service-name.namespace'.

Can they find each other with "pod-name.service-name.namespace.cluster.local"? I ask that because, earlier, you said "But when I exec the into one of the containers, all fqdns are resolvable."
If you exec into one of the LDAP containers, what does /etc/hosts look like and what is the slapd process running with?

It works for me.
Here is the log from ldap server booting:

*** --- process file : /container/environment/01-custom/env.yaml ---
*** --- process file : /container/environment/98-default/default.yaml ---
*** --- process file : /container/environment/99-default/default.yaml ---
*** ignore : LDAP_LOG_LEVEL = 256 (keep LDAP_LOG_LEVEL = 256 )
*** Run commands before process...
*** ------------ Environment dump ------------
*** PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
*** HOSTNAME = ldap-2.my-service.my-namespace
*** SERVICE_NAME = coredbs
...
*** LDAP_PORT = 389
*** LDAPS_PORT = 636
*** ------------------------------------------
*** Running runit daemon...
*** runit daemon started as PID 614
exec /usr/sbin/slapd -h ldap://ldap-2.my-service.my-namespace:389 ldapi:/// -u openldap -g openldap -d 256
5f5853a8 @(#) $OpenLDAP: slapd 2.4.50+dfsg-1~bpo10+1 (May  4 2020 05:25:06) $
Debian OpenLDAP Maintainers<EMAIL_ADDRESS>Sep  9 12:01:44 ldap-2 syslog-ng[620]: syslog-ng starting up; version='3.19.1'
5f5853a8 slapd starting
5f5853a8 slap_client_connect: URI=ldap://ldap-1.my-service.my-namespace
5f5853a8 slap_client_connect: URI=ldap://ldap-1.my-service.my-namespace
5f5853cb conn=1000 fd=14 ACCEPT from IP=<IP_ADDRESS>:58040 (IP=<IP_ADDRESS>:389)
5f5853cb conn=1001 fd=16 ACCEPT from IP=<IP_ADDRESS>:58042 (IP=<IP_ADDRESS>:389)
...
5f5853cc conn=1003 fd=17 TLS established tls_ssf=256 ssf=256
5f5853cc conn=1002 fd=14 TLS established tls_ssf=256 ssf=256
5f5853cc conn=1003 op=1 BIND dn="cn=admin,cn=config" method=128
...
5f5853cc conn=1002 op=1 BIND dn="cn=admin,dc=my-org,dc=org" method=128
...
5f5853cc conn=1003 op=2 SRCH base="cn=config" scope=2 deref=0 filter="(objectClass=)"
5f5853cc conn=1003 op=2 SRCH attr= +
5f5853cc conn=1002 op=2 SRCH base="dc=my-org,dc=org" scope=2 deref=0 filter="(objectClass=)"
5f5853cc conn=1002 op=2 SRCH attr= +
5f5853cd conn=1001 op=3 UNBIND
5f5853cd conn=1001 fd=16 closed

To verify, exec into container:
root@ldap-0:/# hostname
> ldap-0
root@ldap-0:/# hostname --fqdn
> hostname: Name or service not known
root@ldap-0:/# echo $HOSTNAME
> ldap-0.my-service.my-namespace
root@ldap-0:/# cat /etc/hosts
> <IP_ADDRESS> ldap-0.my-service.my-namespace.svc.cluster.local ldap-0.my-service.my-namespace
> # Kubernetes-managed hosts file.
> <IP_ADDRESS>       localhost
> ::1     localhost ip6-localhost ip6-loopback
> fe00::0 ip6-localnet
> fe00::0 ip6-mcastprefix
> fe00::1 ip6-allnodes
> fe00::2 ip6-allrouters
root@ldap-0:/# python3
> Python 3.7.3 (default, Dec 20 2019, 18:57:59)
> [GCC 8.3.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import socket
> >>> socket.gethostbyname('ldap-0.my-service.my-namespace')
> '<IP_ADDRESS>'
> >>> socket.gethostbyname('ldap-1.my-service.my-namespace')
> '<IP_ADDRESS>'
> >>> socket.gethostbyname('ldap-2.my-service.my-namespace')
> '<IP_ADDRESS>'
> >>> quit()

It's abnormal that hostname --fqdn command not working. It means FQDN is not set. However, this solution still works...
Since 'ldap-2.my-service.my-namespace' and 'ldap-2.my-service.my-namespace' are dns resolvable from 'ldap-0.my-service.my-namespace', I can do ldapsearch like this from 'ldap-0':
root@ldap-0:/# ldapsearch -x -H ldap://ldap-2.my-service.my-namespace -b "cn=config" -D "cn=admin,cn=config" -W -Z
> Enter LDAP Password:
> # extended LDIF
> #
> # LDAPv3
> # base <cn=config> with scope subtree
> # filter: (objectclass=*)
> # requesting: ALL
> #
>
> # config
> dn: cn=config
> objectClass: olcGlobal
> cn: config
> ...

@pcolmer
Thanks a lot for the info contained in this thread. It really helped me out particularly when setting the domain and hostname.
I'm running osixia 1.40.
The actual replication between two docker images located on two different VMs seems to be working.  I can make changes on the one instance and see the changes on the other.  I'm getting this in the logs of one of the instances:
TLS: can't connect: (unknown error code).
5ff0efb2 slap_client_connect: URI=ldap://openldap.openldap-net Error, ldap_start_tls failed (-11)
5ff0efb2 do_syncrepl: rid=101 rc -11 retrying

I following parameters were set automatically:
olcTLSCipherSuite: SECURE256:+SECURE128:-VERS-TLS-ALL:+VERS-TLS1.2:-RSA:-DHE-DSS:-CAMELLIA-128-CBC:-CAMELLIA-256-CBC
olcSyncrepl: {0}rid=101 provider=ldap://openldap.openldap-net binddn="cn=admin,dc=ldap,dc=gohilton,dc=com" bindmethod=simple credentials="admin" searchbase="dc=ldap,dc=domain,dc=com" type=refreshAndPersist interval=00:00:00:10 retry="60 +" timeout=1 starttls=critical
olcSyncrepl: {1}rid=102 provider=ldap://openldap2.openldap.net binddn="cn=admin,dc=ldap,dc=gohilton,dc=com" bindmethod=simple credentials="admin" searchbase="dc=ldap,dc=domain,dc=com" type=refreshAndPersist interval=00:00:00:10 retry="60 +" timeout=1 starttls=critical
The certificates I'm using are SSL self-signed certificates.
With the starttls command -- I'm really wondering about something regarding TLS.  I does the one of the replication serverspresent client certs to the other replication server, or does this not occur?  I'm assuming by my error presented above the TLS connection is failing and the transfer is occurring without TLS?
Here are my environmental variables:
      LDAP_LOG_LEVEL: 256
      LDAP_ORGANISATION: domain
      LDAP_DOMAIN: openldap.domain.com
      LDAP_BASE_DN: dc=ldap,dc=domain,dc=com
      LDAP_ADMIN_PASSWORD: **PASS*
      LDAP_CONFIG_PASSWORD: config
      LDAP_TLS: "true"
      LDAP_TLS_CRT_FILENAME: cert.pem
      LDAP_TLS_KEY_FILENAME: key.pem
      LDAP_TLS_CA_CRT_FILENAME: ca.pem
      LDAP_TLS_DH_PARAM_FILENAME: "dhparam.pem"
      LDAP_TLS_ENFORCE: "false"
      LDAP_TLS_PROTOCOL_MIN: 3.4
      LDAP_TLS_VERIFY_CLIENT: try
      KEEP_EXISTING_CONFIG: "false"
      LDAP_REMOVE_CONFIG_AFTER_SETUP: "false"
      LDAP_SSL_HELPER_PREFIX: ldap
      LDAP_OPENLDAP_UID: 439
      LDAP_OPENLDAP_GID: 439
      LDAP_BACKUP_TTL: 15
      LDAP_REPLICATION: "true"
      LDAP_REPLICATION_HOSTS: "#PYTHON2BASH:['ldap://openldap.openldap-net','ldap://openldap2.openldap-net']"

@pcolmer
Thanks a lot for the info contained in this thread. It really helped me out particularly when setting the domain and hostname.
I'm running osixia 1.40.
The actual replication between two docker images located on two different VMs seems to be working.  I can make changes on the one instance and see the changes on the other.  I'm getting this in the logs of one of the instances:
TLS: can't connect: (unknown error code).
5ff0efb2 slap_client_connect: URI=ldap://openldap.openldap-net Error, ldap_start_tls failed (-11)
5ff0efb2 do_syncrepl: rid=101 rc -11 retrying

I following parameters were set automatically:
olcTLSCipherSuite: SECURE256:+SECURE128:-VERS-TLS-ALL:+VERS-TLS1.2:-RSA:-DHE-DSS:-CAMELLIA-128-CBC:-CAMELLIA-256-CBC
olcSyncrepl: {0}rid=101 provider=ldap://openldap.openldap-net binddn="cn=admin,dc=ldap,dc=gohilton,dc=com" bindmethod=simple credentials="admin" searchbase="dc=ldap,dc=domain,dc=com" type=refreshAndPersist interval=00:00:00:10 retry="60 +" timeout=1 starttls=critical
olcSyncrepl: {1}rid=102 provider=ldap://openldap2.openldap.net binddn="cn=admin,dc=ldap,dc=gohilton,dc=com" bindmethod=simple credentials="admin" searchbase="dc=ldap,dc=domain,dc=com" type=refreshAndPersist interval=00:00:00:10 retry="60 +" timeout=1 starttls=critical
The certificates I'm using are SSL self-signed certificates.
With the starttls command -- I'm really wondering about something regarding TLS.  I does the one of the replication serverspresent client certs to the other replication server, or does this not occur?  I'm assuming by my error presented above the TLS connection is failing and the transfer is occurring without TLS?
Here are my environmental variables:
      LDAP_LOG_LEVEL: 256
      LDAP_ORGANISATION: domain
      LDAP_DOMAIN: openldap.domain.com
      LDAP_BASE_DN: dc=ldap,dc=domain,dc=com
      LDAP_ADMIN_PASSWORD: **PASS*
      LDAP_CONFIG_PASSWORD: config
      LDAP_TLS: "true"
      LDAP_TLS_CRT_FILENAME: cert.pem
      LDAP_TLS_KEY_FILENAME: key.pem
      LDAP_TLS_CA_CRT_FILENAME: ca.pem
      LDAP_TLS_DH_PARAM_FILENAME: "dhparam.pem"
      LDAP_TLS_ENFORCE: "false"
      LDAP_TLS_PROTOCOL_MIN: 3.4
      LDAP_TLS_VERIFY_CLIENT: try
      KEEP_EXISTING_CONFIG: "false"
      LDAP_REMOVE_CONFIG_AFTER_SETUP: "false"
      LDAP_SSL_HELPER_PREFIX: ldap
      LDAP_OPENLDAP_UID: 439
      LDAP_OPENLDAP_GID: 439
      LDAP_BACKUP_TTL: 15
      LDAP_REPLICATION: "true"
      LDAP_REPLICATION_HOSTS: "#PYTHON2BASH:['ldap://openldap.openldap-net','ldap://openldap2.openldap-net']"

@kevdogg

The certificates I'm using are SSL self-signed certificates.

This is probably the root cause.
SSL certificates work best when there is a trusted Certificate Authority (CA) issuing the certificates. The hassle with self-signed certificates is that there is no such CA because the individual server has signed its own certificate and that server doesn't exist as a CA on the other server(s).
Any system trying to connect to a service that is using self-signed certificates will have a problem with trusting that certificate. In a production environment, your choices are:

Set up a private CA and add that CA to the trusted list of CAs on all systems or

(Preferably) Use publicly issued certificates such as those from Let's Encrypt (free!)

Once you make that change, the TLS issues should go away, but you may hit other problems as you seem to have several different domains being referenced, although that may just be incomplete redaction on your part.

@kevdogg

The certificates I'm using are SSL self-signed certificates.

This is probably the root cause.
SSL certificates work best when there is a trusted Certificate Authority (CA) issuing the certificates. The hassle with self-signed certificates is that there is no such CA because the individual server has signed its own certificate and that server doesn't exist as a CA on the other server(s).
Any system trying to connect to a service that is using self-signed certificates will have a problem with trusting that certificate. In a production environment, your choices are:

Set up a private CA and add that CA to the trusted list of CAs on all systems or

(Preferably) Use publicly issued certificates such as those from Let's Encrypt (free!)

Once you make that change, the TLS issues should go away, but you may hit other problems as you seem to have several different domains being referenced, although that may just be incomplete redaction on your part.

@pcolmer
Thanks for reply.  Yea the self-signed certs may be a problem however Let's Encrypt and other such authorities which I regularly utilize don't offer the ability for client SSL certificates so I'm kind of stuck rolling my own. I've created all the certificates with only local CA and then manually distribute them from that source.  I've verified all the client and server certificates can validate against the CA so I'm pretty sure I've set the certificates up correctly as I'm using similar certificates in other containers.
I don't know how to describe things, but this container is "fragile". I get things up and running and then the container will totally break when trying to add replication.  I've bin mounted the directory containing the ssl certs, but I've noticed that if I change some of the SSL certs in the bin mount, they don't get copied over to the /var/run directory unless the container is rebuilt. I wasn't aware the actual start process worked this way.   I wish I had more errors to post to help debug the problem, however it seems that anytime I make a change with the replication parameters, the entire container "breaks" and the slapd wont start.  I've noticed if I removed the replication parameters from the docker-compose file, the changes do make it back into the container since the flag "started-with-replication" is still present although I've removed these from the docker-compose.  The way things are constructed it appears things are kind of setup as a one way process --- set things up initially and you can add features along the way (such as tls, replication, SSL clientetc), but later its really hard to remove these features.

@pcolmer
Thanks for reply.  Yea the self-signed certs may be a problem however Let's Encrypt and other such authorities which I regularly utilize don't offer the ability for client SSL certificates so I'm kind of stuck rolling my own. I've created all the certificates with only local CA and then manually distribute them from that source.  I've verified all the client and server certificates can validate against the CA so I'm pretty sure I've set the certificates up correctly as I'm using similar certificates in other containers.
I don't know how to describe things, but this container is "fragile". I get things up and running and then the container will totally break when trying to add replication.  I've bin mounted the directory containing the ssl certs, but I've noticed that if I change some of the SSL certs in the bin mount, they don't get copied over to the /var/run directory unless the container is rebuilt. I wasn't aware the actual start process worked this way.   I wish I had more errors to post to help debug the problem, however it seems that anytime I make a change with the replication parameters, the entire container "breaks" and the slapd wont start.  I've noticed if I removed the replication parameters from the docker-compose file, the changes do make it back into the container since the flag "started-with-replication" is still present although I've removed these from the docker-compose.  The way things are constructed it appears things are kind of setup as a one way process --- set things up initially and you can add features along the way (such as tls, replication, SSL clientetc), but later its really hard to remove these features.

@kevdogg

set things up initially and you can add features along the way (such as tls, replication, SSL client certs, etc), but later its really hard to remove these features once added

Some of this is likely to be down to how OpenLDAP itself works. The software stores its configuration in itself so trying to remove features does require some careful attribute editing/removal.

Let's Encrypt and other such authorities which I regularly utilize don't offer the ability for client SSL certificates so I'm kind of stuck rolling my own.

Correct - if you are wanting to do client certificates then you will need your own CA.
You may find it easier to build your own derivative of the OpenLDAP Docker image. See https://github.com/osixia/docker-openldap#advanced-user-guide for an example, including adding certs.

@kevdogg

set things up initially and you can add features along the way (such as tls, replication, SSL client certs, etc), but later its really hard to remove these features once added

Some of this is likely to be down to how OpenLDAP itself works. The software stores its configuration in itself so trying to remove features does require some careful attribute editing/removal.

Let's Encrypt and other such authorities which I regularly utilize don't offer the ability for client SSL certificates so I'm kind of stuck rolling my own.

Correct - if you are wanting to do client certificates then you will need your own CA.
You may find it easier to build your own derivative of the OpenLDAP Docker image. See https://github.com/osixia/docker-openldap#advanced-user-guide for an example, including adding certs.

@pcolmer
Building a custom container is only going to solve a little bit of the issues.  For example you could add custom certs via a Dockerfile, or add these as a bind mounted volume.  The problem however with the example is that the certs are only copied to the /var/run subdirectory tree upon container creation. Additionally part of screwing around with openldap for me is "I don't really know exactly what I want initially".  I kind of learn by doing which I think is a lot of DevOps work.  I always thought a motto of containers was to "treat them like cattle" whereby the data and application were separate.  You could through away the container separate from the configuration and data.  In this implementation however this truly isn't the case.  Since the configuration is intermingled within the actual data it's really hard to "start over" with a fresh configuration because remnants of the old configuration are contained within the data.  Thank goodness for zfs snapshots on my end, however it's just really difficult to work with.  I'm honestly learning a lot more about openldap than I really cared to know, but it's very confusing since there isn't great documentation on where this project is storing it's modifications.  Perhaps if I spun up a VM with with openldap/slapd I would likely understand a lot better, however in doing so this in some ways defeats the abstraction layer containers are supposed to present.
Thanks for your posts since with some of your suggestions I really would not have made it as far as I have.

@pcolmer
Building a custom container is only going to solve a little bit of the issues.  For example you could add custom certs via a Dockerfile, or add these as a bind mounted volume.  The problem however with the example is that the certs are only copied to the /var/run subdirectory tree upon container creation. Additionally part of screwing around with openldap for me is "I don't really know exactly what I want initially".  I kind of learn by doing which I think is a lot of DevOps work.  I always thought a motto of containers was to "treat them like cattle" whereby the data and application were separate.  You could through away the container separate from the configuration and data.  In this implementation however this truly isn't the case.  Since the configuration is intermingled within the actual data it's really hard to "start over" with a fresh configuration because remnants of the old configuration are contained within the data.  Thank goodness for zfs snapshots on my end, however it's just really difficult to work with.  I'm honestly learning a lot more about openldap than I really cared to know, but it's very confusing since there isn't great documentation on where this project is storing it's modifications.  Perhaps if I spun up a VM with with openldap/slapd I would likely understand a lot better, however in doing so this in some ways defeats the abstraction layer containers are supposed to present.
Thanks for your posts since with some of your suggestions I really would not have made it as far as I have.

@pcolmer
I'm often running into the following error: 5ff76144 read_config: no serverID / URL match found. Check slapd -h arguments.
It seems the problem is with my /etc/hosts file the doesn't contain the <IP_ADDRESS> entry.
With a non-working setup, my /etc/hosts file appears as:
# cat /etc/hosts
<IP_ADDRESS>	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
<IP_ADDRESS>	openldap2.domain.com openldap2
<IP_ADDRESS>	openldap2.domain.com openldap2

Whereas for a working container, it will appear as the following:
# cat /etc/hosts
<IP_ADDRESS> openldap.domain.com openldap
<IP_ADDRESS>	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters

On both sets of containers (working and non-working):
(Non-working)
hostname
openldap2
hostname --fqdn
openldap2.domain.com
(Working)
hostname
openldap
hostname --fqdn
openldap.domain.com
I've noticed the <IP_ADDRESS> entry is added at the very end of the /container/service/slapd/startup.sh file the following:
FQDN="$(/bin/hostname --fqdn)"
if [ "$FQDN" != "$HOSTNAME" ]; then
    FQDN_PARAM="$FQDN"
else
    FQDN_PARAM=""
fi
ETC_HOSTS=$(cat /etc/hosts | sed "/$HOSTNAME/d")
echo "<IP_ADDRESS> $FQDN_PARAM $HOSTNAME" > /etc/hosts
echo "$ETC_HOSTS" >> /etc/hosts

So it would appear this startup.sh file should be executed everytime as there is really no conditional for not writing the <IP_ADDRESS> parameter.
Why in some cases as demonstrated above is this line not added to the /etc/hosts file?

@pcolmer
I'm often running into the following error: 5ff76144 read_config: no serverID / URL match found. Check slapd -h arguments.
It seems the problem is with my /etc/hosts file the doesn't contain the <IP_ADDRESS> entry.
With a non-working setup, my /etc/hosts file appears as:
# cat /etc/hosts
<IP_ADDRESS>	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
<IP_ADDRESS>	openldap2.domain.com openldap2
<IP_ADDRESS>	openldap2.domain.com openldap2

Whereas for a working container, it will appear as the following:
# cat /etc/hosts
<IP_ADDRESS> openldap.domain.com openldap
<IP_ADDRESS>	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters

On both sets of containers (working and non-working):
(Non-working)
hostname
openldap2
hostname --fqdn
openldap2.domain.com
(Working)
hostname
openldap
hostname --fqdn
openldap.domain.com
I've noticed the <IP_ADDRESS> entry is added at the very end of the /container/service/slapd/startup.sh file the following:
FQDN="$(/bin/hostname --fqdn)"
if [ "$FQDN" != "$HOSTNAME" ]; then
    FQDN_PARAM="$FQDN"
else
    FQDN_PARAM=""
fi
ETC_HOSTS=$(cat /etc/hosts | sed "/$HOSTNAME/d")
echo "<IP_ADDRESS> $FQDN_PARAM $HOSTNAME" > /etc/hosts
echo "$ETC_HOSTS" >> /etc/hosts

So it would appear this startup.sh file should be executed everytime as there is really no conditional for not writing the <IP_ADDRESS> parameter.
Why in some cases as demonstrated above is this line not added to the /etc/hosts file?

So it would appear this startup.sh file should be executed everytime as there is really no conditional for not writing the <IP_ADDRESS> parameter.

To be honest, I think that this needs to be raised as a separate GitHub issue as it is more to do with how the Docker image is initialised than specifically LDAP itself.

So it would appear this startup.sh file should be executed everytime as there is really no conditional for not writing the <IP_ADDRESS> parameter.

To be honest, I think that this needs to be raised as a separate GitHub issue as it is more to do with how the Docker image is initialised than specifically LDAP itself.

I appreciate that a lot of people have been using this GitHub issue as a common contact point for getting help with their LDAP replication problems, and I'm happy to help when I can, but it is becoming a little bit unwieldy handling conversations this way.
Since the original problems I documented in this issue have been dealt with by the incorporation of https://github.com/osixia/docker-openldap/pull/420, I'm going to close this comment.
If anyone is having LDAP replication issues with the latest Docker container, I suggest you create a new GitHub issue and, if you want, tag me in your comment. I can't guarantee that I can help but I will do my best 😉

I appreciate that a lot of people have been using this GitHub issue as a common contact point for getting help with their LDAP replication problems, and I'm happy to help when I can, but it is becoming a little bit unwieldy handling conversations this way.
Since the original problems I documented in this issue have been dealt with by the incorporation of https://github.com/osixia/docker-openldap/pull/420, I'm going to close this comment.
If anyone is having LDAP replication issues with the latest Docker container, I suggest you create a new GitHub issue and, if you want, tag me in your comment. I can't guarantee that I can help but I will do my best 😉


python3Packages.github3_py: update list of dependencies

Description of changes
ZHF: #172160
According to Hydra, this package does not build because of a missing dependency. After checking the repo, some of the dependencies used weren't needed and others were missing. Also enabled tests and added a Python version constraint.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 172416 run on x86_64-linux 1

2 packages built:

python310Packages.github3_py
python39Packages.github3_py


Should this plugin inline @imports?

According to the CSS Modules documentation:

All URLs (url(...)) and @imports are in module request format (./xxx and ../xxx means relative, xxx and xxx/yyy means in modules folder, i. e. in node_modules).

I noticed that postcss-modules does not inline @import statements. Is this a bug, or should I be using postcss-import for this?

@cameron-martin I'm pretty sure this is expected behavior :)

@cameron-martin @alexisvincent Nope, it shouldn't. Because there is another plugin for this. You should use postcss-import.


[WIP] Explore script block approach to integration

This code is not ready for merge, but is intended to provide a sandbox and means of communicating different strategies for integrating ccdb5-ui and this repository
Additions

Removals

Changes

Testing

Screenshots
Notes

Todos

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] New functions include new tests
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated
[ ] Reviewers requested with the Assignee tool :arrow_right:

@sephcoster @sebworks

I think we should explore adding a package.json file and gulp file in the ccdb5 directory.  The gulp task can take care of handling installation and setup. You can then decide if you want to make all the assets an NPM module or just use git in package.json. The assets can be copied to the the static directory via gulp.
I think we are moving away from loading assets off of a CDN.  Check out Platform issues 384 and 713. cc @ascott1.


me.code does not have style parameters support

Is your feature request related to a problem? Please describe.
me.code or me.markdown with code blocks, does not have a css styling option. Hence it is not possible to change the background color, height of the code block etc.
Describe the solution you'd like
IMO me.code should not be calling me.markdown under the hood and should support style parameters for the code block.
Describe alternatives you've considered
Additional context

I think at minimum we can add a Style parameter to code and then pass the Style down to me.markdown. You'll still be a bit limited in what changes you can make.
But I think if you want more control, you can try something like:
with me.box(classes="my-code-styles"):
   me.code("Your code")

Then you can import a stylesheet to override the code styles. I have not tried this yet so there could be limitations. But I think would actually give you more control over the styles than adding a style parameter (though we should still add that for consistency).

My apologies for the delayed response.
Actually, there are some limitations. Changing only the style of me.box doesn't actually change the style of the code block. If I don't include my-code-styles > * { background: lightblue; } in the CSS file, the background color of the code block doesn't change. This is because the markdown code block has its own CSS styling, which is not accessible from me.markdown.
In my use case, I wanted to change the background color as well as the height of the code block. me.code doesn't support styling, so I tried:
me.markdown(f"```{code}```", style=me.Style(background_color="lightblue", height="500px"))

but it didn't work. If you try passing the me.code style to me.markdown, the result would be the same as above, which is surprising behavior. Passing the style parameter to me.code should change the style of the code block, not just the wrapper around it.
In the case of markdown, there may be multiple code blocks in a single markdown string, so more control over each code block would be needed. This is why I thought me.code should have special style support that actually changes the style of the code block.

Thanks a lot for the additional details. I agree about style not working that well for the code blocks or even markdown. I think this makes sense since we use highlight js themes for the code blocks, so makes it hard to override with style.
The CSS styles for the code blocks starts here: https://github.com/google/mesop/blob/main/mesop/web/src/app/styles.scss#L368
In terms of the box class suggestion, that's good to know. I haven't tested it out yet. But I feel like there's a way to make it work. I agree it will be a bit hacky though.
In terms of a cleaner way to handle this, it's possible we could add a classes parameter to markdown / code component (if it's too hard to do with me.box). But that's still fairly challenging.
Another option is having a way to specify different highlight js themes per code block. I haven't thought too much about how that could work. We'd also want it to work cohesively with Material Theming (https://github.com/google/mesop/issues/669) when we get there.
The other option may be more in line with your suggestion which would be to include parameters to say change the background color of the code block, font size. I think that would end up being too limiting and stray a bit from our current API designs.
I do agree that it would be nice to be able to control the code block styling rather than force someone to use a pre-existing theme.
Right now not readily apparent what would be a good API design for this yet.

Once #989 lands, then I think writing CSS and using box class is a sufficient solution for styling the code blocks and markdown.
But if you really want full control over the markdown, you can create your own web component and wrap a markdown library like marked.js (which is what we use internally). I've done this for my own Mesop app because of some advanced use cases and it's worked well.

Great point about web components. Funny thing is we actually have an example here: https://github.com/google/mesop/tree/main/mesop/examples/web_component/markedjs
We created this one before we moved our markdown component implementation to using markedjs.

Going to close this issue. Although I think adding a style parameter to code could be good for symmetry, I'm not sure how helpful it actually is in this case.


Feedback - Migration, Policies etc.

Hi guys,
I really like the idea to move the json archetype configurations & policies in a separate terraform provider, so that we can have input validation, type linting and naming for the configurations.
here are my two cents on the module

really looking forward to having type linting and input validation for the configurations
will there be a migration strategy from caf-enterprise scale terraform module to this one? how will they work together?
where are the resources defined, e.g. log analytics workspace and identities which do the remediation of the policies? will I need to pass in the log analytics workspace into every archetype where I have a policies with logs?
some initiatives have a lot of parameters. it would be amazing to have typelinting to overwrite the defaults for them.

BR,
Stephi

Thank you @steph409 for taking the time to read and feed back on the specification. It is very much appreciated.
In terms of your feedback:

really looking forward to having type linting and input validation for the configurations

Thanks! We feel this was the correct approach for assignments, given the parameters are likely to come from the Terraform resource graph.

will there be a migration strategy from caf-enterprise scale terraform module to this one? how will they work together?

Migration of management groups will be a re-deploy and move subscriptions. Migration of networking, LA workspace, etc will be either a state migration, or left alone in the old module. We are working on some automation for the state migration.

where are the resources defined, e.g. log analytics workspace and identities which do the remediation of the policies? will I need to pass in the log analytics workspace into every archetype where I have a policies with logs?

Resources will be defined in separate modules, we are creating them now. There is one here https://github.com/Azure/terraform-azurerm-hubnetworking and we also have the alz management module completed. VWAN is in progress.
In terms of well known parameters, yes unfortunately these will have to be passed into each archetype data source. I do not think we can have references within provider blocks. I will test this and if we can move it into the provider block then we will do that.

some initiatives have a lot of parameters. it would be amazing to have typelinting to overwrite the defaults for them.

We can investigate this. For built-in and ALZ default policy sets this could work, but not sure we could do this for custom policy sets.

@matt-FFFFFF are you (or the Azure on Terraform team as a whole) also working with Hashicorp to potentially extend the terraform expressiveness? E.g. by introducing base classes. In such a base class one could configure the central log analytics workspace. Another way of factorizing the resource declarations would be letting the provider having access to the full module scope. The provider could then first read the general settings resource declaration and then start processing the other resource declarions.

Closing as initial experimental version is released


Enable national cloud support for Azure Search SDK

Search SDK currently hard codes the audience to the public cloud. We need to enable the support for multiple audience to target non public clouds by the SDK.

Duplicate: https://github.com/Azure/azure-sdk-for-net/issues/30306
This feature has been implemented here: https://github.com/Azure/azure-sdk-for-net/pull/30316


Boilerplate free case class serialization with implicit macros

See discussion: https://groups.google.com/d/msg/spray-user/zTORNzACIp0/qTmDWi9TZ8EJ
Experimentation branch:
https://github.com/jrudolph/spray/tree/feature/json-case-class-implicit-macro

What's left to do to get macro-based case class serialization in? Is there anything I can do to help?

@zackangelo Thanks for offer to help. We currently don't have any capacity for major changes/extensions to spray-json, so we can't really say when macro-based case class support will be available. Sorry!

@sirthias Thanks for responding! I'm sure you guys are swamped.
I went ahead and wrote a small macro library that supports my use cases (automatic formats for case classes with greater than 22 fields, field renaming, etc). I'll just leave a link here in case someone else finds it useful: https://github.com/zackangelo/spray-json-macros
Any feedback would be greatly appreciated.

Great, thanks for the pointer!

FYI https://github.com/fommil/spray-json-shapeless should solve this problem without any new macros.

@sirthias btw, it is worth pointing out that Miles convinced me that Apache v2 was more appropriate for spray-json-shapeless.


Silee2/fix prebuilt mkldnn

pre-installed mkldnn support was broken. This PR fixes it. Same as #2574 for r0.15

TESTNOW


Added compatibility with WirelessPalaControl

I built the DIY box from @Domochip, but when I wanted to associate it with this component, I had a connection error with the IP address.
I don't think it's very complex to add the compatibility, since they work with the same commands.
Would you have time to look into it ?
Thank you.
https://github.com/Domochip/WirelessPalaControl

Hello,
I'm currently working to get WPalaControl compatible with all different plugins that handle CBox.
Stay tune on WPalaControl repo :-)

Ah, that's perfect so, thank you !

Hello,
The new version has been released : https://github.com/Domochip/WirelessPalaControl/releases/tag/v1.4.1-2.0.0
You should now be able to use this HomeAssistant plugin like any CBox.
Let me know if you encounter any issue.

Bonjour, merci beaucoup, je teste ça dès que j'ai du temps !


metabase: 0.46.6 -> <IP_ADDRESS>

Description of changes
https://github.com/metabase/metabase/releases/tag/v<IP_ADDRESS>

Upgrade your Metabase installation IMMEDIATELY.
A recently discovered security vulnerability almost certainly affects you, and we recommend you upgrade your Metabase Installation right away.

diff: https://github.com/metabase/metabase/compare/v1.46.6...v<IP_ADDRESS>
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Duplicate of #244675 and the other one as already one approval.


feat(): add interaction service for a11y

Closes #289.

We're working on a fastclick solution in core that should make some of this easier (handling touch delay, possibly new APIs for global event listeners, etc).   It would probably make sense to wait on this functionality a bit and re-visit after.
cc @jelbourn

@kara I'm fine with that, this PR can wait for the fastclick solution, but it should be also able to land, because once the Fastclick Solution is landed, we just need to remove the buffer, which is really small effort.

@DevVersion I chatted with @kara about this and we agreed that it would be better to wait for the fastclick to land because it will make this service simpler.

@jelbourn Okay cool, just come back if fastclick is available and I will rebase on top of that ;)

@marcysutton do you have any thoughts about the combination "progressbar" / "button" from any a11y standpoint?
I would guess that you'd want the two to be siblings in the DOM rather than one inside of the other. I'd also think that you'd want the "progressbar" to be aria-hidden until the button interaction, meaning you might want it to seem like the progress bar is "inserted" after the button is clicked. Not sure if it would make any sense whatsoever to manipulate focus during this interaction.

Just realized my previous comment was on the wrong issue. It should have been on #290

@DevVersion, we're going to try to approach this instead by adding a feature to Zones that can tell you the type of the most recent event was, and then only show if it was a KeyboardEvent. Going to close this.

I'll be very curious to see what that feature looks like. Differentiating between keydown and click can get tricky depending on the element's role.

@jelbourn Sounds interesting. Also wouldn't that mean that the events have to be triggered inside of a zone?

@DevVersion not sure, depends on how @robertmesserle implements it.
@marcysutton I can CC you on the PR once it exists (and if we validate that the idea works in general).


Can't cast unless I update the Google Play services

After the latest update of the MicroG (<IP_ADDRESS>5) whenever I try to cast the notification will pop up saying:
"YouTube Vanced won't run unless you update Google Play services."
Clicking on this notification leads me to infinite loading in Google Play.
Phone: Sony Xperia 5 II
OS: Android 11 (58.1.A.5.159)
Manager: 2.6
Vanced: 16.16.38
Google Play: 25.1.24-21

Duplicate of #688


Differentiate users invited as admin by showing bolt icon

Fixes one issue in #7194

I think it's mostly good but maybe give it 2px of margin-left and set the opacity to 0.5 so it doesn't stand out too much.

@brockwhittaker Updated. Looks better now :)

Looks good, good to merge @showell!

Merged, thanks @hackerkid @brockwhittaker
@hackerkid We may want to refine the relevant backend test to assert that field is provided.


Cherry-pick: Remove statsd-to-prometheus from remaining config and test files

Includes:

1bfd98e3378151e18032d7498b0c4bc3a1bcbba4

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: douglas-reid
If they are not already assigned, you can assign the PR to them by writing /assign @douglas-reid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS
tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: mandarjog
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS [mandarjog]
tests/OWNERS [mandarjog]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Register rabug.localplayer.dev

Requirements

[x] You have completed your website.
[x] The website is reachable.
[x] The CNAME record doesn't contain https:// or /.
[x] There is sufficient information at the owner field.
[x] There is no NS Records (Enforced as of Sepetember 4th, 2024)

Description

Link to Website

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.
I have read the CLA Document and I hereby sign the CLA
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

recheck


Ajax Data Loading demo not working

http://ng-table.com/#/demo/5-2
The above demo isn't working.

Yeah those demo's are falling into disrepair.
I am in the process of rewriting the entire demo-site. Should be ready Monday next week.

OK so maybe not... still working through changes before being able to complete the demo site.
In the meantime check out the codepen's that are forming the bulk of the demo-site: http://codepen.io/collection/XOLQPM/

The demo site is now sufficiently complete so look there for working examples


MKIAU_JCAP issue

I am seeing an error when trying to use MKIAU_JCAP to filter replay increments on the develop branch:
Applying Mass Divergence Fix ...
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
GEOSgcm.x<PHONE_NUMBER>1C78DD  Unknown               Unknown  Unknown
libpthread-2.22.s  00002AAAB5966C10  Unknown               Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>2AFAA3  Unknown               Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>9F0949  Unknown               Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>AE2F39  Unknown               Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>ADE137  mkiau_specmod_mp_         578  mkiau_specmod.F90
GEOSgcm.x<PHONE_NUMBER>ADCDA1  mkiau_specmod_mp_         425  mkiau_specmod.F90
GEOSgcm.x<PHONE_NUMBER>AC97A4  geos_mkiaugridcom        3022  GEOS_mkiauGridComp.F90
GEOSgcm.x<PHONE_NUMBER>AC83F1  geos_mkiaugridcom        2974  GEOS_mkiauGridComp.F90
GEOSgcm.x<PHONE_NUMBER>A71E2D  geos_mkiaugridcom        2193  GEOS_mkiauGridComp.F90
GEOSgcm.x<PHONE_NUMBER>A4A8A6  geos_mkiaugridcom        1135  GEOS_mkiauGridComp.F90
libesmf_fullylink  00002AAAB17B6847  _ZN5ESMCI6FTable1     Unknown  Unknown
libesmf_fullylink  00002AAAB17B9CB3  ESMCI_FTableCallE     Unknown  Unknown
libesmf_fullylink  00002AAAB1BCC395  ZN5ESMCI2VM5ente     Unknown  Unknown
libesmf_fullylink  00002AAAB17B792D  c_esmc_ftablecall     Unknown  Unknown
libesmf_fullylink  00002AAAB1DDA731  esmf_compmod_mp_e     Unknown  Unknown
libesmf_fullylink  00002AAAB1F8FC0F  esmf_gridcompmod     Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>42860F  geos_gcmgridcompm        1349  GEOS_GcmGridComp.F90
libesmf_fullylink  00002AAAB17B6847  _ZN5ESMCI6FTable1     Unknown  Unknown
libesmf_fullylink  00002AAAB17B9CB3  ESMCI_FTableCallE     Unknown  Unknown
libesmf_fullylink  00002AAAB1BCC395  ZN5ESMCI2VM5ente     Unknown  Unknown
libesmf_fullylink  00002AAAB17B792D  c_esmc_ftablecall     Unknown  Unknown
libesmf_fullylink  00002AAAB1DDA731  esmf_compmod_mp_e     Unknown  Unknown
libesmf_fullylink  00002AAAB1F8FC0F  esmf_gridcompmod     Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>EDE787  mapl_capgridcompm        1051  MAPL_CapGridComp.F90
GEOSgcm.x<PHONE_NUMBER>EDE39E  mapl_capgridcompm         998  MAPL_CapGridComp.F90
GEOSgcm.x<PHONE_NUMBER>EDE053  mapl_capgridcompm         720  MAPL_CapGridComp.F90

@wmputman Can you remind me how to actually turn this on? I always forget

I'm also getting a crash when enabling this. Will look into.

The CVS build of Jason-3_2 which appears to have an identical mkiau grid comp and ncep_sp libraries runs fine. The git/cmake build crashes in the NCEP_sp with an initialized variable in the ncep library. Tried using the cmake built libraries for iau and ncep_sp in the Jason-3_2 build, that ran ...

The issue turned out to be that the GEOSgcm.x is now linking to the r8i4 version of the NCEP_sp library. The CVS build was linking to the r4i4 version. The reason it is picking up the r8i4 version is that GMAO_tranf in shared was linking to the r8i4 version which apparently the linking in the GEOSgcm.x inherits. Telling GMAO_transf that it should use the r4i4 version via the CMakeLists.txt file allows GEOSgcm.x to link to the r4i4 version and lets the JCAP option in mkiau run but when the stochastic physics is turned on this results in a non-zero diff to the original code.


[qttools] copy traget tools only if they exists.

They do not exist if you crosscompile qt.
This fixes the build error that happens here when using an arm64-osx triplet on an x64-osx host:
https://github.com/daschuer/vcpkg/actions/runs/4476393103/jobs/7866710248

[X] Changes comply with the maintainer guide
[X] SHA512s are updated for each updated download
[X] The "supports" clause reflects platforms that may be fixed by this new version
[X] Any fixed CI baseline entries are removed from that file.
[X] Any patches that are no longer applied are deleted from the port's directory.
[X] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[X] Only one version is added to each modified port's versions file.

Note: I will be converting your PR to draft status. When you respond, please revert to "ready for review".
That way, I can be aware that you've responded since you can't modify the tags.


A new tab opens evertime I start VSCode

Besides the fact that the Glow feature doesn't work as the command does not exist, there seems to be another issue. Everytime I start VSCode, a new tab titled "What's new for Synthwave '84" opens. I really like the extention, but I can't use it like this. I'll keep an eye for an update.
Thank you.

Same

I opened the "src/extension.js" for this extension and added a "return" statement at the top, and the problem went away:
function showUpdatePage() {
    return; // add this line

		const panel = vscode.window.createWebviewPanel(
			`synthwave.whatsNew`, // Identifies the type of the webview. Used internally
			'What\'s new for Synthwave \'84', // Title of the panel displayed to the user
			vscode.ViewColumn.One, // Editor column to show the new webview panel in.
			{ enableScripts: !0 } // Webview options. More on these later.
		);

		const viewPath = path.join(this.cntx.extensionPath, "whats-new", "view.html");
		const viewResourcePath = panel.webview.asWebviewUri(viewPath);
		const htmlContent = fs.readFileSync(viewPath, "utf-8");
		panel.webview.html = htmlContent;
}

I removed the tab altogether - sorry about that this comment closes #7


Fixed Serbian translation

Thanks @fejes713!

Thanks for your contribution to Homebrew!  You rock!


❌ Software Removal | Disroot

Description
Disroot does not meet the Email Provider Criteria.
Why I am making the suggestion
Disroot does not support MTA-STS and TLS-RPT.
The DMARC is broken.
https://www.hardenize.com/report/disr.it/1620291629
My connection with the software
user

[x] I will keep the issue up-to-date if something I have said changes or I remember a connection with the software.

You are using incorrect addresses to test them.


Is there a way to specify what directory to look in for linting?

What problem does this feature solve?
I posted the following question on StackOverflow...
https://stackoverflow.com/questions/51392974/how-to-change-the-lint-directory-in-a-vue-js-app
Wasn't sure if there was something that could be done in exclude? Maybe exclude everything that is not the directory I want?
exclude: !/src/vue-app,
What does the proposed API look like?
include: '/src/vue-app',

You can specify that in the command line, it accepts a glob.


Bug in fanfox.net provider

Tried in 1.3.18:
pipenv run manga-py --cbz --zero-fill --rename-pages --destination Manga/ http://fanfox.net/manga/isekai_de_kuro_no_iyashi_te_tte_yobarete_imasu/

It downloaded:
vol_c001-0.cbz
vol_c002-0.cbz
...
vol_c017-0.cbz
vol_c018-0.cbz
vol_c020-0.cbz
vol_c020-5.cbz
vol_v01-0.cbz
vol_v02-0.cbz
vol_v03-0.cbz
vol_v04-0.cbz

Are missing 21 to 30 and 19 chapters, while 1 to 4 are duplicate.

I changed the name generation. Please check now.
Unfortunately, some of the chapters are not sorted.
So I added a non-significant 0 at the end. (In case the chapter does not contain an additional index):
Before:

After:

Please, check version 1.3.19

Works OK in 1.3.19 :+1:
Collateral damage: Reload all when updating something downloaded with manga-py < 1.3.19, but is not much price to pay.

:cry: Unfortunately, this is so for this provider


Emojis do not work in large readmes

GitHub recently made this change, now the emojis in the README don't work. Just fyi.

Didn't see that one, thanks :smile:


Freeing Hierarchy from any dependence on streaming libraries

Hi @capitanbatata,
I think I'd like to take back maintenance of hierarchy, since I'm going to be using it a fair bit in the hnix project. I'm also going to generalize this library away from any dependence on a particular streaming library, instead choosing the simpler LogicT transformer as the foundation and implementation of ListT. See pull request #7 which implements this change. I have a supporting PR for pipes-files coming shortly. I will likely also create a new files like library that doesn't need pipes, which is really overkill for the problem we're solving.

Hi John, sure. The reason why I needed hierarchy was to be able to put another package (argon) which depended on pipes-files. After trying to update pipes-files I discover that this package could not be built on windows (a requirement of my current project). So I removed the pipes-files dependency from argon.
All in all feel free to remove me as maintainer, and I'm happy to see that some work is taking place around file streaming libraries for Haskell.

@capitanbatata Ah, fair enough. I wonder if the new formulation of this code on LogicT would indeed work on Windows, using unix-compat?

That'd be awesome, since such library would be very useful for us. I'll keep an eye on these packages. Thanks!

@capitanbatata We don't even need a dependence on logict! All that was needed was MonadPlus.  See: http://www.haskellforall.com/2014/11/how-to-build-library-agnostic-streaming.html
I'm going to apply this same logic to pipes-files to make hierarchy-files.


Sync groups for user

Hi
Is there any function to Sync groups for User ?
something like this :
$user->groups()->sync(array(1, 2, 3));
Thanks

You can try the updateGroups method that was recently added.


Q: what does human feedback do?

AutoGPT by default stops after each step for human input. You can simply type y to go on; internally the y response is translated into
            if console_input.lower().rstrip() == "y":
                user_input = "GENERATE NEXT COMMAND JSON"
                break

But if you actually type something, your response is incorporated into the memory:
        while True:
            console_input = utils.clean_input(Fore.MAGENTA + "Input:" + Style.RESET_ALL)
            if console_input.lower().rstrip() == "y":
                user_input = "GENERATE NEXT COMMAND JSON"
                break
            elif console_input.lower().startswith("y -"):
                try:
                    next_action_count = abs(int(console_input.split(" ")[1]))
                    user_input = "GENERATE NEXT COMMAND JSON"
                except ValueError:
                    print("Invalid input format. Please enter 'y -n' where n is the number of continuous tasks.")
                    continue
                break
            elif console_input.lower() == "n":
                user_input = "EXIT"
                break
            else:
                user_input = console_input
                command_name = "human_feedback"
                break

        if user_input == "GENERATE NEXT COMMAND JSON":
            logger.typewriter_log(
            "-=-=-=-=-=-=-= COMMAND AUTHORISED BY USER -=-=-=-=-=-=-=",
            Fore.MAGENTA,
            "")
        elif user_input == "EXIT":
            print("Exiting...", flush=True)
            break

Of course, if your input is EXIT, you kill the agent.
Question is, what does AutoGPT do with your 'human feedback'? It gets added to the memory:
    # Execute command
    if command_name is not None and command_name.lower().startswith( "error" ):
        result = f"Command {command_name} threw the following error: " + arguments
    elif command_name == "human_feedback":
        result = f"Human feedback: {user_input}"
    else:
        result = f"Command {command_name} returned: {cmd.execute_command(command_name, arguments)}"
        if next_action_count > 0:
            next_action_count -= 1

    memory_to_add = f"Assistant Reply: {assistant_reply} " \
                    f"\nResult: {result} " \
                    f"\nHuman Feedback: {user_input} "

    memory.add(memory_to_add)

Then what? Does AutoGPT then send memory_to_add to GPT as context for the next action?

Let's follow up memory.add(). This is in the folder memory, and we look at local.py. It seems that we take the memory_to_add as a string text, and get the embedding of text by get_ada_embedding(text). This embedding vector gets concatenated with previous embedding vectors.
    def add(self, text: str):
        """
        Add text to our list of texts, add embedding as row to our
            embeddings-matrix

        Args:
            text: str

        Returns: None
        """
        if 'Command Error:' in text:
            return ""
        self.data.texts.append(text)

        embedding = get_ada_embedding(text)

        vector = np.array(embedding).astype(np.float32)
        vector = vector[np.newaxis, :]
        self.data.embeddings = np.concatenate(
            [
                self.data.embeddings,
                vector,
            ],
            axis=0,
        )

        with open(self.filename, 'wb') as f:
            out = orjson.dumps(
                self.data,
                option=SAVE_OPTIONS
            )
            f.write(out)
        return text

The same function for pinecone.py is even simpler:
    def add(self, data):
        vector = get_ada_embedding(data)
        # no metadata here. We may wish to change that long term.
        resp = self.index.upsert([(str(self.vec_num), vector, {"raw_text": data})])
        _text = f"Inserting data into memory at index: {self.vec_num}:\n data: {data}"
        self.vec_num += 1
        return _text

You can use human feedback to basically chat with GPT. For example, in one of the tasks where I asked GPT to do research and write a business plan, it continued to 'research' endlessly. When it stopped for human feedback, I said "great work. Now write me the business plan". And it did. Then you can say "revise the business plan to provide more details", etc.


Internationalization

Description
react-i18next library was added and is ready to use throughout the UI code.
So far the assumptions is that translations are kept in single files (one for every language). However, it is possible to split them (or even put them next to their respective components) if we decide it would improve workflow.
Setting the strict flag in tsconfig.json was necessary for correct typing of the translation function (to provide a warning if a given key does not exist in translation file). All new warnings due to introduced rules were resolved.
The UI in AppSettings.tsx is a functional placeholder meant for demonstration and testing purposes - the proper implementation will be made later as described in #84
Linked issues
Resolves #95

@LostInStatic one more thing is that it would be good to add the @types/react-i18next and @types/i18next to the project as dev dependencies. The packages did not seem to come with built in types.

@LostInStatic one more thing is that it would be good to add the @types/react-i18next and @types/i18next to the project as dev dependencies. The packages did not seem to come with built in types.

AFAIK, they have built-in types. They are typed correctly on my end. Also, trying to yarn add them results in following messages:
warning<EMAIL_ADDRESS>This is a stub types definition. i18next provides its own type definitions, so you do not need this installed.
warning<EMAIL_ADDRESS>This is a stub types definition. react-i18next provides its own type definitions, so you do not need this installed.

Could you double-check on your side,  and if the issue persists provide me with your environment details?

@LostInStatic one more thing is that it would be good to add the @types/react-i18next and @types/i18next to the project as dev dependencies. The packages did not seem to come with built in types.

AFAIK, they have built-in types. They are typed correctly on my end. Also, trying to yarn add the types results in following messages:
warning<EMAIL_ADDRESS>This is a stub types definition. i18next provides its own type definitions, so you do not need this installed.
warning<EMAIL_ADDRESS>This is a stub types definition. react-i18next provides its own type definitions, so you do not need this installed.

Could you double-check on your side, and if the issue persists provide me with your environment details?

@LostInStatic ahh yes this makes sense, I think I might have not runyarn install on my machine while looking at the code. 🤦


20230109 frontend optimization

This completes the frontend optimizer and allows *standard-cl-22* to enable it.
The optimizer has a few improvements when enabled:

Constants can be designated as being 'tabled', which means they enter the left environment.
The repl's evaluator is tried and its output taken instead of the basic inliner in cl21 if it saves space.
Defuns created by let forms are marked 'synthetic' and the compiler tries producing them without inlining, taking the choices that save space.
The repl's evaluator now emulates both halves of the environment.
The repl's evaluator now combines (f (c x _)) to x and (r (c _ x)) to x as a short circuit optimization.
A few other fixes for things exposed by enabling this machinery.

Based on feedback, going to un-gate this after adding a couple of optimizations.


tour and tools for metaproteomics

Did not find the tool on toolshed, therefore tour is missing the correct link
- name: split_tabular_columns
  owner: jjohnson

ping @blankclemens

Seems like the repository of this tool has been deprecated, do you know what will happen with this tool in future @jj-umn ?

I included the split_tabular_columns functionality in the iuc tools: filter_tabular and query_tabular as the "normalize list columns" input line filter.
On 9/18/17 3:27 AM, Clemens Blank wrote:

Seems like the repository of this tool has been deprecated https://toolshed.g2.bx.psu.edu/repository?repository_id=ac73ee5351e5391b, do you know what will happen with this tool in future @jj-umn https://github.com/jj-umn ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/galaxyproject/training-material/pull/528#issuecomment-330156534, or mute the thread https://github.com/notifications/unsubscribe-auth/ACBMw-PWMudK2_bhVufufOfjHSAlJqqBks5sjilugaJpZM4PZGVj.

--
James E. Johnson Minnesota Supercomputing Institute University of Minnesota

@polkhe @blankclemens   I UN-deprecated that tool for now.
I included the split_tabular_columns functionality in the iuc tools: filter_tabular and query_tabular as the "normalize list columns" input line filter.  I think that provides a more flexible way to include that functionality for any new workflows.

@blankclemens I can change this part of the tour ofr this tool, but also the workflow has to be adjusted as well...

@blankclemens what do you think about that?

Sorry, this somehow got lost for me. We celebrate Christmas in our lab today, let me have a look tomorrow! Thanks for all the effort!

Heads-up: I'm going to discuss this further with the Galaxy-P community and will let you know if there are any news and if necessary update the tutorial accordingly.

I updated the tutorial to the latest workflow which relies on tools from IUC only #642
To reflect this workflow in the tools this would require some minor changes to the tour, sorry for that - would be great if you can include them!

@subinamehta @bebatut - I see this issue for metaproteomics tour. I was told this involves a yaml file - but not sure how to test it. Any help / inputs are appreciated.

You can install the Galaxy Tour Builder: it is a web extension you can install on your web browser (Chrome or Firefox) via the extensions. You can then open a page with any Galaxy instance, launch the extension, paste the tour there and test it

More recent PR: #1058


Hot! Slick centerMode infinite slickGoTo Bug

Hi, I need an advice!
I change slides by click on the image previews on the carousel.
Demonstration: https://jsfiddle.net/er712r5z/4/
With 5 visible slides doesn't work clicking on -2 index slide when 0 slide is active!
But -1 slide click works fine and if I go to 1,2,3 and other slides, -2 index click slide works fine (ever click on the left slide). How do I need to correct my code to work negative -2 index?
`$("#setSlider .setSliderListOneImage").click(function(){
var index = $(this).parent().data("slick-index");
$(".setSliderList").slick("slickGoTo",index);
});
$('.setSliderList').slick({
    autoplay:false,
    infinite: true,
    arrows:true,
    dots: true,
    speed: 600,
    slidesToShow: 5,
    swipe: false,
    centerMode: true,
    initialSlide: 0,
    responsive: [
        {
            breakpoint: 1400,
            settings: {
                slidesToShow: 3
            }
        },{
            breakpoint: 768,
            settings: {
                slidesToShow: 1
            }
        }
    ]
});`

One more bug I found. When I use centerMode carousel with 5 showed slides if I have only 5 slides, carousel doesn't show arrows and dots and I can't change any slide to Center - I think it is really  a shortage, because it may be needed for this configuration of carousel (centerMode)

@sl1rt I have recently found the same issue today. Have you been able to find any resolution yet?

No. I hope the developers of the plug-in will respond because this problem is very important for use it in projects.

@sl1rt agreed, it's a pretty big miss. I assume (without reading the libraries code) they were accounting for the scenario where you only have one slide and the code isn't dynamic enough to deal with multiple slides.

Yeah centerMode has some issues. It was one of the first major additional features added and it's heavy with side effects. There might be a PR in for this though. If you find one or make one yourself, I will work with you to get it into the master.
Fairly sure this is a duplicate, so I'm closing, and either way I tagged it so when we review centerMode I can make sure I've addressed it. Thanks!


null from optional types conflicts with null as addr_none in TS wrappers

Currently, optional types are described as nullable ones (e.g. boolean | null), which is fine for all types except Address.
@ton/core handles addresses in such way that null means addr_none, and it was implemented in Tact in a similar way due to the original developer being the same person who made the library.
We want to rework addresses (#1119) for consistency and predictability, but nulls prevent us from doing so without refactoring either the ton-core library itself or generated TS wrappers for Tact contracts. Current maintainer of the library suggested the second option.
More precisely, we can make some type wrapper for Maybe that will be used instead of the | null thing to support null as a value itself. This will potentially resolve the problem and allow us to describe Maybe MsgAddress type which will look like Maybe<Address | ExternalAddress | null>.
What do you think? @anton-trunov @verytactical

As far as I understand, we have Maybe<T> = T | null, while we should have done Maybe<T> = [T] | null.
export type Maybe<T> = Just<T> | Nothing
export type Just<T> = [T]
export const Just = <T>(t: T): Just<T> => [t];
export const getJust = <T>(t: Just<T>): T => t[0];
export const isJust = <T>(t: Maybe<T>): t is Just<T> => Boolean(t);
export type Nothing = null
export const Nothing: Nothing = null;
export const isNothing = <T>(t: Maybe<T>): t is Nothing => !t;


[HOTFIX] Fixed count(*) issue when MV is created with simple projection.

Problem:
When MV is created with simple projection and select count(*) is fired on main table then it is wrongly taking MV table with wrong projections.
Solution:
Simple projection MV should not be selected when count(*) is fired.
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

[ ] Any interfaces changed?

[ ] Any backward compatibility impacted?

[ ] Document update required?

[ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3319/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11585/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3517/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3450/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11717/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3653/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3517/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11787/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3721/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3565/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11839/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3772/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/3571/

Build Success with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/11845/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/3778/

LGTM


About Flutter Installation

I download Flutter and install in H:/flutter/flutter folder and update my Environment path for the  flutter.
but when i used flutter doctor command it shows
my Environment Path for flutter is H:\Flutter\flutter\bin;
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel beta, v0.8.2, on Microsoft Windows [Version 6.3.9600], locale en-US)
Unhandled exception:
SocketException: Write failed (OS Error: The pipe is being closed.

, errno = 232), port = 0
#0 _rootHandleUncaughtError. (dart:async/zone.dart:1112:29)
#1 _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#2 _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#3 _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:391:30)
#4 _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
#5 _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)
exit code 255

I am not getting this error and due to this i am cant able to try or learn flutter also i used both editor visual code and android studio but still same problem occurs
hope u will help me
Thankyou!

Is there anything special with the H: drive? Does your current user have write permissions?

Actually it's normal windows drive incase my all android related stuff is
in H: drive including Android SDK,Android studio etc..
On Tue 2 Oct, 2018, 12:25 AM Günter Zöchbauer<EMAIL_ADDRESS>wrote:

Is there anything special with the H: drive? Does your current user have
write permissions?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/22515#issuecomment-426021567,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AdRjxAtanQdo6Eha740U1Ve1ddGifAcLks5ugmUKgaJpZM4XChlR
.

Closing as dup of #22223


A question about code generating by core/rebuild_async_stubs.js

When I tried to locally implement an op function with 14 arguments in deno/ext, and the build failed due to the argument limit imposed by core/rebuild_async_stubs.js.
Looking at the contents of core/rebuild_async_stubs.js, it looks like the function is defined each time depending on the number of arguments.
Is there a reason why this can't be replaced by using rest parameters (...theArgs) as shown below?
function setUpAsyncStub(opName, originalOp) {
  const fn = function async_op(...theArgs) {
    const id = nextPromiseId;
    try {
      const maybeResult = originalOp(id, ...theArgs);
      if (maybeResult !== undefined) {
        return PromiseResolve(maybeResult);
      }
    } catch (err) {
      ErrorCaptureStackTrace(err, async_op);
      return PromiseReject(err);
    }
    if (isLeakTracingEnabled) {
      submitLeakTrace(id);
    }
    nextPromiseId = (id + 1) & 0xffffffff;
    return setPromise(id);
  };

  ObjectDefineProperty(fn, "name", {
    value: opName,
    configurable: false,
    writable: false,
  });
  return fn;
}

https://github.com/denoland/deno_core/blob/7258aa325368a8e2c1271a25c1b4d537ed41e9c5/core/00_infra.js#L193-L236

We benchmarked it and using rest argument was way slower.
As a side note: using 14 arguments might not be the best idea. Consider using and array to "group" some arguments together.

Thanks to you I have resolved my doubts!

Consider using and array to "group" some arguments together.

Good to hear. I will use it as a reference.👀


bump dependencies

Applied mkpr on package.json
/tmp/mkpr.sh

:tada: This PR is included in version 1.31.19 :tada:
The release is available on:

npm package (@latest dist-tag)
v1.31.19
GitHub release

Your semantic-release bot :package::rocket:


Reground by just grounding again

I'll be updating this.  In the meantime for the compositional we're regrounding by rereading.


Import EventAggregator into view-model 404 error

When I import EventAggregator into a view-model I get a 404 in the browser. To test I put this at the top of welcome.js in the esnext skeleton:
import { EventAggregator } from 'aurelia-event-aggregator';
I can see the aurelia-event-aggregator files in jspm_packages/npm.

Is aurelia-event-aggregator in your jspm section of your package.json and is it also listed as a top level mapping in your config.js?
Something like this should be in your config.js:
"aurelia-event-aggregator"<EMAIL_ADDRESS>

@AshleyGrant is right. If it's not there, you must explicitly jspm install aurelia-event-aggregator

(I believe pretty strongly that this is a design flaw in how system.js work, but there's nothing we can do about it unfortunately.)

I think this needs to be documented in a reminder.md file or something. Just cost me 20 mins


Settings.config callback crashes when passed option containing encoded percent sign

Hi, I have have several users of Grosh for Pebble reporting they can login to the app in the settings web page running inside Pebble Time app, but the watch fails to connect using the credentials transferred back to the watch app from the web page.
Did some tests, and it turns out you cannot transfer password containing % sign.
The web page encodes the options
           var location = "pebblejs://close#" + encodeURIComponent(JSON.stringify(options));             console.log("Warping to: " + location);
When entering<EMAIL_ADDRESS>- G1234% and looking at the console, this looks fine:
Warping to: pebblejs://close#%7B%22email%22%3A%22g%40gmail.com%22%2C%22password%22%3A%22G1234%25%22%7D
So the % is nicely escaped. But the watch app dies:
URIError: URI error decodeURIComponent@[native code]     at onCloseConfig (settings/settings.js:186:43)
Any ideas how to get around this, except telling users to stop using % in their password?

As workaround you can encode password (like base64) in your settings page, and than decode after settings page is closed.
I cannot reproduce this error, you are using Android or iOS?

Yes, it was iOS that i tested on. Yes, base64 encoding should do the trick, will do that for now.


Improve integration test flexibility

Closes #730

revised data_builder

reduced scope to function (removes any inter-dependence between tests, getting the tests closer to parallel execution; it leads to more requests, but didn't run into any performance issues.)
supports gears and users (with api keys)
auto-creates parent containers (eg if test only needs to reference an acquisition, it may explicitly create only that, parents will be implicitly added)
auto-deletes any resource created with the data_builder (explicitly or implicitly)
enables tests to recursively delete a resource (eg uid upload test creates hierarchy under a group via an upload request; which can be cleaned up calling a recursive delete on the group)
as a consequence the with_* fixtures were replaced w/ pure data_builder usage

revised requests accessors (as_*)

got rid of the external bootstrappers and now inject users using the new as_drone accessor
user and admin used to have the same access key
separated as_admin and as_root for now

cleanup tasks

passing in and loading env vars without renaming (eg SCITRAN_CORE_DRONE_SECRET)
remove unused/sort imports
remove unused logger setups
switched from bunch to attrdict

PS: Sorry about the ginormous diff :|
Review Checklist

Tests were added to cover all code changes
Documentation was added / updated
Code and tests follow standards in CONTRIBUTING.md

I looked over the diff again today and didn't see anything we haven't already discussed. Great work! I'm excited to have this merged so I can use it to write some tests for some PRs I have open 😉.
Since tests are passing I don't have any worry about unseen issues in the mammoth diff. Go ahead and merge when you're ready!

Just stopping by to say this is solid work :+1:


[hotfix][docker] Removed extra equals in mail user in docker.md

Purpose of the pull request
Removed extra equal after mail_user in docker doc
Brief change log
Updated MAIL_USER= to 'MAIL_USER` in the docker.md
Verify this pull request
This pull request is code cleanup without any test coverage.

Thanks @SbloodyS


Add confirmation before delete/save an endpoint

https://github.com/hellofresh/janus-dashboard/pull/72

Toasters (issue #33) about actions results will be added after resolving this issue


Clean up console output for dev mode and generate features from ServerFeatureUtil

Partial fix for https://github.com/OpenLiberty/ci.maven/issues/1369
See https://github.com/OpenLiberty/ci.maven/pull/1477 and https://github.com/OpenLiberty/ci.gradle/pull/729
Signed-off-by: Kathryn Kodama<EMAIL_ADDRESS>

I think both these lines should be changed to debug:
https://github.com/OpenLiberty/ci.common/blob/bef758b339b5bfb501e851aff653ec453b353878/src/main/java/io/openliberty/tools/common/plugins/util/DevUtil.java#L2467
https://github.com/OpenLiberty/ci.common/blob/bef758b339b5bfb501e851aff653ec453b353878/src/main/java/io/openliberty/tools/common/plugins/util/DevUtil.java#L2483