enable openldap uid/gid to be specified at runtime

Hello,
thanks for the PR, can you describe the problem solved by this ?
Thanks

Hi Bertrand,
I would like to specify the UID/GID as env vars at time of container start
and not build time.
The motivation is the following:
I have a environment where the docker stack is defined in an ansible play.
The docker user is also specified in the ansible playbook.
The user uid/gid is then passed in the docker-compose at the time of
running the playbook to create the docker stack.
E.g., The following docker-compose.yml template (jinja2 template) is used
by ansible to specify the inputs at the time the docker admin stack
playbook is ran:

#Requirement: Set environmental variables: USERDIR, PUID, PGID,
MYSQL_ROOT_PASSWORD, and TZ as explained in the reference.
version: "3.6"
networks:
default:
driver: bridge
net:
external: false
attachable: true
traefik-public:
external: true
services:
######### FRONTENDS ##########
Traefik Reverse Proxy
traefik:
hostname: traefik
image: traefik:latest
container_name: traefik
restart: always
domainname: {{ external_domainname }}
networks:
- default
- traefik-public
ports:
- "80:80"
- "443:443"
- "XXXX:8080"
environment:
  - PUID={{ docker_user_uid }}
  - PGID={{ docker_user_gid }}
  - TZ={{ timezone }}
  - CLOUDFLARE_EMAIL={{ cloudflare_email }}
  - CLOUDFLARE_API_KEY={{ cloudflare_apikey }}
  - CF_API_EMAIL={{ cloudflare_email }}
  - CF_API_KEY={{ cloudflare_apikey }}
volumes:
  - /var/run/docker.sock:/var/run/docker.sock:ro
  - {{ docker_container_dir }}/traefik:/etc/traefik
  - {{ docker_container_dir }}/shared:/shared
  - {{ docker_container_dir }}/traefik/certs:/certs
labels:
  - "traefik.enable=true"
  - "traefik.backend=traefik"
  - "traefik.frontend.rule=Host:traefik.{{ internal_domainname }}"
  - "traefik.port=8080"
  - "traefik.docker.network=traefik-public"
  - "traefik.frontend.headers.SSLRedirect=true"
  - "traefik.frontend.headers.STSSeconds=315360000"
  - "traefik.frontend.headers.browserXSSFilter=true"
  - "traefik.frontend.headers.contentTypeNosniff=true"
  - "traefik.frontend.headers.forceSTSHeader=true"
  - "traefik.frontend.headers.SSLHost={{ external_domainname }}"
  - "traefik.frontend.headers.STSIncludeSubdomains=true"
  - "traefik.frontend.headers.STSPreload=true"
  - "traefik.frontend.headers.frameDeny=true"

#Portainer - WebUI for Containers
portainer:
image: portainer/portainer
container_name: portainer
restart: always
command: -H unix:///var/run/docker.sock
environment:
- TZ={{ timezone }}
networks:
- traefik-public
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- {{ docker_container_dir }}/portainer/data:/data
- {{ docker_container_dir }}/shared:/shared
labels:
- "traefik.enable=true"
- "traefik.backend=portainer"
- "traefik.frontend.rule=Host:portainer.{{ internal_domainname }}"
- "traefik.port=9000"
- "traefik.docker.network=traefik-public"
- "traefik.frontend.headers.SSLRedirect=true"
- "traefik.frontend.headers.STSSeconds=315360000"
- "traefik.frontend.headers.browserXSSFilter=true"
- "traefik.frontend.headers.contentTypeNosniff=true"
- "traefik.frontend.headers.forceSTSHeader=true"
- "traefik.frontend.headers.SSLHost={{ external_domainname }}"
- "traefik.frontend.headers.STSIncludeSubdomains=true"
- "traefik.frontend.headers.STSPreload=true"
- "traefik.frontend.headers.frameDeny=true"
must have --copy-service for runtime bootstrap to work
ref: https://github.com/osixia/docker-openldap/issues/163
openldap:
container_name: "openldap"
image: {{ docker_registry }}/openldap:latest
image: {{ openldap_image }}
command: --copy-service --loglevel debug
ports:
  - 389:389
restart: "unless-stopped"
networks:
  - net
volumes:
  - {{ docker_container_dir }}/openldap/slapd/database:/var/lib/ldap
  - {{ docker_container_dir }}/openldap/slapd/config:/etc/ldap/slapd.d
  - {{ docker_container_dir

}}/openldap/slapd/certs:/container/service/slapd/assets/certs
- {{ docker_container_dir
}}/openldap/ldif:/container/service/slapd/assets/config/bootstrap/ldif/custom
## custom schema
## ref: https://github.com/osixia/docker-openldap/issues/177
- {{ docker_container_dir
}}/openldap/schema/sudo.schema:/container/service/slapd/assets/config/bootstrap/schema/sudo.schema
environment:
PUID: {{ docker_user_uid }}
PGID: {{ docker_user_gid }}
  LDAP_OPENLDAP_UID: {{ docker_user_uid }}
  LDAP_OPENLDAP_GID: {{ docker_user_gid }}
  LDAP_ORGANISATION: "{{ ldap_internal.ldap_organisation }}"
  LDAP_DOMAIN: "{{ ldap_internal.ldap_domain }}"
  LDAP_BASE_DN: "{{ ldap_internal.ldap_base_dn }}"
  SLAPD_ADDITIONAL_MODULES: "memberof"
  LDAP_ADMIN_PASSWORD: "{{ ldap_internal.ldap_admin_password }}"
  LDAP_READONLY_USER: "{{ ldap_internal.ldap_readonly_user }}"
  LDAP_READONLY_USER_USERNAME: "{{

ldap_internal.ldap_readonly_user_username }}"
LDAP_READONLY_USER_PASSWORD: "{{
ldap_internal.ldap_readonly_user_password }}"
ref: https://github.com/alterrebe/docker-mail-relay
postfix:
container_name: "postfix"
image: {{ postfix_image }}
restart: "unless-stopped"
networks:
- net
healthcheck:
test: [ "CMD", "sh", "-c", "netstat -an | fgrep 587 | fgrep -q LISTEN" ]
  test: [ "CMD", "sh", "-c", "netstat -an | fgrep 25 | fgrep -q LISTEN" ]
  interval: 10s
  timeout: 5s
  start_period: 10s
  retries: 2
ports:
  - "1025:25"
  - "1587:587"
environment:
  TZ: {{ timezone }}

INBOUND_DEBUGGING: 1
  RELAY_HOST_NAME: {{ inventory_hostname }}
  EXT_RELAY_HOST: {{ smtp_relay_host }}
  EXT_RELAY_PORT: {{ smtp_relay_port }}
  SMTP_LOGIN: {{ smtp_relay_username }}
  SMTP_PASSWORD: {{ smtp_relay_password }}
  ACCEPTED_NETWORKS: {{ smtp_relay_accepted_networks }}

ALLOWED_SENDER_DOMAINS: {{ postfix_allowed_sender_domains }}
  ROOT_ALIAS_ADDRESS: {{ root_alias_email }}
  MAILMAN_TRANSPORT: {{ mailman_transport }}
  RELAY_EXCLUDES: {{ smtp_relay_excludes }}

cobbler:
container_name: "cobbler"
image: {{ docker_registry }}/cobbler:latest
image: {{ cobbler_image }}
privileged: true
ports:
  - 69:69
  - 9080:80
  - 9443:443
  - 25151:25151
restart: "unless-stopped"
networks:

- internal
  - net
volumes:
  - /sys/fs/cgroup:/sys/fs/cgroup:ro
  - {{ docker_container_dir }}/cobbler:/etc/cobbler
  - {{ docker_container_dir

}}/cobbler/var/www/cobbler/images:/var/www/cobbler/images
- {{ docker_container_dir
}}/cobbler/var/www/cobbler/ks_mirror:/var/www/cobbler/ks_mirror
- {{ docker_container_dir
}}/cobbler/var/www/cobbler/links:/var/www/cobbler/links
- {{ docker_container_dir
}}/cobbler/var/lib/cobbler/config:/var/lib/cobbler/config
- {{ docker_container_dir }}/cobbler/var/lib/tftpboot:/var/lib/tftpboot
- {{ docker_container_dir }}/cobbler/dist/centos:/mnt:ro
environment:
PUID: {{ docker_user_uid }}
PGID: {{ docker_user_gid }}
DEFAULT_ROOT_PASSWD: cobbler
HOST_IP_ADDR: {{
hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}
HOST_HTTP_PORT: 80
HOST_HTTP_PORT: 9080
HOST_HTTP_PORT: 9443
  COBBLER_WEB_USER: cobbler
  COBBLER_WEB_PASSWD: cobbler
  COBBLER_WEB_REALM: Cobbler
  COBBLER_LANG: en_US
  COBBLER_KEYBOARD: us
  COBBLER_TZ: {{ timezone }}
  TZ: {{ timezone }}

The ansible play would look like:

BLOCK: Create docker_admin_server group and user, enable sudo access

block:

name: create {{ docker_user_group }} group
group: name={{ docker_user_group }} system=yes state=present

name: Create docker user to run admin stack containers
user:
name: "{{ docker_user_username }}"
password: "{{ docker_user_password | password_hash('sha512') }}"
group: "{{ docker_user_group }}"
update_password: on_create
state: present
createhome: yes
shell: "{{ docker_user_shell }}"
groups: adm,cdrom,dip,plugdev,docker
append: yes
register: docker_user_verify_user

#- debug:
var: docker_user_verify_user
verbosity: 1

name: Set Fact "docker_user_uid"
set_fact: docker_user_uid={{ docker_user_verify_user.uid }}

name: Set Fact "docker_user_gid"
set_fact: docker_user_gid={{ docker_user_verify_user.group }}

name: Set Fact "docker_user_uid"
set_fact: docker_user_home={{ docker_user_verify_user.home }}

name: Enable sudo for admin user
user:
name={{ docker_user_username }}
groups=sudo
append=yes
when: docker_user_sudo_access | bool

Create admin_server directories and configs

name: Ensure admin dirs exist
file:
path: "{{ item.path }}"
state: directory
owner: "{{ item.owner | default( docker_user_username ) }}"
group: "{{ item.owner | default( docker_user_group ) }}"
mode: "{{ item.mode | default(omit) }}"
recurse: "{{ item.recurse | default( omit ) }}"
with_items:

{ path: "{{ script_dir }}", owner: "root", group: "root", recurse: yes  }
{ path: "{{ fwbackups_dir }}", owner: "root", group: "root",
recurse: yes }
{ path: "{{ backups_dir }}", owner: "root", group: "root", recurse: yes }
{ path: "{{ backups_dir }}/records/daily", owner: "root", group: "root" }
{ path: "{{ backups_dir }}/records/monthly", owner: "root",
group: "root" }
{ path: "{{ backups_log_dir }}", owner: "root", group: "root" }
{ path: "{{ docker_container_dir }}/traefik" }
{ path: "{{ docker_container_dir }}/traefik/rules" }
{ path: "{{ docker_container_dir }}/traefik/certs" }
{ path: "{{ docker_container_dir }}/traefik/acme" }
{ path: "{{ docker_container_dir }}/shared" }
{ path: "{{ docker_container_dir }}/portainer" }
{ path: "{{ docker_container_dir }}/postfix" }
{ path: "{{ docker_container_dir }}/openldap" }
{ path: "{{ docker_container_dir }}/openldap/certs" }
{ path: "{{ docker_container_dir }}/openldap/ldif" }
{ path: "{{ docker_container_dir }}/openldap/schema" }
{ path: "{{ docker_container_dir }}/openldap/slapd" }
{ path: "{{ docker_container_dir }}/openldap/slapd/database" }
{ path: "{{ docker_container_dir }}/openldap/slapd/config" }
{ path: "{{ docker_container_dir }}/cobbler" }
{ path: "{{ docker_container_dir }}/cobbler/etc" }
{ path: "{{ docker_container_dir }}/cobbler/var/www/cobbler/images" }
{ path: "{{ docker_container_dir }}/cobbler/var/www/cobbler/ks_mirror" }
{ path: "{{ docker_container_dir }}/cobbler/var/www/cobbler/links" }
{ path: "{{ docker_container_dir }}/cobbler/var/lib/cobbler/config" }
{ path: "{{ docker_container_dir }}/cobbler/var/lib/tftpboot" }
{ path: "{{ docker_container_dir }}/cobbler/dist/centos" }

name: Install admin_server app config settings
copy:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
mode: "{{ item.mode | default('0664') }}"
owner: "{{ item.owner | default( docker_user_username ) }}"
group: "{{ item.owner | default( docker_user_group ) }}"
with_items:

{src: 'ldap/sudo.schema', dest: "{{ docker_container_dir
}}/openldap/schema/sudo.schema"}
{src: 'ldap/johnson.local.ldif', dest: "{{ docker_container_dir
}}/openldap/ldif/bootstrap.ldif"}
{src: 'ldap/samba_indices.ldif', dest: "{{ docker_container_dir
}}/openldap/ldif/samba_indices.ldif"}
{src: 'scripts/backups/job-backup-incremental.sh', dest: "{{
script_dir }}", mode: "0770", owner: "root", group: "root"}
{src: 'scripts/backups/rsync-incremental-backup-local', dest:
"{{ script_dir }}", mode: "0770", owner: "root", group: "root"}
{src: 'scripts/backups/fwbackup.sh', dest: "{{ script_dir }}",
mode: "0770", owner: "root", group: "root"}
{src: 'scripts/docker/docker-utils.sh', dest: "{{
docker_container_dir }}", mode: "0770"}
{src: 'scripts/reset-openldap.sh', dest: "{{
docker_container_dir }}", mode: "0770"}

name: Install admin_server docker config settings
template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
mode: "{{ item.mode | default('0664') }}"
owner: "{{ docker_user_username }}"
group: "{{ docker_user_group }}"
with_items:

{src: 'config-traefik.toml.j2', dest: "{{ docker_container_dir
}}/traefik/traefik.toml"}
{src: 'traefik/rules/nginx.toml.j2', dest: "{{
docker_container_dir }}/traefik/rules/nginx.toml"}
{src: 'postfix/master.conf.j2', dest: "{{ docker_container_dir
}}/postfix/master.cf"}
{src: 'postfix/main.conf.j2', dest: "{{ docker_container_dir
}}/postfix/main.cf"}
{src: 'config-smbcredentials.conf.j2', dest:
"/root/.smbcredentials", mode: "0600" }
{src: 'config-fwbackup.conf.j2', dest: "/root/.fwbackup.cfg",
mode: "0600" }
{src: 'compose.env.j2', dest: "{{ docker_container_dir }}/.env" }
{src: 'docker-compose.yml.j2', dest: "{{ docker_container_dir
}}/docker-compose.yml" }

name: Ensure admin mount dirs exist
file:
path: "{{ item.name }}"
state: directory
with_items: "{{ admin_mounts }}"

name: Add admin mounts to fstab
mount:
name: "{{ item.name }}"
src: "{{ item.src }}"
fstype: "{{ item.fstype }}"
opts: "{{ item.options }}"
state: mounted
with_items: "{{ admin_mounts }}"

create acme.json for letsencrypt

name: ensure acme.json file exists
copy:
content: ""
dest: "{{ docker_container_dir }}/traefik/acme/acme.json"
force: "{{ overwrite_configs }}"
owner: "{{ docker_user_username }}"
group: "{{ docker_user_group }}"
mode: 0600

for some reason copy above does not enforce correct chmod perms
set correct perms on acme.json

name: ensure acme.json has correct perms
file:
path: "{{ docker_container_dir }}/traefik/acme/acme.json"
owner: "{{ docker_user_username }}"
group: "{{ docker_user_group }}"
mode: 0600

BLOCK: run docker-compose to setup admin_server services

block:

name: Init a new swarm with default parameters
docker_swarm:
state: present

name: Create docker networks
docker_network:
name: "{{ item }}"
with_items:

- internal
  - net
  - traefik-public

run docker compose
ref: https://stackoverflow.com/questions/44962282/how-to-write-an-ansible-playbook-with-docker-compose

name: run the services defined in docker-compose.yml
docker_compose:
project_src: "{{ docker_container_dir }}"
remove_orphans: "{{ docker_remove_orphans }}"

BLOCK: Configure firewalld

block:

name: Installing Firewall Packages
package:
name: "{{ firewalld_packages }}"
state: present

state: latest
update_cache: yes

name: Allow admin services through the firewall
firewalld:
service: "{{ item }}"
permanent: yes
state: enabled
immediate: yes
with_items:
"{{ exposed_services }}"
notify:

reload firewalld

name: Allow admin_server ports through the firewall
firewalld:
port: "{{ item }}"
zone: internal
permanent: true
state: enabled
immediate: yes
with_items:
"{{ exposed_ports }}"
notify:

reload firewalld

ref: https://forums.docker.com/t/no-route-to-host-network-request-from-container-to-host-ip-port-published-from-other-container/39063/5

name: Add docker networks to firewall
firewalld:
source: <IP_ADDRESS>/8
permanent: true
zone: internal
state: enabled
immediate: yes
notify:

reload firewalld

tags:
- firewall
I hope this helps in understanding the motivation.
On Thu, Aug 1, 2019 at 8:36 AM Bertrand<EMAIL_ADDRESS>wrote:

Hello,
thanks for the PR, can you describe the problem solved by this ?
Thanks
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/osixia/docker-openldap/pull/336?email_source=notifications&email_token=AAP3JD3M36WM7YHDCBZ4SATQCLKE3A5CNFSM4IEQ3GF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3KODEI#issuecomment-517267857,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAP3JD2FK7BKKY23YWOZDFTQCLKE3ANCNFSM4IEQ3GFQ
.

This is very usual practise - e.g. all LinuxServer/ images have this setting, and others. Usually this is ony needed when you want to use host-mounts, not named volumes, and thus need to align with the host uid/gid for other tooling or e.g. shared volume / smb / nfs permissions.

Thanks for you feedbacks. This sounds good to me.
groupadd -g 911 -r openldap
useradd -u 911 -r -g openldap
does this can break existing configurations ?
May the user adjusments done in entrypoint.sh would fit better in startup.sh on container first start section ?

Hi Bertrand,
To tell you the truth, I wasn't sure where the startup.sh was ultimately getting called from and did not want to adversely impact any of the existing code.
So - in order to minimize the impact, I researched the least invasive approach available for this which seemed to me the approach taken by the jenkins docker implementation for the related feature of defining the user/group id at startup used here:  https://github.com/sudo-bmitch/jenkins-docker/blob/master/entrypoint.sh
That said, if the startup.sh is simply getting kicked off at container start, then I imagine it should work fine with no adverse impact.  However, to formalize this, I would imagine that this would require testing with the existing configurations to make sure that is the case.
Is there test automation for each of the configurations available to run this change through to see that it does not break anything?

Thanks Lee,
startup.sh is called when the container start.
i guess group and user adjustments would fit great just before
https://github.com/osixia/docker-openldap/blob/2a03b392b019adbf810f11bd64a2fed753d77b9d/image/service/slapd/startup.sh#L45
tests are run via travis, but you can run them locally with make test.
Make sure to have bats installed before :
https://github.com/bats-core/bats-core

Hi Bertrand,
As requested, I moved the logic from entrypoint.sh into the startup.sh and tested using successfully.
administrator@admin2:~/repos/docker/docker-openldap-1$ docker container prune -f
Deleted Containers:
1afff9f0e9fd2fa8bcdbf823eaefe5b8a63159cc312393705d61374b6db5504a
db2bcd843030125852d83e12d62862c0b5aa911f85c4372c2a5a299d88be575d

Total reclaimed space: 2.589MB
administrator@admin2:~/repos/docker/docker-openldap-1$ docker volume prune -f
Total reclaimed space: 0B
administrator@admin2:~/repos/docker/docker-openldap-1$ docker rmi osixia/openldap:1.2.4
Untagged: osixia/openldap:1.2.4
Deleted: sha256:5382adc0ab451abaaf9cca27a924b818bf7453ee83ff985c405254beb4051922
Deleted: sha256:88c74a283ea9c835c95645a94d9bcb802599813568a9d4ada068cae2a7c03753
Deleted: sha256:c10c04aa1b6bb8e60413bf3aa593a2d44c56b33438eb58c2edf71061e57b527a
Deleted: sha256:512ae00d3dee67cccae921b9255c43e115085a80399e9a205c7a505b68967450
Deleted: sha256:70d8f269ebfe3e7bcbc8b04846778fa256fe5f15fde25b7c1f40759d5726200f
Deleted: sha256:4fb1cbdf6b7ccd19d27635997ce07d928fc784aa54de08de907ec1e0351bf0cc
Deleted: sha256:dfe96b03a7bd267d44822690adb3fd7945bc2664f7a4db8263ccaa9d436960a3
administrator@admin2:~/repos/docker/docker-openldap-1$
administrator@admin2:~/repos/docker/docker-openldap-1$ make test
env NAME=osixia/openldap VERSION=1.2.4 bats test/test.bats
 ✓ image build
 ✓ ldapsearch new database
 ✓ ldapsearch database from created volumes
 ✓ ldapsearch new database with strict TLS
 ✓ ldapsearch new database with strict TLS and custom ca/crt
 ✓ ldapsearch new database with strict TLS and custom ca/crt and custom dhparam
 ✓ ldapsearch existing hdb database and config
 ✓ replication with new databases and strict TLS

8 tests, 0 failures

administrator@admin2:~/repos/docker/docker-openldap-1$

HI! Any news on this PR? It looks promising!

HI! Any news on this PR? It looks promising! (missing documentation though)

Hi - I just added a brief description for the new env vars in the environments section of the doc here:
https://github.com/osixia/docker-openldap/pull/336/commits/c6844f624299cf6c23bdc0a92b4f0dd9cf4664b6

Thanks a lot !


Chrome Around Graph Explorer doesn't respect System Theme.

Describe the bug
Graph Explorer dark them gets set quite nicely based on Windows theme which is pretty cool!!!. However, it seems that the chrome around Graph Explorer doesn't respect the System Theme (Dark theme) and breaks the experience.
To Reproduce
Steps to reproduce the behavior:
Set OS theme to dark mode.
Graph Explorer adapts to the theme accordingly, however the chrome around stays white.
Expected behavior
I expected that when the theme changes, all components in the window would change accordingly.
Screenshots

Desktop (please complete the following information):

OS: Windows 11 Insider
Browser: Edge
Version: latest

Has external dependency


Release of <IP_ADDRESS>

Greetings and thx for the great lib :slightly_smiling_face:
I was wondering when will the v<IP_ADDRESS> be available on maven central? It includes the fix for #775 and kc:3.7 is default for spring boot 3.3+

We are preparing release right now - just ironing out some kinks in the release pipeline after CI migration.
Should be out this week.


Numeric-string labels do not display correctly

var data = {
    labels: ['A', 'B', 'C', 'D', 'E', 'F', 'G'] ...

works, but:
var data = {
    labels: ['1', '2', '3', '4', '5', '6', '7'] ...

shows up as "rgb(0,0,0)" in the tooltip

@MrAndMrsK this is already fixed in the v2.0 codeline but you'd need to rebuild yourself to get it.
To build the source code. The npm install is necessary to fix this one since the issue actually came from one of the dependencies pulled in to the lib.
npm install
gulp build


[SPARK-37750][SQL] ANSI mode: optionally return null result if element not exists in array/map

What changes were proposed in this pull request?

Add a new configuration spark.sql.ansi.failOnElementNotExists which controls whether throwing exceptions or returning null results when element not exists in the [] operator in array/map type
The default value of the new configuration is true.
Why are the changes needed?

Provide an alternative way for Spark SQL users who replies on null results when element not exists in array/map, e.g. select .. where array[index] is not null or select .. where map[key] is not null
Map type is not part of the ANSI SQL type. There can be arguments that map[key] should return null if key not exist.

Does this PR introduce any user-facing change?

Yes, providing a new option spark.sql.ansi.failOnElementNotExists which can optionally return null result if element not exists in array/map. However, the default behavior is not changed.
How was this patch tested?

Unit tests

cc @cloud-fan @entong @allisonwang-db

shall we limit it to [] operator only? For functions, I think a better idea is to add try_xxx version instead of a config.

shall we limit it to [] operator only? For functions, I think a better idea is to add try_xxx version instead of a config.

Do you mean the scope of the configuration?
The behavior of [] and element_at are so similar except for the array index. It is odd to control [] only. Also, it is also hard to explain the idea to users.
Shall we just change the error message for [] and element_at?

The behavior of [] and element_at are so similar

They are very different. element_at uses 1-based index and support negative index. It's also confusing for end-users that element_at has a config to control the error behavior and also a try_element_at.
I think it's easier to explain if this config only applies to [] operator, instead of a lof of places and list them in the doc.

@cloud-fan I have updated the code as per your suggestion.

merging to master

I think failOnNonexistentElement would be a better name since, in English, on should not be followed by a phrase containing verb (exists).
If you agree, I can send out a PR.

@gengliangwang you forgot to update the PR description...

@tedyu @cloud-fan sorry I forgot to update the configuration name in the PR description. It should be spark.sql.ansi.strictIndexOperator


NewValidation: CSCwh68103: Enhance PD for sanitizing the tDN of class infraRsHPathAtt and fabricRsOosPath

(use upvote :thumbsup: for attentions)
Validation Type
[ ] - Fault
[X] - Config
[ ] - Bug
[ ] - Other
What needs to be validated
There are multiple scenarios described in the defect that can lead to failure of config deployment on upgrade to version with fix for CSCwh68103.

Wrong class ID, for example user enter uni/infra/funcprof/accbundle-legacy as target
Malformed port like Eth1/1 (capital letter), eth/1, eth1//4
FEX fabricPathEP should be topology/pod-1/paths-102/extpaths-133/pathep-[eth1/1] instead of topology/pod-1/paths-102/pathep-[eth133/1/1]
Wrong fex id -> it should be greater than 101
Leaf Interface profile contains breakout port that exceeds 16 (Example: 1/1/17 or 1/1/50)

Why it needs to be validated
Such misconfiguration on the source version will result in policy deployment failure on the destination. This includes VPC configuration failure and outage.
Additional context
The validation should detect any of the 5 scenarios, identify the problematic configuration for the user, and recommend the method of resolution depending on impacting scenario.

A new scenario was recently discovered where a customer configured etherbnet interfaces starting from 0.
dn           : uni/infra/hpaths-1_eth1_0/rsHPathAtt-[topology/pod-1/paths-1/pathep-[eth1/0]]
dn           : uni/infra/hpaths-2_eth1_0/rsHPathAtt-[topology/pod-2/paths-2/pathep-[eth1/0]]
dn           : uni/infra/hpaths-3_eth1_0/rsHPathAtt-[topology/pod-2/paths-3/pathep-[eth1/0]]
dn           : uni/infra/hpaths-4_eth1_0/rsHPathAtt-[topology/pod-1/paths-4/pathep-[eth1/0]]
dn           : uni/infra/hpaths-5_eth1_0/rsHPathAtt-[topology/pod-1/paths-5/pathep-[eth1/0]]

After the upgrade to 5.3(2c), it triggered the following error:
"Error: 400 - Failed to decode IfIndex, id: 0x1ffff000, error: 1".


Verify that fuzz targets are not hampered by checksums

Fuzzing lewton goes through the "vorbis inside ogg" codepath, which verifies CRC32 checksum on the input. This seems to prevent any kind of meaningful fuzzing.
I have disabled CRC32 checks in ogg crate during fuzzing (using conditional compilation as described in honggfuzz-rs readme) and immediately got panics on out-of-bounds access to a slice. It seems that because of CRC32 lewton was never really fuzzed.
Steps to reproduce the crash: git clone https://github.com/Shnatsel/lewton-fuzz,
download files http://rpg.hamsterrepublic.com/ohrrpgce/File:Slash8-Bit.ogg and https://commons.wikimedia.org/wiki/File:Example.ogg put them in honggfuzz input directory, run
cargo hfuzz run lewton-fuzz
I had to separate the lewton fuzz target into its own repo because I'm on stable and this repo requires nightly to build. The Cargo.toml files in its repo is repointed to the patched lewton crate that refers to patched ogg crate that doesn't perform CRC32 check in fuzzing mode. I also had to drop all .unwrap()s in the fuzz target code because they were causing panics that were false positives (duh). I'm not sure why they were there in the first place.
This problem with checksums is likely not exclusive to lewton; most multimedia formats employ checksums of some description, and this prevents unmodified libraries from being fuzzed. This needs to be verified for every fuzz target.

I have removed checksum verification in 3 crates via conditional compilation:
https://github.com/Shnatsel/ogg/commit/818731bcf8ebb99537fe36459f5621964bf4eba6
https://github.com/Shnatsel/image-png/commit/1a5c7866eb3d017dcea40ec6f4200bcb89d90c50
https://github.com/Shnatsel/lodepng-rust/commit/22c87dcd99ffbbe78cb85dc9e7f5d90350901892
After that fuzzers have immediately uncovered bugs in these libraries.
Is this the right approach to fuzzing formats with checksums? If so, I will open pull requests with these changes to the crates themselves.
Also, checksums should be mentioned in fuzzing documentation.

@Shnatsel I've really enjoyed your post on reddit. Do you want to continue to work on this? I'd love to add you to the rust-fuzz orga!

I will be preoccupied by other things in the coming month, but after that - sure, why not.

Also, I've verified that all the fuzz targets in the repo are not hampered by checksums (as far as format specifications for them go, anyway; I didn't read all of the code).
This just leaves documenting that checksums should be conditionally disabled in quickstart guides.


Fix quantifiedcode issues.

Coverage remained the same at 100.0% when pulling df0b799a0e560588024058b3f813bd1687a69a76 on fixMinorIssues into 5fbb7bc74ae6d30e7579c4b964874a6b6f1fe667 on master.

Changes Unknown when pulling 332a063feb371d289a0178f610ffde153aceb683 on fixMinorIssues into ** on master**.

Coverage remained the same at 100.0% when pulling 332a063feb371d289a0178f610ffde153aceb683 on fixMinorIssues into f9eca7481ca637ac5af5a0f19704e85b4191a37e on master.


When the application being profiled is finished it is impossible to recompile it as its pdb is locked.

The profiler's window locks application pdb. To proceed with compilation one
must close the window.
The expected behavior is that the user shall be able to recompile the
application even, if profiler's window is open.

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 10 Apr 2012 at 11:04

Duplicate of a defect tracked.

Hi!
I have made a small release, that caches symbols upon application exit and releases PDBs. This does not handle abnormal application termination (or terminate from debugger). This will be fixed later.
Please find the release here: https://github.com/tyoma/micro-profiler/releases/tag/v1.4.604.0
Thank you!

I had the opportunity to try it today and it worked as expected. Thanks a lot !

Thanks!
I'll leave this opened until the underlying functionality is made asynchronous (for .pdb >100MB it takes significant time to read the symbols, and now it just freezes the UI).

In 1.5 the debug info is requested asynchronously and then released, so that the application can be rebuilt immediately after profiling session.


Add basic docker integration

As requested, the dockerfile from my build.
I added the Dockerfile in the root since the other build files reside there as well, and docker does not like relative paths that move up the hierarchy... so only the resources are in the docker subfolder.
There are 3 entries there:

the Readme describing the options you need to launch the container
the hibernate template with all the placeholders present
the startup script that does the replacing. It's the v2 of my startup script that also allows the host/port to be changed (otherwise, you get redirects back to localhost when you use a public hostname).

We could discuss if this is the right approach, my aim was to provide the basic things as parameters... if you want full control, you can simply take this as a base and overwrite with your own hibernate/properties file without too much hassle.

It seems that another docker configuration was merged a few days ago in 8c4f7b15fc07d61cc439d44d37b4e62fdcb2e95b (the original PR was #205).
Could you please check if this covers all your needs?

@kullervo16 you did a great work, thanks!
But as @xdarklight mentioned, #205 did a awesome docker setup by leveraging docker-compose, so I'm closing this PR, ok?


Compare to certlm.msc and certmgr.msc

How is following these steps different than just running certlm.msc or certmgr.msc?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f950f479-532b-ab38-a676-653495777197
Version Independent ID: 30fa2c1a-86bd-e683-3e6f-8a268f4dc4e5
Content: How to: View Certificates with the MMC Snap-in
Content Source: docs/framework/wcf/feature-details/how-to-view-certificates-with-the-mmc-snap-in.md
Product: dotnet-framework
GitHub Login: @dotnet-bot
Microsoft Alias: dotnetcontent

@HappyNomad I'm going to close this one and keep #7586 open. That article needs to be rewritten with and also needs to show certlm.msc or certmgr.msc


FFmpeg-run Error

Keep getting an error every time I try to train a model using the WAV files I downloaded to use for the dataset.
start preprocess
['trainset_preprocess_pipeline_print.py', 'C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0\datasets\', '48000', '8', 'C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0/logs/dominic-fike-1', 'False']
C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0\datasets/DominicFike->Traceback (most recent call last):
File "C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0\my_utils.py", line 96, in load_audio
ffmpeg.input(file, threads=0)
File "C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0\runtime\lib\site-packages\ffmpeg_run.py", line 325, in run
raise Error('ffmpeg', out, err)
ffmpeg._run.Error: ffmpeg error (see stderr output for detail)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0\trainset_preprocess_pipeline_print.py", line 84, in pipeline
audio = load_audio(path, self.sr, DoFormant, Quefrency, Timbre)
File "C:\Users\mwaka\OneDrive\Desktop\RVCV2\Mangio-RVC-v23.7.0_INFER_TRAIN\Mangio-RVC-v23.7.0\my_utils.py", line 103, in load_audio
raise RuntimeError(f"Failed to load audio: {e}")
RuntimeError: Failed to load audio: ffmpeg error (see stderr output for detail)
end preprocess

Same error for me as well. Did you resolve it?

Same error for me as well. Did you resolve it?

I just gave up on training using Mangio and used a diff training site for my voice models instead.


CORE-5331: Use a single entity manager factory across all of DB message bus

After talking with @driessamyn I also made the auto topic creation use a pessimistic write lock rather than worry about catching various exceptions.  This should be a better way to avoid duplicate topic creations.


A tag can not include TABLE tag.

Action:
Nokogiri::HTML::DocumentFragment.parse('

').to_html

Output (Expected):

Output (Actual):

Mb cuz  is an inline element, however  is a block-level element and result is the same

Hi @tashua314,
Thanks for asking this question.
Neither the  nor the
 elements can contain block-level elements as children. See https://www.w3.org/TR/html4/struct/links.html#edef-A and https://www.w3.org/TR/html4/struct/text.html#edef-P to see that these are both inline elements.
To wit:
The P element represents a paragraph. It cannot contain block-level elements (including P itself).

I think you'll find that if you change  to a block-level element like
 that this works as expected.
The behavior you're seeing is libxml2 attempting to correct invalid markup (much as browsers do) by following some simple rules. In this case, because a table can't be contained in an a tag, the a tag is closed before the table is opened.
Hope this helps!

Thank you for the reply, @flavorjones .
I have another question.
HTML5 seems to be allowed to add block-level elements to  tags.
Is this excluded?
https://www.w3.org/TR/html5/text-level-semantics.html#the-a-element

Sorry, Nokogiri's parser doesn't support HTML5. You can try the Nokogumbo
project, which extends Nokogiri to support HTML5.
On May 10, 2017 2:32 AM, "yuta takahashi"<EMAIL_ADDRESS>wrote:

Thank you for the reply, @flavorjones https://github.com/flavorjones .
I have another question.
HTML5 seems to be allowed to add block-level elements to tags.
Is this excluded?
https://www.w3.org/TR/html5/text-level-semantics.html#the-a-element
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sparklemotion/nokogiri/issues/1638#issuecomment-300388746,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAgDwBS3qTOADO8y1zL7tX1qk6VEKQbks5r4VoQgaJpZM4NUuNY
.


[IOTDB-6239] Show regions display error create time

Jira: https://issues.apache.org/jira/browse/IOTDB-6239
Bug fix test: the create time is correct in this PR.

Compatibility test: the ConfigNode is still compatible with earlier version, through the create time of Region can't be fixed to correct.

Codecov Report

Merging #11481 (7d99db6) into master (1df8ec2) will increase coverage by 0.00%.
Report is 1 commits behind head on master.
The diff coverage is 67.53%.

@@            Coverage Diff            @@
##             master   #11481   +/-   ##
=========================================
  Coverage     49.02%   49.03%
- Complexity    24598    24603    +5
=========================================
  Files          2772     2772
  Lines        172995   173059   +64
  Branches      20761    20769    +8
=========================================
+ Hits          84818    84854   +36
- Misses        88177    88205   +28

Files
Coverage Δ

.../persistence/partition/DatabasePartitionTable.java
39.80% <100.00%> (+0.30%)
:arrow_up:

...onfignode/persistence/partition/PartitionInfo.java
28.86% <100.00%> (+0.36%)
:arrow_up:

...ain/java/org/apache/iotdb/db/conf/IoTDBConfig.java
78.07% <ø> (-0.04%)
:arrow_down:

...rageengine/dataregion/flush/MemTableFlushTask.java
83.49% <100.00%> (ø)

...torageengine/dataregion/tsfile/TsFileResource.java
68.73% <100.00%> (ø)

.../java/org/apache/iotdb/db/utils/DateTimeUtils.java
83.85% <ø> (-0.25%)
:arrow_down:

...va/org/apache/iotdb/commons/conf/CommonConfig.java
89.03% <100.00%> (+0.07%)
:arrow_up:

...pache/iotdb/commons/utils/CommonDateTimeUtils.java
40.00% <100.00%> (+40.00%)
:arrow_up:

...in/java/org/apache/iotdb/db/audit/AuditLogger.java
0.00% <0.00%> (ø)

...nfignode/manager/load/balancer/RegionBalancer.java
6.52% <0.00%> (-0.15%)
:arrow_down:

... and 5 more

... and 22 files with indirect coverage changes
:mega: Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!


Add setup.cfg + update travis

Set universal on so that wheels are tagged py2.py3

Bah - I need to learn how to test travis better. Sorry about all the spam!


Make a "ding" sound when a tx confirms

One thing that has connected NEM community members throughout the years is that annoying "ding" sound when txs confirm, but it really grows on people and they really love it after a while. It really just works. It is also a great notifier that the tx has confirmed and now you can move on to the next step, whatever that is, in your operation. It makes things more efficient. The same file can easily be found in the NanoWallet Github.

Thanks @jabo38 . Closing it as it's Duplicate of #1362


Elgato Eve - Extreme Brightening Lag & On to 1%

I installed OpenHAB with the InsteonPLM binding and have a number of Insteon dimmer items.  Everything is very responsive through the OpenHAB web interface and through the OpenHAB iOS app.  I installed OpenHAB-HomeKit-Bridge without any issues.  I attached the bridge to Elgato Eve on iOS.  All of the items pulled in and are controllable through the app.  There are no issues with Siri interactions like "Turn ON/OFF" or "Set to 50%".
The issue:
Dimmer items in Eve are exhibiting two issues.

Tapping ON sets them to 1%
The brightness slider has extreme lag.  If I swipe the slider quickly from 0% to 100%, the slider and its corresponding light will brighten at random increments from 0% to 100% over the course of a minute; as if the slider is sending tens or hundreds of requests to "set to 1%, set to 2%, set to 5%" etc.

Is anyone else experiencing these two issues?

Same behavior here for devices bound with the InsteonPLM binding, though I'm using the "Home" app on iOS 10, build 14A5297c instead of Elgato Eve.

Removing "switchSupport" from the item in the sitemap improves the experience for me. Tapping ON will set the dimmer to 100%, though the readout on the "Home" app shows it bouncing back to 1% before retuning and remaining at 100%.

Thanks @BrianTillman for the input. If this is still an issue please reopen.


current property conflicts with method current()

It is not possible to call the current() method of DocumentClient. If you try to do that an exception is thrown saying that current is not a method. The current () method should be renamed e.g. to currentElement()

There is no current() method in DocumentClient, do you mean the queryIterator returned by the query or readFeed method ?

Sorry. I mean the current() method of QueryIterator. This class also has a property current, which stores the index of the current object in the resource array.

Makes sense, thanks for notifying us about the bug, we will get it fixed as soon as possible

This has now been fixed.


allowUnfree is broken

Describe the bug
After https://github.com/NixOS/nixpkgs/commit/1c49b81263858c69b932da05ae63a7767b308e74 setting nixpkgs.config.allowUnfree = true has no effect.
(Probably same for allowBroken and allowUnsupportedSystem, didn't check)
(I guess it still works when set as explicit nixpkgs import attribute?  Didn't check either)
Steps To Reproduce
Steps to reproduce the behavior:

Have a system/home config that sets nixpkgs.config.allowUnfree in a module (not as nixpkgs import attribute)
Move to nixpkgs past https://github.com/NixOS/nixpkgs/commit/1c49b81263858c69b932da05ae63a7767b308e74
Watch you config break

Additional context
My config is a flake, could that be somehow relevant?
Notify maintainers
@Artturin
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 5.17.5, NixOS, 22.05 (Quokka), 22.05.20220506.e23ece8`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.0`
 - channels(root): `""`
 - channels(cmm): `""`
 - nixpkgs: `/home/cmm/.config/home/input/nixpkgs`

Thanks for missing allowUnfreePredicate, so that one still works :)

Yeah, it's actually specific to my home-config (system config's fine), so probably this: https://github.com/nix-community/home-manager/issues/2942
Sorry for the noise!

Thanks for missing allowUnfreePredicate, so that one still works :)

For someone encountering this same issue, a workaround is to add this your nixpkgs config:
nixpkgs.config.allowUnfreePredicate = (pkg: true);
This will allow any unfree package to be installed.

It looks like the change also caused issues with flake-utils-plus, this fails now:
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/1c49b81263858c69b932da05ae63a7767b308e74";
    utils.url = "github:gytis-ivaskevicius/flake-utils-plus/v1.3.1";
  };

  outputs = inputs@{ self, utils, ... }: utils.lib.mkFlake {
    inherit self inputs;

    hosts.default.modules = [
      ({ pkgs, ... }: {
        environment.systemPackages = [ pkgs.discord ];
        nixpkgs.config.allowUnfree = true;
      })
    ];
  };
}

This issue is still relevant for me

nixos or home-manager? please provide a reproducer

This should be working as designed on the NixOS/nixpkgs side, and even the HM bug was fixed in a prior release.
If there's still an issue with the behaviour, please provide an example to reproduce and file a new issue (if an issue does not already exist for the topic).


Reason:

In func xx_sigdeliver() restore stage, when saved_irqcount == 0, that means
rtcb NOT in critical_section before switch to xx_sigdeliver(), then we need
reset the critical_section state before swith back.

Fix:

Add condition to cover saved_irqcount == 0.

Impact
Testing

@GUIDINGLI
I can reproduce the bug with sabre6quad:smp and confirmed that this PR fixes the problem.
Could you please apply the same logic to armv6-m/arm_sigdeliver.c and amend the commit?


Huge deadlocks count on the last phase operations ALTER INDEX RENAME TO

Hello, Maxim!
Thank you for very useful tool. We have deadlock errors in postgresql logs related to last phase of pgcompacttable - "REINDEX".
For example:
2020-10-31 21:05:37 MSK [21303] <IP_ADDRESS> PostgreSQL JDBC Driver queue2@queue2 40P01 UPDATE ERROR:  deadlock detected
2020-10-31 21:05:37 MSK [21303] <IP_ADDRESS> PostgreSQL JDBC Driver queue2@queue2 40P01 UPDATE DETAIL:  Process 21303 waits for RowExclusiveLock on relation 38747785 of database 16619; blocked by process 20639.
Process 20639 waits for AccessExclusiveLock on relation 39109132 of database 16619; blocked by process 21303.
Process 21303: UPDATE public.queue_message2 as x SET sended_at = now(), state='SENDED'
WHERE id = (
SELECT id
FROM public.queue_message2
WHERE queue_id = $1
AND state = 'QUEUED'
AND router_tag = $2
AND expired_at > now()
ORDER BY queue_id, state, router_tag,
priority DESC, queued_at ASC FOR UPDATE SKIP LOCKED LIMIT 1)
RETURNING x.*
Process 20639:
ALTER INDEX "public".pgcompact_index_20528 RENAME TO "pgcompact_index_14090";
Can there be a positive effect from using "Options controlling the behaviour" like --reindex-*** or any else?
May be --print-reindex-queries will help in such situation to perform manual reindex later?
Problem arising only on hule DML-loaded database shardes.

2020-10-31 21:05:28 MSK [20639] [local] [unknown] postgres@queue2 57014 ALTER INDEX ERROR:  canceling statement due to statement timeout
2020-10-31 21:05:28 MSK [20639] [local] [unknown] postgres@queue2 57014 ALTER INDEX STATEMENT:
ALTER INDEX "public".pgcompact_index_20528 RENAME TO "idx_queue_message2_main";

I've set execution hourly by crontab:
/u01/postgres/pgcompacttable-master/bin/pgcompacttable --dbname queue2 --verbose info --reindex-retry-count 5 --reindex-retry-pause 10 --reindex-lock-timeout 5000 -f
and still having errors:
[postgres@ ~]$ grep -c ERR /u01/postgres/main/log/postgresql-2020-11-01.log
26
[local]:5432 postgres@queue2=# \d+ queue_message2
Table "public.queue_message2"
...
Indexes:
"pk_queue_message1" PRIMARY KEY, btree (id)
"pgcompact_index_28802" UNIQUE, btree (id)
"idx_queue_message1_msg_id" btree (message_id)
"idx_queue_message2_cnt" btree (expired_at, queue_id) WHERE state = 'QUEUED'::text
"idx_queue_message2_content" btree (queue_id, state, router_tag, content_type, priority DESC, queued_at, expired_at, id)
"idx_queue_message2_expired" btree (state, expired_at, sended_at, id)
"idx_queue_message2_main" btree (queue_id, state, router_tag, priority DESC, queued_at, expired_at, id)
"idx_queue_message2_message_id" btree (queue_id, message_id) WHERE state = 'SENDED'::text
"idx_queue_message2_mt" btree (queue_id, state, router_tag, message_type, priority DESC, queued_at, expired_at, id)
"pgcompact_index_14090" btree (queue_id, state, router_tag, priority DESC, queued_at, expired_at, id)
"pgcompact_index_4300" btree (queue_id, state, router_tag, priority DESC, queued_at, expired_at, id)
"pgcompact_index_4803" btree (queue_id, state, router_tag, priority DESC, queued_at, expired_at, id)
"queue_message2_qualified_content_name_idx" btree (qualified_content_name)
It seems strange that pgcompact_index**** not removed and all of them + their "master" index (idx_queue_message2_main) lead to deadlock errors by ALTER INDEX RENAME TO operations.
[local]:5432 postgres@queue2=# select * from pg_stat_user_indexes where indexrelname='idx_queue_message2_main';
-[ RECORD 1 ]-+------------------------
relid         | 16632
indexrelid    | 39485407
schemaname    | public
relname       | queue_message2
indexrelname  | idx_queue_message2_main
idx_scan      | 1652
idx_tup_read  | 3945
idx_tup_fetch | 78

[local]:5432 postgres@queue2=# select * from pg_stat_user_indexes where relname='queue_message2';
relid | indexrelid | schemaname |    relname     |               indexrelname                | idx_scan | idx_tup_read | idx_tup_fetch
-------+------------+------------+----------------+-------------------------------------------+----------+--------------+---------------
16632 |   39485382 | public     | queue_message2 | idx_queue_message2_message_id             |       25 |       626576 |             0
16632 |   39485389 | public     | queue_message2 | idx_queue_message2_cnt                    |       56 |         3689 |             0
16632 |   39485402 | public     | queue_message2 | idx_queue_message1_msg_id                 |   683526 |       591117 |        377035
16632 |   39485403 | public     | queue_message2 | pk_queue_message1                         |      408 |           10 |            10
16632 |   39485405 | public     | queue_message2 | pgcompact_index_28802                     |  4752409 |       305091 |        272196
16632 |   39485406 | public     | queue_message2 | queue_message2_qualified_content_name_idx |        0 |            0 |             0
16632 |   39485407 | public     | queue_message2 | idx_queue_message2_main                   |     1652 |         3945 |            78
16632 |   39485408 | public     | queue_message2 | pgcompact_index_14090                     |     1251 |          771 |            63
16632 |   39485409 | public     | queue_message2 | pgcompact_index_4300                      |     1903 |         2173 |            93
16632 |   39485410 | public     | queue_message2 | pgcompact_index_4803                      |  3989263 |      3574874 |        318559
16632 |   39485423 | public     | queue_message2 | idx_queue_message2_mt                     |     9799 |            5 |             2
16632 |   39485436 | public     | queue_message2 | idx_queue_message2_expired                |      594 |     10766611 |         17919
16632 |   39485437 | public     | queue_message2 | idx_queue_message2_content                |   724183 |       209192 |         77533
(13 rows)

[postgres@ ~]$ /u01/postgres/pgcompacttable-master/bin/pgcompacttable -V
/u01/postgres/pgcompacttable-master/bin/pgcompacttable v1.0.10
We using the latest tool's version

One more type of errors - canceling  autovacuum task
2020-11-02 17:30:28 MSK [6018] ERROR:  canceling autovacuum task
2020-11-02 17:30:28 MSK [6018] CONTEXT:  automatic vacuum of table "queue2.public.queue_message2"

Hello
For some applications, it is important not to change the index names. To do this, we first have to build a new index (named pgcompact_index_N), then swap new and old indexes under the index rename lock (with max lock duration --reindex-lock-timeout and it's exactly reason for "canceling statement due to statement timeout" errors in PG log). When we can't rename index, we sleep for --reindex-retry-pause seconds. We trying rename again and again up to --reindex-retry-count retries. print-reindex-queries will print SQL that tool uses for reindex.
I didn't actually expect a deadlock here. I think that in this case we have no further error handling and pgcompacttable will leave the new index. I suggest checking all pgcompact_index_N indexes, most likely they are redundant and should be removed.

I've set execution hourly by crontab

I doubt this is good idea at all. For regular maintenance it would be better to tune autovacuum settings and investigate root cause of your table (or index) bloat. The usual reason is long transactions and default (means lazy) settings of autovacuum. Typically postgresql with appropriate tuning of autovacuum will not need pgcompacttable (excepts after massive deletes).

One more type of errors - canceling autovacuum task

Yes, this is expected behavior. pgcompacttable run some commands (e.g. VACUUM, ANALYZE) which definitely conflicts with autovacuum - autovacuum notices this and cancels itself with such log message.
If you have modern PostgreSQL (12 and above) - it would be better to use native REINDEX CONCURRENTLY command instead of manual indexes rename. It works with indexes in the same way as pgcompacttable, but has no concurrent locking issues during index swap. I am one who worked on this feature in postgresql core, but unfortunately still have no time to add support in pgcompacttable.


Label does not hide when start entering Location on a signup page

No I noticed that when I write text in the input that appears on a map, this text will appear below. What should be the correct  behavior here? Maybe the input below should not be editable, it shouldn't even be an input field?

What browser and device is this on?
The red error also appears a bit too eagerly imo

But the behaviour you are talking about is correct - it populates automatically, but you can change it to whatever you want to display on your profile

chrome 92.0.4515.159

Imo, the input below is more visible than search bar on the map - so maybe it should be disabled till the location from map is not chosen?

That's a good idea.

I can try to do that

I wonder if I broke something here by accident by making the LocationAutocomplete more re-usable so it can be shared in the create/edit event page 😅  please check this form (by creating/editing an event) in say, the global community as well when you get around to make the fix


Latest DJI SDK 4.14-trial1 is not compatible with latest available Beta UX SDK

I have the following pods in Podfile for my project:
pod 'DJI-SDK-iOS', '> 4.14-trial1'
pod 'DJI-UXSDK-iOS', '> 4.14-trial1'
pod 'DJI-UXSDK-iOS-Beta', '> 0.3.0'
pod 'DJIWidget', '> 1.6'
Unfortunately, build fails with the errors:
Property 'isObstacleAvoidanceSensorInVerticalDirectionEnabled' not found on object of type 'DJIFlightAssistantObstacleAvoidanceSensorState *'; did you mean 'isVisualObstacleAvoidanceSensorInVerticalDirectionEnabled'?
Property 'isObstacleAvoidanceSensorsInHorizontalDirectionEnabled' not found on object of type 'DJIFlightAssistantObstacleAvoidanceSensorState *'; did you mean 'isVisualObstacleAvoidanceSensorsInHorizontalDirectionEnabled'?

@YaroTs a new iOS UX SDK beta is coming soon and it will support MSDK/UXSDK 4.14-trial1.


Owlet component fails getting device info

Home Assistant release with the issue:

0.90.1
Last working Home Assistant release (if known):
Operating environment (Hass.io/Docker/Windows/etc.):

Docker
Component/platform:

owlet
Description of problem:
Error during setup of component owlet
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

Traceback (if applicable):

Traceback (most recent call last):
  File "/usr/src/app/homeassistant/setup.py", line 154, in _async_setup_component
    component.setup, hass, processed_config)  # type: ignore
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/app/homeassistant/components/owlet/__init__.py", line 45, in setup
    device = PyOwlet(username, password)
  File "/usr/local/lib/python3.7/site-packages/pyowlet/PyOwlet.py", line 27, in __init__
    self.dsn = self.get_dsn()
  File "/usr/local/lib/python3.7/site-packages/pyowlet/PyOwlet.py", line 56, in get_dsn
    return json_data[0]['device']['dsn']
IndexError: list index out of range

Trying the example:
printing response ------> <Response [200]>
printing json_data ---->   []
Traceback (most recent call last):
  File "example.py", line 15, in <module>
    pyowletClient = PyOwlet(sys.argv[1], sys.argv[2])
  File "/usr/local/lib/python3.7/site-packages/pyowlet/PyOwlet.py", line 27, in __init__
    self.dsn = self.get_dsn()
  File "/usr/local/lib/python3.7/site-packages/pyowlet/PyOwlet.py", line 58, in get_dsn
    return json_data[0]['device']['dsn']
IndexError: list index out of range

Additional information:

@craigjmidwinter Can you confirm if the example in pyowlet currently works?

bump

Will you please confirm if the change on #25962 fixes this issue once it is released?

Just tried 100.1 with change #26892 and this didn't solve it for me.

Can confirm, this still hasnt been fixed. It seems the API urls have changed again.


Autocompletion happening without colon getting appended

Autocompletion for a dependent field happens without a colon. Is this a known issue?
For e.g. spec is a field dependent on another field type as shown in below schema (it's a partial schema, can share complete schema, if needed)
connector field is of type object and works fine, completing with a colon.
But spec field, which is also of type object coming from an allOf section, doesn't.
https://user-images.githubusercontent.com/48940704/107954149-1e711c80-6fc2-11eb-8ff1-328ec6654db5.mov
        "allOf": [
          {
            "if": {
              "properties": {
                "type": {
                  "const": "Http"
                }
              }
            },
            "then": {
              "properties": {
                "spec": {
                  "$ref": "#/definitions/BitbucketHttpCredentials"
                }
              }
            }
          },
          {
            "if": {
              "properties": {
                "type": {
                  "const": "Ssh"
                }
              }
            },
            "then": {
              "properties": {
                "spec": {
                  "$ref": "#/definitions/BitbucketSshCredentials"
                }
              }
            }
          }
        ]

@vardanbansal-harness is there any chance to find an instruction for integrating this plugin into monaco editor?

@vardanbansal-harness when you wrote this issue, the project wasn’t maintained actively. yaml-language-server was outdated and dependencies of this project were invalid. The upstream issue may or may not have been fixed even before you created it.
Could you maybe try if the issue still persists with the latest version in the main branch? I think the easiest way to do so is to fork this project and edit the demo example. In version 3.0.1 the yaml-language-server is the same as it was when you created this issue. In the main branch it has been updated to the latest version at the moment of writing (0.22.0).

I’m going to close this issue, because it’s outdated.


crash: invalid memory address or nil pointer dereference on disk full

1. What version of Caddy are you using (caddy -version)?
Caddy 0.10.10 (non-commercial use only)
7. What did you see instead (give full error messages and/or log)?
When the disk got full caddy crashed with a nil pointer deref...
Feb 21 16:00:24 example.com caddy[19791]: 2018/02/21 16:00:24 [INFO] Scanning for stale OCSP staples
Feb 21 16:00:24 example.com caddy[19791]: 2018/02/21 16:00:24 [ERROR] Checking OCSP: unable to write OCSP staple file for [slack-invite.rclone.org]: write /etc/ssl/caddy/ocsp/slack-invite.rclone.org-5f08d61a: no space left on device
Feb 21 16:00:24 example.com caddy[19791]: panic: runtime error: invalid memory address or nil pointer dereference
Feb 21 16:00:24 example.com caddy[19791]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x801bdf]
Feb 21 16:00:24 example.com caddy[19791]: goroutine 6 [running]:
Feb 21 16:00:24 example.com caddy[19791]: github.com/mholt/caddy/caddytls.DeleteOldStapleFiles()
Feb 21 16:00:24 example.com caddy[19791]: src/github.com/mholt/caddy/caddytls/maintain.go:323 +0x1df
Feb 21 16:00:24 example.com caddy[19791]: github.com/mholt/caddy/caddytls.maintainAssets(0xc42005c180)
Feb 21 16:00:24 example.com caddy[19791]: src/github.com/mholt/caddy/caddytls/maintain.go:76 +0x1df
Feb 21 16:00:24 example.com caddy[19791]: created by github.com/mholt/caddy/caddytls.init.1
Feb 21 16:00:24 example.com caddy[19791]: src/github.com/mholt/caddy/caddytls/maintain.go:32 +0x4e
Feb 21 16:00:24 example.com systemd[1]: caddy.service: main process exited, code=exited, status=2/INVALIDARGUMENT

Huh, interesting. Do you know what made the disk full? (Was it in the .caddy folder?) Just want to be sure there's not something else going on.

No it was another process that filled the disk - nothing to do with caddy....
Caddy has crashed again after I have emptied the disk 5 or 10 mins after restarting it.
I guess I need to clean something out?
Feb 21 17:18:02 example.com caddy[24104]: 2018/02/21 17:18:02 [INFO] Scanning for stale OCSP staples
Feb 21 17:18:02 example.com caddy[24104]: panic: runtime error: invalid memory address or nil pointer dereference
Feb 21 17:18:02 example.com caddy[24104]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x801bdf]
Feb 21 17:18:02 example.com caddy[24104]: goroutine 6 [running]:
Feb 21 17:18:02 example.com caddy[24104]: github.com/mholt/caddy/caddytls.DeleteOldStapleFiles()
Feb 21 17:18:02 example.com caddy[24104]: src/github.com/mholt/caddy/caddytls/maintain.go:323 +0x1df
Feb 21 17:18:02 example.com caddy[24104]: github.com/mholt/caddy/caddytls.maintainAssets(0xc42005c180)
Feb 21 17:18:02 example.com caddy[24104]: src/github.com/mholt/caddy/caddytls/maintain.go:76 +0x1df
Feb 21 17:18:02 example.com caddy[24104]: created by github.com/mholt/caddy/caddytls.init.1
Feb 21 17:18:02 example.com caddy[24104]: src/github.com/mholt/caddy/caddytls/maintain.go:32 +0x4e
Feb 21 17:18:02 example.com systemd[1]: caddy.service: main process exited, code=exited, status=2/INVALIDARGUMENT

Aha. The code leading up to the line of the panic is:
# in for loop
resp, err := ocsp.ParseResponse(ocspBytes, nil)
if err != nil {
	// contents are invalid; delete it
	err = os.Remove(stapleFile)
	if err != nil {
		log.Printf("[ERROR] Purging corrupt staple file %s: %v", stapleFile, err)
	}
}
if time.Now().After(resp.NextUpdate) {  # line 323 - PANIC

There's no continue inside the if err != nil check. I'll add that and push it in a moment. Thanks for the report.

Great - thanks!  From the look of the code my caddy will be OK now I think as I saw it had deleted the file in question and re-created it.

We just had the following error with 0.11.4
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x47817e]
goroutine<PHONE_NUMBER>0 [running]:
bufio.(*Writer).Available(...)
        /usr/local/go/src/bufio/bufio.go:592
bufio.(*Writer).WriteString(0x0, 0xe4372f, 0x19, 0xc01c385c20, 0x509430, 0x6b5c3f)
        /usr/local/go/src/bufio/bufio.go:673 +0x7e
net/http.(*expectContinueReader).Read(0xc00c90e600, 0xc005b67000, 0x812, 0x1000, 0x0, 0xffffffffffffffff, 0x0)
        /usr/local/go/src/net/http/server.go:889 +0x166
io.(*teeReader).Read(0xc00c90e780, 0xc005b67000, 0x812, 0x1000, 0x2, 0x1, 0xc00e6e5b70)
        /usr/local/go/src/io/io.go:535 +0x55
github.com/mholt/caddy/caddyhttp/limits.(*maxBytesReader).Read(0xc007866780, 0xc005b67000, 0x812, 0x1000, 0x408bdb, 0xc00001c000, 0xd3f5c0)
        /tmp/gopath_02-22-0906.228675989/src/github.com/mholt/caddy/caddyhttp/limits/handler.go:74 +0x89
net/http.transferBodyReader.Read(0xc009007360, 0xc005b67000, 0x812, 0x1000, 0xd69220, 0xc00e6e5c01, 0x0)
        /usr/local/go/src/net/http/transfer.go:62 +0x56
io.(*LimitedReader).Read(0xc00f28d940, 0xc005b67000, 0x1000, 0x1000, 0x0, 0x0, 0x5)
        /usr/local/go/src/io/io.go:448 +0x63
bufio.(*Writer).ReadFrom(0xc007867340, 0xf622e0, 0xc00f28d940, 0x7fb4b31ba140, 0xc007867340, 0xc000044f01)
        /usr/local/go/src/bufio/bufio.go:707 +0xe4
io.copyBuffer(0xf61420, 0xc007867340, 0xf622e0, 0xc00f28d940, 0x0, 0x0, 0x0, 0xd76aa0, 0x1, 0xc00f28d940)
        /usr/local/go/src/io/io.go:388 +0x303
io.Copy(0xf61420, 0xc007867340, 0xf622e0, 0xc00f28d940, 0xc0003b0f00, 0x0, 0x1)
        /usr/local/go/src/io/io.go:364 +0x5a
net/http.(*transferWriter).writeBody(0xc009007360, 0xf61420, 0xc007867340, 0x2, 0x2)
        /usr/local/go/src/net/http/transfer.go:362 +0x5b8
net/http.(*Request).write(0xc00d5ce700, 0xf61420, 0xc007867340, 0x0, 0xc0053fa270, 0xc00c90f8a0, 0x0, 0x0)
        /usr/local/go/src/net/http/request.go:645 +0x6e8
net/http.(*persistConn).writeLoop(0xc006265440)
        /usr/local/go/src/net/http/transport.go:1888 +0x1b8
created by net/http.(*Transport).dialConn
        /usr/local/go/src/net/http/transport.go:1339 +0x966

I don't know if it's related or I should create another issue?


Revert invalid date input when blurring

Fixes #106
When blurring the date input, either sets the value to the formatted date, or clears it if no date is selected.  Has the effect of wiping out invalid dates.

Current coverage is 91.82%

Merging #327 into master will increase coverage by +0.16% as of c989962

@@            master    #327   diff @@
======================================
  Files            9       9
  Stmts          204     208     +4
  Branches        44      45     +1
  Methods        100     101     +1
======================================
+ Hit            187     191     +4
  Partial          4       4
  Missed          13      13

Review entire Coverage Diff as of c989962

Powered by Codecov. Updated on successful CI builds.

lgtm


Replace internal use of deprecated OsVersion.rawVersion with raw_version

Change-type: patch
Signed-off-by: Thodoris Greasidis<EMAIL_ADDRESS>
Contributor checklist

[ ] Includes tests
[ ] Includes typings
[ ] Includes updated documentation
[ ] Includes updated build output

@balena-ci I self-certify!


Weibull Adstock transformation results are different for training vs whole dataset

Project Robyn
Describe issue
We have some questions about the Weibull Adstock function. My understanding of applying the Adstock transformation is that we're carrying over the effect of previous investment into the current time, therefore each value of the transformed variable gets adjusted using the previous time point value. In mathematical terms,
decay_{t,j} = x_{t,j} + f(x_{t-1,j}, param_{j})
where x_{t,k} is the variable at time t and for media j, f is some function and param are some parameters for the function (theta, scale, shapes...) for media j.
What we see is that when we pass to the Weibull Adstock function two datasets for the same media, one with only the training data and then a second one that includes more recent data (test set), the outputs are not matching. The behaviour we expected was that adding more data at the end of the time series, should only affect the last values, but what we actually see is that the transformed results are completely different even for the starting data-points. We think this is faulty behaviour as Adstock transformations should affect only the future and not the past, therefore adding extra data-points at the end of a time series should not affect the value for the transformed variable at the start. We've observed this behaviour only for the Weibull adstock, not for the Geometric.
I would like to understand if this is an expected behaviour and if so, how could we apply this transformation for out-of-sample data. Thanks!
Sample Code
We've run a reproducible test of this behaviour using the simulated data provided with Robyn. Below the sample code, where we're applying the Adstock transformation to the Search spend, but in two different ways, first passing the whole dataset (208 observations), and then only providing the first 150 observations.
shape <- 1.55
scale <- 0.04
m <- dt_simulated_weekly[, "search_S"]
ads_whole <- adstock_weibull(x = m$search_S, shape = shape, scale = scale)
ads_train <- adstock_weibull(x = m$search_S[1:150], shape = shape, scale = scale)
x_ads_whole <- ads_whole$x_decayed
x_ads_train <- ads_train$x_decayed

In both of these cases, only the first two values remain the same, but all other values of the transformed variable are different, see screenshot below.

In the case of the Geometric adstock function, we're not seeing this type of results, the values are matching as expected.
theta <- 0.4
m <- dt_simulated_weekly[, "search_S"]
ads_whole <- adstock_geometric(x = m$search_S, theta=theta)
ads_train <- adstock_geometric(x = m$search_S[1:150], theta=theta)
x_ads_whole <- ads_whole$x_decayed
x_ads_train <- ads_train$x_decayed

In this case, the results from both transformations are matching:

Environment & Robyn version
R version 4.1.3 (2022-03-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 10 (buster)
Robyn version: Robyn_3.6.2

Hi - thanks for the question and apologies for the delay in getting back to you. This is working as intended, since Weibull adstock (esp. scale param) is in fact dependent on the length of the dataset. I'll illustrate here with an example -

Looking at this chart in the bottom left, where for example shape =9 and scale = 0.5. when m1 is 100 days, inflexion is at about day 40, or let's say around position 40% of X axis. When the user cuts the first 50 days for transformation, the relative position of inflexion point changes to day 20.
So the best way to work around your use case would be to do all of the adstocking transformation prior to doing the splitting of the data into train/test, so then there will be no issue with the window length anymore.


Added increment-decrement rule

Equivalent to tslint-microsoft-contrib's no-increment-decrement. Bans ++/-- unary expressions in favor of the += equivalents. Allows an "allow-post" option for the i++ scenario.
Fixes #3556.
CHANGELOG.md entry:
[new-rule] increment-decrement

First iteration failure:

TypeError: ts.isPostfixUnaryExpression is not a function

tsutils it is.

This is yet another rule that could be implemented with a generic ban-syntax rule described in #3591 and #2497
Though that doesn't mean this PR shouldn't land.

@JoshuaKGoldberg Hi, so this is not yet part of 5.10? Waiting for implementation using a generic ban-syntax? Thanks & Cheers, Marcel


A bug was introduced (latest update?)

While editing Javascript file in OS X:
Select block of code w/comments (not sure having comments is required) hit TAB:
Expected: increase indent.
Result: text is messed-up thru the end of file. Undo does not recover text.

This also happens in typescript with shift+tab.
Restarting code fixes the issue temporarily.

Dup of #2829. We are preparing a recovery build.

Recovery build 0.10.9 is now available


tag and build Poolv0.6.3

Tag and release Poolv0.6.3 now that #451 is merged, to allow Pool users who need to generate new L402 certificates to enter the Pool market.

Done: https://github.com/lightninglabs/pool/releases/tag/v0.6.3-beta


XML-specific annotations

jackson supports also XML serialization and deserialization through some extra dependency, and even JAXB XML-specific annotations. Those don't work with javaslang-jackson.. That may be on purpose (jackson supports so many features, it's really difficult to support it all), but in case you actually intend to support these use cases, I added two failing tests: one using jackson XML-specific annotations, and one using JAXB annotations.
In the tests I serialize the "same" object, once with java.util.List, once with javaslang.collections.List, and expect the same XML output. I commited the test in the xml_annotations branch of my fork, in commit https://github.com/emmanueltouzery/javaslang-jackson/commit/6fbad5f8a0bbeb18134b0fe5ce30b81d8ec860cc

Until now it is only csv used here :)
thank you

It seems that the problem is localized here.
java.util.ArrayList is an Jackson indexed type, but javaslang.collection.List is not.

I was looking a little more...
I think the criteria is inheritance from java.util.Collection, or being a raw array, which we obviously don't have.
Following the call, we find this.
they call JavaType.isContainerType. Also line 35 of that same file, in another codepath, they do Collection.class.isAssignableFrom(cls) (and array check).
in jackson, JavaType is constructed only in TypeFactory, according to the javadoc.
this seems relevant.
There are several JavaType subclasses, CollectionLikeType would be good.
I found this way where you get a collection type built, and sure enough it checks for Collection.class. There's also something for arrays.
So it seems we may have to submit a patch for jackson-databind... We could suggest that they could also accept as a criteria for a collection implementation of java.util.Iterable, but they may be worried about back-compatibility...
What do you think?

So it seems we may have to submit a patch for jackson-databind... We could suggest that they could also accept as a criteria for a collection implementation of java.util.Iterable, but they may be worried about back-compatibility...

I believe this is right way, because it seems we can't do anything here in Javaslang.
We can start with opening issue with description of our problem. Could you create it?

github did link the issue, but in case you did not get notified, I opened https://github.com/FasterXML/jackson-databind/issues/1430. Let's see what they suggest...

Thank you!

i don't know if you saw, we got a pretty amazing answer on the upstream bug.
From what I can see, CollectionLikeType does return true for isContainerType() so it should work I think.
I've tried myself to make javaslang List a CollectionLikeType, but I broke a huge number of tests with that (3 failures, 21 errors):
test1(javaslang.jackson.datatype.ScalarTest)  Time elapsed: 0.003 sec  <<< ERROR!
com.fasterxml.jackson.databind.JsonMappingException: Can not find a Value deserializer for abstract type [collection-like type; class javaslang.collection.List, contains [simple type, class java.lang.Integer]]

If that can help, what I tried is there.
If you think I'm not that far from making it, or if you don't have the time, I can try to make this work, but I think I'd need some guidance from you, otherwise it'll take me inordinate amounts of time :-(

Hi Emmanuel
First of all, JavaslangTypeModifier should be modified on something like this:
final Class<?> raw = type.getRawClass();
if (List.class.isAssignableFrom(raw)) {
   return CollectionLikeType.construct(
         raw, type.containedTypeOrUnknown(0));
}
return type;

because in fact javaslang.collection.List instance represents by javaslang.collection.List$Cons implementation. And here we can observe next problem: Jackson doesn't find custom serializer if serializable type is container type (see this cycle). It seems this can be solved with implementing findCollectionLikeSerializer() method in JavaslangSerializers
I believe this is right way :)
Last few days I have not enough time for Javaslang projects, but I'll take a look on this issue soon.


XMLHTTPRequest set Cookie and read Set-Cookie

I'm not sure if this is a pebblejs issue, or a Pebble JS runtime issue on the phone; I've had little luck on the SDK forum, so I'm x-posting here:
I'm aware of the forbidden headers in the XHR spec: https://fetch.spec.whatwg.org/#forbidden-header-name
There is two distinct problems with this, however:

In development, the emulator CAN set Cookie's and read Set-Cookie's. I imagine this is because the underlying implementation of XMLHTTPRequest in the emulator is python's urllib or something similar.
JS runtimes on the phone CANNOT set Cookie's and read Set-Cookie's using the same code.

There is a disconnect between the two, but I believe that the phone's JS runtime should support general HTTP interaction as if your were interacting with some HTTPConnection. Is there a simple configuration flag I can put on the XMLHTTPRequest object call to support these headers? I haven't had much success with just withCredentials = true, either.
Thanks for your help!

I imagine the use case outlined is so rare that it's simply not worth the time and energy to investigate. Since I've found a decent enough workaround for my problem, I'm closing this issue.
For those interested in the workaround: don't rely on cookies! If you are integrating with a third party service that relies on cookies (like I was), your best bet is to build a service to proxy the calls with the authenticated session for you. On the plus side, it makes web requests initiated by Pebble very simple, as it was intended to be.

I ran into the same problem as you, but I disagree about the solution to use a proxy: I really don't want to handle other people's passwords. I'd rather just send them directly to the correct server (not controlled by me). It also means I have to run my own proxy just for a small server, and that my app stops working if I have server issues.
Maybe consider re-opening the issue?

I agree that it's a problem, and the workaround is not ideal. I also think the problem lies on the Pebble JS runtime on the phone, so it really falls outside of the scope of PebbleJS. Ultimately the issue falls into this technical grey area: whether or not cross origin sandboxing should stay true to the JS specification. Of course there are advantages and disadvantages in either case, but it would certainly be worth discussing to those involved in the runtime project.
As far as proxying the requests, AWS's API Gateway makes this task pretty easy. I believe once you start proxying, a world of features open up to you that were closed before.

+1 to reopen.  I'd really rather not handle passwords, either.

You can store credentials once as a Pebble configuration page and the information is stored on the watch, which is no different than having a native app that does the same thing. You can still proxy the requests with API Gateway, and accomplish this task so I don't quite understand where the "handling passwords" hold up is coming from (for you to have a session, you need the credentials anyway). That said, having a definitive answer on what should happen with remote requests using a XMLHTTPRequest would be great, so I'm opening the ticket for that reason.

Is the issue that you're sending a user to a third-party service to authenticate, and want to get the cookies returned in the authenticated session so that you can then send requests as if you are said user (and therefore authenticated)?  Seems like a case for something like OAuth, though I'm guessing this has to do with services that don't provide OAuth?

Correct. The original problem is trying to communicate to a third party service that doesn't support OAuth, and we don't have control over it. Ideally the service would simply support modern authentication methods and no workaround would be needed.

I don't mind storing credentials in the app -- in that case, they're not leaving the device.  However, I'd rather not have user credentials passing through a server that I control.
(On the other hand, in the case of the service that I am proxying for, you can call them up on the telephone, and provide the e-mail address in question and a year of birth, and they'll just read your password out to you.  But it's the principle of the matter!)

Hmm, this is a problem with the PebbleKit JS JavaScript VM as you have found, and not particular to Pebble.js, but it's relevant enough to warrant a ticket here. Have you tried contacting devsupport at pebble.com? That is the normal process for potential issues with the base SDK, though I am curious to this problem as well. I will write some thoughts here.
I agree that it is pretty nasty to pass credentials through your own server. If this cookie header setting restriction were removed however, I'm not sure it would solve the idea behind that nastiness -- something would need to intercept the cookie in order to pass it to your service JavaScript. And to intercept the cookie, that would mean having the potential to intercept the password.
AFAIK, the only way to guarantee that code originating from you doesn't touch, or doesn't even have the potential to touch the password is if you use the WebView directed to the service you're trying to access. But after the user logs in, something needs to grab the cookie and close the WebView. If the service is just an old fashioned cookie setting server from yesteryear's web days, ways to do so is to fake the request with your own code, or worse, encapsulate one or the other with an iframe that somehow is able to communicate which I don't think is possible with modern browsers anymore. If the user were to not trust any code that has the potential to grab their password, these methods are just as bad as a proxy from their perspective. Why they hold their password dear and not authority over the account is a different puzzle ;)
I think, then, that OAuth or some derivative that utilizes the WebView and redirects to a page after authentication is the solution, and that would be on the part of the service you're trying to access to implement unfortunately. If you do come up with a way to gain the cookie without ever having the potential to access the user password that works for all cookie based services however please do post here, though I'm not in charge of the implementations of the JavaScript VMs that are in use with PebbleKit JS.
I can think of at least one alternate method, but even then there are still pitfalls. If there was a way for a user's JavaScript to abide to a CORS-like contract -- that it can only communicate to certain blessed domains -- then on the surface it would seem to lower how much it matters that the code can intercept the password since it limits how the password can travel. But let's say the old web service is a forum. An evil JavaScript app could appear as a regular forum viewer, but can secretly post the password in a thread or over the forum's direct message functionality. Or more straightforward, the target service has a web exploit that allows outside communication. Furthermore, this also assumes that both the PebbleKit JS VM and the Pebble C are impenetrable; the C app is considered since this functionality would be available for all Pebble apps, not just Pebble.js apps. The Pebble App would also have to prompt the user each time the contract updates as well.
Without a method to obtain the cookie without having guarantees that the password cannot be transmitted or saved elsewhere, it would be ethically difficult to encourage other developers to use as a pattern in their app since it means encouraging the possibility to intercept passwords, which might be what the W3C imagined, and is of course what OAuth was made to address. With all that said however, you're free to implement a proxy to work around this issue. I can understand the frustration with being unable to access legacy systems with what seems like small restrictions, but I also see W3C's intention on the other hand.

For me the problem is not the user trusting the developer. Yes, OAuth would be nicer, but we don't have it, so it's up to the user to decide it they trust the code or not.
For me the problem is the developer not trusting the developer. I'm smart enough to realize that plenty of things can go wrong when you start sending passwords around to proxy servers (whether it's a 3rd party server or something I run myself).
Using the cookie approach I need to ask the user for the password once, and after that I can throw it away, without having access to any sensitive information after that. Not because I think my code will do something bad with the login details on purpose, but because I think it might do something bad with them by accident (or due to some exploit).
Normal Javascript does NOT have access to the cookie data, so it's not even possible to intercept it. The User-agent handles all of that in the background. So after the login, from the perspective of my own code, I just need to do normal HTTP requests, and the authentication is handled behind the scenes, out of reach of my own code. The password is only needed at the time of login, and is not needed for subsequent requests.
All of this, in fact, seems to work in a normal web browser, and on iOS, but not on Android. This has been confirmed by devsupport, so they know about the issue as well.

Ah, so it works on iOS but not Android? That's interesting, I'll have to ask about how that would be resolved. When I get an answer I'll be sure to post here again.

For me the problem is the developer not trusting the developer

In creating platforms, the worst case is taken into consideration so that they may be prevented, including misbehaving developers. That said, I surely don't think anyone posting in this ticket is a misbehaving developer.

Not because I think my code will do something bad with the login details on purpose, but because I think it might do something bad with them by accident (or due to some exploit).

That's exactly why limited platforms exists, so these worst cases can't happen. It also results in very frustrating APIs. I was merely expanding on why the limitation exists. I'll be sure to question the discrepancy between iOS and Android since I wasn't aware of that.

Normal Javascript does NOT have access to the cookie data, so it's not even possible to intercept it.

I was referring to document.cookie which only works if the page is in the same domain of course, but I admit that was silly of me to point out since the page must originate from the same domain. Sandbox breaking functionality would be needed for it, such as unrestricted iframes. However, the password being intercepted is the main concern.
I can see this feature being possibly supported because one platform allows it, but I didn't want to give false hope, so I attempted to expand on the issue. It's really a question of how far a platform should go to stop passwords from leaking. Fitbit's OAuth API even forbids the use of WebViews! It's a bit farfetched, but they're not wrong; the WebView can be hooked in many arbitrary ways. It's as good as compromised.

Use of WebView is prohibited. From the Fitbit OAuth2 docs

Once the problem is framed that way, you could say a platform that doesn't stop passwords from leaking is asking for it. In this particular case, the idea is whether we want to support middleman login forms. It might actually even be easier to implement than using OAuth, and could end up being used as a means when it shouldn't. Of course, this appears as a slippery slope, but I only mention this as a possibility and not an inevitability. We would have to deploy the functionality to know if that would occur at all, but the risk has to be taken into account before doing so.
In any case, it seems iOS has already taken that risk! I'll be asking around just which platform should change. It's unfortunate but the outcome could be anything, including no change. When only one platform supports a feature, developers generally avoid it since it's not fully supported. iOS is one to have very limited API's, so even if it was wanted to be disabled on iOS, it could ironically be difficult to do so. But note that I'm not in any particular position to make comment on the iOS implementation or what will get decided. I just want to give a heads up.

I develop apps for my own use, so I'm not a huge fan of restrictions that are there to protect myself from me stealing my own data :-) (I am OTOH, a fan of restrictions that are there to protect myself from my own incompetence).
 However, in the source code there is a switch which eliminates this step, so it's possible to bypass it by proving that you have actually looked at the source code. I'm thinking of making the source-code based switch mandatory so that simply checking the box is not enough.
So, I fully agree that users should not be typing their password in random places. It's just that sometimes that user is me, and I know exactly what the app is doing.

@Meiguro -- "Without a method to obtain the cookie without having guarantees that the password cannot be transmitted or saved elsewhere, it would be ethically difficult to encourage other developers to use as a pattern in their app since it means encouraging the possibility to intercept passwords, which might be what the W3C imagined, and is of course what OAuth was made to address."
This is "kind of" what the W3C imagined.  The goal in making XMLHTTPRequest cookie-unsafe is to protect web sites from other web sites ... not to protect web sites from applications.  XMLHTTPRequest is used in PebbleJS as a convenience API (exposed to applications that the user has already trusted by explicitly installing them), for compatibility with what developers already know; compare the usage of XHR on the web, where applications are expected to be untrusted, and no user input is required to execute an XHR (i.e., it could be part of a hidden iframe dropped by malware on a web page).
Similarly, the W3C is, for better or worse, generally not concerned with how web sites manage authentication credentials.  If the W3C enforced that everyone used OAuth, or secure practices in general, then indeed, we wouldn't have this problem :-) But in the mean time, we still have to live with these awful legacy APIs, and we need a way to make them work: if the remote end wanted secure remote usage, then they'd provide OAuth.
So yes, the W3C's design makes sense for the web.  When just commandeering the API for another programming interface, though, it makes sense to augment it to not strip cookies.

I basically agree with jwise here (and this is an old conversation we keep come back around to — CORS enforcement was basically the thing, for instance).
This also has nothing to do with Pebble.js.

The goal in making XMLHTTPRequest cookie-unsafe is to protect web sites from other web sites ... not to protect web sites from applications

After last night, this is what I had come to realize. Basically, if I am to view the Pebble as a computer, it should be able to have clients. And if a legacy system requires cookie based authentication, it should be possible to write a suitable client. The decision of whether a client should have access to a password should be up to the user, just like with desktop clients.
Thanks for the discussion everyone, I was viewing the issue through the lens of OAuth. That only works if everything spoke OAuth as jwise pointed out.

This also has nothing to do with Pebble.js.

Yes, it's a bit of an issue when I get non-Pebble.js tickets. Would it make sense to have a Pebble SDK issue tracker?

We're looking into it! For now, emailing devsupport gets the job done.

Sorry to post here, but it seems to be the central thread for this issue.  I noticed the issue persists with v4.0.  Are there any plans to implement cookies in PebbleKit JS in the near (or not so near) future?  Just wondering if I should keep my hopes up.  I have a few projects that have fallen flat due to the lack of cookie access.
Thank you.

Already another year over?
I have the same question like @robisodd:

Are there any plans to implement cookies in PebbleKit JS in the near (or not so near) future?


RUMM-1388: Add GDPR related tests for the Trace feature

What does this PR do?
This change is pretty much similar to the #591, just for the Trace feature.
Review checklist (to be filled by reviewers)

[x] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[x] Make sure you discussed the feature or bugfix with the maintaining team in an Issue
[x] Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

Codecov Report

Merging #616 (36ade03) into master (50ba133) will increase coverage by 0.15%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #616      +/-   ##
============================================
+ Coverage     88.75%   88.90%   +0.15%
- Complexity     1612     1614       +2
============================================
  Files           181      181
  Lines          5385     5385
  Branches        655      655
============================================
+ Hits           4779     4787       +8
+ Misses          372      364       -8
  Partials        234      234

Impacted Files
Coverage Δ

...nternal/tracking/OreoFragmentLifecycleCallbacks.kt
89.74% <0.00%> (-5.13%)
:arrow_down:

...g/android/rum/internal/domain/scope/RumEventExt.kt
81.82% <0.00%> (-3.64%)
:arrow_down:

...droid/rum/internal/domain/scope/RumSessionScope.kt
96.00% <0.00%> (ø)

...g/internal/domain/event/DdSpanToSpanEventMapper.kt
93.88% <0.00%> (ø)

...l/net/info/BroadcastReceiverNetworkInfoProvider.kt
79.17% <0.00%> (ø)

...c/main/kotlin/com/datadog/android/DatadogConfig.kt
94.53% <0.00%> (+0.50%)
:arrow_up:

...oid/src/main/kotlin/com/datadog/android/Datadog.kt
83.33% <0.00%> (+1.04%)
:arrow_up:

...m/internal/domain/scope/ExternalResourceTimings.kt
79.41% <0.00%> (+5.88%)
:arrow_up:

...android/rum/internal/ndk/DatadogNdkCrashHandler.kt
81.58% <0.00%> (+6.58%)
:arrow_up:


Issues with creating chroot on NixOS

Hi,
I'm trying to use your script, but I'm encountering issues on the way. First I had to change version of deboostrap, because
version 1.0.128 is no longer present in the repository -> using  1.0.137 instead. I'm running on fresh installation of NixOS 23.11
But no matter what version of NixOS or debootstrap I use, I still encounter following issue when creating chroot:
...
I: Extracting zlib1g...
W: Failure trying to run: chroot "/opt/chroot" /bin/true
W: See /opt/chroot/debootstrap/debootstrap.log for details
chroot /opt/chroot unsucessful creation
InstallChroot->createChroot: run sudo rm -rf /opt/chroot and do it again

deboostrap.log looks like:
...
2024-09-16 18:15:02 URL:http://deb.debian.org/debian/pool/main/z/zlib/zlib1g_1.2.13.dfsg-1_i386.deb [82644/82644] -> "/opt/chroot//var/cache/apt/archives/partial/zlib1g_1%3a1.2.13.dfsg-1_i386.deb" [1]
/usr/share/debootstrap/functions: line 1142: chroot: command not found
/usr/share/debootstrap/functions: line 1148: grep: command not found
/usr/share/debootstrap/functions: line 1148: head: command not found
/usr/share/debootstrap/functions: line 1148: sed: command not found
/usr/share/debootstrap/functions: line 1148: cut: command not found

I'm not sure what I'm doing wrong..
Any advice?
Thank you!! :)

@ruyrybeyro did you by any chance had a moment to take a look at this?


add CI job to check for broken links

This PR proposes adding a continuous integration job that will scrape all the external links out of the source code in this repo and check that they're valid.
To be respectful of other sites and to avoid introducing friction into the PR process in this project, this PR proposes that this new check be run once a day, NOT on every pull request.
As of this change, it will also be possible to manually run this job any time you want from https://github.com/saturncloud/examples/actions.
How to test this
To test manually, pull this branch and run
make check-links

Prefect re-arranges their docs and breaks a link we use in one of the notebooks, and we don't find out about it until a customer tries the notebook.

Ah yeah good point


User Story 4: User can select next delivery date

As a user
When I visit /user/profile
I can see a button to change the next delivery date of a crate
Once I click that button
There is a popup with a calendar
Once I select a date on the calendar
And click the checkbox
The new delivery date I set is now listed

the user can only pick 1sxt or 15th


implement fade out of text before paywall

In our previous Headline theme, you had implemented the nice fade-out before the paywall. That was nice, please do again.

And we'd like the following copy in the paywall block please (as in the above) - (maybe we'll be able to change that copy again soon after we plan a bit of your time for next theme enhancements):
Subscribe to keep reading.
Get a deeper understanding of the money and special interests behind the headlines. Subscribe for just $5 a month for the full story and to directly support our independent journalism.


Spanish updated

Changed Spanish for Español, for Foundry not to duplicate spanish in language list.

Hi Lozalojo,
Thanks for the contribution. I'll review and hopefully merge tonight.
Michael

I'm sorry for the delay. School has been keeping me rather busy. I'll try to find time to look at this soon.

Thanks for the changes Lozalojo. Everything looks good.


send to new connections

Hi.
i have code from the example SimpleWsRouter.php and i want to send some information once to only new connected clients, how can i do that. Thanks for any help :)

@FakerSenpai - You can use the subscription meta module here: https://github.com/voryx/ThruwaySubscriptionMeta to see when people subscribe to send information to them.


CSRF token missing or incorrect in https request

Hi
I am having an issue related to CSRF token while using https end points.
Issue Description
I am using django-auth-adfs for authentication and authorization, my application which works perfectly as expected in localhost , I am not getting any CSRF token issue in POST/PUT request.
Deployed the application in azurewebapp and the endpoints became https.
Now in azure endpoint PUT/POST request I am getting below exception
HTTP 403 Forbidden
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept
{
"detail": "CSRF Failed: CSRF token missing or incorrect."
}
Could you please help me to resolve the issue, your help will be much appreciated

@JonasKs

CSRF isn't directly related to authentication, but APIs in general. So, nothing I really can help you with.
You should read this.


Fix When serialization fails, the code value in org.apache.dubbo.rpc.RpcException is set incorrectly. (#12279)

fix https://github.com/apache/dubbo/issues/12279

RpcException

RpcException uses the internal code variable to indicate which type of exception has occurred. When a serialization exception occurs, the code value is set to NETWORK_EXCEPTION, which is incorrect. The correct value for code should be SERIALIZATION_EXCEPTION.

@CrazyHZM @chickenlj PTAL


Storage: Disabled failing UploadAsync_ProgressReporting live test

We've seen UploadAsync_ProgressReporting failing during the live test runs.  Marking it Explicit until it can be made more reliable.

Hi @tg-msft, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.


promobot-generate-manifest: Fix name of bazel target

This lets us do bazel build //cmd/promobot-generate-manifest

/lgtm
/approve


Fix warnings with terraform v0.12.17

After updating to the latest terraform I ended up with a few warnings complaining about quoted interpolations being deprecated. This PR addresses those warnings for a clean init/apply output :smiley:.

This change is

Anybody with write access that can review these changes so we can get them merged?

@AlexRoman , @guimaluf , @felddy , @jsf9k , @dav3r
This repo is in my queue to convert to TF 0.12. It is not enough to just replace syntax but we have requirements to cover module with tests. If you want to contribute it in a proper way (see example) you are most welcome! Give it a try, then ping me, so I could review and run tests. This will help us and speed up the process. Otherwise we all have to wait until it will be done as well as many other modules in queue


Unable to download webpage: IncompleteRead

[youtube] qA9u6N4Rp98: Downloading webpage
ERROR: Unable to download webpage: IncompleteRead(15017 bytes read, 1341 more expected) (caused by IncompleteRead(15017 bytes read, 1341 more expected)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
C:\Green software\youtube>youtube-dl -U
youtube-dl is up-to-date (2017.04.28)
C:\Green software\youtube>

Be sure to call youtube-dl with the --verbose flag and include its complete output.


[docs] Add release notes (draft/in progress) for 0.2

Build and tests passed.

Build and tests passed.


⚠ Bofrost.be has degraded performance

In 760e4da, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 5271 ms

Resolved: Bofrost.be performance has improved in 426319d.


Plots are not generated in allinone

Hello.
I'm trying to use the allinone image to proceed benches and plot generation.
The logs shows:
No log file found with pattern !
No log file found with pattern !
However, the documentation should indicate the use of write_{bw,iops}_logs in FIO file.

Ok, PLOTNAME as to NOT contain any space.


[Feature] Add a check for Admins

Awesome script! However was just thinking, if there could be added an option into the config to enable/disable a check for admins, if enabled their accounts won't show up on the leaderboard? Or even an option to drop an identifier either char/license/steamid/etc to not include that account, etc.
Just a thought, thanks!

Sounds like a good suggestion, I already reply you on Discord. Cheers!