dokku_config does not update values if they exist already

If you're using Dokku - especially for commercial purposes - consider donating to project development via OpenCollective or Patreon. Funds go to general development, support, and infrastructure costs.
If you'd like to sponsor specific functionality, see the project's Sponsoring document.
If you need support for a version of Dokku that is more than a year old, your issue may be closed without an answer. Please upgrade to a recent version before filing an issue.

Description of problem
Ansible role that sets Dokku config using dokku_config will not update keys that already exist. Is there a workaround for this?
How reproducible
Steps to Reproduce
In an Ansible role, I calculate a var clickhouse_database_url.  I then set the env var with Dokku like this:
- name: Set DOKKU env vars for our app
  dokku_config:
    app: "{{ dokku.application }}"
    restart: false
    config:
      CLICKHOUSE_DATABASE_URL: "{{ clickhouse_database_url.stdout }}"

However, if the value changed and we run Ansible again, Dokku ansible will not update the value if the key exists already in the config:
https://github.com/dokku/ansible-dokku/blob/978b45ac436ad2ce563d93955d5d2de30b8c423a/library/dokku_config.py#L86-L87
Actual Results
CLICKHOUSE_DATABASE_URL is still the old value
Expected Results
CLICKHOUSE_DATABASE_URL gets overwritten with the new value, or workaround suggested?
Environment Information

dokku report APP_NAME output

-----> uname: Linux plausible-analytics 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
-----> memory:
                     total        used        free      shared  buff/cache   available
       Mem:           1923         831          76         115        1015         843
       Swap:             0           0           0
-----> docker version:
       Client: Docker Engine - Community
        Version:           24.0.2
        API version:       1.43
        Go version:        go1.20.4
        Git commit:        cb74dfc
        Built:             Thu May 25 21:52:22 2023
        OS/Arch:           linux/amd64
        Context:           default

       Server: Docker Engine - Community
        Engine:
         Version:          24.0.2
         API version:      1.43 (minimum version 1.12)
         Go version:       go1.20.4
         Git commit:       659604f
         Built:            Thu May 25 21:52:22 2023
         OS/Arch:          linux/amd64
         Experimental:     false
        containerd:
         Version:          1.6.21
         GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
        runc:
         Version:          1.1.7
         GitCommit:        v1.1.7-0-g860f061
        docker-init:
         Version:          0.19.0
         GitCommit:        de40ad0
-----> docker daemon info:
       Client: Docker Engine - Community
        Version:    24.0.2
        Context:    default
        Debug Mode: true
        Plugins:
         buildx: Docker Buildx (Docker Inc.)
           Version:  v0.10.5
           Path:     /usr/libexec/docker/cli-plugins/docker-buildx
         compose: Docker Compose (Docker Inc.)
           Version:  v2.18.1
           Path:     /usr/libexec/docker/cli-plugins/docker-compose

       Server:
        Containers: 11
         Running: 8
         Paused: 0
         Stopped: 3
        Images: 30
        Server Version: 24.0.2
        Storage Driver: overlay2
         Backing Filesystem: extfs
         Supports d_type: true
         Using metacopy: false
         Native Overlay Diff: true
         userxattr: false
        Logging Driver: json-file
        Cgroup Driver: cgroupfs
        Cgroup Version: 1
        Plugins:
         Volume: local
         Network: bridge host ipvlan macvlan null overlay
         Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
        Swarm: inactive
        Runtimes: io.containerd.runc.v2 runc
        Default Runtime: runc
        Init Binary: docker-init
        containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
        runc version: v1.1.7-0-g860f061
        init version: de40ad0
        Security Options:
         apparmor
         seccomp
          Profile: builtin
        Kernel Version: 5.4.0-153-generic
        Operating System: Ubuntu 20.04.2 LTS
        OSType: linux
        Architecture: x86_64
        CPUs: 1
        Total Memory: 1.878GiB
WARNING: No swap limit support
        Name: plausible-analytics
        ID: c4517169-92b4-4913-b234-65513c0af112
        Docker Root Dir: /var/lib/docker
        Debug Mode: false
        Experimental: false
        Insecure Registries:
         <IP_ADDRESS>/8
        Live Restore Enabled: false

-----> git version: git version 2.25.1
-----> sigil version: 0.6.0
-----> herokuish version:
 !     Herokuish image gliderlabs/herokuish:latest-20 is not available
-----> dokku version: dokku version 0.30.8
-----> plugn version: plugn: 0.6.1
-----> dokku plugins:
         00_dokku-standard    0.30.8 enabled    dokku core standard plugin
         20_events            0.30.8 enabled    dokku core events logging plugin
         app-json             0.30.8 enabled    dokku core app-json plugin
         apps                 0.30.8 enabled    dokku core apps plugin
         builder              0.30.8 enabled    dokku core builder plugin
         builder-dockerfile   0.30.8 enabled    dokku core builder-dockerfile plugin
         builder-herokuish    0.30.8 enabled    dokku core builder-herokuish plugin
         builder-lambda       0.30.8 enabled    dokku core builder-lambda plugin
         builder-null         0.30.8 enabled    dokku core builder-null plugin
         builder-pack         0.30.8 enabled    dokku core builder-pack plugin
         buildpacks           0.30.8 enabled    dokku core buildpacks plugin
         caddy-vhosts         0.30.8 enabled    dokku core caddy-vhosts plugin
         certs                0.30.8 enabled    dokku core certificate management plugin
         checks               0.30.8 enabled    dokku core checks plugin
         clickhouse           1.34.0 enabled    dokku clickhouse service plugin
         common               0.30.8 enabled    dokku core common plugin
         config               0.30.8 enabled    dokku core config plugin
         cron                 0.30.8 enabled    dokku core cron plugin
         docker-options       0.30.8 enabled    dokku core docker-options plugin
         domains              0.30.8 enabled    dokku core domains plugin
         enter                0.30.8 enabled    dokku core enter plugin
         git                  0.30.8 enabled    dokku core git plugin
         haproxy-vhosts       0.30.8 enabled    dokku core haproxy-vhosts plugin
         letsencrypt          0.13.0 enabled    Automated installation of let's encrypt TLS certificates
         logs                 0.30.8 enabled    dokku core logs plugin
         network              0.30.8 enabled    dokku core network plugin
         nginx-vhosts         0.30.8 enabled    dokku core nginx-vhosts plugin
         plugin               0.30.8 enabled    dokku core plugin plugin
         postgres             1.34.0 enabled    dokku postgres service plugin
         proxy                0.30.8 enabled    dokku core proxy plugin
         ps                   0.30.8 enabled    dokku core ps plugin
         registry             0.30.8 enabled    dokku core registry plugin
         repo                 0.30.8 enabled    dokku core repo plugin
         resource             0.30.8 enabled    dokku core resource plugin
         run                  0.30.8 enabled    dokku core run plugin
         scheduler            0.30.8 enabled    dokku core scheduler plugin
         scheduler-docker-local 0.30.8 enabled    dokku core scheduler-docker-local plugin
         scheduler-null       0.30.8 enabled    dokku core scheduler-null plugin
         shell                0.30.8 enabled    dokku core shell plugin
         ssh-keys             0.30.8 enabled    dokku core ssh-keys plugin
         storage              0.30.8 enabled    dokku core storage plugin
         trace                0.30.8 enabled    dokku core trace plugin
         traefik-vhosts       0.30.8 enabled    dokku core traefik-vhosts plugin
=====> plausible app-json information
       App json computed selected:    app.json
       App json global selected:      app.json
       App json selected:
=====> plausible app information
       App created at:<PHONE_NUMBER>
       App deploy source:             docker-image
       App deploy source metadata:    dokku/master-9993a7a4-1688631847
       App dir:                       /home/dokku/plausible
       App locked:                    true
=====> plausible builder information
       Builder build dir:
       Builder computed build dir:
       Builder computed selected:
       Builder global build dir:
       Builder global selected:
       Builder selected:
=====> plausible builder-dockerfile information
       Builder dockerfile computed dockerfile path: Dockerfile
       Builder dockerfile global dockerfile path: Dockerfile
       Builder dockerfile dockerfile path:
=====> plausible builder-herokuish information
       Builder herokuish computed allowed: true
       Builder herokuish global allowed: true
       Builder herokuish allowed:
=====> plausible builder-lambda information
       Builder lambda computed lambdayml path: lambda.yml
       Builder lambda global lambdayml path: lambda.yml
       Builder lambda lambdayml path:
=====> plausible builder-pack information
       Builder pack computed projecttoml path: project.toml
       Builder pack global projecttoml path: project.toml
       Builder pack projecttoml path:
=====> plausible buildpacks information
       Buildpacks computed stack:     gliderlabs/herokuish:latest-20
       Buildpacks global stack:
       Buildpacks list:
       Buildpacks stack:
=====> plausible caddy information
       Caddy image:                   lucaslorentz/caddy-docker-proxy:2.7
       Caddy letsencrypt email:
       Caddy letsencrypt server:      https://acme-v02.api.letsencrypt.org/directory
       Caddy log level:               ERROR
       Caddy polling interval:        5s
       Caddy tls internal:            false
=====> plausible ssl information
       Ssl dir:                       /home/dokku/plausible/tls
       Ssl enabled:                   true
       Ssl hostnames:                 pa.jessedobbelae.re
       Ssl expires at:                Sep 21 21:00:24 2023 GMT
       Ssl issuer:                    C = US, O = Let's Encrypt, CN = R3
       Ssl starts at:                 Jun 23 21:00:25 2023 GMT
       Ssl subject:                   subject=CN = pa.jessedobbelae.re
       Ssl verified:                  self signed
=====> plausible checks information
       Checks disabled list:          none
       Checks skipped list:           none
       Checks computed wait to retire: 60
       Checks global wait to retire:  60
       Checks wait to retire:
=====> plausible cron information
       Cron task count:               0
=====> plausible docker options information
       Docker options build:          --link dokku.clickhouse.plausible-db:dokku-clickhouse-plausible-db --link dokku.postgres.plausible-db:dokku-postgres-plausible-db
       Docker options deploy:         --link dokku.clickhouse.plausible-db:dokku-clickhouse-plausible-db --link dokku.postgres.plausible-db:dokku-postgres-plausible-db --restart=on-failure:10 -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-City.mmdb:/geoip/GeoLite2-City.mmdb -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-Country.mmdb:/geoip/GeoLite2-Country.mmdb
       Docker options run:            --link dokku.clickhouse.plausible-db:dokku-clickhouse-plausible-db --link dokku.postgres.plausible-db:dokku-postgres-plausible-db -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-City.mmdb:/geoip/GeoLite2-City.mmdb -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-Country.mmdb:/geoip/GeoLite2-Country.mmdb
=====> plausible domains information
       Domains app enabled:           true
       Domains app vhosts:            pa.jessedobbelae.re
       Domains global enabled:        true
       Domains global vhosts:         dokku.me
=====> plausible git information
       Git deploy branch:             master
       Git global deploy branch:      master
       Git keep git dir:              false
       Git rev env var:               GIT_REV
       Git sha:                       641e4de
       Git source image:              dokku/master-81bd2795-1689765780
       Git last updated at:<PHONE_NUMBER>
=====> plausible haproxy information
       Haproxy image:                 byjg/easy-haproxy:4.3.0
       Haproxy letsencrypt email:
       Haproxy letsencrypt server:    https://acme-v02.api.letsencrypt.org/directory
       Haproxy log level:             ERROR
=====> plausible logs information
       Logs computed max size:        10m
       Logs global max size:          10m
       Logs global vector sink:
       Logs max size:
       Logs vector sink:
=====> plausible network information
       Network attach post create:
       Network attach post deploy:
       Network bind all interfaces:          false
       Network computed attach post create:
       Network computed attach post deploy:
       Network computed bind all interfaces: false
       Network computed initial network:
       Network computed tld:
       Network global attach post create:
       Network global attach post deploy:
       Network global bind all interfaces:   false
       Network global initial network:
       Network global tld:
       Network initial network:
       Network static web listener:
       Network tld:
       Network web listeners:                <IP_ADDRESS>:8000
=====> plausible nginx information
       Nginx access log format:
       Nginx access log path:         /var/log/nginx/plausible-access.log
       Nginx bind address ipv4:
       Nginx bind address ipv6:       ::
       Nginx client max body size:
       Nginx disable custom config:   false
       Nginx error log path:          /var/log/nginx/plausible-error.log
       Nginx global hsts:             true
       Nginx computed hsts:           true
       Nginx hsts:
       Nginx hsts include subdomains: true
       Nginx hsts max age:            15724800
       Nginx hsts preload:            false
       Nginx computed nginx conf sigil path: nginx.conf.sigil
       Nginx global nginx conf sigil path: nginx.conf.sigil
       Nginx nginx conf sigil path:
       Nginx proxy buffer size:       4096
       Nginx proxy buffering:         on
       Nginx proxy buffers:           8 4096
       Nginx proxy busy buffers size: 8192
       Nginx proxy read timeout:      60s
       Nginx last visited at:<PHONE_NUMBER>
       Nginx x forwarded for value:   $proxy_add_x_forwarded_for
       Nginx x forwarded port value:  $server_port
       Nginx x forwarded proto value: $scheme
       Nginx x forwarded ssl:
=====> plausible proxy information
       Proxy enabled:                 true
       Proxy port map:                http:80:8000 https:443:8000
       Proxy type:                    nginx
=====> plausible ps information
       Deployed:                      true
       Processes:                     1
       Ps can scale:                  true
       Ps computed procfile path:     Procfile
       Ps global procfile path:       Procfile
       Ps procfile path:
       Ps restart policy:             on-failure:10
       Restore:                       true
       Running:                       true
       Status web 1:                  running (CID: 6d1b484dedc)
=====> plausible registry information
       Registry computed image repo:      dokku/plausible
       Registry computed push on release: false
       Registry computed server:
       Registry global push on release:
       Registry global server:
       Registry image repo:
       Registry push on release:
       Registry server:
       Registry tag version:
=====> plausible resource information
=====> plausible scheduler information
       Scheduler computed selected:   docker-local
       Scheduler global selected:     docker-local
       Scheduler selected:
=====> plausible scheduler-docker-local information
       Scheduler docker local disable chown:
       Scheduler docker local init process: true
       Scheduler docker local parallel schedule count:
=====> plausible storage information
       Storage build mounts:
       Storage deploy mounts:         -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-City.mmdb:/geoip/GeoLite2-City.mmdb -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-Country.mmdb:/geoip/GeoLite2-Country.mmdb
       Storage run mounts:            -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-City.mmdb:/geoip/GeoLite2-City.mmdb -v /var/lib/dokku/data/storage/plausible/geoip/GeoLite2-Country.mmdb:/geoip/GeoLite2-Country.mmdb
=====> plausible traefik information
       Traefik api enabled:           false
       Traefik api vhost:             traefik.dokku.me
       Traefik basic auth password:
       Traefik basic auth username:
       Traefik dashboard enabled:     false
       Traefik image:                 traefik:v2.8
       Traefik letsencrypt email:
       Traefik letsencrypt server:    https://acme-v02.api.letsencrypt.org/directory
       Traefik log level:             ERROR
       Traefik priority:

This is required! Issues missing this information may be closed.
For problems affecting all applications, the report output for a broken application is useful for our debugging.
In these cases, you may run dokku report without any arguments to display the top-level reporting information.

How (deb/make) and where (AWS, VirtualBox, physical, etc.) was Dokku installed?:
Additional information

App container inspect output (if applicable) via dokku ps:inspect APP_NAME
The nginx configuration (if applicable) via dokku nginx:show-config APP_NAME
Link to the exact repository being deployed (if possible/applicable):
Output of failing Dokku commands after running dokku trace:on
(BEWARE: trace:on will print environment variables for some commands, be sure you're not exposing any sensitive information when posting issues. You may replace these values with XXXXXX):

Hi @jessedobbelaere , thanks for the report!
The code you link to should actually only skip, if the key already exists and value for that key is the one you are trying to set.
Maybe you can open a pull request adding a failing test to https://github.com/dokku/ansible-dokku/blob/master/molecule/default/verify.yml that demonstrates the problem you are seeing?

Right, it checks the value 🤦  I will do another test and see how to exactly reproduce it! Will close until I have a reproducible case


rerun after image removed cause error

Additional Information
Version of nydus being used (nydusd --version)
Version: 	v2.3.0
Git Commit: 	a4683baa1ea8cafecc9013bc4d6a5259b5a82678
Build Time: 	2024-10-11T02:51:53.888907475Z
Profile: 	release
Rustc: 		rustc 1.72.1 (d5c2e9c34 2023-09-13)
Version of nydus-snapshotter being used (containerd-nydus-grpc --version)
Version:     v0.14.0
Revision:    8e72f94dcad2d4898b6437fc84d6c9c36f78ea50
Go version:  go1.21.12
Build time:  2024-08-02T06:25:54
Kernel information (uname -r)
5.4.0-48-generic
GNU/Linux Distribution, if applicable (cat /etc/os-release)
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
containerd-nydus-grpc command line used, if applicable (ps aux | grep containerd-nydus-grpc)
root     53507  0.3  0.1 1272200 40048 ?       Ssl  16:34   0:01 /usr/local/bin/containerd-nydus-grpc --config /etc/nydus/config.toml
root     65231  0.0  0.0   6436  2416 pts/0    S+   16:43   0:00 grep --color=auto containerd-nydus-grpc

client command line used, if applicable (such as: nerdctl, docker, kubectl, ctr)
nerdctl --snapshotter nydus run --network=host -it --rm harbor.info/machine-learn/mpioperator_tensorflow-benchmarks:0.3.0-nydus bash

nerdctl image rm harbor.info/machine-learn/mpioperator_tensorflow-benchmarks:0.3.0-nydus

mount -l | grep rafs

Screenshots (if applicable)

Details about issue
After I remove the image and re-run the container, there are some problems with the new mount point. I could not get the content inside the file .
I try v2.2.5, it works well in this case.

@VincentLeeMax Hi, any error logs can be found in nydus snapshotter (containerd-nydus-grpc) process?

@VincentLeeMax Hi, any error logs can be found in nydus snapshotter (containerd-nydus-grpc) process?

No error logs found in nydus-snapshotter.log. The old mount point is /data/containerd/io.containerd.snapshotter.v1.nydus/snapshots/24/mnt, and the new mount point is /data/containerd/io.containerd.snapshotter.v1.nydus/snapshots/50/mnt.
Everytime this happen, I need to remove the whole $containerd_home to resume. It's horrible.

@VincentLeeMax Hi, I test in nyudsd v2.3.0 and containerd-nydus-grpc v0.14.0, but i can't reproduce your issue. Can you provide more infomations,? Such as your nerdctl containerd version and your config for nydusd.
It's even better if you can provide the image you used or can you test with ghcr.io/dragonflyoss/nydus/busybox:latest.

@Desiki-high Did you check some files in the mount point?

@Desiki-high Did you check some files in the mount point?

I checked the dir bin and the file in  etc.

这个问题可能和 chunk_map 文件清理有关，请尝试 nydus-snapshotter v0.15.0, 相关 PR：https://github.com/containerd/nydus-snapshotter/pull/619 cc @VincentLeeMax @Desiki-high


mock custom function

Hi,
Is there a way to mock custom functions?
something like
let(:pre_condition) { "define Puppet::Parser::Functions::my_func() {return 'hi'}" }
let(:pre_condition) { "define my_func() {return 'hi'}" }
Thx, Avihay

Nowadays you can stub functions directly on the scope property. There's an example in the README.

This is still not possible for manifests, however.

It should be possible in a before(:each){} block?

Does rspec-puppet-utils help at all? I recently updated it to work better with argument matchers.

@DavidS it is not possible in a manifest spec. https://github.com/binford2k/binford2k-node_encrypt/blob/master/spec/defines/file_spec.rb#L18-L31

This is biting me now as well, any go back on this?

Reminder, this is mostly covered by #310 (which has a more implementation focused slant), and can be worked around by adding the mocked functions to fixture modules, or directly defining them in code like https://github.com/binford2k/binford2k-node_encrypt/blob/master/spec/defines/file_spec.rb#L18-L31 does.

Namespaced V4 API ruby functions can be mocked using let(:pre_condition) and a mock function written in the puppet language.  eg see https://github.com/voxpupuli/puppet-archive/pull/323/files#r174119645
Thanks to @TraGicCode for suggesting this.


Image compressed unnecessarily

The input is being copied to target.jpg, but by doing this, the file is being compressed unnecessarily, adding compression artifacting: https://github.com/brian-yang/table-parser-opencv/blob/master/main.py#L27
Should I open a PR to change it to target.png?

Yes, that would definitely be helpful! I can merge the change in ASAP once the pr's made. Thanks!


refactor(functions): yaml indentation

Align yaml like other yaml files: https://cloud.google.com/functions/docs/testing/test-cicd#setting_up
No functional change.

Will use a fork next time.

A branch on this repo is totally fine with me! Whatever you prefer.


insurance_leads.rb

FILE
ActiveAdmin.register InsuranceLead do
  config.sort_order = 'created_at_desc'

  menu parent: 'Loans & Insurance', label: 'Insurance Leads'
  # See permitted parameters documentation:
  # https://github.com/activeadmin/activeadmin/blob/master/docs/2-resource-customization.md#setting-up-strong-parameters
  #
  # Uncomment all parameters which should be permitted for assignment
  #
  permit_params :email, :date_of_birth, :premium, :first_name, :last_name, :gender, :phone_number, :country
  #
  # or
  #
  # permit_params do
  #   permitted = [:email, :date_of_birth, :premium, :first_name, :last_name, :gender, :phone_number, :country]
  #   permitted << :other if params[:action] == 'create' && current_user.admin?
  #   permitted
  # end

end

This ActiveAdmin configuration sets up an admin interface for managing InsuranceLead records. Here’s a breakdown of the key components:
1. Menu Configuration

menu parent: 'Loans & Insurance', label: 'Insurance Leads':
This line places the InsuranceLead resource under the "Loans & Insurance" menu in the ActiveAdmin interface and labels it as "Insurance Leads." This categorization helps administrators easily find and manage insurance-related leads within the broader context of financial services.

2. Sort Order Configuration

config.sort_order = 'created_at_desc':
This sets the default sort order for InsuranceLead records to be by created_at in descending order. This ensures that the most recently created leads appear at the top of the list, which is often useful for administrators when reviewing new leads.

3. Permitted Parameters

permit_params :email, :date_of_birth, :premium, :first_name, :last_name, :gender, :phone_number, :country:
This line defines the attributes of the InsuranceLead model that can be modified through the ActiveAdmin interface. By explicitly permitting these parameters, it ensures that only the necessary and safe attributes can be altered by administrators.
The permitted parameters include:

email: The email address of the insurance lead.
date_of_birth: The lead's date of birth, likely relevant for calculating insurance premiums.
premium: The insurance premium associated with the lead.
first_name: The first name of the lead.
last_name: The last name of the lead.
gender: The gender of the lead.
phone_number: The phone number of the lead.
country: The country of residence of the lead.

4. Commented Configuration

# permit_params do ... end:
This commented-out block shows an alternative way to define permitted parameters. It demonstrates how to conditionally permit additional parameters based on the context, such as the action being performed or the role of the user.
permit_params do
  permitted = [:email, :date_of_birth, :premium, :first_name, :last_name, :gender, :phone_number, :country]
  permitted << :other if params[:action] == 'create' && current_user.admin?
  permitted
end

This is useful if you need more control over which parameters can be assigned, depending on specific conditions.

Summary
This configuration is designed to manage InsuranceLead records efficiently. It places the resource within a relevant menu, ensures new leads are easily accessible, and securely defines which attributes can be modified by administrators. The flexibility provided by the commented-out block allows for advanced configurations if necessary, such as conditional parameter permissions.


NOTARY_VERSION_ERROR is wrong and NOTARY_COMMAND_ERROR is undocumeted

I tried running az acr check-health --name myregistry and it first failed with NOTARY_COMMAND_ERROR  which is not documented on the site. What you need to do is go to https://github.com/notaryproject/notary/releases, download notary-Windows-amd64.exe, put it in the PATH and rename it to just notary.exe or it won't be found by az.
Then I got the error NOTARY_VERSION_ERROR:
"Current notary version is not recommended. Please downgrade your notary client to version 0.6.0."
The documentation says to use a previous version from 0.6.0 though. So I followed the documentation instead, which resulted in the error: An error occurred: NOTARY_VERSION_ERROR
"Current notary version is not recommended. Please upgrade your notary client to version 0.6.0."
So it looks like only 0.6.0 works, contrary to the docs.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 63a4a6d7-2f3b-1f3a-fbfe-f10b55a35c03
Version Independent ID: 8acdf23c-1d6b-0734-a345-a7a1863440e1
Content: Error reference for registry health checks - Azure Container Registry
Content Source: articles/container-registry/container-registry-health-error-reference.md
Service: container-registry
GitHub Login: @dlepow
Microsoft Alias: danlep

@ChristopherManthei Thank you for your feedback!  We've assigned this to the document author for further review.

Reassigning to current writer
#assign: @tejaswikolli-web

In my case, notary was not installed. I thought it was coming with docker but does not look the case anymore (both on Linux and Docker for Windows).

Thank you for feedback on our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner, and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary.
#please-close

Requesting reopen


WantedBy = default.target

The multi-user.target doesn't exist in user-level systemd services (try systemctl --user -t target), so this service will never be activated. Instead this should be WantedBy = default.target, I think.

Yes, my bad.
Thank you


Build issue while building atkmm

I have trouble building gtkmm:
Computing installation plan...
The following packages will be built and installed:

atkmm[core]:x64-windows
bzip2[core]:x64-windows
cairo[core]:x64-windows
cairomm[core]:x64-windows
dirent[core]:x64-windows
expat[core]:x64-windows
fontconfig[core]:x64-windows
freetype[bzip2,core,png]:x64-windows
gdk-pixbuf[core]:x64-windows
gtk[core]:x64-windows
gtkmm[core]:x64-windows
harfbuzz[core,glib,ucdn]:x64-windows
libepoxy[core]:x64-windows
libpng[core]:x64-windows
pango[core]:x64-windows
pangomm[core]:x64-windows
pixman[core]:x64-windows
ragel[core]:x64-windows
tool-meson[core]:x64-windows
Additional packages (*) will be modified to complete this operation.
Starting package 1/19: atkmm:x64-windows
Building package atkmm[core]:x64-windows...
-- Using cached D:/software/vcpkg/downloads/atkmm-2.24.2.tar.xz
-- Using source at D:/software/vcpkg/buildtrees/atkmm/src/atkmm-2-de808b48b0
-- Building D:/software/vcpkg/buildtrees/atkmm/src/atkmm-2-de808b48b0/MSVC_Net2013/atkmm.sln for Release
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message):
Command failed: msbuild D:/software/vcpkg/buildtrees/atkmm/src/atkmm-2-de808b48b0/MSVC_Net2013/atkmm.sln /p:Configuration=Release /t:atkmm /p:Platform=x64 /p:PlatformToolset=v142 /p:VCPkgLocalAppDataDisabled=true /p:UseIntelMKL=No /p:WindowsTargetPlatformVersion=10.0.18362.0 /m /p:ForceImportBeforeCppTargets=D:/software/vcpkg/scripts/buildsystems/msbuild/vcpkg.targets /p:VcpkgTriplet=x64-windows
Working Directory: D:/software/vcpkg/buildtrees/atkmm/x64-windows-rel
Error code: 1
See logs for more information:
D:\software\vcpkg\buildtrees\atkmm\build-x64-windows-rel-out.log

Call Stack (most recent call first):
scripts/cmake/vcpkg_build_msbuild.cmake:119 (vcpkg_execute_required_process)
ports/atkmm/portfile.cmake:25 (vcpkg_build_msbuild)
scripts/ports.cmake:90 (include)
Error: Building package atkmm:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: atkmm:x64-windows
Vcpkg version: 2020.02.04-nohash
Additionally, attach any relevant sections from the log files above.

@licon  please attach the D:\software\vcpkg\buildtrees\atkmm\build-x64-windows-rel-out.log file so the issue can be looked at.

Pinging... @licon

Thanks for posting this issue. Please reopen this issue if this is still a problem for you.


Data movement workload

Put description here...
Code-Reviewer Section
The general guidelines can be found here.
Please check each of the following things and check all boxes before accepting a PR.

[ ] The PR has a description, explaining both the problem and the solution.
[ ] The description mentions which forms of testing were done and the testing seems reasonable.
[ ] Every function/class/actor that was touched is reasonably well documented.

For Release-Branches
If this PR is made against a release-branch, please also check the following:

[ ] This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or master if this is the youngest branch)
[ ] There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

I've committed this through a different path.


Text for the latest rows doesn't show correctly

Hello,
when i tried the example code in iphone 7 plus the text for the last row doesn't show correctly, it's kinda shifted to right, so i was wondering how i can resolve this.
another thing if i want to add an image how this can be done ?
Thanks a lot.

thanks for reporting the issues.
Regarding to the first one, which iOS and XCode version are you using? (Currently, this lib still works fine on swift 3, iOS 10.2, xcode 8.2.1)

Thanks for reporting the issue.
Could you let me know which iOS version and swift version you are using? I have just rechecked the issue on ip7 plus iOS 10.2 and swift 3.0, it still works fine at my end.
Regarding to your question, you mean that you wanna use an image instead of black curve background, right? If yes, unfortunately, the currently doesn't support this feature. Anw, if you can show me the design that you expected, I will schedule to implement it if possible.

Hello Thanks for your reply,
I'm using Xcode 8.3.1 swift 3.1, this issue raised on iphone7 plus simulator, please try to write two words rather than one in the last cell.
And for my question, I'm sorry I didn't explain it right, I want to add an image to the cell, before the text, as it doesn't appear when i added one.
Thanks a lot.


Cannot Run on Windows 10 (Lenovo G40)

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

No information provided or steps to replicate - Closing 😞


[nrf fromtree] Support for low power mode in UARTE driver

From https://github.com/zephyrproject-rtos/zephyr/pull/30693 and https://github.com/zephyrproject-rtos/zephyr/pull/33708

Upmerge went first.


increased thread safety of establish_conda_env.sh

Pull Request Description
What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)
Closes #754.
What are the significant changes in functionality due to this change request?
establish_conda_env.sh only writes to .ravenrc when running under "install" mode, not during "load" mode.
Also moves to using bash natively instead of python to read .ravenrc to support older python on some systems.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[ ] 1. Review all computer code.
[ ] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[ ] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[ ] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[ ] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[ ] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[ ] 8. If an analytic test is changed/added is the the analytic documentation updated/added?

Job Mingw Test on efce052 : invalidated by @PaulTalbot-INL
checking repeatability of failure, there are "/" missing in the Windows conda path

Job Test mac on 4fc71a6 : invalidated by @PaulTalbot-INL

Job Mingw Test on 4fc71a6 : invalidated by @wangcj05
FAIL \tests\framework\CodeInterfaceTests.RAVENrunningRAVEN FAILED (TIMEOUT) [FINISHED]

Hm, it's odd this test is failing from timeout regularly; on the contrary, the changes should have sped up RrR run times, not slowed them down, although they should barely have affected it at all.

This has redundantly been included in #770, so closing here.


Azure pipelines setup

Working on setting up CI with Azure Pipelines

Codecov Report

Merging #190 into master will decrease coverage by 0.41%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #190      +/-   ##
==========================================
- Coverage   93.49%   93.08%   -0.42%
==========================================
  Files          25       25
  Lines        3413     3413
==========================================
- Hits         3191     3177      -14
- Misses        222      236      +14

Impacted Files
Coverage Δ

tslearn/tests/sklearn_patches.py
89.55% <0%> (-1.9%)
:arrow_down:

tslearn/datasets.py
88.69% <0%> (-1.74%)
:arrow_down:

tslearn/tests/test_estimators.py
88.09% <0%> (-1.2%)
:arrow_down:

tslearn/utils.py
93.41% <0%> (-0.95%)
:arrow_down:

tslearn/shapelets.py
93.82% <0%> (-0.78%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6f25e0...772ec55. Read the comment docs.

So there are several issues while trying tests,

Python 2.7 the dependencies (and in particular numba) won't install, because of version mismatch https://github.com/rtavenar/tslearn/issues/191 I'm not sure it's worth spending time on this, dropping python 2.7 might be a solution.
Python 3.5: some tests are very slow in CI. Locally tests run for me in a minute or so, while in CI some individual tests timeout after 90s. I believe this is related to TensorFlow, maybe due to CPU oversubscription: in other CI the detected number of CPU used to be much larger than the number of available CPU i.e  2. This is usually addressed by setting environement variables, but it looks like Tensorflow 1.15  dropped support for OMP_NUM_THREADS. This needs more investigation, meanwhile maybe

I'm planning to add other OSs once Linux test pass.

Hi @rth
Thanks for this. Yeah I think for the moment you can move forward with Python 2 disabled and skip ShapeletModel-related tests. I can try to have a deeper look at the specific problem related to TF.

Some of the tests in check_estimators (especially those with ShapeletModels as @rtavenar pointed out) are indeed rather slow. This is because the default hyper-parameters of ShapeletModel will have a long runtime (e.g. max_iter of 10000...). Perhaps we should look to extend it with early stopping or smth like that, as it converges quickly on the benchmark datasets.

This is because the default hyper-parameters of ShapeletModel will have a long runtime (e.g. max_iter of 10000...).

Thanks! I see, maybe it could be worth setting it to a lower value in tests similarly to what is done in https://github.com/scikit-learn/scikit-learn/blob/c3969f43d9e4ba66d7f211d3835bb4cbe616c12b/sklearn/utils/estimator_checks.py#L481
For now I will skip them.

OK so I skipped ShapeletModel tests for now and all tests run in less than 4 min. We can add wheel building (and Python 3.8 https://github.com/rtavenar/tslearn/issues/193) in a follow up PR. Current setup is a bit repetitive but it's a start.

That is a great idea, and that reminded me that we actually tried to do something similar:
https://github.com/rtavenar/tslearn/blob/master/tslearn/tests/test_estimators.py#L168
But it seems to not be working as it should?

that reminded me that we actually tried to do something similar:

Indeed, I opened https://github.com/rtavenar/tslearn/issues/194 to discuss how to do that without redefining check_estimator, but generally yes that should have worked.

OK, so the nice thing (correct me if I'm wrong) is that here you discarded Shapelet tests only for Azure pipelines, so we can still use Travis in parallel for the moment until we have found a fix for Shapelet timeouts, right?
If this is true, I would merge and we can have a look at that #194 you opened.

Hello @rth! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file tslearn/tests/test_utils.py:

Line 105:1: E302 expected 2 blank lines, found 1

In the file tslearn/utils.py:

Line 808:5: E722 do not use bare 'except'
Line 818:1: E302 expected 2 blank lines, found 1
Line 861:5: E722 do not use bare 'except'
Line 891:1: E302 expected 2 blank lines, found 1
Line 931:5: E722 do not use bare 'except'
Line 941:1: E302 expected 2 blank lines, found 1
Line 983:5: E722 do not use bare 'except'
Line 1006:1: E302 expected 2 blank lines, found 1
Line 1037:5: E722 do not use bare 'except'
Line 1055:1: E302 expected 2 blank lines, found 1
Line 1101:5: E722 do not use bare 'except'
Line 1186:1: E302 expected 2 blank lines, found 1
Line 1220:5: E722 do not use bare 'except'

would prefer that we skipped only pandas-related conversion functions (ie. those involving sktime, pyflux and tsfresh) only. Maybe we could split the tests into two functions (one with pandas required, one that should pass even without pandas)?

OK, I split that test in multiple functions depending on dependencies.
I would be good to merge this, if only to avoid CI failures in other PRs.

OK, I added a  conftest.py to skip doctests that require optional dependencies (as it is done in scikit-learn). Hopefully that would fix Travis.
For now Azure doesn't run doctests indeed, I would rather add it in a follow up PR. For one thing, it would also be good to move the doc/ folder one level up, to avoid excluding it from tests.

What's the difference between dev and master branches BTW? This is a PR into master, should it be done to dev instead?. I guess the naming of branches doesn't matter much, but it would be good to have the default branch on github to be the one in active development, so that PRs are made to the correct one..

What's the difference between dev and master branches BTW? This is a PR into master, should it be done to dev instead?. I guess the naming of branches doesn't matter much, but it would be good to have the default branch on github to be the one in active development, so that PRs are made to the correct one..

You're right. What we have right now is:

master is the branch from which wheels, conda packages, and docs are built
dev is a branch used to prepare next release (contains new features to appear, etc)


setup_requires nose in setup.py

We host our python dependencies in an internal repository, and this section in the setup.py:
setup_requires=[
'nose'
]
forces it to look at the pypi website, ignoring the --index-url provided.
Is it possible to just add it to the requires.txt and exclude it from setup.py?
Thank you for a superb library, it works beautifully.

No update here in a while, assuming the workaround is sufficient.


Improves readability for specific error messages

reference: https://github.com/standardhealth/shr-models/pull/17
This is a more readable option for choice values in errors (in response to feedback from Mark).
Turns:

[19:25:20.965Z] ERROR shr: No cardinality found for field: ChoiceValue<IdentifiableValue<shr.core.CodeableConcept>|RefValue<shr.core.Range>|IdentifiableValue<shr.core.Quantity>>. ERROR_CODE:12004 (module=shr-expand, shrId=shr.careplan.ResultTargeted)

into:

[19:19:23.316Z] ERROR shr: No cardinality found for field Choice(CodeableConcept, Range, Quantity) in ResultTargeted. ERROR_CODE:12004 (module=shr-expand, shrId=shr.careplan.ResultTargeted)

(with the addition of some base class tracking improvements by @cmoesel).

Unnecessary after https://github.com/standardhealth/shr-models/commit/b176e7cb405d5bb6177cdcec31e2da13c3e3463e


Nature's Minerals Compatability

pretty neat mod for tinkers but its now fabric only (also uses alloy forgery)

Done. Will be making a release in the following days.


Distribution includes d3 functions, which get duplicated (& maybe version skew) when sites also include d3.js

I think flamegraph/dist bundles a copy of many d3 functions, increasing page weight and risking version skew.
To Reproduce

Open<EMAIL_ADDRESS>Search for node_modules/d3-
note 139 matches, including (a selection of files from top-level modules):

node_modules/d3-selection/src/selector.js
node_modules/d3-format/src/exponent.js
node_modules/d3-hierarchy/src/partition.js
node_modules/d3-array/src/ticks.js
node_modules/d3-color/src/define.js
node_modules/d3-interpolate/src/basis.js
node_modules/d3-scale/src/init.js
node_modules/d3-ease/src/cubic.js
node_modules/d3-dispatch/src/dispatch.js
node_modules/d3-timer/src/timer.js
node_modules/d3-transition/src/interrupt.js

I think this is true of the minified JS (but to a lesser extent because unused functions will be removed). I'm just using the unminified JS as it clearly demonstrates the files included.
Expected behavior

No d3 functions included, because:

The expectation is on the user to include d3.js separately on the webpage, and we want just one version of each d3 function included (to reduce page weight).
When used in npm, importing d3-flame-graph imports the dist version, which might (not sure) prevent npm from deduplicating dependencies (say, if your project depends on d3-select and d3-flame-graph, you'll get two versions of d3-select in the output).

I wonder if this might lead to version skew sometimes if d3-flame-graph bundles one version of d3's functions and the site includes another? I suppose this is unavoidable in general; d3-flame-graph calls d3 functions, and expects some set of functions to be there.
I'm not sure the best way to solve this. Here's some brainstorming:

Maybe in npm, don't ship a dist version, but rather ship the raw source, depending on d3 libraries? This would prevent duplicating in npm builds. But if we're serving straight from npm to websites, this might break the websites.
Maybe we could keep the dist folder for websites, but for npm imports, don't use the dist version? Like in the top-level index.js, don't export all of dist/.

I should add, I don't know that this is an urgent issue or anything (nothing seems to be really 'breaking' yet!), just thought I'd file it to track in case this was unexpected.

Thanks for opening the issue @mhansen! This is likely related to a bundling configuration we might have missed during the migration to Webpack, where we need to specify packages that are available already, so they don't get bundled and duplicated.
No reports of issues so far (and assuming the same major version is being used, should be fine), but we shouldn't be shipping duplicated code. I'll check the build configs to see if there's anything missing.


Correct ES6 export

Currently Moment.js ES6 export is not really an ES6 export, it's just an older ES5 export rewritten with imports and exports.
The goal of ES6 is not using different words but rather exporting different parts of the package independently.
This PR shows the right way of doing an ES6 export for a package.
It is meant as an illustration which can be merged if the moment internals allow that.
Otherwise it's 180 KiloBytes which is unacceptable and will eventually have to go.

Does this actually get you a size reduction? Internally, moment is using duration for tons of computations, so I cant think that this would actually reduce the size - duration would get picked up in it's entirety even without being explicitly loaded.
There's extensive discussion on several threads about how to reduce the size of the library. The short answer is that this isn't like lodash where every iterator has it's own function. There is tons of shared code, and it won't unwind easily. If we were to unwind the shared code, the full library would get even bigger.
The lowest-hanging fruit on the size reduction front is to pull out the custom parser, and leave only the ISO parser. I got half way through this and it did reduce size by 10%.

You can close this if you feel like it.
This PR is not stating that it can reduce the imported code amount: it's simply showing you that as it is right now your ES6 export is purposeless and misleading and could as well be removed at all because it doesn't bring anything new compared to the old ES5 export.
This PR is meant as an illustration of how the real ES6 export would look like – without any "export it all" objects.
In a real ES6 export each function must be exported individually, not as part of the whole.

Certainly refactoring the exports might make the API cleaner, but it's a substantial amount of work - which is why it hasn't been done yet.
To be clear, we are 100% open to contributions in this space. At this time though, the existing maintainer team for this project is at 100% capacity just handling issues and the immutability project we've already taken on. We're happy to add maintainers if we see great contributions!


feat: add custom search modal

Closes #530

the search modal will show automatically when open the the module docs page

oh whoops some debugging that was left on

the issue is a comrak import


Feature/robot scene

This PR makes some minor modifications to the robotscene tmplit:

Update README to improve clarity
Add explicit dependency on ThreeJS
Throw an error if callback has issues

Is this ready to merge?


click_timestamp is 3 hours in future?

We have integrated the Branch SDK into our project. In our BranchReferralInitListener, we are passing the referring params up to Mixpanel for tracking.
On the Mixpanel dashboard, I noticed the value of the click_timestamp attribute was<PHONE_NUMBER>. This value is documented to be an epoch time value, but it converts to 18:56:12 (UTC) and 13:56:12 (EST). Since it is currently 11:08:45 (EST), this value appears to be three hours in the future.
I have verified that my device (HTC One M7, Android 4.4.2) has the correct timezone (GMT-05:00, Eastern Standard Time) and correct time set.
What could be causing this discrepancy?

FYI, the SDK version we're is 'io.branch.sdk.android:library:1.10.5'

Hey @efung - sorry for the confusion! That click time stamp is in UTC, as are all raw dates stored in our database. I hope this helps.

Right, as epoch timestamps are. But interpreted as UTC, the converted timestamp was three hours ahead.


fix: no version field

Hi! Faced with such problem:
error loading plugin schemas: on plugin 'py-geocode': Plugin "py-geocode" cannot be loaded because its VERSION field does not follow the "x.y.z" format, got: "nil".

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

This is the exact change I hotfixed in my Kong deployment.


Partial incompatibility with vscodevim extension

OS: macOS 10.13.3
Opened VS Code, uninstalled all extensions, clicked "Reload"
Uninstalled VS Code using AppCleaner
Downloaded/installed VS Code, version 1.22.2 (1.22.2), 3aeede733d9a3098f7b4bdc1f66b63b0f48c1ef9
Opened a new file, entered some Markdown:

# This is a heading

This is some text

## This is another heading

This is some more text

Saved the file: ~/Desktop/file.md
Installed extension: vscodevim 0.11.3, clicked "Reload"
In file.md

Moved cursor to top of file
Pressed esc to enter vim command mode
Pressed / to start a search
Typed This as the search value
Hit enter to complete the search text
Hit n multiple times to cycle through search results
Worked as expected

Installed "Docs Authoring Pack" extension, clicked "Reload." This installed:

DocFX 0.1.1
Docs Authoring Pack 0.0.1
docs-markdown 0.0.1
markdownlint 0.14.1

In file.md

Moved cursor to top of file
Pressed esc to enter vim command mode
Pressed / to start a search
Typed This as the search value
Hit enter to complete the search text

Instead of "completing" the search, enter inserts a newline in the file at the cursor's location (unexpected)
Since the search text hasn't been completed, typing n just appends "n" to the search text (rather than moving to the next search result as expected)

Change filetype of document to plaintext (in lower-right corner of VS Code)

vim search works as anticipated

Reverted document type to Markdown
Disabled Docs Authoring Pack extension (only that extension, not its dependencies), clicked "Reload"

vim-style searching still broken

Disabled docs-markdown extension, clicked "Reload"

vim-style searching works as expected

Can confirm .. same behaviour here.

Thanks @bradumbaugh and @mpepping for the information.  I'll start looking into this now.

I think I know what's causing this.  The docs-markdown extension uses "enter" for the automaticList command.  We should be able to resolve this issue by checking to see if vim is active i.e. && !vim.active in keybindings.  I've tested this out and it appears to work as expected.  Will submit a PR and schedule this for the 0.0.3 release.

Hi @bradumbaugh , @mpepping .  The fix for this was deployed with yesterday's release.  Please don't hesitate to let us know if you run into any other issues.  Thanks

Thanks @jamarw! Appreciate the help!


Pass custom headers

Our image provider requires a custom header "API-KEY" to be set to a certain value.
Is there a way to do that?

The feature was released in the pro version of v2.10.0


Use requests.session for performance

By using a persistent requests.session, requests can use Keep-Alives, resulting in better performance, when the client is doing repeated requests to Consul.
When testing the ansible consul_io inventory script, it takes ~12 seconds (because it does 774 HTTP requests!) to hit all nodes without this, and about ~8 seconds with this, which is a pretty decent speedup.
Cc: @sgargan, @sudarkoff, @aconrad

So I've been thinking about this a little more... doesn't having persistent connections defeats the purpose of load balancing the requests across many nodes? Or maybe we should have a slow TTL which would force to reconnect after say, 1 minute or so?
I don't know, just a random thought I had.

@aconrad: Yeah this makes you hit the same Consul server for the same session, which reduces availability a bit in the small chance that the Consul server fails.
However, saving the HTTP connection setup and teardown results in a big performance boost, because when we invoke the ansible consul inventory script, it results in 774 (!) HTTP requests to Consul, which means it takes ~12 seconds to complete. With this change, it's reduced to around 8 seconds.

@msabramo can you reason about what the 774 requests are for? Do HTTP requests scale up and down w/ inventory size?

@davidbirdsong: Yes the requests scale with inventory size because the way this plugin is written, it queries /catalog/node/<node> for each node.
We have 200 nodes in our inventory.
We have:

201 /catalog/node/<node> requests.
374 /kv/ lookups
601 /health/ lookups.

I'm sure the inventory script can use a lot of optimization and I plan to attack that as well but the keep alives seem to help a bit too and that was easier to do.

If I comment out settings in consul.ini for a few optional features, then I can get it down to a more understandable 201 requests.
# looks up the node name at the given path for a list of groups to which the
# node should be added.
# kv_groups=ansible/groups

# looks up the node name at the given path for a json dictionary of metadata that
# should be attached as metadata for the node
# kv_metadata=ansible/metadata

# looks up the health of each service and adds the node to 'up' and 'down' groups
# based on the service availibility
# availability = true
# available_suffix = _up
# unavailable_suffix = _down

Then we have just:

1 /catalog/node request so it can get the list of all nodes
200 /catalog/node/<node> requests (one request per node)
201 total requests.

Hi thanks for this. This is great. Apologies for the delay in getting it merged. I've been swamped in marriage preparations the last month.

Tagged a pushed as v0.4.5


Sprite End leaves the sprite on screen

Kernel bug

When sprite updated just checking for newStatus <> 0, should be checking either status non-zero (active) or new status non-zero (activating)


Build Command

The command in the "build.sh" file and in the one in the comments at the top of the "cpp" file are different. Only the former works for me on my Pi1 with libcec version 3.0.1. - have yet to try it on my Pi2.

Fixed.
I've harmonized the build commands.  I've also fixed a compile issue.
Could you git pull and let me know of any issues?
Thanks,
Geoff

Had a quick look, and:
 (1) "-ldl" still seems to be missing from the version in the .cpp

comments. Perhaps erroneously, I had attributed my compilation error to this
  (2) "-g4.8" is now missing from both - no idea whether this is

important, but I think it is needed for compiling libcec itself.
but sorry I may not be in a position to try a new version for a couple
of days.
Regards,
EMJB
On 31/07/2015 10:54, DrGeoff wrote:

Fixed.
I've harmonized the build commands. I've also fixed a compile issue.
Could you git pull and let me know of any issues?
Thanks,
Geoff
—
Reply to this email directly or view it on GitHub
https://github.com/DrGeoff/cec_simplest/issues/1#issuecomment-126630117.

Further to my previous email, have managed to give the build file a
quick try with the old cpp file- the "-4.8" appears to be optional, but
the "-ldl" to be essential when run on a Pi1, but haven't had a chance t
try on a Pi2 yet - have some other problems on that getting in the way.
Regards,
EMJB
On 31/07/2015 10:54, DrGeoff wrote:

Fixed.
I've harmonized the build commands. I've also fixed a compile issue.
Could you git pull and let me know of any issues?
Thanks,
Geoff
—
Reply to this email directly or view it on GitHub
https://github.com/DrGeoff/cec_simplest/issues/1#issuecomment-126630117.

Hi EMJB,
I've pushed a new build.sh that just grabs the example build command out of the cpp file.  The two can never be out of sync now.  And thanks for noting that one had -ldl and the other didn't.  Yes it is necessary and that is also fixed.
Best wishes,
Geoff

Thanks,
EMJB
On 01/08/2015 10:18, DrGeoff wrote:

Hi EMJB,
I've pushed a new build.sh that just grabs the example build command
out of the cpp file. The two can never be out of sync now. And thanks
for noting that one had -ldl and the other didn't. Yes it is necessary
and that is also fixed.
Best wishes,
Geoff
—
Reply to this email directly or view it on GitHub
https://github.com/DrGeoff/cec_simplest/issues/1#issuecomment-126889771.


add SLO migration

move migrate-slo-form to shared/components/migrate-slo and fix eslint…
fix checkboxs hiding check marks
move migrate-slo-form to shared/components/migrate-slo
add migrate button to entity nerdlet
handle slo migration, exclude migrated slos
change 'OR' to 'AND' in if stmt
add logic to disable non-migratable slos

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: ricegi:x: shahramkYou have signed the CLA already but the status is still pending? Let us recheck it.

/sk


Overlay files

This PR adds (most) of the files extracted from the various root overlay tarballs that exist at the moment. They are grouped functionally under data/files/ and the idea is to include them in the configuration as shown in data/csp/ec2/sle15/files.yaml.
What do you think?

I've added configurations for the overlay files. It's not 100% complete but will hopefully be useful to go ahead with the implementation in keg (if all agree).

A couple of things are sill missing from the overlay configuration, namely:

/etc/iscsi/iscsid.conf

This is only included in the following images:
SLES12-SP2-SAP-EC2-HVM-BYOS
SLES12-SP2-SAP-EC2-HVM
SLES12-SP2-EC2-HVM-BYOS
SLES12-SP3-EC2-SAPCAL-HVM
SLES12-SP2-SAP-GCE-BYOS
SLES12-SP2-SAP-GCE

So this is almost obsolete, only SLES12-SP3-EC2-SAPCAL-HVM is still inactive. Given that it's only included in some images for some platforms, it would be a bit tricky to integrate nicely. Do we need to care about this?

/opt/aws/bin/install-aws-agent.sh

Included in SLES 12 SAP images on EC2. Since it's only in EC2 and only in SAP it doesn't fit nicely in how the configuration is broken down. I would need to be in its own data module that is only included in the EC2 profile of the SAP image. As a side node, SLES15-SP1-SAPCAL includes the cloud.cfg directive to run this script but the script is not included. Either it needs to be added (if required on SAPCAP) or the cloud.cfg file should be dropped.

/etc/systemd/system/sapinit.service.d/type.conf

Only in SLES12-SP3-EC2-SAPCAL-HVM and SLES12-SP3-GCE-SAPCAL and as such also doesn't fit in nicely. I wonder why it's not in the Azure image though?

A couple of things are sill missing from the overlay configuration, namely:

/etc/iscsi/iscsid.conf

This is only included in the following images:
SLES12-SP2-SAP-EC2-HVM-BYOS
SLES12-SP2-SAP-EC2-HVM
SLES12-SP2-EC2-HVM-BYOS
SLES12-SP3-EC2-SAPCAL-HVM
SLES12-SP2-SAP-GCE-BYOS
SLES12-SP2-SAP-GCE

So this is almost obsolete, only SLES12-SP3-EC2-SAPCAL-HVM is still inactive. Given that it's only included in some images for some platforms, it would be a bit tricky to integrate nicely. Do we need to care about this?

Well me may not care for this particular file but we care about the use case, which is "a specific file for multiple providers but only for certain images"

/opt/aws/bin/install-aws-agent.sh

Included in SLES 12 SAP images on EC2. Since it's only in EC2 and only in SAP it doesn't fit nicely in how the configuration is broken down. I would need to be in its own data module that is only included in the EC2 profile of the SAP image. As a side node, SLES15-SP1-SAPCAL includes the cloud.cfg directive to run this script but the script is not included. Either it needs to be added (if required on SAPCAP) or the cloud.cfg file should be dropped.

Should be dropped from SAPCAL cloud.cfg. The agent is specifically to make things work nicely with the AWS maintained SAP qucik start. So the agent should be in all SAP images for EC2 and in all cloud.cfg files for SAP images in EC2.

/etc/systemd/system/sapinit.service.d/type.conf

Only in SLES12-SP3-EC2-SAPCAL-HVM and SLES12-SP3-GCE-SAPCAL and as such also doesn't fit in nicely. I wonder why it's not in the Azure image though?

SLES 12 SP3 based SAPCAL image is on it's way out and is no longer getting updates, we can ignore this.

Ok, so I'll simply remove modules systemd-sapinit and iscsid-conf since they are on the brink of obsolescence. As for the use case "specific to platform and product", as mentioned, this can be solved by adding a separate data module to be explicitly included in the appropriate profiles. AFAICS it's very rare (only the aws-agent at the moment really), so that approach should be serviceable IMO. I'll add a module for aws-agent.

Overlay configuration adjusted as discussed. I set a couple of keys to 'null' to overwrite them, I thought it looks a bit nicer than using an empty include list. This will require a small change to keg though. I'd open a PR for that if people are ok with the approach.

LGTM


gensio: 2.2.5 -> 2.2.7

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for gensio is: "General Stream I/O"
meta.homepage for gensio is: "https://sourceforge.net/projects/ser2net/"
meta.changelog for gensio is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/gsfn1x0j3rcpmy11v8k0y2aiplq95v53-gensio-2.2.7/bin/gensiot passed the binary check.

1 of 1 passed binary check by having a zero exit code.

0 of 1 passed binary check by having the new version present in output.

found 2.2.7 with grep in /nix/store/gsfn1x0j3rcpmy11v8k0y2aiplq95v53-gensio-2.2.7

found 2.2.7 in filename of file in /nix/store/gsfn1x0j3rcpmy11v8k0y2aiplq95v53-gensio-2.2.7

directory tree listing: https://gist.github.com/fbb7b069155481a0e8e9f0c57bd30166

du listing: https://gist.github.com/d73f3e402d1890105336bb7faa4aef1b

Rebuild report (if merged into master) (click to expand)

7 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
2 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
gensio
ser2net

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/gsfn1x0j3rcpmy11v8k0y2aiplq95v53-gensio-2.2.7 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A gensio https://github.com/r-ryantm/nixpkgs/archive/b819b82ef3900e693a9821c3dbe4a1d60bd863b6.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/gsfn1x0j3rcpmy11v8k0y2aiplq95v53-gensio-2.2.7
ls -la /nix/store/gsfn1x0j3rcpmy11v8k0y2aiplq95v53-gensio-2.2.7/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

gensio
ser2net

Maintainer pings
cc @emantor for testing.

Result of nixpkgs-review pr 127020 at b819b82e run on aarch64-linux 1

2 packages built successfully:

gensio
ser2net

1 suggestion:

warning: unclear-gpl
gpl2 is a deprecated license, please check if project uses gpl2Plus or gpl2Only and change meta.license accordingly.
Near pkgs/development/libraries/gensio/default.nix:35:5:
   |
35 |     license = licenses.gpl2;
   |     ^


Add another improved Twitter Bridge

This bridge is based on TwitterBridgeClean but shows RTs in user timelines correctly with the recycle sign. hashtag timelines have not been modified

This is getting too much Twitter bridges :( Someday I'll consider if it's worth merging all in one, with selectable options for RTs etc.

I know this is getting out of hand but I (and others) desperately needed these modifications and wanted to share them. If you'd be able to merge all of them, that would be great. Unfortunately that would exceed my (very marginal) PHP skills :(


Sound is bad on 5.1

After an upgrade from 5.0 to 5.1 the sound of unreal is really bad. We currently experience a bunch of 'hickups' and 'ticks' in the sound. This is not the case when we downgrade again to 5.0. I do not know where the issue comes from, only that this happens when we build in 5.1.
We did notice that 'red' is used now for sound and that more additional parameters are used in the sound of the pixel stream. Disabling these from the unreal PixelStreaming Plugin and the signalling server side did not result in any improvement in the sound.
Next to this the main issue that we face is that the recording of the microphone sounds really bad in the new version of the pixel stream. It sounds like the sample rate changed a little bit, even though the audio consumer still reports the sample rate to be 48khz. We cannot really use 5.1 now for our project because of this and we will revert to 5.0 for the time being, but we would like to move to 5.1 in the future.
I hope someone with more knowledge of webrtc can look into this and fix these issues such that 5.1 will work with normal sound.
Edit:
Here are the specs of the PC so you can rule that out:
Laptop: Omen 16n-0690nd
CPU: Amd ryzen 6800H
GPU: Nvidia RTX 3060 laptop gpu
RAM: 32GB DDR 5
it runs our game at ~150 fps

Hey @KungFuDonkey,
We've recently pushed the fix for the poor mic input to all versions on the Pixel Streaming Infrastructure.
Give it a go and let us know how it goes!
Cheers

yes I noticed the fix, however we now notice that the bitrate goes over the MaxBitrate setting. We think this might be an issue with the renegociation that happens in the new version, but we cannot say this for certain.

@KungFuDonkey Which bitrate? The audio or video bitrate?

@Belchy06 From the top of my head it was the video bitrate that exceeded the MaxBitrate setting. I will close this issue for now and make a new issue next week in which I will have more details about this issue. For now thanks for fixing the audio :).


can't set cookie

my serve ip,for example ,is  a.com,and client ip is b.com,
i want to set cookie for b.com in my serve a.com,
i have already set cors in serve,
and cline have set with credential,
but just cant set cookie
i can get the Set-Cookie in the res header,
but the browser don't set

exports.login = async (req, res) => {   let { username, password } = req.body   if (username == user.username && password == user.password) {     const t = token.sign({ username })     res.cookie('token', t)     res.json({       code: 0,       data: t     })   } else {     res.json({ code: 1, data: '用户名密码错误' })   } }

app.use(
  cors({
    origin: true,
    credentials: true,
    maxAge: '1728000'
  })
)

have set cors and the clinet
  axios: {
    // See https://github.com/nuxt-community/axios-module#options
    credentials: true,
 }

better way to setup your code...
exports.login = async (req, res) => {
  let { username, password } = req.body
  if (username == user.username && password == user.password) {
    const t = token.sign({ username })
    res.cookie('token', t)
    res.json({
      code: 0,
      data: t
    })
    return
  }
  res.json({ code: 1, data: '用户名密码错误' })
}

better way to setup your code...
exports.login = async (req, res) => {
  if (req.body.username == user.username && req.body.password == user.password) {
    const token = token.sign({ username })
    res.cookie('token', token)
    res.json({
      code: 0,
      data: token
    })
    return
  }
  res.json({
    code: 1,
    data: '用户名密码错误'
  })
}

better way to setup your code...
exports.login = async (req, res) => {
  if (req.body.username == user.username && req.body.password == user.password) {
    const token = token.sign({ username })
    res.cookie('token', token)
    res.json({
      code: 0,
      data: token
    })
    return
  }
  res.json({
    code: 1,
    data: '用户名密码错误'
  })
}

i dont think there are any different

right, they still function the same, but notice the omission of the else statement.  it might not seem like anything at first, but once I started coding this way, it made coding so much cleaner. Faster coding and less prone to error.  Maybe it's just me...just throwing it out there.


[python] make install fails with sed: can't read : No such file or directory

Steps to reproduce:

git clone -b python https://github.com/jreese/znc-push
cd znc-push
make install

What happens:
$ make install
cp push.py push.copy.py
sed -i "" -e "s|VERSION = '.*'|VERSION = 'v2.0.0-rc-53-gb7a4a77'|" push.copy.py
sed: can't read : No such file or directory
make: *** [push.pyc] Error 2

OS: Ubuntu 14.04.2 LTS

Changing the Makefile to run sed -i -e "..." push.copy.py" fixes this.

I was about to submit a pull request, but then I noticed 66d91f251b749e6ca280b24c84e31a1bcdc9fad3, so you must've added that "" for a reason.  What was it?
Perhaps you can use
sed -e "..." < push.py > push.copy.py.tmp && mv push.copy.py.tmp push.copy.py

and avoid inplace modification altogether?

Hmm, this seems to be an incompatibility between BSD sed and GNU sed.  I recently switched to using a Mac for my dev work.  I'll find a way that works with both platforms.


[Publishing] Adds image component and font system

Thanks @eessex for pairing on this & getting me started in reaction-force! Main thing I changed since the pair sesh is how the fonts work. I tried making it feel more similar to Force/Positron so whenever a font needs to be used, we can ${Fonts.garamond("s15")} in the styled component.
This screenshot doesn't show the new font since it's not in my system yet, but the fallback is good old Arial.

No blockers from my side, looks great! 🚀

Thanks for the comments! I simplified the image component to a function (good call this will likely never have a state) and added a long caption image to the storybook. Will merge when 🍏


Fix OBJECT_ID Errors in FeatureLayer Tests

Hi dear GeoBlazor Team,
The issue #306 was resolved by our team member @adrien426.
Description:
This PR resolves recurring OBJECT_ID errors encountered during ApplyEdits operations in FeatureLayer tests. It addresses failures in TestCanUpdateManyFeaturesWithApplyEdits and TestCanDeleteManyFeaturesWithApplyEdits, where OBJECT_ID was being read from undefined.
Changes:
Ensured OBJECT_ID is properly initialized and passed to ApplyEdits.
Adjusted test setup to correctly assign OBJECT_ID to features before edits.
Impact:
These changes fix the test failures, allowing both update and delete operations to complete as expected, thus ensuring the integrity and reliability of FeatureLayer functionality.

Yours sincerly,
David from Team GIS - POST Luxembourg

@adrien426, I pushed the correct code now, can you give it a check and when everything is ok, give me the go that I will change the PR back to "Ready for review".

@CyberDNS @adrien426 I've been doing a lot of testing for our beta v3 work, and I think I stumbled on the same bug, so I might already have a fix. But I'd still be happy to take a look (right now this only has the unit test changes in it).

@CyberDNS @adrien426 I believe this has all been addressed now on develop, and will be also fixed in the upcoming 3.0.0 release. Thank you again for bringing the issues to attention.


one file isn't downloading on travis

So I'm using git-lfs with travis to download files.  I added a new file and it's not getting downloaded for some reason.  I can't tell why.
I'm pretty sure I've updated the .gitattributes properly.  If I make a fresh clone of the repo and then checkout the branch I added the file on, I get the file as expected.  It's only when I getting it on travis that I'm having problems.
  The file is src/test/resources/large/CEUTrio.20.21.gatk3.4.gvcf.
I've included the log.
 Use fetch to download.
trace git-lfs: run_command: 'git' version
trace git-lfs: run_command: 'git' config -l

Errors logged to /home/travis/build/broadinstitute/gatk-protected/.git/lfs/objects/logs/20160620T193718.646396764.log
Use `git lfs logs last` to view the log.

 Use fetch to download.

File missing and download is not allowed
goroutine 15 [running]:
github.com/github/git-lfs/lfs.Stack(0x0, 0x0, 0x0)
	/Users/rick/go/src/github.com/github/git-lfs/lfs/errors.go:566 +0x80
github.com/github/git-lfs/commands.logPanicToWriter(0x7f38d1205358, 0xc820020160, 0x7f38d11c9518, 0xc82014c0d0)
	/Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:197 +0xf80
github.com/github/git-lfs/commands.logPanic(0x7f38d11c9518, 0xc82014c0d0, 0x0, 0x0)
	/Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:161 +0x412
github.com/github/git-lfs/commands.handlePanic(0x7f38d11c9518, 0xc82014c0d0, 0x0, 0x0)
	/Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:136 +0x4e
github.com/github/git-lfs/commands.LoggedError(0x7f38d11c9518, 0xc82014c0d0, 0xa58040, 0x42, 0xc820269e78, 0x1, 0x1)
	/Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:86 +0x82
github.com/github/git-lfs/commands.checkoutWithChan(0xc82004fa40)
	/Users/rick/go/src/github.com/github/git-lfs/commands/command_checkout.go:200 +0x5b2
github.com/github/git-lfs/commands.checkoutWithIncludeExclude.func1(0xc82004fa40, 0xc82000c510)
	/Users/rick/go/src/github.com/github/git-lfs/commands/command_checkout.go:110 +0x21
created by github.com/github/git-lfs/commands.checkoutWithIncludeExclude
	/Users/rick/go/src/github.com/github/git-lfs/commands/command_checkout.go:112 +0x1a0

ENV:
LocalWorkingDir=/home/travis/build/broadinstitute/gatk-protected
LocalGitDir=/home/travis/build/broadinstitute/gatk-protected/.git
LocalGitStorageDir=/home/travis/build/broadinstitute/gatk-protected/.git
LocalMediaDir=/home/travis/build/broadinstitute/gatk-protected/.git/lfs/objects
LocalReferenceDir=
TempDir=/home/travis/build/broadinstitute/gatk-protected/.git/lfs/tmp
ConcurrentTransfers=3
BatchTransfer=true
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
AccessDownload=none
AccessUpload=none
GIT_TRACE=1

The LFS object469ec5f88061c5a6fab0a453626a4f018e7c6d4a9bff0acaf619fd7a113ce11c in https://github.com/broadinstitute/gatk doesn't exist. Do you have a local copy of it somewhere? If you run git lfs push --all to upload it, you should be able to download it from Travis.

@technoweenie I'm not sure I understand how it was happening though.  It may be gone now because I ended up force pushing over the branch, at the time though it seemed like it was an issue specific to running it on travis though.
Does git-lfs keep a global cache on your local machine?  I tried testing to see if it was an issue with the file or if it was an issue only on travis by cloning a fresh copy of the repo on my local machine.  The clone worked perfectly including downloading the full file.  Only on travis did it fail to download it.  Is it possible it was getting it from some other source on my local machine?

Git LFS keeps a cache in your working directory under .git/lfs/objects. A fresh clone should not share any objects, unless you use the --references flag.
 I don't see d2cf11165c826a642f8208b32a81f9e6325ab655 on the repo on GitHub.

Sorry, I ended up forcing over d2cf11 in order to get my branch in.
See commit e41c1ac23df5e2653915f2333224a6ac4abefcab which exhibits the same problem.
It fails to download the file on travis, but works with a fresh clone.
git clone git@github.com:broadinstitute/gatk-protected.git
cd gatk-protected/src/test/resources/large
git checkout e41c1ac23df5e2653915f2333224a6ac4abefcab

It checks out the file correctly when I do that.  ( gatk-protected/src/test/resources/large/CEUTrio.CEUTrio.20.21.gatk3.4.g.vcf).  However, on travis it gets that same error.
[404] Object does not exist on the server
[469ec5f88061c5a6fab0a453626a4f018e7c6d4a9bff0acaf619fd7a113ce11c] Object does not exist on the server

https://travis-ci.org/broadinstitute/gatk-protected/builds/141661926
If it doesn't exist on the server I don't understand how my local clones are getting it.

I can confirm checking it out works on other machines too, so it's not some hidden caching.

I don't have that commit either. Are e41c1ac23df5e2653915f2333224a6ac4abefcab and d2cf11165c826a642f8208b32a81f9e6325ab655 from other branches besides master?

Neither of them has been merged to master.  Try fetching, I just put e41 up this morning after hearing back from you.  It's in the branch lb_add_gvcf_to_lfs

I don't see that remote branch or the commit after fetching.

That's really strange...   Can you see this branch on github?
https://github.com/broadinstitute/gatk-protected/tree/lb_add_gvcf_to_lfs
Is it possible you've accidentally cloned broadinstitute/gatk instead of broadinstitute/gatk-protected?
I'm sorry this has been such a pain to look into.  Thank you for taking the time.

Hey, broadinstitute/gatk-protected isn't actually protected (in terms of GitHub permissions), so I hope that's not your intention :)
I cloned gatk based on the first repository reference I saw in the messages above. I just checked out gatk-protected, and e41c1ac23df5e2653915f2333224a6ac4abefcab checked out just fine. Here are the objects I have in .git/lfs/objects:
06d71f4306957dea0f46adf78fce2969e80c37ecb070c2d51088b8db88ef9ab1
233c4c703ac071c37ff052c12cd5ad2016a0e84021e9ca3d01764b7165821aa7
469ec5f88061c5a6fab0a453626a4f018e7c6d4a9bff0acaf619fd7a113ce11c
5b62a51ebafae25cc735b16a90c4233a13372e66aa75c2410c4819c4140d5f94
671c073f3aedded63b187dbee993d1e441af74d21184135add78f497b7f9543b
6b1304800e60c0ac0358df137bdad48b7857a36465b04fef3fbbb09380f04746
8e5daa85c50bc06de53d26794dc02ed3bddcad083badae7a601e524982eeb58d
bafe4b1274a4ee6f9b5d9348a6a5f31d49781595bcda25cc452ad6fd6ad3cb34
e0c7060c55e35cc879da11b0ac9f3f6772fe95dc8b927fafb80f0139e6c4c065

Here's the ls-files output:
$ git lfs ls-files
469ec5f880 * src/test/resources/large/CEUTrio.20.21.gatk3.4.g.vcf
6b1304800e * src/test/resources/large/CEUTrio.HiSeq.WGS.b37.NA12878.20.21.bam
06d71f4306 * src/test/resources/large/CEUTrio.HiSeq.WGS.b37.NA12878.20.21.bam.bai
e0c7060c55 * src/test/resources/large/dbsnp_138.b37.20.21.vcf
5b62a51eba * src/test/resources/large/dbsnp_138.b37.20.21.vcf.idx
671c073f3a * src/test/resources/large/human_g1k_v<IP_ADDRESS>bit
233c4c703a * src/test/resources/large/human_g1k_v37.20.21.dict
bafe4b1274 * src/test/resources/large/human_g1k_v37.20.21.fasta
8e5daa85c5 * src/test/resources/large/human_g1k_v37.20.21.fasta.fai

Everything that Git LFS thinks it should have has been downloaded. I don't see any reference to "src/test/resources/large/CEUTrio.20.21.gatk3.4.gvcf", or a file with contents matching "469ec5f88061c5a6fab0a453626a4f018e7c6d4a9bff0acaf619fd7a113ce11c".
I don't know how gatk and gatk-protected are related, but you will have to push LFS objects to both remotes. That should work through its git pre-push hook. I'd really like to know why that's failing. It's tough to track down, since people don't report it until they have problems downloading, like this issue :)

Ack!  I think you've discovered the problem!   I'm the one who's checking out the wrong repo...
The script on travis is looking in gatk's lfs store, not gatk-protected.  So it's working fine when I have git lfs installed on my computer locally, but correctly failing when the script is setting up the remote to point to the wrong repo...
I'm sorry for wasting your time with my own typo.

Gatk-protected is meant to be visible on github, thanks for pointing out the potential opsec failure :)  The protected indicates it's under a different license than plain gatk.

Ha, glad you got it sorted. The protected suffix made sense when I finally read the repo description :)


Hope to add a PIN verification

Is your feature request related to a problem? Please describe.
I would like to request the introduction of a secondary authorization for the file sending or receiving functions.
Occasionally, I find myself needing to share links publicly for temporary access. To facilitate easy receipt, I often turn on the auto-accept feature, but this can inadvertently allow unauthorized access by others when left on for long periods, which is a source of distress.
Describe the solution you'd like
Hence, I am seeking the inclusion of a secondary authentication (using a PIN or password) that necessitates an additional PIN or password verification prior to downloading via shared links.
Additionally, I'm hoping for the incorporation of a similar functionality on the client-side, where transactions of sending or receiving files could be authenticated via a PIN or password. This would eliminate the need for manual confirmations on the client-end through password verification, proving particularly advantageous for unmanned computers.
Describe alternatives you've considered
No response
Additional context
No response

Hi, I like this idea a lot. It's true that someone could just scan every computer for the port 53317 use by LocalSend.
But easier than using a PIN/Password, I propose that the generated URL in the LocalSend client should include a Session Key/ID or something. That way the "PIN" verification is done directly when opening the link. That also means less effort to implement it because it doesn't require to make a "Enter Pin" UI.
It should be noted that while this is safer, it isn't secure by definition. It is still vulnerable to brute-force attacks if the pin code isn't long enough. Also if the transfer is done without encryption (HTTPS), the URL with the key/pin would be sent in clear text over the network, which could be intercepted by malicious actors. So should we enforce HTTPS for this mode ? Or we could just make it very clear on the UI (Orange/warning text that explains why it isn't secure blah blah....) and assume the user is responsible.

I think this can be easy added to the protocol by returning on if a PIN is required. The sending device must then proceed the same request again but with the secret / pin included in the JSON payload. e.g.401``POST /api/localsend/v2/prepare-upload
{
  "key": "hello123" // the new PIN
  "info": {
    "alias": "Nice Orange",
    "version": "2.0",
    "deviceModel": "Samsung",
    "deviceType": "mobile",
    "fingerprint": "random string",
    "port": 53317,
    "protocol": "https",
    "download": true,
  },
  "files": {
    "some file id": {
      "id": "some file id",
      "fileName": "my image.png",
      "size": 324242, // bytes
      "fileType": "image/jpeg",
      "sha256": "*sha256 hash*",
      "preview": "*preview data*"
    }
  }
}

The password could also be included in the header - I don't have a strong opinion here but I would like to keep everything simple with URL + JSON
Edit: This is only for the upload process. To download via shared links, it is similar: respond on , then add the key in the subsequent request: 401``POST /api/localsend/v2/prepare-download``POST /api/localsend/v2/prepare-download?key=hello123
Edit2: It would also be safer if the key is hashed before it gets sent to the server.
@Caesarovich If you are okay with this, then I would add this to https://github.com/localsend/protocol

Hello, thank you for responding to my suggestion.
I think this idea is quite good.
Regarding security, it would be even better if LocalSend could utilize user-defined certificates. For instance, I possess an ICANN-accredited domain certificate; a website hosted with this certificate via HTTPS is recognized by the majority of web browsers, thereby avoiding any browser warning messages.

I think this can be easy added to the protocol by returning on if a PIN is required. The sending device must then proceed the same request again but with the secret / pin included in the JSON payload. e.g.401``POST /api/localsend/v2/prepare-upload
{
  "key": "hello123" // the new PIN
  "info": {
    "alias": "Nice Orange",
    "version": "2.0",
    "deviceModel": "Samsung",
    "deviceType": "mobile",
    "fingerprint": "random string",
    "port": 53317,
    "protocol": "https",
    "download": true,
  },
  "files": {
    "some file id": {
      "id": "some file id",
      "fileName": "my image.png",
      "size": 324242, // bytes
      "fileType": "image/jpeg",
      "sha256": "*sha256 hash*",
      "preview": "*preview data*"
    }
  }
}

The password could also be included in the header - I don't have a strong opinion here but I would like to keep everything simple with URL + JSON
Edit: This is only for the upload process. To download via shared links, it is similar: respond on , then add the key in the subsequent request: 401``POST /api/localsend/v2/prepare-download``POST /api/localsend/v2/prepare-download?key=hello123
Edit2: It would also be safer if the key is hashed before it gets sent to the server.
@Caesarovich If you are okay with this, then I would add this to https://github.com/localsend/protocol

Hello
YES, my main concern stems from URLs potentially carrying extra information. I'm hoping for a more simplified approach when manually typing URLs, akin to the current format of IP/domain plus port number. This is particularly important because it can be quite inconvenient to verbally share URLs that include "session keys/IDs" with others.
On the matter of security, I completely agree with your perspective. The feature ought to default to employing https, with users having the option to manually disable it. Doing so should trigger a yellow warning message, succinctly elucidating the rationale behind maintaining https for safety purposes. This ensures that users are thoroughly informed before they opt for less secure configurations.

@Tienisto :

Edit: This is only for the upload process. To download via shared links, it is similar: respond 401 on POST /api/localsend/v2/prepare-download, then add the key in the subsequent request: POST /api/localsend/v2/prepare-download?key=hello123

While I like this idea for download via URL, I wouldn't be so sure as to add a "Autosave files verified with PIN" kind of feature, it could leave users with a false sense of security as the key could be brute-forced. For the download I wouldn't flinch as much because we could just generate a unique PIN every time, whilst the "PIN to receive" would have to be defined permanently or at least defined at each session.
But I guess it can be added to the official protocol and we can discuss about its implementation in the client at a later point.
@Tienisto :

Edit2: It would also be safer if the key is hashed before it gets sent to the server.

Could you clarify why this would be needed ?
@CATXYLS :

This is particularly important because it can be quite inconvenient to verbally share URLs that include "session keys/IDs" with others.

Yes I agree that having to type https://192.168.1.x:58000?pin=616aze68 might be a bit of a pain. But I like this solution in cases where the user can click on the link directly or scan the QR code as it "auto-fills" the pin via the url. On the UI we could indicate that it is optional in such a manner:

With the important part of the URL in bolder/stronger font that the pin part.
In any case if the PIN is optional from the URL, that would mean we have to build a PIN UI.

Could you clarify why this would be needed ?

Thinking about it again, I think we can skip that (I meant that we hash the pin similar to passwords, i.e. https://<IP_ADDRESS>:1234?pin=<sha256 hash>
Regarding autosave, I think the issue can be mitigated if we just return 429 Too Many Requests if an IP address sent too many requests. The block would be IP address based. In my opinion, this is sufficient enough because I don't think it is feasable to have so many IP addresses in a local wifi network (a 6 character PIN with lower+capital letters and digits have 56 billion combinations).
UX wise, there can be a "default" mode after enabling PIN verification. Here,
(1) a static PIN is generated to receive files from other devices and
(2) a new PIN is generated every time the "send via link" feature is used
According to

To facilitate easy receipt, I often turn on the auto-accept feature, but this can inadvertently allow unauthorized access by others when left on for long periods

the user might want to share some files publicly and having to tell others the exact PIN everytime they want to download it seem to be the opposite of what "auto-accept" promises.

Could you clarify why this would be needed ?

Thinking about it again, I think we can skip that (I meant that we hash the pin similar to passwords, i.e. https://<IP_ADDRESS>:1234?pin=<sha256 hash>
Regarding autosave, I think the issue can be mitigated if we just return if an IP address sent too many requests. The block would be IP address based. In my opinion, this is sufficient enough because I don't think it is feasable to have so many IP addresses in a local wifi network (a 6 character PIN with lower+capital letters and digits has (26+26+10)^6 = 56 billion combinations).429 Too Many Requests
UX wise, there can be a "default" mode after enabling PIN verification. Here,
(1) a static PIN is generated to receive files from other devices and (2) a new PIN is generated every time the "send via link" feature is used
The user can configure a custom PIN afterwards but this is optional.
According to

To facilitate easy receipt, I often turn on the auto-accept feature, but this can inadvertently allow unauthorized access by others when left on for long periods

the user might want to share some files publicly and having to tell others the exact PIN everytime they want to download it seem to be the opposite of what "auto-accept" promises.

Hello,
I believe this better ensures security and privacy. If users only want to share with others who know the PIN, such as in a blog post or a thread, sharing the LocalSend link can be done (in this case, the PIN should not be included in the link, e.g., https://<IP_ADDRESS>:443 (the PIN for this link is 1234)). This feature ensures that only users who know the PIN can access it.
Regarding auto-accept,
it seems a bit conflicting, but I think it's like a public link versus a private link. To resolve this conflict, I think it can be set up so that when auto-accept is enabled, PIN verification cannot be enabled, and vice versa.


Publish to Chrome Web Store and Firefox extensions

I'll probably rebrand the whole thing while reworking the big contentScript

Released v1.0.0 of Roller, currently under review on chrome and FF ext stores