bug: pkgx's git is slow

While making my first steps with pkgx, I found that my shell became really slow at giving back the prompt.
Running zsh -x showed me that it was caused by Git. I was about to start a contribution for the pantry and just executed a dev command. I realized it installed the git package from pkgx and that my shell was not using the system Git, but the one from pkgx.
For example, git branch went from taking about ~3ms to more than 1s!
Here's a reproduction link: https://github.com/aslafy-z/pkgx-reproduce-slow-git
Do you have any idea where this issue might be coming from?

git uses a shim to try and detect if it needs to load any git-* commands. You can read the shim here
So, do you see the slowdown using ~/.pkgx/git-scm.org/v2/libexec/git ...? That would tell us if it's the shim, or possibly something else. It's worth nothing that 500ms is something on the order of slowdown we see from deno startup when calling the pkgx binary, so it's possible there is an additional pkgx invocation (or two) in the thread resulting in a detectable delay.
You can also run the shim with bash -x to see what calls it's making: bash -x ~/.pkgx/git-scm.org/v2/bin/git ....

Yup, I tracked it down to the ~/.pkgx/git-scm.org/v2/libexec/git binary

Testing with hyperfine shows some but not as much variance as you're seeing:
$ pkgx hyperfine -w3 -m1000 "~/.pkgx/git-scm.org/v2/libexec/git --version"
Benchmark 1: ~/.pkgx/git-scm.org/v2/libexec/git --version
  Time (mean ± σ):       2.2 ms ±   0.1 ms    [User: 1.4 ms, System: 0.6 ms]
  Range (min … max):     2.0 ms …   3.3 ms    1000 runs

  Warning: Command took less than 5 ms to complete. Note that the results might be inaccurate because hyperfine can not calibrate the shell startup time much more precise than this limit. You can try to use the `-N`/`--shell=none` option to disable the shell completely.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

$ pkgx hyperfine -w3 -m1000 "/usr/bin/git --version"
Benchmark 1: /usr/bin/git --version
  Time (mean ± σ):       4.2 ms ±   0.1 ms    [User: 1.9 ms, System: 1.3 ms]
  Range (min … max):     3.8 ms …   5.6 ms    1000 runs

  Warning: Command took less than 5 ms to complete. Note that the results might be inaccurate because hyperfine can not calibrate the shell startup time much more precise than this limit. You can try to use the `-N`/`--shell=none` option to disable the shell completely.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

running it under a pantry checkout isn't much different:
$ pkgx hyperfine -w3 -m1000 "/usr/bin/git --version"
Benchmark 1: /usr/bin/git --version
  Time (mean ± σ):       4.3 ms ±   0.2 ms    [User: 2.0 ms, System: 1.4 ms]
  Range (min … max):     3.9 ms …   7.2 ms    1000 runs

  Warning: Command took less than 5 ms to complete. Note that the results might be inaccurate because hyperfine can not calibrate the shell startup time much more precise than this limit. You can try to use the `-N`/`--shell=none` option to disable the shell completely.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

$ pkgx hyperfine -w3 -m1000 "~/.pkgx/git-scm.org/v2/libexec/git --version"
Benchmark 1: ~/.pkgx/git-scm.org/v2/libexec/git --version
  Time (mean ± σ):       2.3 ms ±   0.4 ms    [User: 1.4 ms, System: 0.7 ms]
  Range (min … max):     2.0 ms …   9.1 ms    1000 runs

  Warning: Command took less than 5 ms to complete. Note that the results might be inaccurate because hyperfine can not calibrate the shell startup time much more precise than this limit. You can try to use the `-N`/`--shell=none` option to disable the shell completely.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

so, what platform and architecture are you running on? this was tested under darwin-aarch64.

I can reproduce this within a linux-amd64 8vcpu codespace environment, within the repro repository linked in OP. Wait for it to initialize and run bash bench.sh.

FYI, I updated my initial answer. I can confirm the latencies comes from the shim, specifically the part that runs pkgx --provider.

That, unfortunately, sounds unavoidable, due to pkgx overhead. However, based on the shim, if there's no command (a la git --version) or a command that ships with git, then pkgx shouldn't get called. That's everything under libexec.
Looks like the buggy line is:
elif [ -x "$libexec/git-$_CMD" -a -f "$libexec/git-$cmd" ]

if it's fixed to
elif [ -x "$libexec/git-$cmd" -a -f "$libexec/git-$cmd" ]

then it works as expected. i'll make that change to our package. you can either make that change locally, or remove and re-install git-scm.org once it finishes building.
good catch.

https://github.com/pkgxdev/pantry/actions/runs/6438659399
after this, it should only be slow when searching for external commands it doesn't ship.

So, 500ms from pkgx git to the shim, and 500ms for the extra check with pkgx --provider.
Two pkgx startups for git branch, 500ms each. A single one for git --version.
Git is often used in shell prompts like it is in the default codespace setup. It is really slow to work with.
I've read in #341 the plan was to rewrite the cli in rust after the 1.0 release. Is it still something you want to do?

we certainly need the oft-called bits to be the fast. that'll always be our goal.

I stared at this weird $_CMD earlier and was a bit confused. I could have sworn it appeared in both conditions. Thanks for the quick solution @jhheider, it changes everything!

what are friday nights for? ;)

Thanks all.
pkgx needs to be way faster in general. We intend to make a cache for pantry reads which should reduce as much as deno makes possible. Eventually we would like to rewrite in rust, but I have too many TODO items before that.
The cache should make it so even this situation would have been fast, though I'm glad it wasn't since we caught the bug.
Need to use set -u more often I think.

Is it possible to disable pkgx's custom git? I'm having http://starship.rs in my shell which displays the git status and branch in the prompt. Usually its super fast, but when running dev my shell gets super slow and starship --timings shows that its because of the git commands.

Hm.
$ pkgx hyperfine '/usr/bin/git status'
Benchmark 1: /usr/bin/git status
  Time (mean ± σ):      32.7 ms ±   2.1 ms    [User: 8.3 ms, System: 25.5 ms]
  Range (min … max):    30.2 ms …  38.8 ms    79 runs

$ pkgx hyperfine '~/.pkgx/git-scm.org/v2/bin/git status'
Benchmark 1: ~/.pkgx/git-scm.org/v2/bin/git status
  Time (mean ± σ):      35.3 ms ±   2.1 ms    [User: 9.8 ms, System: 25.6 ms]
  Range (min … max):    32.6 ms …  43.2 ms    77 runs

can you tell which git command? if you have the current version of the shim, it should only be slower if it can't find the git command you're running:
#!/bin/sh

set -e

find_git_command() {
  for arg in "$@"; do
    case $arg in
    -*);;
    *)
      echo "$arg"
      return  # found the command!
    esac
  done
}

libexec="$(cd "$(dirname "$0")/.." && pwd)"/libexec

# extract the git subcommand
cmd=$(find_git_command "$@")

_provides() {
  if foo="$(pkgx --provider git-"$cmd")"; then
    echo "$foo"
  else
    # syncing is slow but let’s be sure about it
    pkgx --sync --provider git-"$cmd"
  fi
}

if [ -z "$cmd" ]; then
  exec "$libexec/git" "$@"
elif [ -x "$libexec/git-$cmd" ] && [ -f "$libexec/git-$cmd" ]; then
  exec "$libexec/git" "$@"
elif type "git-$cmd" >/dev/null 2>&1; then
  exec "$libexec/git" "$@"
elif command -v pkgx >/dev/null 2>&1 && pkg=$(_provides); then
  exec pkgx +"$pkg" "$libexec/git" "$@"
else
  exec "$libexec/git" "$@"
fi

if that doesn't match your shim exactly, you can rm -rf ~/.pkgx/git-scm.org && pkgx +git-scm.org to reinstall it. however:
$ pkgx hyperfine '~/.local/bin/git status'
Benchmark 1: ~/.local/bin/git status
  Time (mean ± σ):     616.7 ms ±   8.7 ms    [User: 376.4 ms, System: 106.8 ms]
  Range (min … max):   600.0 ms … 631.5 ms    10 runs

if you have that shim installed, it basically always results in a pkgx invocation. so, a quick fix is to have git installed from your package manager and remove ~/.local/bin/git, or alias/link/directly reference the ~/.pkgx/git-scm.org/v2/bin/git shim.

You are right git status is actually fast! I've checked what starship is doing (by using STARSHIP_LOG=trace env variable) and found this command:
git -C . --no-optional-locks status --porcelain=2 --branch

I can confirm that this command takes 1.023s with dev and just 0.042s after dev off. Let me know if you can reproduce it.

that's a good one. this is finding the command as .:
find_git_command() {
  for arg in "$@"; do
    case $arg in
    -*);;
    *)
      echo "$arg"
      return  # found the command!
    esac
  done
}

so, it's looking for a pkgx package called git-.. i think i can exclude that case.
this seems to work:
find_git_command() {
  for arg in "$@"; do
    if test "$SKIP" = 1; then
      SKIP=0
      continue
    fi
    case $arg in
    -C) SKIP=1; continue;;
    -*);;
    *)
      echo "$arg"
      return  # found the command!
    esac
  done
}

you can make that change directly to your git shim; i'll push it to the repository. after applying:
$ pkgx hyperfine '~/.pkgx/git-scm.org/v2/bin/git -C . --no-optional-locks status --porcelain=2 --branch'
Benchmark 1: ~/.pkgx/git-scm.org/v2/bin/git -C . --no-optional-locks status --porcelain=2 --branch
  Time (mean ± σ):      35.5 ms ±   3.5 ms    [User: 10.4 ms, System: 26.2 ms]
  Range (min … max):    31.0 ms …  53.5 ms    51 runs

it's a very weird case. one of the few ways a non-flag argument can precede the command.

Amazing, thanks for the quick fix! 🥳

thanks for providing in-depth troubleshooting. you made it easy to find the bug by narrowing down exactly which conditions triggered it.

What about other global options that takes attributes like -c for eg.?

Should definitely be handled.

If you already have the latest version of git cached you will need to delete it to get the update, otherwise pkgx +git@latest will work. In a future bug fix release git@latest will work in all cases.

I'm sorry, I didn't quite follow what the fix was here. I just installed brew, pkgx and starship on a brand new macbook pro today, and I get a delay when I hit enter in a git repo (which triggers a redraw of the starship prompt) as well as missing git status symbols in the prompt.
The output of env STARSHIP_LOG=trace starship timings shows the following:
[DEBUG] - (starship::modules::git_status): New repo status created
[TRACE] - (starship::context): Executing command "git" with args ["-C", "/Users/fredrik/code/dotfiles", "--no-optional-locks", "status", "--porcelain=2", "--branch"] from context
[TRACE] - (starship::utils): Creating Command for binary "git"
[TRACE] - (starship::utils): Using "/Users/fredrik/.pkgx/git-scm.org/v2.42.1/bin/git" as "git"
[DEBUG] - (starship::modules::git_status): get_repo_status: git status execution failed
[TRACE] - (starship::modules): Took 516.221667ms to compute module "git_status"
[TRACE] - (starship::config): No config found for "git_status": Option "git_status" not found

If I run env -git-scm.org in the git repo and hit enter, there is no longer a delay in redrawing the prompt.

max is saying, once pkgx downloads git-scm.org for a specific version, it doesn't re-download it, unless it's missing (or it needs a different version).
to get the fixes i've added, you can pkgx +git+latest (forces a re-download), or rm -rf ~/.pkgx/git-scm.org (removes all cached binaries; subsequent calls to the shim at ~/.local/bin/git, if pkgx install gited, will download the updated bottle.)

@jhheider I just tried what you mentioned (rm -rf ~/.pkgx/git-scm.org which forced a re-download after re-entering the git project directory), but I still see the delay/failed git status problem. I now have git-scm v2.42.1.
Thing is you added the fix a couple of days ago, and I installed pkgx and hit dev in this project today and I still see this issue. Do you think I'm doing something wrong on my end?

hm. strangely, if i download git fresh, it has the old shim, not the new one. you can easily fix yours by replacing ~/.pkgx/git-scm.org/v2/bin/git with https://github.com/pkgxdev/pantry/tree/main/projects/git-scm.org/git-shim. let me see why it's not built into the new one.

it's possible CI didn't trigger the build, since I didn't change the package.yml. sloppy on my part, if so. i'll trigger a rebuild now.

Awesome, the rebuilt GHA workflow did the trick!
Thanks @jhheider!

this rebuild got it. interestingly, @mxcl, pkgx +git@latest didn't seem to do it, but rm -rf ... did.

you beat me to it, @fredrikaverpil . glad to be of assistance.

@jhheider I'm actually having similar issues with https://github.com/sindrets/diffview.nvim, where git becomes slow and I wish I could just not have git-scm.org loaded by default in my projects.
Is there any way to globally configure pkgx never to use git-scm.org?

Seems like it's this line that causes the delay. I added some debug printing:
#!/bin/sh

set -e

find_git_command() {
  for arg in "$@"; do
    if test "$SKIP" = 1; then
      SKIP=0
      continue
    fi
    case $arg in
    -C) SKIP=1; continue;;
    -*);;
    *)
      echo "$arg"
      return  # found the command!
    esac
  done
}

libexec="$(cd "$(dirname "$0")/.." && pwd)"/libexec

# extract the git subcommand
cmd=$(find_git_command "$@")

# echo $cmd into log file /tmp/gitlog.log
echo "cmd: $cmd" >> /Users/fredrik/code/dotfiles/gitlog.log

_provides() {
  if foo="$(pkgx --provider git-"$cmd")"; then
    echo "_provides returns: $foo" >> /Users/fredrik/code/dotfiles/gitlog.log
    echo "$foo"
  else
    echo "_provides goes into else clause for cmd: $cmd" >> /Users/fredrik/code/dotfiles/gitlog.log
    # syncing is slow but let’s be sure about it
    pkgx --sync --provider git-"$cmd"
  fi
}

if [ -z "$cmd" ]; then
  exec "$libexec/git" "$@"
elif [ -x "$libexec/git-$cmd" ] && [ -f "$libexec/git-$cmd" ]; then
  exec "$libexec/git" "$@"
elif type "git-$cmd" >/dev/null 2>&1; then
  exec "$libexec/git" "$@"
elif command -v pkgx >/dev/null 2>&1 && pkg=$(_provides); then
  exec pkgx +"$pkg" "$libexec/git" "$@"
else
  exec "$libexec/git" "$@"
fi

And this was the resulting log file:
cmd: status
cmd: rev-parse
cmd: ls-files
cmd: version
cmd: rev-parse
cmd: rev-parse
cmd: log
cmd: gc.auto=0
_provides goes into else clause for cmd: gc.auto=0
cmd: submodule.recurse=false
_provides goes into else clause for cmd: submodule.recurse=false
cmd: show
cmd: submodule.recurse=false
_provides goes into else clause for cmd: submodule.recurse=false
cmd: show
cmd: status

Does this help to figure out what's going on?

this rebuild got it. interestingly, @mxcl, pkgx +git@latest didn't seem to do it, but rm -rf ... did.

yeah I need to make git@latest check for an updated tarball, currently it does not.

Is there any way to globally configure pkgx never to use git-scm.org?

This has been requested before. I'm not yet sure how to handle the configuration. We add git so the user doesn’t need the Xcode Command Line Tools to dev which is a feature, though in cases where the user doesn't want that it is reasonable we allow such opt outs.

@jhheider I opened a PR here: https://github.com/pkgxdev/pantry/pull/4121
This is quite brittle, as it's just a matter of time before git adds arguments which will require a change here. I would personally like to disable git-scm.org globally for when using dev because of this, like I mentioned above.


Make TargetPixelFile.interact() more reponsive

Problem description

As stated in the doc, TargetPixelFile.interact() could be quite slow / sluggish.
One possibility to make it effectively faster / more responsive is to let user specify a time / cadence range to plot, rather than plotting for the entire file.
Proposal 1: Add an optional cadence_mask argument to interact(), e.g.,
  # handle only 5 days of plot, rather than the entire 20+ days of data
  tpf.interact(cadence_mask=(tpf.time >= 2010) & (tpf.time < 2015))

Proposal 2 Provide a general filter on TargetPixelFile, e.g. add a select(cadence_mask) method to TargetPixelFile
   # a new TPF object with only 5 days of data
   tpf_subset = tpf.select((tpf.time >= 2010) & (tpf.time < 2015))
   tpf_subset.interact()

Environment

lightkurve version: v1.11

I believe we already support this using the following syntax:
tpf[(tpf.time.value >= 2010) & (tpf.time.value < 2015)].interact()

This also works for a LightCurve object, e.g. you can do lc[lc.time.value > 2000].plot().
However the docstring of interact() could certainly be improved by advertising this syntax as a way to speed up the tool!

Ah, so it's a new feature for v2.0 .
I recalled I wanted to do something like that for LightCurve in the past too.

I believe this slicing functionality has been there since Lightkurve v1.0 (or thereabouts), though we haven't advertised it as much as we should!

You're correct - I had a typo in my trial script for v1 lightkurve, mistakenly believing it's not there.
It'd be good to have the feature documented.

Yep.  We'll want to do this using a combination of having more tutorials and having more "Examples" sections in docstrings.
I will go ahead and close this issue, but any and all new issues or PRs related to improving these matters are very welcome!


system ID mismatch, node belongs to a different cluster - after reboot

Hi,
I ended up with a node out of the cluster after a minor postgresql upgrade and system reboot.
I did this before in the past but never with patroni 1.2.5.
Error:
2017-10-10 11:20:44,915 CRITICAL: system ID mismatch, node sql2 belongs to a different cluster:<PHONE_NUMBER>861444259 !=
2017-10-10 11:20:44,931 ERROR:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/patroni/__init__.py", line 134, in patroni_main
    patroni.run()
  File "/usr/local/lib/python3.5/dist-packages/patroni/__init__.py", line 110, in run
    logger.info(self.ha.run_cycle())
  File "/usr/local/lib/python3.5/dist-packages/patroni/ha.py", line 946, in run_cycle
    info = self._run_cycle()
  File "/usr/local/lib/python3.5/dist-packages/patroni/ha.py", line 903, in _run_cycle
    sys.exit(1)
SystemExit: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/patroni/dcs/etcd.py", line 329, in wrapper
    retval = func(self, *args, **kwargs) is not None
  File "/usr/local/lib/python3.5/dist-packages/patroni/dcs/etcd.py", line 511, in delete_leader
    return self._client.delete(self.leader_path, prevValue=self._name)
  File "/usr/local/lib/python3.5/dist-packages/etcd/client.py", line 584, in delete
    self.key_endpoint + key, self._MDELETE, params=kwds)
  File "/usr/local/lib/python3.5/dist-packages/patroni/dcs/etcd.py", line 211, in api_execute
    return self._handle_server_response(response)
  File "/usr/local/lib/python3.5/dist-packages/etcd/client.py", line 928, in _handle_server_response
    etcd.EtcdError.handle(r)
  File "/usr/local/lib/python3.5/dist-packages/etcd/__init__.py", line 304, in handle
    raise exc(msg, payload)
etcd.EtcdCompareFailed: Compare failed : [sql2 != sql1]

I found this: https://github.com/zalando/patroni/issues/438
Edited that file, indented return to proper position but still the same issue.
Any ideas on how to debug this?
Thank you.

Hi,
without looking on your config file it's hard to say anything, but I think either postgresql.data_dir points to the wrong place or Patroni can't find postgres binaries.

Config:
scope: sqlmcp
name: sql2
restapi:
  listen: <IP_ADDRESS>:8008
  connect_address: <IP_ADDRESS>:8008
etcd:
  host: etcd0.local.tld:2379
  discovery_srv: local.tld
bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      use_slots: True
      parameters:
        archive_mode: "on"
        wal_level: hot_standby
        max_wal_senders: 10
        wal_keep_segments: 32
        archive_timeout: 1800s
        max_replication_slots: 10
        hot_standby: "on"
        wal_log_hints: "on"
        config_file: /var/lib/postgresql/data/sql2mcp/postgresql.conf
        max_connections: 1000
    pg_hba:
    - local all all  trust
    - host all all <IP_ADDRESS>/21 md5
    - hostssl all all <IP_ADDRESS>/21 md5
    initdb:
    - encoding: UTF8
    - lc-collate: en_US.UTF-8
    - lc-ctype: en_US.UTF-8
postgresql:
  listen: <IP_ADDRESS>:15432
  connect_address: <IP_ADDRESS>:15432
  data_dir: /var/lib/postgresql/data/sql2mcp
  pgpass: /tmp/pgpass0
  authentication:
    pg_rewind:
      username: postgres
      password: ***
    replication:
      username: replicator
      password: ***
      network: <IP_ADDRESS>/21
    superuser:
      username: postgres
      password: ***
    admin:
      username: admin
      password: ***
  parameters:
    unix_socket_directories: '.'
    archive_command: mkdir -p /var/lib/postgresql/wal_archive && test ! -f /var/lib/postgresql/wal_archive/%f && cp /var/lib/postgresql/data/sql2mcp/%p /var/lib/postgresql/wal_archive/%f
    shared_buffers: 10GB
    effective_cache_size: 32GB
    work_mem: 12582kB
    maintenance_work_mem: 2GB
    min_wal_size: 80MB
    max_wal_size: 1GB
    checkpoint_completion_target: 0.5
    wal_buffers: 16MB
    default_statistics_target: 100
  recovery_conf:
    restore_command: cp /var/lib/postgresql/wal_archive/%f /var/lib/postgresql/data/sql2mcp/%p
    archive_cleanup_command: '/var/lib/postgresql/omnipitr/bin/omnipitr-cleanup -l /var/log/omnipitr/cleanup.log -a /var/lib/postgresql/wal_archive/ %r'
tags:
    nofailover: False
    noloadbalance: False
    clonefrom: False

All postgresql binaries are present and datadir is present.
Having an option in patronictl for health check would be nice. Checking needed binaries, access to etcd or whatever is configured. But I guess that is something that some one new to this would need :)
I tried with: bin_dir: /usr/lib/postgresql/9.5/bin/, no change.
And after I tried with: data_dir: data/sql2mcp, patroni started but in logs I have: pg_basebackup: could not create directory "data/sql2mcp": Permission denied.
So it looks like it is related to data dir indeed. Will dig further.

I renamed the current data dir and now it is creating a replica from master.
No idea how this could of happened after a minor postgresql upgrade. Still using 9.5 just as before and I am sure I did this before in the past but with an older patroni version.

I'm seeing instances of the same error -
2017-12-04 12:14:41,012 CRITICAL: system ID mismatch, node postgresql82_131 belongs to a different cluster:<PHONE_NUMBER>219842093 !=<PHONE_NUMBER>632441982
In my case, I am using a custom bootstrap method which is restoring a physical backup (created with pg_basebackup), performing a pg_upgrade on the backup and then subscribing a replica via the default pg_basebackup subscription.
The master and replica pg_controldata both show the same system ID
Database system identifier:<PHONE_NUMBER>632441982
Etcd has a different system ID:
{
  "node": {
    "createdIndex": 3872,
    "modifiedIndex": 3873,
    "value": "6334130145219842093",
    "key": "/patroni/postgresql/initialize"
  }

I've done some more testing, and it looks like a race condition somewhere.  I'm feeding the same backup through an automated deployment pipeline (following the same steps).  Sometimes it deploys cleanly, sometimes it comes up with the system ID mismatch error.
My restore/upgrade process does the following

copy the backup to the host
decompress the backup to a staging data dir (not the PG data dir Patroni uses)
start Patroni
Patroni custom bootstrap script then copies the data from the staging area to the PG data dir (that Patroni knows about) and then runs pg_upgrade on it.

I'm speculating that there is a race condition where Patroni can get the System ID before or after the PG upgrade runs and changes it.

In my case, I am using a custom bootstrap method which is restoring a physical backup (created with pg_basebackup), performing a pg_upgrade on the backup and then subscribing a replica via the default pg_basebackup subscription.

It looks like very complicated setup. I would like to see your patroni.yaml and scripts for custom bootstrap and post_bootstrap/post_init.
But I can guess what is going on: after a custom bootstrap script has been executed patroni starts postgres and later updates initialize key with the value of Database system identifier (of the original cluster). In order to avoid running pg_controldata all the time patroni caches value of cluster sysid in memory, because nobody replaces data directory in runtime.
pg_upgrade requires you to run initdb, which generates a new Database system identifier, dumps/restores schemas and copying (or relinking) datafiles to a new directory. Also pg_upgrade starts and stops original cluster. Basically after pg_upgrade you get TWO data directories, the first one is original and another - upgraded to the new version.
What happens next? Do you somehow rename directories in order to give the new one to Patroni? But this is absolutely new cluster with the new sysid. And moreover, it requires a new postgres binaries (from another major version).
The only possible way to do a major upgrade right now is:

stop patroni
pg_upgrade
update patroni yaml
remove initialize key from DCS or wipe complete cluster state from DCS. The second one could be achieved by running patronictl remove <cluster-name>
may be copy patroni.dynamic.json from old data dir to the new one (if you wiped cluster state on previous step). It will help you to retain some postgres parameters you set before.
start patroni

For our use case, since we are doing the pg_upgrade on newly provisioned DB's only, the simplest approach is to upgrade the data before Patroni sees it.

I was using pgbackrest and ran into this problem as well. To fix it, I deleted the initialize key from Consul.

I was using pgbackrest and ran into this problem as well. To fix it, I deleted the initialize key from Consul.

hi c4milo could you pls share with us how to delete initialize key from consul ,
because i got same problem , i see in my logs liks this :
2020-02-12 18:11:08,133 CRITICAL: system ID mismatch, node odoo-psql2 belongs to a different cluster:<PHONE_NUMBER>479795020 !=<PHONE_NUMBER>744947325
2020-02-12 18:11:08,674 INFO: Lock owner: odoo-psql; I am odoo-psql2

I doubt that you want to delete the key from Consul, as it reports that there is an existing lock owner (odoo-psql).  The initialize key comes from PostgreSQL, and can be seen using the pg_controldata command.  All instances in your Patroni cluster should have the same ID, which will be the case if the second node was not initialized as a separate database but rather created as a copy of the first.
# /usr/lib/postgresql/12/bin/pg_controldata -D /var/lib/postgresql/12/main | grep "Database system identifier"
Database system identifier:<PHONE_NUMBER>013953695

1.list keys in etcd: etcdctl --endpoints=http://host:2379 get / --prefix --keys-only
2.delete old cluster keys: etcdctl --endpoints=http://host:2379 del /db/your_cluster_path --prefix
3.systemctl restart patroni
4.journalctl -u patroni


Workfiles API for Hosts

It seems my branch is ahead or differs from main getavalon/core here and there so this PR might be a bit messy. To see the clean changes that need to be made for this to actually work, see this PR: https://github.com/Colorbleed/core/pull/42
I will investigate the rest of the differences here and comment on this PR

Implement Workfiles API
This implements a Workfiles API for the hosts.

How to implement a Workfiles API for a host is documented in the README of the Workfiles tool
Note: Nuke is still missing its Workfiles API here.

Example implementations

https://github.com/BigRoy/core/blob/workfiles/avalon/fusion/workio.py
https://github.com/BigRoy/core/blob/workfiles/avalon/houdini/workio.py
https://github.com/BigRoy/core/blob/workfiles/avalon/maya/workio.py

References

Issue: https://github.com/getavalon/core/issues/365
Gitter discussion: https://gitter.im/getavalon/Lobby?at=5cdd928583ae782aee7b3046

See files changed overview for my comments on the "other" commits that are in this branch. Sorry for the mess. :)

About the conflicting files
avalon/houdini/pipeline.py
avalon/maya/pipeline.py

So the conflict in this PR seems to be on a change from self.name to self.data["subset"] in the creation of instances for the node names - which has been discussed that self.name was actually the preferred pick here
To resolve the conflict thus the best pick would be go for self.name - which is interesting to see that it wasn't actually resolved in avalon/core (yet). Let's also make sure the Nuke and Fusion integration to that too while we're at it.

To resolve the conflict thus the best pick would be go for self.name - which is interesting to see that it wasn't actually resolved in avalon/core (yet).

Oh, nooo.. I forgot this one :(
Looks like if the conflict get resolved here and we are safe ? :thinking:

Fixed the conflicts as described above.

Looks like if the conflict get resolved here and we are safe ?

Yup! :)

@tokejepsen have you been able to play with this yet? :)

Not yet :( Hope to get onto pipeline next week/month.

Seems to work fine here. Only tested Maya though as I don't have Houdini or know Fusion.
Happy to merge this :)

@davidlatwe is this something you can test too? Just want to make sure things behave as everyone wants at this stage. :)

Yes, for sure :)
I was planning to do the test this weekend.
I can test on Maya and Houdini tomorrow.

Tested, working good on both Maya and Houdini :D
One thing though, I did not set any default_dirs for Houdini, so at first it gives me an error saying that path/to/houdini-workspace/scenes dir not exists, I think this worth be mentioned in workfile's README ?
Or auto makedirs if not exists ?

One thing though, I did not set any default_dirs for Houdini, so at first it gives me an error saying that path/to/houdini-workspace/scenes dir not exists, I think this worth be mentioned in workfile's README ?
Or auto makedirs if not exists ?

Ah - wait. That's definitely a good point. I'm wondering if there's a more customizable approach to this for a studio, without altering avalon core... any ideas? Maya has the workspace you can customize. But what would Houdini have? Or even Fusion, or Nuke? Should we have an environment variable for applications that don't have a default fallback for "workspace customization"?

Maybe it should use the first entry in the default_dirs if it exists?
Could also have a workfiles_dir entry in the toml files?

Maybe it should use the first entry in the default_dirs if it exists?
Could also have a workfiles_dir entry in the toml files?

I believe inside the Host the .toml file itself isn't readily accessible - or at least it's non-trivial. So maybe it's best to have lik a AVALON_SCENEDIR kind of environment variable so that hosts that do not have their own workspace management that this scene dir name gets appended to the AVALON_WORKDIR path for work files to be saved into.
set AVALON_WORKDIR=houdini
set AVALON_SCENEDIR=scenes

Makes houdini/scenes. When AVALON_SCENEDIR is not set then Workfiles will operate directly in the root of the avalon workdir?

Makes houdini/scenes. When AVALON_SCENEDIR is not set then Workfiles will operate directly in the root of the avalon workdir?

Sounds good to me.

Sounds good to me, too :D

Nobody wants to hit the merge button ? :)
@Bigroy are you happy to merge this?

Sorry, I'm a bit swamped with other things.
The only remainder is still: https://github.com/getavalon/core/pull/387#issuecomment-500879708 specifically the AVALON_SCENEDIR

Implemented AVALON_SCENEDIR to define the scenes directory for hosts that do not have their own mechanism. So Fusion and Houdini now fall back to using that variable. Maya still uses its own workspace mechanism to define where to save scene files.
It's important to note that when the scenes directory does not exist the Work Files tool will still fail to launch. When using a custom folder name in your own pipeline, please make sure the folder is created prior it, e.g. through the use of default_dirs in Application .toml files.

Looks nice ! Will have a test later today !

How about we use AVALON_WORKDIR for all apps?
We could essentially get rid of the AVALON_SCENEDIR and just rely on a single environment variables for all apps.

How about we use AVALON_WORKDIR for all apps?
We could essentially get rid of the AVALON_SCENEDIR and just rely on a single environment variables for all apps.

That way one could never set the "root" folder in AVALON_WORKDIR where you might also save images/references/other content for a specific shot. It really is the root of the asset's work directory... not necessarily where the scenes get saved. So, what you are saying is to always save the files in the root of AVALON_WORKDIR - which I think wouldn't be the best way...
And if we'd never have the "project root" then we'd miss out on that.
In a way, it's similar to Maya's workspaces like:
cmds.workspace(query=True, rootDirectory=True) == AVALON_WORKDIR
cmds.workspace(fileRuleEntry="scene") == AVALON_SCENEDIR

How about just unifying using AVALON_WORKDIR in all apps?

We could, but..

I'm just afraid of backwards incompatibility for studios operating on Maya for which workfiles currently works as they'd expect since it relies on Maya's workspace system.
Personal preference. ;)

I'm just afraid of backwards incompatibility for studios operating on Maya for which workfiles currently works as they'd expect since it relies on Maya's workspace system.

Good point! Continue as is :)

@davidlatwe once you have tested it feel free to merge.

I have tested in Houdini and Maya, and they both worked smoothly :D
Merging this !


FAM support Silva: return user email information in Cognito

Describe the task
Team Silva wants to get the email information of the login user, we need to include that in the return data from Cognito. Update the read_attributes in the infrastructure/server/oidc_clients_silva.tf file to include email.
Acceptance Criteria

[ ] Return email of the login user for Silva

Need to make sure in our PIA or STRA, we're allowed to share these information to downstream applications


Remove Prop-Types from Production Build

It would be nice if PropTypes wasn't in the production build in any fashion. I believe this could be achieved by utilizing the following plugin:
https://www.npmjs.com/package/babel-plugin-transform-react-remove-prop-types
This would allow using the library without having to include the prop-types library as a pre-requisite in production.

Hi,
If you're talking about the umd build, I think that the prop-types module is mark as external and so not included there.
But if not, I don't think that the library should do that but you clearly should use this plugin for your app in production.
As said in #91
Recommended way for libraries: https://github.com/facebook/prop-types#how-to-depend-on-this-package
Also: https://github.com/facebook/prop-types/issues/44

I am referring to the UMD build.
You are wrong in your assumptions on how external works. That refers to whether the referenced library is included.
This PR is in regards to the use of Prop-Types ... it should make use of the remove prop types plugin as numerous other application libraries do allowing inheriting apps to not have a required dependency in production.

Got you, so we should strip prop-types for the minified version of the umd build ?

Yes. Not necessarily "strip" but use the proper mode (I believe wrap) with the plugin to remove as much as possible and where appropriate transforms certain situations to {} where constants dot-walks won't fail. ... Effectively removing as much as possible such that a referenced application does not need to either manually include prop-types for this to work.

Ok the need to include prop-types is there because we use externals with the webpack conf, by removing prop-types from the externals you don't need to manually include it anymore.
But the size of the bundle is bigger even in the production build (I thought that uglify could remove the dead_code and so the prop-types).
What would be the prefered result here ? Not having to include prop-types using the umd build even with a bigger module size ?

The original issue outlines everything in detail. Please read up on how things work.
1 - prop-types should be listed as external. This allows consuming applications to handle as necessary and the distribution builds will EXCLUDE the prop-types library. This is desired.
2 - However, References to prop types inside the library are now unnecessary and have references to constants used by the prop-types library. This is not needed in production as it requires an additional include for applications that don't already include the prop-types in production (most don't).  Additionally, it results in extra code and large dist size.
Please read up on prop types, before continuing any further in this thread.

I understand that, I already took some time yesterday to add the babel-plugin that I also use on almost all my react projects.
The part were I'm a bit confused is related to the use of the UMD build. (Maybe an example could help me there)
Also if you can find a repo that is already setup the way want please share it because on all the repos that I looked at no one was making the prop-types as external.

A problem for this lib would be this issue
One solution would be to handle contextTypes & childContextTypes ourself, like so:
Arrow.contextTypes = process.env.NODE_ENV !== "production" ? {
  popper: PropTypes.object.isRequired,
} : {
  popper: () => {},
}

What if we simply wait for the new context API? Once we get it we'll be able to get rid of PropTypes all together

@FezVrasta yeah I thought about waiting for the new context API too, but if @virgofx doesn't want to wait #94 is still a solution.
I wonder if after the refactoring for the new context we'll be able to support the old context.
An idea on this subject too ?

@Justkant The plugin prevents writing code which you did. I haven't had time to test; however, it's literally as simple as marking prop-types as external, and using the preset for all builds with appropriate flags (mode:wrap). It still should leave in context and a few other areas (That's what needs to be tested).
Sample Library
Material-UI

@virgofx You're right and if you take a look at #94 you can see the prop-types plugin in use. (with the mode: wrap)
Also for the contextTypes & childContextTypes that are not handle by the plugin, I followed the test of someone else
I also tested everything from jsfiddle to be sure it was working without the prop-types (new vs old -- look at the console)
🤣Material-UI was one of the lib that I looked at but they're not using externals for prop-types (umd webpack conf)
That's why I was asking for it in the first place.


Do not keep all association records in the memory

Depends on

[x]  https://github.com/ManageIQ/manageiq/pull/13997

This is optimization of second+ refresh, where we always load everything we have in our DB, in order to figure out what needs to be created/delete/updated by comparing to data we got from the API.
The biggest difference is visible on large table, e.g. a lot of templates, s3 objects, etc.

number of records
before
after

200k mocked MiqTemplates
2.6GB
1.3GB

100k mocked MiqTemplates
2.3GB
868MB

78k real AWS public images
1.5GB
1.1GB

so this can save a lot of memory for large tables (in the real example, the memoory spikes to 1.1GB and does not get GCed, because of aws-sdk)

@miq-bot assign @kbrock

cool. dependency is merged. rebase away

SaveCollection::Helper#save_inventory_collection!
unique_index_keys = inventory_collection.manager_ref_to_cols
# ...
index = inventory_collection.object_index_with_keys(unique_index_keys, record)

may want to keep in the inventory_collection?


Broken in Firefox 69 on OSX

This add-on stopped working in Firefox version 69 on OSX

I will need more info. From all I know, you may be experiencing problems that are produced by YouTube itself. Lately, YouTube has way more bugs than usual.

Ok, Good luck..!

The pop-up does seem to remain in place for me also, sometimes, even after focusing the tab, but the audio is still playing, it's not paused. Personally, I don't mind, as it does the job.

Please check the newest version(v0.7.0). It should fix the popup staying open.
Btw, @zmixnyc in case there is a misunderstanding, this extension doesn't make the popup not appear at all, it is just supposed to autoclick it so that it doesn't stay open.
I'll leave this issue open for a week or so if someone wants to add something or if it still doesn't work.


can we convert network to libtorch  by torch script and libcaffe (for object detection and segmentation)

above all

You may give a try.
link to object detection & instance segmentation https://github.com/zhanghang1989/detectron2-ResNeSt
link to semantic segmentation https://hangzhang.org/PyTorch-Encoding/model_zoo/segmentation.html

ResNeSt is also adopted by MMDetection https://github.com/open-mmlab/mmdetection/tree/master/configs/resnest.


Schema does not include an element for the geospatial reference system

Schema does not include an element for the coordinates reference System (WMDS element 3-07). Consider for Schema Version 2.0

This is now included in the EA model for the next version of the XSD (publication date unknown)
		<xs:element name="coordinateReferenceSystem" type="gml:ReferenceType" minOccurs="0" maxOccurs="1">
			<xs:annotation>
				<xs:appinfo>
			<gmlexr:targetCodeList>CoordinateReferenceSystemType</gmlexr:targetCodeList>
				</xs:appinfo>
			</xs:annotation>
		</xs:element>


Modify color panel and add labels to imgs click problem

Modify color panel and add labels to imgs click problem
fix issue about read only mode warning undisappear about #4741

Coverage decreased (-0.02%) to 64.38% when pulling 973c179893cb2404895d9b2158ad9d0b2edac033 on pengpengshui:colorPicker into 00538f424697441544c9871445145e8e3ca97d7c on vmware:release-1.5.0.


Invalid ColorCode

Description
Hello, I encounter an issue regarding the start of the plugin. I updated my onenord them using packer and now I have an error for a missing value:
packer.nvim: Error running config for no-neck-pain.nvim: ...acker/start/no-neck-pain.nvim/lua/no-neck-pain/color.lua:57: `colorCode` none  does not match the regex ^#[abcdef0-9][abcdef0-9][abcdef0-9][abcdef0-9][abcdef0-9][abcdef0-9]$

I tried to find how to fix it but without success, does it ring a bell to someone?

Hey there, thanks for opening the issue!
What does your setup call look like? Did you provided a custom backgroundColor value?

Hello,
Sorry I should have put the config, here it is:
use {"shortcuts/no-neck-pain.nvim", tag = "*", config = function()
    require("no-neck-pain").setup({
      width = 250,
    })
  end}

I didn't update much and I'm using onenord without any modifications

guess for anyone that wants to work on a fix, or me this evening:

we might get NONE returned here: https://github.com/shortcuts/no-neck-pain.nvim/blob/main/lua/no-neck-pain/color.lua#L70

if so, this should check it https://github.com/shortcuts/no-neck-pain.nvim/blob/main/lua/no-neck-pain/color.lua#L73
if not, this condition might be wrong

if none of the above apply, either:

we are not correctly getting the current background color
onenord doesn't set their background to this highlight group?

can confirm I'm not able to reproduce without the onenord theme, will try with it

neither with it, is your config open-source? I'll however still work on making it non-blocking

I encounter the same issue while using nvchad config so perhaps that can help with issue repro.

https://github.com/NvChad/NvChad
docker quick-start: https://nvchad.com/quickstart/install#try-in-docker-container

thanks for reporting! is the config similar to the one of the issue?

found the issue! the fix is available in 0.9.2 :)

Thanks a lot @shortcuts !

Thanks both of you for using the plugin :D


e2e: add CEL to accesslog

Positive test worked as expected.

will add negetive tests in this one.


Bluetooth: controller: Fix regression in Coded PHY CI Change

Fix the regression in Coded PHY CI change implementation
introduced by reverting the commit 9d1ca9c3906b ("Bluetooth:
controller: remove redundant PPI channel and TIMER CC").
Signed-off-by: Vinayak Kariappa Chettimada<EMAIL_ADDRESS>

Codecov Report

Merging #14506 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #14506   +/-   ##
=======================================
  Coverage   51.96%   51.96%
=======================================
  Files         309      309
  Lines       45576    45576
  Branches    10554    10554
=======================================
  Hits        23683    23683
  Misses      17083    17083
  Partials     4810     4810

Impacted Files
Coverage Δ

...oller/ll_sw/nordic/hal/nrf5/radio/radio_nrf5_ppi.h
93.1% <ø> (ø)
:arrow_up:

...oth/controller/ll_sw/nordic/hal/nrf5/radio/radio.c
81.53% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 959abdf...5532746. Read the comment docs.

Is this trying to get into 1.14?

@galak @carlescufi We are have not claimed BT conformance for the coded PHY feature before, hence, this PR is not for 1.14.

@cvinayak do we need this PR for 1.14?

If you desire this fix in 1.14, please open or associate this PR with an Issue.

@aescolar @ioannisg @carlescufi can I get some re-review?

@cvinayak are you gonna change the code to  use nrf_ppi?

are you gonna change the code to use nrf_ppi?

@carlescufi, you mean the comment from @aescolar ? I have addressed it. Is there something else?


[Question] PostgreSQL

Hello,
i saw from your .env files that you use Doctrine ORM for DB management but specify "MySQL or MariaDB" as a requirement in your documentation.
As Doctrine support PostgreSQL, i was wondering what that requirement really means for you.
Are you using specific instruct sets that only works on MySQL which cannot be translated to PostgreSQL?
Or is it that mosparo is only tested for MySQL as it is (i suppose) the dbms you're most used to but it "could" work with PostgreSQL?
If it's the second guess, could you consider adding PostgreSQL as a supported DBMS for mosparo in your documentation?

Hi @skid9000
Thank you very much for your question.
Doctrine, Symfony, and (I guess) mosparo should work perfectly with PostgreSQL. From my perspective, we do nothing special with the database, so there should be no problem with the queries and the schema (I think).
We investigated that issue before starting with the private beta and had some general troubles. I think - but I cannot remember - we had problems with the Doctrine Migrations Bundle, which we use for the database schema.
I just googled and tried to find out the problem again, but I cannot find anything. It could be a different problem (not Doctrine Migrations Bundle), or we didn't try it too much back then.
I will have another look at it and try to find out the reason why we didn't do it (or try to find the solution for this).
Thank you very much for bringing this to our attention!
Kind regards
zepich

Hi @skid9000
I've tested it with the latest mosparo version and found the problem.
 The reason is that these are plain SQL queries that are incompatible between MySQL and PostgreSQL.
The other, more significant issue is the comments on the column for some special Doctrine column types that we use (in our type and the types from the Doctrine Encryption Bundle by Gracious). These define the COMMENT for a column directly in the SQL statement, and therefore, it's impossible to create any SQL queries to create the tables for PostgreSQL. It's also a solvable problem. I tried that in a test environment, and it will work, but it needs some work.
We will put that on our TODO list and make mosparo support PostgreSQL databases. The number one priority for us is to support as many different setups as possible.
The negative part is that this will take some time and will not be available for the next weeks. In the best case, we can add it in v1.2, which is planned for the next months (release around April/May).
Kind regards,
zepich

That's great to hear! Thank you for looking into this!

I'm closing this issue because I created a new development issue with the to-dos for this enhancement. See #185

Hi @skid9000
As you may seen, we've released version 1.2 today with the official support for PostgreSQL.
Please let me know if you need any help.
Kind regards,
zepich


The list of cutscene command ids that are CS_NPC_ACTION_LIST is incomplete or wrong

CS_NPC_ACTION_LIST can use different cs cmd ids, it seems that ZAPD's list of those cmd ids is incomplete/wrong, as some are extracted as CS_UNK_DATA_LIST instead
The example Fig noticed:
gTempleOfTimeFirstAdultCs in assets/scenes/indoors/tokinoma/tokinoma_scene.c
has CS_UNK_DATA_LIST(0x4A, 2),
but 0x4A = 74 is a "real"/implemented command id:
case 74: in https://github.com/zeldaret/oot/blob/9647f50451f26fa8b96653bf792f86c6148f6d12/src/code/z_demo.c#L1728
So this should be a CS_NPC_ACTION_LIST
Idk if there are other occurrences

Resolved with PR #282.


Fix git-changes-action when module was unchanged

Description
As #1397 pointed out if a module is changed and then unchanged, the needs-go-generate action is not removed. For this reason, this pr:

consolidates actions for include_deps=true and include_deps=false. This will also help with #1390 by halving api requests by git-changes-action. Further work could be done here by consolidating goreleaser into go.yml the future
Remove unchanged module labels. This is honestly kind of a pain and could lead to api rate limiting issues. The way we're going to deal with this is by starting an action for each label, but only running it if the event is a pull request and this condition is met. Once this is done we can also add this to the needs-go-generate check to reduce the number of api calls to remove a lebl further helping w/ #1390

Note, this will still break is a module is deleted entirely, but I believe this is an acceptable edge case

issue as of ebe5e58092a9a805dc33b6272426119e30a5baa6 (for my ref)

testing this out again following this event: https://github.com/synapsecns/sanguine/pull/1399#event-10465688166


Showing bit value on mysql table

my table contains a BIT value, the database is on MySQL, at querying its does not show as 1 or 0.

Probably it is showing the raw structure type. I guess the data field contains the bit value.

Any one is looking into this?

@enVolt I guess nobody is looking into that. Would you like to fix that? I'm almost sure the place to fix that is here https://github.com/sqlectron/sqlectron-gui/blob/master/src/renderer/utils/convert.js#L19.


Dynamically change the log level in winston

how to dynamically change the log level in winston which will reflect a level change in multiple files.
I have two js files, in both files, I'm including logger, Let say If I change the level of the logger in index.js, but it will not reflect in readfile.js
Here is my code
winston_logger.js
var winston = require('winston');
var logger;
module.exports.init = function (logLevel) {
      logger = new(winston.Logger)({
        transports: [
            new(winston.transports.Console)({
                level: logLevel
            })
        ],
        exitOnError: false
    });
}

module.exports.getLogger = function (config) {
    if (!module.exports.logger) {
        exports.init('info')
    }
    return logger;
}

module.exports.debugLevel = function(){
  logger.transports.console.level = 'debug'
}

Index.js
var  readFile = require('./readFile')
var winston = require('./winston_logger')
var log = winston.getLogger()
log.info("info message" );

/*Here Im changing the log level as debug*/
winston.debugLevel ();

setTimeout(function(){
    readFile(function(){})
}, 5000)

readfile.js
var  readFile = require('./readFile')
var winston = require('./winston_logger')
var log = winston.getLogger()
log.info("info message" );

/*Here Im changing the log level as debug*/
winston.debugLevel ();

setTimeout(function(){
    readFile(function(){})
}, 5000)

Don't add these as a duplicate issue. these answers are not helped me

Please close the issue
winston_logger.js
   var winston = require('winston');
    winston.configure({
        transports: [
          new (winston.transports.Console)({ timestamp: true, json : false , colorize: true}),
          new (winston.transports.File)({filename: './somefile.log' , json : true, timestamp: true })
        ]
     });

module.exports = winston;

I can able to change the log level from anywhere.  Thanks

Closing as a duplicate of #1191. This is on our roadmap and will be fixed before<EMAIL_ADDRESS>gets released on May 29th, 2018.


JsonConvert.DeserializeObject does not throw exception on single opening brace

Source/destination types
JObject

Source/destination JSON
{

Expected behavior
Expected an exception to be thrown for invalid JSON.
Actual behavior
No exception is thrown.
Steps to reproduce
Using the v12.0.3-beta1 Newtonsoft.Json nuget package.
string payload = "{";
JsonConvert.DeserializeObject<JObject>(payload); //does not throw exception (I expected it to throw an exception like the below call to JObject.Parse does)
JObject.Parse(payload); //throws Newtonsoft.Json.JsonReaderException: Error reading JObject from JsonReader. Path '', line 1, position 1.

Thanks for the simple repro. Fixed.


using terraform plan output files

Codecov Report

Merging #11 into master will increase coverage by 4.51%.
The diff coverage is 92.3%.

@@            Coverage Diff             @@
##           master      #11      +/-   ##
==========================================
+ Coverage    93.1%   97.61%   +4.51%
==========================================
  Files           1        1
  Lines         116       84      -32
==========================================
- Hits          108       82      -26
+ Misses          8        2       -6

Impacted Files
Coverage Δ

main.go
97.61% <92.3%> (+4.51%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7d8a2a...9b6a7eb. Read the comment docs.


[REVIEW] victoriacheng15

GitHub Profile Link
https://github.com/victoriacheng15

Heya @victoriacheng15! Here are a few things that caught my eye:
Things I liked:

The labels under what I'm learning and plan to learn are a nice touch - mind if I steal that?
The blog posts are a nice touch! It's really nice that they're automated, GH Actions ftw! ✨

Things to improve on:

The font size of the social handle logos are on the smaller side, thus a bit difficult to read:

This might be a pet peeve of mine, but I feel that the banner height could be a lot smaller? It feels like there's unnecessary empty space

On the whole, it's a pretty cool profile - hope my review helped!

Hi @Akshu-on-github !

The labels under what I'm learning and plan to learn are a nice touch - mind if I steal that?

Sure, please feel free to do so!

The blog posts are a nice touch! It's really nice that they're automated, GH Actions ftw! ✨

GitHub action is so cool. I am planning to look into it more once I have time!

The font size of the social handle logos are on the smaller side, thus a bit difficult to read:

My original plan is to have links look different from skill badges, but you have a point there, it looks small. I will look into badge styles.

This might be a pet peeve of mine, but I feel that the banner height could be a lot smaller? It feels like there's unnecessary empty space

Good thinking! I am in process of changing my banner. I will take it into consideration!

On the whole, it's a pretty cool profile - hope my review helped!

Thank you so much for your time and feedback. I am appreciated them!

I love your profile! My only idea would be to turn on the activity overview.

And just because I'm being really picky, I'd use full stops on your about me section as you have exclamation marks, just to keep it consistent.

I love your profile! My only idea would be to turn on the activity overview.

Thank you, @hellodeborahuk! I used to have activity overview on, but I don't find it attractive/useful since it is pretty much my commits. So I am curious, what is your thoughts on it?

And just because I'm being really picky, I'd use full stops on your about me section as you have exclamation marks, just to keep it consistent.

I just re-read the about me section and realized that they are not consistent. Thank you for bringing it up. I will make changes.

Coding environment dark mode I like it 🔥


Stabilize config-cli

This stabilizes the --config CLI argument as per this FCP.
It also makes the adjustment suggested by @ehuss to allow stabilizing --config path without also stabilizing config-include.
I took a guess that this would land in 1.63 and put that in the tombstone entry in the unstable docs, but let me know if that's likely to be wrong.
Closes https://github.com/rust-lang/cargo/issues/7722.
Also, I think this should probably be tagged relnotes.

r? @ehuss
(rust-highfive has picked a reviewer for you, use r? to override)

There is also https://github.com/rust-lang/cargo/pull/10742 which would help with it.

Hmm, the new test failure appears to be related to the clap update changing a help message somewhere @epage?

I force-pushed to build on top of the landed 3.2 upgraded.

:umbrella: The latest upstream changes (presumably #10764) made this pull request unmergeable. Please resolve the merge conflicts.

Fixed conflict (which was just a trailing newline to unstable.md) by force-push.

@ehuss gentle bump on this — would be nice to land it for 1.63 beta.

Thanks!
@bors r+

:pushpin: Commit 7eefb4221fc6980bd3e6dceacdd40e6c280d2977 has been approved by ehuss

:hourglass: Testing commit 7eefb4221fc6980bd3e6dceacdd40e6c280d2977 with merge ed549d87178e47cb9a05ba309bd821c9595b770c...

:sunny: Test successful - checks-actions
Approved by: ehuss
Pushing ed549d87178e47cb9a05ba309bd821c9595b770c to master...

@weihanglo Can you tag this with relnotes?


joe mama gaming

yes

can you put the joe mama gaming button with other youtube channels?

oh ok

ok  i did it

ok cool

now i have to pull the things onto my local repo


[BUG] VirtualMachineCustomImage creation fails with OsState invalid error

Describe the bug
VirtualMachineCustomImage creation fails with OsState invalid error when .withWindowsFromDisk option is specified with the option OperatingSystemStateTypes.SPECIALIZED whereas when we pass OperatingSystemStateTypes.GENERALIZED everything seems to be working fine.
Exception or Stack Trace
Caused by: com.microsoft.azure.CloudException: The value of parameter osDisk.osState is invalid.
    at com.microsoft.azure.AzureClient.createExceptionFromResponse (AzureClient.java:771)
    at com.microsoft.azure.AzureClient.access$100 (AzureClient.java:33)
    at com.microsoft.azure.AzureClient$3.call (AzureClient.java:160)
    at com.microsoft.azure.AzureClient$3.call (AzureClient.java:157)
    at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext (OnSubscribeMap.java:69)
    at retrofit2.adapter.rxjava.CallArbiter.deliverResponse (CallArbiter.java:120)
    at retrofit2.adapter.rxjava.CallArbiter.emitResponse (CallArbiter.java:102)
    at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call (CallExecuteOnSubscribe.java:46)
    at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call (CallExecuteOnSubscribe.java:24)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:48)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:33)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OnSubscribeSingle.call (OnSubscribeSingle.java:81)
    at rx.internal.operators.OnSubscribeSingle.call (OnSubscribeSingle.java:27)
    at rx.internal.operators.SingleToObservable.call (SingleToObservable.java:39)
    at rx.internal.operators.SingleToObservable.call (SingleToObservable.java:27)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:48)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:33)
    at rx.internal.operators.OnSubscribeLift.call (OnSubscribeLift.java:48)
    at rx.internal.operators.OnSubscribeLift.call (OnSubscribeLift.java:30)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.DeferredScalarSubscriber.subscribeTo (DeferredScalarSubscriber.java:153)
    at rx.internal.operators.OnSubscribeTakeLastOne.call (OnSubscribeTakeLastOne.java:32)
    at rx.internal.operators.OnSubscribeTakeLastOne.call (OnSubscribeTakeLastOne.java:22)
    at rx.internal.operators.OnSubscribeLift.call (OnSubscribeLift.java:48)
    at rx.internal.operators.OnSubscribeLift.call (OnSubscribeLift.java:30)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:48)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:33)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:48)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:33)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:48)
    at rx.internal.operators.OnSubscribeMap.call (OnSubscribeMap.java:33)
    at rx.Observable.unsafeSubscribe (Observable.java:10327)
    at rx.internal.operators.OperatorSubscribeOn$SubscribeOnSubscriber.call (OperatorSubscribeOn.java:100)
    at rx.internal.schedulers.CachedThreadScheduler$EventLoopWorker$1.call (CachedThreadScheduler.java:230)
    at rx.internal.schedulers.ScheduledAction.run (ScheduledAction.java:55)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:304)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:834)
Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: retrofit2.Response.class
    at rx.exceptions.OnErrorThrowable.addValueAsLastCause (OnErrorThrowable.java:118)
    at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext (OnSubscribeMap.java:73)

To Reproduce
Steps to reproduce the behavior:
Execute the code shared below.
Code Snippet
 VirtualMachineCustomImage virtualMachineCustomImage = azure.virtualMachineCustomImages().define(sourceVmName + "-Image")
                .withRegion(region)
                .withExistingResourceGroup(sourceVmRgName)
                .withHyperVGeneration(HyperVGenerationTypes.V1)
                .withWindowsFromDisk(virtualMachine.osDiskId(), OperatingSystemStateTypes.SPECIALIZED)
                .withDataDiskImageFromManagedDisk(virtualMachine.dataDisks().get(0).id())
                .create();

Expected behavior
A specialized image should be created.
Screenshots
If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):

OS: MacOS
IDE : IntelliJ
1.35.1

Additional context
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

hi @xccc-msft could you pls have a look? thanks

@Shanky2304 The cloud exception comes from service response. I just added label of Service Attention so we could get help from service team. Meanwhile, I will try to reproduce from my end. It would be better if you could share more info about the configurations of the virtual machine you created. Thanks.

@Vaibhav-Agar, could you please take a look at this issue.

Apologies for the delay.

@Shanky2304 The cloud exception comes from service response. I just added label of Service Attention so we could get help from service team.

Yes, I suspected that. It would seem that specialized images are only supported in Shared Image Gallery? Why so?
Btw even SIG doesn't support SPECIALIZED images via the SDK right now. @xccc-msft I think you're already aware of #13073

Meanwhile, I will try to reproduce from my end. It would be better if you could share more info about the configurations of the virtual machine you created. Thanks.

Oh, you mean the source VM's configuration out of which I tried to create the CustomImage?
It was a vanilla Standard D2_V3 Windows 10 1903 VM. With a S10 managed data disk. I didn't generalize it since I want to create a specialized image.

Investigating

Hi @Shanky2304,
As you correctly pointed out, specialized images are only supported in SIG. SIG images offer more feature capabilities than Managed images. SIG is a newer offering compared to Managed images and in general, we have been investing more in SIG compared to Managed images. As for SDK support, I believe you got your answer via #13073.
Thanks!

@Vaibhav-Agar Why is this not documented at https://learn.microsoft.com/en-us/rest/api/compute/images/create-or-update?tabs=HTTP#operatingsystemstatetypes?


4.11 paks now don't load properly as of latest update

Tested in Days Gone, using the latest repak.exe causes the pak contents to not be read, reverting back the exe fixes it

What command are you using to make the pak? (version/compression settings)

"C:\modding\unreal_engine\repak\repak.exe" pack --version V3 "C:\Users\Mythical\Downloads\pak_dir"

Ha I was unaware versions that old have even been tested. Seems like I did introduce a bug and fixed it in 468eab37e406752259883d924c1657c8936a9ee8. I'm fairly confident this should be fixed in the latest release.

Aye that fixed it, thanks!
Hyped to use the new compression feature
and regarding other versions 4.11, 4.14. 4.15 also work if you never tested those (atleast they did before these last commits, have not tested those with new exe)

also not a big deal but this last one flagged windows defender, never had that happen before

also not a big deal but this last one flagged windows defender, never had that happen before

Pretty much expected for small time software unfortunately.


allow kleebuilder to fail

instead of exploding and using all RAM + a testcase from building LLVM that explodes and uses all RAM without this patch
this code isn't too pretty, it takes a code path that used to always succeed and allows it to fail. but I believe it is the right answer here, we've already tried several mitigations for explosion in kleebuilder and this solves all future problems of that sort.

anyone have time to look this over? I'd like to move on with some other changes as we move towards the PLDI deadline

Argh some tab chars sneaked in-- should be fixed now.


Module 1 exercises [Boiki Ramatho]

Please "drag & drop" exercise 1 pdf file below.
POPULATION 2022.pdf

Please "drag & drop" exercise 2 pdf file below.
BOTSWANA POPULATION DENSITY - 2022.pdf

Please UPDATE THE TITLE of this "Issue" in the title bar above with your name.

If you haven't done so, please go back to Module 1 and make sure to take
the post-training survey. Participating in surveys is required for receiving
the "GALUP Certificate" by the University of Florida.

Well done, Boiki. You clearly grasped the material outlined in Module 1 and successfully applied it to these exercises. Thank you for your participation!


The application is still signed even if the "signed" option is disabled when using Android one-click deploy

Godot version
v3.5.beta1.official [b9b23d222]
System information
w10 64
Issue description
This happens on android.
Option signed deactivated

The application is still signed when exporting

Steps to reproduce
see the images
Minimal reproduction project
...

When using one-click deploy, signing is always enabled no matter the setting's value: https://github.com/godotengine/godot/blob/0393057e36e486b2944e6166e6572327944b2fa7/platform/android/export/export_plugin.cpp#L1849
(5th argument should_sign is always set to true.)
When exporting traditionally, the signing flag is supposed to be obeyed though: https://github.com/godotengine/godot/blob/0393057e36e486b2944e6166e6572327944b2fa7/platform/android/export/export_plugin.cpp#L2620

The reason is to save time generating the apk by avoiding the signature. I did not know that the latest versions no longer allow unsigned installation.

I am referring to the use in "one click deploy". The reason is to save time generating the apk by avoiding the signature. I did not know that the latest versions no longer allow unsigned installation.

One click deploy installs directly to the device, and as @Calinou mentioned, Android does not allow installation of unsigned apk, hence that apk is always signed.
If you want to generate an unsigned apk, you have to use the regular export mechanism.


gofumpt: 2020-10-27 -> 0.1.0

Motivation for this change
Upstream update
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build gofumpt

@GrahamcOfBorg build gofumpt

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 109321 run on x86_64-linux 1

1 package built:

gofumpt

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 109321 run on x86_64-linux 1

1 package built:

gofumpt


Remove the dependency of DB from soda-controller

Is this a BUG REPORT or FEATURE REQUEST?:
Currently DB is accessed by two module, soda-apiserver and soda-controller. soda-controller usses DB just to update the status of provisioning service(For ex: after volume,fileshare creation, it update the status).
Since we are making hotpot to three seperate and independent project, let's remove the controller interaction with DB. And to update the status controller should sent the reuest to APISERVER only. And only API Server will update the status/anyother things. This way DB  will be completly mange by API Server.
Let's do the POC.
i.e scenario 2 from below diagram

Uncomment only one, leave it on its own line:
/kind bug
/kind feature

What happened:
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

Hotpot(release/branch) version:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

@vaibhav2ghadge As per discussion, lets work on this. Let me know if you find any other issue. Thanks


Fix --style parameter for az interactive command

Fixes: https://github.com/Azure/azure-cli/issues/5807
This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

View a preview at https://prompt.ws/r/Azure/azure-cli/5809
This is an experimental preview for @microsoft users.


Certificate chain is not split correctly

Description:
We are using google CAS as a delagated certificate authority and our complete certificate chain is :
[Root CA cert - RCA] -> [Intermediate CA cert - ICA] -> [GoogleCAS CA cert - CASCA] -> [Leaf certificate - CERT]
In certificate secrets, the certificate chain is not split properly :

tls.crt contains CASCA and CERT in a single encoded string using a mix of "\r\n" and  as new line separator.

CERT : this part of the string uses "\r\n" as new line separator and contains the leaf certificate.
CASCA : this part of the string uses "\n" as new line separator and contains the Google CAS certificate authaurity public certificate.

ca.crt contains RCA + ICA in a single encoded string with "\r\n" as new line separator (except at the end of the string where only "\n" is used.

What's expected:

From what we undestood from cert-manager FAQ, the secret should contains :

tls.crt with the full certificate chain within (except the CA cert)
ca.crt should only contain the root certificate

All encoded string should be using the same new line separator (might be an external issue without impact to google-cas-issuer, but...??)

What's happening:
As the web server using the leaf certificate publishes cert.crt as certificate chain, our TLS handshake is timeout, as the client does not trust the ICA, but only the RCA.
If we modify the client config and we add the ICA to its truststore, the TLS handshake ends succesfully, and the TLS connexion is established.
Versions affected:
google-cas-issuer: 0.8.0
cert-manager: 1.14.4
How to reproduce:
Create the CA chain described, and you should reproduce the issue.

The google private-certificate GO api provide,, as fullchain certificate, an array of certificates (as string). In this issue's context, the latest element of this array contains 2 certificates in one string.
I'm writing an issue on google side regarding this topic, but it might be a good idea to "sanitize" received certificate chain. Here is a snipet of code that fix current issue in my case :
import (
  "regex"
  (...)
  'k8s.io/klog/v2"
)

(...)

func extractCertAndCA(resp *casapi.Certificate) (cert []byte, ca []byte, err error) {
	klog.Info("Starting extractCertAndCA")
	defer klog.Info("Quit extractCertAndCA")
	if resp == nil {
		return nil, nil, errors.New("extractCertAndCA: certificate response is nil")
	}

	certBuf := &bytes.Buffer{}
	var certs []string

	re := regexp.MustCompile(`(?sU)-{5}BEGIN CERTIFICATE(?:.+)END CERTIFICATE-{5}`)

	// parse the certificate and store it in certs slice
	match := re.FindString(resp.PemCertificate)
	if match == "" {
		return nil, nil, errors.New("extractCertAndCA: leaf certificate is not properly parsed")
	}
	certs = append(certs, match)

	klog.Info("The original Certificate Chain length is ", len(resp.PemCertificateChain))
	// Write any remaining certificates except for the root-most one
	// for _, c := range resp.PemCertificateChain[:len(resp.PemCertificateChain)-1] {
	for _, casCert := range resp.PemCertificateChain {
		match := re.FindAllString(casCert, -1)
		if len(match) == 0 {
			return nil, nil, errors.New("extractCertAndCA: the certificate chain is not properly parsed")
		}
		// Append all matched certs from the certificate chain to the certs slice
		certs = append(certs, match...)
	}

	for _, cert := range certs[:len(certs)-1] {
		// For all the certificate chain, but the most root one (CA cert)
		// We write it to the cert buffer
		certBuf.WriteString(cert)
		certBuf.WriteRune('\n')
	}

	klog.Info("Finally, the Certificate Chain length is ", len(certs)-1, ", including the leaf certificate")
	// Return the root-most certificate in the CA field.
	return certBuf.Bytes(), []byte(certs[len(certs)-1] + "\n"), nil
}


Consider returning the legacy registration method

I'd like to make use of this updated module that has some goodies in the SDK , but it has switched to the new registration scheme.
Unfortunately, it's hard to make use of it in the SDK for the following reason: the old scheme is used throughout the SDK tests to create the fixtures in the DB (like create a device before testing the move).
So while the registration itself doesn't make sense in the browser, for example, it is still needed for the browser tests.
As a transitional step I suggest returning the old method as registerLegacy or smth like that, upgrading to this module in the SDK and using this method.
The method will report deprecation.
Later we can properly think about what to do when the method is removed.
Your thoughts?

What is preventing the SDK to make use of the new registration schema in the browser (and update the tests accordingly)? Is it randomstring?
In any case, if registration doesn't make sense in the browser, then it should probably be not part of
the SDK at all.
I'm happy with the registerLegacy transitional step :+1:

I can't see any reason why the SDK can't use this to register a device, you just put the provisioned device key in the config.json instead of the provisioning key and it should be ok?

Yes, I don't say it can't be used at all. Of course, when you have the
provisioning key you can use this method.
But I'm not sure to which of the statements you reply here.
The old method is currently used in the SDK tests to create the test device
records (when testing other parts of the SDK). And it works because it
doesn't need any secrets. But now how would I use this new method for the
same purpose?
On Wed, Dec 14, 2016 at 9:33 AM, Page-<EMAIL_ADDRESS>wrote:

I can't see any reason why the SDK can't use this to register a device,
you just put the provisioned device key in the config.json instead of the
provisioning key and it should be ok?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/resin-io-modules/resin-register-device/issues/25#issuecomment-266954657,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgGCLSG7z7bNQQHF5P7PpREYLq-L6OVks5rH42jgaJpZM4LLnqe
.

--
Eugene Mirotin
Senior Frontend Engineer
site: Resin.io https://resin.io/, twitter: @resin_io
https://twitter.com/resin_io

The SDK tests already have to use an api key to use this module right? Using the same api key should be absolutely fine as it has to be a valid provisioning key to have worked previously (it will just have a superset of the permissions of a provisioning-only key)

Oh, I didn't realize that and assumed the provisioning keys are special
one-time keys issued for the specific device ID.
OK then, I will try the new version and see if it works, let's keep this
issue hanging for a while.
On Wed, Dec 14, 2016 at 11:26 AM, Page-<EMAIL_ADDRESS>wrote:

The SDK tests already have to use an api key to use this module right?
Using the same api key should be absolutely fine as it has to be a valid
provisioning key to have worked previously (it will just have a superset of
the permissions of a provisioning-only key)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/resin-io-modules/resin-register-device/issues/25#issuecomment-266972788,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgGCFFO-7dMDFihL0zT-d8xB0r5zbDcks5rH6grgaJpZM4LLnqe
.

--
Eugene Mirotin
Senior Frontend Engineer
site: Resin.io https://resin.io/, twitter: @resin_io
https://twitter.com/resin_io

I'm fairly sure this can be closed now @emirotin, unless you had some other concerns here.

Yeah I think this can be closed
On Tue, Jan 24, 2017 at 7:58 PM, Tim Perry<EMAIL_ADDRESS>wrote:

I'm fairly sure this can be closed now @emirotin
https://github.com/emirotin, unless you had some other concerns here.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io-modules/resin-register-device/issues/25#issuecomment-274883531,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgGCD7BPyg3PTV7Cu4kMjgHyMMjCh6Rks5rVjutgaJpZM4LLnqe
.

--
Eugene Mirotin
Senior Frontend Engineer
site: Resin.io https://resin.io/, twitter: @resin_io
https://twitter.com/resin_io


Use reference instead of value type to match Lucene's logic

Introduce a nullable long in DocValuesConsumer, instead of using just long. The need for this is best illustrated if you look at the changes in Lucene45DocValuesConsumer and in NumericDocValuesWriter. The Writer had iterator implementation that did not match Lucene version, ignoring cases of DocsWithField.Get(doc) evaluating to false in which case a null value needs to be returned. Null value executes a code branch in Lucene45DocValuesConsumer that was simply ignored before. The result of that is the resulting index structure being incorrect, and requests such as DocsWithFields would return true for all documents even though the index had documents that did not have a field in question.
The tests that used to break are in TestLucene45DocValuesFormat and TestPerFieldDocValuesFormat, e.g. BaseDocValuesFormatTestCase.TestTwoNumbersOneMissing. This fixes about 20 failing tests by my count.
I was considering porting over Numeric class over, but I think we can do that in stages, if necessary. Since all the signatures were using long, migrating to long? was the first step to get the logic right and tests passing. And then we can improve on the approach itself.
I think I see more code places where iterators have been ported incorrectly and numbers are used in non-nullable form causing different code execution paths, but will get those as we go down the list of failing tests.

Merging now - will this get us below 150 failures? :)

No need to port Numeric IMO, all usages I've seen of it mattered only for inheritance which I could get by one way or another.

@synhershko ha! We wish. If I had to guess it will get us down to 220 range, you know how those random tests go. But what it will allow me to do is move on to the next failures. This particular issue had me stomped. It is so subtle, I just couldn't see what was wrong until looking at the byte level of what the index streams are writing... it was brutal.


settings-view package styles being polluted

Specifically, the
button { margin-right: 10px;}
was causing the buttons in settings view to be spaced incorrectly.

Nice catch


Atom 1.40.1 starts infinite copies of itself (fork bomb!)

Prerequisites

[x] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
When I restarted atom today (I think it was just upgraded?) It started to create infinite copies of itself stacked on top of each other to the point my computer was unusable. I'd restart my computer and atom would do the same thing. Finally I was able to ssh into my own computer and remove the atom package so it couldn't start and from there I was able to experiment a bit. I tried starting it with --clear-window-state and with --safe, but I saw the same fork-bomb behavior.
Finally, I moved my ~/.atom dir to ~/.atom-backup and then atom was able to start successfully. I copied the packages dir back to the new ~/.atom and started fresh. I still have a copy of that backup dir in case it would be helpful to reproduce the bug.
Steps to Reproduce

Upgrade to atom 1.40.1
(re)Start atom
Be ready to killall atom a few times!

Expected behavior:
Atom should not fork-bomb
Actual behavior:
Atom fork-bombs
Reproduces how often:
100%
Versions

$ atom --version
Atom    : 1.40.1
Electron: 3.1.10
Chrome  : 66.0.3359.181
Node    : 10.2.0
$ apm --version
apm  2.4.3
npm  6.2.0
node 10.2.1 x64
atom 1.40.1
python 2.7.12
git 2.7.4
$ lsb_release -a
LSB Version:	core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch:printing-9.20160110ubuntu0.2-amd64:printing-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noarch
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.5 LTS
Release:	16.04
Codename:	xenial

Additional Information

Can you confirm how you installed Atom? Also, do yo have Ubuntu set to auto-update packages or did you manually update Atom by installing the latest .deb or using apt-get?

I have atom installed from the official apt repo.
zac@zac-prodesk:~/mbot$ dpkg --status atom
Package: atom
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 587924
Maintainer: GitHub<EMAIL_ADDRESS>Architecture: amd64
Version: 1.40.1
Depends: git, libgconf-2-4 (>= 3.2.5) | libgconf2-4, libgtk-3-0 (>= 3.9.10), libgcrypt11 | libgcrypt20, libnotify4, libxtst6, libnss3 (>= 2:3.22), python, gvfs-bin, xdg-utils, libx11-xcb1, libxss1, libasound2 (>= 1.0.16), libxkbfile1, libcurl3 | libcurl4, policykit-1
Recommends: lsb-release
Suggests: libsecret-1-0, gir1.2-gnomekeyring-1.0
Description: A hackable text editor for the 21st Century.
 Atom is a free and open source text editor that is modern, approachable, and hackable to the core.
zac@zac-prodesk:~/mbot$ cat /etc/apt/sources.list.d/atom.list
deb [arch=amd64] https://packagecloud.io/AtomEditor/atom/any/ any main

When you say you removed the Atom package, do you mean that you uninstalled Atom at this point (with apt-get remove for example)? And then reinstalled it to try clearing saved state and starting in safe mode?

I guess my system is set to automatically open the applications from the last saved session because atom (and other apps) kept opening automatically when I would restart and I wouldn't have time to open a terminal and killall atom. So the only way I could use my system was to apt-get remove atom via ssh. Then I could log in, reinstall atom via apt-get and be ready with the killall command to experiment with different startup options.
I tried atom --safe and atom --clear-window-state but saw the same behavior.
I don't have any customization to the config or init scripts other than what atom gives me access to in the Settings UI. I actually looked at conif.cson and init.coffee in my broken ~/.atom dir and they're empty aside from the commented out boilerplate.
I just tried swapping back my old ~/.atom dir and I'm still able to reproduce the fork bomb behavior.
One interesting tidbit is that at the time I originally saw this behavior I had an error in my .bashrc. I'd implemented a work-around for another electron bug by adding ELECTRON_TRASH=gvfs-trash atom but I forgot to remove the atom at the end! Originally I thought this could be the cause of the fork bomb. It would make sense. But I fixed that, restarted, and still saw the same behavior. Again, I do not see the behavior with my reset ~/.atom dir. Is there may some way that broken .bashrc setting was persisted in atom's config?
Here's my broken ~/.atom dir minus the packages if anyone else wants to try to reproduce this.
atom-broken.tar.gz


purs-nix docs doesn't support searching

In DMs you suggested this repo could help:
https://github.com/purescript/purescript-docs-search

Added in 204d33a5e7fc4cfa559300b0f08558bbcc02a79c


Configure the URL to match with your repository

The URL in the file config/_default/config.yaml (line 7) must match your repository name:
Replace template-project-website with your repository name.

Done


Memory improvements

There is probably more that could be done here, but I'd say this fixes #72 and #63. My city took 40GB of RAM originally, and then 20GB after #74. It now only takes 2-3GB of RAM for the same city.
The main improvement comes from reducing the size of the Block struct. It's now only 1 byte long. This has also improved performance a bit.

Amazing, I can't keep up with my changes since you're so fast haha!
Huge thanks for your contributions. Sorry for being a bit late on merging them, I'm trying to find some more time for Arnis again. I'm really motivated on pushing the development of this project forward to a new release. :)

No worries at all!


How can I do some process like segment_sum to a batch data in parrelel

For example..
I got an embedding matrix as follow:
embed_matrix = [
[0,0,0,0],
[1,1,1,1],
[2,2,2,2],
]
I have another input which decides which embedding should I use for summation to get a new embedding.
ids = [
[0,1,],
[1,2,],
[0,1,2]
]
And I want a function f as folow.
f(embed_matrix, ids)
= [
[0, 0, 0] + [1, 1, 1],
[1, 1, 1] + [2, 2, 2],
[0, 0, 0] + [1, 1, 1] + [2, 2, 2],
]
= [
[1,1,1],
[3,3,3],
[3,3,3],
]

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.
If you think we've misinterpreted a bug, please comment again with a clear explanation, as well as all of the information requested in the issue template. Thanks!


Hide groups

I think will be better if the Groups could be hide  or minimized on side panel.

@fsobolev FYI
Do you mean the whole groups section should be collapsable?
I don't really see this as a useful feature IMO.

When you have a lot of groups, I think its doesnt looks good and not useful. Could be better if we can minimize it.
Sorry, I cant explain better, my english isnt good.

Ah I see. When there are a lot of groups the list can be really long, making it hard to navigate to calendar for example. Can be fixed by using Gtk.ScrollWindow for groups list.

Fixed in V2023.1.0-beta2
The update is being published to flathub-beta now and should be available in 10-30 minutes :)

This feature was also added to WinUI. Will be available in the next beta:
Groups Shown:

Groups Hidden:

This feature was also added to WinUI. Will be available in the next beta:
Groups Shown:

Groups Hidden:


Error page redirects to login

I'm not sure if this is actually an issue or if I'm just missing something.. On my Publish instance the user gets redirected to the login page [1] when hitting a page that doesn't exist. If I'm logged in, I get the error page just fine.
Looking at the code it sounds like this is handled in the function "doHandle404()". However, where does it actually check if we are on an Author or on a Publish instance? It seems for me it always meets the first condition:
        if (this.getStatusCode(request) == SlingHttpServletResponse.SC_NOT_FOUND
                && this.isAnonymousRequest(request)
                && AuthUtil.isBrowserRequest(request)) {

It's a 404 request
it's anonymous (user is not logged in)
it's a browser request

Am I missing something or how does it check for Author/Publish?
[1] /content/geometrixx/en/toolbar/account/login.html

Yes, I agree it makes sense to support OOTB behavior.
I like your idea of a path-pattern based selection. An admin could say "these areas of the site are not access controlled (ie. don't have special permissions) and thus we do not need to authenticate the request". On the other hand you could also have a field where you specify which paths you DO need to authenticate. Sort of like exclude and include fields. Not sure if that makes it too complicated... wdyt?

Yes, I agree it makes sense to support OOTB behavior.
I like your idea of a path-pattern based selection. An admin could say "these areas of the site are not access controlled (ie. don't have special permissions) and thus we do not need to authenticate the request". On the other hand you could also have a field where you specify which paths you DO need to authenticate. Sort of like exclude and include fields. Not sure if that makes it too complicated... wdyt?

@paulrohrbeck Adding this functionality; Adding 2 new OSGi configs .. thinking we may want to set the default to "respond-with-404" since has been the usual behavior prior to the above "fix".
not-found.behavior = redirect-to-login || respond-with-404
not-found.exclusion-path-patterns = [ /content/foo/.* ]

If not-found.behavior = redirect-to-login , then a 404'ing request to /content/foo/bar  result in an immediate 404
If not-found.behavior = respond-with-404, then a 404'ing request to/content/foo/bar`  result in a redirect to login page (assuming the other usual conditions are met; anon, browser req, etc.)

Thoughts?

@davidjgonzalez I didn't do any testing on it yet but it looks fine to me!
Is there a timeline for the 1.10.0 release?

@paulrohrbeck mm.. it might be 1.9.4 .. reviewing the PRs most are bug fixes or "enhancements" that you could argue were "bugs" in the behavior.
I still need to test this PR out locally (its a PITA to try to unit test error page handler since so much is orchestrated by Sling)
@justinedelson thoughts? (I know we have the repo.adobe issue open which hopefully will be resolved in the next day or two)

Hey @davidjgonzalez, any progress on a 1.9.4 or 1.10 release?

Sorry @paulrohrbeck works been crazy! Would a release early next week for for you? I still need to test this one out and dont think ill have a chance till the weekend; if youve had a chance would love to hear the feedback.
@justinedelson WDYT? We have a handful of minor enhancements/fixes.

Early next week works for me.
On Wednesday, January 14, 2015, david g<EMAIL_ADDRESS>wrote:

Sorry @paulrohrbeck https://github.com/paulrohrbeck works been crazy!
Would a release early next week for for you? I still need to test this one
out and dont think ill have a chance till the weekend; if youve had a
chance would love to hear the feedback.
@justinedelson https://github.com/justinedelson WDYT? We have a handful
of minor enhancements/fixes.
—
Reply to this email directly or view it on GitHub
https://github.com/Adobe-Consulting-Services/acs-aem-commons/issues/401#issuecomment-70024749
.

@paulrohrbeck 1.9.4 is out w this fix

Awesome, thanks so much for your help on this!