WIP: bind mount for acme.sh is missing

This is a suggestion, and WILL BREAK EXISTING INSTALLATIONS since it mounts a new directory into the /etc/acme.sh please do not blindly apply this
Having no fixed mount for acme.sh can cause a loss of your private key and all your signing keys for your domains, which hapend to me yesterday.
I noticed this when I got greeted with lots of:
too many certificates already issued for exact set of domains
A restart of the containers confirmed this suspicion:
nginx-letsencrypt    | [Wed Dec  9 16:48:07 UTC 2020] Create account key ok.
nginx-letsencrypt    | [Wed Dec  9 16:48:07 UTC 2020] Registering account: https://acme-v02.api.letsencrypt.org/directory
nginx-letsencrypt    | [Wed Dec  9 16:48:08 UTC 2020] Registered

Now having this bind-mount for the /etc/acme.d will put them where they should be.
Reference: Readme: nginx-proxy/docker-letsencrypt-nginx-proxy-companion

Wow @rwese, thank you very much for you PR, that's was definitely missing in our repo.
I will check the breaking changes as you suggested, but it seems I can have a work around script for that...

@evertramos
I was bondering a fix, and fixing this without breaking would be adding an init-script which copies existing keys to the new mounted location and symlink the mounted-dir to /etc/acme.sh. This would be hands-off for the users.
Or modify the nginx-letsencrypt container init to use a different /etc/acme.sh location and migrate existing scripts with a init script.
Anyway it is not easy to make this fix non-breaking.

@rwese
This changes is due to an update on the companion image...
https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion#step-2---letsencrypt-nginx-proxy-companion
Which is not using simp.le anymore...
We were expecting an update on simp.le but never happened.
It will be necessary some extra updates in this repo in order to fix it... I am working in a side project that will have a bunch of automation including this repo as well, but it will take me a coupld weeks to finish it. But for now it still works for the newer users.
I will update this to the new companion way of doing things... as acme.sh is a good way of generating the certificates.
Catch you up later on this.

Same problem here after few functional tests
Generating new certificate private key
ACME server returned an error: urn:ietf:params:acme:error:rateLimited :: There were too many requests of a given type :: Error creating new order :: too many certificates already issued for exact set of domains: maparla.es: see https://letsencrypt.org/docs/rate-limits/

Finally you solve that? Thanks in advance

@MarioProjects can you checkout v2 branch and test the docker-compose file in your environment as well? Also if you do so, could you post here which environment you have, linux version, distro, docker version, docker-compose version etc. thanks!

@evertramos this branch solves this certificate thing problem? I tried, I accessed to the branch and downloaded using git clone -b v2 https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git proxy but when calling `bin/start.sh
ERROR: The Compose file './../docker-compose.yml' is invalid because:
services.nginx-proxy-automation-web.ports is invalid: Invalid port "::/0:443:443", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]
services.nginx-proxy-automation-web.ports is invalid: Invalid port "::/0:80:80", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

I did a fresh installation. Should I had the files and containers running from this branch first and then call that start.sh? In the README references fresh_start but it does not exists

@evertramos For example, If I run a container with flask and I want to update the code... this will call lets encrypt again and again, and will cause the same errors? How should we deploy the container and reuse the cert?

you shouldn't use the start script... just docker-compose
$ docker-compose up -d

docker-compose up -d

ERROR: The Compose file './docker-compose.yml' is invalid because:
services.nginx-proxy-automation-web.ports is invalid: Invalid port "::/0:443:443", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

comment the ipv6 and check if you have anything running in port 80 or 443 stop everything and try again please

What I did:
git clone -b v2 https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git proxy
cd proxy
docker network create proxy  # As your docker-compose requires 'proxy' network
# comment ipv6 lines in docker-compose.yml
docker-compose up -d
# 3 containers running:  nginx-proxy-automation-web / nginx-proxy-automation-letsencrypt / nginx-proxy-automation-gen

Now create a simple site with Dockerfile
FROM nginx:alpine
COPY . /usr/share/nginx/html

Then, create container and attach with your config:
docker build -t personal-site .

docker run -d -e VIRTUAL_HOST=maparla.es \
              -e LETSENCRYPT_HOST=maparla.es \
              -e<EMAIL_ADDRESS>\
              --network=proxy \
              --name nginx-personal-site \
              personal-site

LetsEncrypt container logs:
[Wed Mar  3 15:10:29 UTC 2021] Create account key ok.
[Wed Mar  3 15:10:29 UTC 2021] Registering account: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:10:30 UTC 2021] Registered
[Wed Mar  3 15:10:30 UTC 2021] ACCOUNT_THUMBPRINT='YWBvw20BMukszEukj3GDEIq3zsSVple0M4CHcnRGB4I'
Reloading nginx docker-gen (using separate container nginx-proxy-automation-gen)...
Reloading nginx (using separate container 490d233c8554b03c902578b8f30fe6debaae666b6fbc0a30c4ac32f489cd8cb3)...
Creating/renewal maparla.es certificates... (maparla.es)
[Wed Mar  3 15:10:31 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:10:31 UTC 2021] Creating domain key
[Wed Mar  3 15:10:32 UTC 2021] The domain key is here<EMAIL_ADDRESS>[Wed Mar  3 15:10:33 UTC 2021] Single domain='maparla.es'
[Wed Mar  3 15:10:33 UTC 2021] Getting domain auth token for each domain
[Wed Mar  3 15:10:34 UTC 2021] Create new order error. Le_OrderFinalize not found. {
  "type": "urn:ietf:params:acme:error:rateLimited",
  "detail": "Error creating new order :: too many certificates already issued for exact set of domains: maparla.es: see https://letsencrypt.org/docs/rate-limits/",
  "status": 429
}
[Wed Mar  3 15:10:34 UTC 2021] Please check log file for more details: /dev/null
Sleep for 3600s

After that I tried with one subdomain from maparla.es -> info.maparla.es.
docker run -d -e VIRTUAL_HOST=info.maparla.es \
              -e LETSENCRYPT_HOST=info.maparla.es \
              -e<EMAIL_ADDRESS>\
              --network=proxy \
              --name nginx-personal-site \
              personal-site

Here the logs:
2021/03/03 15:12:40 Debounce minTimer fired
2021/03/03 15:12:40 Generated '/app/letsencrypt_service_data' from 5 containers
2021/03/03 15:12:40 Running '/app/signal_le_service'
Creating/renewal info.maparla.es certificates... (info.maparla.es)
[Wed Mar  3 15:12:41 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:12:41 UTC 2021] Creating domain key
[Wed Mar  3 15:12:42 UTC 2021] The domain key is here<EMAIL_ADDRESS>[Wed Mar  3 15:12:42 UTC 2021] Single domain='info.maparla.es'
[Wed Mar  3 15:12:42 UTC 2021] Getting domain auth token for each domain
[Wed Mar  3 15:12:44 UTC 2021] Getting webroot for domain='info.maparla.es'
[Wed Mar  3 15:12:44 UTC 2021] Verifying: info.maparla.es
[Wed Mar  3 15:12:48 UTC 2021] Success
[Wed Mar  3 15:12:48 UTC 2021] Verify finished, start to sign.
[Wed Mar  3 15:12:48 UTC 2021] Lets finalize the order.
[Wed Mar  3 15:12:48 UTC 2021] Le_OrderFinalize='https://acme-v02.api.letsencrypt.org/acme/finalize/114524160/8232878452'
[Wed Mar  3 15:12:49 UTC 2021] Downloading cert.
[Wed Mar  3 15:12:49 UTC 2021] Le_LinkCert='https://acme-v02.api.letsencrypt.org/acme/cert/04be838c10624a41f2cb447cae28047a8259'
[Wed Mar  3 15:12:50 UTC 2021] Cert success.
-----BEGIN CERTIFICATE-----
MIIGJDCCBQygAwIBAgISBL6DjBBiSkHyy0R8rigEeoJZMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAzMDMxNDEyNDhaFw0yMTA2MDExNDEyNDhaMBoxGDAWBgNVBAMT
D2luZm8ubWFwYXJsYS5lczCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB
ANGTULroG/iVqkJsb6AuNVfG9K9vpsXhBvq8xXzmBmHPRBhfGpKAvDa+VMWsX+Kk
P5wVrrZW0xSIEUNsgy1oIJnBRvdHR+ZS1XceHCPYScz7rm4tVduMo6nCoY5Maqyd
f9QdQc0Eu1TdvvF78FSETeE69L8ZElRzh0EZBAihu3aj91VUS3c5R6YJ1djxrlAC
XOjLXuYPsbwgAmtwBbrZQ1ob8jACDOE/5ILXSmDKf7R71qiN7+ZJlgQ27fTQaIXu
pfPzMVyyqW9E+T1u0bHgjXVlVuV0AfmY/IsoiUaDd+MR3vch3ZqUbbckS3DwtN3Y
K/pQCctCRLEsSCpFE6ARQg0aMujU+WYvW69K8RHKoMCstYR6AZUCfUIKDabbcmJc
Pu9+pK5UnGx8hwaVqDhNY9JvoSflJ4OZ8GkQQxl4oA9jCsLTXHw58pU6JKCgb6er
0IMjse5xLd7qMKZPpjcBvJq7sR19JrqRLq/r2QDg8gOpIwPrKMJ+wUkNmYaK3Xp6
HLwLiizY+iM3j/RF6MRwyxJfgupyqYFqgFCziIFh9v86Dvj52UW+oqqQBfVjC2P9
YP1c1TBQFEmw1mYgGTLKPwDXE8jEH4hJ7IuqPSDoXEN6V9lluVMduJxOzG++PD56
+yDhGjlcCCiQfaA66Jf9ZVPpLiMfM38Avzc00PRZ4bwvAgMBAAGjggJKMIICRjAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwG
A1UdEwEB/wQCMAAwHQYDVR0OBBYEFCbBZL9YGDC4NGOMvC2LpzTUPSGOMB8GA1Ud
IwQYMBaAFBQusxe3WFbLrlAJQOYfr52LFMLGMFUGCCsGAQUFBwEBBEkwRzAhBggr
BgEFBQcwAYYVaHR0cDovL3IzLm8ubGVuY3Iub3JnMCIGCCsGAQUFBzAChhZodHRw
Oi8vcjMuaS5sZW5jci5vcmcvMBoGA1UdEQQTMBGCD2luZm8ubWFwYXJsYS5lczBM
BgNVHSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIB
FhpodHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQQGCisGAQQB1nkCBAIEgfUE
gfIA8AB2AJQgvB6O1Y1siHMfgosiLA3R2k1ebE+UPWHbTi9YTaLCAAABd/imEZ8A
AAQDAEcwRQIhAKfVfGqrXBk/x+Wq2M1gyMI9jOgdaykqdbqgOiRPqCfsAiBbwRBe
oX9fbQT6PCxMDPsJ1/7dU0IztT4jI7oyH6FOiwB2AH0+8viP/4hVaCTCwMqeUol5
K8UOeAl/LmqXaJl+IvDXAAABd/imEcoAAAQDAEcwRQIhAMXvCjBCF4RjNI7ZNYHh
hCb1613OdQDvH4xeBBj9zyF3AiB49luGVh8tycmmRBKPR5HdsxRL0dzMyl0COo5k
bW661DANBgkqhkiG9w0BAQsFAAOCAQEARgY4eA166oHnAoNPziQwGbZ0hoByfCB6
YhvTPVPiSUBuMzgUsxUqlZGct28y9ZjO66BN+3QSkGmpwpeD8kH9f+Tvon/sEj4o
zeETWcx+s+zi6Grga1JF87uSxlRWL1I92/8cHCs69tIMY/O75cub1NSQS8fzrH4b
78b1BLydKv0T0+2SnXweFeQs01i+Lzd2TnWWOILrStWUn6H/XTNh8xGjeEqOmEJN
Iz5rwOZCbuZM80FTKJsyBy9xjRVoAuGilrIipO204I7Yb6bBSn7Lj/qvzDnf22q2
/J7lVyt4gTtcxoD8ilJ/XBdwFT9+yW/jSvSnWC1M/pwo/WFWn3ylDQ==
-----END CERTIFICATE-----
[Wed Mar  3 15:12:50 UTC 2021] Your cert is in<EMAIL_ADDRESS>[Wed Mar  3 15:12:50 UTC 2021] Your cert key is in<EMAIL_ADDRESS>[Wed Mar  3 15:12:50 UTC 2021] The intermediate CA cert is in<EMAIL_ADDRESS>[Wed Mar  3 15:12:50 UTC 2021] And the full chain certs is there<EMAIL_ADDRESS>[Wed Mar  3 15:12:50 UTC 2021] Installing cert to:/etc/nginx/certs/info.maparla.es/cert.pem
[Wed Mar  3 15:12:50 UTC 2021] Installing CA to:/etc/nginx/certs/info.maparla.es/chain.pem
[Wed Mar  3 15:12:50 UTC 2021] Installing key to:/etc/nginx/certs/info.maparla.es/key.pem
[Wed Mar  3 15:12:50 UTC 2021] Installing full chain to:/etc/nginx/certs/info.maparla.es/fullchain.pem
Creating/renewal maparla.es certificates... (maparla.es)
[Wed Mar  3 15:12:51 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:12:51 UTC 2021] Creating domain key
[Wed Mar  3 15:12:52 UTC 2021] The domain key is here<EMAIL_ADDRESS>[Wed Mar  3 15:12:52 UTC 2021] Single domain='maparla.es'
[Wed Mar  3 15:12:52 UTC 2021] Getting domain auth token for each domain
[Wed Mar  3 15:12:54 UTC 2021] Create new order error. Le_OrderFinalize not found. {
  "type": "urn:ietf:params:acme:error:rateLimited",
  "detail": "Error creating new order :: too many certificates already issued for exact set of domains: maparla.es: see https://letsencrypt.org/docs/rate-limits/",
  "status": 429
}
[Wed Mar  3 15:12:54 UTC 2021] Please check log file for more details: /dev/null
Reloading nginx docker-gen (using separate container nginx-proxy-automation-gen)...
Reloading nginx (using separate container 490d233c8554b03c902578b8f30fe6debaae666b6fbc0a30c4ac32f489cd8cb3)...
Sleep for 3600s

info.maparla.es is working with HTTPS :) The error to maparla.es I think is not solvable, I have to wait one week I read in the letsencrypt forums, but the subdomain worked. What do you think?
If I have to update the code of the personal-site container... How should I create and update the container? Thanks in advance and for your work!

thanks you very much for the tests! I will publish everything next thursday!

I will close ok? Thanks for your help!

I haven't yet had time, but will try it later this week and notify you if I find anything.


Set QEMU path accordingly to the EL systems

Since the default host for building the images is EL.
Add qemu_binary option from qemu-system-x86_64(default) to /usr/libexec/qemu-kvm.
Fix Failed creating Qemu driver: exec: "qemu-system-x86_64": executable file not found in $PATH on EL systems.
Signed-off-by: Elkhan Mammadli<EMAIL_ADDRESS>

Closed for #40


Trilium Notes

Project description
Build your personal knowledge base with Trilium Notes

homepage URL: https://github.com/zadam/trilium/wiki
source URL: https://github.com/zadam/trilium
license: AGPL v3
platforms: linux, darwin

Isn't trilium-desktop already in nixpkgs?

Right, my fault


build: use npm5 instead of yarn

npm5 is not faster than yarn, and I have heard that it can be buggy in some cases. I would recommend staying with yarn.

I agree with you that it's not faster, and that there are bugs. But the reason for moving isn't so much the speed, but the interop.
Right now it's pretty painful to develop with a linked cli and npm5. You need to npm link && rm -rf node_modules/ && yarn on the CLI dir. And then on the linked project, linking and then installing something else will break the modules as well. This is very frustrating as a CLI dev.
Another factor is that even though the CLI itself uses yarn, when in our e2e tests we use npm to ensure compatibility with most users. So using npm everywhere also has benefits.
It also helps that package-lock.json is pretty decent, and that it doesn't depend on the package manager version like yarn does.
Overall I do expect package-lock.json to become the standard.

So bringing this topic back up again, I noticed Angular's now switched to Yarn with https://github.com/angular/angular/pull/19328. Is the team's stance changing again?

No plan to change at the moment, no.


luminosity retrieval for popIII source that mirrors popII

Codecov Report

Merging #28 (d097c8f) into main (585bfc8) will increase coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main      #28      +/-   ##
==========================================
+ Coverage   54.57%   54.66%   +0.09%
==========================================
  Files         112      112
  Lines       22210    22210
==========================================
+ Hits        12120    12142      +22
+ Misses      10090    10068      -22

Impacted Files
Coverage Δ

ares/sources/SynthesisModel.py
66.48% <0.00%> (+0.54%)
:arrow_up:

ares/inference/ModelFit.py
62.99% <0.00%> (+1.17%)
:arrow_up:

ares/analysis/TurningPoints.py
75.00% <0.00%> (+2.34%)
:arrow_up:

ares/sources/StarQS.py
84.00% <0.00%> (+5.59%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 585bfc8...d097c8f. Read the comment docs.


Incorrect date with "* * * * *" cron statement

I think I've found an issue with Crontab.get_next_run_date("* * * * *").
When I call this function at e.g. 09:42:10 I get {:ok, ~N[2017-01-18 09:42:00]}.
However, I think the result should be {:ok, ~N[2017-01-18 09:43:00]}.
The Crontab.get_previous_run_date("* * * * *") function works as expected and returns {:ok, ~N[2017-01-18 09:42:00]}.
What do you think?

Hmm. I think you're right about that. I'll make a fix for that.

Can you please check if your issue is solved with v0.8.4?

Awesome!
Many thanks for patching this issue. It works as expected, now.
👍 👍 👍

Great :)


Kibana4

Issue #14: sets a good default repository/revision pair and allow user to specify their own revision preference. Bump to v0.3.0.

I set the default ref to the v4.0.0 tag to be less of a moving target. Thanks spredzy.


Updating the plotly.js version used in the plotly mimetype renderer

Hi, plotly.py maintainer here.  First off, thanks for supporting the plotly mimetype. This makes vscode-python a really nice environment for plotly.py users.
I'm working on a refactor/generalization of the plotly.py rendering subsystem (See https://github.com/plotly/plotly.py/pull/1474) and one of the things I'd like to do is automatically apply a dark theme (https://medium.com/@plotlygraphs/introducing-plotly-py-theming-b644109ac9c7) when plotly.py detects that it's running in vscode.  For this to work, the plotly.js version used in the vscode-python mimetype renderer would need to be updated so that it includes theme support.
I'd be happy to submit a PR for this if someone could point me to where the mimetype renderer is maintained (I didn't find it when searching this repo).
As a side question, do you have a recommendation for the best way to detect that a Python session is running in VSCode?  Right now I'm looking for the presence of the VSCODE_PID environment variable.
Thanks! 🙇
Environment data

VS Code version: Version 1.32.3 (1.32.3)
Extension version (available under the Extensions sidebar): 2019.3.6139

Hi @jonmmease, thanks so much for the offer.
Our mimetype rendering is here:
https://github.com/Microsoft/vscode-python/blob/325e1f025565a4455cb2d3e8e23a53c60a47b20a/src/datascience-ui/history-react/cell.tsx#L278
We actually use nteract transforms to do our rendering of different mimetypes.
For your second question, which python session are you referring to? A general python process launched, or an iPython console hosted by Jupyter? We could probably add another environment variable when we start the Jupyter server if you need it for something.

We actually use nteract transforms to do our rendering of different mimetypes.

Thanks @rchiodo, that was helpful.  Looks like the plotly.js version is being determined by the @nteract/plotly package (https://www.npmjs.com/package/@nteract/plotly), which is locked to version 1.0.0 from a couple of years ago:
https://github.com/Microsoft/vscode-python/blob/419ecb1f593b4c58af77a9230d1179e34aa4d5bb/package-lock.json#L1128-L1133
There is a 1.43.0 version available that was up to date as of a few months ago. so I think a good starting point would be bump @nteract/transform-plotly to version 5.0.0 which should cause @nteract/plotly to jump to 1.43.0.
Do you have any established processes for dependency updates like this?  I'm happy to submit a PR with just the package.json/package-lock.json updates, but let me know if there's anything else that would make things easier on your end.

For your second question, which python session are you referring to? A general python process launched, or an iPython console hosted by Jupyter?

The latter, I think.  Basically I would like to know when the Python session is likely to be displaying plotly figures in VSCode using the mimetype renderer.  It is already possible to detect whether a Python session is an IPython session, so it might be accurate enough so simply combine this with looking for the VSCODE_PID environment variable.

Do you have any established processes for dependency updates like this? I'm happy to submit a PR with just the package.json/package-lock.json updates, but let me know if there's anything else that would make things easier on your end.

Submitting a PR would be the quickest/easiest I think. Our CI system for the PR should verify that the update works.

Hi @jonmmease! I've ran into some issues trying to use latest plotly charts with VSCode's interactive window. Looks like those issues are caused by the old version of plotly bundled with VSCode Python extension. Are you going to make a pull request with version bump? I would much appreciate this :)

Hi @olegtarasov, I was hoping for https://github.com/nteract/minimal-plotly/pull/12 to get merged and published first so that we could update to the latest version of plotly.js.  I just followed up in https://github.com/nteract/minimal-plotly/issues/13.

@jonmmease Thanks for following up on the other thread! I've merged and released the bump to @nteract/plotly. Moving forward, updates will be released on a weekly cadence alongside updates in the nteract monorepo. I hope this unblocks your work here.


[WIP] Meta lamp/nft marketplace/change auction duration to end time purescript

TODO: convert Number to BigInteger. Number -> Int -> BigInteger fails, probably Number is out of bounds.

We use duration on frontend again


[AA-1300] Admin app 2.2 Azure testing

@vimayya @stephenfuqua @plioi
Please help review and merge the changes.
Changes include separate powershell script file and ARM templates which are specifically defined for AdminApp 2.2.0 in-place upgrade in Azure environment.
This implementation designed to work with existing azure resource group and ODS API.
Added new folder and files, since don't want to customize the existing code for unique purpose/ use case.
Thanks

@CSR2017 Is the intension here is to keep these additional scripts until next release and discard it when default release supports admin app 2.2 or higher or this needs to be kept longer of older version upgrades?


Fix linting, husky preventing from committing

Fix issues

[ ] Issue not raised, new issues incl. fixes

Description

Fix: can't commit in repo due to husky, working after update:

husky > pre-commit (node v14.15.3) $ eslint src husky > commit-msg (node v14.15.3) ⧗ input: My commit message here ✖ subject may not be empty [subject-empty] ✖ type may not be empty [type-empty] ✖ found 2 problems, 0 warnings ⓘ Get help: https://github.com/conventional-changelog/commitlint/#what-is-commitlint husky > commit-msg hook failed (add --no-verify to bypass)

Fix: can't commit until either installing eslint globally (not recommended) or using already installed dev dependency.
Disable react/jsx-props-no-spreading in .eslintrc and remove exceptions accordingly

Check List

[x] respect code conventions and usages
[x] tests and 100% coverage
[ ] well documented (no need for documentation?)
[x] self review

Review targets:

husky, git, eslint

Thanks @Gustl22 !


Fix: Auto-refresh file content when selected file changes

This PR fixes the issue where the file content in the workspace panel does not automatically refresh when the selected file changes externally. Now, when a file is selected and its content changes, it will automatically update without requiring the user to click on the file again.
Changes:

Added Redux store selector to get current agent state in TreeNode component
Added useEffect hook to trigger file content refetch when:

File is currently selected
File is not a directory
Agent state changes

Should fix: https://github.com/All-Hands-AI/OpenHands/issues/4968


xeus-robot-wheel not for Python 3.8

Hi,
it looks like the latest Windows wheel for Python 3.8 on pip is 0.3.3. Latest xeus-robot release for Windows is Python 3.7 only.
Could you please generate one for 3.8 too?
Thank you

Thanks for reporting!
Done! :)


Module Page

Add A module page which can list more information about the selected module

Readme
License
Ratings
Comments

Download sources like packagist github etc
Contributors to this module
Owner of the module

Nudge - any news or can I close please?

I have not feedback from community.. If this feature is not interesting we can close this issue


Add .NET Core 3.0 target that includes COM interop features

Since .NET Core 3.0 supports COM interop, a netcoreapp3.0 target should be added that defines FEATURE_COM (so Microsoft.Scripting.ComInterop stuff will be included in the assembly).

I took a quick look and it seems like the following steps would be required (to get it to compile):

Add FEATURE_COM to the props file.
Add pragmas to disable the obsoletion warnings which are probably no longer (see https://github.com/dotnet/corefx/issues/34949). File an issue with corefx to see if the ObsoleteAttribute should be removed?
Add some #if FEATURE_ASSEMBLYBUILDER_DEFINEDYNAMICASSEMBLY so it doesn't call AppDomain.DefineDynamicAssembly on .NET Core
Use reflection to invoke SignatureHelper.GetMethodSigHelper(CallingConvention, Type). I believe the API is currently available in .NET Core but it's not in reference assemblies, could file an issue with corefx to see if they're open to adding it?
ComEventSinkProxy uses remoting APIs and should be excluded with a FEATURE_REMOTING and then figure out what to do about it's use in ComEventSink.

I think we might even be able to target .NET Core 2.1? Only difference seems to be a few more obsoletion warnings.
I don't use COM myself so I'm unlikely to work on this any time soon, but I'm open to PRs.

@reflectronic @slozier I've just put out a PR for adding the support for a .NET Core 3.0 target with COM support.


apply default serializer.req before using configured serializer.req

This serializes the req using the internal asReqValue before passing the result to any configured serializer.req. This is to make it work in combination with pino-noir and a configuration redacting a value in the req object. e.g.
{
    plugin: require('hapi-pino'),
    options: {
      serializers: noir(['req.headers.authorization'])
    }
  }

It is inspired by the discussion in this issue: https://github.com/pinojs/express-pino-logger/issues/14
This change is BC breaking as before any configured serializer got the original req object without being passed through asReqValue. One test case is changed to reflect this.
I am not sure if this makes sense. Another solution would be to do this within a function passed to
options.serializers.req directly. Although this would slightly complicate the usage of pino-noir together with this plugin.
I am also not sure if this should be done for res as well and if you would like to see this as copied code (just for res instead of req) or if you prefer to have it in a separate function called with res or req returning the function.
Let me know if I should change anything.

Coverage decreased (-0.08%) to 98.229% when pulling 1fc78ce405198c5ea5df8ebd6b8866477bbc1f6b on benib:overriding-req-serializer into 4d67abf7bfc69d301d86d3725f0ed58892610da5 on pinojs:master.

Coverage decreased (-0.08%) to 98.229% when pulling 1fc78ce405198c5ea5df8ebd6b8866477bbc1f6b on benib:overriding-req-serializer into 4d67abf7bfc69d301d86d3725f0ed58892610da5 on pinojs:master.

Coverage increased (+0.06%) to 98.368% when pulling e4c3ece202628c7a9f002d7e76d23ead4d8fea9e on benib:overriding-req-serializer into 4d67abf7bfc69d301d86d3725f0ed58892610da5 on pinojs:master.

Coverage increased (+0.06%) to 98.368% when pulling e4c3ece202628c7a9f002d7e76d23ead4d8fea9e on benib:overriding-req-serializer into 4d67abf7bfc69d301d86d3725f0ed58892610da5 on pinojs:master.

Mhh..I don't know. This results in an issue that the serializes doesn't get the whole information which breaks existing serializers and doesn't feel like best practice.
Perhaps it's better so pass req and additionally asReqValue to the serializer so the dev can decide if he wants the whole request object or just the simplified:
options.serializers.req = (req, asReqValue) => (
  (options.serializers.req | asReqValue)(req, asReqValue)
)

@felixheck Yes, this would break existing serializers that are implemented against the complete request object. This is a problem. I agree.
I'd rather export asReqValue from hapi-pino instead of passing it in thought.
As far as I know me as a dev has to apply asReqValue at some point though to stay compatible with https://github.com/pinojs/pino-toke#supports.
I do not have a lot of experience with pino, so I am not sure what the strategy about keeping these serializers "compatible" as in producing the same req log object. But I guess this should be synced with https://github.com/pinojs/express-pino-logger/issues/14 somehow.
I am happy to help with any implementation but would rather leave the decision to people who have more experience with pino and these http-style serializers as it can have quite some implications for the future.

Mhh..yeah, exposing asReqValue seems to be the smartest solution. At least in my point of view.

@felixheck so should we go for not overriding but exposing asReqValue and documenting that one should use asReqValue in any custom serializer passed to options.serializers.req? I would send another PR in favor of this one then.
Any hard feelings from @davidmarkclements as the implementation in this PR is based on your comment here: https://github.com/pinojs/express-pino-logger/issues/14 and handling this in the same way would make a lot of sense imo.

@mcollina has the last word ;-)

How about attaching req.raw as a non-enumerable property when invoking the original serializer?

Coverage increased (+0.06%) to 98.368% when pulling d5fdeef1aa5c0535ae25b92d2a2d538883e1e498 on benib:overriding-req-serializer into 4d67abf7bfc69d301d86d3725f0ed58892610da5 on pinojs:master.

Coverage increased (+0.06%) to 98.368% when pulling d5fdeef1aa5c0535ae25b92d2a2d538883e1e498 on benib:overriding-req-serializer into 4d67abf7bfc69d301d86d3725f0ed58892610da5 on pinojs:master.

See https://github.com/pinojs/pino-http/pull/43

@jsumners the plan is that hapi-pino could or should wrap around pino-http?

@benib unfortunately I don't think so. But I believe whatever solution we settle upon there should be used here.

hey guys thanks for all the input here!
@benib  - pino-http is the basis for express and koa loggers, but Hapi is too different to integrate with efficiently so it has it's own implementation
@felixheck - the full req serialization is a bug that occurs when adding multiple serializers, which is unfortunate. However it doesn't happen in the main case. To retain the full request behaviour we simply provide a new serializer, let's not change the serializer format as this is more disruptive.
Regarding how we collaborate on Pino and Pino's ecosystem:
@mcollina and I wrote pino core together, @mcollina  wrote hapi-pino, I wrote pino-http (express-pino-logger, koa-pino-logger, ...) and as in pretty much all cases when there's a hot topic, the "last word" tends to come from agreement between @mcollina, @jsumners and myself. In this particular case I trust @jsumners judgement (as does @mcollina) on which way to take pino-http, which should tease out a direction for this issue.

thanks for all the explanations. I will leave this PR open and close it once I come up with one fixing the issue along the lines of pino-http once https://github.com/pinojs/pino-http/pull/43 is settled. Or anyone else picks this up. No hard feelings here.

thanks you @benib !

pino-http has been updated to v3.0.0 which solves this same problem. I recommend looking at how it was solved there and applying that solution to this PR.

@benib would you mind updating this PR?

@mcollina sorry was pretty overwhelmed with work the last days. will try to get to that before the end of the year. If someone has the resources to take over, feel free.


Fix quantization tutorial

Fix this tutorial on quantization.

Remove outdated dependencies, as pointed out in #10463.

The change made in #10592 was incomplete. The example on how to run transform_graph to quantize a graph should follow this tutorial about graph_transforms, but the argument --inputs='Mul' is missing.

@petewarden I'm also wondering when/how will the tutorial on tensorflow.org get updated? I'm assuming it should be the same as quantization.md that I'm fixing?
Thanks!

Can one of the admins verify this patch?

@saxenasaurabh does that look right?
Jenkins, test this please.

This looks correct, and if it's wrong I'm sure we'll get another PR to fix it :P


the code for .15 is off is should be this

rom random import randint
board = []
for x in range(0, 5):
board.append(["O"] * 5)
def print_board(board):
for row in board:
print " ".join(row)
print_board(board)
def random_row(board):
return randint(0, len(board) - 1)
def random_col(board):
return randint(0, len(board[0]) - 1)
ship_row = random_row(board)
ship_col = random_col(board)
print ship_row
print ship_col
Everything from here on should be in your for loop
don't forget to properly indent!
for turn in range(4):
print "Turn", turn + 1
guess_row = int(raw_input("Guess Row: "))
guess_col = int(raw_input("Guess Col: "))
if guess_row == ship_row and guess_col == ship_col:
print "Congratulations! You sank my battleship!"
else:
if guess_row not in range(5) or
guess_col not in range(5):
print "Oops, that's not even in the ocean."
elif board[guess_row][guess_col] == "X":
print( "You guessed that one already." )
else:
print "You missed my battleship!"
board[guess_row][guess_col] = "X"
print_board(board)

@acfarris1 Thanks for your issue. Can you please provide more details, which section and which exercises is that. Might be worth to format the code with markdown appropriately, to make it more readable.


Use parquet read speed-ups from fastparquet.api.paths_to_cats.

This PR propagates parquet partition read speed-ups merged in https://github.com/dask/fastparquet/pull/471/
Before this change, code was duplicated from fastparquet.
After this change, an optimized function is imported from fastparquet.
Since fastparquet versions range is not explicit in dask,
the import is for now made optional, reverting to existing implementation if
using older fastparquet.
@rjzamora I am not entirely happy with "try-catch imports" and would rather have fastparquet as e.g. an optional dependency with well-specced version ranges, but for now this could be a work-around.

[x] Tests added / passed
[x] Passes black dask / flake8 dask

Potentially related issues
https://github.com/dask/dask/issues/5272
https://github.com/dask/dask/issues/4701

OK, I've released fastparquet 0.3.3 to PyPI. CF will start building in a bit.
For now, let's merge this. I opened https://github.com/dask/dask/issues/5865 as a followup for removing this compat code.

Thanks @ig248!


Memory (System): 3.88GB (1.98GB free)

VSCode Version:
OS Version:

Steps to Reproduce:

Please provide more details about what kind of problem you are running into.

Closing because of lack of information


Merge video fixes

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/desktop/beyond-the-basics-content-sharing.md
:white_check_mark:Succeeded

docs/desktop/chat-room-features-in-lync-sdk.md
:white_check_mark:Succeeded

docs/desktop/conversation-manager.md
:white_check_mark:Succeeded

docs/desktop/core-concepts-persistent-chat.md
:white_check_mark:Succeeded

docs/desktop/establish-trust-on-contextual-conversation-clients.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-content-sharing.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-desktop-application-and-display-sharing.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-lync-contact-lists.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-lync-controls.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-lync-conversations.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-lync-meetings.md
:white_check_mark:Succeeded

docs/desktop/get-started-with-persistent-chat.md
:white_check_mark:Succeeded

docs/desktop/how-to-add-enhanced-presence-to-a-web-application.md
:white_check_mark:Succeeded

docs/desktop/how-to-add-rename-or-remove-a-custom-group.md
:white_check_mark:Succeeded

docs/desktop/how-to-attach-a-file-to-the-content-stage-of-a-conversation.md
:white_check_mark:Succeeded

docs/desktop/how-to-create-a-silverlight-page-that-displays-a-lync-presence-control.md
:white_check_mark:Succeeded

docs/desktop/how-to-display-and-publish-custom-availability-in-lync-sdk.md
:white_check_mark:Succeeded

docs/desktop/how-to-send-and-receive-context-information-in-a-lync-conversation.md
:white_check_mark:Succeeded

docs/desktop/how-to-share-a-powerpoint-slide-deck.md
:white_check_mark:Succeeded

docs/desktop/how-to-sign-a-user-in-to-lync.md
:white_check_mark:Succeeded

docs/desktop/how-to-start-a-lync-im-conversation.md
:white_check_mark:Succeeded

docs/desktop/understanding-lync-controls.md
:white_check_mark:Succeeded

docs/desktop/what-s-new-in-lync-2013-sdk.md
:white_check_mark:Succeeded

docs/desktop/what-you-can-do-with-content-sharing.md
:white_check_mark:Succeeded

docs/desktop/what-you-can-do-with-enhanced-presence.md
:white_check_mark:Succeeded

docs/desktop/what-you-can-do-with-lync-controls.md
:white_check_mark:Succeeded

docs/technical-articles/set-up-the-ucwa-event-channel-to-receive-incoming-notifications.md
:white_check_mark:Succeeded

docs/ucma-sdk/creating-a-generic-bot.md
:white_check_mark:Succeeded

docs/ucma-sdk/creating-a-lync-bot.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


tf.unravel_index (Was: tf.argmin across all dimensions)

Hi,
tf.argmin only works in one dimension.
Let's say I have a picture which is a 2x2 array of pixels (each pixels is a 3 value array) and I want to know which pixel is closest to a certain color. In that case I have to reshape the 2x2 array to a 1-D array then get the min index, then find out that index to what 2x2 array position corresponds.
Couldn't I just as tensorflow to give me the index in the 2x2 array. For instance [1,15]?
I really don't understand the reason for having to translate between lineal index and array location all the time.
Thanks.

I think it'd be better to handle this the way numpy does it, with np.unravel_index: https://bytes.com/topic/python/answers/509074-numpy-argmin-multidimensional-arrays.  Making tf.argmin do this directly would either require a new op or an unpleasant boolean flag that indexes over everything, and it still wouldn't be what you want since most of time you want to minimize over some but not all of the dimensions (e.g., the last three dims of a 4D batched image tensor).
Cc @aselle since unravel_index sounds vaguely index related, but I'll mark this contributions welcome for now.

where is tf.unravel_index in the document? I cannot find it.
Thank you.

Not sure which document you mean, but tf.unravel_index doesn't exist yet.  np.unravel_index is described in that thread I linked to.

Yeah. I know np.unravel_index. but can np.unravel_index be used to build the graph?
I write a simple example. How to achieve that using Tensorflow?
Thank you.
ksize = 3
stride = 1

input_image = tf.placeholder(tf.float32, name='input_image')

#conv1
kernel = tf.Variable(tf.truncated_normal([ksize, ksize, 3, 16],stddev=0.1),
                    name='kernel')
conv = tf.nn.conv2d(input_image, kernel, [1,stride,stride,1], padding='SAME')
biases = tf.Variable(tf.constant(0.0, shape = [16]), name = 'biases')
bias = tf.nn.bias_add(conv, biases)
conv1 = tf.nn.relu(bias, name='conv1')

#pool1
pool1, pool1_indices = tf.nn.max_pool_with_argmax(conv1, ksize=[1, 2, 2, 1],
                                                  strides=[1, 2, 2, 1],
                                                  padding='SAME', name='pool1')

#upsample by assigning the values of pool1 to the position in unpooling Tensor according to pool1_indices
indices = pool1_indices
unravel_pool1_indices = np.unravel_index(indices,[4,32,32,16])
unravel_pool1_coordinates = np.array(unravel_pool1_indices)
coor_shape = np.shape(unravel_pool1_coordinates)
unravel_pool1_coordinates = np.reshape(unravel_pool1_coordinates,(coor_shape[0],coor_shape[1]*coor_shape[2]*coor_shape[3]*coor_shape[4]))
unravel_pool1_coordinates = unravel_pool1_coordinates.T

values = pool1
values = np.reshape(values,(np.size(values)))

up1 = tf.constant(0.0, shape = [4,32,32,16])
delta = tf.SparseTensor(unravel_pool1_coordinates, values, shape = [4,32,32,16])

result = up1 + tf.sparse_tensor_to_dense(delta)

with tf.Session() as session:
    session.run(tf.initialize_all_variables())
    test_image = np.random.rand(4,32,32,3)
    sess_outputs = session.run([pool1, pool1_indices],
                               {input_image.name: test_image})

Someone who'd have to write a TensorFlow version of np.unravel_index, which could be called tf.unravel_index.  We might not do that soon, so PRs adding it would be welcome.  tf.unravel_index could either be a new C++ op or something written in Python.

Here's a sketch of a possible Python implementation using cumprod:
import tensorflow as tf

def unravel_index(indices, shape):
  with tf.name_scope('unravel_index'):
    indices = tf.expand_dims(indices, 0)
    shape = tf.expand_dims(shape, 1)
    strides = tf.cumprod(shape, reverse=True)
    strides_shifted = tf.cumprod(shape, exclusive=True, reverse=True)
    return (indices // strides_shifted) % strides

s = tf.Session()
out = unravel_index([22, 41, 37], (7, 6))
print(s.run(out))
# ==> [[3 6 6]
#      [4 5 1]]

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

@ibab The last line in the function unravel_index should be return (indices % strides) // strides_shifted.

I think unravel_index is quite a useful feature that could be used in many places. Created a PR #14895 to add the C++ kernel for it. Please take a look if interested.


Non-minified package distribution

@eavichay Hey, man! Thanks for your brilliant work!
Using Chrome Dev Tools, the Slim class appears as u
u?
I found that this is because, even though I'm using a stack with npm, vite, esbuild, rollup... the import { Slim } from 'slim-js' actually brings in the code from the minified index.js at node_modules/slim-js/dist/index.js
:scream:
Could you please let the src folder intact in the package exports? So I can import from slim-js/src or, really, any other file on it?
From your README:
 It also provides access to it's internals, if you want to hack things out.

This is great! If it can also provide the non-minified internals for us, mere mortals, it will become even greater!
Thank you very much. :bow:

Done. https://github.com/slimjs/slim.js/commit/c04cc04968f1bd86fbaaf937b70f5d91dd1e809f
Published with 5.0.9


Error when run test.py

Hi, there is a problem hope you can help me, thank you.
File "/home/hd/ssd.pytorch/data/voc.py", line 222
gts.append([label, *(int(bb.text) - 1 for bb in bbox)])
^
SyntaxError: invalid syntax
This error occur when I run the test.py. Thank you again.
^ under the *

Hey, I can't reproduce that on my computer and can't really think of why that would be an issue off the top of my head.  I will look into this more after class and get back to you, sorry about that.

It's my bad. I find my python version is 2.7......sorry to disturb

No worries at all! We are working to support python 2.7, so maybe check back in the future.


Does a single AMD GPU can be shared among containers? Things like MPS of Nvidia

Does a single AMD GPU can be shared among several containers?
Nvidia GPU has things like MPS https://docs.deep-hybrid-datacloud.eu/en/latest/technical/others/gpu-sharing-with-mps.html, but I'm not aware of any similar feature of AMD GPU.

The ROCm driver supports multiple concurrent processes on the GPU natively. We don't need MPS for that. This should also work for multiple containers.
The processes end up sharing GPU resources, which can lead to performance and functionality problems. Compute queues virtual memory IDs can be oversubscribed, but when that happens, performance drops significantly. GPU memory cannot be oversubscribed. This may result in out-of-memory errors causing memory-hungry applications to fail.

The ROCm driver supports multiple concurrent processes on the GPU natively. We don't need MPS for that. This should also work for multiple containers.
The processes end up sharing GPU resources, which can lead to performance and functionality problems. Compute queues virtual memory IDs can be oversubscribed, but when that happens, performance drops significantly. GPU memory cannot be oversubscribed. This may result in out-of-memory errors causing memory-hungry applications to fail.

Thank you!  Any guides on how to config ROCm-docker start several containers and share the same GPU device?

I don't think you need to do anything special. Just start multiple containers. They will share the GPU in the same way as multiple processes in the same container.

I don't think you need to do anything special. Just start multiple containers. They will share the GPU in the same way as multiple processes in the same container.

But then how to do isolation? e.g limit the gpu resources usage if coming from a low priority container.  Do you aware any solution to do GPU performance QoS among different processes/containers?
Thanks!

I don't think you need to do anything special. Just start multiple containers. They will share the GPU in the same way as multiple processes in the same container.

But then how to do isolation? e.g limit the gpu resources usage if coming from a low priority container. Do you aware any solution to do GPU performance QoS among different processes/containers?
Thanks!

Well, that's a different question. There is currently no support in the driver for QoS or resource isolation between containers. Some proposals for cgroup controls to manage GPU resources are being discussed upstream to solve this problem.


Earlier fixes - Piotr

@PiotrFusik The MR #2 can be closed. Please use this one

Do not understand do we need still this PR ?


Bump minimum version to Go1.8

Go1.8 went GA on AppEngine.
The lack of Go1.8 support was the primary reason for much
of these backwards compatibility hacks.
Bumping to Go1.8 still ensures that we're supporting at least
the latest 2 versions of Go, which are Go1.8 and Go1.9.

Not yet, please. The cloud libraries may still need to support 1.6.
/cc @zombiezen

I'll be holding off on this until Jan 2nd, 2018.

Do I still have to hold off on this?
According to https://cloud.google.com/appengine/docs/standard/go/release-notes, Go1.8 has been GA since October 25th, 2017.

Until AppEngine is off of 1.6, we need to support it.
golang.org/x/oauth2 still works with 1.6 (since we depend on it).

(If I understood correctly) In a meeting we had some time ago, my understanding was to only support the last 2 versions of Go on AppEngine and that anything older would be the responsibility of the user to vendor their dependencies. Now that AppEngine has both Go1.8 and Go1.9, doesn't that remove our responsibility to keep supporting Go1.6?
If we do have keep Go1.6 support, when is that going to be removed from AppEngine? (Tangent) I should also note, that starting on April 30th, we're going to merge the dev branch of protobufs, which will effectively raise the minimum version to Go1.9.

With the release of Go1.11 on AppEngine, I'd like to revisit dropping support for anything lower than Go1.8. I'd like to make the equivalent change to protocol buffers as well.
Anyone objections to dropping support?
\cc @sbuss @jba @neild @zombiezen

Wait until Nov. 1.

Times up!


"Web login required" error for gmail

I'm attempting to write an app that checks my gmail, but I'm getting the following exception:
* NO [WEBALERT <<long link>>] Web login required.
When I visit the link provided, it says "Google prevented a sign-in from a less secure app".  I was able to note in my Google account that that device is trusted, but it still does not work and I still get the same error.
Any ideas?  I have the SSL flag set to true on the initialization of the ImapClient.

You need to configure gmail to allow "insecure access" for IMAP.
The link tries to explain that you need to jump thru quite a few hoops before it works. (had the same thing with smtp not long ago)

Thanks!

How did you solve it?

I googled configure gmail to allow "insecure access" for IMAP and followed the instructions.


Updating requirements-dev filename

The dependencies txt file "requirements-dev" appears to be named "dev-requirements" by mistake which results in pip package installation error.
Proposed changes:

...

Status (please check what you already did):

[Y] made PR ready for code review
[ ] added some tests for the functionality
[Y] updated the documentation
[ ] updated the changelog

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[5.1] Storage has(null) returns true

If have a valid filesystem and if has() receives a null value as argument it returns true. It seems to take the root of it as result.
Storage::disk('mydisk')->has($file) returns true if $file == null

Closing since this issue belongs to flysystem.

Upstream notification: This is fixed and tagged as 1.0.21.

Thanks @frankdejonge. ;)


Add config option for locale date

Is there a possibility for Ter to be translated to other languages? I can try to translate it to Portuguese, if at all possible.

Hmm I'm not sure what exactly the translation would be I this case. 99% of text on the pages is user's content.
Can you give a specific example?

Yes, there are only a few strings that can be translated, like "Child pages" or "Child Tags". Since there aren't much more, perhaps this could be directly translated by the user in config.json? Also, the date format could be added as an option as well. What do you think?

Yeah I can provide options for these labels directly in the config file so that user can customize if needed.For the date it’s also possible to add a config option for locale so that dates are formatted accordingly: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toLocaleDateStringOn 28 Sep 2022, at 18:52, Tiago Jacinto @.***> wrote:
Yes, there are only a few strings that can be translated, like "Child pages" or "Child Tags". Since there aren't much more, perhaps this could be directly translated by the user in config.json? Also, the date format could be added as an option as well. What do you think?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

That's perfect, awesome :)

The date formatting should be configurable in the latest release. Here's the config example: https://ter.kkga.me/usage#options.
For Portuguese you can use pt (https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)
Let me know if it works as intended for you.

I can't seem to use Ter 0.8.0, I get this error:
error: Uncaught TypeError: Cannot convert undefined or null to object     return handleCache(template, options)(data, options);                                          ^     at Function.keys (<anonymous>)     at eval (eval at compile (https://deno.land/x/eta@v1.12.3/compile.ts:24:16), <anonymous>:57:8)     at render (https://deno.land/x/eta@v1.12.3/render.ts:90:42)     at buildPage (https://deno.land/x/ter@0.8.0/build.ts:101:16)     at buildContentFiles (https://deno.land/x/ter@0.8.0/files.ts:61:24)     at async Promise.all (index 0)     at async generateSite (https://deno.land/x/ter@0.8.0/main.ts:116:59)     at async main (https://deno.land/x/ter@0.8.0/main.ts:216:3)

hmm... hard to say what went here. Can you try running with -r flag to make sure it reloads all dependencies?
deno run -A -r --unstable https://deno.land/x/ter/main.ts

WHen I run that command, this happens:
error: Uncaught TypeError: Cannot convert undefined or null to object     return handleCache(template, options)(data, options);                                          ^     at Function.keys (<anonymous>)     at eval (eval at compile (https://deno.land/x/eta@v1.12.3/compile.ts:24:16), <anonymous>:57:8)     at render (https://deno.land/x/eta@v1.12.3/render.ts:90:42)     at buildPage (https://deno.land/x/ter@0.8.0/build.ts:101:16)     at buildContentFiles (https://deno.land/x/ter@0.8.0/files.ts:61:24)     at async Promise.all (index 0)     at async generateSite (https://deno.land/x/ter@0.8.0/main.ts:116:59)     at async main (https://deno.land/x/ter@0.8.0/main.ts:216:3)

If run this command: deno run -A -r --unstable<EMAIL_ADDRESS>--serve
It works, both generating _site and --serve.
But it does not work if I use<EMAIL_ADDRESS>

Is there any chance I can

Sure, here: https://github.com/tiagojacinto/tgjcnt

Thanks for sharing. This was helpful as I found the issue.
I'll push an update soon, but in the meantime if you want to build a site with 0.8.0, just add an empty navigation object in the config.json:
{
  "navigation": {}
}

New release should work (without having to adjust config file): https://github.com/kkga/ter/releases/tag/0.8.1
Thanks 👍🏼

I confirm that it is working!
Thanks!


Namig bug

I did forget about this thing we've discussed earlier... So here's the update to the error name.

:+1:

Gimme a sec to up the version


Array type is not cast or checked

✏ Describe the bug
Transforming JSON to Data object, type of elements in array is not checked or converted.
I set "cast_and_transform_iterables" but it changed nothing (no, config is not cached)
↪ To Reproduce
it('array type checked', function () {

class TypedArray extends Data {
    public bool $bool;
    public bool $bool2;
    public bool $bool3;
    public bool $bool4;
    /** @var bool[] */
    public array $booleans1;
    /** @var array<int, bool> */
    public array $booleans2;
    /** @var array|bool[] */
    public array $booleans3;
    /** @var bool[] $booleans4 */
    public array $booleans4;
}

$sample = <<<JSON
{
    "bool": 1,
    "bool2": true,
    "bool3": 0,
    "bool4": false,
    "booleans1": [1,0],
    "booleans2": [1,0],
    "booleans3": [1,0],
    "booleans4": [1,0,"Hello"]
}
JSON;

dd(TypedArray::from($sample));

});

you get
TypedArray {#1226 ▼
  #_additional: []
  #_dataContext: null
  +bool: true
  +bool2: true
  +bool3: false
  +bool4: false
  +booleans1: array:2 [▼
    0 => 1
    1 => 0
  ]
  +booleans2: array:2 [▼
    0 => 1
    1 => 0
  ]
  +booleans3: array:2 [▼
    0 => 1
    1 => 0
  ]
  +booleans4: array:3 [▼
    0 => 1
    1 => 0
    2 => "Hello"
  ]
}

✅ Expected behavior
Should convert types or throw exception if type is incorrect. Instead, nothing is done to the array contents.
I expected it to at least be consistent with bool fields, which are converted to boolean.
🖥 Versions
Laravel: 10.48.7
Laravel Data: 4.6
PHP: 8.3.7

This will be fixed in the next release, since it relies on PHP's way of casting you might end up with some stranger results in such case you probably want to write your own cast.

I think that's good enough, thanks for fixing it!

I tried to use this in a controller method like so:
public function localNames(GetLocalNames $request)

class GetLocalNames extends Data
{
    /**
     * @var string[] - array with translation keys (%blabla|subs)
     */
    public array $tokens;
}

Unfortunately, instead of throwing some validation exception, it now crashes.
ErrorException: Array to string conversion
/srv/flowbox/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php:255
/srv/flowbox/vendor/spatie/laravel-data/src/Casts/BuiltinTypeCast.php:35
/srv/flowbox/vendor/spatie/laravel-data/src/Casts/BuiltinTypeCast.php:25
/srv/flowbox/vendor/spatie/laravel-data/src/DataPipes/CastPropertiesDataPipe.php:181
/srv/flowbox/vendor/spatie/laravel-data/src/DataPipes/CastPropertiesDataPipe.php:153
/srv/flowbox/vendor/spatie/laravel-data/src/DataPipes/CastPropertiesDataPipe.php:110
/srv/flowbox/vendor/spatie/laravel-data/src/DataPipes/CastPropertiesDataPipe.php:42
/srv/flowbox/vendor/spatie/laravel-data/src/Support/ResolvedDataPipeline.php:46
/srv/flowbox/vendor/spatie/laravel-data/src/Resolvers/DataFromSomethingResolver.php:44
/srv/flowbox/vendor/spatie/laravel-data/src/Support/Creation/CreationContextFactory.php:186
/srv/flowbox/vendor/spatie/laravel-data/src/Concerns/BaseData.php:46
/srv/flowbox/vendor/spatie/laravel-data/src/LaravelDataServiceProvider.php:53

Can I plug in my own version of BuiltinTypeCast instead of monkeypatching the library?


Add DoF extraction function

So the idea was that I could throw any Function, subfunction or expression into a function and get a numpy vector back that I could plot. Basically it does:

Function -> f.vector()
subfunction -> interpolate(f, fs).vector()
expression -> project(expr).vector()

and I'm not sure anymore how useful this is. Let me know.
Edit: Reason for build failure: the behaviour seems to have changed going from 2017.2 to 2018.1 for the subfunction. Will investigate later.

Is this still relevant?
A more general approach would transform a function/subfunction/expression (optional argument is the function space/mesh?) into a Function. This can then be plotted. If you prefer the numpy.array, call .vector()[:] afterwards.
use case could be:
f = XDMFFile(....)
f.write(helpers.to_function(u), t)
f.write(helpers.to_function(sqrt(u)), t)
f.write(helpers.to_function(sqrt(u), super_fine_plotting_mesh), t)


Suggestion: button to transfer pokemon

It would be neat if a button can be added which triggers an API call to PokemonGo to transfer the pokemon clicked. This would cleaning up a list of 200+ pokemons with lots of duplicates more easy as the app itself is not keen in this as you have to tab multiple times to do one transfer.

Great idea! I'm going to work on this tonight

I see it works however it takes quite some time to make the transfer (probably because the API is slow on their end). Maybe some loading/transferring indication would be nice.


Namespace in configuration management

After discussion with @abhinavdas and @jessesuen, the current implementation of configuration management is not ideal from the namespace perspective. The current behavior is to limit namespace as the username of the user whom created the configuration.
The changed behavior is user can create configuration with different namespace as far as the namespace/config_name is unique.

Not going to do it. Closing now


sql/mysql: Ent based RevisionReadWriter implementation for mysql

💯
Code looks really good but added a few questions/comments.


Clear local storage in web browser

Hi @Adrian,
Is there a way to clean local storage of the browser via opentest?
The test will accept a token that is valid for 1 hour so iget for the firat time a window with the option to click on a button that saves the token, but for the second time if i run the test the token is valid so i do not get that windows. If i can for every run clean the local storage i will force the window to appear every time

You should be able to do this by running  client-side JavaScript:
- description: Clear local storage
  action: org.getopentest.selenium.ExecuteScript
  args:
    script: |
      localStorage.clear();

ok i use it and sounds it works but not sure how to check it.

If you want to verify that the local storage was cleared add this line to your client-side script:
alert(JSON.stringify(localStorage))

You should get a dialog box displaying {}.


[Flaky Test] ProducerCannotRetrieveMetadataWhenProxyIsInvalid

ProducerCannotRetrieveMetadataWhenProxyIsInvalid

Result Details
Failed 24m ago on Mac-1614625516396
Duration0:07:28.150
Ownernot available
Date started3/1/2021, 11:24:05 AM
Date completed3/1/2021, 11:31:33 AM
Failing since24m ago
Failing since build20210301.2

Debug

Work items

Attachments

History

Attachments*

Error message
Expected: instance of <System.Net.WebSockets.WebSocketException> or instance of <System.TimeoutException>
But was:  <System.NullReferenceException: Object reference not set to an instance of an object.
at System.Net.WebSockets.ClientWebSocket.SendAsync(ArraySegment1 buffer, WebSocketMessageType messageType, Boolean endOfMessage, CancellationToken cancellationToken) at Microsoft.Azure.Amqp.Transport.WebSocketTransport.WriteAsync(TransportAsyncCallbackArgs args) at Microsoft.Azure.Amqp.AsyncIO.AsyncBufferWriter.Write(TransportAsyncCallbackArgs args) --- End of stack trace from previous location where exception was thrown --- at Azure.Messaging.EventHubs.Amqp.AmqpConnectionScope.CreateAndOpenConnectionAsync(Version amqpVersion, Uri serviceEndpoint, Uri connectionEndpoint, EventHubsTransportType transportType, IWebProxy proxy, String scopeIdentifier, TimeSpan timeout) in /_/sdk/eventhub/Azure.Messaging.EventHubs/src/Amqp/AmqpConnectionScope.cs:line 402 at Microsoft.Azure.Amqp.FaultTolerantAmqpObject1.OnCreateAsync(TimeSpan timeout)
at Microsoft.Azure.Amqp.Singleton1.GetOrCreateAsync(TimeSpan timeout) at Microsoft.Azure.Amqp.Singleton1.GetOrCreateAsync(TimeSpan timeout)
at Azure.Messaging.EventHubs.Amqp.AmqpConnectionScope.OpenManagementLinkAsync(TimeSpan timeout, CancellationToken cancellationToken) in /_/sdk/eventhub/Azure.Messaging.EventHubs/src/Amqp/AmqpConnectionScope.cs:line 243
at Microsoft.Azure.Amqp.FaultTolerantAmqpObject1.OnCreateAsync(TimeSpan timeout) at Microsoft.Azure.Amqp.Singleton1.GetOrCreateAsync(TimeSpan timeout)
at Microsoft.Azure.Amqp.Singleton1.GetOrCreateAsync(TimeSpan timeout) at Azure.Messaging.EventHubs.Amqp.AmqpClient.GetPartitionPropertiesAsync(String partitionId, EventHubsRetryPolicy retryPolicy, CancellationToken cancellationToken) in /_/sdk/eventhub/Azure.Messaging.EventHubs/src/Amqp/AmqpClient.cs:line 330 at Azure.Messaging.EventHubs.Amqp.AmqpClient.GetPartitionPropertiesAsync(String partitionId, EventHubsRetryPolicy retryPolicy, CancellationToken cancellationToken) in /_/sdk/eventhub/Azure.Messaging.EventHubs/src/Amqp/AmqpClient.cs:line 367 at Azure.Messaging.EventHubs.EventHubConnection.GetPartitionPropertiesAsync(String partitionId, EventHubsRetryPolicy retryPolicy, CancellationToken cancellationToken) in /_/sdk/eventhub/Azure.Messaging.EventHubs/src/EventHubConnection.cs:line 370 at Azure.Messaging.EventHubs.Producer.EventHubProducerClient.GetPartitionPropertiesAsync(String partitionId, CancellationToken cancellationToken) in /_/sdk/eventhub/Azure.Messaging.EventHubs/src/Producer/EventHubProducerClient.cs:line 439 at Azure.Messaging.EventHubs.Tests.EventHubProducerClientLiveTests.<>c__DisplayClass36_1.<<ProducerCannotRetrieveMetadataWhenProxyIsInvalid>b__2>d.MoveNext() in /Users/runner/work/1/s/sdk/eventhub/Azure.Messaging.EventHubs/tests/Producer/EventHubProducerClientLiveTests.cs:line 1188 --- End of stack trace from previous location where exception was thrown --- at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter1.BlockUntilCompleted()
at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
at NUnit.Framework.Internal.ExceptionHelper.RecordException(Delegate parameterlessDelegate, String parameterName)>

Stack trace
   at Azure.Messaging.EventHubs.Tests.EventHubProducerClientLiveTests.ProducerCannotRetrieveMetadataWhenProxyIsInvalid() in /Users/runner/work/1/s/sdk/eventhub/Azure.Messaging.EventHubs/tests/Producer/EventHubProducerClientLiveTests.cs:line 1188

cc @jsquire

This failure occurred within the AMQP library in a code path not controlled by the test suite or client library.   From inspection, it appears to be a transient network issue, as the other two operations using the same approach passed their assertion.
I'm going to close this for now, but if we continue to see this occur,  I'll open an issue for the AMQP library and ask them to investigate further.


Add enterprise Kubernetes test, installation, and deployment notes.

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] I have read the CONTRIBUTING document.
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

This PR adds enterprise Kubernetes test, installation, and deployment notes to the documentation.
I have tested this locally and our Staging AKS cluster. The Prometheus metrics were consumed through DataDog and matched the output the Metrics.md document.

@utkuozdemir I was wondering if we can merge this soon?
Also wanted to say a massive thank you for putting this exporter!

@echoblag Ah, I'm sorry that I couldn't give attention to this PR. I'll try to go through it as soon as possible, then we can merge.
Many thanks for the PR (especially documentation is very welcome :) ) and for using the exporter. I was wondering if anyone was using it in an enterprise setting.


Prblm

how to  run sudo commands
also cannot dwnload packages

If you run the robot on heroku I think it is not possible to run there suod commnds, on vps servers it is possible!

vps also not yet working
sudo commands dead

Please send a screenshot or the error!


Add HPKE DHKEM provider support for EC, X25519 and X448.

This is a implementation of the HPKE DHKEM for EC and ECX.
The KEM algorithm is implemented using the KEM provider interface.
Extara APIS were added to support authentication.
The interface for deriving keys using DHKEM uses the keygen API, by supplying the seed (ikm) as a settable param

Checklist

[x] documentation is added or updated
[x] tests are added or updated

CIs are relevant.

@sftcd this is a first pass, that just does the X25519 & X448 KEM.
It includes code that is shared with the rest of hpke in the hpke_util.c
We will have to have a talk about that I guess - but you can modify it to suit your needs.
(In order to commit this I obviously need those bits for it to work).
Rather than jamming an API into the KEM to do the keyderive -  I have added to the keygen which seems to fit much nicer (i,.e. this gets rid of the dummy key I was creating)_
The supplied test shows the interfaces...
I have added an EVP_PKEY_auth_encapsulate_init() and EVP_PKEY_auth_decapsulate_init() for the auth case.
The KEM already had an EVP_PKEY_encapsulate_init()/EVP_PKEY_encapsulate() interface.
Inside the DHKEM code it it setting up all the params related to kem_id, kdf and kdfdigest based on whether it is X25519 or X448. If we were to require overrides in the future they could be set via set_params() - but they are not really needed.
Let me know if you want to set up a meeting at some point..

@sftcd this is a first pass, that just does the X25519 & X448 KEM. It includes code that is shared with the rest of hpke in hpke_util.c We will have to have a talk about that I guess

Good stuff. I'll ping ya for a time to chat next week.

Just started to play with this - added a bit of test code for when doing HPKE encryption, just for x25519 and with base mode, and not pushed anywhere yet. I'm getting a failure at the 2nd call to EVP_PKEY_encapsulate() if no initial key material (ikm) is supplied (or if a param with a NULL/O ikm is supplied) - what's the intent for that case? RFC9180 says that means to generate some random ikm, but should the HPKE code generate the random ikm and give that to your provider or should generation of the random ikm happen within the provider if no specific ikm is supplied by the caller? My initial guess was the latter but seems that's not working. (For ECH usage, I think the norm will be to not have any caller-supplied ikm from TLS clients.)

It doesnt do it currently.. I wasnt sure if I should do that internally if no ikm is supplied, or whether the caller does it..

It doesnt do it currently.. I wasnt sure if I should do that internally if no ikm is supplied, or whether the caller does it..

I'd say it might be better to generate the random ikm inside the provider, for two reasons: 1) that code should know better how many random octets to use and whether e.g. some h/w is available and 2) I'd guess some applications might be better off never knowing how to re-generate the same private key. That said, for ECH it doesn't make much difference, so long as the sender's ephemeral private key can be used a 2nd time by the same process in the event of a HelloRetryRequest.

Ok will change it - its not a big change...

Do you have any comment on the test vector for X25519 private keys..
Looking at RFC 9180 A.1.1
it has this
skEm:
52c4a758a802cd8b936eceea314432798d5baf2d7e9235dc084ab1b9cfa2f736
The initial section describes this as being in serialized form...
Which should do this to the private key...
privkey[0] &= 248;
privkey[X25519_KEYLEN - 1] &= 127;
privkey[X25519_KEYLEN - 1] |= 64;
    When it does this the first byte is then 0x50 and the last byte is 0x76>>
    Am I interpreting this incorrectly?

Yeah I think the above is correct (as per https://www.rfc-editor.org/rfc/rfc7748#section-5) , but not sure your new code needs to do that - for test purposes I just compare the result of using the IKM to the generated public key, which should be good enough I think.

I will do the masking in the test code.. I really think the test vector should be updated to be correct.

Do you happen to have any X448 KEM test vectors?

IIRC there are some in the test-vectors.json file at https://github.com/cfrg/draft-irtf-cfrg-hpke/

On the good news front, when I do provide ikm, and whack in a call to your kem stuff into my shim code, my pre-existing decryption code works fine (for base mode, x25519 anyway). Seems like a good sign:-) https://github.com/sftcd/happykey/blob/ecxdhkem/hpke.c#L2160

Just to ensure we're on the same page - after I make a call to your kem API when encrypting, My HPKE code has to do a bunch of extracts and expands using the shared secret output from your kem API before I call the aead function. One example label input to the first of those extracts is "psk_id_hash", so that string shouldn't occur in your code but just in mine. I'm just checking that I'm clear on where we want the boundary to be before we start thinking about other curves modes etc.
Put another way, the code from here to the end of that function wouldn't be changed due to this PR. (It'll change based on other comments and I'll be doing the 9180 based API stuff, but ignore that for the moment.)
Sound about right?

Yes I am only doing the KEM part..(Encap / Decap)
The key schedule is a separate operation that belongs in  HPKE..
I would suggest putting the schedule into a separate internal function.

The keygen related changes are just to create the recipient (and sender auth) keys from ikm values..

I am getting close with the EC code now.. Just need to test and write a bit of code for the auth mode.. I am hoping to get close to submitting by the end of my day today..

Grand so. I can play more with it tomorrow (here) and/or Wed. Just send me a ping here or via mail.

@sftcd - just getting the errors out of the CI loop now..
The interface is very similiar to the ecx case.

Just checked that latest works for all groups in base mode and it does, which is neat. Will check with other HPKE modes tomorrow.
Looks like a non-NULL ikm still has to be supplied in this version, and (perhaps worse?) the ikm length has to match the private key length for the group. That's a level of clue it's a bit hard to expect from a caller IMO. I'd argue both restrictions ought go, noting that length of input != entropy of input if someone says you need that many random bits of ikm to be supplied by the caller:-)

Generation of random IKM in EC has been added.. Documentation is also updated.

Generation of random IKME inside the EC KEM for the sender ephemeral key has been added.. Documentation is also updated.

... and confirmed to work, thanks. (Where "work" so far is still just that code I whacked into my test shim to use this when encrypting and older code when decrypting, but it'll be same code.)

Fairly comprehensive Coverage testing added..

ping
This is now ready for review.

ping
can someone review this please..

On 06/09/2022 10:31, Shane wrote:

ping
can someone review this please..

I'll be looking at it again next day or two if that helps
(though I guess you're looking from reviews from committers
so it probably doesn't:-)
S.

(Starting to try this out more with the new HPKE API stuff now, and thanks for your comments on that, I'll respond to those in a bit...)
Am I right in reading this as follows: to use the HPKE AUTH modes one needs to make calls to e.g. EVP_PKEY_auth_encapsulate_init() whereas for the base modes one calls EVP_PKEY_encapsulate_init()?
If so, won't that lead to lots more code that is effectively only called by the test harness? (There being no current uses of HPKE AUTH modes afaik.)
That'd also mean similar increases in lines of code in the HPKE PR I think, compared to APIs that allow setting the AUTH private or public when needed as an attribute of the context fed into one EVP_PKEY_encapsulate_init() call.
I'm not seeing the benefit of that approach, what am I missing? Or maybe I'm reading it wrong?

I cant change the existing API.. It is now fixed in stone.. Hence a new API.

if (auth)
auth_encapsulate_initi()
else
encapsulate_init()
Is not a lot of code :)

Sorry, not following. What's EVP_PKEY_encapsulate_init() currently used for? Somewhere in there you must be calling the new hpke_expand() I guess, so not clear what calls for EVP_PKEY_auth_encapsulate_init(). Maybe I need to read it more:-)
(If it were just one call to _init() that'd be easy, sure but there's a bunch of 'em needed)

ah sorry, reading more it's less bad than I figured - so the same EVP_PKEY_CTX var can be used in both cases? If so, that;'s ok

ah sorry, reading more it's less bad than I figured - so the same EVP_PKEY_CTX var can be used in both cases? If so, that;'s ok

The pattern is this
EVP_PKEY_encapsulate_init(key)
//Potentially do other things such as setparams()
EVP_PKEY_encapsulate(enc, secret)
This init(), set(), dooperation() is a common pattern for most operations in openssl.

Yes, to create a sender you create a recipient key and use that for the ctx.. that is then used by either init().
My tests just then go  if (data->authkey) do the auth_encapsulate_init() else do the encapsulate_init() followed by the encapsulate() call that is used by both.. My thinking is that maybe the auth mode does something completely different for some new kem that comes along at some point...

The one remaining thing I havent done (which can be done later) is limit the ikm length. Currently it will just fail in the extract or expand functions since they use fixed size buffers. For interop reasons I think you should be applying the 64 byte limits to the other HPKE fields that are listed in the RFC.. The code may need to do 2 passes when using the wpacket code, one to calculate the size (and if the size is too large for the fixed buffer - then use a malloced buffer instead).

making progress with the AUTH mode stuff now, not sure I'll reach success tonight (late here), but direction == good:-)

making progress with the AUTH mode stuff now, not sure I'll reach success tonight (late here), but direction == good:-)

Excellent. You have some ref code in my old PR that worked with auth 😉. The DHKEM stuff should be good as it has been fairly well tested.

rebased to fix test/build.info conflict

I've confirmed this all works for my (shim) HPKE build so as soon as this is merged I can simply my HPKE PR #17172 significantly and easily ensure this works for ECH. IOW, please merge soon:-)

We should probably discuss the proposed API changes at OTC (or did we already do so when we discussed HPKE previously??).

I redid quite a few things since the initial discussion, so I am happy to discuss again..

On 12/09/2022 22:32, Shane wrote:

We should probably discuss the proposed API changes at OTC (or did
we already do so when we discussed HPKE previously??).

I redid quite a few things since the initial discussion, so I am
happy to discuss again..

I don't think I'm needed for that discussion but would be
happy to take part if that helped.
I can say Shane's current API works fine for my HPKE PR, for
all combinations of HPKE mode, and suite (kem/kdf/aead) when
round-tripping. It also works fine for all the test vectors
I've tried so far. It was also easy enough to figure out how
to use it from the test code, which seems pretty thorough.
It'll also help me a lot to see this merged; before then it's
a bit tricky to test the combination of Shane's KEM stuff,
and my HPKE stuff and ECH stuff all together and it's that
combo that's the important interop target for the near term,
i.e. OpenSSL-enabled clients working with CF's servers and
browsers working with servers using OpenSSL. If it were going
to be a while before this were merged (which'd be a pity), it
would be good to know that so I can figure a way to do those
interop tests on an ongoing basis.
Lastly, for now, I also plan to maintain a version of my HPKE
shim code that works against a released library. That should
be potentially useful until there's a library release that
has HPKE, which I guess will be some time (but if I'm wrong
there, be good to know that).
Cheers,
S.

On 12/09/2022 22:41, Shane wrote:

Yes I have had a discussion with the authors and raised an errata. I
assume other implementations must just be testing the public key
(since it is derived from the private).
One can argue that that's not wrong. IIRC implementations
presented with random octet strings are supposed to do the
masking anyway so from that POV it doesn't matter that the
output from the deterministic key gen isn't masked and is
used for test vectors, but on the whole it'd probably have
been better if the key gen had said to do the masking and
the test vectors had been masked. But, it could be the
right outcome is to just document that the test vectors in
the RFC aren't masked.

Cheers,
S.

Addressed review comments..

rebased with no changes..

removed extra #defines for auth encapsulate/decapsulate ops
@mattcaswell I think the only outstanding thing now is the question related to the public from private function..

rebased to fix merge conflicts.
Changes: Remove extra spaces at end of lines in pod files
Collapsed all commits into 1 commit + the merge conflict commit.
Changed the commit message, to a summary of all commits.

ping - Can someone review this please

Buitblot failures are not related to this PR.

The commit messages do not look correct. Could you please amend them so the commit title is meaningful, etc.

Commit message updated..

Issues have been addressed..

ping - Can I get a second reviewer please..

NITS addressed

@mattcaswell Still OK?

@slontis This needs rebasing now. So close...

Merging.

Merged to master. Thank you.

Thanks for reviewing!
@sftcd time to rebase :)

On 23/09/2022 09:51, Shane wrote:

Thanks for reviewing!
@sftcd time to rebase :)

Will do. Good to see progress progressing.
S.


[Event Request] Codeunit 5804 Item Cost Management - OnAfterUpdateUnitCost

Hi Team,
In codeunit 5804 we need possibility to replace and avoid using standard code. Having event in the end of procedure UpdateUnitCost, would be a perfect solution :

[IntegrationEvent(false, false)]
local procedure OnAfterUpdateUnitCost(var Item: Record Item; CalledByFieldNo: Integer)

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Update parameters in metadata (from ibmq-provider)

Summary
Copying issue from https://github.com/Qiskit/qiskit-ibmq-provider/pull/1065
Fixes #273
Details and comments

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 15 (13.33%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.08%) to 37.357%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit_ibm_provider/utils/json_encoder.py
2
15
13.33%

Totals

Change from base Build<PHONE_NUMBER>:
-0.08%

Covered Lines:
1758

Relevant Lines:
4706

💛  - Coveralls


Improve floating point to text conversion

Floating point conversion had a limited range that was supported. This improves the range but requires space in the buffer for the trailing null.

I think there is a better way to do this, so I'm closing it and will rework it.


Snow depth latitude flipped for select dates

It seems that for a select few dates the snow_depth variable is flipped (e.g., the latitudes are reversed) from this dataset gs://gcp-public-data-arco-era5/ar/full_37-1h-0p25deg-chunk-1.zarr-v3/. I only checked values in 6 hour chunks from 1979 to 2021, but I found the fields are flipped for these dates:
Flipped data at 1981-03-16T00:00:00.000000000
Flipped data at 1981-03-16T06:00:00.000000000
Flipped data at 1981-03-16T12:00:00.000000000
Flipped data at 1981-03-16T18:00:00.000000000
Flipped data at 1982-04-06T00:00:00.000000000
Flipped data at 1982-04-06T06:00:00.000000000
Flipped data at 1982-04-06T12:00:00.000000000
Flipped data at 1982-04-06T18:00:00.000000000
Flipped data at 1985-12-11T00:00:00.000000000
Flipped data at 1985-12-11T06:00:00.000000000
Flipped data at 1985-12-11T12:00:00.000000000
Flipped data at 1985-12-11T18:00:00.000000000
Flipped data at 1987-11-30T00:00:00.000000000
Flipped data at 1987-11-30T06:00:00.000000000
Flipped data at 1987-11-30T12:00:00.000000000
Flipped data at 1987-11-30T18:00:00.000000000
Flipped data at 1990-03-05T00:00:00.000000000
Flipped data at 1990-03-05T06:00:00.000000000
Flipped data at 1990-03-05T12:00:00.000000000
Flipped data at 1990-03-05T18:00:00.000000000
Flipped data at 1990-04-02T00:00:00.000000000
Flipped data at 1990-04-02T06:00:00.000000000
Flipped data at 1990-04-02T12:00:00.000000000
Flipped data at 1990-04-02T18:00:00.000000000
Flipped data at 1990-08-12T00:00:00.000000000
Flipped data at 1990-08-12T06:00:00.000000000
Flipped data at 1990-08-12T12:00:00.000000000
Flipped data at 1990-08-12T18:00:00.000000000
Flipped data at 1997-05-15T00:00:00.000000000
Flipped data at 1997-05-15T06:00:00.000000000
Flipped data at 1997-05-15T12:00:00.000000000
Flipped data at 1997-05-15T18:00:00.000000000
Flipped data at 2002-03-17T00:00:00.000000000
Flipped data at 2002-03-17T06:00:00.000000000
Flipped data at 2002-03-17T12:00:00.000000000
Flipped data at 2002-03-17T18:00:00.000000000
Flipped data at 2003-11-26T00:00:00.000000000
Flipped data at 2003-11-26T06:00:00.000000000
Flipped data at 2003-11-26T12:00:00.000000000
Flipped data at 2003-11-26T18:00:00.000000000
Flipped data at 2004-02-10T00:00:00.000000000
Flipped data at 2004-02-10T06:00:00.000000000
Flipped data at 2004-02-10T12:00:00.000000000
Flipped data at 2004-02-10T18:00:00.000000000
Flipped data at 2006-04-12T00:00:00.000000000
Flipped data at 2006-04-12T06:00:00.000000000
Flipped data at 2006-04-12T12:00:00.000000000
Flipped data at 2006-04-12T18:00:00.000000000
Flipped data at 2007-06-19T00:00:00.000000000
Flipped data at 2007-06-19T06:00:00.000000000
Flipped data at 2007-06-19T12:00:00.000000000
Flipped data at 2007-06-19T18:00:00.000000000
Flipped data at 2009-03-05T00:00:00.000000000
Flipped data at 2009-03-05T06:00:00.000000000
Flipped data at 2009-03-05T12:00:00.000000000
Flipped data at 2009-03-05T18:00:00.000000000
Flipped data at 2013-11-11T00:00:00.000000000
Flipped data at 2013-11-11T06:00:00.000000000
Flipped data at 2013-11-11T12:00:00.000000000
Flipped data at 2013-11-11T18:00:00.000000000
Flipped data at 2014-05-11T00:00:00.000000000
Flipped data at 2014-05-11T06:00:00.000000000
Flipped data at 2014-05-11T12:00:00.000000000
Flipped data at 2014-05-11T18:00:00.000000000
Flipped data at 2017-03-17T00:00:00.000000000
Flipped data at 2017-03-17T06:00:00.000000000
Flipped data at 2017-03-17T12:00:00.000000000
Flipped data at 2017-03-17T18:00:00.000000000
Flipped data at 2020-05-19T00:00:00.000000000
Flipped data at 2020-05-19T06:00:00.000000000
Flipped data at 2020-05-19T12:00:00.000000000
Flipped data at 2020-05-19T18:00:00.000000000

Hi thank you for your quick response! After further investigating it I realized the problem is more subtle than realized. The latitudes themselves are not flipped but the data itself is flipped. Here is the minimum reproducible code to this bug. The code below produces a matplotlib imshow image for the two dates in your example code (I also attached those images below). During these "flipped" dates I mentioned in the previous comment the outline of Antarctica and Greenland are in the incorrect locations (e.g., flipped over the equator).
import xarray as xr
import matplotlib.pyplot as plt

ar_full_37_1h = xr.open_zarr('gs://gcp-public-data-arco-era5/ar/full_37-1h-0p25deg-chunk-1.zarr-v3/')[['snow_depth']]

correct_orientation = ar_full_37_1h.sel(time='1980-03-16T00:00:00.000000000')['snow_depth'].values
incorrect_orientation = ar_full_37_1h.sel(time='1981-03-16T00:00:00.000000000')['snow_depth'].values

plt.imshow(correct_orientation)
plt.title('Correctly Orientated Data Example')
plt.show()

plt.imshow(incorrect_orientation)
plt.title('Flipped Data Example')
plt.show()

Hello @Arcomano1234 If possible please share the script with us which you used to generate the flipped data or run your script on all of the year(1940-2023) so we can fix all the flipped data of the dataset.

I admit this is not the best method for detecting the flipped data but this is the quickest and most simple. I assumed that the top latitude band (e.g., the North Pole) had no snow depth after inspecting the data. So the script checks any time there is snow depth where there shouldn't be. This script was able to detect all of the problem data from 1979 - 2023, however, I can not vary if it works for older data.
import numpy as np
import xarray as xr

years = np.arange(1979,2022)
for year in years:
    ds = xr.open_dataset(f'{year}.nc')
    times = ds.time
    for i in range(len(times)):
        var = ds['snow_depth'][i,0,:].values

        if np.mean(var) > 0.1:
            print('Flipped data at',times[i].values)

https://github.com/google-research/arco-era5/issues/71 is a duplicate of this issue.
It appears that every variable has this issue on these dates.

Thank you for the update. I just checked and yes all of the variables I have used from ARCO-ERA5 dataset are flipped at the dates mentioned in my original post.

@Arcomano1234 @shoyer we were able to identify the root cause of the issue. The discrepancy arises from the fact that certain variables have a resolution of 0.5 * 0.5 degrees, whereas all the other variables have a resolution of 0.25 * 0.25 degrees. Consequently, during the creation of the dataset, the latitude value is reversed, resulting in reversed data for that particular date.
The following variables have a spatial resolution of 0.5 * 0.5 degrees for the specified date.
'1965-11-22' -> wave_spectral_kurtosis
'1981-03-16' -> mean_wave_period_based_on_first_moment_for_swell
'1982-04-06' -> benjamin_feir_index
'1985-12-11' -> benjamin_feir_index
'1987-11-30' -> mean_wave_period_of_second_swell_partition
'1990-03-05' -> mean_direction_of_wind_waves
'1990-04-02' -> period_corresponding_to_maximum_individual_wave_height
'1990-08-12' -> mean_period_of_total_swell
'1997-05-15' -> significant_wave_height_of_third_swell_partition
'2002-03-17' -> peak_wave_period
'2003-11-26' -> mean_direction_of_wind_waves
'2004-02-10' -> mean_wave_direction_of_first_swell_partition
'2006-04-12' -> mean_wave_period_based_on_first_moment_for_swell
'2007-06-19' -> v_component_stokes_drift
'2009-03-05' -> wave_spectral_directional_width_for_wind_waves
'2013-11-11' -> mean_wave_period_of_third_swell_partition
'2014-05-11' -> significant_wave_height_of_third_swell_partition
'2017-03-17' -> total_precipitation
'2020-05-19' -> benjamin_feir_index

P.S.: I already updated the .nc file for the above date for the above variables so you can't see the old files now. 😄
Here is an example code snippet for creating the datasets:
def fun():
  year, month, day = 1981, 3, 16
  root_path = pathlib.Path("gs://gcp-public-data-arco-era5/raw") # GCP path
  output = {}
  for variable in ['mean_wave_period_based_on_first_moment_for_swell', "mean_wave_period_based_on_first_moment"]:
      relative_path = SINGLE_LEVEL_SUBDIR_TEMPLATE.format(year=year, month=month, day=day, variable=variable)
      output[variable] = _read_nc_dataset(root_path / relative_path)
      print(output[variable])
  print("-------------------------")
  final_answer = xr.Dataset(output)
  print("final answer : ", final_answer)

P.P.S.: It should be noted that the .nc files have already been updated, so using the above code will yield accurate results (data is not flipped).
Furthermore, we will update the data of the zarr file ('gs://gcp-public-data-arco-era5/ar/full_37-1h-0p25deg-chunk-1.zarr-v3/') within the next 2-3 days.

@Arcomano1234 @shoyer I updated data of the zarr file(gs://gcp-public-data-arco-era5/ar/full_37-1h-0p25deg-chunk-1.zarr-v3/). After a thorough examination, I have concluded that there are no flipped data points; therefore, I am closing this issue.
Please feel free to reopen the issue if you find any flipped data in the future.

Thanks @dabhicusp, really appreciated!


Status codes 404/500?

Hi all -
I've been getting status codes as below -- 404 and 500.  Followed the setup precisely and cannot get past this.
As Wix no longer supports the 'crypto' library I tried moving it to crypto-js but it looks to be more than just a few lines of code and so that did not work either.
/_functions-dev/batchCheckUpdateState
Request failed with status code 404
Any ideas?  Thanks!

i get the same error too...


Validate clusterserviceversion

Signed-off-by: Chun Lin Yang<EMAIL_ADDRESS>Fixed https://github.com/operator-framework/operator-lifecycle-manager/issues/1155

Thanks for submitting this PR @clyang82!
#1157 was created to address this issue.

@awgreene That is great. Good to hear that this issue will be addressed. Thanks. I am going to close this PR once your fix was merged.

@awgreene That is great. Good to hear that this issue will be addressed. Thanks. I am going to close this PR once your fix was merged.

@clyang82 Great, I will work on getting the PR through quickly.

@awgreene That is great. Good to hear that this issue will be addressed. Thanks. I am going to close this PR once your fix was merged.

@clyang82 Great, I will work on getting the PR through quickly.

Thanks @awgreene

#1157 was merged. so close this one...

@awgreene the original issue(#1155) I reported was not fixed, because the deploy/upstream/quickstart/crds.yaml was not updated.
@gallettilance Do we need to update release dir by using controller-gen? Thanks.

check the code again. the crds.yaml and olm.yaml will be generated when a new release is made.


Errors during publishing / installing an extension in VS Code: "A task was canceled"

In some cases, although the compiler in VS Code shows no error, an extension can not be published or installed.
Currently there is only a generic error message "A Task was canceled".
When you take a look in the event log, or use the publish / install powershell methods, a more precise error is shown.
E.g license issues, or callstacks...
It would be nice if you could show these errors in VS Code as well.

We believe that this should have been fixed on the latest version of AL Language (0.15) and recent VS Code versions (1.21+). Please comment on this issue if you are still running into a problem
You will still not see call stacks in VS Code but errors about duplicate IDs, duplicate extension, license problems etc. will all be shown in VS Code.

I am receiving the same error message but there are no events in the application event log related to it AL.
I start a my extension in VS Code (F5 or Ctrl+F5, doesn't matter) and after an extended period of time (several minutes), the Error: A task was canceled. message appears.
The default AL Project (Hello World) works so that implies that it is a problem my extension but I have no where to start looking without some feedback from the system.
This is on premise using AL version 0.12.17720 (NAV 2018 CU04 build 21441).
Thoughts?

For NAV 2018 issues please contact our Support team.

So how do I do this without using up my support incidents or having to pay for one?


feat: add stats to convert-to-delta operation

Description
Collect stats during conversion of a parquet dir to a Delta table and add to the actions.
Related Issue(s)
Closes #2490
Documentation

I'm not sure why the parse decimal overflow appears in the Python tests.
It's indicative that I see the decimal column min/max ends up as Value(Number(2560.0))/Value(Number(3584.0)) even though the pyarrrow seems to be properly persisting the stats to the file
% parquet meta python/tests/part-0.parquet

File path:  python/tests/part-0.parquet
Created by: parquet-cpp-arrow version 15.0.2
Properties:
  ARROW:schema: /////5ADAAAQAAAAAAAKAAwABgAFAAgACgAAAAABBAAMAAAACAAIAAAABAAIAAAABAAAAA4AAAAwAwAA5AIAALACAAB8AgAASAIAABACAADgAQAAtAEAAIgBAABMAQAAHAEAAOgAAABkAAAABAAAABj9//8AAAEMFAAAABwAAAAEAAAAAQAAABQAAAAEAAAAbGlzdAAAAAAM/f//RP3//wAAAQIQAAAAGAAAAAQAAAAAAAAABAAAAGl0ZW0AAAAAfP3//wAAAAFAAAAAdP3//wAAAQ0YAAAAIAAAAAQAAAACAAAAPAAAABQAAAAGAAAAc3RydWN0AABs/f//pP3//wAAAQUQAAAAFAAAAAQAAAAAAAAAAQAAAHkAAACQ/f//yP3//wAAAQIQAAAAFAAAAAQAAAAAAAAAAQAAAHgAAAD8/f//AAAAAUAAAAD0/f//AAABChAAAAAcAAAABAAAAAAAAAAJAAAAdGltZXN0YW1wAAAA8v7//wAAAgAk/v//AAABCBAAAAAYAAAABAAAAAAAAAAGAAAAZGF0ZTMyAAAe////AAAAAFD+//8AAAEHEAAAACAAAAAEAAAAAAAAAAcAAABkZWNpbWFsAAgADAAEAAgACAAAAAUAAAADAAAAiP7//wAAAQQQAAAAGAAAAAQAAAAAAAAABgAAAGJpbmFyeQAAeP7//7D+//8AAAEGEAAAABgAAAAEAAAAAAAAAAQAAABib29sAAAAAKD+///Y/v//AAABAxAAAAAYAAAABAAAAAAAAAAHAAAAZmxvYXQ2NADS////AAACAAT///8AAAEDEAAAACAAAAAEAAAAAAAAAAcAAABmbG9hdDMyAAAABgAIAAYABgAAAAAAAQA4////AAABAhAAAAAYAAAABAAAAAAAAAAEAAAAaW50OAAAAABw////AAAAAQgAAABo////AAABAhAAAAAYAAAABAAAAAAAAAAFAAAAaW50MTYAAACg////AAAAARAAAACY////AAABAhAAAAAYAAAABAAAAAAAAAAFAAAAaW50MzIAAADQ////AAAAASAAAADI////AAABAhAAAAAgAAAABAAAAAAAAAAFAAAAaW50NjQAAAAIAAwACAAHAAgAAAAAAAABQAAAABAAFAAIAAYABwAMAAAAEAAQAAAAAAABBRAAAAAcAAAABAAAAAAAAAAEAAAAdXRmOAAAAAAEAAQABAAAAAAAAAA=
Schema:
message schema {
  optional binary utf8 (STRING);
...
  optional fixed_len_byte_array(3) decimal (DECIMAL(5,3));
...

Row group 0:  count: 5  288.20 B records  start: 4  total(compressed): 1.407 kB total(uncompressed):1.408 kB
--------------------------------------------------------------------------------
                   type      encodings count     avg size   nulls   min / max
utf8               BINARY    S _ R     5         16.00 B    0       "0" / "4"
...
decimal            FIXED[3] S _ R     5         17.00 B  0       "10.000" / "14.000"
...

Hi @roeap, @ion-elgreco, I think this is ready for review. There's both a feat added (convert stats) as well as a bugfix (decimal stats parsing from byte arrays) now.
Can you provide some feedback on it? Thanks!

Thankss

@gruuya can you rebase so that we can merge

@ion-elgreco weirdly a couple of tests seem to be stuck, and have been running close to 2h hours now (test_concurrency_local, test_integration_local). I presume this is a CI problem of some sort (the previous runs were ok), can you restart the job please?

@gruuya  yeah it's a flaky test, re-executed it


Use hidraw instead of libusb on linux

This makes bluetooth, and thus joycons, show up!

After making this change, I noticed that the joycon image in slimevr-wrangler does not render correctly on linux (renders a black box)