Issue with network topology involving VPN and 2 sites

Hello,
I am having issues with WebRTC protocol in my particular network topology involving 2 sites connected via VPN.
My setup:
Site1
Docker server running Frigate 0.12 rc2
2023-04-04 12:11:33.629540773  [INFO] Preparing go2rtc config...
2023-04-04 12:11:33.719847311  [INFO] Not injecting WebRTC candidates into go2rtc config as it has been set manually
2023-04-04 12:11:33.733325568  [INFO] Starting go2rtc...
2023-04-04 12:11:33.801569519  12:11:33.800 INF go2rtc version 1.2.0 linux/amd64
2023-04-04 12:11:33.801572791  12:11:33.801 INF [api] listen addr=:1984
2023-04-04 12:11:33.801576488  12:11:33.801 INF [rtsp] listen addr=:8554
2023-04-04 12:11:33.801578018  12:11:33.801 INF [srtp] listen addr=:8443
2023-04-04 12:11:33.801579341  12:11:33.801 INF [webrtc] listen addr=:8555
2023-04-04 12:11:43.630126325  [INFO] Starting go2rtc healthcheck service...

Site1 is connected to Site2 via Wireguard VPN.
On Site2, unable to start a webrtc session in Safari, iPhone.
tcpdump shows the data flowing freely, in both directions, and non-stop on UDP port 8555 between the Frigate server in Site1 and the device in Site2. However, the video never loads which is quite puzzling.
There is no NAT to deal with, all IPs are reachable directly between Site1 and Site2.
I have the following setup in Frigate:
go2rtc:
  streams:
    Cam1:
      - rtsp://user:pass@<IP_ADDRESS>:554/Streaming/Channels/101
    Cam2:
      - rtsp://user:pass@<IP_ADDRESS>:554/Streaming/Channels/101
  webrtc:
    candidates:
      - <IP_ADDRESS>:8555  # docker image internal IP
      - <IP_ADDRESS>:8555  # docker host

Would it be possible to tell go2rtc to NOT try to use any STUN or TURN server, since the connection should work internally. Maybe adding "local" subnets in the go2rtc configuration?
Even without Frigate in the equation, for example using the "stream.html WebRTC stream" link from the go2rtc page, same result.
Thank you!

I'm having the same issue. Using Frigate. Site to Site VPN using Wireguard on the routers. In Wireshark I see a constant stream of UDP traffic coming from the server on port 8555. But the video player just remains at loading.

What viewers (html) have your tried? What about Frigate 12 release?

I'm using Frigate 0.12.0-DA3E197. Trying to access the stream from the /live/webrtc/links.html?src=backyard page. I've tried in both Chrome and Firefox if that is what you mean for viewers. stream.html?src=backyard&mode=mse works while at the second site. But mode=webrtc only seems to work when my laptop is on the same network as the server.

Hey,
I have this problem too.
All my networking is internal (ie, the viewers and go2rtc are on different sites connected by wireguard at their respective routers). The moment I fire a stream (I'm using the webrtc-camera card), I see dozens of public ip-to-public ip UDP requests between the two routers (from the router behind which go2rtc is to the router behind which the viewer is), and this triggers the destination router's port scan alarm and ultimately blocks the originating router's ip.
Now, I can either a) whitelist the originating router's IP (which is a PITA, because it's dynamic and has I'd have to script out something) or b) disable port scanning altogether, but since both go2rtc and the viewer's host are directly reachable (through their private IPs) and I have no intention to make the cameras available on the internet, I am looking for a way to, somehow, instruct go2rtc to just use private IPs and therefore avoid the whole STUN situation altogether - because it's ultimately not necessary.
I've added
webrtc:
  listen: ":8555"
  candidates:
   - <IP_ADDRESS>:8555

to the go2rtc.yml config file, but that didn't alter this behaviour. I fully admit I don't understand these config params deeply (or the protocols in question, for that matter) but I was wondering if there's a way to keep everything flowing through the private IPs.
Thanks

Basic docker users (also Frigate docker users) shouldn't have any problems with LAN connections. And shouldn't add any local IP to go2rtc config. They should use docker network host option. And they will be fine.
Frigate addon users should proxy WebRTC port and set local candidate IP manually. Because they can't use network host.

@gedl do you have problems with local or with external connections?

Hi @AlexxIT
I don't expose the cameras externally at all - there is no domain name (or route, for that matter) into the go2rtc box. The STUN kicks in for internal to internal network requests, like so:
<IP_ADDRESS> (browser) -> <IP_ADDRESS> / <IP_ADDRESS> (router 1 Lan/Wg)  ---> internet ---> <IP_ADDRESS> / <IP_ADDRESS> (router 2 Lan/wg) -> <IP_ADDRESS> (proxmox) -> <IP_ADDRESS> (go2rtc).
Right when I open the webrtc card in homeassistant, the go2rtc fires a lot of UDP packets to router 2's public IP, instead of simply using the readily available private network path.
The network hosts on either side of the internet have open-bar access to each others, on all ports.

WebRTC (in the browser and in the go2rtc) will try all possible ways to connect. Because they don't know which is the best route.

I understand that. I guess what @J-Prince and I are asking is if there's a way to instruct either party (but particularly go2rtc) to use a specific route, or to disable STUN altogether and skip straight to the next alternative (until eventually settles on using the internal, direct route between the hosts).

You can disable STUN in the go2rtc yaml

Maybe I'm after a red herring with the STUN remark, but with
webrtc:
  ice_servers: []

I still see a blast of UDP connections from one router's public IP (the one behind which go2rtc runs) to the other router's public IP (behind which the browser is). These two hosts are directly reachable through private IPs, and my goal was to make them do so rather than trying to punch UDP holes on the each other's firewalls.
This is what tcpdump shows when the page loads
23:09:13.280426 IP 18x.58.xxx.xx3.15543 > 2xx.187.xxx.xx0.53805: UDP, length 100
23:09:13.280813 IP 18x.58.xxx.xx3.15543 > 2xx.187.xxx.xx0.61214: UDP, length 100
23:09:13.281087 IP 18x.58.xxx.xx3.15543 > 2xx.187.xxx.xx0.53805: UDP, length 100
23:09:13.281302 IP 18x.58.xxx.xx3.15543 > 2xx.187.xxx.xx0.61214: UDP, length 100
23:09:13.291919 IP 18x.58.xxx.xx3.15543 > 2xx.187.xxx.xx0.53805: UDP, length 100
23:09:13.292146 IP 18x.58.xxx.xx3.15543 > 2xx.187.xxx.xx0.61214: UDP, length 100

Browser still has ice settings. You can be changed only with JS code. No any settings for that.
But it shouldn't be a problem.

But the IP blasting the requests is go2rtc's router, not the browser's.
I agree there shouldn't be any problem in the general case. In my particular case this is identified as a port scan (about 50 connections within 2 seconds).
Totally understand if this is a niche situation that doesn't warrant a generally available setting to avoid, but since it is go2rtc's side blasting the browser's side, I was hoping there would be a setting there for it just not to.

You can enable trace log level for webrtc module and check all local and remote candidates there

I have the same problem. Has anyone found a solution? @gedl @J-Prince @sovapatr
The two subnets are directly connected with wireguard, freely passable, yet webrtc cannot pass through.
I created an EOIP tunnel, the client got the ip address directly from the server subnet, but still it was not able to establish the connection. I haven't got around to tracing my connections yet.

I have the same problem. Has anyone found a solution? @gedl @J-Prince @sovapatr The two subnets are directly connected with wireguard, freely passable, yet webrtc cannot pass through.

Unfortunately no. I have to resort to use jsmpeg for Frigate live view, which less than ideal.

I started testing the problem from many directions and came to the following conclusions:

Even though I created an EOIP tunnel between the two subnets, and my laptop received an IP directly from my home network, the webrtc still did not work.
I monitored the server side, client side, and the two routers: the go2rtc, after failing to reach the internet, finds its way to the internal network, but sends UDP packets in vain, no video connection is established.

The most interesting observation:

My home network is sent out to the garage with wifi, two antennas, and then a wifi AP is in place. It's exactly the same network as the one inside, just bridge-connected. Despite this, WEBRTC DOES NOT WORK in the garage either!!!

Could there be any connection between WEBRTC and the jump points? Because we can clearly state that we are not talking about an IP fault here.
@AlexxIT  Please help us to investigate the error so we can find a solution.
Any way I can help you to solve it.

I think it's hard task for really hi level network engineers to build solid local routing between multiple local subnets.
There is no magic here. Two clients are exchanging IP addresses. And both are trying to reach each other. They can or can't do it. Based on your internal routing.

@gedl Did you find a solution to your problem?


Dependabut doesn't ignore major version when using nuget ( Azure Devops)

Package ecosystem
Nuget
Package manager version
Latest on Azure Devops
Language version
c#
Manifest location and content prior to update
Updated dependency
Bump Microsoft.EntityFrameworkCore from 5.0.0 to 6.0.1
What you expected to see, versus what you actually saw
I expected to see the latest nuget of Microsoft.EntityFrameworkCore with 5.x.x. Which is 5.0.13
Link to available package versions
Images of the diff or a link to the PR, issue or logs

🕹 Bonus points: Smallest manifest that reproduces the issue
Test project on Azure Devops

Dependabot pipeline file
Dependabot config

Note: Configure PAT as a variable in Azure Devops ( Personal Access Token)

Hey, it looks like you're running dependabot-script, it doesn't support anything the the config file, if you want to pass in an ignored version, you'll have to pass it into the UpdateChecker manually as an IgnoreCondition object, the argument on the UpdateChecker is ignored_versions.
You'll have to modify the script in order to be able to do this.

UpdateChecker doesn't take IgnoreCondition object, it only takes ignored_versions object -- which means that there isn't a ready way that I found so far to skip major/minor versions. I saw config has this, but new to Ruby and trying to figure out how to get it to skip major/minor still. Any ideas?

Also tagging @NicoJuicy to see if you found some way to do something like this

@craigta This was an innovation project at my company. I currently decided to not continue this for Azure Devops until it get's easier.
Since Microsoft bought Github and Github bought dependabot, it doesn't seem an unrealistic predicition.


Not working ssl

Set to 1 to enable HTTPS, 0 to run only over HTTP
NGINX_ENABLE_HTTPS=1

Hi @developcreativo and sorry for the long delay.
I've just setup a new project with NGINX_ENABLE_HTTPS=1 and I can't reproduce your issue.
I got HTTPS working (with a dummy index.php):

which version of laradhoc are you using?
which files do you have under <project>.docker/images/nginx/ssl?

I think you should see

fikrea.test.crt
fikrea.test.key

In my example, I got

laradhoc.test.crt
laradhoc.test.key

which are then mounted in the nginx container:
❯ docker exec -it laradhoc_nginx /bin/bash
root@32ec4d01ee46:/var/www# cd /etc/ssl/certs/
root@32ec4d01ee46:/etc/ssl/certs# ls -la
total 20
drwxr-xr-x 2 root root 4096 Oct 14 14:15 .
drwxr-xr-x 1 root root 4096 Oct 14 14:15 ..
-rw-r--r-- 1 root root   86 Oct 14 14:13 .gitignore
-rw-r--r-- 1 root root    0 Oct 14 14:13 .gitkeep
-rw-r--r-- 1 root root 1818 Oct 14 14:15 laradhoc.test.crt
-rw-r--r-- 1 root root 3272 Oct 14 14:15 laradhoc.test.key
root@32ec4d01ee46:/etc/ssl/certs#

Did you also try to rebuild the containers?


[TS migration] Migrate MenuItem to Typescript

Details
This is quite a delicate migration - 50 props used in all of the places across the app - the MenuItem is used in all the screens in the RIght Hand Panel and some of the screens in the Global Nav screens. Please test those screens.

Fixed Issues

$ https://github.com/Expensify/App/issues/25144
PROPOSAL:
Tests

Verify that no errors appear in the JS console
Add a background color on the outer element of MenuItem for easier testing.
Go through all the possible screens and make any possible actions with the items in Right Hand Panel (Settings).
Do the same with the Global Nav screens - some of them (like Request Money -> Distance) also use MenuItem.

Offline tests

Not affected directly by Offline mode

QA Steps

Verify that no errors appear in the JS console
Go through all the possible screens and make any possible actions with the items in Right Hand Panel (Settings).
Do the same with the Global Nav screens - some of them (like Request Money -> Distance) also use MenuItem.

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes

[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)

[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native
https://github.com/Expensify/App/assets/85628901/db876892-7f08-4398-b113-7e6c69a85247

Android: mWeb Chrome
https://github.com/Expensify/App/assets/85628901/f8b62466-c253-4ee9-bdf0-5d51ba3a51c4

iOS: Native
https://github.com/Expensify/App/assets/85628901/f6da44f7-7a2d-4a59-81a8-2a3b44bd92e3

iOS: mWeb Safari
https://github.com/Expensify/App/assets/85628901/355d3a48-9bd0-45d6-af0e-f6691f731a9f

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/85628901/13af5da5-7259-46cd-8bde-5a80a0304a85

MacOS: Desktop
https://github.com/Expensify/App/assets/85628901/5f89dbc3-e106-4f01-909a-e244cd0d5203

@shubham1206agra please wait a little bit, I forgot to record videos. Unless this is not blocking you

@MaciejSWM Please ping here whenever you have completed the videos

@shubham1206agra  ready

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native
https://github.com/Expensify/App/assets/58412969/dc140cc7-5d01-4361-8724-bcb367a6e52d

iOS: mWeb Safari
https://github.com/Expensify/App/assets/58412969/88c6be3d-300e-447b-9c55-9f76ab47158d

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/58412969/6d534917-b9ac-430b-8452-2f1c7976df25

MacOS: Desktop
https://github.com/Expensify/App/assets/58412969/0e44fdfc-62d9-4569-bbe9-0388efb47ab8

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

Thank you @deetergp!

🚀 Deployed to staging by https://github.com/deetergp in version: 1.4.15-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@deetergp @blazejkustra @kubabutkiewicz @fabioh8010
I'm getting typescript errors of missing props while using MenuItemWithTopDescription. It seems to me that some props that were optional previously became required, like hoverAndPressStyle.

@pac-guerreiro Can you mark other props that also became required by mistake?

🚀 Deployed to production by https://github.com/jasperhuangg in version: 1.4.15-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

@shubham1206agra You can check the props that need to be optional here -> https://github.com/Expensify/App/pull/33242/files#diff-9db0c2baf85f38b9a238d03f1089a686039ab6c6b94ef412bda920293909b7a4
When the above PR is merged, these issues will be fixed!


Haskell bash completion

Motivation for this change
The optparse-applicative library provides a uniform method of adding completions to haskell programs. This introduces two helper functions in haskell.lib that capture that.
This PR also adds completion to a small number of Haskell commands that are built with this library.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@Infinisil, others, I have thought about trying all executables but was a bit wary of the side effects. A package might include both a command line tool and an executable that fails in the sandbox, or takes a very long time to complete.
I have already spent too much time debugging silent failures. These kinds of failures are probably rare, so I'd rather require an override to selectively disable this feature, rather than give a one-line cue in an already verbose build log.
To make an opt-out design work without too nasty surprise, it should involve at least

Conditioning on isExecutable and, depending on what works best, one of

Detecting optparse-applicative in the transitive closure of dependencies
Checking direct dependencies against a list of at least optparse-applicative and optparse-generic

Running the executables with --bash-completion-script and logging that we're doing so
Set up a five minute timeout to catch interactive commands without completion script, fail hard on timeout with explanatory message
Fail hard with explanatory error message on failure, including methods to resolve the issue

Disable completion script extraction entirely
Blacklist an executable

Log that we're done with completion script extraction

An intermediate possibility is to require opt-in but not require to list the command line executables. That should give an opportunity to test the logic above that may become the opt-out solution.

Set up a five minute timeout to catch interactive commands without completion script, fail hard on timeout with explanatory message

This is just for getting the output of --bash-completion-script though, wouldn't it make more sense to fail after like 1 second? Any program that takes longer than that to finish with such an argument probably doesn't support it. Also why fail hard? Wouldn't this make lots of packages fail because of that? I'd rather have all packages at least try to get bash completion, but ignore it if it's not possible, since bash completion is really secondary. And such a failure really doesn't have a lot of other remedies than just blacklisting that package, which will result in the same as just ignoring such failures.

@Infinisil I agree that completion is secondary, but my concern is about bitrot. If we're going to fail silently we should at least test some high profile haskell packages. Five minute timeout is for a builder with high load to recover. It's only 2.5 orders of magnitude more ;)
Do we have a place for (small) integration tests in nixpkgs nowadays? I think I can live with soft failure in this case if we have warnings in the log and some test cases to prevent bitrot.

I'd love to see some thought put into a more general notion of how packages interact with installation environments, but for now some kind of post install hook seems great to me

@roberth By bitrot you mean the completion scripts potentially failing for more packages over time without anybody noticing? Does that really happen? I can't think of anything that could go wrong with such a simple post build hook.

@Infinisil That's what I mean. I don't expect anything in particular to go wrong either. I just want to set things up so that when they do go wrong, they get noticed.

It's interesting to note that the generated completion files are totally generic, i.e. they contain no code that's specific to the executable in questions other than its name and full path:
$ cabal2nix --bash-completion-script=/full/path/to/cabal2nix
_cabal2nix()
{
    local CMDLINE
    local IFS=$'\n'
    CMDLINE=(--bash-completion-index $COMP_CWORD)

    for arg in ${COMP_WORDS[@]}; do
        CMDLINE=(${CMDLINE[@]} --bash-completion-word $arg)
    done

    COMPREPLY=( $(/full/path/to/cabal2nix "${CMDLINE[@]}") )
}

complete -o filenames -F _cabal2nix cabal2nix

So arguably we could generate those files without actually running the executable, which is a trait the cross-compilation people would surely appreciate. The downside is, of course, that we expose ourselves to potential incompatibilities that might occur across different versions of optparse-applicative.

@peti, interesting observation for cross compilation. I should add though that running $executable --bash-completion-script also serves as a check whether it is really an optparse program. For example in the dhall package, dhall-repl does not support completions. If you'd generate a generic completion script for each executable, bash would start the dhall repl as soon as you hit tab.
This is not a concern for the current state of the PR, but it is for the opt-out approach. I think opt-out is feasible and I want to give it a try soon.

I am not particularly fond of the opt-out approach because it seems like
a bad idea to run arbitrary executables as part of the build without
having any clue about what they are doing.

@peti It's all sandboxed anyways though, what bad could happen? If we can't find some serious problem/risk with the opt-out approach, this would be a huge usability improvement, especially since so many packages are using optparse-applicative.

I understand peti’s concerns, it will make debugging harder. It also has a taste of an inverse autotools, running checks that might generate output, or maybe not.

Although I think a well-implemented opt-out approach is valuable, it will take more time than I can realistically expect to spend on it short/mid-term. I will limit the scope of this PR to opt-in.

Ping?

We need a fish user to test this.
Cross compilation can be 'solved' by disabling it during cross compilation.

Don't wait for me if you want to have this in 18.09 :(

Am I too diligent? #49477

Closing in favor of pr #49714 and cross compilation issue #49748


fix(txpool): proper pool ordering, round 2

Description
Fix transaction pool ordering round 2.
For reference see geth's implementation.
Additionally, base fee types used in mempool were demoted to u64, because that's how it is represented in a header.

--> crates/transaction-pool/src/test_utils/mock.rs:573:29

Some missing things on the test-utils feature of txpool, LGTM otherwise.

replaced by #3955


upload artifact per build for easier review

Thanks for the PR! This is a nice idea. Do you know where the artifacts get uploaded?

You can find artifacts at the Summary of each Action. E.g.,

@gtfierro I just updated the name. You were right that artifacts are always zipped at the UI: https://github.com/actions/upload-artifact#zipped-artifact-downloads, though those files are not zipped in their system. I updated the filename. Please let me know if it's ready to be merged.

Looks great! I"ll merge it in

I'm not questioning whether this is a good idea; I agree with you this is what we want to do. My question is whether or not this is the correct syntax for accomplishing having a build on every pull request. I would think there would be some sort of glob pattern, rather than just leaving the trigger blank?

From: Jason B. Koh @.>
Sent: Thursday, April 7, 2022 11:05:38 PM
To: BrickSchema/Brick @.>
Cc: Gabriel Fierro @.>; Review requested @.>
Subject: [External] Re: [BrickSchema/Brick] upload artifact per build for easier review (PR #388)
@jbkoh commented on this pull request.

In .github/workflows/builds.ymlhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FBrickSchema%2FBrick%2Fpull%2F388%23discussion_r845739999&data=04|01|gtfierro%40mines.edu|bda54bba843144722ca408da191d6c35|997209e009b346239a4d76afa44a675c|0|0|637849911419915170|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=IJ27nLfUsrr0Fi1USmWVDKVgocbMLDXYJE885wipU9I%3D&reserved=0:

branches:

   - master

   - v1.*

Yeah, that is intended. Every PR deserves a build and automated validation. Are there reasons not to do so?
—
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FBrickSchema%2FBrick%2Fpull%2F388%23discussion_r845739999&data=04|01|gtfierro%40mines.edu|bda54bba843144722ca408da191d6c35|997209e009b346239a4d76afa44a675c|0|0|637849911419915170|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=IJ27nLfUsrr0Fi1USmWVDKVgocbMLDXYJE885wipU9I%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAANRCS3V6MDAHSFUGYXW23VD65CFANCNFSM5S2UD4CA&data=04|01|gtfierro%40mines.edu|bda54bba843144722ca408da191d6c35|997209e009b346239a4d76afa44a675c|0|0|637849911419915170|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=yAnynGNl3P1D7xX0tZrxMi%2F8%2FuzIkRr5L%2FDz%2Bm8HJG8%3D&reserved=0.
You are receiving this because your review was requested.Message ID: @.***>

I believe the syntax is correct, but I can take a look later; I'm traveling now. Have you seen unexpected behaviors?


fix: prevent all referenced properties from updating to latest regardless of recipe

What's changed?

Fixing a bug in the Properties visitor for the rewrite-gradle plugin which failed to check whether the property currently being visited is referenced by the GroupArtifact that the recipe refers to
Anyone you would like to review specifically?

@timtebeek and @shanman190  have the context for this
Checklist

[ ] I've added unit tests to cover both positive and negative cases
[ ] I've read and applied the recipe conventions and best practices
[ ] I've used the IntelliJ IDEA auto-formatter on affected files

Thanks, @ajohnsonz!


Using obsidian-git as a git front-end / editor in Obsidian

Is there a plan to use the Obsidian Git source view as a Git front-end? I often have to manage docs for various projects. It's easier for me to keep them inside my vault, and I would love to push those changes from Obsidian as well. At the moment, the source is only available for the Obsidian repo but it looks like all the code is there, and there could be a switch, or the source tree could just look for any folder where there is a .git and process that.

At the moment, the source is only available for the Obsidian repo

What do you mean? The plugin works in every vault.


Increased gas price ceiling to 50Gwei

Closes: #1896
Relay entry fee estimate with no callback price difference:

For 60 Gwei ceiling, entry fee is<PHONE_NUMBER>0000000 wei = 0.081964 ETH
For 50 Gwei ceiling, entry fee is<PHONE_NUMBER>0000000 wei = 0.07897 ETH
For 40 Gwei ceiling, entry fee is<PHONE_NUMBER>0000000 wei = 0.075976 ETH
For 30 Gwei ceiling, entry fee is<PHONE_NUMBER>0000000 wei = 0.072982 ETH

Analyzing the historical data from https://etherscan.io/chart/gasprice, the gas price values are as follows:

13/7/2019 - 13/7/2020 median is<PHONE_NUMBER>7 wei
13/7/2019 - 13/7/2020 average is<PHONE_NUMBER>7 wei
1/1/2020 - 13/7/2020 median is<PHONE_NUMBER>1 wei
1/1/2020 - 13/7/2020 average is<PHONE_NUMBER>1 wei
1/7/2020 - 13/7/2020 median is<PHONE_NUMBER>4 wei
1/7/2020 - 13/7/2020 average is<PHONE_NUMBER>0 wei

Increasing the ceiling to 40 Gwei is a must and increasing it to 50 Gwei is also reasonable.
The impact on DKG with 1% contribution margin as it's now:

30 Gwei ceiling, 20 Gwei actual price: DKG triggered every 72 entries
30 Gwei ceiling, 10 Gwei actual price: DKG triggered every 42 entries
40 Gwei ceiling, 20 Gwei actual price: DKG triggered every 54 entries
40 Gwei ceiling, 10 Gwei actual price: DKG triggered every 34 entries
50 Gwei ceiling, 20 Gwei actual price: DKG triggered every 48 entries
50 Gwei ceiling, 10 Gwei actual price: DKG triggered every 30 entries
60 Gwei ceiling, 20 Gwei actual price: DKG triggered every 42 entries
60 Gwei ceiling, 10 Gwei actual price: DKG triggered every 27 entries
60 Gwei ceiling, 1 Gwei actual price: DKG triggered every 13 entries

Detailed breakdown for DKG available at https://github.com/keep-network/keep-core/issues/1896

I would consider even 60 gwei as the price, to stay ahead of the gas price growth trend. Short peaks should be handled by clients submitting transactions at a greater price if necessary and eating the difference, but if the increase in prices continues we don't want our reimbursements to get completely buried under it.

It sounds reasonable to me. I put detailed calculations in the issue, and summarizing them:

For 60 Gwei ceiling, entry fee (with no callback) is<PHONE_NUMBER>0000000 wei = 0.081964 ETH
60 Gwei gas price ceiling, 20 Gwei actual gas price: DKG is triggered every 42 entries.
60 Gwei gas price ceiling, 10 Gwei actual gas price: DKG is triggered every 27 entries.
60 Gwei gas price ceiling, 1 Gwei actual gas price: DKG is triggered every 13 entries.

If we go from 30Gwei to 60Gwei ceiling we'll increase base (no callback) entry price by 12.3%: 0.072982 ETH -> 0.081964 ETH and we'll increase the fee for opening a keep by 13.73%: 0.07418878 ETH -> 0.08437756.
Given the gas price trends, I tend to agree with @eth-r's proposition of going even further and increase to 60 Gwei.

The 13.73% is unfortunate for tBTC, but this just screams "replace the mechanism" to me. I'm for biting the bullet now and going back to the pricing drawing board after this release


Provide toString for events

When initially writing a bot, it's useful to be able to see what events are running through the bot to be able to figure out what you need to act on.
Unfortunately, just printing the event out results in the default toString, which does not provide any useful information about the event.
It would be nice if they had a toString so that the events could be passed straight to System.out.println or similar for debugging and development purposes.

Can you give a use case for this "debugging" that isn't covered by simply putting the logger on TRACE
WebSocketClient.LOG.setLevel(SimpleLog.Level.TRACE);

That function is not discoverable. You have to know it exists to be able to use it, which is the kind of thing I'm trying to work around here. The more you can find out about the code without having to read the entire source code tree, the better.
That function itself only provides part of the story. It does not say which fields in the event contain which values, and it does not say which event it is firing. It might be useful for inspecting the data given to us by the server, but it is not useful for quickly exploring which events are relevant to your bot or for clarifying the structure of the Java objects used, both of which are invaluable in the earlier stages of bot development.

Why do you have to read the source tree to figure out what events exist and what they provide?
We have a pretty simple event hierarchy which is entirely documented with package docs and class level docs, each method is listed and in a logical place to find.
I am not sure how printing every event to console (which is a MASSIVE spam btw) can be helpful for new users in any way and it is certainly a very big time investment to implement this.
Even the wiki has all events listed here: https://github.com/DV8FromTheWorld/JDA/wiki/8)-List-of-Events

It's not massive spam. If you invite a small fledgling bot to a private test server, printing out every event to the console is fine. But more importantly, it precisely points you to the events you're looking for. Rather than having to search the full list of event types, or trying to guess what the official Discord API terminology for the event you're looking for is, you just look at what your bot printed and you will know what your bot needs to handle.

I recommend if you want to learn more about events to just join our discord and ask for the event there. This is not going to be implemented as it is a big task for a little payoff. I hope you can understand that.

If you really insist on printing every event into your console you can try using reflections to access all getters
@Override
public synchronized void onGenericEvent(Event event)
{
    System.out.println("EventType: " + event.getClass().getSimpleName());
    for (Method m : event.getClass().getMethods())
    {
        final String name = m.getName();
        final Class<?> returnType = m.getReturnType();
        if ((name.startsWith("is") || name.startsWith("get")) && !returnType.equals(Void.class))
            System.out.println("\t" + name + ": " + returnType.getSimpleName());
    }
}

But honestly I think asking for events is much better and you will have a much better experience than trying to find it this way.

Closing this as it seems inactive and I made my statement.

What kind of activity are you expecting?

A single response to my two comments I made 6 days ago would be a start.

Fair enough. For the first, I don't see much of a problem with automatically generating the toString method. It doesn't need to get anything special across, just give an idea of the events being sent and the breadth of information that each specific event contains (for example, it's not intuitively obvious that the event for joining a voice chat does not include the muted/deafened state). IntelliJ can do most of the work with alt+insert.
Regarding reflection, that works to an extent but everyone who wants to try the same will either have to write the code from scratch or somehow come across this issue to find that code. The existence of reflection (let alone how to use it) will likely not be known to Java newbies, which rules out the first in arguably a majority of cases.

I'm afraid that java newbies won't even understand how the event system works let alone that every event is a subclass of Event.
I doubt that the first thing a new JDA user would do would be implementing an EventListener and just doing System.out.println(event);. The first thing you would/should do is look at the list of events in the javadocs or in ListenerAdapter or in the wiki or just ask in the development discord. ListenerAdapter is explicitly mentioned in the README as well.
We have bigger priorities than adding a toString implementation to 140 events (client events excluded) such as fixing issues with permissions, moving towards etf or adding implementations for new features (audit logs)/client features (relationships).
My suggestion is that if you want this to be added you will have to do it yourself as our staff is not likely gonna do this any time soon.

I doubt that the first thing a new JDA user would do would be implementing an EventListener and just doing System.out.println(event);.

Then brace yourself, because that's exactly what I did. As for `ListenerAdapter``, I have still yet to find a practical use for it.
If this is a problem of priorities, then why not leave the ticket open? Closing it implies it is complete, or never to be completed. If you close a ticket, that says to any reader that you will not even be accepting pull requests to that effect.

It should imply it is never to be completed. I already said we won't implement it.

Discussed on discord; no plans to accept PRs relating to this.


Add DeletionConfirmation admission controller

The current way of deletion of Shoot resources is a left over from when the Gardener was working as TPR operator and external validation webhooks were not available.
DeletionConfirmation admission controller fixes this issue by allowing deletion of resources to happen when a Shoot resource is annotated with confirmation.garden.sapcloud.io/deletion=true
It is disabled by default to preserve the old behavior.

@vlerenc: Is this change generally OK to you? Some months back you voted for explicitly putting the deletion timestamp into the Shoot annotations to confirm the deletion.
With this DeletionConfirmation admission controller you first need to set an annotation (without any dynamic value, just confirmation.garden.sapcloud.io/deletion=true), and the DELETE request is only allowed if this annotation is set.

If you want a dynamic value, then the value of the annotation can be set to the uid of the resource.

ping @vlerenc
@mvladev: let's definitely use a dynamic value, what about the shoot name itself?

Hi colleagues, for me the most important bit is that we have an annotation in the first place. We, but now even more importantly our users have programmatic access. If we have no annotation, they may accidentally delete a cluster and complain. Whether or not the value is dynamic (timestamp, uid, name) is not that critical to me anymore. If you can agree on either one of them, it sounds OK to me. They all have certain side effects. Name and UID must be read first, time could be used to narrow the time window in which deletions are possible, etc. But even if you decide for a non-dynamic value, I would be still happy. At least, nobody can just call kubectl delete shoot anymore, which was the original "prevention goal" when we introduced it a year ago. ;-)

OK, then let's keep it as Martin proposed it. No dynamic value, but to delete a Shoot, one needs to set the annotation first and only is allowed to execute kubectl delete shoot after the annotation was set.


placeholder support

It would be nice if you could specify a placeholder attribute on all supported html5 input types, specifically text, search, url, tel, email, and password input types.

render_kw={"placeholder": "ur placeholder"}


New snipper

Hi,
I've wnated to add a new snipper called countStringOccurrences, which counts the occurrences of a string in another string
for examle: 'tt' occurs twice at 'attbtt'
right now theres only countOccurrences of an item in an array
is it ok to add it?

Thanks for asking. countOccurences will work with strings if spread to arrays ([...str]), so it's pretty much the same snippet in reality, so we shouldn't add one more for strings.

spread it to array would help you find character occurences but not another string occurences

@coralhayoun Fair enough, String.prototype.search() orString.prototype.indexOf() in combination with String.prototype.split() will most likely do the trick then.


RTL is not working

Environment
Android 9, Samsung galaxy s8
Description
The RTL is working well on example project "BasicPagerViewExample.tsx" but it is not working when I install the package from npm.
Reproducible Demo
I created a sample project
npx react-native init TestRTL --template react-native-template-typescript
And add the package
yarn add react-native-pager-view
And the UI like that
<SafeAreaView style={backgroundStyle}>
  <StatusBar barStyle={isDarkMode ? 'light-content' : 'dark-content'} />
  <PagerView
    style={{width: '100%', height: '100%'}}
    initialPage={0}
    layoutDirection="rtl">
    <View key="1">
      <Text>First page</Text>
    </View>
    <View key="2">
      <Text>Second page</Text>
    </View>
  </PagerView>
</SafeAreaView>

I have the same issue. rtl doesn't work with next branch.

As a work around from the latest version 5.4.11 I set layoutDirection to ltr
It will work on RTL but animation is from the other side unfortunately

I am facing same issue on layoutDirection any thoughts on this. ?

@trumdongnat https://github.com/callstack/react-native-pager-view/pull/613 this pr fixes this issue, please help me get it merged

This has been fixed by @sebasg0 and released in https://github.com/callstack/react-native-pager-view/releases/tag/v6.0.1

RTL still not working in android , in IOS its working.
tested on Pixel 6 API 33.

What’s your issue @ahmad6242 as for me android always worked.
Can you try doing the following in android.
In manifest add this android:supportsRtl="true" . In MainApplication.java add I18nUtil sharedI18nUtilInstance = I18nUtil.getInstance();      sharedI18nUtilInstance.allowRTL(this, true);
This 2 lines of code but don’t forget to do this import at the top import com.facebook.react.modules.i18nmanager.I18nUtil; . Let me know if it works!

@sebasg0 Ok i will try that tonight. Will let you know. Thanks


How to get second page of TimelineFeed results

How can I get second page of TimelineFeed results?

Changed InstagramTimelineFeedRequest to pass pagination (maxId).

@brunocvcunha this doesn't work anymore.  You'll need to pass max_id and various other pagans as post parameters.  The post request however is NOT signed.

As a guide, the getTimelineFeed method here should help: https://github.com/mgp25/Instagram-API/blob/master/src/Request/Timeline.php

To fix the Timeline pagination, you must to change 2 simple things:

Change the method getUrl() of the Class InstagramTimelineFeedRequest like that:
@Override
public String getUrl() {
      String url = "feed/timeline";
if (maxId != null && !maxId.isEmpty()) {
url += "?max_id=" + maxId;
}
return url;
}

Change the attribut long pk; of the Class InstagramComment to String pk;


Allow for  in markdown

This replaces /docs/static with /static in image URLs. This lets them be rendered by github as well as our backend. It doesn't touch existing syntax, but going forward we can use /docs/static/... to allow rendering both in our websites and on github.
For example see here https://github.com/Microsoft/pxt-arcade/blob/dbgconn/docs/hardware/dbg.md

Minor benefit, but after everything is updated so the links work, switching the links over to the new syntax would also resolve https://github.com/Microsoft/pxt/issues/5021, correct?

I suppose, though I guess we should also fix staticpkg...

We need to patch the code cards too.

@mmoskal added a patch for static pkg as well.

Also fixes https://github.com/Microsoft/pxt-arcade/issues/805 besides the simulator portion


fix: moves auro library to dependencies in package.json #41

Alaska Airlines Pull Request
Moves Auro Library into dependencies from devDependencies to avoid installation issues, as originally reported in #573.
Resolves: #41
Summary:

Moved to dependencies within package.json
Regenerated package.lock
Verified build and test ran successfully

Type of change:
Please delete options that are not relevant.

[x] Revision of an existing capability

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Chris Friedberg seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

:tada: This PR is included in version 3.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Worker threads do not initiate parallel executions

Following the discussion on https://github.com/camunda-community-hub/spring-zeebe/issues/424, we added micrometer in the classpath and the worker's thread pool size seems ok using the following:

Zeebe broker : 8.2.13
spring-boot-starter-camunda : 8.2.4
Java : 17

However we notice the following behavior.
We create a process instance in which the first job invokes an external service which has internally a thread sleep of 30 seconds.
While waiting for the response from the external service we create additional process instances and we were expecting these jobs to be executed in parallel with the first. However it seems that they get executed (in parallel) but only after the first job has been executed.
It seems as if the jobs are fetched at a specific point in time, and if the current batch is executing then additional jobs that are created are not picked up, even though that the number of parallel threads allows such an action
This behavior severely impacts parallel job processing

This is related to a behaviour of the zeebe client: https://github.com/camunda/zeebe/blob/main/zeebe/clients/java/src/main/java/io/camunda/zeebe/client/impl/worker/JobWorkerImpl.java#L213-L216
To overcome this, you could enable job streaming: https://github.com/camunda-community-hub/spring-zeebe?tab=readme-ov-file#enable-job-streaming


[designspaceLib] floating point rounding causes "Cannot determine default source to look up family name"

Say I have a designspace with axis mappings affecting the "default design location", like this:
<?xml version='1.0' encoding='UTF-8'?>
<designspace format="5.0">
  <axes>
    <axis tag="wght" name="Weight" minimum="100" maximum="900" default="374.390244">
      <map input="100" output="100"/>
      <map input="400" output="428"/>
      <map input="900" output="900"/>
    </axis>
  </axes>
  <sources>
    <source filename="X-Thin.ufo" name="X Thin" familyname="X" stylename="Thin">
      <location><dimension name="Weight" xvalue="100"/></location>
    </source>
    <source filename="X-Regular.ufo" name="X" familyname="X" stylename="Regular">
      <location><dimension name="Weight" xvalue="400"/></location>
    </source>
    <source filename="X-Black.ufo" name="X Black" familyname="X" stylename="Black">
      <location><dimension name="Weight" xvalue="900"/></location>
    </source>
  </sources>

The default value is generated by glyphsLib in this case, but even if it's manually written to the most exact representation, the bug still occurs.
Now, if I run designspaceLib.getStatNames(designspaceDoc: DesignSpaceDocument) it calls designspaceDoc.findDefault() which tries to match the "default" to a source, doing the following:
def findDefault(self):
  defaultLoc = self.newDefaultLocation()
  for sourceDescriptor in self.sources:
    if sourceDescriptor.getFullDesignLocation(self) == defaultLoc: # <—— bug!
      self.default = sourceDescriptor
      return sourceDescriptor

If findDefault returns None the call fails and a warning is logged (by designspaceLib.getStatNames)
WARNING:fontTools.designspaceLib.statNames:Cannot determine default source to look up family name.

The issue is this comparison:
sourceDescriptor.getFullDesignLocation(self) == defaultLoc

If we print the value of defaultLoc it's clear what is going on:
>>> defaultLoc
OrderedDict([('Optical size', 28.0), ('Weight',<PHONE_NUMBER>066667)])

Let's also print the source descriptors which are being tested for equivalence:
{'Weight': 100.0}
{'Weight': 400.0}
{'Weight': 900.0}

Aha! 400.0 !=<PHONE_NUMBER>066667
The newDefaultLocation produces the default location like this:
def newDefaultLocation(self):
  loc = collections.OrderedDict()
  for axisDescriptor in self.axes:
    loc[axisDescriptor.name] = axisDescriptor.map_forward(
      axisDescriptor.default)
  return loc

Which goes through AxisDescriptor.map_forward:
def map_forward(self, v):
  from fontTools.varLib.models import piecewiseLinearMap
  if not self.map:
    return v
  return piecewiseLinearMap(v, {k: v for k, v in self.map})

Which uses piecewiseLinearMap:
https://github.com/fonttools/fonttools/blob/4.39.3/Lib/fontTools/varLib/models.py#L502-L519
Summary: Lookup of "default" design location fails because it goes through axis mapping using IEEE 754 floating point math, which has limited precision; the result is a computed value that is technically different from the actual source axis values.

I believe the expectation is that there is a explicit axis map for the default value, in your case it's missing hence it gets interpolated and might fail for the reasons you outlined

Also, the source locations xvalue attributes are supposed to be definied in the internal design-space coordinate system (the "output" of the axis map elements), whereas it appears you are using user-space coordinates (the "input" in axis map, as used in fvar axis definitions)

Interesting! Thank you for the help.
I think Glyphs.app might have its UI backwards.
If I do this in Glyphs:

Notice how the "internal coordinates" say 400 and "external coordinates" say 350.
(how I read this: "Map 400 from your internal sources to 350. When user says 350 they actually get 400")
Then this is what designspace gets generated by fontmake:
<designspace format="5.0">
  <axes>
    <axis tag="wght" name="Weight" minimum="100" maximum="900" default="445.454545">
      <map input="100" output="100"/>
      <map input="400" output="350"/>
      <map input="900" output="900"/>
    </axis>
  </axes>
  <sources>
    <source filename="X-Thin.ufo" name="X Thin" familyname="X" stylename="Thin">
      <location><dimension name="Weight" xvalue="100"/></location>
    </source>
    <source filename="X-Regular.ufo" name="X Regular" familyname="X" stylename="Regular">
      <location><dimension name="Weight" xvalue="400"/></location>
    </source>
    <source filename="X-Black.ufo" name="X Black" familyname="X" stylename="Black">
      <location><dimension name="Weight" xvalue="900"/></location>
    </source>
  </sources>

Which is not correct.
However if I invert the mapping in Glyphs, by setting the internal value on the external axis and vice versa, like this:

(how I read this: "Map 350 from your internal sources to 400. When user says 400 they actually get 350")
Then—with the inverted mapping—the correct mapping is produced in the designspace:
<?xml version='1.0' encoding='UTF-8'?>
<designspace format="5.0">
  <axes>
    <axis tag="wght" name="Weight" minimum="100" maximum="900" default="350">
      <map input="100" output="100"/>
      <map input="350" output="400"/>
      <map input="900" output="900"/>
    </axis>
  </axes>
  <sources>
    <source filename="X-Thin.ufo" name="X Thin" familyname="X" stylename="Thin">
      <location><dimension name="Weight" xvalue="100"/></location>
    </source>
    <source filename="X-Regular.ufo" name="X Regular" familyname="X" stylename="Regular">
      <location><dimension name="Weight" xvalue="400"/></location>
    </source>
    <source filename="X-Black.ufo" name="X Black" familyname="X" stylename="Black">
      <location><dimension name="Weight" xvalue="900"/></location>
    </source>
  </sources>

I'm probably just confused about what is what here.
Really what I'm trying to do is to make it so that when the user asks for wght=350 the outlines for wght=400 is used. When the user asks for wght=400, outlines for something like wght=430 is used. Essentially making everything a bit bolder around wght=400 as seen by the users.


Add login-form-using-the-http-connector sample

Purpose

Add sample for login-form-using-the-http-connector.

Until https://github.com/wso2/integration-studio-examples/issues/58 issue fixed, Users need to add the following configuration in conf/deployment.toml in Embedded MI in Integration Studio to enable the message formatter for content type text/html
[[custom_message_formatters]]
content_type = "text/html"
class = "org.apache.axis2.transport.http.ApplicationXMLFormatter"


#159650798 create question and answer details page

What does this PR do?
This PR creates the details page for users to view the whole question and answers to the question
How should this be manually tested?

clone the repo

$ git clone https://github.com/asheuh/StackOverflow-lite
$ cd StackOverflow-lite/UI/templates/mains
$ [your browser] details.html

What are the relevant pivotal tracker stories?
159650798

world class

Dope

Nice design

Thanks guys

It's nice

Nice

Nice


fix: bug 2201 protect against empty messages - e.g. buttons

retrigger build

:tada: This PR is included in version 0.367.11 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


feature: extend ObjectMapper behavior

Please read our contributor guide before
creating an issue. Also consider discussing your idea on
the discussion forum first.
Describe the Feature Request
provide solution to extend  ObjectMapper behavior during deserialize/serialize.
In my case , collecting field name with null value by feature like this.
Describe Preferred Solution
class DgsSchemaProvider(
	private val applicationContext: ApplicationContext,
	private val federationResolver: Optional<DgsFederationResolver>,
	private val existingTypeDefinitionRegistry: Optional<TypeDefinitionRegistry>,
	private val mockProviders: Optional<Set<MockProvider>>,
	private val schemaLocations: List<String> = listOf(DEFAULT_SCHEMA_LOCATION),
	private val dataFetcherResultProcessors: List<DataFetcherResultProcessor> = emptyList(),
	private val objectMapperModuleList: List<Module> = listOf(CustomModule()),
) {
// ...
        // add custom module
	private val objectMapper = jacksonObjectMapper()
		.registerModule(JavaTimeModule())
		.registerModules(objectMapperModuleList)
// ...

Describe Alternatives

This is for serializing the HTTP responses specifically?

@pcyan could you help us understand..

the HTTP Request and the HTTP Response that you are expecting
the GraphQL Schema you present to your users.
are you using Scalars? are you aware of graphql-java's extended scalars?

@pcyan could you help us understand..

the HTTP Request and the HTTP Response that you are expecting
the GraphQL Schema you present to your users.
are you using Scalars? are you aware of graphql-java's extended scalars?

@berngp

my schema example

extend type Mutation {
    updateFoo(
        updateInput:[MyInput]
    ):ID
}

input MyInput {
    """notnull"""
    firstName:String!
    """notnull"""
    lastName:String!
    """nullable"""
    nickName:String
}

my request example

mutation mutationWithNullValueField {
    updateFoo(
        updateInput: [
            {
                firstName: "harry"
                lastName: "porter"
                # nickName with null value
                nickName: null
            },
            {
                firstName: "harry"
                lastName: "porter"
                # nickName not present
            },
            {
                firstName: "harry"
                lastName: "porter"
                nickName: "you know who"
            }
        ]
    )
}

it's not about scalar, i want to collect field name with null value, such as updateInput[0] in the example。 but in java, updateInput[0] and updateInput[1] are not different

This is for serializing the HTTP responses specifically?

@paulbakker it's about deserializing the HTTP request specifically. i have notice that there is a issue about custom objectMapper https://github.com/Netflix/dgs-framework/issues/565
how do we customize deserialize/serialize behavior under dgs-framework?

Just bumping the thread here because I ran into the same problem with a JSON field in the schema where the object being converted to JSON has a java.time.Duration property that requires the JavaTimeModule to be registered with the ObjectMapper.

This is now available in  v4.9.24


DeprecationWarning: register_plugin is deprecated. Use polars.plugins.register_plugin_function instead.

Just a warning I got. May dig into and make PR myself but just making note

Did you have a chance looking at it? I'm facing it now while using polars 1.2.1 and thinking about implementing the PR myself, but my knowledge is the plugins areas of polars is minimal

@Yvictor @ctyler9 i created https://github.com/Yvictor/polars_ta_extension/pull/10 to start handling the issue.

fix with version 0.1.3


Change type from int to string to set a valid data for autorest #6586

[x] The purpose of this PR is explained in this or a referenced issue.
[x] The PR does not update generated files.

These files are managed by the codegen framework at Azure/autorest.go.

[] The PR targets the latest branch.
[ ] Tests are included and/or updated for code changes.
[ ] Updates to CHANGELOG.md are included.
[ ] Apache v2 license headers are included in each file.

Fix for this issue #6586. It can be resolved like that or will be fixed in the autorest package

This package, and the majority of the SDK, is generated from swagger files.  We cannot make hand-edits as they will be lost when the SDK is regenerated.
Looking at the swagger it's properly defined as a string so this appears to be a codegen issue.  Can you please open an issue to track getting this fixed?

Hi @jhendrixMSFT, I'm not sure I understood right what you mean. I mean that issue is already created https://github.com/Azure/azure-sdk-for-go/issues/6586

Oh I see, I missed the issue in the original comment, sorry for the confusion.


Correct by platform archive AQAvitTapFiles.tar.gz location

The "by platform" AQAvitTapFiles.tar.gz archiveArtifact location is wrong, so is not finding it:
13:36:43  + find target/linux/x64/temurin/AQAvitTaps -type f -name *.tap -exec tar -czf target/linux/x64/temurin/AQAvitTaps/AQAvitTapFiles.tar.gz {} +
[Pipeline] fileExists
[Pipeline] {
[Pipeline] archiveArtifacts
13:36:44  Archiving artifacts
13:36:47  ‘AQAvitTapFiles.tar.gz’ doesn’t match anything, but ‘target/linux/x64/temurin/AQAvitTaps/AQAvitTapFiles.tar.gz’ does. Perhaps that’s what you mean?
[Pipeline] }
13:36:47  ERROR: Error during copy and archive artifacts for build job: build-scripts/jobs/jdk23/jdk23-linux-x64-temurin
13:36:47  ERROR: No artifacts found that match the file pattern "AQAvitTapFiles.tar.gz". Configuration error?
13:36:47  Setting overall build result to FAILURE

When I fixed the the platform AQAvitTapFiles.tar.gz’ location, I forgot to change the achived artifact target!

Do these changes imply that we now have many artifacts instead of a single one? (which then needs to be accounted for in later verification and upload steps? I think I missed seeing the initial change, that has the trickle down effect (requires changes in other areas). Guessing this is what @sophia-guo was referring to, regarding verification scripts, requiring updates to handle this new scenario.

@smlambert @sophia-guo Ok, so I was wondering if this looked right!
What we have currently is:

an archive "by platform" in target/OS/ARCH/VARIANT/AQAvitTaps/AQAvitTapFiles.tar.gz
and an overall "single all pipeline" in AQAvitTaps/AQAvitTapFiles.tar.gz

Is that correct or should we just have (2) ?
PR https://github.com/adoptium/ci-jenkins-pipelines/commit/e6947980814dc85266954880202c6adfedadbcfa implies both "Tap files per platforms keep as before."?

I like them organized by platform, but then the final requirements are that:

all TAP files, including any Grinder TAP files that are gathered during triage are verified by these scripts
1 tar.gz containing all TAP files for a release gets uploaded to the binaries repo (related:  https://github.com/adoptium/github-release-scripts/issues/96)
we fix the marketplace API to point to the single tar file (related: https://github.com/adoptium/marketplace-api.adoptium.net/issues/33)

I like them organized by platform, but then the final requirements are that:
* all TAP files, including any Grinder TAP files that are gathered during triage are verified by these [scripts](https://github.com/adoptium/aqa-tests/tree/master/buildenv/jenkins/tapVerification)

* 1 tar.gz containing all TAP files for a release gets uploaded to the binaries repo (related:  [Ensure we are pushing AQAvit TAP files to the binaries repository for each release github-release-scripts#96](https://github.com/adoptium/github-release-scripts/issues/96))

* we fix the marketplace API to point to the single tar file (related: [Reorganize the tap result link aqavit_tapresult_link marketplace-api.adoptium.net#33](https://github.com/adoptium/marketplace-api.adoptium.net/issues/33))

I think the "by platform" tar.gz does now include all the "re-run" sub-testjobs @sophia-guo ? but manually run Grinder's would need adding manually?
I query whether the "single tar.gz" for the whole "openjdkNN-pipeline" job is really useful, it seems to be duplicating the "by platform" ones. The above also states 1-tar per release, and we may not always have 1 "openjdkNN-pipeline", especially if we decide to split "primaries" from "secondaries".

Test with this PR: https://ci.adoptium.net/job/build-scripts/job/openjdk21-pipeline/292/

Andrew, to be clear, there is no expectation that we can produce the single tar file, verify it and upload it as a direct part of the release pipeline.  It will have to be triggered later, due to the fact that on average we have 20+ Grinder TAP files to include (due to infra or other issues during a release).

Test with this PR: https://ci.adoptium.net/job/build-scripts/job/openjdk21-pipeline/292/

Looks good

This is not correct. The reason is the revert changes https://github.com/adoptium/ci-jenkins-pipelines/pull/1040  is merged into taps https://github.com/adoptium/ci-jenkins-pipelines/commit/6e22834340d2f9b95bcef218d77c7f7eabcf4f2c by hit some button of github issues ( not by the PR author). Feels like some new features of github, reviewers has the permission?
Those per platform archive are unnecessary and duplicate. Should be deleted. Archives are done by https://github.com/andrew-m-leonard/ci-jenkins-pipelines/blob/8cc041a8c8e96c5c48a984f972bfd642b78f33d3/pipelines/build/common/build_base_file.groovy#L1014-L1040


Kubuntu returns Neon which causes error

In the file /usr/lib/python3/dist-packages/cloudutils/syscfg.py there is only Ubuntu, but in Kubuntu, it gives Neon if you run
/usr/bin/lsb_release -i Distributor ID:	Neon
This makes cloudstack-setup-management fail,
Cloudstack 4.15.1
Distributor ID:	Neon
Description:	KDE neon User Edition 5.22
Release:	20.04
Codename:	focal
(Kubuntu 20.04)

@abdelouahabb Cloudstack could still work since it reads the contents of /etc/lsb-release and not run any command to get the OS version

@abdelouahabb As of now, vanilla Ubuntu is supported, feel free to add support for Kubuntu if you'd like

@abdelouahabb Kubuntu is not a supported distribution, please use Ubuntu Server. Refer: http://docs.cloudstack.apache.org/en/latest/releasenotes/compat.html

Kubuntu is just a normal Ubuntu with the main difference is the GUI and the pkcon update instead of apt update (and yes, for servers, better use headeless tools)
I will try to make a PR to add it

Hi @abdelouahabb please advise when you have created the PR to fix this

Hi @abdelouahabb community can't commit to supporting and testing non-LTS and non-mainstream distros. But we'll be happy to accept any PRs of somebody wants to maintain support for their favourite distro. Feel free to raise a PR and discuss on dev@ML.
Closing on this remark.


Make candidate questioniarre look better on mobile screens

going to merge this in real quick so its live and i can test better.
may need to reopen.


Set result card lang to its scope

This should give proper CJK variant rendering to the ja and zh scopes. (It also helps screen readers I think.) Not tested 🤐

Thanks for the PR :) Merging and integrating soon.


Support for ST-Link v2 with flash script

Issue
Are there any plans to add support for the ST-Link V2 programmer on the flash utility?
How can I flash the Olimex board without using the command     ros2 run micro_ros_setup flash_firmware.sh from the tutorials?

Hardware description: Olimex stm32-e407
RTOS: NuttX
Installation type: micro_ros_setup
Version or commit hash: foxy

Steps to reproduce the issue
while st-link v2 is plugged in
ros2 run micro_ros_setup flash_firmware.sh
Expected behavior
Successfull upload with ST-link v2
Actual behavior
Flashing firmware for nuttx platform olimex-stm32-e407
Flashing firmware for nuttx platform olimex-stm32-e407
Error. Unsuported OpenOCD USB programmer
Additional information

Hello, there are no plans in our roadmap for implementing this feature. But I think that looking into the other flashing script should be easy to implement this.
Could you consider contributing with a PR?

Sure will work on it this weekend. 👍

Thanks a lot!


Continuous queries - wrong values calculated

My setup:

Influx db version: 0.9.5-nightly-6ecb62e
One measurement, request, holds a point for each web request. The field "counter" always exists.
One continuous queries aggregates request counts into hourly_requests.
One continuous queries aggregates request counts into daily_requests.

After the system has been running for a few days, I can see that the aggregation measurements hold the wrong values when compared to the detailed points:
This is the real data:
> SELECT count(counter) AS "Total" FROM request WHERE time > '2015-11-08' and time < '2015-11-17' GROUP BY time(1d)

request
time	Total
2015-11-08T00:00:00Z
2015-11-09T00:00:00Z
2015-11-10T00:00:00Z	769
2015-11-11T00:00:00Z	1930
2015-11-12T00:00:00Z	1393
2015-11-13T00:00:00Z	795
2015-11-14T00:00:00Z	5414
2015-11-15T00:00:00Z	201
2015-11-16T00:00:00Z	4379

The hourly aggregation is mostly correct, but for one day (2015-11-10) it's missing a query:
> SELECT sum(Total) FROM hourly_requests WHERE time > '2015-11-08' and time < '2015-11-17'  GROUP BY time(1d)

hourly_requests
time                         	sum
2015-11-08T00:00:00Z
2015-11-09T00:00:00Z
2015-11-10T00:00:00Z	768
2015-11-11T00:00:00Z	1930
2015-11-12T00:00:00Z	1393
2015-11-13T00:00:00Z	795
2015-11-14T00:00:00Z	5414
2015-11-15T00:00:00Z	201
2015-11-16T00:00:00Z	4379

The daily aggregation is completely off, missing many queries nearly every day:
> SELECT Total FROM daily_requests

daily_requests
time	                                Total
2015-11-10T00:00:00Z	598
2015-11-11T00:00:00Z	1883
2015-11-12T00:00:00Z	1335
2015-11-13T00:00:00Z	749
2015-11-14T00:00:00Z	5396
2015-11-15T00:00:00Z	188
2015-11-16T00:00:00Z	4379

@caligula1989 can you give the output from SHOW CONTINUOUS QUERIES and SHOW RETENTION POLICIES ON <database>?

And just for completeness, can you also paste in the [continuous_queries] section from your config file? E.g.
[continuous_queries]
  log-enabled = true
  enabled = true
  recompute-previous-n = 2
  recompute-no-older-than = "10m0s"
  compute-runs-per-interval = 10
  compute-no-more-than = "2m0s"

SHOW CONTINUOUS QUERIES
name: mydb

name		query
daily_requests	CREATE CONTINUOUS QUERY daily_requests ON mydb BEGIN SELECT count(counter) AS "Total", sum(Price) AS "Revenue", mean(RunTime) AS "AverageRunTime", count(PerfectMatch) AS "PerfectMatches", count(NoMatch) AS "NoneMatches", count(PossibleMatch) AS "PossibleMatches", sum(Warnings) AS "Warnings", count(ErrorTypeID) AS "Errors" INTO "mydb"."default".daily_requests FROM "mydb"."default".request WHERE APIVersion !~ /undefined/ GROUP BY time(1d), APIKey, OrganizationID, OrganizationName, DeveloperClass, AccountManagerID, AccountManager, AccountDirectorID, AccountDirector, APIVersion, StatusCode END
hourly_requests	CREATE CONTINUOUS QUERY hourly_requests ON mydb BEGIN SELECT count(counter) AS "Total", sum(Price) AS "Revenue", mean(RunTime) AS "AverageRunTime", count(PerfectMatch) AS "PerfectMatches", count(NoMatch) AS "NoneMatches", count(PossibleMatch) AS "PossibleMatches", sum(Warnings) AS "Warnings", count(ErrorTypeID) AS "Errors" INTO "mydb"."default".hourly_requests FROM "mydb"."default".request WHERE APIVersion !~ /undefined/ GROUP BY time(1h), APIKey, OrganizationID, OrganizationName, DeveloperClass, AccountManagerID, AccountManager, AccountDirectorID, AccountDirector, APIVersion, StatusCode END
hourly_errors	CREATE CONTINUOUS QUERY hourly_errors ON mydb BEGIN SELECT count(counter) AS "Total" INTO "mydb"."default".hourly_errors FROM "mydb"."default".request WHERE APIVersion !~ /undefined/ AND ErrorTypeID !~ /undefined/ GROUP BY time(1h), ErrorTypeID END

SHOW RETENTION POLICIES ON mydb
name	duration	replicaN	default
default	0		1		true

This is my configuration:
[continuous_queries]
log-enabled = true
enabled = true
recompute-previous-n = 2
recompute-no-older-than = "10m"
compute-runs-per-interval = 10
compute-no-more-than = "2m"

After upgrading to RC2, this is still happening.

I've set up a long-running test to try and repro. Will report back next week with results.

My test is invalid due to my laptop frequently going to sleep, rendering the CQs non-functional. I am duplicating the test on a new 0.9.5 instance.

My test results suggest there is something wrong with GROUP BY time(1d) continuous queries.
I have Telegraf running on a hosted Ubuntu box, recording system metrics every 10 seconds.
I set up a CQ to downsample counts from the cpu=cpu-total series every hour and every day:
> show continuous queries
...
name: telegraf
--------------
name	query
hourfoo	CREATE CONTINUOUS QUERY hourfoo ON telegraf BEGIN SELECT count(value) AS "count" INTO "telegraf"."default"._cpu_usage_guest_hour FROM "telegraf"."default".cpu_usage_guest WHERE cpu = 'cpu-total' GROUP BY time(1h) END
dayfoo	CREATE CONTINUOUS QUERY dayfoo ON telegraf BEGIN SELECT count(value) AS "count" INTO "telegraf"."default"._cpu_usage_guest_day FROM "telegraf"."default".cpu_usage_guest WHERE cpu = 'cpu-total' GROUP BY time(1d) END

The hourly rollups appear to be working quite well. Here's the raw data, grouped by hour:
> select count(value) from cpu_usage_guest where time > '2015-11-24' and cpu='cpu-total'  group by time(1h)
name: cpu_usage_guest
---------------------
time			count
2015-11-24T00:00:00Z	173
2015-11-24T01:00:00Z	360
2015-11-24T02:00:00Z	360
2015-11-24T03:00:00Z	360
2015-11-24T04:00:00Z	360
2015-11-24T05:00:00Z	360
<snip> (all hourly values are 360)
2015-12-02T15:00:00Z	360
2015-12-02T16:00:00Z	360
2015-12-02T17:00:00Z	360
2015-12-02T18:00:00Z	360
2015-12-02T19:00:00Z	262

and that matches very well with the downsampled measurement from the 1h CQ:
> select * from _cpu_usage_guest_hour
name: _cpu_usage_guest_hour
---------------------------
time			count
2015-11-24T00:00:00Z	173
2015-11-24T01:00:00Z	360
2015-11-24T02:00:00Z	360
2015-11-24T03:00:00Z	360
2015-11-24T04:00:00Z	360
<snip> (all hourly values are 360)
2015-12-02T15:00:00Z	360
2015-12-02T16:00:00Z	360
2015-12-02T17:00:00Z	360
2015-12-02T18:00:00Z	360
2015-12-02T19:00:00Z	252

>

(We expect the last hour to be off by a bit since the CQ only runs every 6 minutes.)
However, the raw data grouped by day does not match the 1d CQ downsampled values.
The raw data grouped by day is clearly correct (86400 seconds in a day, sampling every 10s, means 8640 points per day):
> select count(value) from cpu_usage_guest where time > '2015-11-24' and cpu='cpu-total'  group by time(1d)
name: cpu_usage_guest
---------------------
time			count
2015-11-24T00:00:00Z	8453
2015-11-25T00:00:00Z	8640
2015-11-26T00:00:00Z	8640
2015-11-27T00:00:00Z	8640
2015-11-28T00:00:00Z	8640
2015-11-29T00:00:00Z	8640
2015-11-30T00:00:00Z	8640
2015-12-01T00:00:00Z	8640
2015-12-02T00:00:00Z	7099

The downsampled CQ, however, is missing 661-665 points for each day, consistently.
> select * from _cpu_usage_guest_day
name: _cpu_usage_guest_day
--------------------------
time			count
2015-11-24T00:00:00Z	7788
2015-11-25T00:00:00Z	7975
2015-11-26T00:00:00Z	7976
2015-11-27T00:00:00Z	7976
2015-11-28T00:00:00Z	7977
2015-11-29T00:00:00Z	7977
2015-11-30T00:00:00Z	7978
2015-12-01T00:00:00Z	7979
2015-12-02T00:00:00Z	6252

Difference between the two:
date                                 raw          CQ          difference
2015-11-24T00:00:00Z	8453	7788	665
2015-11-25T00:00:00Z	8640	7975	665
2015-11-26T00:00:00Z	8640	7976	664
2015-11-27T00:00:00Z	8640	7976	664
2015-11-28T00:00:00Z	8640	7977	663
2015-11-29T00:00:00Z	8640	7977	663
2015-11-30T00:00:00Z	8640	7978	662
2015-12-01T00:00:00Z	8640	7979	661
2015-12-02T00:00:00Z	7099	6252	847

Each day the daily CQ is missing ~110 minutes worth of points.

My CQ settings are the defaults:
[continuous_queries]
  log-enabled = true
  enabled = true
  recompute-previous-n = 2
  recompute-no-older-than = "10m"
  compute-runs-per-interval = 10
  compute-no-more-than = "2m"

The pattern continues as usual. The 1h CQ shows 360 points for every hour since I set it up, excepting the first hour and the most recent hour, as expected.
> select * from /_cpu.*hour/ where count != 360
name: _cpu_usage_guest_hour
---------------------------
time			count
2015-11-24T00:00:00Z	173
2015-12-16T00:00:00Z	90

However, the daily CQ is continually missing ~660 events, and the number is not regular, and is slowly increasing (!?):
> select * from /_cpu.*day/
name: _cpu_usage_guest_day
--------------------------
time			count
2015-11-24T00:00:00Z	7788
2015-11-25T00:00:00Z	7975
2015-11-26T00:00:00Z	7976
2015-11-27T00:00:00Z	7976
2015-11-28T00:00:00Z	7977
2015-11-29T00:00:00Z	7977
2015-11-30T00:00:00Z	7978
2015-12-01T00:00:00Z	7979
2015-12-02T00:00:00Z	7979
2015-12-03T00:00:00Z	7980
2015-12-04T00:00:00Z	7979
2015-12-05T00:00:00Z	7980
2015-12-06T00:00:00Z	7980
2015-12-07T00:00:00Z	7981
2015-12-08T00:00:00Z	7982
2015-12-09T00:00:00Z	7981
2015-12-10T00:00:00Z	7982
2015-12-11T00:00:00Z	7982
2015-12-12T00:00:00Z	7983
2015-12-13T00:00:00Z	7983
2015-12-14T00:00:00Z	7984
2015-12-15T00:00:00Z	7983

Any updates on this?

@caligula1989 have you tried this with the latest version of InfluxDB?

I cannot repro this with the latest versions. The query engine was improved with InfluxDB 0.11 and seems to have solved the issue.


Incorrect documentation for SQS.Queue.receive_messages & SQS.Client.receive_message

The documentation for both SQS.Client.receive_message and SQS.Queue.receive_messages has an incorrect list of AttributeNames under Request Syntax
    AttributeNames=[
        'All'|'Policy'|'VisibilityTimeout'|'MaximumMessageSize'|'MessageRetentionPeriod'|'ApproximateNumberOfMessages'|'ApproximateNumberOfMessagesNotVisible'|'CreatedTimestamp'|'LastModifiedTimestamp'|'QueueArn'|'ApproximateNumberOfMessagesDelayed'|'DelaySeconds'|'ReceiveMessageWaitTimeSeconds'|'RedrivePolicy'|'FifoQueue'|'ContentBasedDeduplication'|'KmsMasterKeyId'|'KmsDataKeyReusePeriodSeconds',
    ],

The issue is that it lists attributes of a queue, rather than of a message.
In both cases, the corresponding description under Parameters > AttributeNames has the correct list with further details of each, and is described as "A list of s that need to be returned along with each message." (typo: /s/attributes), so it is just the Request Syntax that has the wrong list of attributes.
As far as I can tell this documentation is generated from data/sqs/2012-11-05/service-2.json in botocore, hence filing the issue here rather than on the boto3 repo:

ReceiveMessage has input shape based on ReceiveMessageRequest...
...whose AttributeNames have shape AttributeNameList which is based on QueueAttributeName

It seems that (instead of using AttributeNames > AttributeNameList > QueueAttributeName) the input shape for ReceiveMessageRequest needs to use something that ends up based on the list of attributes in MessageSystemAttributeName instead, which seems to be the correct list of message attributes.
As per CONTRIBUTING.rst I am filing this issue without attempting to fix it, given the service-2.json files are generated "upstream based on our internal knowledge of the AWS services".

It seems like you're correct here. The possible values listed in the operation documentation are correct, but the auto-generated values in the sample syntax are incorrect. As you've noted we pull in these values from the service model that's from the upstream service team. Unfortunately, this means that we won't be able to fix this directly in botocore. I've forwarded this issue along to the appropriate team.
Thanks for the report. Closing out the issue here as there's no actionable item for botocore.

Many thanks @joguSD, is there a link to track the upstream issue or is that all behind closed doors?


Implement finalization of the order in cart template

In the cart template, below the cart items, price and details must be shown.

The total price must be shown
The template must show unit price and total price of each item
The order button must be connected to the set_order view

The set_order view must then redirect the user to the order_list (orders index).

completed in #98


Introduce linter action

Summary
Fix all linter errors. Add linter GitHub action

Labels

Test Plan

Checklist

[ ] If required, I have updated the Plural documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] I have added a meaningful title and summary to convey the impact of this PR to a user.
[ ] I have added relevant labels to this PR to help with categorization for release notes.

I'd also enable more linter as by default there are only a couple of them used.
https://golangci-lint.run/usage/linters/
Most important I guess would be:

gofmt - formatter
goimports - formatter
gocyclo - checks the cyclomatic complexity of the code
importas - makes sure imports in all files use the same structure
goconst - finds repeated strings that could be a const
misspell - checks comments for common misspells
nestif - checks if there are too many nested ifs


Figlet support

http://www.figlet.org/

Limited options make it easier to use. So keep sensible defaults and fewer options, instead of full support.

How would it play along with normal text tool?


Fix failing build in PR for bumping node in techdocs

Build is failing for bumping library/node from 18.10.0-bullseye to 19.1.0-bullseye in /images/techdocs/context

ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref nkhbkl5rncnjgo0weainvotyd::ga2degctfbtxe0vpo6uyss9ri: failed to walk /tmp/buildkit-mount2019063959/usr/local/share/systemtap/tapset: lstat /tmp/buildkit-mount2019063959/usr/local/share/systemtap/tapset: no such file or directory

https://github.com/coopnorge/engineering-docker-images/actions/runs/3471310263/jobs/5800640310

node has been bumped, no need to fix


Add --altona flag to beacon cmds

Fixes https://github.com/ChainSafe/lodestar/issues/1163.
To start and configure a Beacon node now you just do
lodestar beacon run --altona

The flag will download the genesis file if not in disk, and apply the appropriate config values. This flag is at the beacon level, so it applies both to run and init.

@dapplion why did you close this?

@dapplion why did you close this?

Accidentally closed when migrating branches from my fork to the main repo


fix distribution builds

Issue #, if available:
#1617
Description of changes:
fix distribution builds
CheckList:

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Related issue https://github.com/opensearch-project/alerting/issues/1641


Order content type and management screens

Order management should rely on workflows, in order to allow rich configurability.
Orders are content items. The order part has the following properties:

string or enumeration Status (should this just be an enum field?)
Customer
Currency
Line items (see shopping cart for definition)

Nwazet had a tracking URL on the order part, but that should be handled somewhere else, likely shipping: some orders may be split into several packages, each with its tracking URL.
Nwazet had an activity text field. This doesn't need to be built into the part, and can instead be a standard text field.

Nwazet had a tracking URL on the order part, but that should be handled somewhere else, likely shipping: some orders may be split into several packages, each with its tracking URL.

I agree. If there were an interface to inject additional information into the order this would be an ideal implementation.
Another thing that came up, and I think workflows can take care of, is the fact that some orders may not immediately be valid, but may require some action from a back-office operator.

I will work on this.

What would be the best way for me to add an order to test with?  I was going to do it by going through the checkout process but got an error adding an item to the cart so I'm not sure if I just don't have everything setup right or if that process is not completed yet.  Would it be better if I created a recipe that creates some products and an order?

Please file a bug with a repro for the shopping cart problem, but that won't help you create an order anyway, since checkout is not done. Instead, just create an order from the admin, it's just a content type. I think there's a recipe for creating the order type. LMK if that's not the case, can't check right now, but I can check later. Shouldn't be too hard to create if that's not the case.

I created issue #73  for the shopping cart problem.  You are correct that there is a recipe for creating the order content type.  I am able to create an order from the admin but I don't see how I can add Items to the order.

You can't add items to the order as it is.


Clearer error messaging when token.uri.info is missing and retrieve_scopes() is called

When attempting to refresh the list of scopes available via credentials.retrieve_scopes() you receive the error AttributeError("'NoneType' object has no attribute 'find'",) if token.uri.info is not set on the credentials instance. It's not a very friendly error and it's unclear until you read the code that this value is not set when credentials are generated using flow_from_clientsecrets. When generating credentials from a OAuth2WebServerFlow this value is set to GOOGLE_TOKEN_INFO_URI.
I am not sure the thought process behind token.url.info being set for one flow type vs the other. In any case, better error handling would make the situation clearer.

Thank you for creating this issue, however, this project is deprecatedand we will only be addressing critical security issues. You can read moreabout this deprecation here.
If you need support or help using this library, we recommend that you ask yourquestion on StackOverflow.
If you still think this issue is relevant and should be addressed, pleasecomment and let us know!


more intelligent linking

Moved from https://github.com/lsegal/rubydoc.info/issues/40
tl; dr
References to rubydoc.info in other publications are usually about the subject of the docs, not the docs themselves. Since the docs improve from version to version, links within the current version should exclude the version number in the URL, providing the reader with the most up-to-date documentation for the given subject.
The problem
Documentation for any library improves over time, from version to version. If I go to http://rubydoc.info/gems/rspec-core/ and click on Hooks I end up at http://rubydoc.info/gems/rspec-core/2.7.1/RSpec/Core/Hooks. If you follow that link, you'll see, well, not much. Now go to http://rubydoc.info/github/rspec/rspec-core and click on Hooks, which takes you to http://rubydoc.info/github/rspec/rspec-core/master/RSpec/Core/Hooks. When I release 2.8, that's the documentation that will be linked to from http://rubydoc.info/gems/rspec-core/. It's not only better, it's actually existent!
The problem is that someone who is referencing these docs in a blog post or a tweet is very likely to grab the version-specific URL as/is and post it, which means it is soon linking to stale documentation.
Now one might argue that if you're referencing something you want to reference that thing, but I think that's only the case when the documentation itself is the subject of the blog or tweet. Most of the time it's the thing being documented that is the subject, in which case something is lost by linking to the version-specific doc.
Proposed solution
rubydoc.info renders the current version if there is no version in the URL. Since it knows which version is the current version, it could use that knowledge to render links differently for the current version than older versions - without the version number in the URL.
Older versions would render version-specific links. For that to work, when a new version is published, the docs for the previous version would have to be re-generated. Additional overhead, yes, but it seems like a fair tradeoff to me.
Ideally, we'd also account for links that point to docs that existed in older versions but no longer exist in the current version. In that case, we'd either need to redirect to front of the current version with some flash-like message explaining what happened, or index the old versions and find the most recent version that still has that content.

Another way to do this would be to add a "version_supplied" switch in the request or router object that is set when parsing the URL for routing (in router.rb). Then we just check that flag when generating the base prefix. Seems like the easiest way to do it.

Why not just make the links relative?

I forgot why we did it this way, so I tried.. now I remember: relative links are hell when URLs can be accessible with or without the ending '/'. The relative path of 'x/y' from 'docs/yard' would be 'docs/x/y', but from 'docs/yard/' it would be correct. I don't want to make the server finicky about whether users put an ending / in their URLs.

And deployed on rubydoc.info to test before release :)

Oh just FYI, some older generated projects might already have cached html that use /version/ links-- I'm not going to reset the cache just yet. Just letting you know in case you come across one of those projects and go "omg it's not working!" :) Try this on an unprocessed gem or github if you want to test.

Hey @lsegal - sorry I didn't follow up on this back in December. Looking at http://rubydoc.info/gems/rspec-core/, the links to README, Changlog, and License all have 2.8.0 in them. I'm guessing that rubydoc.info just doesn't have this change. Is that correct?