test dev code

Dear @cedrozor ,
Need one help!
I want to test the code during development from a Windows 10 system.
Currently I am writing my code, building the code and installer; then installing and testing in a a cloud server, Win Server 2019.
Is there any way I can do this with local service and local IIS all running in sync from my Windows 10 Laptop.
Regards,
Sabyasachi Basu

Hi @sabyasachi-basu-i,
Of course yes! it's always better to dev and test locally. You can enable IIS, .NET and all Myrtille pre-requisites on your Win10 laptop (see documentation) or run the following powershell commands: (Myrtille.Web.Install.ps1)
# enable IIS, WebSocket protocol, management console and tools
Enable-WindowsOptionalFeature -NoRestart -Online -FeatureName IIS-WebServerRole, IIS-WebServer, IIS-CommonHttpFeatures, IIS-DefaultDocument, IIS-DirectoryBrowsing, IIS-HttpErrors, IIS-StaticContent, IIS-HttpRedirect, IIS-HealthAndDiagnostics, IIS-HttpLogging, IIS-Performance, IIS-HttpCompressionStatic, IIS-Security, IIS-RequestFiltering, IIS-ApplicationDevelopment, IIS-ISAPIExtensions, IIS-ISAPIFilter, IIS-WebSockets, IIS-ManagementConsole, IIS-ManagementScriptingTools
Write-Output "Enabled IIS, WebSocket protocol, management console and tools`r`n"

# enable .NET extensibility
Enable-WindowsOptionalFeature -NoRestart -Online -FeatureName IIS-NetFxExtensibility45, IIS-ASPNET45 -All
Write-Output "Enabled .NET extensibility`r`n"

# enable WCF/HTTP activation
Enable-WindowsOptionalFeature -NoRestart -Online -FeatureName WAS-WindowsActivationService, WAS-ProcessModel, WAS-ConfigurationAPI, WCF-HTTP-Activation45
Write-Output "Enabled WCF/HTTP activation`r`n"

Then you need Visual Studio on your laptop (enable C# and also C++ if you want to build the FreeRDP fork) and voila :)
You can use your Server 2019 cloud server as target host.

Hi @sabyasachi-basu-i,
Of course yes! it's always better to dev and test locally. You can enable IIS, .NET and all Myrtille pre-requisites on your Win10 laptop (see documentation) or run the following powershell commands: (Myrtille.Web.Install.ps1)
# enable IIS, WebSocket protocol, management console and tools
Enable-WindowsOptionalFeature -NoRestart -Online -FeatureName IIS-WebServerRole, IIS-WebServer, IIS-CommonHttpFeatures, IIS-DefaultDocument, IIS-DirectoryBrowsing, IIS-HttpErrors, IIS-StaticContent, IIS-HttpRedirect, IIS-HealthAndDiagnostics, IIS-HttpLogging, IIS-Performance, IIS-HttpCompressionStatic, IIS-Security, IIS-RequestFiltering, IIS-ApplicationDevelopment, IIS-ISAPIExtensions, IIS-ISAPIFilter, IIS-WebSockets, IIS-ManagementConsole, IIS-ManagementScriptingTools
Write-Output "Enabled IIS, WebSocket protocol, management console and tools`r`n"

# enable .NET extensibility
Enable-WindowsOptionalFeature -NoRestart -Online -FeatureName IIS-NetFxExtensibility45, IIS-ASPNET45 -All
Write-Output "Enabled .NET extensibility`r`n"

# enable WCF/HTTP activation
Enable-WindowsOptionalFeature -NoRestart -Online -FeatureName WAS-WindowsActivationService, WAS-ProcessModel, WAS-ConfigurationAPI, WCF-HTTP-Activation45
Write-Output "Enabled WCF/HTTP activation`r`n"

Then you need Visual Studio on your laptop (enable C# and also C++ if you want to build the FreeRDP fork) and voila :)
You can use your Server 2019 cloud server as target host.

@cedrozor , But, if I change my service and FreeRDP fork code(winpr and additional dependencies) I still need to separately build and deploy in the Win Server. right ? to reflect when I an connecting to the target host.

@cedrozor , But, if I change my service and FreeRDP fork code(winpr and additional dependencies) I still need to separately build and deploy in the Win Server. right ? to reflect when I an connecting to the target host.

No, you don't have to deploy FreeRDP on the Server 2019. It will run as RDP client on your Win10 laptop, along with Myrtille. You need to clone it into the Myrtille.RDP\FreeRDP folder and build it from there, after running cmake. I had this described in the documentation and you can find many closed issues about it.

No, you don't have to deploy FreeRDP on the Server 2019. It will run as RDP client on your Win10 laptop, along with Myrtille. You need to clone it into the Myrtille.RDP\FreeRDP folder and build it from there, after running cmake. I had this described in the documentation and you can find many closed issues about it.

Got it. thank you!!

Got it. thank you!!


Option to expose the slave listener port as host port

What this PR does / why we need it:
Add the option to expose jenkins' slave listener as host port.
Which issue this PR fixes

fixes #11186

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md

/approve
/lgtm


Fix on error if bot is in 0 guilds.

reduce will throw an error if client.guilds.cache is empty.
Also fixed "my prefix is" line

Also apparently info command fix is here too.


Add a readme note for building on macOS

I know it's mentioned at the top of the readme but I think it doesn't hurt to add the additional flag required for macOS in the instructions.

Ah, good point. I just invoked via the toolchain dir directly. This is a better solution (which was non-obvious to me).

Thanks! I don't use macOS, so any further help is appreciated. 😃


dovecot: <IP_ADDRESS> -> 2.2.34

Release notes: https://dovecot.org/list/dovecot-news/2018-February/000370.html
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

/cc @andir

Success on x86_64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/lib/dovecot/lib21_fts_lucene_plugin.so
shrinking /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/lib/dovecot/lib20_mailbox_alias_plugin.so
shrinking /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/lib/dovecot/lib10_last_login_plugin.so
gzipping man pages under /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/share/man/
stripping (with flags -S) in /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/lib  /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/libexec  /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/bin  /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/sbin
patching script interpreter paths in /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34
/nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/jgw8hxx7wzkyhb2dr9hwsd9h2caaasdc-bash-4.4-p12/bin/sh"
checking for references to /tmp/nix-build-dovecot-2.2.34.drv-0 in /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34...
moving /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/sbin/* to /nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34/bin
/nix/store/wzz09ad2smr8whc4n452a2jjwh3fsk6p-dovecot-2.2.34

Success on aarch64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/lib/dovecot/lib11_imap_quota_plugin.so
shrinking /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/bin/doveadm
shrinking /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/bin/doveconf
gzipping man pages under /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/share/man/
stripping (with flags -S) in /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/lib  /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/libexec  /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/bin  /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/sbin
patching script interpreter paths in /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34
/nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/7b94zsymp6g5rzp2pgwhpkw0idjg52x5-bash-4.4-p12/bin/sh"
checking for references to /build in /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34...
moving /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/sbin/* to /nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34/bin
/nix/store/lrrim81ab87i3n86nalmcyk7r8agj7az-dovecot-2.2.34


Document option to allow almost all printable ASCII characters in environment variables

/retitle [WIP]: Allow almost all printable ASCII characters in environment variables

/sig node

/retitle [WIP]: Document option to allow almost all printable ASCII characters in environment variables

@HirazawaUi I think that the target branch should be 1.30 instead of main, since this is related to a feature with a 1.30 milestone - would it be possible to change it?

@HirazawaUi I think that the target branch should be 1.30 instead of main, since this is related to a feature with a 1.30 milestone - would it be possible to change it?

Ah, sorry, that was my mistake.

@sftim May you take a look at this document first? Because you have the full context and understand it well enough.

Hello @HirazawaUi 👋 please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before Tuesday March 12th 2024 18:00 PST. Thank you!

The current PR is already in Ready for review, I did not create the draft PR initially.....

Hey @HirazawaUi! Just a reminder that this needs to be reviewed and ready to merge by Docs Freeze on March 26, including a technical review. An exception request will be required if you cannot meet that deadline. Thank you!
cc: @thockin @mrunalp

Please also amend https://kubernetes.io/docs/concepts/configuration/configmap/, https://kubernetes.io/docs/concepts/configuration/secret/

Maybe we shouldn't modify the documentation of configmap and secret. Our changes only apply to using envFrom to reference the keys of configmap and secret as environment variable names. Before, they did not mention this part.
For me, describing this feature in environment-variable-expose-pod-information and define-environment-variable-container together is a duplication, in define-environment-variable-container we describe how to define an environment variable, and in environment-variable-expose-pod-information we describe how to expose pod information through environment variables, and based on semantics I prefer the current way(describe in define-environment-variable-container)
WDYT?

@sftim  Do you have any follow-up thoughts?

Maybe we shouldn't modify the documentation of configmap and secret. Our changes only apply to using envFrom to reference the keys of configmap and secret as environment variable names. Before, they did not mention this part.
For me, describing this feature in environment-variable-expose-pod-information and define-environment-variable-container together is a duplication, in define-environment-variable-container we describe how to define an environment variable, and in environment-variable-expose-pod-information we describe how to expose pod information through environment variables, and based on semantics I prefer the current way(describe in define-environment-variable-container)

In technical writing, we like to overcommunicate some details. We can't make assumptions about which page readers are looking at to learn about a thing, or which pages they have already read.
I recommend adding a heading and brief text into https://kubernetes.io/docs/concepts/configuration/configmap/#using-configmaps that mentions environment variables, and that links to one of the pages where this new feature is mentioned.
You may well only write 100 extra words.
I recommend adding a mention to https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables that the name of the environment variable is subject to restrictions. The way I'd do it, “restrictions” would be a hyperlink to some other page, eg to an updated version of content/en/docs/tasks/inject-data-application/define-environment-variable-container.md

@sftim May you take a look at this document first? Because you have the full context and understand it well enough.

I'm afraid we can't offer you that luxury; our SIG leads' time is popular. Please work with reviewers for English, me included, but not me exclusively. I wouldn't have the capacity to pick up all the reviews I've been invited to help out with ahead of the docs deadline.

@sftim May you take a look at this document first? Because you have the full context and understand it well enough.

I'm afraid we can't offer you that luxury; our SIG leads' time is popular. Please work with reviewers for English, me included, but not me exclusively. I wouldn't have the capacity to pick up all the reviews I've been invited to help out with ahead of the docs deadline.

Okay, I understand. Thank you for your patient comment. I will communicate directly with the English reviewer later.

/milestone 1.30

Hello @liggitt @thockin a reminder that this needs a technical review before being ready to merge by Docs Freeze on March 26. An exception request will be required if you cannot meet that deadline.
@kubernetes/sig-docs-en-reviews I see there is request for additional review (see comments above). Might someone other than Tim be available to assist review on this one?
Thanks everyone!

Before reviewing by sig-docs-en-reviews, I would like to hear @thockin opinion on comment. We are allowed to relax the restrictions on using EnvFrom to reference configmap/secret keys when feature gate is not enable.

Hello @HirazawaUi  :wave:! I'm reaching out from the Docs team. Just checking in as we approach Docs Freeze on Tuesday March 26th 18:00 PDT. This documentation appears to still be under review. To meet the Docs Freeze, this PR must have a technical review as well as lgtm and approve labels applied, without any unaddressed comments or concerns from SIG Docs. The status of this enhancement is marked as at risk for docs freeze. Thank you!

Hello @HirazawaUi 👋! I'm reaching out from the Docs team. Just checking in as we approach Docs Freeze on Tuesday March 26th 18:00 PDT. This documentation appears to still be under review. To meet the Docs Freeze, this PR must have a technical review as well as lgtm and approve labels applied, without any unaddressed comments or concerns from SIG Docs. The status of this enhancement is marked as at risk for docs freeze. Thank you!

OK, I'm also waiting for the technical review. If @thockin doesn't have any free time, I will try to ask other approvers of sig-node for help.

@kubernetes/sig-node-leads Howdy leads, we're looking for a technical review for these docs to make the Docs Freeze deadline of March 26 – thanks!

Just a heads up that Doc Freeze is approaching fast tomorrow March 26th at 18:00 PDT and starting this cycle, we need to file an Exception after this time.
This one looks pretty close and we'd like to help you get it over the finish line!
@kubernetes/sig-docs-en-owners Does this one look good to you? @HirazawaU I notice we have approvals from SIG Node leads and also there are a couple outstanding comments from Tim H. Do we want to tackle those before merge?

Just a heads up that Doc Freeze is approaching fast tomorrow March 26th at 18:00 PDT and starting this cycle, we need to file an Exception after this time. This one looks pretty close and we'd like to help you get it over the finish line!
@kubernetes/sig-docs-en-owners Does this one look good to you? We could use an approve @HirazawaUi I notice we have approvals from SIG Node leads and also there are a couple outstanding comments from Tim H. Do we want to tackle those before merge?

All comments have been fixed.
@kubernetes/sig-docs-en-owners for lgtm and approve

/lgtm

/milestone 1.30


Chrome compat and other browsers ?

Mozilla has a polyfill:
https://github.com/mozilla/webextension-polyfill
basically s/browser/chrome/g and a couple of other renaming.
Chrome also does not Support SVG icons... so maybe a build tools that generate 2 extensions for Chrome/Firefox ?

Publishing on chrome webstore requires a $5 fee.

Extra things needed for chrome:

Edit your contact email address
Edit your physical address (optional, unless consumer protection laws enforce it, so should be ok if free)
Add a link to your privacy policy
check we own a website associated w/ app. (maybe MyBinder.org ?)


More tests

Codecov Report

Merging #14 (332977a) into main (93098ed) will increase coverage by 4.80%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #14      +/-   ##
==========================================
+ Coverage   69.41%   74.22%   +4.80%
==========================================
  Files           6        6
  Lines         448      450       +2
==========================================
+ Hits          311      334      +23
+ Misses        137      116      -21

Impacted Files
Coverage Δ

...SotoCognitoAuthenticationKit/Authenticatable.swift
68.27% <100.00%> (+8.75%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93098ed...332977a. Read the comment docs.


bug: usage and finish_reason with OpenAI compatible API support

Currently, this is not yet supported due to fragmented implementation between vllm and pytorch.
We will need to clean up some general Python SDK and API for LLM implementation.

This is now implemented.


⚠ Secret Website 2 has degraded performance

In 00c1bb4, Secret Website 2 ($SECRET_WEBSITE_2) experienced degraded performance:

HTTP code: 200
Response time: 6278 ms

Resolved: Secret Website 2 performance has improved in a7c7c26.


Add docs for Asgadeo IDP integration

Description:
Add docs for Asgadeo IDP integration.

Added docs to APK wiki
https://github.com/wso2/apk/wiki/Using-Asgardeo-as-an-IdP


First commit of squeeze break

This is the squeeze break, please check this code
Thanks in advance

 Volume - Squeeze Break


Intermittent 503 Errors Due to upstream immediate_connect_error with CiliumEnvoyConfig for Internal Traffic

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
Description
I have noticed that when I configure my Internal Traffic ServiceMesh using EnvoyConfig, a portion of requests result in 503 errors after passing through Envoy. My upstream services receive approximately 80,000 to 100,000 requests per hour, and around 1% of these requests encounter a 503 error. However, when I don't have the ServiceMesh enabled, no 503 errors occur. The error message received for the 503 errors is as follows:
X-Envoy-Flags: UF,URX
X-Envoy-Details: upstream_reset_before_response_started{connection_failure,immediate_connect_error:_Cannot_assign_requested_address}
Troubleshooting
Regarding Envoy Configuration Attempts
I have tried adjusting the idleTimeout, maxConnections, maxRequestsPerConnection, maxConnectionDuration, and other parameters in Envoy, but it did not improve the situation. Additionally, I attempted to create a separate Envoy instance without using the Envoy within the Cilium Agent. I configured this standalone Envoy with the same parameters, except I changed the Cluster configuration to use LOGICAL_DNS instead of EDS and directly specified the Service or Pod. Interestingly, there were no issues encountered with this setup.
Envoy Log
📌Cannot assign requested address
After examining the Envoy log, I noticed several concerning points. The 503 error occurs after a failed connection creation, likely due to the "Cannot assign requested address" error. Here is the relevant log:
[2023-05-22 05:38:08.028][1340][debug][pool] trying to create new connection
[2023-05-22 05:38:08.028][1340][debug][pool] creating a new connection (connecting=0)
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] current connecting state: true
[2023-05-22 05:38:08.028][1340][debug][client] [C1937] connecting
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] connecting to <IP_ADDRESS>:8523
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] immediate connect error: Cannot assign requested address

The Cannot assign requested address error typically occurs when there is an issue with the IP address or when the OS's port resources are exhausted. However, it is strange because:

<IP_ADDRESS>:8523 is the correct upstream IP.
When checking the port utilization on the current node, it does not exceed the ip_local_port_range.

📌Regarding the Pool Information in the Log
In the log, I noticed information related to the pool, and it seems that the number of connections in the pool is lower than expected for a service with an average of 20-30 requests per second. However, I might be misunderstanding the information in this log entry. Here is the log:
[2023-05-22 05:38:08.028][1340][trace][pool] ConnPoolImplBase 0x1620bdfeeb40, ready_clients_.size(): 0, busy_clients_.size(): 0, connecting_clients_.size(): 0, connecting_stream_capacity_: 0, num_active_streams_: 0, pending_streams_.size(): 1 per upstream preconnect ratio: 1

Additional Findings:

Based on the service traffic patterns, it seems that the occurrence of 503 errors is not necessarily related to the current traffic volume.

Enabling retries can significantly reduce the frequency of 503 errors, but it cannot completely eliminate them. The occurrence rate decreases from around 1% to 0.01%.

Increasing the number of service pods also lowers the probability of 503 errors. For example, increasing from 2 to 4 pods reduces the occurrence. However, when the ServiceMesh is not enabled, even with only 2 pods, no 503 errors are observed.

When conducting load testing with a tool, specifically targeting the service with EnvoyConfig, there are no 503 errors observed when using HTTP GET requests for the healthz API, even with a rate of 100 requests per second ( The /healthz API, which only returns "True" with an average response time of <5ms), shows no issues. However, in actual usage, when using POST requests, the 503 errors occur. The average response time for the POST API is around 40-50ms.

I initially thought that this issue was similar to Issue #23906, as mentioned in the discussion below, stating that it would no longer occur in version 1.14. Therefore, I attempted to upgrade to version 1.14.0-snapshot.2 , but the problem still persists.

Cilium Version
Client: 1.13.2 8cb94c70 2023-04-17T23:19:21+02:00 go version go1.19.8 linux/amd64
Daemon: 1.13.2 8cb94c70 2023-04-17T23:19:21+02:00 go version go1.19.8 linux/amd64
and
Client: 1.14.0-snapshot.2 5d3899fb 2023-04-28T13:23:52-07:00 go version go1.20.3 linux/amd64
Daemon: 1.14.0-snapshot.2 5d3899fb 2023-04-28T13:23:52-07:00 go version go1.20.3 linux/amd64
Kernel Version
Linux 5.15.98-flatcar #1 SMP Thu Apr 13 03:16:20 -00 2023 x86_64 Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz GenuineIntel GNU/Linux
Kubernetes Version
Server Version: version.Info{Major:"1", Minor:"24",  GitVersion:"v1.24.10",  GitCommit:"5c1d2d4295f9b4eb12bfbf6429fdf989f2ca8a02", GitTreeState:"clean",  BuildDate:"2023-01-18T19:08:10Z",  GoVersion:"go1.19.5", Compiler:"gc",  Platform:"linux/amd64"}
Sysdump
No response
Relevant log output
[2023-05-22 05:38:08.027][1340][trace][http] [C1913] onHeadersCompleteBase
[2023-05-22 05:38:08.027][1340][trace][http] [C1913] completed header: key=X-Source-Host value=hostname:80
[2023-05-22 05:38:08.027][1340][trace][http] [C1913] Server: onHeadersComplete size=23
[2023-05-22 05:38:08.027][1340][debug][http] [C1913][S13076914254576433341] request headers complete (end_stream=false):
':authority', 'collector.namespace:80'
':path', '/api'
':method', 'POST'
'content-length', '358'
'accept', 'application/json'
'accept-encoding', 'gzip'
'authorization', '****'
'content-type', 'application/json'
'x-forwarded-port', '80'
'x-forwarded-proto', 'http'
'x-forwarded-scheme', 'http'
'x-k8s-in', '1684733888.026'
'x-original-forwarded-for', '***'
'x-real-ip', '***'
'x-request-id', '***'
'x-scheme', 'http'
'x-source-host', 'hostname:80'

[2023-05-22 05:38:08.027][1340][debug][connection] [C1913] current connecting state: false
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode headers called: filter=cilium.l7policy status=0
[2023-05-22 05:38:08.028][1340][debug][router] [C1913][S13076914254576433341] cluster 'namespace/collector' match for URL '/api'
[2023-05-22 05:38:08.028][1340][trace][filter] cilium.network: in upstream callback
[2023-05-22 05:38:08.028][1340][trace][filter] cilium.ipcache: Looking up key: *, prefixlen: 32
[2023-05-22 05:38:08.028][1340][debug][filter] cilium.ipcache: <IP_ADDRESS> has ID 13852
[2023-05-22 05:38:08.028][1340][debug][router] [C1913][S13076914254576433341] router decoding headers:
':authority', 'collector.namespace:80'
':path', '/api'
':method', 'POST'
':scheme', 'http'
'content-length', '358'
'accept', 'application/json'
'accept-encoding', 'gzip'
'authorization', '***'
'content-type', 'application/json'
'request-id', '***'
'x-forwarded-port', '80'
'x-forwarded-proto', 'http'
'x-forwarded-scheme', 'http'
'x-k8s-in', '1684733888.026'
'x-original-forwarded-for', '***'
'x-real-ip', '***'

'x-request-id', '***'
'x-scheme', 'http'
'x-source-host', 'hostname:80'
'x-envoy-expected-rq-timeout-ms', '2000'
'x-flow', 'mesh-weight'

[2023-05-22 05:38:08.028][1340][debug][pool] queueing stream due to no available connections (ready=0 busy=0 connecting=0)
[2023-05-22 05:38:08.028][1340][debug][pool] trying to create new connection
[2023-05-22 05:38:08.028][1340][trace][pool] ConnPoolImplBase 0x1620bdfeeb40, ready_clients_.size(): 0, busy_clients_.size(): 0, connecting_clients_.size(): 0, connecting_stream_capacity_: 0, num_active_streams_: 0, pending_streams_.size(): 1 per upstream preconnect ratio: 1

[2023-05-22 05:38:08.028][1340][debug][pool] creating a new connection (connecting=0)
[2023-05-22 05:38:08.028][1340][trace][filter] Set socket (534) option SO_MARK to 3e20900 (magic mark: 900, id: 994, cluster: 0), src: <IP_ADDRESS>:58924
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] current connecting state: true
[2023-05-22 05:38:08.028][1340][debug][client] [C1937] connecting
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] connecting to <IP_ADDRESS>:8523
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] immediate connect error: Cannot assign requested address
[2023-05-22 05:38:08.028][1340][trace][pool] not creating a new connection, shouldCreateNewConnection returned false.
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode headers called: filter=envoy.filters.http.upstream_codec status=4
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode headers called: filter=envoy.filters.http.router status=1
[2023-05-22 05:38:08.028][1340][trace][misc] enableTimer called on 0x1620be35bf10 for 300000ms, min is 300000ms
[2023-05-22 05:38:08.028][1340][trace][http] [C1913] message complete
[2023-05-22 05:38:08.028][1340][trace][http] [C1913] body size=358
[2023-05-22 05:38:08.028][1340][trace][misc] enableTimer called on 0x1620be35bf10 for 300000ms, min is 300000ms
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode data called: filter=cilium.l7policy status=0
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode data called: filter=envoy.filters.http.router status=3
[2023-05-22 05:38:08.028][1340][debug][http] [C1913][S13076914254576433341] request end stream
[2023-05-22 05:38:08.028][1340][trace][misc] enableTimer called on 0x1620be35bf10 for 300000ms, min is 300000ms
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode data called: filter=cilium.l7policy status=0
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] decode data called: filter=envoy.filters.http.router status=3
[2023-05-22 05:38:08.028][1340][trace][http] [C1913] parsed 1235 bytes
[2023-05-22 05:38:08.028][1340][trace][connection] [C1913] socket event: 2
[2023-05-22 05:38:08.028][1340][trace][connection] [C1913] write ready
[2023-05-22 05:38:08.028][1340][trace][connection] [C1937] socket event: 3
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] raising immediate error
[2023-05-22 05:38:08.028][1340][debug][connection] [C1937] closing socket: 0
[2023-05-22 05:38:08.028][1340][trace][connection] [C1937] raising connection event 0
[2023-05-22 05:38:08.028][1340][debug][client] [C1937] disconnect. resetting 0 pending requests
[2023-05-22 05:38:08.028][1340][debug][pool] [C1937] client disconnected, failure reason: immediate connect error: Cannot assign requested address
[2023-05-22 05:38:08.028][1340][debug][router] [C1913][S13076914254576433341] upstream reset: reset reason: connection failure, transport failure reason: immediate connect error: Cannot assign requested address
[2023-05-22 05:38:08.028][1340][trace][main] item added to deferred deletion list (size=1)
[2023-05-22 05:38:08.028][1340][trace][main] item added to deferred deletion list (size=2)
[2023-05-22 05:38:08.028][1340][debug][http] [C1913][S13076914254576433341] Sending local reply with details upstream_reset_before_response_started{connection_failure,immediate_connect_error:_Cannot_assign_requested_address}
[2023-05-22 05:38:08.028][1340][trace][misc] enableTimer called on 0x1620be35bf10 for 300000ms, min is 300000ms

[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] encode headers called: filter=cilium.l7policy status=0

[2023-05-22 05:38:08.028][1340][debug][http] [C1913][S13076914254576433341] encoding headers via codec (end_stream=false):
':status', '503'
'x-flow', 'mesh-weight'
'x-envoy-downstream', '<IP_ADDRESS>:0'
'x-envoy-upstream', '<IP_ADDRESS>:8523'
'x-envoy-details', 'upstream_reset_before_response_started{connection_failure,immediate_connect_error:_Cannot_assign_requested_address}'
'x-envoy-flags', 'UF,URX'
'x-envoy-down-local', '<IP_ADDRESS>:80'
'content-length', '175'
'content-type', 'text/plain'
'date', 'Mon, 22 May 2023 05:38:07 GMT'
'server', 'envoy'

[2023-05-22 05:38:08.028][1340][trace][connection] [C1913] writing 425 bytes, end_stream false
[2023-05-22 05:38:08.028][1340][trace][misc] enableTimer called on 0x1620be35bf10 for 300000ms, min is 300000ms
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] encode data called: filter=cilium.l7policy status=0
[2023-05-22 05:38:08.028][1340][trace][http] [C1913][S13076914254576433341] encoding data via codec (size=175 end_stream=true)
[2023-05-22 05:38:08.028][1340][trace][connection] [C1913] writing 175 bytes, end_stream false

Anything else?
CiliumEnvoyConfig
apiVersion: cilium.io/v2
kind: CiliumEnvoyConfig
metadata:
  name: collector
  namespace: namespace
spec:
  backendServices:
    - name: collector
      namespace: namespace
    - name: collector-baseline
      namespace: namespace
  resources:
    - '@type': type.googleapis.com/envoy.config.listener.v3.Listener
      filterChains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typedConfig:
                '@type': >-
                  type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                httpFilters:
                  - name: envoy.filters.http.router
                    typed_config:
                      '@type': >-
                        type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

                rds:
                  routeConfigName: namespace/collector
                statPrefix: namespace/collector
      name: namespace/collector
    - '@type': type.googleapis.com/envoy.config.route.v3.RouteConfiguration
      name: namespace/collector
      virtualHosts:
        - domains:
            - '*'
          name: namespace/collector
          routes:
            - match:
                headers:
                  - name: test
                    stringMatch:
                      exact: 'true'
                prefix: /
              route:
                cluster: namespace/collector-baseline
            - match:
                prefix: /
              requestHeadersToAdd:
                - header:
                    key: x-flow
                    value: mesh-weight
              responseHeadersToAdd:
                - header:
                    key: x-flow
                    value: mesh-weight
                - header:
                    key: x-envoy-downstream
                    value: '%DOWNSTREAM_REMOTE_ADDRESS%'
                - header:
                    key: x-envoy-upstream
                    value: '%UPSTREAM_REMOTE_ADDRESS%'
                - header:
                    key: x-envoy-details
                    value: '%RESPONSE_CODE_DETAILS%'
                - header:
                    key: x-envoy-flags
                    value: '%RESPONSE_FLAGS%'
                - header:
                    key: x-envoy-down-local
                    value: '%DOWNSTREAM_LOCAL_ADDRESS%'
              route:
                retryPolicy:
                  numRetries: 3
                  perTryTimeout: 2s
                  retryOn: reset
                weightedClusters:
                  clusters:
                    - name: namespace/collector
                      weight: 100
                    # Originally, I had configured multiple clusters, but for the sake of simplifying the testing process, I switched to using only one cluster.
    - '@type': type.googleapis.com/envoy.config.cluster.v3.Cluster
      circuitBreakers:
        thresholds:
          - maxConnections: 2048
            maxPendingRequests: 2048
            maxRequests: 2048
            maxRetries: 3
            priority: DEFAULT
      commonLbConfig:
        healthyPanicThreshold: {}
      connectTimeout: 6s
      dns_lookup_family: V4_ONLY
      name: namespace/collector
      outlierDetection:
        consecutiveLocalOriginFailure: 2
        splitExternalLocalOriginErrors: true
      type: EDS
    - '@type': type.googleapis.com/envoy.config.cluster.v3.Cluster
      commonLbConfig:
        healthyPanicThreshold: {}
      connectTimeout: 3s
      name: namespace/collector-baseline
      outlierDetection:
        consecutiveLocalOriginFailure: 2
        splitExternalLocalOriginErrors: true
      type: EDS
  services:
    - listener: ''
      name: collector
      namespace: namespace

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for the detailed report. One thing you could try out is to add a SO_LINGER socket option to the Envoy upstream cluster in your CiliumEnvoyConfig. Turning linger on and specifying a zero timeout should allow the kernel to immediately close the socket when the (cilium-envoy) process closes the socket, hence making the same 5-tuple available for reuse right after. This is based on the premise that the 503 are caused by bind errors due to socket with the same 5-tuple already existing in in the host networking namespace due to the source pod(s) cycling through ephemeral source ports in a fast pace.
Setting zero timeout on enabled SO_LINGER may/will RST connections rather than doing the normal TCP close exchange. As Upstream HTTP sockets have had the complete response read, it may be safe to assume that the request was also already delivered. There may be some streaming scenarios where this is not the case. If zero timeout is problematic, you may experiment with other small timeout values, like 1 or 2 seconds, basically find the largest timeout value that still resolves the issue with 503's.
Another thing you can try is to slow down the pace at which the source test pods start new connections to see if that affects the percentage share of 503 responses. Depending on these results you may determine if the 503's are likely to happen in realistic rather than test workloads.
The yaml snippet to add to the Envoy Cluster configuration looks like this:
    upstream_bind_config:
      source_address:
        address: "<IP_ADDRESS>"
        port_value: 0
      socket_options:
      - description: "Zero SO_LINGER"
        state: STATE_PREBIND
        level: 1 # SOL_SOCKET
        name: 13 # SO_LINGER
        buf_value: "0x01000000000000000000000000000000" # linger ON, 0 timeout -> RST connection immediately on close

It would be great if you could report back here in how this works out for you :-)

Hi @jrajahalme , thank you for your response and the suggested solution.  I appreciate your detailed explanation. However, I have tried configuring the SO_LINGER socket option as you suggested in my CiliumEnvoyConfig, but it did not resolve my issue.
I set the linger option to have a zero timeout as you mentioned, hoping that it would allow the kernel to immediately close the socket when the cilium-envoy process closes it. Unfortunately, I am still experiencing the 503 errors.
However, I noticed something strange when configuring SO_LINGER, and I'm not sure if it has any impact on resolving the issue. I found that when I configure buf_value: "0x01000000000000000000000000000000" in CiliumEnvoyConfig, it appears differently when I retrieve the Envoy Admin dump config:

if I add two more zeros at the end, buf_value: "0x0100000000000000000000000000000000", the strange characters do not appear when I dump the Envoy config:

I'm not sure if this transformation affects the behavior of the SO_LINGER option.
Thank you again for your assistance.

I'm currently dealing with a 503 error problem. Disabling the Envoy between the services resolves the issue, as no problems arise in that scenario. Surprisingly, the error persists even when a service calls another service through the internal API Gateway, there use Envoy.
Nonetheless, I've taken the step of exposing the gateway via BGP to an external host using FRR. Additionally, I've directed the services to consume resources from this external host, resulting in the absence of any observed 503 errors. This leads me to believe that the error is confined to the internal environment. It's possible that upon exiting the cluster, the system is following an alternative path.

I'm also dealing with this issue using 1.14.1 on NixOS 23.11 unstable.

Adding myself to the list, same issue using ingress on 1.14.3 tried both with embedded envoy and daemonset

I got the same error.

Same on 1.15.6
upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: immediate connect error: Cannot assign requested addressMon Jun 17 07:00:29 UTC 2024
with
kind: CiliumClusterwideEnvoyConfig
metadata:
  name: envoy-circuit-breaker
spec:
  services:
    - name: cilium-target
      namespace: default
  resources:
    - "@type": type.googleapis.com/envoy.config.listener.v3.Listener
      name: envoy-lb-listener
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: envoy-lb-listener
                rds:
                  route_config_name: lb_route
                use_remote_address: true
                skip_xff_append: true
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
    - "@type": type.googleapis.com/envoy.config.route.v3.RouteConfiguration
      name: lb_route
      virtual_hosts:
        - name: "lb_route"
          domains: [ "*" ]
          routes:
            - match:
                prefix: "/"
              route:
                timeout: 5s
                weighted_clusters:
                  clusters:
                    - name: "default/cilium-target"
                      weight: 100
    - "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster
      name: "default/cilium-target"
      connect_timeout: 5s
      lb_policy: ROUND_ROBIN
      type: EDS
      outlier_detection:
        split_external_local_origin_errors: true
        consecutive_local_origin_failure: 2

actually for me it might be ipv6 issues - i added this now       dns_lookup_family: V4_ONLY
will track

We are using TLS Inspection and envoy returns 503 error (timeout). Turning off bpf masquerade works.
Talos 1.8.1 with this cilium config:
bpf:
  masquerade: false
cgroup:
  autoMount:
    enabled: false
  hostRoot: /sys/fs/cgroup
hubble:
  relay:
    enabled: true
  ui:
    enabled: true
  tls:
    auto:
      enabled: true
      method: helm
ipam:
  mode: kubernetes
k8sServiceHost: localhost
k8sServicePort: 7445
kubeProxyReplacement: true
kubeProxyReplacementHealthzBindAddr: <IP_ADDRESS>:10256
l2announcements:
  enabled: true
  leaseDuration: 3s
  leaseRenewDeadline: 1s
  leaseRetryPeriod: 200ms
loadBalancer:
  mode: dsr
  dsrDispatch: geneve
operator:
  replicas: 3
  prometheus:
    enabled: true
prometheus:
  enabled: true
rollOutCiliumPods: true
securityContext:
  capabilities:
    ciliumAgent:
    - CHOWN
    - KILL
    - NET_ADMIN
    - NET_RAW
    - IPC_LOCK
    - SYS_ADMIN
    - SYS_RESOURCE
    - DAC_OVERRIDE
    - FOWNER
    - SETGID
    - SETUID
    cleanCiliumState:
    - NET_ADMIN
    - SYS_ADMIN
    - SYS_RESOURCE
tunnelProtocol: geneve

Same on 1.16.3.
It's definitely a 5-tuple collision and not ephemeral port exhaustion; the Cannot assign requested address error stems from __inet_check_established returning EADDRNOTAVAIL.
We were able to mitigate slightly with connectionPoolPerDownstreamConnection which I think helped centralize requests onto connection pools that already had an upstream socket open, but even with --concurrency=1 and connectionPoolPerDownstreamConnection, we still see errors under load.
I think maybe the behavior of cilium-envoy where it attempts to bind to the same port as the upstream request needs to change - it doesn't seem to take into account the effects of distributing these requests across multiple different Envoy threads+connection pools.

from https://github.com/cilium/cilium/issues/36427#issuecomment-2555659428: when using a CNP instead of a CEC, Envoy's sockets don't bind to the source address/port, so the collision scenario no longer exists.

Figured it out:
Set the label cilium.io/use-original-source-address: false on your CiliumEnvoyConfig.
Without that label, a CEC not managed by the Gateway or Ingress controller will cause Envoy to attempt to bind the upstream connection pool socket to the original source IP/port of the local downstream connection, so if two pipelined HTTP requests from the same downstream are distributed to two different connection pools and land on the same destination pod when the upstream ClusterIP is resolved, the 5-tuples will collide and connect() will return EADDRNOTAVAIL.
(At least I think this is what's happening. But setting that label on our CECs eliminated the issue.)


Add studder-formatter

S-s-s-studdering i-is not a-a-a  joke. but it would be a nice formatter!

fixed in v0.7.1


Cleaned up warnings and provided some error checking in build.php

Added a try/catch block to minimize PHP fatal error output while retaining the message.

Is this maybe why our whole build is broken for weeks :DD @jnvsor I don't think we should keep this open, @deanhouseholder thanks for the effort, and sorry we couldn't make use of the code, but people like you is who inspire us to build things like this.

@raveren No, the build isn't broken. @deanhouseholder I'd like to reopen this, but if you don't feel like finishing it up I'll leave it closed.

I can't understand why you prefer ugly errors to clean ones. Haha
On Thu, Mar 21, 2019 at 11:08 AM Jonathan Vollebregt <
<EMAIL_ADDRESS>wrote:

@raveren https://github.com/raveren No, the build isn't broken.
@deanhouseholder https://github.com/deanhouseholder I'd like to reopen
this, but if you don't feel like finishing it up I'll leave it closed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kint-php/kint/pull/299#issuecomment-475319808, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAr0EQpe8qd1HA1STgpRv6u4XXAFJlxWks5vY7yRgaJpZM4aTq4N
.

This gets a bit unixy as far as technicals go, but programs with text input/output generally have 3 streams: stdin (input) stdout (output) and stderr (error)
Why do you have different streams for output and error? Because it's common to connect programs up to eachother. For instance you could pipe a php script's output to another one like so:
php script.php | php script2.php

If you do it like this, only stdout (The output) is piped to script2, and stderr ends up in the terminal. Apache modphp typically sends both stdout and stderr to the browser, so it doesn't matter there.
The second reason is that programs return a number when they end indicating success or failure. A 0 is success and anything else indicates a variety of failures.
php script.php && php script2.php

In this example script2 will only run if script1 succeeded, which it knows by the exit code. A proper fatal error returns a non-0 exit code, which is also used by CI to tell if things succeeded.
In other words by manually exiting the script exceptions in the build won't be caught by CI which is a very bad thing.
Long story short: Smarter people than us thought up these systems and they work very well. No touchy :)


Chore: Update function headers

(No code changes)
Added descriptions
Removed extra (empty) lines
Added _WD_Screenshot return values
Diff for these changes doesn't appear to be working

Added commit for fixing _WD_ScreenShot (Issue #44). I can split these if you don't want them both in the same pull request.


Add cell_top & cell_bottom actions

Add a new action in Controller to move the cursor to the top of the cell: cursor_move_cell_top.
Add a new action in Controller to move the cursor to the bottom of the cell: cursor_move_cell_bottom.

PgDown moves cursor to the last row of the cell.
PgUp moves cursor to the first row of the cell.
CTRL + PgDown moves cursor to the last row of the decision table.
CTRL + PgUp moves cursor to the first row of the decision table.


Use tagged release for go-ftw download

This makes sure changes to the artifacts like filename don't affect the workflow

More context: we are changing the package naming for go-ftw to more sane defaults in https://github.com/coreruleset/go-ftw/pull/91, so in the meantime we don't want to break CRS tests.


Bump to version 2.0.0

Please merge https://github.com/beyondstorage/go-service-s3/pull/95 into this branch.


Groups in @BeforeMethod and @AfterMethod don't work as expected

Either I'm misinterpreting or misusing TestNG-6.8.8, but it seems like the groups attribute in the @BeforeMethod and @AfterMethod aren't working as expected.
Consider following class:
public class MyClass {
    public void method1() {
        System.out.println("Method 1 invocation");
    }
    public void method2() {
        System.out.println("Method 2 invocation");
    }
}

and following test
public class MyClassTest {
    MyClass o = new MyClass();
    @BeforeMethod(groups = "method1")
    public void before1() {
        System.out.println("Before1");
    }
    @BeforeMethod(groups = "method2")
    public void before2() {
        System.out.println("Before2");
    }

    @AfterMethod(groups = "method1")
    public void after1() {
        System.out.println("After1");
    }
    @AfterMethod(groups = "method2")
    public void after2() {
        System.out.println("After2");
    }

    @Test(groups = "method1")
    public void testMethod1() throws Exception {
        o.method1();
    }
    @Test(groups = "method2")
    public void testMethod2() throws Exception {
        o.method2();
    }
}

I would expect the output to be
Before1
Method 1 invocation
After1
Before2
Method 2 invocation
After2

However, the output is
Before1
Before2
Method 1 invocation
After1
After2
Before1
Before2
Method 2 invocation
After1
After2

I have the same problem. AfterMethod don't care about the group specified in groups="".

@jeangb - I just now tried this in TestNG 6.10 and I cannot recreate the problem.
Here's my sample class
package org.rationale.emotions.github.issue549;

public class MyClass {
    public void method1() {
        System.out.println("Method 1 invocation");
    }

    public void method2() {
        System.out.println("Method 2 invocation");
    }
}

Here's my test class
package org.rationale.emotions.github.issue549;

import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class MyClassTest {
    MyClass o = new MyClass();

    @BeforeMethod (groups = "method1")
    public void before1() {
        System.out.println("Before1");
    }

    @BeforeMethod (groups = "method2")
    public void before2() {
        System.out.println("Before2");
    }

    @AfterMethod (groups = "method1")
    public void after1() {
        System.out.println("After1");
    }

    @AfterMethod (groups = "method2")
    public void after2() {
        System.out.println("After2");
    }

    @Test (groups = "method1")
    public void testMethod1() throws Exception {
        o.method1();
    }

    @Test (groups = "method2")
    public void testMethod2() throws Exception {
        o.method2();
    }
}

Here's how my suite xml looks like
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="1265_Suite" parallel="false" verbose="2">
    <test name="92">
        <groups>
            <run>
                <include name="method1"/>
            </run>
        </groups>
        <classes>
            <class name="org.rationale.emotions.github.issue549.MyClassTest"/>
        </classes>
    </test>
</suite>

and here's my output
XmlSuite] [WARN] 'parallel' value 'false' is deprecated, default value will be used instead: 'none'....
... TestNG 6.10 by Cédric Beust<EMAIL_ADDRESS>...

[TestNG] Running:
  /Users/krmahadevan/playground/scrappable-project/src/test/resources/549.xml
Before1
Method 1 invocation
After1
PASSED: testMethod1

===============================================
    92
    Tests run: 1, Failures: 0, Skips: 0
===============================================

@krmahadevan Could you try to include  method2 too like this:
<groups>             <run>                 <include name="method1"/>                 <include name="method2"/>             </run>         </groups>
Does it work this way too for you?

@apreg - If I do what you suggested above, then yes I can re-create what you are seeing.
@juherr - Do you think that this is an issue ? I am a bit confused on this.
Here's what TestNG is currently doing :
When user chooses to run by 1 or more groups, TestNG ends up invoking all the @BeforeMethod methods that belong to the groups chosen, for all the tests, instead of selectively running only the relevant @BeforeMethod methods.

Currently, it is how groups is working.
In fact, groups is just a way to select methods but there is no relation between methods with the same groups.
If you choose method1 group, then every method with method1 will be selected and run. If you choose method1 and method2 groups, then every method with method1 or method2 will be selected and run.
So, it is not a bug.
But many persons don't understand it (some StackOverflow questions about it) because the behavior is not really logic. So, it could be a good idea to fix the issue.
But it is a big (maybe complicate) behavior move with potential regressions and we need to think about it before.
Ping @cbeust

@juherr What is not logic about the current behavior?

@cbeust In fact, many users expect to have groups activated when they select a test.
For example, if they run @Test(groups="a"), they expect @BeforeMethod(groups="a") will be run without specify any group in the test selection.
Another example: if the run @Test(dependsOnGroup="a"), they expect @Test(groups="a")` will be run first.
I'm not saying the current behavior is bad but it is not the most intuitive.

Isn't this what alwaysRun = true is for?

No, it is not exactly the same goal IMO.
alwaysRun will be always invoked when groups will be only invoked if a method has one of the groups.
And if you check the sample of this issue, it goes a step further: it expects the group is only activated for the context of the test.

Has this issue been resolved . i just tried with testng 6.9.14 and it didn't work for aftermethod groups .

@madhukarashok - Can you please help clarify what did you try ? This is not resolved because there's some ambiguity in the expectation. Please feel free to include relevant java and xml samples.

i think i overlooked a condition which caused that confusion and im might be able to rectify that.
but i wanted to check why does multiple @aftermethod shows as skipped in the summary but in actual all the methods were executed succesfully.
![aftermethod]
(https://user-images.githubusercontent.com/3843299/27188001-c8627ed4-51ba-11e7-9c5b-e47f5c173cd9.png)

@madhukarashok - Am guessing that screenshot is from eclipse. But you still haven't shared an example that can be used to reproduce the problem. Can you please do that ?

hi here is the code snippet
package com.testCases;
import org.testng.annotations.AfterGroups;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
public class sampletest extends basetest {
private String EcommerceTestCase;
public Boolean iflag=false;
//private String ecommerceTestCases;
AppDataFromDB apdfdb = null;
cartTestartTest sCart =null;
cartOverlayTest bopisAddtoCart=null;
createOrderTest bopisCreateOrder = null;
sPage bopisPage = null;
sCart  cart=null;
private String ecomID;
public BopisRegressionTests(){

}

@Factory(dataProvider = "DP4")
public BopisRegressionTests(String testCaseID){
	this.EcommerceTestCase = testCaseID;

}

@BeforeMethod(groups={"atc"})
public void setUp(){
	common.setInsertResults(insertResults);
	apdfdb = new AppDataFromDB(insertResults);
	EcommerceTestCases ecommerceTestCases = apdfdb.getNewTestCaseFROMDB(EcommerceTestCase);
	ecomID = ecommerceTestCases.getTestCaseDescription();
	cart = new ShoppingCart(common);
	bopisPage = new BopisPage(common);
	bopisCreateOrder = new BopisCreateOrderTest(EcommerceTestCase, common);
	bopisAddtoCart = new BopisAddtoCartOverlayTest();
}

@BeforeGroups(groups={"cart","atc"})
 public void Login(){
	LoginSetup();
}

@Test(groups={"atc"})
public void baddToCartOverLayVerification(){
if (ecomID.contains("AddToOverLay")) {
	bopisAddtoCart.addToCartOverlay(EcommerceTestCase, common);
	bopisAddtoCart.overlayverification(EcommerceTestCase,apdfdb);
    }

}

@Test(groups={"cart"})
public void cartverfiy (){
   if (ecomID.contains("Cart"))  {
	   sCart = new BopisCartTest();
	   sCart.cart(EcommerceTestCase, common);
    }

}

@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="d1")
public void addToCart1(){
if (ecomID.contains("Cart"))  {
	sCart.addToCart();
}
}

@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="addToCart1")
public void cstoreHeaderValidation1(){
	if (sCart != null ) {
    sCart.storeHeaderValidation();
	}
}

}
here is the xml

<parameter name="configFile" value="config/chrome.win.properties" />
<parameter name="logLevel" value="info" />

<test name="bRegressionRun">
	<groups>
	  <run>
       <include name="atc"/>
       <include name="cart"/>
      </run>
    </groups>
	<classes>
	   <class name="com.sw.testcases.BopisRegressionTests" />
	</classes>
</test>

my base execution flow is i hav two groups "atc" & "cart" and i have different methods for respective groups which needs to get kicked off when @Test is executed. but im having hard time running dependonmethods in conjuncture with @AfterMethod as it keeps spitting out this error "com.sw.testcases.BopisRegressionTests.addToCart1() is depending on method public void com.sw.testcases.BopisRegressionTests.d1(), which is not annotated with @Test or not included."
i jus need one execution for each of this groups in the following order

atc -> baddToCartOverLayVerification method execution completed
2)cart - > cartverify -> (then aftermethod annotations for this group should get kicked off based on dependOnMethod annoations) .

let me know if it makes sense ?

hi i tried dependOnGroups & dependOnMethods annotations like shown below and was unsuccesfull
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
import com.ecommerce.store.dataaccess.EcommerceTestCases;
import com.sw.datahelper.AppDataFromDB;
import com.sw.pages.BopisPage;
import com.sw.pages.ShoppingCart;
public class BopisRegressionTests extends SWBaseTest {
private String EcommerceTestCase;
public Boolean iflag=false;
//private String ecommerceTestCases;
AppDataFromDB apdfdb = null;
BopisCartTest sCart =null;
BopisAddtoCartOverlayTest bopisAddtoCart=null;
BopisCreateOrderTest bopisCreateOrder = null;
BopisPage bopisPage = null;
ShoppingCart  cart=null;
private String ecomID;
public BopisRegressionTests(){

}

@Factory(dataProvider = "DP4")
public BopisRegressionTests(String testCaseID){
	this.EcommerceTestCase = testCaseID;

}

@BeforeMethod(alwaysRun = true)
public void setUp(){
	common.setInsertResults(insertResults);
	apdfdb = new AppDataFromDB(insertResults);
	EcommerceTestCases ecommerceTestCases = apdfdb.getNewTestCaseFROMDB(EcommerceTestCase);
	ecomID = ecommerceTestCases.getTestCaseDescription();
	cart = new ShoppingCart(common);
	bopisPage = new BopisPage(common);
	bopisCreateOrder = new BopisCreateOrderTest(EcommerceTestCase, common);
	bopisAddtoCart = new BopisAddtoCartOverlayTest();
}

@Test(groups={"cart"})
 public void aLogin(){
	LoginSetup();
}

@Test(groups={"atc"})
public void baddToCartOverLayVerification(){
if (ecomID.contains("AddToOverLay")) {
	bopisAddtoCart.addToCartOverlay(EcommerceTestCase, common);
	bopisAddtoCart.overlayverification(EcommerceTestCase,apdfdb);
    }

}

@Test(groups="cart",dependsOnMethods="aLogin")
public void d1 (){
   if (ecomID.contains("Cart"))  {
	   sCart = new BopisCartTest();
	   sCart.cart(EcommerceTestCase, common);
    }

}

@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="d1")
public void addToCart1(){
if (ecomID.contains("Cart"))  {
	sCart.addToCart();
}
}

@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="addToCart1")
public void cstoreHeaderValidation1(){
	if (sCart != null ) {
    sCart.storeHeaderValidation();
	}
}

}
The execution stops at d1 and it doesn't enter the aftermethod annotations which is dependent on groups & dependentonmethod = "d1".
any help would be appreciated .

How is this not a bug? If BeforeMethod with assigned groups also run on tests not assigned to those groups, what's the groups assignment for? Sorry, I just don't see the logic here.

As I said, groups are used for the test selection. As an alternative, you can split your tests in many classes.

The functionality you are looking for is BeforeGroup.
@BeforeGroups(groups = "method1")
public void before1() {
    System.out.println("Before1");
}
@BeforeGroups(groups = "method2")
public void before2() {
    System.out.println("Before2");
}

@AfterGroups(groups = "method1")
public void after1() {
    System.out.println("After1");
}
@AfterGroups(groups = "method2")
public void after2() {
    System.out.println("After2");
}

@Test(groups = "method1")
public void testMethod1() throws Exception {
	System.out.println("Method 1 invocation");
}
@Test(groups = "method2")
public void testMethod2() throws Exception {
	System.out.println("Method 2 invocation");
}

Output:
Before1
Method 1 invocation
After1
Before2
Method 2 invocation
After2
The BeforeMethod or AfterMethod will run for EVERY method in the test class IF its group is enabled.
Since you choose to run the class with groups 'method1' and  'method2', methods before1(), before2(), after1(), and after2() are ALL flagged as true. Therefore, both before1() AND before2() will run before EVERY method, and both after1() AND after2() will run after every method.

No, @BeforeGroup doens't do exactly what is needed here. It is only invoked once per group, whereas it would be nice to have a way to invoke a configuration method before every test from a specific group. Ditto for @AfterMethod. Right now, pretty much the only way to achieve it is the Testcase Class per Test Fixture strategy as xUnit Patterns book puts it. Sometimes it's fine, but sometimes one ends up with too many classes this way, especially if inheritance is involved.
I propose to add a kind of group filter to @BeforeMethod. Perhaps, @BeforeMethod(onlyForGroups = ...). Then this method will only be invoked before a test method belonging to one of the listed groups. I have just tried some minimal implementation, works fine for me. Should I submit a PR to discuss it further?
And as far as I can see, issue #780 is very similar, so this should fix it too.

@stachenov - Please go ahead and raise a PR with your changes. That would help facilitating the discussion on the actual changes more aptly.

I tried both 6.10 and 7.0.0-beta, the test result is not expected. Here is the wiki page with details.
If groups cannot group Before and After methods, why don't remove it to avoid confusions. I am quite confused why we need to add one more attribute onlyForGroups rather than fix groups attribute to make it workable in the expected way. (@juherr, if groups is used for test selection, why did we add it to Before/After Method annotation also?)
But anyway, onlyForGroups can work in 7.0.0-beta7.

The wiki page is broken. Opens up a error page instead.
I also find it confusing, but I understand the historical reasons behind it. If groups are used for test selection, it makes perfect sense that they work in exactly the same way for before/after methods. You may have, say, a “slow” group that also includes rather slow startup, and therefore mark the appropriate before method as belonging to the same group, to avoid expensive startup if the slow tests aren't selected. And once groups are defined as test selection mechanism, it makes little sense to use them for method invocation control.
The problem is that maybe it is wrong to have onlyForGroups attribute too, because it also refers to groups, and therefore should not be used for anything else than test selection. But I couldn't think of anything but groups to, well, group method invocations (save for the test-class-per-fixture approach, which works poorly for testing class hierarchies)... Adding another kind of group would make things even more complicated and force people to duplicate stuff in every test method, like @Test(group="slow", invocationGroup="slow"), which doesn't make any sense either.

Maybe a better name for TestNG groups could be "tags". But as @stachenov said: historical reason.

I find the documentation lacking for the use of onlyForGroups on AfterMethods. Currently it says:

onlyForGroups | Only for @BeforeMethod and @AfterMethod. If specified, then this setup/teardown method will only be invoked if the corresponding test method belongs to one of the listed groups.

src: https://testng.org/doc/documentation-main.html
In reality, must be accompanied with a "alwaysRun = true" for it to work the way you expect.
I'm using version 7.0.0

What happens if you don't specify alwaysRun? I never used it and never had any problems. Maybe it's something v7-specific, but I'd like to clarify what behavior you're expecting and what you're getting instead.

@stachenov
If you meant this wiki page, then it's still alive.

?! I was replying to @CoryKniefel about alwaysRun. I didn't even know about that wiki page.

@stachenov
I replied to your message "The wiki page is broken. Opens up a error page instead.".

Oh. But it was apparently broken more than a year ago. I don't think it matters now whether it was indeed so.

If we exclude the groups or do not use groups tag in testng.xml file, then @BeforeMethod/@AfterMethod(onlyForGroups) methods run. But if we include the groups, then the @BeforeMethod/@AfterMethod(onlyForGroups) methods do not run. So, we need to club it with alwaysRun=true i.e. @BeforeMethod/@AfterMethod(onlyForGroups, alwaysRun=true).


UnhandledPromiseRejectionWarning

After a successful login to cas, i get this error in console :: Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
and then it loads a page which says unathorized.
Any help with this?

I'm not entirely sure, but I'm guessing this is something on your express server, outside of this module, downstream in your middleware.
Start removing middleware from your express stack to find the culprit.  One of them is sending sendHeaders while another one downstream is trying to send additional headers.

@badnuk After some console logs in your code I think it is coming from failed ssl verification. Can we have an optio to disable it for development?

I think this still might work: https://stackoverflow.com/questions/10888610/ignore-invalid-self-signed-ssl-certificate-in-node-js-with-https-request
process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0;

i ll try this on Monday thanks man.


replace includes with indexOf

Hi!
About this issue: https://github.com/fridays/next-routes/issues/32
As my investigation at case When using Router.pushRoute('index', params) from link above.
Root cause:

Because Array.includes is not working in iOS 8.4

Solution:

I replace it with indexOf

Don't think this is the correct way to solve this, better is for @nndung179 to add a polyfill/transpile this?  Otherwise you should add tests for checking that you're not using ES6 features

Thanks a lot for contributing!
Nevertheless we already have spotted this issue in https://github.com/fridays/next-routes/pull/37.
Since you also need to polyfill Array.find I am closing this issue in favor of #37.


DF Curvy Walls: Add support for Button Overflow ~Option to put Controls in main Walls list~

Is your feature request related to a problem? Please describe.
I think a previous update made the Curvy Wall controls into its own element in the UI. I recently found and started using the module Button Overflow, which allows for buttons to roll over into another column. Unfortunately as seen in the picture below, the DF Curvy Wall controls aren't included in the main element and will be obscured instead.

Describe the solution you'd like
Since I don't want to ask for a reversion to a possibly-non-existent behavior, I'd like to request a GM-only option in the module settings to decide whether the Curvy Walls controls are in their own element or tacked onto the existing wall controls. Then I could just toggle that on and be good to go.

So I don't plan to revert back as the new UI is a complete rebuild and is actually running much better than when I had to try and get the condos into the main control set.
Instead I can maybe try grabbing the Controls element and adjusting my X position if the controls element has expanded. As such, this ticket is a conflict issue since Curvy Walls works as expected in core Foundry.

Finally got around to getting this fixed up 👍🏽


How to use it directly in my project?

I want to use it in my project and i have done it by copying your project files in my project and included a continue button in the View Controller and I want the button to trigger a segue when clicked but its not working, Please Help 🙂

Sorry, adding a segue isn't really in the scope of this project. This repo was a little side idea and I, unfortunately, don't have time to work on it these days.
I appreciate your interest in this repo and hope your project goes well. Thank you. 🙇 🌻


Support custom platforms

Add the ability to bring your own metadata service. Add a "custom" platform type and document how to use it by overwriting parts of the coreos-metadata service
DO NOT MERGE: currently untested.

Addressed various typos and fat-fingerings of file modes. Added some comments and made the language a little clearer.

Updated to namedspace everything with COREOS_ and updated the docs to match.

Updated the grammar. Left off the :custom for the config type until the next CT release happens (since the docs sync doesn't use master IIRC)

If I understand this correctly, this is targeting a user who:

writes a custom metadata fetching script
Adds that script on the filesystem
Adds a coreos-metadata.service unit that uses that script

all so that the user can write http://{PRIVATE_IPV4}:2379 instead of http://{COREOS_CUSTOM_PRIVATE_IPV4}:2379 in the CLC. Was this a problem folks were having?

Yeah, there was a coreos-user thread about it. CT doesn't allow any { } templating without specifying a platform, regardless of what the value inside the {} is. This also makes it so the same CLC can target baremetal and non-baremetal platforms.


ci: Update workflow trigger conditions

Pull Request
Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

#168  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @JackPlowman and the rest of your teammates on  Graphite


Enable and document .envrc for lorri/direnv

Codecov Report

Merging #454 (00adea0) into master (9c7bfe1) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #454   +/-   ##
=======================================
  Coverage   74.54%   74.54%
=======================================
  Files          83       83
  Lines       10512    10550   +38
=======================================
+ Hits         7836     7865   +29
- Misses       2676     2685    +9

Impacted Files
Coverage Δ

fedimint/src/net/api.rs
84.03% <0.00%> (-4.15%)
:arrow_down:

fedimint/src/net/connect.rs
94.94% <0.00%> (-0.26%)
:arrow_down:

ln-gateway/src/lib.rs
41.42% <0.00%> (-0.20%)
:arrow_down:

client/cli/src/main.rs
0.68% <0.00%> (-0.03%)
:arrow_down:

fedimint/src/net/peers.rs
93.17% <0.00%> (ø)

modules/fedimint-wallet/src/db.rs
100.00% <0.00%> (ø)

client/client-lib/src/transaction.rs
99.20% <0.00%> (+<0.01%)
:arrow_up:

modules/fedimint-wallet/src/lib.rs
92.52% <0.00%> (+0.02%)
:arrow_up:

client/client-lib/src/lib.rs
89.06% <0.00%> (+0.05%)
:arrow_up:

client/client-lib/src/api.rs
82.48% <0.00%> (+1.02%)
:arrow_up:

... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add separate token required flags for ndt5 and ndt7

This change adds independent flags for requiring access tokens for the ndt5 and ndt7 servers.
This change will allow us to require access tokens on ndt7 before ndt5 in order to remediate some negative client scheduling behavior.

This change is

Pull Request Test Coverage Report for Build 1272

9 of 9 (100.0%)  changed or added relevant lines in 1 file are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.3%) to 80.077%

Files with Coverage Reduction
New Missed Lines
%

ndt7/upload/sender/sender.go
5
67.65%

Totals

Change from base Build 1265:
0.3%

Covered Lines:
1656

Relevant Lines:
2068

💛  - Coveralls

https://github.com/m-lab/dev-tracker/issues/586

https://github.com/m-lab/dev-tracker/issues/587


Use of Promise.reject() in hot reloading code triggers Chrome debugger on every save

Do you want to request a feature or report a bug?
An unfortunate issue.
What is the current behavior?
Every time you save a file and hot update is not possible, Webpack calls Promise.reject() with hot update abort. Unfortunately this pauses Chrome debugger if "break on exceptions" is true. And even if the Promise will get caught, Chrome debugger can't know it.
The issue is described in:

https://github.com/facebookincubator/create-react-app/issues/1880
https://bugs.chromium.org/p/chromium/issues/detail?id=465666

If the current behavior is a bug, please provide the steps to reproduce.
Toggle "break on exceptions" in Chrome.
Save a file with hot reloading enabled but not accept handlers.
It will pause here.

This is very annoying for every edit.
What is the expected behavior?
Somehow, on either Chrome or Webpack side, we need to figure out how to avoid pausing on every refresh. Or how to blacklist this particular rejection. Neither Chrome nor Webpack seem to provide a workaround for this.
cc @paulirish
If this is a feature request, what is motivation or use case for changing the behavior?
Fast iteration.
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
Happening with Webpack 2+. macOS, Chrome stable.

isn't this an issue for https://github.com/webpack/webpack-dev-server/issues ?

This code lives in Webpack repo. I'm not sure if it's fixable on Webpack side anyway but wanted to raise awareness.

I guess we can fix it this way:
https://github.com/webpack/webpack/blob/master/lib/HotModuleReplacement.runtime.js#L231
Instead of this:
			hotApply(hotApplyOnUpdate).then(function(result) {
				deferred.resolve(result);
			}, function(err) {
				deferred.reject(err);
			});

			Promise.resolve().then(function() {
				return hotApply(hotApplyOnUpdate);
			}).then(function(result) {
				deferred.resolve(result);
			}, function(err) {
				deferred.reject(err);
			});

This calls hotApply on the next micro task tick and a reject handler is attached.
Does somebody want to do a PR and test if my assumptions are correct.

@gaearon @sokra
I faced also :

And my fix was to install latest webpack and webpack-dev-server , as followed :
npm install webpack@latest --save
npm install webpack-dev-server@latest --save


Performance issues wis2box-ui for stations-data with data over a long time-range

Rory Burke and David Berry are working on updating data with a long time-range on "wis2box-daycli".
The UI for this wis2box is currently available at http://<IP_ADDRESS>:8080/
wis2box-daycli currently has only 2 stations and Rory is uploading data for a long time-range: 1932-2022, to enable climate-studies through the wis2box-api.
I note that the wis2box-UI for wis2box-daycli gets stuck loading data when opening the "data"-window for one station, and I can see a large number of network requests:

@maaikelimper is this still an issue?

no this was fixed with the various UI-improvements during the sprint


Implement Random Forrest

This PR implements the first draft of a working Random Forest.

Do not merge until PR: fix_feature_indices(https://github.com/NiklasPfister/adaXT/pull/43) has been merged into main, and the changes to main has propagated to Development.

Before being able to do a proper review, I think it is important to first merge the main branch into the Development branch. There currently are a lot of file changes in this PR, which are already on main, and shouldn't be part of this PR.


Limited Tutorial Creation Options

Problem: the current tutorial creation process seems restricted to Slicer4minutes, thereby excluding others tutorials.
Description: there are several tutorials that require updates, in addition to the initial tutorial (Slicer4minutes). It would be beneficial to consider including other tutorials in the tutorial maker database.

Just an update, we are creating the repositories with the tutorials, now we have the 4minute completed and the incomplete welcome tutorial.

We have created the repository https://github.com/SlicerLatinAmerica/TestSlicerTutorials with the tutorials, now, when you open the extension, the tutorials are loaded from this repository.


Add NinjaTech Enterprises from Kerbal Stuff

This pull request was automatically generated by Kerbal Stuff on behalf of Ninja_5tyl3, to add NinjaTech Enterprises to CKAN.
Please direct questions about this pull request to Ninja_5tyl3.

it even has a .version file <3


Use binaryregexp for rx operator

Fixes #688
I did know about this and didn't mention it yet :P I was wishing we may come up with an alternative solution but in the end this seems to be the best. While I'm nervous using a library not updated for several years, at least there is some comfort in being written by Russ Cox himself, and it is somewhat used
https://github.com/search?q=rsc.io%2Fbinaryregexp&type=code
Performance seems to be the same, it doesn't magically bring in a DFA implementation unfortunately, wasilibs is still the best for that.
We don't like adding dependencies, but this seems to be the only way to implement CRS, which needs to be able to execute regexp on arbitrary bytes. In the future when wasm threads support gets worked out, the RAM tradeoffs for using wasilibs will go away and that could be a better default with the same behavior and much better performance, for now I think using that should still be opt-in by users / connectors.

Oops - the docs say utf8 codepoints but wonder if that was copy-pasted, looks like unicode matching fails completely with it. Should we use binaryregexp only when detecting byte control points in the pattern?

Oops - the docs say utf8 codepoints but wonder if that was copy-pasted, looks like unicode matching fails completely with it. Should we use binaryregexp only when detecting byte control points in the pattern?

I would personally like to avoid using multiple regexp libraries, but this is a v3 stopper so looks like the closest solution

Went with it - agree using both engines is not ideal but probably necessary, at least it's an internal decision so we'll improve it in the future.


python38Packages.azure-mgmt-storage: 18.0.0 -> 19.0.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python38Packages.azure-mgmt-storage is: "This is the Microsoft Azure Storage Management Client Library"
meta.homepage for python38Packages.azure-mgmt-storage is: "https://github.com/Azure/azure-sdk-for-python"
meta.changelog for python38Packages.azure-mgmt-storage is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 19.0.0 with grep in /nix/store/77mshlssvldn0a27pbhb0ahb30k1p80q-python3.8-azure-mgmt-storage-19.0.0

found 19.0.0 in filename of file in /nix/store/77mshlssvldn0a27pbhb0ahb30k1p80q-python3.8-azure-mgmt-storage-19.0.0

directory tree listing: https://gist.github.com/6a8303e48553dd03d920ebaf2e0be528

du listing: https://gist.github.com/4e73cca8e4ac1f78069d09a4a0bd8568

Rebuild report (if merged into master) (click to expand)

16 total rebuild path(s)

4 package rebuild(s)

4 x86_64-linux rebuild(s)
4 i686-linux rebuild(s)
4 x86_64-darwin rebuild(s)
4 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
appdaemon
azure-cli
python38Packages.azure-mgmt-storage
python39Packages.azure-mgmt-storage

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/77mshlssvldn0a27pbhb0ahb30k1p80q-python3.8-azure-mgmt-storage-19.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python38Packages.azure-mgmt-storage https://github.com/r-ryantm/nixpkgs/archive/cb7c941b949a3f5ab6a11efba467e45377d1ca14.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/77mshlssvldn0a27pbhb0ahb30k1p80q-python3.8-azure-mgmt-storage-19.0.0
ls -la /nix/store/77mshlssvldn0a27pbhb0ahb30k1p80q-python3.8-azure-mgmt-storage-19.0.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

4 packages built:

appdaemon
azure-cli
python38Packages.azure-mgmt-storage
python39Packages.azure-mgmt-storage

Maintainer pings
cc @jonringer @olcai @mwilsoncoding for testing.

Result of nixpkgs-review pr 137961 at cb7c941b run on aarch64-linux 1

4 packages built successfully:

appdaemon
azure-cli
python38Packages.azure-mgmt-storage
python39Packages.azure-mgmt-storage

1 suggestion:

warning: unused-argument
Unused argument: python.
Near pkgs/development/python-modules/azure-mgmt-storage/default.nix:4:3:
  |
4 | , python
  |   ^


HylaFAX+: init at 5.5.8

Motivation for this change
trees?
Things done
Receiving and sending faxes works.
Email redirection works.
Optional hfaxd client-server service works (mostly for LAN clients) and some local GUIs.
Configuration is somewhat manual\interactive due to the nature of the hardware:

In configuration.nix: services.hylafaxp.enable = true;

As root: faxsetup.

Copy /var/spool/hylafax/etc/config and /var/spool/hylafax/etc/config.tty* to configuration.nix. e.g.

  services.hylafaxp = {
    enable = true;
    config = ''
      LogFacility:                daemon
      CountryCode:                011
      AreaCode:           01
      LongDistancePrefix: 1
      InternationalPrefix:        011
      DialStringRules:    etc/dialrules
      ServerTracing:              1
    '';
    devs = {
      ttyACM0 = ''
        CountryCode:        011
        AreaCode:       01
        FAXNumber:      12345678
        LongDistancePrefix: 1
        InternationalPrefix:    011
        DialStringRules:    etc/dialrules
        ServerTracing:      1
        SessionTracing:     0xFFF
        RecvFileMode:       0600
        LogFileMode:        0600
        DeviceMode:     0600
        RingsBeforeAnswer:  5
        SpeakerVolume:      off
        GettyArgs:      "-h %l dx_%s"
        LocalIdentifier:    companyOrHome
        TagLineFont:        etc/LiberationSans-25.pcf
        TagLineFormat:      "From %%l|%c|Page %%P of %%T"
        MaxRecvPages:       25
        #
        #
        # Modem-related stuff: should reflect modem command interface
        # and hardware connection/cabling (e.g. flow control).
        #
        ModemType:      Class1      # use this to supply a hint
        ModemRate:      19200       # rate for DCE-DTE communication
        ModemFlowControl:   rtscts      # default
        #
        ModemNoFlowCmd:     AT&K0       # setup no flow control
        ModemHardFlowCmd:   AT&K3       # setup hardware flow control
        ModemSoftFlowCmd:   AT&K4       # setup software flow control
        ModemSetupDTRCmd:   AT&D2       # setup so DTR drop resets modem
        ModemSetupDCDCmd:   AT&C1       # setup so DCD reflects carrier (or not)
        #
        ModemMfrQueryCmd:   !Rockwell
        ModemModelQueryCmd: !RC288DPi
        ModemRevQueryCmd:   ATI3        # product information
        #
        ModemDialCmd:       ATDT%s      # T for tone dialing
      '';
    };
  };

nixos-rebuild switch.

This will produce persistent results between reboot & updates.

(Optional) For email dispatching, it's enough to add:

    faxDispatch = ''
<EMAIL_ADDRESS>      FILETYPE=pdf;
    '';

to services.hylafaxp and
  services.postfix = {
    enable = true;
    setSendmail = true;
  };

(Optional) For the client-server protocol service, adding

    hfaxd = {
      enable = true;
      hosts = ''
        localhost
        <IP_ADDRESS>
      '';
    };

to services.hylafaxp should be sufficient for local access. See faxadduser for LAN clients and passwords.
I've only tested one soft-modem, but I wrote the module to support multiple modems.
Built and tested with nixos-rebuild switch -I nixpkgs=/home/user/nixpkgs.

[x] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Note: HylaFAX vs. HylaFAX+ is 2012 vs. 2016.

WIP: Taking a look at the email redirection.

@Mic92 I've cleaned up a few problem spots including the FONTPATH you've requested. It's still being produced by ghostscript since we still need to account for different fonts and changes between ghostscript versions... But it's nicer looking :)
@lethalman Thanks for explaining how to condition the let...in... statement.

@RamKromberg
I have refactored the module:
https://github.com/Mic92/nixpkgs/commit/babb76722c1355aa0d84ccd9847bc69842afb3fb

@Mic92 Have you ever pushed the refactored version to mainline Nixpkgs?

@7c6f434c no

@Mic92 any reason not to push it (and close this PR afterwards)?

@7c6f434c If memory serves, it was suggested the package should probably be rewritten (and updated*) to run in a container since it handles its own ring0->ring3 de-escalation. I think I tried doing it myself but run into some issues... Well, I was pretty satisfied with the C code security-wise and was running it for a few months without any major problems so I just never got around to doing it.
Eventually, it stalled as I became "distracted" with unrelated low-level work and generally gone inactive with NixOS development.
Seeing how I moved my fax-modem to a RasPi and been running Debian there... Well, it would be a shame to see so much work going to waste but I'll understand if you'd want to close the pull request. Though I wish it could be archived or something in case someone ends up wanting to work on it or something.

5.5.9 http://hylafax.sourceforge.net/news/5.5.9.php

@RamKromberg I think the refactored version by @Mic92 addresses some of the discussion. I asked about closing the PR after merging whatever further work on top of this PR has been done.
In general, closed PRs are visible in a separate tab and are displayed in search; availability of the branch actually depends on whether you keep it in your fork.


Unstable performance between adjacent epochs during test phase

Hi, thanks very much for your code.
But, trained with sync-bn layer, my model seems unstable between adjacent epochs during test phase, i.e., on the test dataset, I tested my model every two epochs, and performance curve present to vibrate seriously.
PS. I used the momentum of 0.1 by default, does it seem too large ?

The momentum can be the reason for your unstable training. Could you please try some smaller values such as 1e-2 or 1e-3? I am not sure about your dataset and/or your application, but I think it's worth trying.

Close for now. Please feel free to reopen it if you still have questions.

Hi @lyakaap
Thanks for your information.
I think the choice of the momentum is sometimes a tricky issue, depending on your task/model, and it will be extremely helpful for others if you could provide some details on your model / how you choose the momentum.
In this repo, I use the momentum=0.1 as the default value, which is consistent with the default setting as in PyTorch's BatchNorm.

Thanks for reply.
My target dataset is cityscapes and its distribution is far from imagenet, which pretrained model trained. So such a low momentum rate, 0.001 harm estimating cityscapes stats.

In this repo, I use the momentum=0.1 as the default value, which is consistent with the default setting as in PyTorch's BatchNorm.
Sorry, I confused this repo's settings between other bn repo's (I forgot repo name..).
momentum=0.1 gave rise to no problems, so it is okay to follow pytorch official setting.

Thank you so much for the information! @lyakaap


NNI FAILED: gpu_metrics file does not exist!

Describe the issue:
The NNI failed the moment I run my project, which reported: "gpu_metrics file does not exist". But I can run the mnist-pytorch demo.
I have checked that the gpu_metrics file exist in /tmp/*/nni/script

Environment:

NNI version: 2.3
Training service (local|remote|pai|aml|etc): local
Client OS: ubuntu 20.04
Server OS (for remote mode only):
Python version: python 3.8
PyTorch/TensorFlow version: 1.9
Is conda/virtualenv/venv used?: yes
Is running in Docker?: no

Configuration:

Experiment config (remember to remove secrets!):

authorName: Bowen
experimentName: cem
trialConcurrency: 1
maxExecDuration: 24h
maxTrialNum: 10
trainingServicePlatform: local
searchSpacePath: search_space.json
useAnnotation: false
tuner:
builtinTunerName: TPE
classArgs:
#choice: maximize, minimize
optimize_mode: maximize
localConfig:
useActiveGpu: true
gpuIndices: 1
maxTrialNumPerGpu: 4
trial:
command: python train_nni.py
codeDir: .
gpuNum: 1

Search space:

{
"optimizer":{"_type":"choice", "_value":["Adam", "Adamax", "Adagrad", "RMSprop", "Adagrad"]},
"Transpose":{"_type":"choice", "_value":[0.3, 0.4, 0.5]},
"HorizontalFlip":{"_type":"choice", "_value":[0.3, 0.4, 0.5]},
"VerticalFlip":{"_type":"choice", "_value":[0.3, 0.4, 0.5]},
"ShiftScaleRotate":{"_type":"choice", "_value":[0.3, 0.4, 0.5]},
"hue_shift_limit":{"_type":"choice", "_value":[0.2, 0.3, 0.4]},
"sat_shift_limit":{"_type":"choice", "_value":[0.2, 0.3, 0.4]},
"val_shift_limit":{"_type":"choice", "_value":[0.2, 0.3, 0.4]},
"HueSaturationValue":{"_type":"choice", "_value":[0.3, 0.4, 0.5]}
}

Log message:

nnimanager.log:

nnimanager.log

dispatcher.log:

dispatcher.log

nnictl stdout and stderr:
(pytorch) bowen@fsk-workstation:~$ nnictl log stdout
[2021-09-16 00:38:47] INFO (numexpr.utils/MainThread) Note: NumExpr detected 48 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
[2021-09-16 00:38:47] INFO (numexpr.utils/MainThread) NumExpr defaulting to 8 threads.

            Experiment start time 2021-09-16 00:38:31

(pytorch) bowen@fsk-workstation:~$ nnictl log stderr
[2021-09-16 00:39:15] INFO (numexpr.utils/MainThread) Note: NumExpr detected 48 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
[2021-09-16 00:39:15] INFO (numexpr.utils/MainThread) NumExpr defaulting to 8 threads.
            Experiment start time 2021-09-16 00:38:31

How to reproduce it?:

Solved.

How to solve it?


no matches for kind "ExternalSecret" in version "kubernetes-client.io/v1"

Hi All,
Getting error "no matches for kind "ExternalSecret" in version "kubernetes-client.io/v1"" for the following manifest file, any clues will be helpful:
apiVersion: "kubernetes-client.io/v1"
kind: ExternalSecret
metadata:
name: test-secret
namespace: dev
spec:
backendType: secretsManager
roleArn: arn:aws:iam::<>:role/<>
region: us-east-1
data:
- key: username
name: tutorials/MyFirstTutorialSecret

Looks like you didnt install the CRD.

I encountered the same issue via following Install with kubectl. The reason was if you run helm template --output-dir ./output_dir ./charts/kubernetes-external-secrets/, it won't extract ./charts/kubernetes-external-secrets/crds/kubernetes-client.io_externalsecrets_crd.yaml file therefore the CRD is missing
My fix was manually copy the kubernetes-client.io_externalsecrets_crd.yaml file to output-dir and put in correct location then run helm upgrade.

I think you can use flag --include-crds with helm template

I encountered the same issue via following Install with kubectl. The reason was if you run helm template --output-dir ./output_dir ./charts/kubernetes-external-secrets/, it won't extract ./charts/kubernetes-external-secrets/crds/kubernetes-client.io_externalsecrets_crd.yaml file therefore the CRD is missing
My fix was manually copy the kubernetes-client.io_externalsecrets_crd.yaml file to output-dir and put in correct location then run helm upgrade.
Btw my helm version is version.BuildInfo{Version:"v3.5.4", GitCommit:"1b5edb69df3d3a08df77c9902dc17af864ff05d1", GitTreeState:"dirty", GoVersion:"go1.16.3"} if it's helpful

Awesome!!! Thank you @chenyang-nirovision @Flydiverny


Some dropdowns in inspector fail when dropdown would go off screen (DisplayServer regression)

Godot version
4.0.alpha1.official (31a7ddbf8)
System information
Windows 10
Issue description
When a named enum can have many different values and is given the @export attribute, its dropdown menu in the inspector can lose functionality if there are more values than can fit in the "on screen".
When the mouse button is pressed, the dropdown is opened. If the mouse is over a valid selection when the mouse button is released, this value will be selected and the dropdown will close again immediately. For most users, the issue is easily worked around by pressing and holding the mouse button (once they've worked out that this is what they need to do), but pressing and holding can be an accessibility issue for some users.
Note that since the behaviour depends on screen size and the dropdown's position in the inspector, the number of values at which problems arise may not be consistent from case to case.

Examples
With 35+ values, the dropdown always closes immediately, selecting the first or second value, depending on the mouse's position:

With 34 values, the dropdown may not close if the mouse was high enough:

With 33 or fewer values, the dropdown behaves properly:

Unless the enum's position is lower on the screen:

The problem persists when the dropdown menu has a scrollbar:

A real-world example, editing the type for a custom data layer in the new TileSet:

Note that this doesn't occur with the dropdown for an export with type Resource; the dropdown menu stays open as expected:

Steps to reproduce

Add a Node to an empty scene and attach a script with the following code.

extends Node

enum LengthyEnum{q,w,e,r,t,y,u,i,o,p,a,s,d,f,g,h,j,k,l,z,x,c,v,b,n,m,\
qa,wa,ea,ra,ta,ya,ua,ia,oa,pa,aa,sa,da,fa,ga,ha,ja,ka,la,za,xa,ca,va,ba,na,ma}

@export var lengthy_enum:LengthyEnum

(The enum may need more values, depending on resolution etc. 35 was sufficient for me.)

Select the Node so that it appears in the Inspector.

Open the dropdown menu by clicking it. It will close automatically, and whatever value was under the mouse's position will be selected.

Optionally, add additional export vars above the enum to see that the behaviour depends on the enum's position in the Inspector. Similarly, moving the Godot Editor window within the screen can change the behaviour.

Minimal reproduction project
See steps above.

Does this occur on Godot 3.4.2?

I can't replicate it on v3.4.2.stable.official [45eaa2daf]

It can also happen with the 'view settings' in the main editor viewport:

Definitely looks to be a problem with this type of dropdown in general - one more example, choosing language from the project list:

Sounds like #55712, but the issue was already fixed.

I can reproduce the issue on Linux, at least an issue that has similar symptoms.
The problem happens when clicking on the Label section of the OptionButton (as opposed to clicking on the arrow on the right). If the popup is too big, it overlaps with the area being clicked, and thus the item below the pointer gets selected.
https://user-images.githubusercontent.com/4701338/162742261-b55b9da7-45e5-492b-8ba2-1d704207f23e.mp4

On Windows, only enums produce different the behaviours when clicking on the label vs the arrow. For the other examples (language selection and type selection), the dropdown closes immediately regardless of whether or not the arrow was clicked.

Doesn't seem to happen anymore, at least as of RC2 (on Windows 10). Since there have been multiple windowing and input related fixes in the meantime, I'll close this as fixed. Feel free to comment if it is still reproduceable on your system.