cmp: add OrFunc

Proposal Details
Related to https://github.com/golang/go/issues/60204 - I find cmp.Or very useful, although I would appreciate a variant which does not 'execute' the inputs to subsequent parameters.
I propose a variant of this function, which itself takes in a variable number of functions, and executes each until one of them returns a non-zero result.
The motivation for this is to provide behaviour similar to connect() || die('ohnoes')  in Perl, Ruby, etc. So, each parameter is like a 'fallback' of the previous parameter. It should only be executed if the previous parameter(s) returned zero.
// OrFunc returns the the return-value of the first argument (a function), that is not equal to the zero value.
// If no argument is non-zero, it returns the zero value.
// OrFunc is inspired by Or and Perl's`||` operator
func OrFunc[T comparable, F func() T](funcs ...F) T {
	var zero T
	for _, f := range funcs {
		t := f()
		if t != zero {
			return t
		}
	}
	return zero
}

See here for a working example: https://go.dev/play/p/GYr6qeT7pDg
Thanks!

Regardless of whether we add that function to the library, it should not be called And, as it is not the dual of Or due to the lazy or "short-circuit" behavior. An And that is the strict dual seems like a reasonable function, so we shouldn't foreclose adding it later, even if there seems to be little need.
How often does this need really arise? The resulting code isn't especially pretty, compared to, say, a single function containing an if-else chain of return statements.

I don't think this makes sense in a world without short function literals (#21498). Even with short function literals, it's not clear that this is better than an if-else chain.


customizable static folders' names

First, I like to say that I do love this generator because it really comes handy when I'm trying to work on side projects and testing things out.
Now for the thing I'm asking, is it possible to make the static folders' names customizable to my own style, reason is I don't like the default naming "stylesheets, images, javascripts" because I usually go something short like "css" or "js" or "img".
I believe I might be able to create this and do a PR, if of course this is kinda useful.
Note: if I change the names of those folders then I can to change them also in the template which will probably result if more refactoring than just the cli file
thank you for your consideration.

Note: if I change the names of those folders then I can to change them also in the template which will probably result if more refactoring than just the cli file

I don't believe the names should be changed in the template for personal preference.  Having the full name javascripts, images, etc at least make it explicitly clear what the folders are for (vs js, img, etc).
There's currently a concise list of command line options and I'm not sure adding to that list to just rename something is worth it.  That said, if you're interested in making a PR - go for it!

I'm going to go ahead and close this issue.  Feel free to raise a PR with more semantic naming conventions if you'd like!


Add cluster being stuck on deletion

Issue: https://github.com/giantswarm/giantswarm/issues/24137
Checklist

[x] Update changelog in CHANGELOG.md.
[ ] Add Unit tests
[ ] Request review from oncall area, as well as team (e.g: oncall-kaas-cloud GitHub group).
[ ] Alerting rules must have a comment documenting why it needs to exist.
[ ] Consider creating a dashboard (if it does not exist already) to help oncallers monitor the status of the issue.

@njuettner can we doo the same for Azure?

Always forget about Azure 😅, I added it now 🙂

Always forget about Azure 😅, I added it now 🙂

I was hoping about one query to catch all cluster CRS but if that's not possible LGTM


What if I wish to load css from the server-side? Instead of wait the js to load?

Ok well it seems that in production the css is properly loaded! Never mind!


Improve new F2 GUI

We've got new menu and now its time to improve it.

show current map name and round
...
...

Add short server descriptions. For example: 'Mix' doesn't really specify what it is about. The screenshot makes it more clear, but it could be improved.
Maybe add a list of online playernames?
Add option to join server when the next round/map starts.

when you're in race or mix and click the icon in F2, should get rid of the gui window. Now you have to click F2 again to remove it, clicking above the "You are here" text doesn't do it.

Today the info in F2 was stuck at a map which has passed mins before I opened the GUI. Another map started but map name and author in F2 were on the old map, first time this happened.


Non-linearity in relative depth

Hi
Since Depth Anything provided a model with impressive relative depth output , I tried to convert relative depth (RD) to metric depth by using lidar GT.
According to the paper, RD  is inverse of metric depth up to a scale s and a bias t. I fit RD with lidar GT to get s and t by using least-squares methold mentioned in MiDas paper. Then I plotted prediction vs GT and got the picure below:

It can be seen that there is obvious non-linearity in the data. (I fit RD with lidar in the middle area of the image, the depth range is aroud 20m, so data in range 0 ~ 20 fit better than other range).
I applied a small gamma factor (-0.1) to RD, the error indeed decreased. And the plot is below:

I also found a post  #47  that suggested applying gamma to RD. But I don't get why. Is there any explaination?

Can you please share the code? I am trying to do something similar and I want to check that too. Thanks!

++


Discrepancy between scipy.special.lpmn and jax.special.scipy.lpmn

There is a discrepancy between the values return for the derivatives in both functions, the magnitude of the difference is significant in the first row of the output. Have included a minimum example, additionally enabled 64 bit precision to ensure that is not the cause of the issue.
Edit: Have checked and it appears taking the grad of the jax lpmn function gives the same output as scipy, so the bug seems to be with the jax implementation.
import jax
import numpy as np
from jax import config
config.update("jax_enable_x64", True)

from jax.scipy.special import lpmn
x = jax.numpy.array([0.1], dtype='float64')

a, b = special.lpmn(6, 6, 0.1)
c, d = lpmn(6, 6, x)
print(np.asarray(d[:,:,0])- b)

[[ 0.00000000e+00  0.00000000e+00  0.00000000e+00 -2.22044605e-16
   3.71250000e-01 -3.45262500e+00 -3.78118125e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  -1.77635684e-15  0.00000000e+00 -1.77635684e-15]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00 -1.77635684e-15
  -3.55271368e-15  0.00000000e+00 -7.10542736e-15]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.77635684e-15
  -1.42108547e-14  1.42108547e-14  1.70530257e-13]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
   1.42108547e-14 -2.27373675e-13 -2.27373675e-13]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
   0.00000000e+00  1.13686838e-13  0.00000000e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
   0.00000000e+00  0.00000000e+00  0.00000000e+00]]

I will look into this.

Thanks for catching it. This [PR] (https://github.com/google/jax/pull/10643) should be a fix.

It looks like there's another inconsistency here – the scipy function requires z to be a scalar, while the JAX function requires z to be a length-1 array. Could we also address that, so that the two versions of the function have a consistent call signature?

Thanks for bringing it out, Jake. I will look into the consistent call signature after submitting this change.


MSBuild and Server's wwwroot Configuration for JS Project

Currently js project needs manual pnpm install and node .\esbuild.mjs to build, and output to Server project's wwwroot/js/dist folder.

We want to make the project works "out of box", i.e. after clone, start Server project in Debug mode should automatically build all dependencies.

We want to keep possibility to rebuild JS project while keep Server project running, and simply refresh browser should load updated js files.

We want to support watch configuration for esbuild, so that after we modify our ts files and save, the output js files should be refreshed automatically.

Cosed by #41


Broken build

Looks like 95442dac broke the build for me.
95442dac
[  3%] Generating manpage for sway.5
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
[  3%] Built target man-sway.5
[  7%] Generating manpage for sway.1
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
[  7%] Built target man-sway.1
[  7%] Built target man
Scanning dependencies of target sway
[ 11%] Building C object CMakeFiles/sway.dir/sway/container.c.o
/tmp/build/sway/sway/container.c: In function ‘pointer_test’:
/tmp/build/sway/sway/container.c:516:12: error: dereferencing pointer to incomplete type ‘const struct wlc_point’
  if (origin->x >= view->x && origin->y >= view->y
            ^
/tmp/build/sway/sway/container.c: In function ‘container_under_pointer’:
/tmp/build/sway/sway/container.c:534:19: error: storage size of ‘origin’ isn’t known
  struct wlc_point origin;
                   ^
/tmp/build/sway/sway/container.c:535:2: warning: implicit declaration of function ‘wlc_pointer_get_position’ [-Wimplicit-function-declaration]
  wlc_pointer_get_position(&origin);
  ^
/tmp/build/sway/sway/container.c:534:19: warning: unused variable ‘origin’ [-Wunused-variable]
  struct wlc_point origin;
                   ^
CMakeFiles/sway.dir/build.make:206: recipe for target 'CMakeFiles/sway.dir/sway/container.c.o' failed
make[2]: *** [CMakeFiles/sway.dir/sway/container.c.o] Error 1
make[2]: Target 'CMakeFiles/sway.dir/build' not remade because of errors.
CMakeFiles/Makefile2:169: recipe for target 'CMakeFiles/sway.dir/all' failed
make[1]: *** [CMakeFiles/sway.dir/all] Error 2
[ 30%] Built target swaybg
make[1]: Target 'all' not remade because of errors.
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
make: Target 'default_target' not remade because of errors.

ab3d6f6
[  3%] Generating manpage for sway.5
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
[  3%] Built target man-sway.5
[  7%] Generating manpage for sway.1
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
[  7%] Built target man-sway.1
[  7%] Built target man
Scanning dependencies of target sway
[ 11%] Building C object CMakeFiles/sway.dir/sway/container.c.o
[ 15%] Linking C executable bin/sway
[ 80%] Built target sway
[100%] Built target swaybg

You just need to update wlc.

@SirCmpwn Mind adding wlc-git to makedepends on the AUR? =)

Would that make it be rebuilt every time you build sway-git?

It doesn't make sense to have wlc-git in makedepends when it is already in depends. makedepends is only for packages which are not required at runtime. The user (or aur-helper) is responsible for rebuilding -git packages if required. I believe some aur-helpers can even auto-rebuild -git packages if they are told to do so.

Yeah, I'm not going to add it.

Oops, you're right; it doesn't make a difference.


[Todo] Sass color bug

One thing we could do is actually callout to the CLI and get the colored output that way, and simply forward stderr as an error for esbuild to catch.

This is a third-party thing. Technically we can always call out to the CLI and add --color.


TCP "too many open files" error

Running foursquare importer had errors (see the first few lines below) after that going to the UI resulted in the "too many open files" error in the server logs. Was working on the localhost.
First part (missing public key blob) is already reported as #700 and it might've triggered the "too many open files".
2016/04/24 16:16:11 error fetching public key blob sha1-fd55fb212e2da3e5265f2d28cc3f2a757e88dc68: blob was not fully indexed because of a missing dependency
2016/04/24 16:16:11 error fetching public key blob sha1-fd55fb212e2da3e5265f2d28cc3f2a757e88dc68: blob was not fully indexed because of a missing dependency
2016/04/24 16:16:11 error fetching public key blob sha1-fd55fb212e2da3e5265f2d28cc3f2a757e88dc68: blob was not fully indexed because of a missing dependency
2016/04/24 16:16:11 error fetching public key blob sha1-fd55fb212e2da3e5265f2d28cc3f2a757e88dc68: blob was not fully indexed because of a missing dependency
2016/04/24 16:16:11 error fetching public key blob sha1-fd55fb212e2da3e5265f2d28cc3f2a757e88dc68: blob was not fully indexed because of a missing dependency
2016/04/24 16:16:11 error fetching public key blob sha1-fd55fb212e2da3e5265f2d28cc3f2a757e88dc68: blob was not fully indexed because of a missing dependency
2016/04/24 16:16:11 Error fetching https://api.foursquare.com/v2/venues/4a7e524ef964a520c2f11fe3/photos: Get https://api.foursquare.com/v2/venues/4a7e524ef964a520c2f11fe3/photos?limit=5&oauth_token=2IMCPY0T43Q1OFDK3UBJ5CREUNGSU0H2UFESO4IASGKX2KNJ&v=20140225: dial tcp: lookup api.foursquare.com: no such host
2016/04/24 16:16:11 Foursquare importer: error importing photos for checkin 55ea3638498eee7b334ce640 Get https://api.foursquare.com/v2/venues/4a7e524ef964a520c2f11fe3/photos?limit=5&oauth_token=2IMCPY0T43Q1OFDK3UBJ5CREUNGSU0H2UFESO4IASGKX2KNJ&v=20140225: dial tcp: lookup api.foursquare.com: no such host
2016/04/24 16:16:11 Error fetching https://api.foursquare.com/v2/users/self/checkins: Get https://api.foursquare.com/v2/users/self/checkins?limit=100&oauth_token=2IMCPY0T43Q1OFDK3UBJ5CREUNGSU0H2UFESO4IASGKX2KNJ&offset=800&v=20140225: dial tcp: lookup api.foursquare.com: no such host
2016/04/24 16:16:11 foursquare importer account, sha1-5bc35bbf9ddf28b5236331a3469e2da87d56ff04 error: Get https://api.foursquare.com/v2/users/self/checkins?limit=100&oauth_token=2IMCPY0T43Q1OFDK3UBJ5CREUNGSU0H2UFESO4IASGKX2KNJ&offset=800&v=20140225: dial tcp: lookup api.foursquare.com: no such host
2016/04/24 16:18:27 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 5ms
2016/04/24 16:18:27 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 10ms
2016/04/24 16:18:27 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 20ms
2016/04/24 16:18:35 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 40ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 5ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 10ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 20ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 40ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 80ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 160ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 320ms
2016/04/24 16:18:51 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 640ms
2016/04/24 16:18:52 http: Accept error: accept tcp [::]:3179: accept: too many open files; retrying in 1s

I suspect ulimit maxfds is low.
"dial tcp: lookup api.foursquare.com: no such host" is probably also just out of filedescriptors.
We should make this easier to debug and show the ulimit somewhere from camlistored, and warn when it's too low.
I also suspect the real problem is foursquare is goroutine-bombing the API too widely and connecting lots of TCP connections due to lack of HTTP/2 on Foursquare's side. We should throttle concurrent requests, and verify we're also closing their bodies.


.Wait() and .WaitNot() methods wait "active" window incorrectly

Now these methods make no effect.
Steps:

Start calc.exe
Minimize it
run:

import pywinauto
import time
pwa_app = pywinauto.application.Application()
try:
    w_handle = pywinauto.findwindows.find_windows(title_re=u'\u041a\u0430\u043b\u044c\u043a\u0443\u043b\u044f\u0442\u043e\u0440',
                                                  class_name='CalcFrame',
                                                  active_only=True)[0]
except:
    print time.time(), "The window is not active!"

w_handle = pywinauto.findwindows.find_windows(title_re=u'\u041a\u0430\u043b\u044c\u043a\u0443\u043b\u044f\u0442\u043e\u0440',
                                              class_name='CalcFrame')[0]
print "window handle - %s" % w_handle
window = pwa_app.window_(handle=w_handle)
print time.time(), 'start waiting for active'
print "window handle after wait - %s" % window.Wait('active', timeout=5).handle
print time.time(), 'finish waiting for active'

In pywinauto 0.5.1 the result is:
1441459064.84 The window is not active!
window handle - 264032
1441459064.84 start waiting for active
window handle after wait - 264032
1441459064.84 finish waiting for active

As shown in the listing .Wait returned immediately however the calc app was not active.
The cause is in
pywinauto\application.py:

    289: def Wait
    330:     wait_criteria = self.criteria[:]
    358:     ctrls = _resolve_control(wait_criteria, timeout, retry_interval)

and
pywinauto\findwindows.py

    84:  def find_windows
    122:     if handle is not None:
    123            return [handle, ]

The window handle is copied into wait_criteria and translated as an argument into find_windows where it causes the return without any checks.
As for me, find_windows works as expected since this method should return handles of the windows which satisfy some criteria. If handle already exists, no checks needed.
Proposition is to extract the handle from wait_criteria and to verify whether the handle is in the _resolve_control(wait_criteria, timeout, retry_interval) response.

Decided to change approach in .Wail(). Now it work via _resolve_control-->_get_ctrl-->ctrl = controls.WrapHandle(findwindows.find_window(**ctrl_criteria)). This way much more slower than checking via appropriate method (e.g. IsVisible()). See the listing below:
window handle - 1574902
criteria - [{'handle': 1574902, 'visible_only': True}]
.IsVisible() - 0.082110420911
'visible_only': True + HwndWrapper + Findwindows - 2.05964695336
         1365751 function calls (1364035 primitive calls) in 3.040 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10002    0.386    0.000    0.888    0.000 HwndWrapper.py:111(FindWrapper)
   600315    0.343    0.000    0.343    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
    30287    0.324    0.000    0.331    0.000 handleprops.py:93(classname)
    20004    0.189    0.000    0.503    0.000 HwndWrapper.py:185(__init__)
    20004    0.103    0.000    0.956    0.000 win32_controls.py:830(__init__)

Agree. Attributes of existing window (visible, enabled, active) should be separated from search criteria itself. Probably even find_windows should take only search params in future.

Verified


If Resource uses bool for boolean it will convert param to string.

If the DSC Resource uses [bool] as the param type it will set the value as a string.

Sorry for all of the whitespace being removed. I can't seem to get VSCode to stop removing it. I have it disabled in settings yet it stills trims it on save.


chore: add local shellcheck tools

Resolve: https://github.com/envoyproxy/gateway/issues/314
Signed-off-by: bitliu<EMAIL_ADDRESS>

Codecov Report

Merging #521 (a2bd6f4) into main (5bf94c1) will decrease coverage by 0.93%.
The diff coverage is n/a.

:exclamation: Current head a2bd6f4 differs from pull request most recent head 5efdc93. Consider uploading reports for the commit 5efdc93 to get more accurate results

@@            Coverage Diff             @@
##             main     #521      +/-   ##
==========================================
- Coverage   63.48%   62.54%   -0.94%
==========================================
  Files          40       42       +2
  Lines        4425     4496      +71
==========================================
+ Hits         2809     2812       +3
- Misses       1471     1538      +67
- Partials      145      146       +1

Impacted Files
Coverage Δ

internal/status/status.go
0.00% <ø> (ø)

internal/xds/translator/runner/runner.go
0.00% <0.00%> (-86.67%)
:arrow_down:

internal/provider/kubernetes/gatewayclass.go
70.28% <0.00%> (-5.08%)
:arrow_down:

internal/provider/kubernetes/gateway.go
70.16% <0.00%> (-2.95%)
:arrow_down:

internal/provider/kubernetes/httproute.go
62.12% <0.00%> (-0.72%)
:arrow_down:

internal/cmd/root.go
100.00% <0.00%> (ø)

internal/gatewayapi/sort.go
76.74% <0.00%> (ø)

internal/cmd/versions.go
28.57% <0.00%> (ø)

internal/gatewayapi/translator.go
91.16% <0.00%> (+0.03%)
:arrow_up:

internal/gatewayapi/helpers.go
66.66% <0.00%> (+1.75%)
:arrow_up:

... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Thanks @LukeShu, updated based on your comments.


Docs: Fix auth challenge in rate limit preview

Proposed changes
The docs seem to have a mismatch between the given service and the actually necessary service in the auth challenge for previewing ones rate limits, for an anonymous user.
We rename the service in the token request to correspond to the service presented in the authentication challenge.
Related issues (optional)
Fixes #11660

ping @ingshtrom @arkodg PTAL; I think you were looking into this Yesterday

actually @binman-docker had created a PR yesterday for this: #11656
It so probably best we use that as it fixes an additional spot that I don't think this current PR catches.

perfect, I was indeed to lazy to check the user+password version although I suspected that it probably needed it, too... cheers!


Kill the process created by DataTarget.CreateSnapshotAndAttach

Closes #925

LGTM!  I should have a new version released early next week with this fix.

Shouldn't the Process object be disposed even though it's killed?

That's a good remark @ltrzesniewski! I discovered recently that Process  is disposable but I've never seen patterns where we dispose it like that:
using (Process.Start())
{
}

I guess we rely on the finalizer.


Fix tqdm threading

Fix tqdm for threading
Add customization for number of threads and batch sizes

Fix tests for pull request


Bug fix variable name error in sound player

This PR fixes a minor error in sound_player.
There is an error log event when a sound is encountered without a valid "track" value. The error log intends to include the sound's "action" attribute in the log output, but 8 lines earlier the action value was moved from an attribute to a variable:
action = s['action'].lower()
del s[action']

As a result, encountering the error log causes a KeyError that crashes MPF-MC.
This PR fixes the log output to reference action instead of s['action'].

Thanks!


Fix PRF stage typed stage name and abbreviation for IS type

closes #189

@mico I've moved the remaining task to #203. Merging this.


Update test command

Update one of the test commands to work in the latest version of conda / conda-build. This line was causing a failure because the _test environment no longer exists.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Needs to be re-rendered.

@gbrener we will be able to merge this only after conda-forge moves to conda-build 2.
The following packages will be DOWNGRADED due to dependency conflicts:

    conda-build: 2.0.1-py35_0 defaults --> 1.21.14-py35_0 defaults

Done in #12  Thanks @gbrener


fix: update vulnerable npm packages

Closes #208

:tada: This PR is included in version 0.17.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add piping support

I'd like to know if there's any interest in having imagecram support command line piping.
My main use case is to use imagecram in the following way:
$ git status -s "*.png" | awk '{print $2}' | imagecram

This pipe automatically finds all the uncommitted image files in a repository and then feed them to imagecram.
This is just an example, but there are many other similar use cases.
Solution
There are essentially two main ways to do this:
1. The automatic way
in main.swift we check whether there's any data in the standard input buffer and, if there is, we parse it and feed it to
ImageCramCommand's main(_:).
I have a working implementation of this in my stream-input branch, you can see all the changes in this commit.
2. The flag way (similar to SwiftLint)
We introduce a new @Flag telling ImageCramCommand to use the standard input instead of reading the inputs @Argument.
Please let me know what you think! If there's interest, I'd be happy to send a PR for this :)
Checklist

[x] I've read the guide for contributing.
[x] I've checked there are no other open pull requests for the feature.
[x] I've checked there are no other open issues for the same feature.

Additional context
Thank you very much for building this very handy tool ❤

Thanks, really glad someone is using the tool and finds it helpful!
This feature sounds like a great idea, I like the 'automatic' way from above, it seems intuitive, what I would expect and works like other standard system tools.


Thank you! I will update this file recently!


Socket or port exhaustion when making fast RPC calls to bitcoin core

JsonRpc(Transport(SocketError(Error { kind: TimedOut, message: "connection timed out" })))
This is due to using too many sockets or ports. Probably ports, specifically local ephemeral ports.
This is the rust-jsonrpc issue: https://github.com/apoelstra/rust-jsonrpc/issues/67
Need to reuse the socket, or add hyper transport to rust-jsonrpc.

Potentially explore moving to https://github.com/paritytech/jsonrpsee

maybe try again with this fix: https://github.com/apoelstra/rust-jsonrpc/pull/72


Update WSF Ambition 7 numbers

These documents cited 130,139 articles in total. 1.4% of these (n = 1778) were Wellcome research. However, some individual documents contained a much higher percentage of Wellcome research. For example, in the 2012 WHO technical report ‘Research priorities for helminth infections: technical report of the TDR disease reference group on helminth infections’ there were 539 references cited, 11.7% (n=63) were Wellcome research.

Marta is asking an update on those numbers and I am not 100% sure how to replicate them.
When I run this into the db
select
	count(distinct file_hash)
from document;

I get 5,228,096 which is far from the previous number. Did we filter the references somehow before reporting?
Also how did we calculate the last number?

btw I assume this has an error in it, because 5,228,096 is the number of:
select
	count(distinct id)
from document;

not distinct file_hash which is 5124.
The only things I can think is

the number has genuinely increased loads since other parts of the algo have improved (not sure what data I got this 130,139 number on).
at some point we did have duplicate policy documents in the db (see https://github.com/wellcometrust/policytool/issues/48 and https://github.com/wellcometrust/policytool/issues/104), so there is a chance of the same doc and reference repeating in the db

I did:
select
	count(distinct concat(file_hash, '-', title,'-', author,'-', journal,'-', pub_year,'-', issue,'-', volume))
from document;

and got 1,131,488 counts. Which might mean issue 104 is still at play (duplicated policy docs).
Thus I think there is still an error somewhere with duplicated data, and thus each row of document not truly being a unique reference from a policy document.

The decision has been not to use the database. More information can be asked to @hblanks.


Not symmetric view of the modal window if selected address is very long on New spending limit

Description
Not symmetric view of the modal window if selected address is very long on New spending limit
Environment

Browser: Chrome
Wallet: MetaMask
Chain: Polygon

Steps to reproduce

Go to Polygon
Try to add new spending limit
select long address ( matic:0x448E4E6DD37523A5c7804D75a67754c24bFbFb32)

Expected result
Make sure that the modal window has symmetric view when the address is long
Screenshots

This is also an issue on smaller screens in general.

Hey @liliya-soroka
Please write to Support : Polygon support requests  on the livechat page for swift assistance.


Use requests

Resolves #232.
I removed all references to urllib.request. I tried to test everything manually as I changed them. Hopefully we'll have less SSL errors now that we use requests' certs.

This actually won't help much, because sanction uses urllib, and that's the source of most of our issues

Can we switch oauth libraries to something that does requests (requests-oauthlib perhaps?) or fork sanction to use requests?


wasm-encoder: support extended constant expressions

I've noticed that #687 added the groundwork for implementing the extended constant expressions proposal, but wasm-encoder still does not support them. Are there any plans for this, and if so, when can I expect them to be available? This isn't a pressing issue for me, hence why I don't open a PR at this point, but I wanted to check in on this.

As far as I know, no one has plans to implement extended const expressions for wasm-encoder at this time. I'm sure it will happen eventually, but that could easily be in another year or so.
If you want it badly enough to implement it, I'm happy to point you to the relevant parts of the crate, answer questions, and review PRs.

I presume the solution would just to add some functions which encode the relevant instructions to ConstExpr. Would the functions just look as such?
impl ConstExpr {
    fn i32_add(self) -> Self { /* ... */ }
}

Hmm, avoiding breaking changes would be difficult: the current i32_const & co. functions have the signature
fn i32_const(value: i32) -> Self;

But in order to have multiple i32.consts (e.g. in a (i32.add (i32.const 0) (i32.const 1)) const expr, which is allowed by this proposal), it must have the signature:
fn i32_const(self, value: i32) -> Self;

Should I make this breaking change, or simply add with_i32_const(self, value: i32) -> Self functions? (bikeshed: naming)


man-page: document range values may be zero. Change to say non-negative instead of positivie.

The START value to the -r option may be zero, (as well es the END/COUNT, though that might not be as useful).
The man page now states that more precisely by saying non-negative instead of positive.

Must've been an oversight when writing the manpage, thanks!


how can i use this model?

hi，I'm very interesting in your job,but i don't know how to use this model in my dataset,can you tell me some detail.Very thanks for your answer.

Hi, can you run this project right now?

I've recently added a feature for training the model on a custom dataset. See README.

and yes, it should run.


Suggestions: enable using publications and meteor methods

Not sure if this is possible, but to have more DRY code it would be useful to be able to

turn a Meteor Publication into an API.
Because in a publication you already limit what you publish of a collection. There's a big chance your API exposure won't differ, but that means duplication of code in endpoint actions.

2. use meteor methods in endpoint actions
Because you already have your methods set up to do something with your collections. So if you could use those it means you don't have to duplicate code.
UPDATE: just figured out that is possible, with the exception that Meteor.userId() is not set. So if you protected your methods with check for userId the API can't run the method.

Hey Satya,

This is definitely something that needs to be added to Restivus. I'm sure you're already aware of the http-publish package that has this functionality. I have a few other things that are higher priorities for me (e.g., additional testing, route/endpoint versioning, and JSON PATCH support), but if someone wants to add it I will gladly work on hashing out an acceptable API with them or I will happily accept a pull request if someone would like to be extra ambitious. I just know that it's going to take a couple of weeks of work to design and implement a clean API that plays nicely with everything else, and I need those other features I named for my own API asap. I went ahead and added the feature to the roadmap, but it may be more than a month before I get around to implementing this.

I've used Meteor methods in multiple endpoints, including authenticated endpoints. In my situation, I'm calling through to the method from within the endpoint. Are you talking about just applying the entire Meteor method as the endpoint function? That I haven't tried. If there is an easy way to add this to the API I will do it, but otherwise it will probably be fairly low priority since it's probably only saving a single line of code to call through to the method from within the endpoint (if I understand you correctly). Here's an example of what I do that works with authenticated endpoints:

Restivus.addRoute 'test', authRequired: true,
  get: ->
    Meteor.call('someMeteorMethod', arg1, arg2)

Working on a package that has this as its main feature here: https://github.com/stubailo/meteor-rest
Hoping to have a first version out on Tuesday, but until then here is a hackpad with design and discussion: https://meteor.hackpad.com/Meteor-Hack-Week-REST-APIs-with-Meteor-XK2NNXqhUvj

@stubailo's simple:rest package (referenced above) is designed specifically for this, so I have no plans to provide any additional functionality for it in Restivus. I'm open to debate as always, but I'm going to go ahead and close this for now.


Allow job classes to specify a queue to push onto

It’s a void 😜

From: Daniël Klabbers @.>
Sent: Thursday, March 3, 2022 8:56:40 PM
To: blomstra/flarum-redis @.>
Cc: Ian Morland @.>; Author @.>
Subject: Re: [blomstra/flarum-redis] Allow job classes to specify a queue to push onto (PR #12)
@luceos commented on this pull request.

In src/Overrides/RedisQueue.phphttps://github.com/blomstra/flarum-redis/pull/12#discussion_r819056340:

+namespace Blomstra\Redis\Overrides;

+use Illuminate\Queue\RedisQueue as IlluminateQueue;
+
+class RedisQueue extends IlluminateQueue
+{

/**

*<EMAIL_ADDRESS>

*/

public function push($job, $data = '', $queue = null)
{

   if ($job->queue && !$queue) {

       $queue = $job->queue;

   }

   parent::push($job, $data, $queue);

Should this method return the parent method response?
—
Reply to this email directly, view it on GitHubhttps://github.com/blomstra/flarum-redis/pull/12#pullrequestreview-899505987, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD6OIOHWI3H4VEOAVHSGECLU6ERQRANCNFSM5P3IWGAQ.
You are receiving this because you authored the thread.Message ID: @.***>


add upzmg license

Add license for Universidad Politecnica de la Zona Metropolitana de Guadalajara

No comments

@Palomino0513 First stage of verification passed, official domain is upzmg.edu.jalisco.gob.mx as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://upzmg.edu.jalisco.gob.mx/directorio

@Palomino0513 Thank you for your patience while waiting for a review and processing your request. Pull request merged.


i got two different version of Link.js warning

Describe the bug
i have this warning after i installed next-i18next
warn  - ./node_modules/next/Link.js
There are multiple modules with names that only differ in casing.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Use equal casing. Compare these module identifiers:

here i got two different versions of Link.js, one from next and on from your package.

Occurs in next-i18next version
"next-i18next": "^4.5.0",
OS (please complete the following information)

Device: [e.g. MBP 2017 15"]
Browser: [e.g. Chrome ]

Sorry, I don't understand what you mean. Can you explain further, or provide a reproducible repo?

so, i installed the next-i18next package and configured and when i run yarn dev i receive a warning in the console.
the warning is this:
warn - ./node_modules/next/Link.js
There are multiple modules with names that only differ in casing.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Use equal casing. Compare these module identifiers:

/path_to_app/node_modules/next/Link.js
Used by 1 module(s), i. e.
/path_to_app/components/navigation/desktop/Desktop.js
/path_to_app/node_modules/next/link.js
Used by 1 module(s), i. e.
/path_to_app/node_modules/next-i18next/dist/commonjs/components/Link.js

Take a look in path_to_app/node_modules/next. There is no Link.js file. It's link.js.


[Enhancement] Support multi-part GRF generated by COLOCATE/SHUFFLE_HASH_BUCKET HJ with only one eqConjunct

What type of PR is this：

[ ] bug
[ ] feature
[x] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #
https://github.com/StarRocks/starrocks/issues/6372
Problem Summary(Required) ：

COLOCATE/SHUFFLE_HASH_BUCKET HJ does not produce global runtime filters, this PR support these HJ with only one eqConjunct generated a shuffle-aware(multipart) GRF. In the future, partition_by_exprs will be introduced to support multipart GRF generated by HJ with mutiple eqConjuncts.
When evaluating a GRF generated by COLOCATE HJ; at first,  crc32_hash is used to compute a hash_value, then it is modulo-ed by num_buckets of colocate group to compute the bucket_seq, a bucket_seq to fragment instance ordinal mapping is used to translate bucket_seq to partition index of multipart GRF, then the part of GRF indexed at partition index is probed.

[FE PR Coverage check]
:disappointed: fail : 16 / 43 (37.21%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/qe/Coordinator.java
8
31
25.81%

:large_blue_circle:
com/starrocks/planner/RuntimeFilterDescription.java
7
11
63.64%

:large_blue_circle:
com/starrocks/planner/JoinNode.java
1
1
100.00%

[FE PR Coverage check]
:heart_eyes: pass : 39 / 44 (88.64%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/qe/Coordinator.java
24
28
85.71%

:large_blue_circle:
com/starrocks/planner/RuntimeFilterDescription.java
15
16
93.75%

[FE PR Coverage check]
:heart_eyes: pass : 70 / 76 (92.11%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/qe/Coordinator.java
24
28
85.71%

:large_blue_circle:
com/starrocks/planner/RuntimeFilterDescription.java
15
16
93.75%

:large_blue_circle:
com/starrocks/planner/JoinNode.java
31
32
96.88%

[FE PR Coverage check]
:heart_eyes: pass : 70 / 79 (88.61%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/qe/Coordinator.java
24
28
85.71%

:large_blue_circle:
com/starrocks/planner/JoinNode.java
31
35
88.57%

:large_blue_circle:
com/starrocks/planner/RuntimeFilterDescription.java
15
16
93.75%

@Mergifyio rebase

[FE PR Coverage check]
:heart_eyes: pass : 70 / 79 (88.61%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/qe/Coordinator.java
24
28
85.71%

:large_blue_circle:
com/starrocks/planner/JoinNode.java
31
35
88.57%

:large_blue_circle:
com/starrocks/planner/RuntimeFilterDescription.java
15
16
93.75%


Allow to extend BaseAI

I'm submitting a ...
[ ] bug report
[X ] feature request
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Summary

We have a special AI client that customizes existing AI like Anthropic, Bedrock and OpenAI. In order to use llm-client with them, it helps to build on top of llm-client and requesting to make it possible.

BaseAI - https://github.com/dosco/llm-client/blob/main/src/ai/base.ts#L43 OR
a better alternative.

I'd suggest just extending BaseAI to add your own ai client. Maybe I'm not understanding this well enough. Feel free to msg on our discord.


ci-automation/vendor-testing/vmware.sh: Fix unbound variable use

This gets triggered when the test is rerun and an existing image is reused.
Spotted here: http://jenkins.infra.kinvolk.io:8080/job/container/job/test/3376/console

Needs backporting to 3227 3346 3374 3033 lts-2605.


TransposeConv with Bias

Description of issue:
For generating .tflite file with TFLiteConverter, when model contains
Conv2DTranspose layers, bias cannot fold into Operator TRANSPOSECONV.
It will result with extra Op ADD following Op TRANSPOSECONV.
But with other CONV-like layers (Conv2D, DepthwiseConv2D),
bias will be fold into CONV layer.
(check detailed TF issue: https://github.com/tensorflow/tensorflow/issues/34622)
PR try to resolve it:
So we try to resolve this issue by enable TransposeConv with bias for TFLite:

Update TFLite graph_transform features with:
fill TransposeConv bias with zero if there is no bias
fuse bias add into preceding TransposeConv(TEST added)

Update TransposeConv with bias:
add bias input to TransposeConv
add optional bias to TransposeConv kernels

example of the results:
TRANSPOSE_CONV inputs:

output_shape
weights
activation
bias

Need to discuss:
currently this PR only update reference kernel for transpose_conv, optimised kernal is commented out.
several TEST need to be added as well, but further suggestions are needed for adding additional test.

Hi, thanks for the proposed CL. Until we have an optimized version of the fused bias, I'm not sure this is worth it. Whenever we change op semantics like this (e.g., adding a new attribute, or a new fused input), it impacts all delegates which support a given op. So we would also need to update our nnapi_delegate, gpu_delegate, etc... instances to handle the fused bias.
That may not be a bad thing, but I wonder how much that will really buy us in terms of performance? Have you measured the difference? transpose_conv isn't as common as those other operators, so extracting the absolutely maximal performance (like fusing the bias) isn't quite as critical.

@psunn Could you please check jdduke's comments and keep us posted. Thanks!

@jdduke thanks for quick replying.
@gbaned thanks for remind me to reply.
I was in a week-long conference, sorry for late reply.

Until we have an optimized version of the fused bias, I'm not sure this is worth it.
I'll update code patch, as soon as I can.

That may not be a bad thing, but I wonder how much that will really buy us in terms of performance? Have you measured the difference?
I meet TransposeConv with Bias when I playing with FSRCNN, so I will try to measuring regarding this model.

Until we have an optimized version of the fused bias, I'm not sure this is worth it.

I've updated the patch with optimized Ops. Please have a review.

So we would also need to update our nnapi_delegate, gpu_delegate, etc... instances to handle the fused bias.

Looks like TransposeConv for nnapi [1] and gpu_delegate [1] has bias inside, I think it will be a minor update.

So we would also need to update our nnapi_delegate, gpu_delegate, etc... instances to handle the fused bias.

Looks like TransposeConv for nnapi [1] and gpu_delegate [1] has bias inside, I think it will be a minor update.

I think it's much easier from your side to push the changes regarding delegates, do you think so?
If it's must be pushed from us, shall we resolve it in a follow-up&separate PR, what do you think?

I've update the patch with new test, please have a review.

Thanks for review!

2\. add an entry in lite/tools/versioning/op_version.cc for the new op version, and set it to kPendingReleaseOpVersion

I guess you mean lite/toco/versioning/tflite/op_version.cc?
In lite/toco/versioning/tflite/op_version.cc, shall I add as:
{{OperatorType::kTransposeConv, 1}, "1.9.0"},
{{OperatorType::kTransposeConv, 3}, kPendingReleaseOpVersion}

or  {{OperatorType::kTransposeConv, 2}, "x.x.x"} is also needed?

Good catch, we should add {{OperatorType::kTransposeConv, 2}, kPendingReleaseOpVersion} as well, thanks!

I've update the patch, please have a review.

Can we merge this? Will add test for post-training quantization tooling in separate PR.

Hi, any chance to merge this PR?

Sorry for the delay, there are some internal conflicts that we have to resolve manually, hoping to resolve in the next day or two.

Hi @jdduke @suharshs , can you please review and hopefully re-approve please. I rebased this PR on latest master and resolved conflict. Thanks.

Hi peng, this is messing with some internal tests that I am cleaning up.
Should be merged once that is done. Thanks!
On Fri, Feb 21, 2020, 3:37 AM Peng Sun<EMAIL_ADDRESS>wrote:

@psunn https://github.com/psunn requested your review on: #34903
https://github.com/tensorflow/tensorflow/pull/34903 TransposeConv with
Bias.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/pull/34903?email_source=notifications&email_token=AALCE5VSSOH4AFOXDJZZOFTRD64I3A5CNFSM4JW2PHR2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOWZTNKEA#event-3060192528,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALCE5WUZTP5HU3MRPYEVXTRD64I3ANCNFSM4JW2PHRQ
.

Hi @jdduke @suharshs ,
Is it ok to re-approval this PR?
Hopefully merged soon, if no issues.

I am still working through some internal tests that are failing. Will merge
the pr once those are resolved.
On Wed, Mar 18, 2020 at 3:49 PM Peng Sun<EMAIL_ADDRESS>wrote:

Hi @jdduke https://github.com/jdduke @suharshs
https://github.com/suharshs ,
Is it ok to re-approval this PR?
Hopefully merged soon, if no issues.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/pull/34903#issuecomment-600897142,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALCE5R57S2KDJJSB4USZGDRIFFWFANCNFSM4JW2PHRQ
.

Just an update here: there is an issue i am running into with flatbuffer import in the mlir code path. I am trying to get that fixed, but the optional tensor support has trickled into a few things, will update here when that is resolved. thanks for your patience.

No that's ok, was just wondering. Thanks
On Fri, Apr 3, 2020, 1:43 PM Peng Sun<EMAIL_ADDRESS>wrote:

@psunn commented on this pull request.
In
tensorflow/lite/toco/graph_transformations/fuse_binary_into_preceding_affine.cc
https://github.com/tensorflow/tensorflow/pull/34903#discussion_r403320402
:

 AddMessageF(

     "Not fusing %s because the preceding %s is not of one of the supported "
     "types",
     LogName(*binary_op), LogName(*preceding_op));
 return ::tensorflow::Status::OK();

}

if (preceding_op->type == OperatorType::kTransposeConv &&

I was intend to fused kAdd to kTransposeConv.
Remove if statement, kMul, kSub, kDiv may also be fused to kTransposeConv,
which is also good for graph optimise.
Shall I remove it?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/pull/34903#discussion_r403320402,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALCE5WP7QBYYUXWYQCPRZLRKZC6VANCNFSM4JW2PHRQ
.

@psunn Could you please resolve the conflicts? Thanks!

@psunn Could you please resolve the conflicts? Thanks!

@gbaned done, thanks for the reminder.

Seems auto-merge is not happening but the changes are now committed so we can close this. Thank you for the PR.


Multiapi fixes

@fearthecowboy

Automation for azure-sdk-for-js

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-python

Can one of the admins verify this patch?

Automation for azure-sdk-for-go

Automation for azure-sdk-for-java


Adding Pagination in book-finder

Is your feature request related to a problem? Please describe.
Hi there !!
I want to contribute to this project by Adding pagination the list .
When we are searching book then it shows up the list but it ends upto infinite list.
So I will add pagination to it so that only specific number of list items will be shown like 5-7 etc.
Please assign me under JWOC
Describe the solution you'd like
When we are searching book then it shows up the list but it ends upto infinite list.
So I will add pagination to it so that only specific number of list items will be shown like 5-7 etc.
Describe alternatives you've considered
When we are searching book then it shows up the list but it ends upto infinite list.
So I will add pagination to it so that only specific number of list items will be shown like 5-7 etc.
Additional context
No response

I want to work on this under JWOC

@Jagroop2000 what's the update?

@Jagroop2000 what is the update? if you aren't working on this, will assign it to @deepbeatz

@wasimreja
I would like to work on this under JWOC.
Please assign it to me.


SCIM groups created for a secondary user store does not get deleted when deleting the user store

Add a read write secondary user store.

Create a SCIM group in the secondary user store via SCIM API.
curl -v -k --user admin:admin --data '{"displayName": "EXTERNAL/engineer"}' --header "Content-Type:application/json" https://localhost:9443/wso2/scim/Groups

Delete the user store

Retrieve groups from SCIM
curl -v -k --user admin:admin https://localhost:9443/wso2/scim/Groups

The group created for the secondary user store is still available. But that cannot be deleted as the user store is not found.
{"Errors":[{"code":"500","description":"Cannot delete group through SCIM. SCIM is not enabled for user store EXTERNAL"}]}

Expected behavior:
It's expected that while the user store is being deleted all data related to that user store to be cleared as well. But in this case, some stale data will remain even the user store is not available.
Moved from https://wso2.org/jira/browse/IDENTITY-5832

This issue is reproducible on IS 5.8.0 as well.

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Induct Cancel on UI

Allows invitee to cancel an induction
Adds a Cancel induction on the UI for Invitee, Inviters or Endorsers when the induction is expired
Solves #232

I'll open an issue for this, and these can be tackled later:

An inviter may also want to cancel a non-expired induction if one of the witnesses is not available or not responsive. Or if they're an initial Genesis member and have reached their 15-induction limit.
There may be a use case for an invitee or witness to cancel too. We should figure out how to expose that.
It'd be good if this error message tells the user they should cancel the existing on first and try again.


!fix(share/eds): use separate badger for inverted index

Having same badger instance for dagstore shards and inverted index has proven to have hangup on Put operation for 1-10min. During tests it has shown, that splitting badger into 2 instances solves the hangup issue.
Also once https://github.com/celestiaorg/celestia-node/issues/2479 is implemented and inverted index will needs to be removed. Separate badger will be much easier for users migration.

Considering inverted index is major bottleneck right now an we plan to remove it soon. I think it is worth to do separate badgers just for sake of easy migration.
If we keep inverted index in same badger as shards badger, we would need to manually remove each key. It would mean to find all keys by prefix and remove them one by one, which will take ages.
I've started test to try 20 compactors on single badger

Agreed

We still may wanna keep all the storage breaking PRs together


Upgraded everything to RTM

.NET Core 1.0.0 + preview2 tooling

Also ASP.NET MVC now requires netstandard1.6


feat(kong): support configurable kgo deployment selector labels

What this PR does / why we need it:
We are currently deploying KGO through kustomize and are trying to migrate to helm with zero downtime. One issue I've seen is that the deployment selector labels have changed. Since that is an immutable field it causes issues with the helm deploy of the gateway operator. This PR adds the ability to override the default selector labels.
Special notes for your reviewer:
Checklist

[x] PR is based off the current tip of the main branch.
[x] Changes are documented under the "Unreleased" header in CHANGELOG.md
[ ] New or modified sections of values.yaml are documented in the README.md
[x] Commits follow the Kong commit message guidelines

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Headscale stops accepting connections after ~500 nodes (likely 512) (0.23-alpha2)

Bug description
I have a large headscale instance (~550 nodes). 0.22.3 has extreme CPU usage (i.e., 30 cores), but is able to handle all clients. 0.23-alpha2 has substantially better CPU performance, but stops accepting connections after about 500 nodes. CLI shows "context exceeded" for all queries (e.g. "headscale nodes/users/routes list") and new clients are unable to join.
CPU usage after connections stop is relatively modest (<50%), and connected clients appear to be able to access each other (e.g. ping/login) as expected.
Environment
Headscale: 0.23-alpha2
Tailscale: various, but mostly 1.54+, mostly Linux, some Mac and Windows.
OS: Headscale installed in privileged Proxmox LXC container (Ubuntu 20.04.6), reverse proxied behind nginx per official docs
Kernel: 6.2.16
Resources: 36 cores, 8GB RAM, 4GB swap (ram usage is quite small)
DB: Using postgres backend (sqlite would die after about 100 clients on 0.22.X with similar symptoms)
Config: ACLs are in use based on users. Two users have access to all nodes. Most nodes have access only to a relatively small set of other nodes via ACL. Specifically, there are 7 nodes that have access to everything, but on almost all other nodes "tailscale status" will return only 8 devices.

[X] Headscale is behind a (reverse) proxy
[X] Headscale runs in a container

To Reproduce
I suspect difficult to do, but have large numbers of clients connect in.
Given that the behavior is a direct change from 0.22.3 -> 0.23-alpha2, I don't think the container or reverse proxy have anything to do with it. I can with some effort bypass the reverse proxy, and do a direct port-forward from a firewall instead, but running on bare metal will be substantially more difficult.
Because I'm in container, I can easily snap/test/revert possible fixes.

@jwischka Have you made sure you are not hitting your file descriptor limits.
If ulimit -n is still 1024, (awfully close to double when your issues start to arise)
try to raise it with ulimit -n 4096 and see if that fixes this issue

@jwischka would you be able to test this with current version of main?

@jwischka just released a new alpha4, please give that a go https://github.com/juanfont/headscale/releases/tag/v0.23.0-alpha4 and report back.

Yep, on it, it was a regression in a upstream dependency, see #1755

Could you please test if this is still the case with https://github.com/juanfont/headscale/releases/tag/v0.23.0-alpha5 ?
The postgres issues should now be resolved.

Thank you, could you elaborate on how much way up in terms of memory is? I would expect it to be up as we just keep more stuff around in memory but still nice to compare some numbers

@jwischka I've started some experimental work in https://github.com/juanfont/headscale/pull/1791, which should both improve performance, and add some tunables for high traffic usage, but it isnt done yet so I would not recommend trying it in a prod env. If you have a non-prod env that is similar, you can.

I'm looking to run a large cluster in about a year or so and I can contribute dev time for this effort. Any thing I can help with?

I can't find release binaries for alpha6 so I'm running alpha7. I'm already seeing reduced memory usage a couple of hours in. Anything in particular I should look for?
I see new log lines talking about partial updates.

Sorry, 6 quickly got replaced with 7 because of an error. In principle I would say that the "nodes stays connected over time" would be the main goal, that none of them loose connection. The main change for performance and resource usage is a change in how the updates are batched and sent to the clients.

There are two new undocumented tuning options (please note the might change and/or removed) which could helps us figure out maybe the holdup:
https://github.com/juanfont/headscale/blob/main/hscontrol/types/config.go#L235-L236
The first is the max amount of time it waits for updates before bundling them and sending them to the node.
The seconds is how many updates a node can receive before it will block.
Can you experiment with these numbers?
I would not set the wait for higher than 1s, but lower will flush to client more often. I would expect lower wait to be lower memory, higher cpu.
For the batcher,you might see a super tiny increase in memory, but I doubt it as it's mostly just more ints allowed on a channel, but you could try that for quite some different large numbers and report back: 100, 300, 500, 1000 even.
It would help if you could try different combinations for these and try to make a table with your observation, it's hard for me to simulate environments like yours.
Thanks.

I believe fixes in https://github.com/juanfont/headscale/releases/tag/v0.23.0-alpha12 should resolve this issue, let me now if not and we will reopen it.


invalid 'in' operand a (Selenium::WebDriver::Error::JavascriptError)

error msg: invalid 'in' operand a (Selenium::WebDriver::Error::JavascriptError)
I was trying to get the html source within a web element so I defined the locator in lets say (xyz) and then called xyz_element.html to get the html source.
And I am getting the above error msg.
It was working before and just started seeing this error msg from last week. Any idea?
I was able to find out a workaround so anyone with similar issue can do the following:
src = xyz_element.attribute('innerHTML')

Are you using Firefox? My guess is that these problems started happening last week after upgrading to Firefox 35.0. There have been various JavascriptError issues reported for Selenium-Webdriver with Firefox 35.0. At the moment, I believe the only solution is to downgrade your version of Firefox.

I can confirm that the problem is in FF 35.0. Downgrade to the previous FF version is the simplest way to solve it.
You can find one of the previous Firefox versions in your local cache, for example:
  apt-cache showpkg firefox

and then install it from cache:
  $sudo apt-get install firefox=28.0+build2-0ubuntu2

yes, i am using ff 35

+1

Was this just an issue with FF 35?

I'm using FF 35.0.1 I got the same issue. "Selenium::WebDriver::Error::JavascriptError: invalid 'in' operand a"
In addition to the issue reported above I got below error message when I'm trying to access the parent node of a web element
Code block:
element.parent.attribute('class').include?('selected')
Error Message:
"Selenium::WebDriver::Error::Javascript Error: a is undefined"

Also is it in the latest version of selenium-webdriver 2.45?

Has this issue been resolved or is it still occurring?

I am going to close this since it seems to have passed. Feel free to re-open the issue if you are still experiencing this issue.


CauldronLevelChangeEvent not firing by rain

Timings link
https://timings.aikar.co/?id=4af1514a774e4258b4d1a06f3a597356
Expected behavior
When it rains CauldronLevelChangeEvent should be fired with NATURAL_FILL
Observed/Actual behavior
But acctually its not in latest builds. On git-Purpur-1574 it works, but on git-Purpur-1603 it doesnt
Steps/models to reproduce
Make it rain, place cauldrons, wait for them to fill and check if CauldronLevelChangeEvent is fired
Purpur version
[21:29:58 INFO]: Current: git-Purpur-1574 (MC: 1.18.2)*
[21:29:58 INFO]: Previous: git-Purpur-1603 (MC: 1.18.2)
[21:29:58 INFO]: * You are 29 version(s) behind
[21:29:58 INFO]: Download the new version at: https://purpurmc.org/downloads
Agreements

[X] I am running the latest version of Purpur available from https://purpurmc.org/downloads.
[X] I have searched for and ensured there isn't already an open issue regarding this.
[X] My version of Minecraft is supported by Purpur.

Other
No response

Nvm, i've checked and its beeing reproduced on paperspigot so im moving it there


Publish latest?

It looks like #228 added what I was wanting, but that the latest of the https://www.nuget.org/packages/CacheManager.SystemRuntimeCaching was published the year prior.

CC @MichaCo

Could we have a release with this feature soon?  I've got a situation where a flood of requests causes the MicrosoftMemoryCacheHandle to use all the memory allocated to a container and it crashes.  I need a way to preemptively flush old cache items rather than just crashing with an OOM exception.  Sounds like this feature is exactly what I need.
@MichaCo
Thanks

Yes, please publish the latest.


Bash shell integration -- exit code is not restored when PROMPT_COMMAND is an array

This solves issue #158090 "Bash shell integration -- exit code is not restored when PROMPT_COMMAND is an array".
Note: this fix also works when PROMPT_COMMAND is not an array, because in BASH you can apply the array syntax on single (i.e. non-array) variable.
Example :  "$xxx" == "${xxx[0]}" whatever the type of xxx: a single variable or an array.

@CyrilOtheninGirard thanks for working on this. Using the config I describe here https://github.com/microsoft/vscode/issues/158090#issuecomment-1270195748, I get the same incorrect result with your changes.


[InfluxEnterprise] Document hostname lowercase restriction

As of v1.2 hostnames for InfluxEnterprise meta and data nodes MUST be lowercase. (p928).

Thank you for letting me know, @kostasb!


Check for doctrine migration

New check to make sure all migrations are applied if you use doctrine migrations.
I had to use mocks for the tests because otherwise a database connection is required.
Note: the doctrine/migrations library does not really maintain tags this is why I used dev-master as dev dependency.

other than that minor comment looks good to me.
@Thinkscape ?

@lsmith77 pushed.
And seems like the tests are failing on master too due to the same segfault.

hmm .. I somehow seem to remember having seen similar failures in a different repository as well.

Looks to be the same segfault as #52 - we need to figure out what test is causing it.

ah damn. so it wasnt a different repo. can one of you adjust the travis.yaml to run phpunit in verbose mode so that we can see which test is causig this?

@lsmith77 pushed the verbose flag. I also skipped 2 tests which were in the area where I saw the segfaults. But there is something really weird, different php versions fail on different tests.

argh sorry .. we also need --debug

@glaubinix would be so kind and fix indentations ?

@Thinkscape are you talking about the 2 lines where I marked the tests as skipped. This PR should not be merged like this. I first need to remove all the debugging again. But I am not sure where we are at with the failing build.

Ah, this https://travis-ci.org/zendframework/ZendDiagnostics/jobs/59508302#L335 ?

@Thinkscape yes, but different PHP Versions throw the segfault for different tests.

Tests seem to run locally, only difference is XDebug. Could that cause it?

likely ..

@lsmith77 Confirmed in #59 — xdebug causes the failures.

please rebase

rebase done


cmake -G Ninja fails on windows runner if CMakeLists.txt includes add_subdirectory()

First, thanks for this great github action!
Second, I've been trying to get a windows runner to work with add_subdirector() in CMakeLists.txt, and have not been able get things to work with the Ninja generator. I've distilled things down to a minimal setup here: https://github.com/ugermann/run-cmake-fails-on-windows-runner/runs/2453647696?check_suite_focus=true. The problem is in line 34 of the output. Path prefix and the added subdirectory are concatenated in the wrong order.
I've been battling this for a few days now and have run out of ideas to try. Any suggestions?

@ugermann thanks for setting up a repro of the problem. Could you please re-run the same workflow with ACTION_STEPS_DEBUG set to true? The log of that run may shed some light about this.
Please compare your workflow with the one that validates the run-cmake action, here the last run for the 'basic scenario': https://github.com/lukka/CppBuildTasks-Validation/runs/2407441272
A notable difference is the usage of recent CMake, while your workflow is using whatever is installed already on the runner, which may be a little older and not compatible with the most recent Ninja. It should be possible to reproduce the same problem on Windows using the same version of CMake and Ninja.
Also, you may try using the get-cmake@latest action and see if using a recent CMake is working around this problem.
Let me know!

@lukka the problem is also reproducible natively on Windows with whatever cmake version the latest visual studio and qtcreator ship. Using an altenrative generator eg -G "Visual Studio 16 2019" -A x64 avoids the issue.

@ugermann thanks for enabling debugging on the workflows.
Looks like the problem is manifesting when setting the OUTPUT_NAME target property in the CMakeLists.txt to a specific value. A ticket on CMake looks like worth in this case.
Let me know if there is anything that can be done on run-cmake side of things, thanks.

Hi @lukka,
thanks for looking into this and your follow-up. I was planning to file an issue for cmake but it took me a while to figure out why my parameterization (for demo purposes) via CMAKE variables didn't work on the runner. Lesson learned: cmakeAppendedArgs is ignored by CMakeListsTxtBasic. I'm fairly new to github workflows and still feeling my way around them. I have a working minimal setup now that works for bug reporting.
Again, thanks for your help!

The issue has now been reported here: https://gitlab.kitware.com/cmake/cmake/-/issues/22133


vManageSession object has no attribute _platform_version

Summary
vManageSession object has no attribute _platform_version
Logs and external information
No response
Reproduction script
Establish session on newly deployed vManage.
On what platform(s) does it occur?
Not Applicable
Issue Severity
High: Blocking everything

This issue was migrated from an old repository: https://github.com/CiscoDevNet/catalystwan/issues/286


alias table will be clear after restart the hubot

Hi There,
Can I export the alias table somewhere to avoid the alias table clear after hubot restarted?

@garumaru have you been able to resolve this issue? Im facing the same problem and dont know how to fix it.

Hi @garumaru, I also meet the issue when hubot restart, I use  "hubot-redis-brain" module to store the alias and after restart it still exist. You can try it.
CentOS step
yum install epel-release
yum install redis -y
systemctl enable redis
systemctl start redis
Add "hubot-redis-brain" to your external-scripts.json
Start hubot to test.


feat: display ordinal address before copying

Try out this version of the Hiro Wallet - download extension builds.

Displays the address about to be copied as shown in the modal shown before this one.

Let's merge this and enhance with https://github.com/hirosystems/stacks-wallet-web/issues/3291 by showing the full address in a standard receive modal.


Hey. Why anyone can eat in 1 milisecond 😄 Fix pls

Hey.


Content item index endpoint pagination

https://trello.com/c/OZF9rKeX/487-linked-items-endpoint-should-support-pagination
Added pagination so that content items can be retrieved in batches by publishing apps.

A few mostly stylistic issues, otherwise :+1:

It occurs to me that we need to support an "all items" parameter; policy-publisher is now using this endpoint to get all organisations, persons and working groups, which shouldn't be paginated.

This looks good.
I'm wondering what order the items are returned in?  It looks from a cursory reading of the code like they might be in database order, which isn't guaranteed to be the same between calls to the database.  There may well be a default order that I'm missing, though.  Also, do the callers to this api that need pagination need any particular ordering (eg, publication date?)
Regarding @danielroseman's comment about all items; yes, policy-publisher needs to get all organisations, etc, for the moment, in order to populate a select box.  content-tagger will have a similar requirement.  Having a way to express this in the api would be ideal.  One thing to think about, based on our experience with the search API, is that you might want to make the "count" parameter a required parameter, rather than having a default value (or make the default value be to return everything).  We've had a few bugs caused by assuming that the search API returns everything matching, which is much harder to do by mistake if you have to specify an explicit "count" parameter.

Looks good. One small comment with an idea for how to make Pagination a bit more reusable.
:+1:

From a quick scan of the code I couldn't see anything which is ensuring that there is a sort order on the pagination. This means that you can't guarantee that you're actually paginating over the set correctly. In other words, I think a sort order should be required, but it could have a default value.


NuGet package not updated

Hello there!
It seems that 3 days ago, the NuGet package was updated. As you stated in #113, you also included a TryGetExtension method in your codebase. Unfortunately, it does not seem to be anywhere on Github nor in the package version 1.0.12.
Here are a few screenshots:

As you can see, it is the latest version available and there are no methods added nor XML documentation (even though TryGetMimeType was added with PR #114).

Here I cannot find the TryGetExtension method you were talking about in issue #113 .

Thank you for pointing that out. I'll get another update out later today.

1.0.14 now available. https://www.nuget.org/packages/MimeTypeMapOfficial/

No problem dude! Have a happy holidays season! Stay safe 😄


Main

Description of Changes
This PR introduces [describe feature or fix]. It addresses [issue] by [briefly describe the fix or feature]. The current behavior is [describe the problem], and the new behavior is [describe the solution or improvement].
Related Issues
Fixes #0000.
How Has This Been Tested?
This change has been tested by:

Running npm run test to ensure all unit tests pass.
Performing manual testing in [testing environment] to verify the behavior.
[If applicable] Running integration tests to ensure no regressions were introduced.

Screenshots or Demonstrations
[Include screenshots or video links here, if necessary.]
Pull Request Checklist

[ ] The code is up-to-date with the main branch
[ ] I have performed a self-review of my code
[ ] npm run lint passes with this change
[ ] npm run test passes with this change
[ ] Tests have been added/updated for new functionality
[ ] Documentation has been updated to reflect this change (if applicable)
[ ] The commits follow the conventional commit spec
[ ] This pull request links relevant issues as Fixes #0000

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
:information_source: You can also turn on project coverage checks and project coverage reporting on Pull Request comment
Thanks for integrating Codecov - We've got you covered :open_umbrella:


Alertmanager - Auto Discovery in Prometheus Server setups

Proposal
Why we dont have the auto discovery for the Alertmanager deployed within the same cluster? The cluster pulls the data for other metrics of how cluster is doing and can we also get the node exporter to get discovered and get populated within the Prometheus server configuration?
Why is this important?: It will eases the automation part of the deployment. When we automate the deployment of these tools (Prometheus server, Alertmanager), its an additional manual step to add the Alertmanager name or IP to the Prometheus server configuration.
Thanks
Vikram

This is already how it works. It makes more sense to ask questions like this on the prometheus-users mailing list rather than in a GitHub issue. On the mailing list, more people are available to potentially respond to your question, and the whole community can benefit from the answers provided.

Somehow it didn't worked for me when I deployed it on K8s Cluster. Will review the setups once again and will post the question in the mailing list.
Thanks


Gmail Panel: Accept query parameters for days and hours

User should be able to set how many days of emails we should be checking the unread/important/most frequent sender statistics for. Similarly, User should be able to select how many hours of emails we are checking the recent unread emails statistics for.

The server has support and tests for this. All that needs to be implemented is that the client has to have the option to send the parameters to server.


Allow for scopes when using ApiKeyAuth to support AWS API Gateway & Cognito

Clear and concise description of the problem
Goal: Create a way to use AWS API Gateway with Cognito as an authorizer, and define scopes.
I believe this is currently not possible as you cannot pass in scopes with ApiKeyAuth which is what AWS expects.
Something like
@extension("x-amazon-apigateway-authtype", "cognito_user_pools")
@extension("x-amazon-apigateway-authorizer", {
  type: "cognito_user_pools",
  providerARNs: ["arn:<redacted>"],
})
model CognitoAuth<Scopes> is ApiKeyAuth<ApiKeyLocation.header, "Authorization", Scopes>;

@post
@extension("x-amazon-apigateway-integration", {
  type: "http_proxy",
  httpMethod: "post",
  uri: "http://<redacted>",
  connectionId: vpcLinkId,
  connectionType: "VPC_LINK",
})
@useAuth(CognitoAuth<Scopes>)
post<CreateUser, User, VpcLinkId, ["user/:write"]>(CreateUser): {
  @statusCode statusCode: 201;
  @body _: User;
};

Example output:
   post:
      ...
      security:
      - CognitoAuth:
        - "users/:write"
      x-amazon-apigateway-integration:
        connectionId: "<redacted>"
        httpMethod: "GET"
        uri: "<redacted>"
        responses:
          default:
            statusCode: "201"
        connectionType: "VPC_LINK"
 ...
  securitySchemes:
    CognitoAuth:
      type: "apiKey"
      name: "Authorization"
      in: "header"
      x-amazon-apigateway-authtype: "cognito_user_pools"
      x-amazon-apigateway-authorizer:
        providerARNs:
        - "arn:<redacted>
        type: "cognito_user_pools"

See https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-enable-cognito-user-pool.html
Checklist

[x] Follow our Code of Conduct
[x] Read the docs.
[x] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

I got it somewhat working with the following changes:
@typespec/http/dist/src/types.d.ts
23c23
< export type HttpAuth = BasicAuth | BearerAuth | ApiKeyAuth<ApiKeyLocation, string> | Oauth2Auth<OAuth2Flow[]> | OpenIDConnectAuth | NoAuth;
---
> export type HttpAuth = BasicAuth | BearerAuth | ApiKeyAuth<ApiKeyLocation, string, string[]> | Oauth2Auth<OAuth2Flow[]> | OpenIDConnectAuth | NoAuth;
84a85
>     scopes: OAuth2Scope[];

@typespec/openapi3/dist/src/openapi.js
1251a1252
>                 console.log(httpAuthRef);
1257a1259,1263
>                         continue;
>                     case "any":
>                         if (httpAuthRef.auth.type === "apiKey") {
>                             securityOption[httpAuthRef.auth.id] = httpAuthRef.auth.defaultScopes;
>                         }

@typespec/http/lib/auth.tsp
92c92
< model ApiKeyAuth<Location extends ApiKeyLocation, Name extends string> {
---
> model ApiKeyAuth<Location extends ApiKeyLocation, Name extends string, Scopes extends string[] = []> {
100a101,103
>
>   @doc("Scopes of every flow. Overridden by scope definitions in specific flows")
>   defaultScopes: Scopes;

I did not check that this didn't break existing usage of ApiKeyAuth or anything beyond just compiling, visually checking it, and trying to use the import feature on API Gateway, which seemed to work.
Leaving this here in case someone else picks it up.


Change the Image tag for opendistro-es from specific ODFE version to latest

Issue #, if available:
Description of changes:
Change the Image tag for opendistro-es from specific ODFE version to latest. This will reduce any change to the values.yaml during ODFE release.
Test Results:
No test needed
Note: If this PR is related to Helm, please also update the README for related documentation changes. Thanks.
https://github.com/opendistro-for-elasticsearch/opendistro-build/blob/master/helm/README.md
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

No changes are required on the Helm README.

Are you sure we don't need to test it? I know it is just tag name change but just to be sure?

You want to specifically define a version number so that when user checkout a version tag, they are using the specific version.
Latest means checkout different tags does not matter you are putting user to change the version themselves when they think they are using an older version.
And I have seen many ppl try to use an older version of ES for stability.
All in all, I dont agree with this change and I think we are putting unnecessary hurdles on the user.

Closing the PR as there is direct dependency with the release tag.


SwiftUI.swift file showing StateObject error in macOS app.

How frequently does the bug occur?
All the time
Description

Stacktrace & log output
No response
Can you reproduce the bug?
Yes, always
Reproduction Steps
No response
Version
Realm 10.17
What SDK flavour are you using?
Local Database only
Are you using encryption?
No, not using encryption
Platform OS and version(s)
macOS BigSur Beta Ver 11.5
Build environment
Xcode version: 12.5.1
Dependency manager and version: COCOAPODS: 1.10.1 & SPM

@leemaguire I'm using these pods:
  pod 'MagicalRecord/Shorthand'
  pod 'SDWebImage'
  pod 'PocketSVG'
  pod 'Localize-Swift', '~> 2.0'
  pod 'ReachabilitySwift'
  pod 'Zip', '~> 1.1'
  pod 'AppCenter'
  pod 'Alamofire', '~> 5.2'
  pod 'CocoaLumberjack/Swift'
  pod 'SwiftGen', '~> 6.0'
  pod 'RealmSwift'

@m-afham could you try the fix from the message above?

@IPrototypeI
I just encountered this problem.
Update to the latest version.
It has been fixed.
Thanks

@m-afham were you able to fix this after upgrading from Big Sur beta?


add support for Canary

canary status flowing to events log. Images are not possible to get since there is no any documentation provided yet.

Closed for now as there is no access to other function at this time.


MVP Sklepy

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 4 committers have signed the CLA.:x: grucapawel:x: pawelgruca:x: IgorSwatowski:x: tgarbowskiYou have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 4 committers have signed the CLA.:x: grucapawel:x: IgorSwatowski:x: pawelgruca:x: tgarbowskiYou have signed the CLA already but the status is still pending? Let us recheck it.


Click is not working in Cordova demo app.

From<EMAIL_ADDRESS>on June 23, 2014 22:54:32
What steps will reproduce the problem? 1. Download cordova mega demo app from playstore and extract apk.
2. Click on Accelerometer, next page will be loaded.
3.  What is the expected output? What do you see instead? It should show the accelerometer values. But nothing happens and application hangs. What version of the product are you using? On what operating system? Robotium 5.2.1, Robotium 5.1  .        Android 4.2 Please provide any additional information below.
Original issue: http://code.google.com/p/robotium/issues/detail?id=617

From<EMAIL_ADDRESS>on June 23, 2014 22:59:58
Thanks for reporting this.
Status: Accepted
Owner<EMAIL_ADDRESS>


How to change theme style

I prefer the following theme to red.

https://github.com/hlissner/doom-emacs/blob/develop/docs/faq.org#how-do-i-change-the-theme
https://github.com/hlissner/doom-emacs/blob/develop/docs/faq.org#how-do-i-customize-a-theme-or-faces


fix(core) set SNI server name for upstream SSL connections

Made sure server name for upstream SSL connections match Host header.
Summary
Fixes problem when having configured upstream server using https://... where Kong initiates upstream SSL connection using SNI with server name "kong_upstream".
This does not match the "Host: " header forwarded from the incoming request and is seen by most web servers as an error and they won't process such request.
Full changelog

Added NGINX proxy_ssl_name directive to nginx-kong.conf template

Issues resolved
Fix #2129

I think this warrants a test.

Good idea to write a test for it.
I'm not sure how to do it for Kong and a simulate a HTTPS upstream and verify that SNI server name is set correctly.
My fix adds "proxy_ssl_name" to nginx-kong.conf in section "location /" so it should apply for all proxied requests if if they're not to a HTTPS upstream but I guess NGINX just ignores it when not using SSL for upstream connection.
An existing test spec/02-integration/05-proxy/01-router_spec.lua Router preserve_host preserves downstream host if enabled seems to fail on my PR.
I'm not sure why.

Should this PR also consider the SNI sent in the downstream Client Hello? If Kong has received a TLS connection from the downstream client, should we not consider that SNI as well, rather than just the Host header/upstream_url (at least on a configurable basis)?

@konrade Thanks for raising this issue. We also experienced the same one internally, and were about to implement your fix in #2129. I am glad it is coming from the community instead, at your initiative :)
@p0pr0ck5 I feel like such a use-case would be handled already when configuring the preserve_host property on the API object. Ex: Assuming (maybe wrongly) with enough confidence that the Client Hello SNI is the request Host header value as well, and if the API is configured with preserve_host = true, then $upstream_host will forward provide that value for the Client Hello made from Kong. Thus the upstream Client Hello and HTTP Host header effectively come from the downstream client. Thoughts?

@thibaultcha my question was centered around your presented assumption indeed being incorrect. While I don't think this is a common case, there's certainly nothing preventing the (layer 4) SNI value from being different from the (layer 7) Host HTTP header. Indeed, this is something Kong currently allows for, which is why I brought it up for the sake of completeness.
Certainly I don't want to be objectionist for objections' sake; I just want to make sure we're covering all cases we want to support. My biggest concern is that we're not offering the option to configure the upstream SNI value; indeed, in other cases, the HTTP Host value is not directly tied to the SNI (https://github.com/curl/curl/issues/607). Not saying we need all the flexibility the curl does ;)

there's certainly nothing preventing the (layer 4) SNI value from being different from the (layer 7) Host HTTP header.

This was certainly very clear already, but my assumption was that "in most places this isn't the case". As of now, I think such a configuration would be outside the scope of this hotfix though, and would be considered if requested, which hasn't happened so far.

Agreed, I don't think it's a common use case. My thought is, rather than treating this as a quick hotfix only, is it an opportunity to add some additional flexibility to meet future use cases. It seems like the answer to that question is 'no', which is okay with me- just wanted to explore the opportunity.
@konrade what are your thoughts on the above discussion?

Based on what I've seen other web servers doing I think Kong should reject incoming requests where SNI server name does not match Host header (see: TLS Virtual Host Confusion).
I guess there are two scenarios regarding upstream SNI:

Use same server name from downstream in SNI for upstream (kind of passing TLS straight through Kong)

Should be the behaviour when configured API with preserve_host=true

Map requests on external FQDN to another TLS upstream server name

Scenario 2 could potentially require an option to configure what HTTP Host and SNI server name to use when upstream is specified by IP.
A very real case for me when starting to evaluate Kong was that I already had an older web gateway that did all kinds of reverse proxying (some with upstream TLS). Instead of trying to replicate all mappings directly I tried to put Kong in front of the old gateway with a default routing sending all that didn't match any configured APIs to the old gateway.
In this scenario I ran into another problem with upstream SNI and it was due to Kong/NGINX keeping the upstream connection alive between multiple requests even if they downstream request was on different FQDNs. This caused HTTP Host to missmatch with SNI for subsequent request that wasn't on same FQDN that opened upstream TLS connection in the first place.

Following https://github.com/Mashape/kong/commit/5d86d0912ad07fbd41381ae7487827b545ac340e, it should be fairly easy to test this functionality by using an ssl_by_lua handler to examine the SNI field (https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/ssl.md#server_name).

I mocked up some tests related to this PR here: https://github.com/Mashape/kong/commit/3f3b476a0a62f6c06f2522aaf45682e50675e2e6
Also, @konrade one point that's come up in discussion of this is the handling of the SNI header when preserve_host is false, and the upstream_url element of an API is built by an IP address. In such a case, the value of upstream_host is an IP/port string, which is not valid SNI (https://tools.ietf.org/html/rfc6066#section-3). So we should find a way to handle this case by either using the downstream Host header when preserve_host is false and the upstream_url uses an IP address, or explicitly defining an alternative elsewhere (I'm a fan of the latter, since that's what this PR tries to do already).

A bit tricky since Host header allows for IP but not SNI.
Interestingly curl seems to try to put IP in SNI when doing request towards https address but when I tried on my Mac (curl 7.51.0) it printed:

WARNING: using IP address, SNI is being disabled by the OS.

I have the feeling that the correct way to handle it is to not include SNI for upstream TLS when upstream_url uses an IP and preserve_host is false. This should probably also be the case if downstream Host is an IP and preserve_host is true.

I have the feeling that the correct way to handle it is to not include SNI for upstream TLS when upstream_url uses an IP and preserve_host is false.

Any reason we couldn't use the downstream Host header here?
Also, this PR should still have tests :)

Should I take in your tests @p0pr0ck5 ?
Honestly I've never dug into the test suite/setup for Kong so would require a bit startup time for me to be able to contribute to the test suite.

@thibaultcha what are you thoughts here? Can we merge this PR as-is and bring in the tests later, or is having the tests in the same PR are blocker?
@konrade do you strongly disagree that we should use the downstream Host header as the value of proxy_ssl_name when preserve_host is false and the upstream_url uses an IP?

I would think that the tests here are quite the blocker, since we don't want to have an edge-case indeed... That said, they could be unit for now and simply testing the value of ngx.var.upstream_host after running the access handler... Although I don't think our components are atomic enough for such testing.

@thibaultcha fwiw, there are integration (ish) tests here: https://github.com/Mashape/kong/commit/3f3b476a0a62f6c06f2522aaf45682e50675e2e6

It should be possible to merge in your tests @p0pr0ck5 in patch-1 branch in my fork so the PR gets updated right?

Regarding the edge case when preserve_host is false and upstream is https and an IP then the only problem I see with passing on Host header value as SNI is when downstream request is done with IP. This would wrongly put an IP in SNI.

Regarding the edge case when preserve_host is false and upstream is https and an IP then the only problem I see with passing on Host header value as SNI is when downstream request is done with IP. This would wrongly put an IP in SNI.

In such a case, Nginx would simply silently strip it from the Client Hello (https://github.com/nginx/nginx/blob/master/src/http/ngx_http_upstream.c#L1792). We could add in our own logic to handle this, but it would just be a waste of CPU cycles.

Would Host header also be passed on to upstream in this case? (I would guess not due to preserve_host being false)
Otherwise there could still be a scenario where upstream web server rejects request due to missmatch between SNI and Host header value.

As your PR sits now, there would still be a mismatch between the Host header seen by the upstream, and the SNI seen by the upstream, when preserve_host is false and the upstream_url is built with an IP- the Host header would contain the IP (and port), and the SNI would be blank.
I spose in such a case (IP upstream_url and preserve_host false), we will always have a Host <-> SNI mismatch, because the desired value for the SNI (the Host header) is forbidden by an RFC. Thus I would say that we should document such a case, saying we simply can't support such a configuration at this time.
As for the tests, I'd imagine you can just merge that branch into your PR if you like. No promises someone else won't want some changes though ;)

@konrade can you rebase this? We recently made some additions to the Nginx template that are resulting in a conflict here.
BTW we'd like to get this merged soon; any movements on tests?

Sorry for the delay but I've been traveling the last weeks so haven't been able to look into the tests.
I've rebased my PR branch patch-1 to latest next from Mashape.

@konrade thanks for your work and discussion here! We will be merging this in via https://github.com/Mashape/kong/pull/2457. Thank you again!

No problem. Happy to help and to see that it got fixed.