Support data-driven auth methods, e.g. SAML

The server is gaining a new API feature to let /server_settings describe auth methods with a bit more detail, making it possible to use auth methods that aren't in a hardcoded list in the client code.
The immediate use case for this is SAML authentication, where there's a single protocol that may have any number of providers with different names, so that it makes less sense than it already does for the client to have to hardcode the possible names in advance.
The server-side implementation just landed as zulip/zulip#13307. Docs are said to be coming soon; previous discussion in chat here.
It would be good for us to implement our end soon, in order to better end-to-end validate the API choices before they're made permanent by a server release.
We'll need to keep the existing code for backwards-compatibility with older Zulip servers, but should plan to ignore the old data if the new key is present in the /server_settings data.
/cc @timabbott @mateuszmandera

The docs updates were merged in https://github.com/zulip/zulip/pull/13339.

I'm going through this now to implement the new part of the server_settings API -- just for the existing methods like Google and GitHub auth -- since that seems to be up now on chat.zulip.org.
@timabbott what's a way we can test actual SAML auth? I think you said something about a test machine on a DigitalOcean droplet.
Some thoughts on the API as I have my hands in it:

The name social_backends for this property is pretty odd:

These aren't particularly "social". Google and GitHub auth are marginally so; auth with AzureAD, or remoteuser SSO, or a SAML provider like Okta, is pretty much exclusively about authenticating as a corporate employee, or another institutional affiliate.

I know there's a library we're using with a name like "django-social-auth", but that inaptly-named dependency within the server seems like very much an implementation detail from any client's perspective.

These aren't "backends" in any obviously meaningful sense. They're methods for authenticating, or providers of authentication or identity... but the very next thing we're going to do with any of these is go make an HTTP request from a browser. Which will typically redirect to some other site, for the user to go enter their credentials into a web page. None of that seems particularly more backend-y than basically anything else a user might do on the Internet.

I know Django has a concept with a name like "auth backend", and each of these might correspond to one of those. But to the extent that's a good name at all, it's a good name for the backend of how the given authentication method works -- i.e. the part within the server, and perhaps the server's interaction behind the scenes with the auth/identity provider's server. It doesn't make much sense in the API for clients.

Missing from the name: any hint that this is about authentication at all.

Except to the extent that "social" gives that hint? Or that someone really steeped in Django auth integrations could perhaps hazard a guess from "backends" :wink:
But Zulip itself is rather more social than any of the things that can appear here. So really once you know this is coming from a Zulip server, the name carries very little information at all.

Not sure what a good name would be. It's almost just a list of all authentication methods, right? It's all of them but dev, password, and LDAP... it's all except those with two particular, well-specified login flows (namely trivial, and username+password)... it's the ones that are "generic", just go to some URL... it's the ones that are opaque to the client, and all it can know is to send the user in a browser to some URL.

Maybe generic_authentication? web_authentication_methods?

... OK, I have some smaller things to mention, but hitting send first.

Smaller thoughts on the API:

Docs say display_icon, but example and czo say display_logo.

I'm inclined to say "icon" is the better term -- a "logo" is a possible choice for an icon but some icons aren't logos. What's the "logo" for remoteuser auth?

Docs don't mention name.

The google method has "login_url": "/accounts/login/google/". The notes I have in a comment say about that URL:

    // Server versions through 2.0 accept only this URL for Google auth.
    // Since server commit 2.0.0-2478-ga43b231f9 , both this URL and the new
    // accounts/login/social/google are accepted; see zulip/zulip#13081 .

Should probably give the new URL, no?

It's a little confusing having a field called sort_order and having it sort in reverse. Why not use sort_order values that have the same order they're meant to sort in?

Or if there's a strong desire to use bigger numbers for the first ones to show... perhaps call it something like priority? Or display_priority.

@gnprice @timabbott

Not sure what a good name would be. It's almost just a list of all authentication methods, right? It's all of them but dev, password, and LDAP... it's all except those with two particular, well-specified login flows (namely trivial, and username+password)... it's the ones that are "generic", just go to some URL... it's the ones that are opaque to the client, and all it can know is to send the user in a browser to some URL.

RemoteUser is not a social backend (and it shouldn't be showing in the social_backends list at /server_settings - if it is showing for you, that'd be a weird bug :thinking: )
What these "social backends" all have in common is that the authentication happens with an external provider (rather than something on the zulip server). And all such auth methods that use an external provider should show up  in the social_backends list, even if in the future we add some that for their implementation we use a different library than python-social-auth.
Thus, perhaps external_authentication_methods would be a good name? I feel like generic_authentication is too... well, "generic"  - since these auth methods do have this "externality" as their central defining characteristic :smile:

Docs don't mention name

Hmm, I think this doesn't play any useful role right now, perhaps it's best to just remove this field from the API dicts.

* The `google` method has `"login_url": "/accounts/login/google/"`.

Should probably give the new URL, no?

Oh, this is a weird result of the following django urls setup:
    url(r'^accounts/login/social/([\w,-]+)$', zerver.views.auth.start_social_login,
        name='login-social'),
    url(r'^accounts/login/social/([\w,-]+)/([\w,-]+)$', zerver.views.auth.start_social_login,
        name='login-social-extra-arg'),
    # Backwards-compatibility (legacy) Google auth URL for the mobile
    # apps; see https://github.com/zulip/zulip/issues/13081 for
    # background.  We can remove this once older versions of the
    # mobile app are no longer present in the wild.
    url(r'^accounts/login/(google)/$', zerver.views.auth.start_social_login,
        name='login-social'),

combined with this method of constructing the login url for each backend for the api dicts:
login_url=reverse('login-social', args=(social_backend.name,)),
which translates the case with social_backend.name = 'google' to the old endpoint
Fixed in https://github.com/zulip/zulip/pull/13356

It's a little confusing having a field called sort_order and having it sort in reverse. Why not use sort_order values that have the same order they're meant to sort in?

Agreed.  @mateuszmandera I think it probably makes sense to just flip the direction for these in the code, so that lower numbers appear first.

I'm inclined to say "icon" is the better term -- a "logo" is a possible choice for an icon but some icons aren't logos.

Agreed s/logo/icon/ would be an improvement.  @mateuszmandera can you switch this?

The name social_backends for this property is pretty odd:

We should also keep in mind the existing legacy authentication_methods field, which I imagine we'll remove in a year or two but will be with us until then.  I'm not sure what a good name is either :/.
My best idea for that name is authentication_buttons, since that describes clearly that this is the set of third-party authentication option buttons to render, and if look at the fields of this structure, they detail precisely how to render those buttons and nothing else.  The main concern with that approach is it may be less extensible long-term if we have non-button authentication items.
I suppose we could call them authentication_widgets, and include a type: button field on the items, for future-proofing the possibility of adding additional authentication widgets that aren't precisely a button to this data structure.  (And then the mobile code could just ignore ones that aren't type: button).  I think I like that option best of the ones we've proposed.
Thoughts?

Thus, perhaps external_authentication_methods would be a good name? I feel like generic_authentication is too... well, "generic" - since these auth methods do have this "externality" as their central defining characteristic

That'd be pretty reasonable!
I'd like to probe a little more the accounts/login/sso/ case, though. From the mobile app's perspective, this actually works 100% exactly the same as all these other auth methods:

The client doesn't need to know anything about how to do it, except a URL.
It sends the user to that URL in a browser.
The server does whatever it wants to do -- in most cases redirect to some other service that in turn does whatever flow it wants to do, or in this case something much simpler. But in all cases, the Zulip client knows nothing about the details.
If the server determines auth succeeded, it sends the user back to the client, using a zulip:// URL and the mobile_flow_otp one-time-pad.

So for us it'd be convenient if it did get included in the list. I wasn't observing that, but czo (the server I had handy that has this feature in the first place) doesn't have REMOTE_USER auth set up anyway, so that's what I expected there :slightly_smiling_face:
Moreover I think the description I just gave of what these all have in common would make sense, except the last bullet, for a lot of clients -- any client that might not know up front the details of all possible auth flows. And the last bullet seems like it could be adapted as needed. So I think it's not only mobile, but Zulip clients in general that want to support not-just-password auth at all, that would benefit from having REMOTE_USER auth included in the list the same way as the others we're discussing.

having REMOTE_USER auth included in the list the same way as the others we're discussing.

This wouldn't necessarily make external_authentication_methods an inapt name, either:

it might be the only one for a long time that doesn't involve redirecting to some other ("external") service;
even though it doesn't redirect, this method is "external" in another sense: the client doesn't know any of the details, and has to rely on whatever's found at some URL (something "external" to the Zulip client) to drive the flow.

My best idea for that name is authentication_buttons, since that describes clearly that this is the set of third-party authentication option buttons to render, and if look at the fields of this structure, they detail precisely how to render those buttons and nothing else. The main concern with that approach is it may be less extensible long-term if we have non-button authentication items.
I suppose we could call them authentication_widgets, and include a type: button field on the items, for future-proofing the possibility of adding additional authentication widgets that aren't precisely a button to this data structure.

Hmm. This feels like it's awfully UI-oriented. Which is fine for something aimed at the web frontend, but for something in the API for other clients to use which have their own UIs it doesn't seem like a great fit.
For example, already today on mobile if you go to log into a realm that has both password auth and something else, we show you a button for password auth -- it's no different from the third-party auth methods in that respect. On web (at least desktop web) it works well to show the email+password fields right there at the same time as the buttons for alternate methods, but on mobile that's too crowded.
It's true that these have some details like icon and display name that are meant for presentation to the user; but those are at a similar level of abstraction to a user's name and avatar, and are similarly flexible to fit naturally in a variety of UI contexts.

Docs don't mention name

The only role this plays right now is that if someone configures multiple SAML IdPs without setting their display_names, they'll all end up with it set to SAML and name just is a key to distinguish between them. But maybe this field should just be deleted from these API dicts, since it doesn't really do anything.

Yeah, not sure it's essential -- I don't expect we'd do much with it in the mobile app, either.
One thing I could see it being potentially useful for is if the client does have (or thinks it has?) any special knowledge about some of the options. For example maybe it sees there's Google auth, and it's the Android app in some future where we've found a way to successfully use the nice system flow for authenticating with the Google account the device is logged into. Or maybe something more boring, like a tweak to the presentation or just a workaround for some bug. If a client does ever for some reason want a special case like one of these, name: 'google' is likely a much better way to target it than testing the login_url or something.

Docs don't mention name

Yeah, agreed, we want the field because it provides opportunity just what Greg describes.
On authentication_widgets:

Hmm. This feels like it's awfully UI-oriented. Which is fine for something aimed at the web frontend, but for something in the API for other clients to use which have their own UIs it doesn't seem like a great fit.
This wouldn't necessarily make external_authentication_methods an inapt name, either:

So, the reason I was thinking something more like authentication_widgets, is that it's theoretically possible we might want to later migrate password authentication to that system (basically moving all the other password/LDAP related keywords into that object), so that it can e.g. benefit from the sort_order feature.  And external_authentication_methods would be a poor name for a thing that included the internal authentication options.
Really, I want to call it authentication_methods.  I suppose we could add an argument one must include in the /server_settings request to ask for a new format if we want to transition away from external_authentication_methods down the line.  So I'm fine with that option if folks prefer.
For the REMOTE_USER backend:

So for us it'd be convenient if it did get included in the list. I wasn't observing that, but czo (the server I had handy that has this feature in the first place) doesn't have REMOTE_USER auth set up anyway, so that's what I expected there slightly_smiling_face

I think I agree it should be in this data structure; that makes live easier for mobile and terminal apps.  And actually @mateuszmandera if you look at zerver/views/login.html, you can see that the structure of the "Login in with SSO" button that only_sso provides has more or less the same structure as a SAML backend button.  Which suggests to me that setting this up the same way the SAML ones work would clean up that template significantly.  I think you should be able to test such a migration in a test production system with https://zulip.readthedocs.io/en/latest/production/authentication-methods.html#apache-based-sso-with-remote-user; you'll just want to configure Apache digest/basic auth to make it work.
Schedule-wise I'd prioritize the other format changes we've been discussing before this, though, since this will have the least impact on the testing experience.

sort order

Finally, I realized we should drop sort_order from the objects; we're sending a list of dicts, and we can just sort it before sending to the client.

Finally, I realized we should drop sort_order from the objects; we're sending a list of dicts, and we can just sort it before sending to the client.

Hmm indeed! Good idea. :slightly_smiling_face:

Really, I want to call it authentication_methods. I suppose we could add an argument one must include in the /server_settings request to ask for a new format if we want to transition away from external_authentication_methods down the line. So I'm fine with that option if folks prefer.

That sounds like a good plan to me.

Really, I want to call it authentication_methods. I suppose we could add an argument one must include in the /server_settings request to ask for a new format if we want to transition away from external_authentication_methods down the line. So I'm fine with that option if folks prefer.

That sounds like a good plan to me.

To clarify, what is the first name change we want to go with then?  external_authentication_methods instead of social_backends?
And in the (possible) future move that to authentication_methods in a new format, requested through an additional argument, where it'll include all the backends?

Yeah, let's go with external_authentication_methods for now.
There's a possible future revised version where we don't even send the new data structure unless extended_authentication_methods=True is passed, and if it is, we just send it under the name authentication_methods.  But we can't go there until we have a plan for migrating the communication about the non-external authentication methods via this data structure, which will require significant design work for uncertain gains, so I think it makes sense to not do any active work in that direction at this time.

OK, we just merged #13361, which executes the main API changes discussed in this thread:

sort_order is removed from the object and instead just used for sorting :)
The objects are now called external_authentication_methods
display_logo -> display_icon
Documented the name field.

I think those are all the easy changes.  We should try to get converting RemoteUserBackend to this framework done next, but probably mobile folks can again proceed with getting their changes working under the assumption we'll succeed at that.

OK, we just merged zulip/zulip#13361, which executes the main API changes discussed in this thread:

Awesome -- thanks to you both!
One small further point I just noticed:

For display_icon, when there's no icon it'd be cleaner to indicate that with the value null rather than an empty string.

In Python this corresponds to using None vs. an empty string.

zulip/zulip#13368 should merge with that change shortly.

We should try to get converting RemoteUserBackend to this framework done next, but probably mobile folks can again proceed with getting their changes working under the assumption we'll succeed at that.

@mateuszmandera Do you have time to try moving that over, so that remoteuser appears in external_authentication_methods when enabled?
If we move it after 2.1 is out, that'll be OK but will just add a few more lines of compatibility code we'll need to carry in the client for as long as 2.1 is around.

@gnprice I'll start working on it

@mateuszmandera Excellent, thanks!
Meanwhile I've just merged (with #3675) an implementation of this feature.
Once the remoteuser change is in, I'll make the small mobile-side adjustment for that. The mobile code that's now in master (after #3675) will treat remoteuser the same way older mobile versions do, so it doesn't add any new compatibility constraints (even if we make a mobile release with this version of the code.)


No status bar in iOS 11.1

There is no status bar(network signal, battery etc.) in iOS 11.1 please fix this

@varun-naharia , thanks reporting.
Fix this problem in version 2.0.5


Same vehicle requirement is not enforced with high-cost

Hello,
Setting AddSoftSameVehicleConstraint(Nodes, Cost) works well with a small number of Nodes, and the desired vehicle serves all these Nodes on time.  However, after adding Nodes to the problem (but not to the condition) the solver also assigns another vehicle which serves the majority of the Nodes listed in the condition - even when an extra-high Cost is set in the condition.
The desired vehicle should have easily served all these Nodes and save that cost, but the solver assigns it to other Nodes.  FSS is ParalleChepaerInsertion.
Any exlpanation?

Hello,
Any prior observations of this behavoir?

Maybe this is the only feasible solution.

Any news? Please reopen if needed.


Ladybug GitHub Actions succeeded even though copyright notices were not OK.

We observed this here:

@MarkvdVorst suggested that the ladybug backend GitHub test may not test the Maven build of the backend. We know that the ladybug backend tests trigger ladybug-frontend, but that may be the only thing the test does.

It would be nice to have a dedicated action for backend testing instead of only having a yaml file for ladybug-frontend tests.
I'm guessing the ladybug-frontend yaml also takes the backend into account somehow, but skips the tests copyright and/or unit tests as it did not give me any failed tests earlier when I submitted a PR with wrong copyright year.

nevermind. the skip copyright is still present in this

Ik denk dat de Ladybug backend op Jenkins wordt getest, zie https://jenkins.frankframework.org/job/ladybug-ci/job/ladybug/. Daar wordt de volledige Maven build gedaan, dus zonder opties om stappen over te slaan.


B11009028 4

Checks

[ ] Your YAML file has required fields as listed in Specs
[ ] All checks have passed yaml-lint passed
[ ] You agree to contribute your work under MIT license

Continuous integration -> A


Use KeyVault client for retrieving BLS key

Description
The BLS key will be used to generate the salt for a given phone number mapping. As such, leaking this key can potentially compromise the privacy of users on the network. This change allows us to store the BLS key within a secure KeyVault instead of as an environment variable.
Other changes
Migrate the AzureKeyVaultClient to the contractkit utils directory since it's used by more than just account signing.
Tested
Tested via mock object as well as using an actual KeyVault.
Related issues

Fixes https://github.com/celo-org/celo-labs/issues/429

Backwards compatibility
These changes are not backwards compatible. We will need to deploy a KeyVault, create the corresponding key, and update the CloudFunction config before we can deploy the update.

one thing i'm unclear on -- if these keys can only be generated in the keyvault, how do we generate the threshold keys?
https://github.com/celo-org/blind-threshold-bls-wasm
// Generate the secret shares for a 3-of-4 threshold scheme
const t = 3;
const n = 4;
const keys = threshold.thresholdKeygen(n, t, crypto.randomBytes(32))

^ it looks like all keys get generated at once

@aslawson since cloud KeyVaults don't support BLS keys, I'm using the generic secret functionality of KeyVault. It acts like a secure Key/Value dictionary and allows us to provide any format for the secret object.
Example: https://docs.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal

@aslawson since cloud KeyVaults don't support BLS keys, I'm using the generic secret functionality of KeyVault. It acts like a secure Key/Value dictionary and allows us to provide any format for the secret object.
Example: https://docs.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal

I see -- so we can generate the keys externally and upload them to the vaults.  I had thought you mentioned a limitation that keys had to be generated in the vault, but perhaps that was for other key types.

@aslawson since cloud KeyVaults don't support BLS keys, I'm using the generic secret functionality of KeyVault. It acts like a secure Key/Value dictionary and allows us to provide any format for the secret object.
Example: https://docs.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal

I see -- so we can generate the keys externally and upload them to the vaults. I had thought you mentioned a limitation that keys had to be generated in the vault, but perhaps that was for other key types.

Yeah, that was a limitation for HSM keys. Unfortunately, HSMs don't support BLS keys so we have to use the less secure approach of storing/retrieving the raw key value.


Updates to processResult

This PR fixes:

testdef.RANDOM in passResultIntoSessionMap() didn't have a random seed and would always pick the same item. It now uses the UnixNano to seed the randomizer

This PR adds:

the JSONPath handling in processResult() is added back based on github.com/oliveagle/jsonpath

Thanks for your PR! I've merged it.


is it possible to add a search box to filter the picker's list?

Hello @razmaimon
I think it's possible but I need to investigate ;)
I keep you in touch.

@razmaimon You can try it on the branch add-search
Tell me what do you think about it.

it works fine, can you please merge your code with the master branch and create a new pod version with this feature?

It's merged and the pod 0.3.0 version is available !

Hi,
The pod 0.3.0 is not available currently

@AsimmParvezz Please do a pod update to get the latest version

I get the following error:
[!] Failed to connect to GitHub to update the CocoaPods/Specs specs repo - Please check if you are offline, or that GitHub is down


bincapz --ignore-self silently drops samples/

directory scans seem to be broken at HEAD:
% go run . -verbose samples/

Returns no results, but shows that it is at least trying:
time=2024-05-05T08:59:22.407-04:00 level=INFO source=/Users/t/src/bincapz_fix/bincapz.go:56 msg="bincapz starting"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=evasion/binary-opaque.yara warning="string \"$word_with_spaces\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=ref/email.yara warning="string \"$e_re\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=ref/ip.yara warning="string \"$ipv4\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=ref/ip_port.yara warning="string \"$ipv4\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=shell/background-sleep.yara warning="string \"$cmd_bg\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=systemd/no_docs_or_comments.yara warning="string \"$ex_comment\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=third_party/mthcht_thk_yara_rules.yar warning="string \"$string22_DynastyPersist_offensive_tool_keyword\" may slow down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=third_party/mthcht_thk_yara_rules.yar warning="rule is slowing down scanning"
time=2024-05-05T08:59:23.403-04:00 level=WARN source=/Users/t/src/bincapz_fix/pkg/rules/rules.go:70 msg=warning namespace=third_party/mthcht_thk_yara_rules.yar warning="string \"$string7_RDPassSpray_offensive_tool_keyword\" may slow down scanning"
time=2024-05-05T08:59:25.337-04:00 level=DEBUG source=/Users/t/src/bincapz_fix/pkg/action/scan.go:87 msg=scan config="{IgnoreSelf:true IgnoreTags:[] IncludeDataFiles:false MinFileScore:0 MinResultScore:1 OCI:false OmitEmpty:false Output:<nil> Renderer:{w:0x14000118048} Rules:0x14000174000 ScanPaths:[samples/] Stats:false}"
time=2024-05-05T08:59:25.337-04:00 level=INFO source=/Users/t/src/bincapz_fix/pkg/action/scan.go:100 msg="14531 rules loaded"
time=2024-05-05T08:59:25.337-04:00 level=INFO source=/Users/t/src/bincapz_fix/pkg/action/scan.go:23 msg="finding files in samples/ ..."

I also checked to see if it was a recursion issue, but it happens for any other directory, for example go run . samples/Linux/clean/. Looking at recent PRs, I think this could be a regression from #174. We'll also need a test case to make sure this doesn't happen again.

cc @egibs in case this rings a bell

This is so weird that maybe this is just my macOS machine.
Running bincapz against /bin works.
Running  go run . samples/does-nothing/does-nothing doesn't work, but it does work if I copy the directory to /tmp. I thought maybe it was an issue with relative paths, but absolute paths within my repository don't seem to work either.
What a curious thing.. I wonder if anyone else is seeing this behavior.

Also, the weirdness does not seem dependent on directory scanning.
go run . samples/Linux/2024.sbcl.market/sbcl.dirty also shows no results.

Confirmed on my Linux box that go run . samples/ shows no results either.
I did figure it out though; if I set --ignore-self=false it works! I think the logic needs to be tightened up a bit.

cc @egibs in case this rings a bell

Yep, definitely a consequence of #167. I'll work on improving this 👍🏻

@egibs - I think if we change to just matching the YARA rule, and tightening the YARA rule up to include say, 2 of 3 unique strings from the bincapz binary, it should be sufficient. For example:
bincapz/pkg/rules.FS
bincapz/pkg/report

After looking at the PR implementation, I think there was some confusion about #134 based on my vague wording: my intent was to only ignore the bincapz binary if encountered: for example, in a container image or in your $PATH. It was not to ignore content within the bincapz repository.
My apologies for the imprecise wording.


Showing two scroll bars (even when nothing to scroll) on an Account in Firefox

Tested on Windows 10, Firefox (latest) (this issue is not present in Chrome)
Two scroll bars
When there is nothing to scroll through:

When there is something to scroll through:

We did change a few things with global styles and scrolling behavior in this release. I don't have a Windows machine to repro this, @kyranjamie do you?

@fbwoolf this happens on MacOS firefox too.

@fbwoolf this happens on MacOS firefox too.

I didn't see it on MacOS 🤔 ...not sure why but thanks for fixing!

@fbwoolf you need to have scroll bars always visible, else they'd be hidden

Try this:

From the Apple menu, select System Preferences.
Click General. Once you click General the following menu will appear.
Set the Show Scroll Bars setting to Always.
Close System Preferences.


chore: support windows shell for just

closes #4794

Thank you.

Asking this purely out of curiosity.  How I was using it on my Windows machine until now😆I didn't use the init command though.
Does it only fail on cmd? Because I use PowerShell as my default shell on Windows and that might be the reason it used to work even before this fix.

Now I noticed in the original issue you were also using PowerShell, So it has to be because of WSL. WSL exposes a bash command to the shell.


[RELAY][BYOC] Add support for composite functions in BYOC

This PR includes two changes to support composite functions as part of the BYOC flow. First, 'check' functions have been added to the MergeComposite pass. Currently you can only do structural pattern matches which don't test any attributes or tensor sizes. The check function accepts the matching expression and can implement arbitrary logic to test whether that match should be supported. See the new test for an example which queries convolution layout.
The other change enables the annotation of composite functions in the AnnotateTarget pass. For a composite function to be recognised, you must name it according to the convention:
{compiler}.{composite_name}
eg. dnnl.add_relu

cc @masahi @zhiics @trevor-m

cc @soiferj

Partitioning looks good for me with a simple pattern, conv2d_bias_relu. I'm seeing the check fail in IsOp for composite functions however. Maybe for another PR.
  bool IsOp(const CallNode* call, const std::string& op_name) const {
    const auto* op_node = call->op.as<OpNode>();
    CHECK(op_node) << "Expects a single op.";
    Op op = GetRef<Op>(op_node);
    return op == Op::Get(op_name);
  }

https://github.com/mbaret/tvm/blob/a197bca20a8effc48d06388190aa1b5f3b525ef6/src/relay/backend/contrib/codegen_c/codegen_c.h#L191-L205
TVMError: Check failed: op_node: Expects a single op.

Using MergeComposite, AnnotateTarget and PartitionGraph, I get the following graph for conv + bias + relu pattern:
def @dnnl_0(%dnnl_0_i0: Tensor[(1, 3, 224, 224), float32], Inline=1, Compiler="dnnl", global_symbol=runtime.String(0x55a8d9cddbd0), Primitive=1) -> Tensor[(1, 1, 224, 224), float32] {
  %2 = fn (%data: Tensor[(1, 3, 224, 224), float32], %weight: Tensor[(1, 3, 3, 3), float32], %bias: Tensor[(1, 1, 1), float32], Composite="dnnl.conv_bias_relu") -> Tensor[(1, 1, 224, 224), float32] {
    %0 = nn.conv2d(%data, %weight, padding=[1, 1, 1, 1], channels=1, kernel_size=[3, 3]) /* ty=Tensor[(1, 1, 224, 224), float32] */;
    %1 = add(%0, %bias) /* ty=Tensor[(1, 1, 224, 224), float32] */;
    nn.relu(%1) /* ty=Tensor[(1, 1, 224, 224), float32] */
  };
  %2(%dnnl_0_i0, meta[relay.Constant][0] /* ty=Tensor[(1, 3, 3, 3), float32] */ /* ty=Tensor[(1, 3, 3, 3), float32] */, meta[relay.Constant][1] /* ty=Tensor[(1, 1, 1), float32] */ /* ty=Tensor[(1, 1, 1), float32] */) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

def @main(%data1: Tensor[(1, 3, 224, 224), float32]) -> Tensor[(1, 1, 224, 224), float32] {
  @dnnl_0(%data1) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

Is it possible to inline composite function %2 there into dnnl_0? What I want is this:
def @dnnl_0(%dnnl_0_i0: Tensor[(1, 3, 224, 224), float32], Inline=1, Compiler="dnnl", global_symbol=runtime.String(0x5599b307c370), Primitive=1) -> Tensor[(1, 1, 224, 224), float32] {
  %0 = nn.conv2d(%dnnl_0_i0, meta[relay.Constant][0] /* ty=Tensor[(1, 3, 3, 3), float32] */ /* ty=Tensor[(1, 3, 3, 3), float32] */, padding=[1, 1, 1, 1], channels=1, kernel_size=[3, 3]) /* ty=Tensor[(1, 1, 224, 224), float32] */;
  %1 = add(%0, meta[relay.Constant][1] /* ty=Tensor[(1, 1, 1), float32] */ /* ty=Tensor[(1, 1, 1), float32] */) /* ty=Tensor[(1, 1, 224, 224), float32] */;
  nn.relu(%1) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

def @main(%data: Tensor[(1, 3, 224, 224), float32]) -> Tensor[(1, 1, 224, 224), float32] {
  @dnnl_0(%data) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

Otherwise I have to support function call in DNNL codegen. @zhiics @mbaret

Using MergeComposite, AnnotateTarget and PartitionGraph, I get the following graph for conv + bias + relu pattern:
def @dnnl_0(%dnnl_0_i0: Tensor[(1, 3, 224, 224), float32], Inline=1, Compiler="dnnl", global_symbol=runtime.String(0x55a8d9cddbd0), Primitive=1) -> Tensor[(1, 1, 224, 224), float32] {
  %2 = fn (%data: Tensor[(1, 3, 224, 224), float32], %weight: Tensor[(1, 3, 3, 3), float32], %bias: Tensor[(1, 1, 1), float32], Composite="dnnl.conv_bias_relu") -> Tensor[(1, 1, 224, 224), float32] {
    %0 = nn.conv2d(%data, %weight, padding=[1, 1, 1, 1], channels=1, kernel_size=[3, 3]) /* ty=Tensor[(1, 1, 224, 224), float32] */;
    %1 = add(%0, %bias) /* ty=Tensor[(1, 1, 224, 224), float32] */;
    nn.relu(%1) /* ty=Tensor[(1, 1, 224, 224), float32] */
  };
  %2(%dnnl_0_i0, meta[relay.Constant][0] /* ty=Tensor[(1, 3, 3, 3), float32] */ /* ty=Tensor[(1, 3, 3, 3), float32] */, meta[relay.Constant][1] /* ty=Tensor[(1, 1, 1), float32] */ /* ty=Tensor[(1, 1, 1), float32] */) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

def @main(%data1: Tensor[(1, 3, 224, 224), float32]) -> Tensor[(1, 1, 224, 224), float32] {
  @dnnl_0(%data1) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

Is it possible to inline composite function %2 there into dnnl_0? What I want is this:
def @dnnl_0(%dnnl_0_i0: Tensor[(1, 3, 224, 224), float32], Inline=1, Compiler="dnnl", global_symbol=runtime.String(0x5599b307c370), Primitive=1) -> Tensor[(1, 1, 224, 224), float32] {
  %0 = nn.conv2d(%dnnl_0_i0, meta[relay.Constant][0] /* ty=Tensor[(1, 3, 3, 3), float32] */ /* ty=Tensor[(1, 3, 3, 3), float32] */, padding=[1, 1, 1, 1], channels=1, kernel_size=[3, 3]) /* ty=Tensor[(1, 1, 224, 224), float32] */;
  %1 = add(%0, meta[relay.Constant][1] /* ty=Tensor[(1, 1, 1), float32] */ /* ty=Tensor[(1, 1, 1), float32] */) /* ty=Tensor[(1, 1, 224, 224), float32] */;
  nn.relu(%1) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

def @main(%data: Tensor[(1, 3, 224, 224), float32]) -> Tensor[(1, 1, 224, 224), float32] {
  @dnnl_0(%data) /* ty=Tensor[(1, 1, 224, 224), float32] */
}

Otherwise I have to support function call in DNNL codegen. @zhiics @mbaret
UPDATE: hmm if I inline the composite function, I lose the composite attribute and hence cannot detect fused call. Is supporting function call in the DNNL codegen a better option?

Hi @masahi, I think it is best to leave this to the codegen. That way we have the option to handle a composite all at once at call node or invididually by just doing VisitExpr(func->body) when we encounter one.

@trevor-m Yeah agree. I'm looking to support either function call or "manual inline" in the codegen.

Ok got the DNNL conv + bias + relu working (https://github.com/masahi/tvm/compare/byoc-composite...masahi:dnnl-composite). This is a reimplmentation of https://github.com/apache/incubator-tvm/pull/4741 based on composite annotate support in this PR, rather than the manual approach. I think it is much cleaner.
I can send a new PR as soon as this PR is merged.

@masahi @trevor-m can you guys help review as well?

@alexbooth Thanks for pointing out. The purpose of the codegen_c is mainly for quick prototyping. Would you be interested to send a PR to add the support for composite function after this is merged?

@masahi yeah, inlining the function you pointed out is a little tricky because it varies case by case. I intentionally made it a bit more conservative and left it for the external codegen to handle. In the long run, it would be more helpful if we make the passes more configurable.

@zhiics Yes, I'll review and merge this today
@alexbooth The error you get is because now with composite, op inside CallNode can be a FunctionNode, rather than OpNode. I'll send a DNNL change today so you can take a look.

Thanks @mbaret @zhiics


feat: Actually de-dupe Navigation entries + take latest LCP entry

Previous attempts have not fully fixed this issue, so actually implement deduping the Navigation entries list. This time we will compare several key/values to remove duplicate entries.
Also change to take only the latest LCP value.
It's possible (though unlikely?) that we have dupe LCP/Navigation entries that span across 2 recording segments.

by owned properties do you mean on sentry.io?

by owned properties do you mean on sentry.io?

yeah

Yeah that'd be a good idea - I have a PR coming up that can do some logging to breadcrumbs that we can use.

@JoshFerge I refactored this a bit so we also select the latest LCP as well

Will be easier to review if you look only at the latest commit: https://github.com/getsentry/sentry-replay/pull/209/commits/0591a14bd4dd8cef0921a083eff7aaa288806cc6


[Lake] Support alter lake tablet in BE

What type of PR is this：

[ ] bug
[x] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Like local tablets, there are three schema change types,
including linked, direct and sorted schema change.
In sorted schema change, use DeltaWriter to sort data.
Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

run all

run starrocks_admit_test

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

run starrocks_admit_test

run starrocks_admit_test


warmup is slow when using "moduleOptions.styles.configFile"

For some reason this config takes more than 40 seconds to warm up.
import { defineNuxtConfig } from "nuxt/config";

export default defineNuxtConfig({
  ssr: true,
  modules: ["vuetify-nuxt-module", 'nuxt-purgecss'],
  css: ["vuetify/styles"],
  devtools: { enabled: true },
  vuetify: {
    moduleOptions: {
      styles: { configFile: "/styles/settings.scss" },
    },
  },
  experimental: {
    inlineSSRStyles: false,
  },
});

/*settings.scss*/

@use "vuetify/settings" with (
  $utilities: false,
  $color-pack: false,
  $button-color: red
);

without setting the configFile, it takes less than 2 seconds.
help? Am I doing something wrong?

@omerman this is a problem with SASS and Vuetify, we cannot do anything here, it is also slow using Vite (without Nuxt) or Webpack 5

ok I understand, thank you


Add instructions regarding vcpkg (fix #116)

Preview: https://github.com/myd7349/simdjson/tree/doc-vcpkg-guide#usage-using-vcpkg-on-windows-linux-and-macos
Some words are borrowed from https://github.com/googleapis/google-cloud-cpp#windows-using-vcpkg

Great.


Update @types/leaflet installation guide

We don't really need @types/leaflet package in runtime dependencies, so we can install it as a dev dependency.

The original statement
npm install @types/leaflet
doesn't actually save the dependency. It leaves it up to the developer to decide whether or not to save the dependency and whether or not to save it as a dev or prod dependency.
I'd rather just keep it simple and leave it as a basic npm install. If you think this part was confusing, I'd prefer to add a comment to the paragraph explaining the need for the typings.

@reblace but now npm save all dependencies by default.

oh, hah... I didn't even notice they changed that. Ok, then this should be fine.


rootfs: Bump golang version to 1.11.1

Bump the golang version to 1.11.1, that is the "newest-version"
currently specified in the runtime version file.
Fixes: #208

lgtm

/test

Not sure why the tests are not firing :thinking: Let's try another keyword:
/retest


Revisit the logic in Architecture.cpp for X64 emulation of ARM64 devices.

Description of the new feature/enhancement
Given the X64 emulation of ARM64 devices went publicly preview this week. We'd better add X64 to ApplicableArchitecturesVector for ARM64 devices.
Proposed technical implementation details (optional)
It should be a trivial change by adding an entry to ApplicableArchitecturesVector. However I don't know how to check if the system supports X64 emulation.

We will be looking at multi architecture support in #132

Detection of x64 support on ARM64 requires a new API.
bool SupportsX64_OnlyCallOnARM64()
{
  // TODO: Dynamically load GetMachineTypeAttributes function pointer
  MACHINE_ATTRIBUTES ma = 0;
  GetMachineTypeAttributes(IMAGE_FILE_MACHINE_AMD64, &ma);
  return ma & UserEnabled;
}

GetMachineTypeAttributes is a brand new API, and thus we will have to load it from wherever it lives (we should be able to GetProcAddress from the onecore umbrella) to support older OS versions.


compiler-rt 15.0.0.rc2

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:

It looks like the 'compiler-rt' output doesn't have any tests.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Cold start taking a long time for c# azure function

This is our first use of Azure Functions and we are happy with the idea so far. We have a couple functions setup that will get hit randomly throughout the day so it will never be called warmed up. The issue that we are having is the cold startup time is 30s+.
Here is an example:
Monitor logs says that function was only 2,640ms running time. On our end we saw 30.30 seconds. This request happened 18:34 on Thursday, October 27, 2016 (UTC).
If we call it again right after the first it's a 200ms response time. If we wait 10 minutes it'll take another 30+seconds again.
I've created a dummy app name if you want to look into this: dumm2345
Is this 30 second startup time expected?

Thanks for the info, we will investigate. Request was from RestSharp, right?

Yup. One solution we are starting with right now is to have a timer function run every 30 seconds. It literally does nothing but keeps the service alive and waiting for our random web calls. It's a terrible solution, but it's a hack we can live with for now.

Is it ok to discuss function names here as long as we keep the Web App name private (which is the host name)?

Yup. Go for it.

So initial finding is that I see the following sequence of requests:

18:36:08: Requests for / coming from ReadyForRequest/1.0. Takes 26.5 seconds
18:36:09: Requests for / coming from ReadyForRequest/1.0. Takes 1.3 seconds
18:36:11: Request for /api/CreateZipLink from RestSharp/<IP_ADDRESS>. Takes 29.4 seconds.
18:36:15: Request for /api/GetZipFile/[guid] from Chrome. Takes 3.9 seconds and returns a large response (1MB)

Though probably the first one is the relevant one that kicks in the cold start and that we need to understand.

Yup: 18:36:11: Request for /api/CreateZipLink from RestSharp/<IP_ADDRESS>. Takes 29.4 seconds.
That's the one that we care about. All it does is put a document into documentdb. But I'm pretty sure it's the cold start that's the issue. Thanks for looking into this!

Assigning to @davidebbo for an update

We have done a number of optimizations, and you should be seeing better cold start time than before. We're still working on additional optimizations that will make it yet faster.

We have a similar problem with cold start.
We have an Azure Function used as a webhook for Slash Comman on Slack. One of the requirments of the command is that response must be served within 3000ms. Every time that command used after sometime of inactivity there is a timeout.
You can check the source code here. I tried to strip out everything from it, but the plain response.
Is there only way to keep function "warm" in Consumption Plan? Is pinging it with some scheduler OK?

@davidebbo do these optimizations have implications for .NET functions a a whole? (We're using F# azure functions)

They apply to all function apps. Previously, there would be occasional 30s+ cold starts even for a trivial Function (see first entry in issue). Now those peaks have been eliminated, and the cold start is typically 4 to 10s in the trivial case.

Do you have plans on improving it below 4s?

Yes, see my earlier comment: We're still working on additional optimizations that will make it yet faster. No ETA at this point, but active work is happening now.

I have just created my first Azure function in order to compare it to AWS lambda. All the function does is return a random entry from a static List<string> that contains around 100 elements. When calling the function with a 'cold start' it sometime takes around 35 secs to respond. Subsequent requests take about 0.2 seconds.
My concern is that one of the reasons for exploring serverless functions is to avoid paying for the times that the function isn't being called. Having to periodically call the function in order to avoid cold starts seems to defeat that purpose.

@DavidBrower 35s does not sound right, as the current expectation is 4s to 10s (aside from the large npm tree issue). Can you share the name of your app and the time when you saw such timing?

@davidebbo the name of the function is https://functionsa7b1a72e.azurewebsites.net/api/ObliqueStrategiesV3. Really, most of the time when I haven't called the function for about 5 minutes then I get the delayed response. I'm simply calling curl -sk https://functionsa7b1a72e.azurewebsites.net/api/ObliqueStrategiesV3 from my desktop.

@DavidBrower it really helps to get the UTC time of one such incident, so we are looking at the same thing. I looked at all requests to that function today (i.e. 1/31 UTC) and the longest it took was 7s.

Interestingly, I have not had the very long response times since the weekend. I'm down to 5-6 secs for my cold starts.
I've checked the Azure Functions Invocation Log and I'm afraid that it doesn't seem to keep the invocations for more than a day ago. The best I can do is to say that the first of the long response times started around 16:00 30th January UTC. Sorry that I can't give you exact times. I have been running my tests using curl so I don't have an audit trail of function calls.

Ok, that's good news. Let's keep observing, and let us know if you hit some extreme slow starts again.

Thanks @davidebbo

$ time os
Mechanicalise something idiosyncratic
real    0m13.450s
user    0m0.105s
sys     0m0.652s

Just received 13.45 secs with a cold start at 3 February 18:35 UTC. os is just the alias I am using in Bash to call my Azure function.

Going to chime in here and say that slow start-up, even for simple C# Functions is impacting key functionality we are using them for. We're on standard consumption plans with Always On switched on, but still see substantial ramp-up on initial invocation after some period (I assume 5 minutes). I've resorted to putting a timer triggered app in to our App Service just to try and keep it all warm and toasty.

@sjwaight does that then increase your costs on Azure?

@sjwaight what do you mean by 'standard consumption plans'? There is Consumption, and App Service Plan (which includes Standard), so the two term are mutually exclusive.
Note that I'd like to focus this thread on the issue in Consumption mode. Outside of Consumption, with Always On, it should always stay hot. If it doesn't for someone, let's open a separate issue an investigate why Always On is not working.

89663ms cold start at 2017-07-02 9:55 UTC for a GET on https://cbtraindev.azurewebsites.net/api/bert which is a node function. Had a couple in that region last night as well (no timestamps though, sorry).
Consecutive warm runs are mostly ~100ms sometimes going near 1000ms which is all good.
Are there lately any issues with coldstart perf compared to the previous comments?

We have the same issue, some cold starts take 20s and we require <1s.
I logged a feature request to add the always on option to consumption plans:
https://feedback.azure.com/forums/355860-azure-functions/suggestions/31332643-allow-always-on-mode-on-the-consumtion-plan

everyone should vote the feature request @DL-LiveTiles  made it will benefit all of us :)

Is it still suggested to have a 5min timer to keep things warm?
And when doing so it it enough to just have the timer, or is it needed to make an http request to keep web triggers warm?

We have optimizations that are partially deployed, and should be fully deployed in the next month. They take the cold start down to about 2 seconds. You can try it right now in the West US 2 region.

I have the same issue. I have a testing Function App in consumption plan in south central US. I setup a bi-hourly ping and it takes ~30 seconds to response.

I know this thread is about c# but I ended up here with my javascript app as well. So for anyone having issues on JS: Using the https://github.com/Azure/azure-functions-pack on JS significantly helps!

Looking forward to the hybrid approach between consumption and app-service plans.

I am having the same issue with slow cold starts which led me to set up a function to call two other functions, time response time, and notify me if either function takes longer than five seconds to respond (I wasn't sure if the warm-up needed to occur per-function or per-function app).
I started with triggering this "health check" to be called every five minutes and eventually turned it down to three, but I can still experience hours and hours with cold-start times of 10+ seconds on every single request.
The only input binding on the slow function is to Azure Table Storage, which returns a single row.
Can a function still "go cold" when called every three minutes? Do functions needs to be "warmed up" at the function level or at the function app level?

@davidebbo We use C# precompiled functions and function proxies that we are deploying using ARM and VSTS. We were nowhere near the 2s cold start response time with a simple function. For instance, making a request to a function proxy using an unknown route took the proxy about 17-18s to respond with 404.
I did some testing and found out that disabling PHP on a function app added about 12-14s to the cold start response time. Setting PHP to v5.6 improved the cold start performance considerably. We disabled PHP in our ARM templates because we do not use it and will turn it back on now.

@romandres indeed, there is a known issue that hurts the cold start when PHP is removed. Note that this PHP setting is actually meaningless for Function Apps, so there is no harm in leaving it the way it is by default. We will work to address the issue, but for now just remove all traces of that setting from your templates.

@davidebbo Good to know, thanks. Are there any other settings that are known to impact the cold start performance at the moment?

@romandres sorry, missed this one earlier. I would not expect App Insights to affect cold start. If you see this consistently (fast without, slow with), can you open a separate issue so we can better track?

I am going to add the information I found on this topic:

To speed up the cold start process, run your functions as a package file when possible.
https://docs.microsoft.com/en-us/azure/azure-functions/run-functions-from-deployment-package

Azure Functions Pack (JS) packs dependencies for fast cold start
https://github.com/Azure/azure-functions-pack

A blog on cold starts
https://blogs.msdn.microsoft.com/appserviceteam/2018/02/07/understanding-serverless-cold-start/

@davidebbo I too have this problem. To me, the hybrid approach doesn't sound too great because I'd need to buy dedicated VMs for most of my expected consumption (with low latency) and use serverless functions only for the bursts (and of course the bursts will take a long time). This almost defeats the purpose of going truly serverless. Also, I was content with using the Web API framework as Functions seems to lack quite a bit of features, but it was a better choice to shave off costs. If I'm going to use actual VMs, I'd rater go back to Web APIs and have access to an actual HTTP pipeline and things like [Authorize] attributes.
Having said that, I may be misunderstanding the hybrid approach here. Why not use a dedicated VM only for the bursts instead? If there are no warm instances available, then use the dedicated VM while the infrastructure warms up another instance behind the scenes.

@adrianknight89 I'm no longer in this project, so I will let others respond (e.g. @paulbatum). But this is an old issue and things have improved quite a bit since. I suggest that you add details on your scenario: e.g. what language, what OS, what runtime version, are you using Run As Package, what kind of timing you are observing, etc...

@paulbatum I’m using C#, Windows 10 (idk why OS matters though), v2, and zip deployment on the consumption tier.
My biggest complaint at the moment is every time I publish new changes (which I do very frequently), everything seems to be deployed to another server, and I incur the full cold start penalty.
I’ve read somewhere that v2 wasn’t as optimized as V1, but this was when it was still in preview. Do you still believe v2 lacks optimization compared to the older version?
As for dependencies, this is probably not the greatest idea, but why don’t you install some of the most popular libraries on your servers (and say the last 5-10 releases of each) and fallback to copying the files on the fly in case they don’t exist on the destination server? A lot of people use Newtonsoft, EF, Azure client SDKs, etc. If the source and the target have the same dependencies and versions, you could skip moving them over the wire.

I'm experiencing cold startup times exceeding 20 seconds in Consumption plan for my trivial C# function. I ping it every 5 minutes using App Insights to keep it warm, but it still shows sporadic >20s responses. What do I do?

@ogvolkov  I've started to experience the same issue too.
I have an availability test, that pings my API Management, that calls my AF v2

This issue has been opened for a while and has covered a few different scenarios over the years. To make sure we're investigating new issues reported (as they won't have the same root cause), please open new issues so we have the right context and things are not missed.
Closing this.


[typescript] react union types props is broken by @types/styled-components ?

Environment

macOS Big Sur 11.1
node v14.3.0
npm: 6.14.11

Reproduction
Example:
https://codesandbox.io/s/dazzling-sun-2drq9
Steps to reproduce

create typescript styled-components project ( install @types/styled-components etc. )
define a union object type, like this.

type Props = {
  type: "a";
  hoge: number;
} | {
  type: "b";
  hoge: string;
};

create react component using Props, like this.

const Component: React.FC<Props> = () => <div />

wrap it in styled-components, like this.

const StyledComponent = styled(Component)``;

use component, like this.

    <StyledComponent type="a" hoge={1} />     // <- OK: type is correct
    <StyledComponent type="a" hoge="str" /> // <- NG: should types error

Expected Behavior
The component props type should be inferred, and report type error, same as normal react component.
    <Component type="a" hoge="string" />
    ...
    Type '{ type: "a"; hoge: string; }' is not assignable to type '(IntrinsicAttributes & { type: "a"; hoge: number; } & { children?: ReactNode; }) | (IntrinsicAttributes & { type: "b"; hoge: string; } & { ...; })'.
  Types of property 'hoge' are incompatible.
    Type 'string' is not assignable to type 'number'.

Actual Behavior
It does not occur types error.
Why it is happened
I think it is occurred by using Omit in StyledComponentProps.

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/styled-components/index.d.ts#L73-L82
https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/styled-components/index.d.ts#L308-L310

Typescript Omit make union type changed other type.
type AorB = {
  type: "a";
  hoge: number;
} | {
  type: "b";
  hoge: string;
};

type A = Omit<AorB, never>
const a: A = {type: "a", hoge: "str"}; // not error:

Type A is inferred following type by typescript.
type A = {
    type: "a" | "b";
    hoge: React.ReactText;
}

So hoge is allowed number and string regardless of type: "a".
Suggestion

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/styled-components/index.d.ts#L73-L82

If type A is never, can it avoid Omit ?
type IsExtendsNever<T> = T extends never ? true : false;

export type StyledComponentProps<
    // The Component from whose props are derived
    C extends string | React.ComponentType<any>,
    // The Theme from the current context
    T extends object,
    // The other props added by the template
    O extends object,
    // The props that are made optional by .attrs
    A extends keyof any
> =
    // Distribute O if O is a union type
    O extends object
        ? WithOptionalTheme<
            false extends IsExtendsNever<A> ?    //  <- If A is not never then use Omit
              Omit<
                  ReactDefaultizedProps<

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/styled-components/index.d.ts#L308-L310

WithOptionalTheme does not use Omit.
type WithOptionalTheme<P extends { theme?: T }, T> = P & { theme?: T };

@ka2jun8 Great research! This is very annoying bug imo!
Can you make PR about your suggestion so we get this problem solved?

@henrikra
Thank you.
I send the PR:
Please review it.
https://github.com/DefinitelyTyped/DefinitelyTyped/pull/51252

I'll just post my usual TypeScript snippet here for reference:

We don't maintain any TypeScript typings in the main repository and are not updating the ones that live in definitely-typed, so we can't offer any help on this here. It's possible that if you browse past issues you'll find similar questions, but overall this repository's issues are not the right place to report these issues or to discuss them.

@ka2jun8 In general, I see that the above has been merged and closed. Please don't forget to close issues here as well, especially if it doesn't end up relating to the main repo. 🙏 Cheers!


network, masquerade: remove dead code

What this PR does / why we need it:
This code removes an unused function from the masquerade infrastructure configurator; the same functionality is provided by
https://github.com/kubevirt/kubevirt/blob/5083ee7b91bd25ce9186f0dc2d68322ee810d7c6/pkg/network/infraconfigurators/masquerade.go#L476
Release note:

NONE

/cc @rhrazdil

/lgtm


Set my.cnf permissions

I completed the 3 steps below:

[x] I've read the Contribution Guide.
[x] I've updated the documentation. (refer to this for how to do so).
[x] I enjoyed my time contributing and making developer's life easier :)

Perhaps change pull request title to “Set my.cnf permissions”. Was not clear what permissions you wanted to change here. Does seem like a possible useful change . But did you have permission problems with it then?

Yes, if the rights are not set, the custom configuration is not applied.


Settings button

Settings icon button created

Great work thank you!
@all-contributors please add @pari-27 for code


Better handling of invalid coordinate values

In https://github.com/Leaflet/Leaflet/issues/3424, @hs06 reported that when importing a KML via leaflet-omnivore with an empty coordinate array, Leaflet crashes.  I've also duplicated this issue with a KML which returns a [NaN] array.  You should be able to duplicate this via the file below, as it is a KMZ renamed to ZIP to conform to Github's attachment file type restrictions.
Portland Trails Presumpscot River Preserve.zip
This code snippet reads in a KML string from a file and fits the map view to the bounds of the paths:
    leafletData.getMap('map-preview').then(function(map) {
        var kml = fileReader.getLocalFile();  //Angular service to store text read from file

        if (kml === null)
            $log.error("No file loaded");
        else {
           //transform KML string to DOM object and parse
            var kmlDom = (new DOMParser()).parseFromString(fileReader.getLocalFile(), 'text/xml');
            var pathData = toGeoJSON.kml(kmlDom);

            angular.extend($scope,  {
                geojson : {
                    data : pathData,
                    style : {
                        dashArray : '3',
                        color : 'black',
                        weight : 8
                    }
                }
            });

            var latlng = [];

            //iterate over GeoJSON features to create an array of all coordinates in data set
            for (var feature in pathData.features) {
                var paths = pathData.features[feature].geometry.coordinates;
                paths = _.compact(paths);

                for (var path in paths) {
                    var prospectiveCoords = pathData.features[feature].geometry.coordinates[path];
                    var coordinate = L.GeoJSON.coordsToLatLng(prospectiveCoords);
                    //$log.debug(coordinate);
                    latlng.push(coordinate);
                }
            }

            map.fitBounds(latlng);

            //Use parsed GeoJSON data to add a layer to the map.
            //Cause of Error: Invalid LatLng object: (NaN, NaN) in Leaflet.
            L.geoJson(pathData, {
                style : {
                    color : 'black',
                    weight : 10
                }
            }).addTo(map);
        }
    });

I can see two possible solutions to this:

Throw a verbose error when encountering an invalid numerical value when building coordinate arrays
If any invalid or NaN value is found during processing, remove that coordinate array from the parsed results and display a verbose error

I'm inclined to keep the behavior as-is. toGeoJSON is a KML converter, not a KML validator, and it should not be any more tolerant of incorrect data than it must be.

Agreed, but raising an exception when invalid data is encountered is good practice, and the developer can then decide whether to act upon said error or ignore it.  Currently there is no way to make this choice.

Parsers do not always throw when invalid data is encountered; they throw when the source data is incorrect in a way that breaks the parser. Throwing whenever 'invalid data is encountered' necessitates building a KML validator - anything else will be fixing this narrow case you found, and letting the infinite other invalid potentials pass through, such that toGeoJSON will be a validating parser for one particular, relatively rare, issue.
Closing; please use a KML validator if you want to ensure that your data is valid.


✨ Introducing Peaceout Omini

PR containing all the info in the JSON file and the associated media to integrate peaceout on Starknet Ecosystem. Thanks

Hey @kelvinnguyenIT thanks for your contribution, can you please point us some links where you explain your involvement on Starknet? Thanks

Hey @kelvinnguyenIT thanks for your contribution, can you please point us some links where you explain your involvement on Starknet? Thanks

Our team is testing deploying our NFTs on the starknet and are taking the final steps to go on the testnet.


Sync package-lock.json to package.json version

#27 didn't update the version in package-lock.json. I don't know what (if anything) this will break, but I assume they should be synced.

Is it possible to add a test / check for this as part of merging a PR?

This seems promising: https://stackoverflow.com/q/57998744/172999

I've noticed that a some OSS libraries simply don't ship a package-lock.json at all. Not sure what the pros/cons of doing this are.

The NPM utility itself encourages you to ship package-lock.json. It prevents you from getting different results locally / on CI / on users' machines, in the case where a newer version of a dependency has been released in between the last time you did npm install and the time when a user does npm install.
Personally I really hate checking in long autogenerated files that produce giant pull request diffs, but to me the above benefit weighs stronger than the dislike :smile:


[DMS-251-v1] - Creation of DataStrictness. feature file

As a general comment, I would suggest running the tests and enabling the ones that are passing.

Please mark the ones that are failing with the @ignore Tag

A big lesson I learned with these updates: if the scenario is going to run a query that accesses OpenSearch (that is, any GET request other than a request for a specific /{id}), then it is best to setup the initial data in a background scenario with @addwait, so that there is time for the Kafka/OpenSearch eventual consistency synchronization to complete before running the assertions.


CAMEL-10543: possible new minimalist logo

Another possible logo, very minimalist.

This minimal logo reminds me of that man that was drawn from a line
https://en.wikipedia.org/wiki/La_Linea_(TV_series)

Hi we are voting on the received proposals, the vote is open till 15.5. see all the proposals and cast the vote for the three designs you prefer the most.
Also read the news announcement on Camel blog.

Since the vote ended I'm closing the related PRs


Component:- Locale Modal:- Search by Location or Language modal: Close button

Issue Description:-Close button is announced as "Select geographic area button" instead.
Recommended Fix:- Provide label as "close"
Happening on screen reader.
url:- https://ibmdotcom-react-canary.mybluemix.net/?path=/story/components-locale-modal--default

@chetankan Could you please test again?
https://ibmdotcom-react.s3.us-south.cloud-object-storage.appdomain.cloud/deploy-previews/3447/index.html?path=/story/*

@Praveen-sr mentioned on QA call today now it's working as expected.
I'm going to convert the Draft PR to an official one and will ask engineers to review it.
@RobertaJHahn FYI

Working fine. Now the label is announced as "close modal" Tested in FF with JAWS

@Praveen-sr Thanks! Moving to review to wait for PR review.


Convert Static functions into global functions

It would be cool if you convert static functions into global functions. For example instead:
import 'package:file_utils/file_utils.dart';

main() {
  print(FileUtils.getCwd());
}

it would be better just to do:
import 'package:file_utils/file_utils.dart';

main() {
  print(getCwd());
}

or:
import 'package:file_utils/file_utils.dart' as fileUtils;

main() {
  print(fileUtils.getCwd());
}

Is that still necessary for someone?

Each of my project often depends on my other projects.
This means if I make correction in some my project then I'll have to make corrections in my other projects.
The problem is that the popularity (or unpopularity) of the Dart language aroused in me a reluctance to work with the that programming language.
I just don't want to waste my time.
Given the fact (if you believe the words of Kasper Lund), that Dart is the most quickly growing in the popularity language in Google, it should be expected in the near future the emergence of a large number of professional programmers from Google.
I think that their code will surpass the quality of my code by many times.
About the functionality I even did not want to say: "As an amateur like me will be able to compete with the professionals of Google?".
Compared to them, I am a louse that lives in underpants.

I'll be more than happy if you transfer the ownership of this repo to me.

Catch a repository, friend!


Fix troubleshooting page URL.

Troubleshooting reference in Github issues template is old. Let's point to the brand new page dedicated to troubleshooting.

Well spotted, thanks a lot for this !

Thanks fir the quick merge! :)


components: Component instances don't think they are component instances

Given this input:
(component
  (component $c
    (component (export ""))
  )
  (instance $x (instantiate (component $c)))
  (alias export $x "" (component $f))
)

which is:
  0x0 | 00 61 73 6d | version 65546 (Component)
      | 0a 00 01 00
  0x8 | 05 18       | [component 0] inline size
    0xa | 00 61 73 6d | version 65546 (Component)
        | 0a 00 01 00
   0x12 | 05 08       | [component 0] inline size
     0x14 | 00 61 73 6d | version 65546 (Component)
          | 0a 00 01 00
   0x1c | 07 04       | component-export section
   0x1e | 01          | 1 count
   0x1f | 00 01 00    | export ComponentExport { name: "", kind: Component(0) }
 0x22 | 06 05       | instance section
 0x24 | 01          | 1 count
 0x25 | 00 01 00 00 | [instance 0] Component { index: 0, args: [] }
 0x29 | 09 05       | alias section
 0x2b | 01          | 1 count
 0x2c | 00 01 00 00 | alias [component 1] InstanceExport { kind: Component, instance: 0, name: "" }

this yields from validation:
Error: instance 0 is not a component instance (at offset 0x2c)

I believe this is because the type registered for the component is TypeDef::Component rather than TypeDef::Instance since the type originated from a component type.
cc @peterhuene

I believe another consequence of this is that this component:
(component
  (module $m (import "b"))
  (instance (export "i") (instantiate (module $m)))
)

  0x0 | 00 61 73 6d | version 65546 (Component)
      | 0a 00 01 00
  0x8 | 01 03       | type section
  0xa | 01          | 1 count
  0xb | 4f 00       | [type 0] Module([])
  0xd | 02 04       | import section
  0xf | 01          | 1 count
 0x10 | 01 62 00    | [module 0] ComponentImport { name: "b", ty: 0 }
 0x13 | 06 05       | instance section
 0x15 | 01          | 1 count
 0x16 | 00 00 00 00 | [instance 0] Module { index: 0, args: [] }
 0x1a | 07 05       | component-export section
 0x1c | 01          | 1 count
 0x1d | 01 69 02 00 | export ComponentExport { name: "i", kind: Instance(0) }

fails validation with as:
Error: instance 0 is not a component instance (at offset 0x1d)

Currently the spec does not allow exporting module instances, only component instances. It may import and export modules, but not their instances. Thus I think the second failure is correct.
I don't think that's changing with the pending updates, either.

Of course, I may have read the spec wrong given the overloaded instance index space...

The updated spec also seems to imply that module instances cannot be imported or exported, FYI.

Aha yes no that's my mistake!
I do think though that this issue affects this module:
(component
  (component $m (import "b"))
  (instance (export "i") (instantiate (component $m)))
)

which should validate but doesn't

Definitely, that's a bug. I plan on knocking all of these validation bugs off today so that we can get as many tests running in main before I land the spec update so we have some confidence in the changes, at least.


How to use csv file to import data?

hallo:
I want to use csv file to import the price data,or other files(ex.hdf5 file)?
Thanks

Hi,
Please see issue https://github.com/enigmampc/catalyst/issues/65 for details.
Lena


update use of namespaces in chart

To ensure Helm releases are installed in the desired namespace it is important to always specify the release namespace: helm install -n <namespace>.
This MR fixes the chart so it will follow this approach (which is the standard way since I think Helm 3 at least), instead of using a value to control the namespace.
If you helm install without -n it is the same as using kubectl without -n, it should use the default namespace configured in the current context of your kubeconfig. However the current non-standard behaviour of the chart (which I would argue is a bug) overrides this to end up in kube-system instead of whatever your current default is - IF the user doesn't specify it with -n (but you really should).
Actually the more serious bug is that the current values-based namespace selection could potentially prevent the standard release-based namespace selection mechanism from working at all, so a user doing the right thing (helm install -n mynamespace) would potentially end up with the helm release details going in the right namespace and the k8s objects going in the wrong namespace, I am not sure how that would work.
fix https://github.com/ROCm/k8s-device-plugin/issues/60
Also:

There is no such thing as a namespace for a ClusterRoleBinding, k8s ignores it. Remove 'default'
There is no reason to be granting RBAC privileges to the default service account in the default namespace, removed

Here are my tests:
$ helm -n amd-device-plugin install --create-namespace  -f ~/amd-values.yaml  amd-device-plugin   ~/amd-k8s-device-plugin/helm/amd-gpu/
NAME: amd-device-plugin
LAST DEPLOYED: Fri Apr 19 21:36:48 2024
NAMESPACE: amd-device-plugin
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
amd-gpu-device-plugin-daemonset deployed in namespace 'amd-device-plugin'
amd-gpu-labeller-daemonset deployed in namespace 'amd-device-plugin'

# daemonsets end up in the right namespace as expected
$ kubectl -n amd-device-plugin get daemonset
NAME                              DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                                                                                AGE
amd-gpu-device-plugin-daemonset   0         0         0       0            0           feature.node.kubernetes.io/pci-0300_1002.present=true,feature.node.kubernetes.io/pci-1002.present=true,kubernetes.io/arch=amd64   4m13s
amd-gpu-labeller-daemonset        0         0         0       0            0           feature.node.kubernetes.io/pci-0300_1002.present=true,feature.node.kubernetes.io/pci-1002.present=true,kubernetes.io/arch=amd64   4m13s

# this confirms that the RBAC works correctly
$ kubectl auth can-i get nodes --as=system:serviceaccount:amd-device-plugin:default
Warning: resource 'nodes' is not namespace scoped
yes

@y2kenny can you please take a look?

@y2kenny  Sorry to bother but have you had a chance to take a look?

Hi @rptaylor, sorry for the delay.  I see that some of the changes replace .Values.namespace with .Release.Namespace while in other places .Values.namespace is removed entirely.  Can you tell me the reason behind the inconsistency?

oh wait... I think you explained it in your note.  Never mind.


chore(@embark-site): add all links to status network projects

Well done @jrainville !

Website updated and deployed


feat: JAAS groups schema and CRUD

Description
This PR implements a resource for jaas groups, updates the jaas client to include required functions and covers the added functionality with tests.
Includes previous work on jaas resources by @kian99, review only the latest commit to see changes unique to this PR.
CSS-9770 / CSS-9772
Type of change

Add new resource

Just taking a brief looking at this,

Change the target branch to jaas-resources rather than main.
I suggest splitting up this PR to add the client code and tests in one PR then the crud in a separate PR.

/merge


The pre-trained model can't achieve the accuracy presented in the paper

Hi hwwang55, I cloned the repo and wrote the code below to evaluate the performance of pre-trained model on CA-GrQc test set, but only got 75.98% using generator embedding, which is far away from 84.9% stated in the paper. Is there anything wrong?
n_node = 5242
def evaluation():
    lpe = lp.LinkPredictEval(
        config.pretrain_emb_filename_g,
        config.test_filename,
        config.test_neg_filename,
        n_node,
        config.n_emb
    )
    result = lpe.eval_link_prediction()
    print(result)

if __name__ == '__main__':
    evaluation()

Did you try to increase the value of update_ratio in config.py? And, you can also try tuning other hyperparameters.

@hwwang55 No, I didn't tune the model further and just loaded the embedding in "pre_train/link_prediction/CA-GrQc_pre_train.emb". Is it necessary to fine tune the model in order to get 84%+ accuracy?

I think I


use non main instance

see https://github.com/dimagi/commcare/pull/218

Should target 2.25 if it is cross-requested with https://github.com/dimagi/commcare/pull/218

Recreated at https://github.com/dimagi/javarosa/pull/206


Adding proper error handling

Currently, with most of the error, the canary tool just panics leaving the application to exit.
We should think about adding proper error handling where it makes sense, trying a retry or recovery logic, or even just exiting.

+1 with the idea of retry, but we need to make sure that an SRE observing the metrics have sufficient metrics to know that the canary is in trouble.  I'd suggest a monotomicly increasing error count metric.

I totally agree with this, having an error metric with one or more labels specifying the kind of error. But do you agree that it can be addressed working on #5? It's more a wider problem than just this issue.
Maybe you can add this comment on #5 so that we can track the idea about a corresponding metric?


Backport — SD-1603 Add LookupType term to Mounting algebra

In support of downstream sub-path permission handling

@mossprescott I assigned you since you've already reviewed these changes.

LGTM

[Current coverage][cc-pull] is 79.94%

Merging [#1184][cc-pull] into [backport/v4.5.6+][cc-base-branch] will decrease coverage by -0.17%

2 files (not in diff) in ...rc/main/scala/quasar were modified. more
3 files (not in diff) in ...ain/scala/quasar/std were modified. more

Misses +1
Hits -1

2 files (not in diff) in ...ain/scala/quasar/sql were modified. more
2 files (not in diff) in .../physical/mongodb/fs were modified. more

Misses +1
Hits -1

12 files (not in diff) in ...sar/physical/mongodb were modified. more

Misses +7
Hits -5

2 files (not in diff) in .../scala/quasar/jscore were modified. more
2 files (not in diff) in ...cala/quasar/fs/mount were modified. more
2 files (not in diff) in ...main/scala/quasar/fs were modified. more

Misses +1
Hits -1

2 files (not in diff) in .../scala/quasar/effect were modified. more

Misses +4
Hits -4

5 files (not in diff) in ...rc/main/scala/quasar were modified. more

Misses +3
Hits -3

@@           backport/v4.5.6+      #1184   diff @@
==================================================
  Files                   163        163
  Lines                  8938       8943     +5
  Methods                8724       8721     -3
  Messages                  0          0
  Branches                 91         99     +8
==================================================
- Hits                   7161       7149    -12
- Misses                 1777       1794    +17
  Partials                  0          0

Powered by Codecov. Last updated by [d43b0fa...c59e244][cc-compare]
[cc-base-branch]: https://codecov.io/gh/quasar-analytics/quasar/branch/backport/v4.5.6%2B?src=pr
[cc-compare]: https://codecov.io/gh/quasar-analytics/quasar/compare/d43b0fafe94824d95e2d4d695f585501c08240c5...c59e244dee2f8594117d44b724a09d3507503d4a
[cc-pull]: https://codecov.io/gh/quasar-analytics/quasar/pull/1184?src=pr


Transversers lose precise types

Currently all methods in transversers.Api return Tree as result type:
val q"..$mods object $name extends $template" = defn
val newTemplate: Template = template.transform { // found   : scala.meta.Tree
                                                 // required: scala.meta.Template
  case q"..$mods def $name[..$tparams](...$paramss): $tpeopt = $expr" =>
    q"???"

q"..$mods object $name extends $newTemplate // requires Template"
}

Possible workaround is to use asInstanceOf.

Improving on the signature of Tree.transform would be nice, but noone had an idea how to do that in almost a year. Since this functionality is not on our roadmap, I'll be closing this issue.


Add right icon variant to IC-Button

What is the purpose of the work?
To add an option to have an icon to the right of the label for th eic-button component
Why do we need it?
This is initially required for the buttons on the datepicker, but would be a generally useful enhancement

could we also fix an issue where the stories have all the buttons heights as 9rem?

Introduce leftIcon and rightIcon, deprecate icon and add a new ticket to remove in v3


Add license header to all files

Add licensei to the tooling and CI to verify headers (example: https://github.com/bank-vaults/vault-secrets-webhook)
Add license headers to Go files (addlicense can make it easier for the first time)

Copyright holder is Tailfin Cloud Inc..

can i do this @sagikazarmark, actually i was reading the openmeter codebase and this would be a great start for contributing

@KunalSin9h feel free to!
I updated the description.


Add requirePorts to Minuteman and Octarine test apps

The marathon ports API is weird, when you don't say  requirePors, the port being specified is the apps "servicePort", and the app gets launched / assigned a random port on the host.
This does two things, it adds requirePorts, so we're specifying the exact port to get, and it changes the port so that we're not conflicting with anything for both the apps.

cc: @sargun, @nlsun

LGTM


Make TileMap toolbar a FlowContainer

Closes https://github.com/godotengine/godot/issues/63851

Thanks!


fix: Ignore terraform working files

Hi @jribble, thanks for your contribution! It looks like this PR is failing a validation step. The workflow is expecting one of the following prefixes in your commit message:
      fix
      docs
      style
      refactor
      perf
      test
      build
      ci
      revert
      chore```

I noticed that the contribution guidelines don't reference this validation rule, so I'll go ahead and update the doc and push this through.


Move predictor into caffe2/caffe2/predictor

Summary: One part of refactor predictor. Move all the files into predictor dir.
Differential Revision: D8845276

Have these header files already been moved (or will be soon)? It seems they're still under https://github.com/pytorch/pytorch/tree/master/caffe2/core ?
Also, have those changes already been propagated to the latest pytorch-caffe2* conda packages under https://anaconda.org/caffe2 ?

I have another pr https://github.com/pytorch/pytorch/pull/9548 for the rest of the changes there. Maybe I should land that first and then this one.


About the formula in the appendix

Hi,
I would like to consult you about the formula in A.2.Derivation in the paper. For "which is a differential of log-SNR(t) regarding time-step t", the differential of log-SNR(t) should be SNR(t)'/SNR(t)，not -SNR(t)/SNR(t)'
If I misunderstood, please reply me as soon as possible, thank you very much

As written in eq.7 of section 2.2 Training Objectives, the weight lambda_t is minus reciprocal of the differential of log-SNR(t).

And yes, the differential of log-SNR(t) is SNR(t)'/SNR(t).

Thanks for your replying.


Cam problem by changing unity scenes

Description
Hello,
I'm not sure where to report this.
I'm using the Holistic solution in an university project and I'm quite new to Unity.
So here is my go:
I have two scenes: A gameScene and a menuScene.
I start the menuScene and press play, so that the gameScene gets loaded. The calibration starts (in the now loaded gameScene), I map the Mediapipe model on a player model and "play". After the game ends, I switch back to the menuScene. If i come back to the game scene (by clicking "play" again), I want to calibrate new.
By doing this, I get the following error:

After I removed the exception throws, I got this error:

I think the access for the webcam is still in use and after creating the new scene the reference is still there.

Is there a way to be sure the camera is not occupied before the new  initialization?

I tests a bit,
if i connect two cameras to the PC and change the Cams,back and forth, after the scene change.

Is there a way to be sure the camera is not occupied before the new initialization?

Just stop the WebCamTexture before starting a new one (cf. OnDestroy).


fix env switch declare err from rdev -> wallet

LGTM, we'll have to raise another issue for the other edge case @stranger80


pi-blaster works but pi-blaster.js does not

I have attempted to use the pi-blaster.js and have tried everything I can think of to debug it...I have followed the examples with no luck.
When I use pi-blaster directly from the command line everything works fine, I can send pwm signals. When I attempt to use pi-blaster.js from within a very simple nodejs application of even at the node console nothing happens.

Can you try again? I think that was fixed thanks to #8.

The changes in #8 plus adding curly braces around the other single line if statements did the trick! thanks for your help. It looks like the GitHub file already has those braces but the version npm downloads does not...Thanks again!

Thanks for pointing this out. I had fixed on github but not republished. I just published v0.1.1. NPM should be ok now.


NYU Dataset training error

Hello,
When I tried to run train_local.py for NYU dataset, it gave an error in eval stage and error is:
"fuse_features = local_features * (scale+1) + shift_
RuntimeError: The size of tensor a (552) must match the size of tensor b (559) at non-singleton dimension 3"
How can I solve this error?
Thanks.

When I interpolated the tensor to match the size, RGB results are far from yours' : ODS=0.709, OIS=0.725, AP=0.722

When I interpolated the tensor to match the size, RGB results are far from yours' : ODS=0.709, OIS=0.725, AP=0.722

could you tell me how to  interpolated the tensor to match the size？

We update the code, please refer line 385-386,436-437.

I used these changes, now training is successfully finished. However, my results, ODS=0.702, OIS=0.718, AP=0.699,  are too lower than the published results. What could be the possible reason for these differences?


removes deprecated role attribute, adds tests for base_role_type

Coverage increased (+0.0%) to 82.08% when pulling 1bea060715fbf8b298117ccf77feff787b869555 on feature/canvas-role-labels into a311c86ba280ef5a9a77e70cc3a8593d24ba8fba on develop.


[bug] Strict input checking

GitHub actions don't validate that inputs were passed to an action even if the input is marked as required.
We should be more strict about input checking and produce errors if the inputs were not given.
For example, a call to secure-project-checkout-go could omit the go-version input even though it's required. This would result in an empty value being passed to setup-go which would install the "latest version" as determined by setup-go.
      - name: Checkout the Go repository
        uses: ./__BUILDER_CHECKOUT_DIR__/.github/actions/secure-project-checkout-go
        with:
          path: __PROJECT_CHECKOUT_DIR__

Shall we create an issue on their runner repo?

Shall we create an issue on their runner repo?

Seems there is an issue already https://github.com/actions/runner/issues/1070

In the meantime, it looks like we should use the required option on getInput when getting required inputs in TypeScript actions.
We should also do explicit checks in composite actions.
Maybe something like:
- name: validate inputs
  id: validate-inputs
  env:
    REQUIRED_INPUT: ${{ inputs.required-input }}
  run: |
    if [[ "${REQUIRED_INPUT}" == "" ]]; then
      >&2 echo "required-input is required!"
      exit 1
    fi

We could even create an action that would do this for us based on the required flag on the input but it would probably need to be like the TRW inputs thing and download the action.yml, parse it, and look for required inputs.


fix(network): ensure accurate btc balance

We had satoshisToBtc hardcoded for our network (channel) balance. This updates it to use the Value component to ensure it displays correct balance depending on if the user is viewing in sats, bits or btc

Pull Request Test Coverage Report for Build 3850

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 14.964%

Totals

Change from base Build 3837:
0.0%

Covered Lines:
571

Relevant Lines:
3417

💛  - Coveralls

Pull Request Test Coverage Report for Build 3850

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 14.964%

Totals

Change from base Build 3837:
0.0%

Covered Lines:
571

Relevant Lines:
3417

💛  - Coveralls


onItemAtEndLoaded(itemAtEnd: Any) returns first element

I am implementing paged list using PagedList.BoundaryCallback. I need to load next page based on last loaded item in a list. Callback for this is onItemAtEndLoaded(itemAtEnd: T). But why itemAtEnd is the first item of PagedList? It should be last item of PagedList

Sorry that we never got to this - the issues here are mainly for the samples, the best way to get traction on an issue / question with the library is to file a bug against our issuetracker at: https://issuetracker.google.com/issues/new?component=413106&template=1096385
However the v2 API PagedList.BoundaryCallback has been superseded by RemoteMediator in v3, and realistically there won't be any more work on the v2 APIs as v3 approaches a stable release.


No 1.35.0 tag

Supermarket shows the latest version of the java cookbook as 1.35.0 but I can't seem to find any tag.

Tagged


Add a StacCollection.search method

Purely as a convenience, it'd be nice to have a StacCollection.search method that uses pystac-client to search an endpoint with a specific collection.
cat = intake.open_stac_catalog("/path/to/catalog")
collection = cat["my-collection"]
collection.search(bbox=bbox)

The .search method would use pystac-client

Find the link with a "rel": "search". Set that as the endpoint
Specify collections=[self.id], to limit the search to just that collection.

I see now that intake's base Catalog class apparently defines a search method, which appears to do some kind of text-based search on the items.

This sounds great @TomAugspurger! I personally don't see any good reason to avoid overriding the base class search method but we should ask @martindurant for his thoughts.

Please do make specialised versions of search(), the one in Catalog is super-simplistic and only meant to be a fallback when nothing better is available.

The .search method would use pystac-client

@TomAugspurger I really like this idea, but it would need some docs / error handling for the cases that "rel":"search" doesn't exist.
I'm thinking of the case of a static catalog/collection, where there is no API endpoint. For that case we could:

Stick with default intake.search() that I think just does some string pattern matching.

Or implement a simple 'local api' search with the same keywords that to filter by bbox and datetime (e.g. geopandas operations on the static catalog represented as a GeoDataFrame https://github.com/intake/intake-stac/issues/36). @matthewhanson probably has some ideas on performance here, and things would likely be quite slow if someone tries this on a really big catalog.

I do rather like the idea of being able to do a search on a static catalog, but that seems like it should be implemented in pystac-client as well (which is named as such as it's a client for both static catalogs and APIs).
Currently pystac-client will raise an APIError if there is no rel=search link

The main limitation is that it depends on having a geodataframe version of the stac metadata

I am not in a good position to reckon how much of a limitation that is.