Enhancements to @bind - globalization and conversions

Edit: @rynowak hijacking the top post for great justice
Summary
@bind when used with <input ...> elements is the primary way that users parse and convert user input into .NET types.  This makes @bind the primary touch point for globalization - dealing with input and display of information in a locale specific way.
Goals:

Support users using any locale in client-side/server-side Blazor
Support users using many locales with a single server for server-side Blazor
Support flexible custom conversions and formats
Provide good support for common built-in .NET types

TLDR My recommendations for 3.0:

[ ] Make sure user's culture is reflected in .NET

[ ] Client-side
[ ] Prerendering
[ ] Server-side (circuits)

[ ] Flow type="..." to .NET to use smarter defaults for parsing
[ ] Add @bind support for:

[x] DateTime?
[ ]  DateTimeOffset
[ ]  DateTimeOffset?

[x] Support TypeConverter

CUT

Format strings for numbers
Specifying culture per @bind
Customize how browser-side JS code determines the culture
Unusual primitive types like byte
Our own converter primitive (use TypeConverter)

Related user-feedback

https://github.com/aspnet/AspNetCore/issues/5471 Runtime Culture and Localization
https://github.com/aspnet/AspNetCore/issues/5523 Add support for binding decimal, double, float data type with configurable formatting
https://github.com/aspnet/AspNetCore/issues/5651 'bind' does not accept values of type System.DateTimeOffset
https://github.com/aspnet/AspNetCore/issues/7655 Razor Components (Blazor 0.8) - "double/float/decimal" error parse
https://github.com/aspnet/AspNetCore/issues/8203 Razore Component Doesn't Support "short" Data Type
https://github.com/aspnet/AspNetCore/issues/8493 TypeConverter for parameters
https://github.com/aspnet/AspNetCore/issues/9435 Blazor Custom formatting for numbers and strings
https://github.com/aspnet/AspNetCore/issues/9632 typeparam bind error
https://github.com/aspnet/AspNetCore/issues/9939 Blazor form component InputSelect should support binding of type Guid
https://github.com/aspnet/AspNetCore/issues/10077 EventCallback Error cannot convert from 'method group' to 'EventCallback'
https://github.com/aspnet/AspNetCore/issues/8385 EventCallback not working as expected

Challenges
HTML5 field definitions make dealing with globalization more complicated, because depending on the field type, the culture-sensitivity expectations are different.
<input type="text">Enter a number in German</input>
<input type="number">Enter a number in German</input>

In the type="text" field the .value attribute will contain exactly what the user typed in - this implies that the value needs to parsed, or formatted according the users' locale.
In the type="number" field the .value attribute will contain a number value in an standard non-locale-specific format. The value must be parsed and formatted according to this standard format. It's up to the browser to support the users' preferred style of display for their locale.
It stands to reason that users will want to use text fields for numbers, dates, etc in some cases instead of the HTML5 equivalents. Using text is much more flexible since it's essentially free-form text, as opposed to number which is restricted by the browser, and has a set of UI behaviors the developer might not want.
Additionally, for the server-side Blazor programming model, there's no automatic way to flow the culture from the client to the server. We have to build it in to Blazor.
Fixes and Solutions
More control over @bind
Some ideas here... we don't have to do all of these, and probably shouldn't try to do all of these in the first release.

Flow the field type into .NET: We could easily capture data about the field type in the JS side of the binding code. This would allow the .NET code to be aware of what the type="..." attribute specifies - which we could use to perform the conversion using invariant culture or the user's culture. With this done, we could make the default be culture-sensitive, and apply invariant culture where appropriate based on the field type.

Allow users to specify the culture for @bind We could allow users to specify the culture at the call-site. This could be a desirable level of control for application developers, but it's not a good default experience - essentially we'd be picking a default behavior and allowing an override. This isn't a good default user experience by itself -  because as we've already seen, HTML5 fields like number want a different default than `text.
Directive attributes make it easy for us to build something like this
<input type="text" @bind="coolnessLevel" @bind:culture="CultureInfo.InvariantCulture">
  Rate the coolness of Blazor (in a non-locale-specific way)
</input>

Allow format strings for more types Currently we support format strings for DateTime and only datetime. This is useful because DateTime is used to represent a variety of different kinds of values, times, dates, both, etc.  This is also useful because DateTime supports a format string on both formatting and parsing.
It might seem like allowing format strings on more types will free us from the need to do some of these other solutions - however:

Format strings in .NET are non-locale-specific.
If you could write a format string is locale-specific, then that ties you to a single locale.
Types other than DateTime don't use format strings for parsing. (ouch)

So while format strings might be useful, they don't really solve any globalization related problems.
Its definitely possible to support format strings for more types, but we need to make sure users understand that the format string they specify is only used in formatting, and not in parsing. The parsing behaviour is specified by NumberStyles. This implies that we need to be really permissive in what we parse if we want users to specify a format for numbers.

Add support for missing types We could add support to bind for types that don't currently support like DateTimeOffset, DateTime? and a bunch of the rarely-used integral types. Look at the types supported by System.Convert
Since bind currently only supports enums and the types we hardcode we don't have support for types like ushort or byte. It seems obvious to me that there's a priority order of these things:

DateTimeOffset & DateTimeOffset? & DateTime?
everything else

I think an important question when you consider more exotic types like char or sbyte is - what is the expectation of someone who reaches for that type?. I don't think we know. It's easy to try and understand the motivation of someone using DateTime or int or decimal.
It gives me a little bit of pause when I think about this because I don't know what that user wants - do they want Hex? Octal? Why?
Paired with adding support for type-converters, we could end up in a scenario where not providing hand-coded bind support in this release and adding it in a future release results in a breaking change.

Add support for type converters We could easily add support for arbitrary types to fall back to type converters. Type convert is a well-known extensibility point that .NET provides, and other parts of ASP.NET Core use for conversions.
This seems like a clear win, because we get support for things like Guid and TimeSpan without writing much additional code. Users can write their own TypeConverter and plug it in using a non-Blazor mechanism and it will just work.
This is actually somewhat high priority because users hit this trying to use @bind with a generic type. Since we don't overload @bind for arbitrary types, this will result in a compile error and a bad experience.

Allow users to specify a custom converter` We could allow users to specify a converter at the call-site. This could be a desirable level of control for application developers when they are doing something complex.
Directive attributes make it easy for us to build something like this
<input type="text" @bind="coolnessLevel" @bind:converter="MyConverter.Default">
  Rate the coolness of Blazor (in a totally custom way)
</input>

However, this doesn't really solve any globalization problems. This is just added flexibility.
Flow the culture
We have to make sure that .NET code is running with the correct CultureInfo.CurrentCulture and CultureInfo.CurrentUICulture. This is required for localization (using .resx as well as doing any kind of culture-specific conversion).
Having the properties reflect the user's locale is essential for making Blazor feel like .NET because without this, a bunch of .NET infrastructure will work incorrectly. You normally accomplish this in .NET using the localization middleware and Accept-Language, but I'm not sure that applies to our scenario. Asterix 1
Concerns that need to be handled:

How does this work in client-side?? We want this to be free and provided by Mono
How does this work in prerendering?? (localization middleware)
How does this work with Circuits?? (I assume we have to build this)

This of course is considering that we can assume navigator.language reflects what the user wants to see. I think we need to consider providing an override or some other mechanism to customize this to be out of scope for 3.0 and react to feedback.
Appendix
Asterix 1
Users really like to get fancy with the localization middleware and do all kinds of things that aren't based on Accept-Language. Typically means using the query string or URL path. These mechanisms aren't inherently visible to JS in the same way that Accept-Language would be - since the browser generates Accept-Language.
I expect we're going to a substantial amount of feedback from users that want something fancy to work, but I don't see a way we can make that possible for either Blazor scenario. The trap here is that the localization middleware will be required for pre-rendering, but won't be used for non-prerendering - resulting in subtle bugs in users' applications. The solution would be for users to write their own logic to resolve the culture (in JS).

If possible I would like to add my 2 cents to the discussion about culture. In my opinion the best practice is (example for 2 cultures):

design your url scheme with culture at the beginning (this is required for step 2): example.com/en-us/customers, example.com/pl-pl/kontrahenci, etc.
create 3 main page urls (one more than number of supported cultures) example.com, example.com/en-us, example.com/pl-pl

How to decide which culture to use?

Culture in url has priority and we don't look any further. Each time user displays any page with culture in the url we set a cookie with preferred culture. This is necessary for step 2.

There is only one address without culture: example.com. When user visits this address (usually because he/she entered it manually in the browser) server will check our "preferred culture" cookie first and if it is set it will return HTTP 302 with new address based on the "preferred culture" cookie. If there is no cookie server will check Accept-Language and use this information for redirect. If our web page doesn't support any culture from Accept-Language header then it redirects to main page with default culture.

Why is this the best practice?
User experience
Let's test it on the most complicated example. We have a web page with support for en-us and pl-pl culture. Default culture is pl-pl because I live in Poland and our customers are mainly from Poland. User from Germany visits our main page. We don't support de-de culture - he is redirected to example.com/pl-pl. Unfortunately this user doesn't speak Polish but he can easily switch to English. Next day he visits our page again. He has entered example.com and he is automatically redirected do example.com/en-us - their preferred culture on our site.
SEO
Every page have to have distinct address. You cannot display different content (language) for the same url based on some "magic" like cookie. As far as I know search engines index only one content per url. Some guidelines from Google: https://support.google.com/webmasters/answer/189077?hl=en
Works everywhere (server, client, prerendering, api)
We have only one page without culture in the url - the main page, which essentially does not render anything because it is only used to redirect user to the page with culture. Every other page has culture in the url so it is easy to find this information on the server (including prerendering) and on the client (Blazor working in web assembly). Blazor can also call web api with culture in the url (example.com/en-us/products) With this information in the URL server can return product names and descriptions in proper language.
Of course culture can be encoded in query string instead of url, but in my humble opinion url is better.
Proof that it works
You can check this site: tajskiespa.pl to see how it works in practice (it was written in ASP.NET Core 1.1 and it supports pl and en language, not full culture.). You can also check how it is displayed in search results. In Google search for "tajskie spa wilanow branickiego". If you have English preference in the browser you should see link to https://tajskiespa.pl/en/our-salons/wilanow-branickiego (scroll down if you don't use add blocker and you see some Google adds at the top). Change your browser to prefer Polish language and you should see link to https://tajskiespa.pl/pl/nasze-salony/wilanow-branickiego.
I understand that good framework is probably not opinionated, but at the same time it should promote best practices and as you can see culture in the url (eventually query string) is the best solution which at the same time probably resolves all problems.

I expect we're going to a substantial amount of feedback from users that want something fancy to work, but I don't see a way we can make that possible for either Blazor scenario.

Implement solution which is in line with best practices and don't worry about people who want something fancy.
What is not acceptable
You should not use Accept-Language header for anything other than redirection from main page. Example: English speaking tourist visits Paris. In the hotel or internet cafe he uses French browser to look for some attractions. He has no idea what Accept-Language header is and how to change it in the browser. The only chance for him is to change the culture using the user interface available on the website.

thanks for the comments @Andrzej-W  - I'm going to look at putting this together as a sample and see what the blockers are.

Super rudimentary sample of the culture propagation part of this here: https://github.com/rynowak/ServerSideBlazorLocalization
This uses the localization middleware (as well as some other custom middleware) to set a cookie on the client. I also created a custom thing to treat the locale as part of the path+pathbase. I don't generally feel positively about using routing to track the culture - it's massively complex, and it feels wrong when you have the assumption that you're going to use a consistent localisation scheme for your app.
Due to the fact that I'm using a cookie for locale, it works for free in server-side Blazor, because I can just rely on the same localisation middleware. This also makes things easy in client-side Blazor because the cookie is available to the app trivially.
What I don't like about this is that we need to find a home for all of the custom middleware that this requires. We'd never put something like this in a sample directly, it's too nuanced.

I don't generally feel positively about using routing to track the culture

@rynowak Unfortunately we cannot ignore the fact that any web site have to be properly indexed by search engines. Otherwise this site simply "does not exists". Recently I have read the study and it looks that people who want to open google.com search for google! It is interesting that they prefer to hit Enter and then use mouse to click on the link in the search results than simply write additional .com.
As I have already written in my previous post each multilingual site should have different urls for the same page in different languages (cultures). It is necessary for proper indexing. Now when you have different urls for each culture search engine will send you directly to appropriate address. If you visit this site the first time (no cookie in the browser) there are two ways to select the culture: url or Accept-Language header. You have seen in my previous post that Accept-Language header is far from perfect for people who need to use not their own computers. There are a few possible url schemes (I will use customers and Polish translation kontrahenci).

example.com/customers
example.com/kontrahenci
example.com/en-us/customers
example.com/pl-pl/kontrahenci
example.com/en-us/customers
example.com/pl-pl/customers
en-us.example.com/customers
pl-pl.example.com/kontrahenci
en-us.example.com/customers
pl-pl.example.com/customers

I have never seen option 1, at least I don't remember. The most popular are options 2 and 3. We can say that urls in option 2 are fully localized and it is probably the best from the SEO point of view. Option 3 is probably the easiest to implement for programmers. We have to remember that each page have to have links to itself and all other languages/cultures (SEO requirements).
<link rel="alternate" hreflang="pl-pl" href="http://example.com/pl-pl/kontrahenci" />
<link rel="alternate" hreflang="en-us"  href="http://example.com/en-us/customers" />

We also want to have "language switcher" UI element on every page. When we switch the language we should stay on the same page. Option 3 is the easiest because we (programmers) only have to change culture in the url to create alternate addresses. Personally I prefer option 2. Options 4 and 5 are probably seldom used. Usually different subdomain means that web server is physically located in different data center (on different continent) to optimize access time for users around the globe. But in that case application is not really multilingual - each application instance serves pages for only one culture.
It can be complex but I think we have no other choice - urls should be tightly coupled with culture. If someone is not interested in true multilingual/multiculture application then it should be possible to set some default culture and use it on the server and on the client. Eventually single language web site can support multiple cultures and this for simplicity can be based on Accept-Language header (first request) and then you can use cookies. Here you don't need different urls because you always display the same content and culture is probably only used for date and numbers formatting.

@Andrzej-W - thanks for your input on this.
When I said that I didn't think routing was a good solution for this, I didn't mean that URLs aren't a good solution. Ultimately I think you're right and I'm wrong after reading your reply.  I'm torn on this because we don't currently offer a solution for translated URLs.
I did a little bit of sample building based on what you described. The part that's not going to easily be possible in this release is translating the URL path into various languages. The routing system in Blazor is pretty primitive right now.
As far as what we're shipping in the box, we're going to recommend that users use a cookie as the source of truth for server-side blazor. This works with the SignalR services, with self-hosted SignalR, and is already supported by ASP.NET Core.
To set this cookie, or decide what locale to give users by default, that's going to be up to you to figure out. We're not going to build in a specific localization scheme to server-side Blazor.


Autozoom on maps

I noticed that when using e.g. DB Navigator to show the calculated route, the route is perfectly shown using the maps from opensciencemap but it is not zoomed into the interesting region where the route is drawn.

Please report in GmsCore if the problem persists in latest nightly.


Issue with the created windows

Describe the bug
Issue with the created windows, possibly not responding
To Reproduce
Steps to reproduce the behavior:

Start Krita
See error pop-up

Screenshots
TypeError
Python 3.8.1: C:\Program Files\Krita (x64)\bin\krita.exe
Sat Dec 31 03:05:17 2022
A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.
C:\Users\Justin\AppData\Roaming\krita\pykrita\krita_diff\extension.py in setup(self=<krita_diff.extension.SDPluginExtension object>)
18         self.update_timer.start(REFRESH_INTERVAL)
19         script.config_updated.connect(lambda: self.update_global())
20         self.instance.notifier().windowCreated.connect(lambda: self.update_global())
21         script.action_update_config()
22
self = <krita_diff.extension.SDPluginExtension object>
self.instance = <PyKrita.krita.Krita object>
self.instance.notifier =
).windowCreated undefined
self.update_global = <bound method SDPluginExtension.update_global of <krita_diff.extension.SDPluginExtension object>>
TypeError: connect() failed between windowCreated(Window*) and unislot()
cause = None
class = <class 'TypeError'>
context = None
delattr = <method-wrapper 'delattr' of TypeError object>
dict = {}
dir =
doc = 'Inappropriate argument type.'
eq = <method-wrapper 'eq' of TypeError object>
format =
ge = <method-wrapper 'ge' of TypeError object>
getattribute = <method-wrapper 'getattribute' of TypeError object>
gt = <method-wrapper 'gt' of TypeError object>
hash = <method-wrapper 'hash' of TypeError object>
init = <method-wrapper 'init' of TypeError object>
init_subclass =
le = <method-wrapper 'le' of TypeError object>
lt = <method-wrapper 'lt' of TypeError object>
ne = <method-wrapper 'ne' of TypeError object>
new =
reduce =
reduce_ex =
repr = <method-wrapper 'repr' of TypeError object>
setattr = <method-wrapper 'setattr' of TypeError object>
setstate =
sizeof =
str = <method-wrapper 'str' of TypeError object>
subclasshook =
suppress_context = False
traceback =
args = ('connect() failed between windowCreated(Window*) and unislot()',)
with_traceback =
The above is a description of an error in a Python program.  Here is
the original traceback:
Traceback (most recent call last):
File "C:\Users\Justin\AppData\Roaming\krita\pykrita\krita_diff\extension.py", line 20, in setup
self.instance.notifier().windowCreated.connect(lambda: self.update_global())
TypeError: connect() failed between windowCreated(Window*) and unislot()
Desktop (please complete the following information):

OS: Windows
Possibly related to commit a41f84fbdb1973a2f17b2956ff78caf39995ca51

What version of Krita are you using?

What version of Krita are you using?

Startup says 4.4.8

Weird, I don't use the official API to change config options at all, so I don't know where the validation error is coming from. Could it be from another extension instead of mine?
Other than that, try updating the webUI as well. WebUI just had some breaking changes 10 hours ago I just fixed.
Anyways, since it seems to work, I will close the issue.


Unseal Authorize Attribute

Problem:
Asp.net Cores Authorize attribute is not sealed. This allows overriding for custom behavior, such as having policys be an enum, then resolve it and set in the constructor. Hot chocolates is Sealed, not allowing for customization.
Solution:
Unseal Authorize attribute, to mimic Microsofts API
Alternatives:
Simply hard coding the strings, or making a const file that everything can call from, however that would only slightly solve this one scenario.

The same thing goes for StateAttribute.

@Dispersia do you want to do a pr?

This is now merged.


Create Chinese Strings

What changes does this PR bring?
Enter a summary of the changes introduced in this PR
Issue Reference
Enter issue number and link or state "None"
Screenshots
If applicable
Additional details
Enter other relevant information (of any)
Check all of the following

[ ] This PR does not conflict with target branch
[ ] I have appropriately labelled this PR
[ ] I have added appropriate issue reference

HI @sr093906, thanks for adding this! Will be in next release.


Python 3.9 support

Thank you for developing this plugin! Would it be possible to add support for Python 3.9? As far as I can tell, the only 3.10 feature you are using is the str | None syntax. If that is the case, Python 3.9 support could be added simply by including from __future__ import annotations at the top of admonition.py.
The reason I'm asking is that in my current project we aim to support all the currently maintained Python versions, which includes 3.9 for another year.

Sure! I'll add it this week.

New release available on the repository, probably on PyPI tomorrow.

Great, thank you for solving this so quickly!

The release is now on PyPI. I would appreciate it if you let me know if everything seems to work as expected. Cheers!

I’ve tested it with Python 3.9, and it seems to work perfectly. Thanks again for the quick fix!


Figure out user authentication

@a-ilango you are working on this, right? Would be good to assign this issue to yourself if yes. Maybe next meeting we can discuss what the authentication API will look like on the frontend.

Yes, I was working on this. Let me assign it to myself.

Maybe next meeting we can discuss what the authentication API will look like on the frontend

Sounds good. It is captured in the design document. We can add missing details if there are any.

@a-ilango ah, great! I forgot about the design document, thank you. I'll take a look again.

@a-ilango Hey Arun! Are you still working on this? Just wanted to check in.

@maxkwallace the /auth_token endpoint code is about finished with python tests passing, though I've just discovered that it's broken on the wsgi server (returns 400) and am looking into this.

@maxkwallace Sorry, I'm not working on this. It's been busy for a while now but I'm hoping to contribute to this project again when I get some time in the future.
Thanks @wittejm for taking this up!

@wittejm Can we close  this?

@erikjasso The only feature listed here that is missing from the app is to use persistent RSA keys in the prod environment, whereas the dev configuration works as intended (generating a random rsa key on app start).
I suppose we could write a more specific issue and toss it into the project backlog, to unclutter the project board a bit. The feature is blocked on having the app use any prod configuration at all, since the wsgi.py launcher is hard-coded to use only the dev configuration

The only remaining feature here has been moved to #581


Hard to understand k8s secret name in Helm chart for TLS certificates

Hi folks,
In values.yaml file[1] also in README.md[2], I see secret name as vernemq-certificates-secret for TLS certificates, however in StatefulSet template[3] it is defined as secretName: {{ include "vernemq.fullname" . }}-certs which will be vernemq-certs if default full name is not overwritten.
Is there any mistake or am I missing something? Thank you in advance ❤
[1] https://github.com/vernemq/docker-vernemq/blob/7eb6ab3fe8d11bd334e3b65f953105a0f597bc44/helm/vernemq/values.yaml#L173-L178
[2] https://github.com/vernemq/docker-vernemq/blob/7eb6ab3fe8d11bd334e3b65f953105a0f597bc44/helm/vernemq/README.md?plain=1#L156
[3] https://github.com/vernemq/docker-vernemq/blob/7eb6ab3fe8d11bd334e3b65f953105a0f597bc44/helm/vernemq/templates/statefulset.yaml#L196

Must have been introduced by this commit: https://github.com/vernemq/docker-vernemq/commit/ddc6ee6c8f8432620941bfb27d8c82d278c1f2fa
Do you see an issue with it? What would be your proposed solution? (as I understand, it's an inconsistency in docs and yaml files where outcommented values might be misleading)

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

To be compatible, should not the secretName: vernemq-certificates-secret  name be vernemq-certs? That's why I asked if I am missing something. Are they different secrets?

In my view vernemq-certificates-secret is just an example name. The PR mentioned above introduced an additional way to configure TLS certs over helm values. You'll have to choose how to setup the secrets. (And yes, it's always about the TLS secrets (server certs & key) here).

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

Thanks a lot @ioolkos , that's what I'm trying to be sure 👍


Add closedAt and createdAtBlockNumber to Allocation

@Jannis requested to have these two fields added:
allocation.closedAt
allocation.createdAtBlockNumber

Both are now available, added them in #124 and #125


⚠ Web Busy Light Demo has degraded performance

In 3455631, Web Busy Light Demo (https://workfromhome.capricehosting.com/) experienced degraded performance:

HTTP code: 200
Response time: 6631 ms

Resolved: Web Busy Light Demo performance has improved in 320e7d5 after 16 minutes.


Relative timestamps

An example, how it looks in ls++.

how are the current timestamps not readable?

accidentally closed sorry.

Renamed. )

I don't think this is a needed change because you can sort by time right now. Sorting by time puts the last modified files in order going up. Because there is a option to sort by time I do not think this is a needed change.

It is just a convenience. Most timestamps on the web are relative nowadays.

Gmail also cuts down the amount of information printed. Maybe not relative, but more readable for humans, who usually don't need a second accurancy on daily+ intervals.

implementing this will probably require -f --full option to get back exact data.

I think I will have the exact data on by default and maybe a flag to enable relative time. I could probably implement it with https://docs.rs/chrono/0.4.0/chrono/struct.DateTime.html#method.signed_duration_since.

It still doesn't convert timestamp to human verbs. I am not sure what is available for Rust. For JavaScript there are plenty of alternatives https://www.npmjs.com/search?q=human readable time


HasBorder and BorderStyle (?) should be on Control level (important for ListView,TreeView,Edit,ListBox)

HasBorder now only on Window level but in any application developer needs to have controls with and without standard border

I am closing this issue as HasBorder was added to all controls we need (ListBox, TextBox, ListView, TreeView, CheckLIstBox)

We need HasBorder in

ComboBox
DateTimePicker
Numeric Input
WebBrowser

wxComboBox and wxChoice don't support HasBorder feature

wxDatePicker and wxTimePicker don't support wxBORDER_NONE feature

wxNumericUpDown supports wxBORDER_NONE but badly (buttons are repainted incorrectly)

Added WebBrowser.HasBorder. I am closing this issue for now.


Flax NNX checkpointing is not that straightforward

I have been trying to use Orbax for checkpointing Flax NNX models and getting checkpointing to work for models with Dropout layers which also hold JAX RNG keys is not very straight forward. After various attempts this was the only way I could get it to work.
import flax.nnx as nnx
import jax.numpy as jnp
import orbax.checkpoint as ocp

class Test(nnx.Module):
    def __init__(self, dim, rngs: nnx.Rngs):
        self.layer1 = nnx.Linear(dim, dim, rngs=rngs)
        self.layer2 = nnx.Dropout(0.1, rngs=rngs)
        self.layer3 = nnx.Linear(dim, dim, rngs=rngs)

    def __call__(self, x):
        x = self.layer3(self.layer2(self.layer1(x)))
        return x

checkpoint_manager = ocp.CheckpointManager(
    ocp.test_utils.erase_and_create_empty(
        "/var/local/ML/TRAIN/STAGE/troubleshoot_checkpointing/"
    ),
    options=ocp.CheckpointManagerOptions(
        max_to_keep=2,
        keep_checkpoints_without_metrics=False,
        create=True,
    ),
    item_names=("state", "layer2_dropout_key"),
)

model = Test(10, nnx.Rngs(0))
model_state = nnx.state(model).flat_state()
layer2_dropout_key = model_state[("layer2", "rngs", "default", "key")].value

# The Dropout layers RNG key had to be replaced with a dummy to
# allow checkpoint saving
# Error seen
# TypeError: Cannot interpret 'key<fry>' as a data type
model_state[("layer2", "rngs", "default", "key")] = nnx.VariableState(
    type=nnx.Param, value=jnp.array([])
)

# The RNG key had to be saved with its special checkpointer
checkpoint_manager.save(
    0,
    args=ocp.args.Composite(
        state=ocp.args.StandardSave(nnx.State.from_flat_path(model_state)),
        layer2_dropout_key=ocp.args.JaxRandomKeySave(layer2_dropout_key),
    ),
)
checkpoint_manager.wait_until_finished()

abs_model = nnx.eval_shape(lambda: nnx.State.from_flat_path(model_state))

# Checkpoint restoration also does not work
# The two items have to be restored separately
restored = checkpoint_manager.restore(
    0,
    args=ocp.args.Composite(
        state=ocp.args.StandardRestore(abs_model),
    ),
)

restored_key = checkpoint_manager.restore(
    0,
    args=ocp.args.Composite(
        # state=ocp.args.StandardRestore(abs_model),
        layer2_dropout_key=ocp.args.JaxRandomKeyRestore(),
    ),
)

# Model restoration is equally not straightforward
restored_model_state = restored["state"].flat_state()
restored_model_state[("layer2", "rngs", "default", "key")] = nnx.VariableState(
    type=nnx.Param, value=restored_key["layer2_dropout_key"]
)

abs_graph_def, abs_state = nnx.split(Test(10, nnx.Rngs(0)))
restored_model = nnx.merge(
    abs_graph_def, nnx.State.from_flat_path(restored_model_state)
)

print(nnx.state(restored_model).flat_state().keys())

The comments describe the issues faced.
It would be good to address ease of use for cases where the model has Dropout layers.

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Hi @prabhudavidsheryl! I also ran into this issue. It seems that jax.random.key is now preferred over jax.random.PRNGKey (see https://jax.readthedocs.io/en/latest/jep/9263-typed-keys.html), but it comes with this key<fry> data type which Orbax doesn't seem to be able to reconcile simultaneously with the state of the nnx.Module yet.
To get around this I made a simple RNG using the old jax.random.PRNGKey with nnx.Object
class RNG(nnx.Object):

    def __init__(self, rng_int: int):
        self.key = nnx.Variable(jax.random.PRNGKey(rng_int))
        self.count = nnx.Variable(0)

    def __call__(self):
        self.count += 1
        _, key = jax.random.split(self.key.value, 2)
        self.key.value = key
        return key

You can then make a model like this, for example:
class Model(nnx.Module):

    def __init__(self, rng_int: int):
        self.rng = RNG(rng_int)

    def __call__(self, x):
        xr = jax.random.normal(self.rng(), x.shape)
        return jnp.dot(x, xr)

We can then just save the model as usual
kwargs = dict(rng_int=0)
model = Model(**kwargs)

path = ocp.test_utils.erase_and_create_empty('/path/to/my/awesome/model')
options = ocp.CheckpointManagerOptions(max_to_keep=3, save_interval_steps=2)
manager = ocp.CheckpointManager(path, options=options, item_names=('kwargs', 'state'))

# Dummy training loop
for i in range(11):
    graphdef, state = nnx.split(model)
    save_args = ocp.args.Composite(
        kwargs=ocp.args.JsonSave(kwargs),
        state=ocp.args.StandardSave(state)
    )
    manager.save(i, args=save_args)
manager.wait_until_finished()

# Restore model
restore_args = ocp.args.Composite(
    state=ocp.args.StandardRestore(nnx.state(model)),
    kwargs=ocp.args.JsonRestore(kwargs)
)
restored = manager.restore(manager.latest_step(), args=restore_args)

restored_model = Model(**restored.kwargs)
print(restored_model.rng.key) # [0, 0]
nnx.update(restored_model, restored.state)
print(restored_model.rng.key) # different

Works for me! But, @cgarciae, do you think this is a good work-around for the moment, or is there something I'm missing?

I am having a similar issue with Orbax-checkpoint in flax.linen. I followed the tutorial in flax.linen for Dropout. The TrainState in that case included another attribute key, which is the key for dropout. When saving the TrainState with orbax_checkpoint, I got an error of TypeError: Cannot interpret 'key<fry>' as a data type.
Hence, it is not about flax.nnx, but also flax.linen as well.

I encouter the same problem on a dropout layer in flax nnx nnx.Dropout(rate=0.3, rngs=rngs)
It would really be nice to fix it, to facilitate greatly model savings and allow more users to really on jax and flax

I meet the same issue too. Based on your code, I come out a slightly different approach which include a ADAM optimizer.
import flax.nnx as nnx
import jax
import jax.numpy as jnp
import numpy as np
import optax
import orbax.checkpoint as ocp

ckpt_dir = '/tmp/checkpoints'

class Test(nnx.Module):
    def __init__(self, din: int, dmid: int, dout: int, *, dropout: float, rngs: nnx.Rngs):
        self.linear1 = nnx.Linear(din, dmid, rngs=rngs)
        self.dropout = nnx.Dropout(rate=dropout, rngs=rngs)
        self.linear2 = nnx.Linear(dmid, dout, rngs=rngs)

    def __call__(self, x: jax.Array):
        return self.linear2(nnx.gelu(self.dropout(self.linear1(x))))

def loss_fn(model, x, y):
    y_pred = model(x)
    return ((y_pred - y) ** 2).mean(), y_pred

sample = np.random.random(10)
target = np.random.random(2)

model = Test(10, 10, 2, dropout=0.5, rngs=nnx.Rngs(jax.random.key(1)))
optimizer = nnx.Optimizer(model, optax.adam(learning_rate=0.001))

grad_fn = nnx.value_and_grad(loss_fn, has_aux=True)
_, grads = grad_fn(model, sample, target)
optimizer.update(grads)

with ocp.CheckpointManager(
        ckpt_dir,
        item_names=('state', 'opt_state', 'dropout_key',),
) as ckpt_mgr:
    print('save checkpoint')

    _, state = nnx.split(model)
    # The Dropout layers RNG key had to be replaced with a dummy to allow checkpoint saving
    dropout_key = state['dropout']['rngs']['default']['key']
    state['dropout']['rngs']['default']['key'] = nnx.VariableState(
        type=nnx.Param,
        value=jnp.zeros(1),
        **dropout_key.get_metadata()
    )
    _, opt_state = nnx.split(optimizer.opt_state)

    ckpt_mgr.save(
        1,
        args=ocp.args.Composite(
            state=ocp.args.StandardSave(state),
            opt_state=ocp.args.StandardSave(opt_state),
            dropout_key=ocp.args.JaxRandomKeySave(dropout_key.value),
        ),
    )
    ckpt_mgr.wait_until_finished()

    print('restore from checkpoint')

    model = Test(10, 10, 2, dropout=0.5, rngs=nnx.Rngs(0))
    graph_def, state = nnx.split(model)
    # replace Dropout layers RNG key by a dummy to allow checkpoint restoration
    state['dropout']['rngs']['default']['key'] = nnx.VariableState(
        type=nnx.Param,
        value=jnp.zeros(1),
        **state['dropout']['rngs']['default']['key'].get_metadata()
    )

    opt_def, opt_state = nnx.split(optimizer.opt_state)

    restored = ckpt_mgr.restore(
        1,
        args=ocp.args.Composite(
            state=ocp.args.StandardRestore(nnx.eval_shape(lambda: state)),
            opt_state=ocp.args.StandardRestore(nnx.eval_shape(lambda: opt_state)),
            dropout_key=ocp.args.JaxRandomKeyRestore(),
        ),
    )
    # restore Dropout layers RNG key
    restored['state']['dropout']['rngs']['default']['key'] = nnx.VariableState(
        type=nnx.rnglib.RngKey,
        value=restored['dropout_key'],
        **restored['state']['dropout']['rngs']['default']['key'].get_metadata()
    )

    model = nnx.merge(graph_def, restored['state'])
    optimizer.opt_state = nnx.merge(opt_def, restored['opt_state'])

I think orbax should support key<fry> Arrays but in the mean time you can manually convert the data back and forth to regular int32 arrays using jax.random.key_data and jax.random.wrap_key_data:

class Model(nnx.Module):
  def __init__(self, din, dmid, dout, rngs: nnx.Rngs):
    self.linear = nnx.Linear(din, dmid, rngs=rngs)
    self.bn = nnx.BatchNorm(dmid, rngs=rngs)
    self.dropout = nnx.Dropout(0.2, rngs=rngs)
    self.linear_out = nnx.Linear(dmid, dout, rngs=rngs)

  def __call__(self, x):
    x = nnx.relu(self.dropout(self.bn(self.linear(x))))
    return self.linear_out(x)

# saving
model = Model(2, 64, 3, rngs=nnx.Rngs(0))  # eager initialization

keys, state = nnx.state(model, nnx.RngKey, ...)
keys = jax.tree.map(jax.random.key_data, keys)
...  # save keys and state checkpoints

# loading
...  # load keys and state checkpoints
keys = jax.tree_map(jax.random.wrap_key_data, keys)
nnx.update(model, keys, state)

Looks code became slightly simple. But need a new function to merge the rngs key back to state.
import flax.nnx as nnx
import jax
import numpy as np
import optax
import orbax.checkpoint as ocp

ckpt_dir = '/tmp/checkpoints'

def merge_state(dst: nnx.State, src: nnx.State):
    for k, v in src.items():
        if isinstance(v, nnx.State):
            merge_state(dst[k], v)
        else:
            dst[k] = v

class Test(nnx.Module):
    def __init__(self, din: int, dmid: int, dout: int, *, dropout: float, rngs: nnx.Rngs):
        self.linear1 = nnx.Linear(din, dmid, rngs=rngs)
        self.dropout = nnx.Dropout(rate=dropout, rngs=rngs)
        self.linear2 = nnx.Linear(dmid, dout, rngs=rngs)

    def __call__(self, x: jax.Array):
        return self.linear2(nnx.gelu(self.dropout(self.linear1(x))))

def loss_fn(model, x, y):
    y_pred = model(x)
    return ((y_pred - y) ** 2).mean(), y_pred

sample = np.random.random(10)
target = np.random.random(2)

model = Test(10, 10, 2, dropout=0.5, rngs=nnx.Rngs(jax.random.key(1)))
optimizer = nnx.Optimizer(model, optax.adam(learning_rate=0.001))

grad_fn = nnx.value_and_grad(loss_fn, has_aux=True)
_, grads = grad_fn(model, sample, target)
optimizer.update(grads)

with ocp.CheckpointManager(
        ckpt_dir,
        item_names=('state', 'opt_state'),
) as ckpt_mgr:
    print('save checkpoint')

    _, state = nnx.split(model)
    # The RNG key had to be convert to int to allow checkpoint saving
    rngs_key = jax.tree.map(jax.random.key_data, state.filter(nnx.RngKey))
    merge_state(state, rngs_key)

    _, opt_state = nnx.split(optimizer.opt_state)

    ckpt_mgr.save(
        1,
        args=ocp.args.Composite(
            state=ocp.args.StandardSave(state),
            opt_state=ocp.args.StandardSave(opt_state),
        ),
    )
    ckpt_mgr.wait_until_finished()

    print('restore from checkpoint')

    model = Test(10, 10, 2, dropout=0.5, rngs=nnx.Rngs(0))
    graph_def, state = nnx.split(model)
    # replace RNG key by a dummy to allow checkpoint restoration
    rngs_key = jax.tree.map(jax.random.key_data, state.filter(nnx.RngKey))
    merge_state(state, rngs_key)

    opt_def, opt_state = nnx.split(optimizer.opt_state)

    restored = ckpt_mgr.restore(
        1,
        args=ocp.args.Composite(
            state=ocp.args.StandardRestore(nnx.eval_shape(lambda: state)),
            opt_state=ocp.args.StandardRestore(nnx.eval_shape(lambda: opt_state)),
        ),
    )
    # restore RNG key
    rngs_key = jax.tree.map(jax.random.wrap_key_data, restored['state'].filter(nnx.RngKey))
    merge_state(restored['state'], rngs_key)

    model = nnx.merge(graph_def, restored['state'])
    optimizer.opt_state = nnx.merge(opt_def, restored['opt_state'])

Orbax team is looking into it and planing to add support for RNG in early Jan.  Thanks!

Thanks, for working on this. Looking forward to being able to easily save models with dropouts.

Waiting for better support as well :)
Currently when trying to save/restore a simple module like:
import orbax.checkpoint as ocp
checkpoint = ocp.PyTreeCheckpointer()
checkpoint.save("models/renderer-nnx", nnx.state(renderer))

I get:
TypeError: JAX array with PRNGKey dtype cannot be converted to a NumPy array. Use jax.random.key_data(arr) if you wish to extract the underlying integer array.
I think this is related to this issue? Happy to see an update soon, thanks!


notes get added to variables that don't have them

Attribute notes should only added if they exists for a specific variables, it seems like the tool is attaching the note for pressure level interpolation to other variables?? see tas example

This was due to not making a proper copy of ctx.obj['attrs'] in the function so notes where getting added to the obj and used by next variable! Now fixed in main!


Functions datatable

making PR here so that I can build a preview of the docs

probably won't be using this, closing for now.


add sync flow to workload identity

Adds GetTokenSync impl to Workload Identity credential

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-identity

/check-enforcer evaluate


Cookie options attribute from init call mostly not respected

While attempting to modify cookie options, I noticed that despite setting the corresponding configuration options, the cookies are still generated with same original attributes.
Expected Behavior
I expected the sameSite: 'None' and secure: true attributes to be applied to newly created cookies
Current Behavior
Original attributes sameSite: 'Lax' and secure: false are still set.
Possible Solution
It looks like the object spreading on the following line might be done in an incorrect location, keeping most properties fixed and not overridable:
https://github.com/amplitude/Amplitude-TypeScript/blob/main/packages/analytics-browser/src/config.ts#L189
Steps to Reproduce

add the following code to be executed in the browser on page load

amplitude.init(<apikey>, {
        // ...
        cookieOptions: {
          sameSite: 'None',
          secure: true,
        }
})

make sure no AMP_ cookies are set in the browser
load the application
check the attributes of the auto-created cookies - observe that sameSite is 'Lax' and secure is false

Environment

JS SDK Version: 2.2.0
Installation Method: yarn
Browser and Version: Chrome 114.0.5735.106

Is this still an issue?  I'm initializing the browser sdk like this:
amplitude.init('f01c77366af58e34a071913e9f75ab7f', {
        logLevel: 4,
        cookieOptions: {
          sameSite: 'None',
          secure: true,
        }
      }

And the cookie still has sameSite: 'Lax' (after deleting cookies and trying again)


hotfix branch also needs an option -k to keep branch after performing finish

I'm submitting a ...

[ ] bug report
[X] feature request
[ ] puppy => You're not submitting a puppy. I already have one and he's adorable

What is the current behavior?
There is an option "Keep branch after performing finish [-k]" for features and releases branches, but there is no option for hotfixes. By consequence pull requests for hotfixes are closed instead of merged on github.

What is the expected behavior?
Create the an option -k for hotfixes' branches and

What is the motivation / use case for changing the behavior?
Every person who uses this plugin will need to swap to console to finish hotfixes because it doesn't have an option to keep the branch after performing branch finish.

Please tell me about your environment:

Gitflow4idea version: <IP_ADDRESS>
Gitflow version: 1.11.0 (AVH Edition)

PhpStorm 2018.1
Build #PS-181.4203.565, built on March 28, 2018
JRE: 1.8.0_152-release-1024-b15 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Linux 4.14.34-1-MANJARO

Luckily this is an easy feature to implement. I love those 😀
It'll be included in the next version

Will be available in 0.7.0

Version 0.6.9 is now on the Jetbrains Repo


Intelligence box issue

When intelligence box (Auto Suggest) appears, labels are not seen, as background color and label colors both are black

@badgujar-nagesh Can you provide me with a screenshot of your issue? Thanks! 😄


Logs out actual error number, support for non-standard baud rates

the result is always -1. The actual error is stored in errno.

Thanks for the pull request! Adding support for non standard baud rates has been on my agenda for a while, so this is really nice. I'll plan to test and (hopefully) merge this tomorrow.

I tested this this morning. Unfortunately, I found that it doesn't work with either Keyspan or Prolific adapter chipsets/drivers. Instead, for those devices, the baud rate seems to get set to the next closest standard baud rate. I'm curious to know what kind of USB to serial adapter you're using?

I’m using a PL2303HX rev.D and I tested at 31250 baud rate which worked fine on Yosemite 10.10.2…

On 23 Mar 2015, at 16:25, Andrew Madsen<EMAIL_ADDRESS>wrote:
I tested this this morning. Unfortunately, I found that it doesn't work with either Keyspan or Prolific adapter chipsets/drivers. Instead, for those devices, the baud rate seems to get set to the next closest standard baud rate. I'm curious to know what kind of USB to serial adapter you're using?
—
Reply to this email directly or view it on GitHub https://github.com/armadsen/ORSSerialPort/pull/45#issuecomment-85076096.

I was able to get this to work at 31250 baud using an FTDI adapter on 10.10.2. On the same adapter, other arbitrary baud rates (e.g. 23456) didn't work. According to this, "the driver for the underlying serial hardware ultimately determines which baud rates can be used", which is what I'd expect.
Assuming it does no harm to use IOSSIOSPEED instead of the termios, I suppose that the unreliability/driver dependence of the ability to use arbitrary baud rates can just be well documented. I want to keep exploring this a little bit before merging, though.

Ok, I didn’t try a completely random baud rate but originally 31250 didn’t work and using IOSSIOSPEED did, so I assumed that would work for everything.

On 23 Mar 2015, at 16:53, Andrew Madsen<EMAIL_ADDRESS>wrote:
I was able to get this to work at 31250 baud using an FTDI adapter on 10.10.2. On the same adapter, other arbitrary baud rates (e.g. 23456) didn't work. According to this https://discussions.apple.com/message/7514136#7514136, "the driver for the underlying serial hardware ultimately determines which baud rates can be used", which is what I'd expect.
Assuming it does no harm to use IOSSIOSPEED instead of the termios, I suppose that the unreliability/driver dependence of the ability to use arbitrary baud rates can just be well documented. I want to keep exploring this a little bit before merging, though.
—
Reply to this email directly or view it on GitHub https://github.com/armadsen/ORSSerialPort/pull/45#issuecomment-85093560.

I'm going to close this in favor of #49 which implements the same fundamental feature. Thanks again for your contribution!


Doble items

Idk why, but the plugin duplicates items when the users join in, its no error in the console

I'm going to assume you mean it does two backups instead of one when a player joins instead of it duplicating items for the player.
It doesn't do this by default so it must be a conflicting plugin. Inventory Rollback listens to the PlayerJoinEvent and logs whenever that is fired. There must be another plugin you have running that for whatever reasons makes the event fire twice causing duplicate backups.
Are you able to provide any more information about your setup or if you have any ideas yourself what could be causing it?

Im so sorry, it was another plugin, not yours, thank you for answer me


feat: work_group_enforced setting to reduce AWS throttling

Description
The adapter performs a GetWorkGroup operation on each thread, which is later cached. When dbt build is started for 10 independent models, it issues 10 AWS requests to get the same information. If dbt is orchestrated via Airflow to run 32 dbt tasks concurrently, it sends 320 GetWorkGroup requests, causing throttling on the AWS side.
This PR introduces a work_group_enforced setting, which instructs dbt to skip checking if a WorkGroup contains an enforced output location. In this mode, dbt will use the provided boolean value. If a value is not provided, the default previous behavior is in place.

[x] You added unit testing when necessary
[x] You added functional testing when necessary

@amacal implementation looks good.
Please have a look at the ci, precommit checks are failing.
Provide unit testing for is_work_group_output_location_enforced function cover the introduced feature.
Nice to have, and not mandatory, add some "integration tests" to fully test the feature in a real environment.

I added a unit test and documented new parameter in the documentation. I cannot do functional test, because I cannot push it via org AWS account.

@amacal implementation looks fine, have a look at the CI, there is a failure

Is there anything I need to do to have it merged?

@amacal you changes looks good. As you can see repository ownership is being moved to dbtlabs, they are the official owners.
@colin-rogers-dbt could you have a look here?


RTD Documentation

Started writing some installation/setup instructions on RTD

Won't be very easy to autodoc everything, since autodoc imports each submodule and that requires that any external python libraries must be imported with requirements.txt
This is fine, except for mariadb, which cannot be pip installed without the MariaDB Connector/C tool.


Better error when launched without AVA cli

Perhaps throw a more explicit error because at the moment it appears to crash and complain about lack of process.send:
> node test/index.js
Warning: a promise was rejected with a non-error: [object Undefined]
    at /Users/timoxley/Projects/tests/node_modules/ava/lib/runner.js:163:20
From previous event:
    at Runner.run (/Users/timoxley/Projects/tests/node_modules/ava/lib/runner.js:161:4)
    at Immediate._onImmediate (/Users/timoxley/Projects/tests/node_modules/ava/index.js:53:9)
    at processImmediate [as _immediateCallback] (timers.js:374:17)

Unhandled rejection TypeError: process.send is not a function
    at exit (/Users/timoxley/Projects/tests/node_modules/ava/index.js:42:10)
From previous event:
    at Immediate._onImmediate (/Users/timoxley/Projects/tests/node_modules/ava/index.js:53:15)
    at processImmediate [as _immediateCallback] (timers.js:374:17)

Agreed. I've already seen this, but forgot to open an issue.


[It should have a non-async HasKey function]

It's not async already. I think the type definitions are incorrect.

very nice library bro. Great work... :)
Here return type is Promise
storage.indexer.hasKey(key)

@ammarahm-ed What about indexer.getKeys() and indexer.[x].getAll()? Are they also synchronous already or would it be possible / make sense to make them synchronous?

@ammarahm-ed What about indexer.getKeys() and indexer.[x].getAll()? Are they also synchronous already or would it be possible / make sense to make them synchronous?

They are async for now.


Ability to use Psuggest in other servers.

Is your feature request related to a problem? Please describe.
Setting up a bot just for a suggestion feature can be quite a tedious and annoying task, and the fact that the Psuggest feature is only available on the TheoTown Discord server just makes this feature a bit of a missed opportunity (if I'm even using the correct term, haha).
Describe the solution you'd like
I'd love to have the Psuggest feature available on other servers. You could set up the feature via Pconfiguration, if you want the feature on/off and in which channel should the suggestions go.
Describe alternatives you've considered
Right now, the only alternative solutions I could come up with is using a different bot or just handling the suggestions manually.
Additional context
No additional context.

The suggest command is currently specifically implemented for TheoTown server because it directly interacts with TheoTown and stores the suggestions to a database.
Although, I could change it for other guilds to provide a similiar functionality.

In work by #28

This is currently being worked on in the master branch.


ithc - Update YAML definitions

Automated updates from daily job running on ithc cluster.

Plan Result (ptlsbox-dt-availability-dashboards)
No changes. Your infrastructure matches the configuration.

Plan Result (ithc-dt-availability-dashboards)
Plan: 2 to add, 0 to change, 0 to destroy.

Create

dynatrace_http_monitor.availability["cmc-citizen-frontend-nodejs"]
dynatrace_http_monitor.availability["cmc-claim-store-java"]

Change Result (Click me)

  # dynatrace_http_monitor.availability["cmc-citizen-frontend-nodejs"] will be created
  + resource "dynatrace_http_monitor" "availability" {
      + enabled   = true
      + frequency = 1
      + id        = (known after apply)
      + locations = [
          + "SYNTHETIC_LOCATION-CC3E4D2657A13D18",
        ]
      + name      = "CFT-MONEY-CLAIMS-ITHC-cmc-citizen-frontend-nodejs"

      + anomaly_detection {
          + loading_time_thresholds {
              + enabled = true
            }

          + outage_handling {
              + global_outage = true
              + local_outage  = false

              + global_outage_policy {
                  + consecutive_runs = 1
                }
            }
        }

      + script {
          + request {
              + description = "moneyclaims.ithc.platform.hmcts.net"
              + method      = "GET"
              + url         = "http://moneyclaims.ithc.platform.hmcts.net/health"

              + configuration {
                  + accept_any_certificate = true
                  + follow_redirects       = true
                }

              + validation {
                  + rule {
                      + pass_if_found = false
                      + type          = "httpStatusesList"
                      + value         = ">400"
                    }
                }
            }
        }

      + tags {
          + tag {
              + context = "CONTEXTLESS"
              + key     = "SVCOFF1"
              + source  = "RULE_BASED"
              + value   = "SVCOFF0001322"
            }
        }
    }

  # dynatrace_http_monitor.availability["cmc-claim-store-java"] will be created
  + resource "dynatrace_http_monitor" "availability" {
      + enabled   = true
      + frequency = 1
      + id        = (known after apply)
      + locations = [
          + "SYNTHETIC_LOCATION-CC3E4D2657A13D18",
        ]
      + name      = "CFT-MONEY-CLAIMS-ITHC-cmc-claim-store-java"

      + anomaly_detection {
          + loading_time_thresholds {
              + enabled = true
            }

          + outage_handling {
              + global_outage = true
              + local_outage  = false

              + global_outage_policy {
                  + consecutive_runs = 1
                }
            }
        }

      + script {
          + request {
              + description = "cmc-claim-store-ithc.service.core-compute-ithc.internal"
              + method      = "GET"
              + url         = "http://cmc-claim-store-ithc.service.core-compute-ithc.internal/health"

              + configuration {
                  + accept_any_certificate = true
                  + follow_redirects       = true
                }

              + validation {
                  + rule {
                      + pass_if_found = false
                      + type          = "httpStatusesList"
                      + value         = ">400"
                    }
                }
            }
        }

      + tags {
          + tag {
              + context = "CONTEXTLESS"
              + key     = "SVCOFF1"
              + source  = "RULE_BASED"
              + value   = "SVCOFF0001322"
            }
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Plan Result (sbox-dt-availability-dashboards)
No changes. Your infrastructure matches the configuration.

Plan Result (stg-dt-availability-dashboards)
No changes. Your infrastructure matches the configuration.

Plan Result (demo-dt-availability-dashboards)
No changes. Your infrastructure matches the configuration.

Plan Result (test-dt-availability-dashboards)
No changes. Your infrastructure matches the configuration.


Zombieland (Tallahassee next) (draft)

Tallahassee is great! It's powerful enough to test your applications entire client side. It's "hackable" enough to emulate most scenarios / quirks occurring in an actual browser, making it suitable for both feature and unit style testing. It's lightweight enough to not be an excuse for skimping on testing.
However it isn't perfect...
Proposed solution is to delegate all the DOM stuff to jsdom. Other key features are built as a toolkit of independent single purpose extensions - Zombieland.

This issue used to be longer! WTH?

Moved over documentation into README in the draft branch.


Version issue when building from source archive (version 1.2.3)

Seeing as we're having issues with newer lua stability, I have decided to start by packaging version 1.2.3 for Gentoo.
There are a few issues with the build, at this stage 2 Errors (1 for Python and 1 for R), looking at the CMake files, it seems however, that the R issue is related to the missing version warning seen at the start:
>>> Unpacking simpleitk-1.2.3.tar.gz to /var/tmp/portage/sci-libs/simpleitk-1.2.3/work
>>> Source unpacked in /var/tmp/portage/sci-libs/simpleitk-1.2.3/work
>>> Preparing source in /var/tmp/portage/sci-libs/simpleitk-1.2.3/work/SimpleITK-1.2.3 ...
 * Working in BUILD_DIR: "/var/tmp/portage/sci-libs/simpleitk-1.2.3/work/simpleitk-1.2.3_build"
 * Hardcoded definition(s) removed in SuperBuild/CMakeLists.txt:
 *    set(CMAKE_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR} CACHE PATH "Where all
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sci-libs/simpleitk-1.2.3/work/SimpleITK-1.2.3 ...
 * Working in BUILD_DIR: "/var/tmp/portage/sci-libs/simpleitk-1.2.3/work/simpleitk-1.2.3_build"
cmake -C /var/tmp/portage/sci-libs/simpleitk-1.2.3/work/simpleitk-1.2.3_build/gentoo_common_config.cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr -DUSE_SYSTEM_LUA=ON -DUSE_SYSTEM_SWIG=ON -DUSE_SYSTEM_GTEST=ON -DUSE_SYSTEM_ITK=ON -DUSE_SYSTEM_VIRTUALENV=ON -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/simpleitk-1.2.3/work/simpleitk-1.2.3_build/gentoo_toolchain.cmake  /var/tmp/portage/sci-libs/simpleitk-1.2.3/work/SimpleITK-1.2.3
loading initial cache file /var/tmp/portage/sci-libs/simpleitk-1.2.3/work/simpleitk-1.2.3_build/gentoo_common_config.cmake
-- The C compiler identification is GNU 10.2.0
-- The CXX compiler identification is GNU 10.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/x86_64-pc-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at CMake/sitkSourceVersion.cmake:41 (message):
  Unable to determine source version!

  Please use the git repository or an official source distribution.

Call Stack (most recent call first):
  Version.cmake:17 (include)
  CMakeLists.txt:23 (include)

-- Building SimpleITK version "1.2.3.dev-gGITDIR-NOTFOUND"
-- Performing Test CXX_HAS_stdcxx11
-- Performing Test CXX_HAS_stdcxx11 - Success
-- Checking if c++11 is required...
-- Checking if c++11 is required... NO
-- Performing Test CXX_HAS-Wall
-- Performing Test CXX_HAS-Wall - Success
-- Performing Test CXX_HAS-Wno-long-double
-- Performing Test CXX_HAS-Wno-long-double - Failed
-- Performing Test CXX_HAS-Wno-long-long
-- Performing Test CXX_HAS-Wno-long-long - Success
-- Performing Test CXX_HAS-Wno-unused-local-typedefs
-- Performing Test CXX_HAS-Wno-unused-local-typedefs - Success
-- Performing Test CXX_HAS-Wno-strict-overflow
-- Performing Test CXX_HAS-Wno-strict-overflow - Success
-- Performing Test CXX_HAS-Wextra
-- Performing Test CXX_HAS-Wextra - Success
-- Performing Test CXX_HAS-Wformat_2
-- Performing Test CXX_HAS-Wformat_2 - Success
-- Performing Test CXX_HAS-Wno-format-nonliteral
-- Performing Test CXX_HAS-Wno-format-nonliteral - Success
-- Performing Test CXX_HAS-Wunused
-- Performing Test CXX_HAS-Wunused - Success
-- Performing Test CXX_HAS-Wpointer-arith
-- Performing Test CXX_HAS-Wpointer-arith - Success
-- Performing Test CXX_HAS-Winvalid-pch
-- Performing Test CXX_HAS-Winvalid-pch - Success
-- Performing Test CXX_HAS-Wcast-align
-- Performing Test CXX_HAS-Wcast-align - Success
-- Performing Test CXX_HAS-Wdisabled-optimization
-- Performing Test CXX_HAS-Wdisabled-optimization - Success
-- Performing Test CXX_HAS-Woverloaded-virtual
-- Performing Test CXX_HAS-Woverloaded-virtual - Success
-- Performing Test CXX_HAS-Wshadow
-- Performing Test CXX_HAS-Wshadow - Success
-- Performing Test CXX_HAS-Wwrite-strings
-- Performing Test CXX_HAS-Wwrite-strings - Success
-- Performing Test CXX_HAS-Wstrict-null-sentinel
-- Performing Test CXX_HAS-Wstrict-null-sentinel - Success
-- Performing Test CXX_HAS-Wno-invalid-offsetof
-- Performing Test CXX_HAS-Wno-invalid-offsetof - Success
-- Performing Test SITK_HAS_CXX11_STATIC_ASSERT
-- Performing Test SITK_HAS_CXX11_STATIC_ASSERT - Success
-- Performing Test SITK_HAS_CXX11_FUNCTIONAL
-- Performing Test SITK_HAS_CXX11_FUNCTIONAL - Success
-- Performing Test SITK_HAS_CXX11_TYPE_TRAITS
-- Performing Test SITK_HAS_CXX11_TYPE_TRAITS - Success
-- Performing Test SITK_HAS_CXX11_UNORDERED_MAP
-- Performing Test SITK_HAS_CXX11_UNORDERED_MAP - Success
-- Performing Test SITK_HAS_CXX11_NULLPTR
-- Performing Test SITK_HAS_CXX11_NULLPTR - Success
-- Performing Test SITK_HAS_CXX11_UNIQUE_PTR
-- Performing Test SITK_HAS_CXX11_UNIQUE_PTR - Success
-- Performing Test SITK_HAS_CXX11_ALIAS_TEMPLATE
-- Performing Test SITK_HAS_CXX11_ALIAS_TEMPLATE - Success
-- Performing Test SITK_HAS_TR1_SUB_INCLUDE
-- Performing Test SITK_HAS_TR1_SUB_INCLUDE - Success
-- Performing Test SITK_HAS_TR1_FUNCTIONAL
-- Performing Test SITK_HAS_TR1_FUNCTIONAL - Success
-- Performing Test SITK_HAS_TR1_TYPE_TRAITS
-- Performing Test SITK_HAS_TR1_TYPE_TRAITS - Success
-- Performing Test SITK_HAS_TR1_UNORDERED_MAP
-- Performing Test SITK_HAS_TR1_UNORDERED_MAP - Success
-- Looking for C++ include stdint.h
-- Looking for C++ include stdint.h - found
CMake Warning at CMakeLists.txt:319 (message):
  Unable to file ITK's NOTICE File!

   This file will not be included in SimpleITK packaging!

-- Performing Test Weak Link MODULE -> SHARED (gnu_ld_ignore) - Success
CMake Warning (dev) at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
  The package name passed to `find_package_handle_standard_args` (Lua) does
  not match the name of the calling package (LuaInterp).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  CMake/FindLuaInterp.cmake:41 (find_package_handle_standard_args)
  CMake/sitkLanguageOptions.cmake:92 (find_package)
  CMakeLists.txt:340 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found Lua: /usr/bin/lua (found version "5.1.5")
-- Found Tclsh: /usr/bin/tclsh (found version "8.6")
CMake Warning (dev) at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
  The package name passed to `find_package_handle_standard_args` (Lua) does
  not match the name of the calling package (LuaInterp).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  CMake/FindLuaInterp.cmake:41 (find_package_handle_standard_args)
  CMake/sitkGenerateFilterSource.cmake:9 (find_package)
  Code/BasicFilters/CMakeLists.txt:4 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Processing json files...
-- Processing json files...done
-- Found SWIG: /usr/bin/swig (found suitable version "4.0.2", minimum required is "3")
-- Found Lua: /usr/lib64/liblua.so;/usr/lib64/libm.so (found version "5.1.5")
-- Found PythonInterp: /var/tmp/portage/sci-libs/simpleitk-1.2.3/temp/python3.7/bin/python (found version "3.7.9")
-- Found PythonVirtualEnv: /usr/bin/virtualenv (found version "virtualenv 20.0.35 from /usr/lib/python3.7/site-packages/virtualenv/__init__.py")
CMake Error at Wrapping/Python/LegacyPackaging.cmake:60 (message):
  In sufficient version of virutalenv for building wheels.  Require
  virtualenv>=13.0.
Call Stack (most recent call first):
  Wrapping/Python/CMakeLists.txt:94 (include)

-- Found TCL: /usr/lib64/libtcl.so
-- Found TCLTK: /usr/lib64/libtcl.so
-- Found TK: /usr/lib64/libtk.so
-- Found R: /usr/lib64/R/include (found version "4.0.2")
CMake Error at Wrapping/R/CMakeLists.txt:64 (math):
  math cannot parse the expression: "9000 + ": syntax error, unexpected $end
  (7).

-- Found GTest: /usr/lib64/libgtest.so
-- Generating tests...
-- Generating tests...Done
-- <<< Gentoo configuration >>>
Build type      Gentoo
Install path    /usr
Compiler flags:
C               -march=native -O3 -pipe
C++             -march=native -O3 -pipe
Linker flags:
Executable      -Wl,-O1 -Wl,--as-needed
Module          -Wl,-O1 -Wl,--as-needed
Shared          -Wl,-O1 -Wl,--as-needed

What would be the best way to help it figure out the version? Seeing as this will be a versioned archive, just patching the string into some file would also be an option — if that's the best apparoach, any idea how this should be done?

CMake Error at Wrapping/Python/LegacyPackaging.cmake:60 (message):
In sufficient version of virutalenv for building wheels.  Require
virtualenv>=13.0.

You can disable using virtualenv with: SimpleITK_PYTHON_USE_VIRTUALENV:BOOL=OFF.

CMake Error at Wrapping/R/CMakeLists.txt:64 (math):
math cannot parse the expression: "9000 + ": syntax error, unexpected $end
(7).

I this is coming from the following line:
elseif(DEFINED SimpleITK_VERSION_DEV)
  math(EXPR DEV "9000 + ${SimpleITK_VERSION_DEV}")
  set(SimpleITKR_VERSION "${SimpleITKR_VERSION}-${DEV}")
endif()

Where SimpleITK_VERSION_DEV is defined but an empty string. This is likely an unexpected side effect of building from wrong tar ball. The one Github automatically creates ( and we can't disable from the downloads ) is missing an added version file. Please download the file label "SimpleITK-1.2.3.tar.gz" in the releases section. Any reason not to be using 1.2.4?

This is likely an unexpected side effect of building from wrong tar ball.

Oh, indeed. Thank you for this.

Any reason not to be using 1.2.4?

The two commits which I was linked to as breaking Lua — 77a3d89 and 0dfae3e — were merged 2019-10-22, and 1.2.4 was released on 2019-11-13 . I was assuming 1.2.3 was the last release that worked with older lua.

The two commits which I was linked to as breaking Lua — 77a3d89 and 0dfae3e — were merged 2019-10-22, and 1.2.4 was released on 2019-11-13 . I was assuming 1.2.3 was the last release that worked with older lua.

That is not correct. The history is not chronological. Patches were applied to 1.2 while 2.0 was under development.
You can run a command line git tag --contains 77a3d89 to verify.

ok, using the proper archive instead of the GitHub autogenerated one fixed this :)


Detecnet example in full screen mode

Hi,
I want to run detectnet example in full screen mode (without any title bar, window decorations). How can I achieve it? I do really appreciate any advice.
Thanks beforehand!

You can try setting the resolution of the stream the same as you screen


No connection and node-red crashes when internet connection is down during and after deploy

Setting:
Using the node on Weidmueller u-controll web device (node red v1.1.2) . Provisioning with DPS (CA based x509).
Scenario for reproduction:

Internet connection breaks (pull ethernet connection).
Deploy application.
Node changes to "disconnected" and later to "error"
Internet connection is reestablished
In case one tries to send a telemetry message, node-red crashes.

Log:
Dec 16 12:39:09 ucm node[5469]: 16 Dec 12:39:09 - [info] Starting flows

Dec 16 12:39:09 ucm node[5469]: 16 Dec 12:39:09 - [info] [azureiotdevice:d2662533.509318] IoTHubDevice_4 -> Initiate IoT Device settings.

Dec 16 12:39:09 ucm node[5469]: 16 Dec 12:39:09 - [info] [azureiotdevice:d2662533.509318] IoTHubDevice_4 -> Initiate IoT Device connection string.

Dec 16 12:39:09 ucm node[5469]: 16 Dec 12:39:09 - [info] [azureiotdevice:d2662533.509318] IoTHubDevice_4 -> Provision IoT Device using DPS.

Dec 16 12:39:10 ucm node[5469]: 16 Dec 12:39:10 - [info] Started flows

Dec 16 12:39:29 ucm node[5469]: 16 Dec 12:39:29 - [error] [azureiotdevice:d2662533.509318] IoTHubDevice_4 -> Error registering device: {"name":"NotConnectedError"}

Dec 16 12:56:14 ucm node[5469]: 16 Dec 12:56:14 - [red] Uncaught Exception:

Dec 16 12:56:14 ucm node[5469]: 16 Dec 12:56:14 - NotConnectedError: rhea: connection disconnected

Dec 16 12:56:14 ucm systemd[1]: uc-ui-node-red.service: Main process exited, code=exited, status=1/FAILURE

@dkby: problem resolved. Will be released with version 0.2.1.


Add support for sending the json as a query string parameter

e.g.
jsonlint.com?q={"test": 123}
And it will validate and format the json.
Thanks for the great service!

@gdoron use json parameter with urlencoded JSON as value. Example: http://jsonlint.com/?json={"test": 123}

Thanks.
BTW, I whitelisted your website on my adblockers, great service!

Thank you @gdoron :)


localhost connection refused for paper trading

Hello, I am very new to gekko. I'm trying to run paper trader on BTC/ETH pair and the strategy is DEMA.
The process of running UI and choosing appropriate parameters goes smoothly. However, when I click 'run' I get an error "Connection refused" in bash. I turned off firewall, but that did not help.
If this may help, I am running windows 10 and the server is not my computer.

Strange, could you post what is visible in the commandline prompt?

Bash screenshot -

Just in case.


[ERROR] java.io.EOFException: Expected to read bytes from the stream

Describe the bug
I am getting EOF exception while running DependencyCheck-5.3.2.
Note: I am still getting dependency-check-report.xml
Version of dependency-check used
The problem occurs using version 5.3.2 of the DependencyCheck(Cli)
Log file
WHS-1625:decompscript darshanmhatre$ ~/Downloads/dependency-check-5.3.2/bin/dependency-check.sh --project testing --scan ./webgoat/ --format XML -disableOssIndex --enableExperimental --disableCentral --disableRetireJS
[INFO] Checking for updates
[INFO] Skipping NVD check since last check was within 4 hours.
[INFO] Check for updates complete (80 ms)
[INFO]
Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user’s risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.
[INFO] Analysis Started
[INFO] Finished Archive Analyzer (0 seconds)
[INFO] Finished File Name Analyzer (0 seconds)
[INFO] Finished Jar Analyzer (0 seconds)
[ERROR] ----------------------------------------------------
[ERROR] .NET Assembly Analyzer could not be initialized and at least one 'exe' or 'dll' was scanned. The 'dotnet' executable could not be found on the path; either disable the Assembly Analyzer or add the path to dotnet core in the configuration.
[ERROR] ----------------------------------------------------
[INFO] Finished Python Distribution Analyzer (0 seconds)
[WARN] An error occurred while analyzing '/Users/darshanmhatre/decompscript/webgoat/jna-4.1.0.jar/com/sun/jna/w32ce-arm/jnidispatch.dll' (PE Analyzer).
[INFO] Finished PE Analyzer (0 seconds)
[INFO] Finished Dependency Merging Analyzer (0 seconds)
[INFO] Finished Version Filter Analyzer (0 seconds)
[INFO] Finished Hint Analyzer (0 seconds)
[INFO] Created CPE Index (1 seconds)
[INFO] Finished NPM CPE Analyzer (1 seconds)
[INFO] Created CPE Index (0 seconds)
[INFO] Finished CPE Analyzer (1 seconds)
[INFO] Finished False Positive Analyzer (0 seconds)
[INFO] Finished NVD CVE Analyzer (0 seconds)
[INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
[INFO] Finished Dependency Bundling Analyzer (0 seconds)
[INFO] Analysis Complete (4 seconds)
[ERROR] java.io.EOFException: Expected to read bytes from the stream
To Reproduce
Link for the codebase : https://github.com/darshanmhatre/webgoat.git
Depcheck command used : dependency-check-5.3.2/bin/dependency-check.sh --project testing --scan ./webgoat/ --format XML -disableOssIndex --enableExperimental --disableCentral --disableRetireJS
Expected behavior
It should run without java.io.EOFException.

A patch will be included in the next release.


1.13.0-rc4 cherry-picks

Convert DanglingOnly to Filters for docker image prune #28535
Fix issue where secret ID is masked by name #28896
refactor plugin install #28963
improve TestServiceLogs for the goroutine issue #28915 #28974
refine api swagger.yaml towards image create status code #29062
Print checkpoint id when creating a checkpoint #29073
Add bash completion for secret management #29109
Fix bash completion for docker service create|update #29111
Add bash completion for docker version --format #29113
Handle logging in compose to swarm #29123
Windows: make.ps1 Throw exception on failure #29145
Windows: Dockerfile 2GB clarification Hyper-V #29149
add 403 for endpoint network create #29166
Add bash completion for docker network create --attachable #29182
Fix docker inspect <unkown object> issue on Windows #29186
fix #29199, reset container if container start failed #29200
modify URLs for bind docker in docs/api #29201
Add zsh completion for 'docker network create --attachable' #29202
hack/make.sh: fix BUILDTIME #29082
Optimize the log info for client test #29026
api/types/container,client: gofmt #29134

I kicked off ARM on this one /cc @anusha-ragunathan

@jhowardmsft looks like it doesn't compile on windows, am I missing something ?

@vieux no, it's the build script that tallies some git stats: https://github.com/docker/docker/blob/master/hack/make.ps1#L192

@friism how come https://github.com/docker/docker/pull/29145 didn't error out ?

Looking

Yes, I see the problem. Working on a fix. PR shortly... (It's tripping on .png files which git diff is returning - for number of lines changed rather than a number).

#29231 fixes it.

#29130 should also be included (but maintainers need to merge it first).

LGTM


Refernence Citations

At the moment, we have the authors and the link to the bib. What is the common practice, probably just the link?

Changed citation to just the link.


Parallelisation - task scheduling potential enhancements

The generic nature of PlantSimEngine makes it harder to know what models or which independent subgraphs of the depency graph should be prioritised.
The exact order will heavily depend on the complexity of the models, and on their coupling, making it very case-specific. Sometimes a specific model might be quite computationally complex to run and a bottleneck to prioritise, sometimes there might be long dependency chains that outweigh the other bottlenecks.
One possible way of determining a usually good order when the user needs to run a simulation over many timesteps would be to have a short dry run that collects a little data, to then determine what seems like the potential ideal order in which the subgraphs and models should be run.
This might not be trivial with multi-scale, hard dependencies and soft dependencies. Perhaps having modelers declare some particularly complex models as being heavier to handle could help, as well.

Thinking back to this enhancement, considering the context, we might already be able to guess which portions are likely to be the most computationally expensive. These'd be :

Light interception calculations on the 3D geometry (not part of PSE)
Models which act on the most numerous organs of the plant (leaves, flowers, fruits, internodes, which will be many orders of magnitude more numerous than the number of plants in an agroforestry context, once the plant has grown a few years)
Models that update the environment or interact with it in the case where its granularity is small enough (for instance, a voxelisation around the plant)

It actually seems likely that a dry run (especially on an immature tree with much fewer terminal organs) might not lead to a more interesting task scheduling, or would have the same predictable bottlenecks in many situations. It might be more valuable to organise scheduling and data around the three cases described in the list above.


Option to ignore certain exceptions

Cassandra does not support IF NOT EXISTS for ALTER table ADD ... queries, so it might be a good idea to be able to ignore failures in these queries.
Might be a global configuration where you specify the codes/parts of cassandra error message to be ignored.

I think I came up with a solution to achieve this in the context of another pull request. I can imagine to have ErrorHandlers inside the Database instance that deal with failures. The default handling would be the current behavior but by making this a separate component, you would be able to overwrite or change parts of the behavior to match your needs. Like this you could implement a handler that ignores the errors you have in mind.

I've raised a PR for this #59 . It should be easily extensible to override the failure mechanism on a task.


Enable CSR approval and issuing by default.

Currently when using the kubectl certificate approve <csr> the kube controller manager does not Issue the certificate (only Approve)
microk8s.kubectl get csr
NAME                AGE    REQUESTOR   CONDITION
uid-val.validator   2m5s   admin       Approved

This is due to missing cert signing arguments in the kube-controller-manager arguments.
By simply adding these in the /var/snap/microk8s/current/args/kube-controller-manager
--cluster-signing-cert-file=${SNAP_DATA}/certs/ca.crt
--cluster-signing-key-file=${SNAP_DATA}/certs/ca.key

it will automatically Approve and Issue the certificate
kubectl get csr
NAME                AGE   REQUESTOR   CONDITION
uid-val.validator   20m   admin       Approved,Issued

Merged to master.

I'm on version the candidate channel, version 1.14.2, where this fix is included, and it doesn't work. microk8s.kubectl certificate approve still does not issue a certificate.
inspection-report-20190527_151632.tar.gz

Looking at inspection-report/args/kube-controller-manager in the report it seems the relevant arguments aren't present, although they most definitely are in /snap/microk8s/current/microk8s-resources/default-args/kube-controller-manager. Strange.

@niklasholm was that deployment upgraded to 1.14.2 from 1.14.1? If that is the case it should be expected that we do not touch the current configuration (unless there is something important).


how to update, uninstall or check?

Let's say I want to update the configuration, should I just build it and install it on the target machine?
Is it better to first run something that uninstalls the current miner and then install the new one?
additionally, is there a way to check the current miner configs in the target machine to check if the configs are updated properly.
Thanks and Regards.

Alright, thanks
closing this


Extension causes high cpu load

Issue Type: Performance
Extension Name: LiveServer
Extension Version: 5.5.1
OS Version: Windows_NT x64 10.0.17134
VSCode version: 1.31.1

:warning: Make sure to attach this file from your home-directory: C:\Users\Pedro Fonseca\ritwickdey.LiveServer-unresponsive.cpuprofile.txt :warning:
Find more details here: https://github.com/Microsoft/vscode/wiki/Explain:-extension-causes-high-cpu-load

Hello. few questions!!,
When you're getting this issue? after load of vscode or after clicking to Go Live ?
Is this happen every-time ? or random ?
Please reply here https://github.com/ritwickdey/vscode-live-server/issues/278
Duplicate of https://github.com/ritwickdey/vscode-live-server/issues/278

[Note: This issue is closed by a javascript code (written by me).
https://gist.github.com/ritwickdey/36682dabe4a992c57e4562c935bfbbdd ]

Thank You.


FOGL-2461 Tests fixed

We should remove comments like; these are sort of debug comment and should be avoided in prod mode.
LOGGER.info("plugin_start called")
Or its content must be similar to what we have in shutdown or reconfigure log statements.
Also there are few inline comments in plugin reconfigure e.g. # plugin_init where the next line is a straight forward call itself as plugin_init. We should avoid such over statements


small fixes

Codecov Report

Merging #1001 (9343302) into master (958a79b) will decrease coverage by 0.02%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master    #1001      +/-   ##
==========================================
- Coverage   76.06%   76.04%   -0.03%
==========================================
  Files         155      155
  Lines        3777     3778       +1
  Branches      742      743       +1
==========================================
  Hits         2873     2873
- Misses        898      899       +1
  Partials        6        6

Impacted Files
Coverage Δ

src/alerts/actions.ts
100.00% <ø> (ø)

src/firmware/alerts/NoDfuHub.tsx
25.00% <ø> (ø)

src/index.tsx
0.00% <ø> (ø)

src/alerts/sagas.ts
94.73% <50.00%> (-5.27%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 958a79b...9343302. Read the comment docs.


[9.x] Add optional params to FormRequest validated method

The PR for [8.x] was closed because of breaking changes, so I guess I should open a new one for [9.x]?
Adds optional params to FormRequest validated() method, so developer can access validated data and provide defaults in cleaner, more readable way IMO.
Before:
$data = $request->validated()
$email = $data['email']
$role = $data['role'] ?? UserRole::GUEST

After:
$email = $request->validated('email')
$role = $request->validated('role', UserRole::GUEST)

Method signature is the same as $request->input($key, $default) so there is no learning curve. It is also much faster than $request->input('key'), about 2-3x faster in my tests, performing similarly to $request->get('key').

Can you rebase this PR and mark it as ready when done? Thanks.

Check #38366

Gonna close this one now. Feel free to resend when you're ready.


Adding extension.ts tests

AB#233670

Codecov Report

Merging #634 into master will increase coverage by 0.18%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #634      +/-   ##
==========================================
+ Coverage   43.96%   44.15%   +0.18%
==========================================
  Files          71       71
  Lines        4890     4890
  Branches      848      848
==========================================
+ Hits         2150     2159       +9
+ Misses       2438     2433       -5
+ Partials      302      298       -4

Impacted Files
Coverage Δ

docs-markdown/src/controllers/no-loc-controller.ts
34.78% <0.00%> (+4.34%)
:arrow_up:

docs-markdown/src/helper/markdown-table.ts
77.22% <0.00%> (+5.94%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81fdfd7...471201f. Read the comment docs.


Ctrl + clicking "File"/"Directory" in the text editor opens empty docs.

Operating system or device, Godot version, GPU Model and driver (if graphics related):
Build 8b52d53
Issue description:
Self-explanatory.

any more detail appreciated, which text editor?

@reduz

This seems to be caused as some classes are prefixed with underscores, namely the ones in core_bind.cpp causing it to search for "_file" rather then "file". It finds the docs correctly when the underscore is removed.


use footcite in otp.py

Reference issue
closes #8926
What does this implement/fix?
citation year was listed as incorrect in doc, added footcite to correct reference.

Thanks @mdclarke !


[Stream Analytics] Bump up version to 0.1.2

Closes https://github.com/Azure/azure-cli/issues/20913
Bump up version to 0.1.2 and support clusters and private endpoints.

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

Stream Analytics

/azp run


Getting an error on cookie.

This is what i did :
const distube = new DisTube(client, {emitNewSongOnly: true, leaveOnStop: false, youtubeCookie:"XXXX", youtubeIdentityToken:"XXXX" });
And it's returning this error when i try to play any song :
DisTube: Downloaded youtube-dl 2020.09.20 (node:9760) UnhandledPromiseRejectionWarning: Error: Error parsing info: Cookie header used in request, but unable to retrieve video metadata

Your cookie in DisTubeOptions is not valid.
If you use document.cookie, it won't return some HttpOnly cookies are sent with the request.
Solution: Read the documentation to know how to get it correctly.


Improve undo manager API

add beginUndoGroup()/endUndoGroup() to perform batch edit
remove prepareTarget(), this is unneeded since the undoManager can subscribe to notifications (as opposed to the model where you create an undo command and apply it to the document, here the undoManager knows after the fact)
remove the index parameters to do multiple undos at once etc. this is unnecessary complicating and defcon API doesn't really provide access to undo indexes in the first place. This also avoids making the API biased towards a particular implementation.

Note: I feel like getUndoText()/getRedoText() could be removed since it's primarily for the UI which will probably call the underlying object rather than the defcon API, for now I left it since I don't mind its presence.
r? @anthrotype
cc @typemytype

Could you provide an short example of how the undo 🐝 would change with patch?
Do we have any tests for the undo manager? If not, we better should.

Well there's no undo manager in defcon, it's just an interface that clients can implement.

Is anybody else besides Trufont also using (or is interested in using) this undo manager interface in defcon?
Frederik, does Robofont uses this at all?
Just asking to see what the impact of merging this patch could be on others.
I also think that, if we don't want the API to be biased towards a particular implementation, the interface should stay abstract enough and it should be properly documented.

RF does not uses this, it has his own implementation.
The problem is that undo can be embedded in multiple different ways:

as a stack of actions, performing the opposite of the actial action: like move((10, 0)) --> move((-10, 0)) (very simplyfied)
as a stack of snapshots of data: copy all data before the action so it can be restored afterwards
a combination of the two above

RF has the stack with snapshot undo method. This is similar as what defcon implemented. I guess it was even based on the RF model.
I dont want to bind everything into a notification. Calling an action inbetween a prepareUndo - action - performUndo provides more control for devs: they can put multiple action in a single undo or even split an action into multiple undos. Or even ignore undos.
Im not a pro to even implement methods related to undos as most of them are very specific for an implementation. They will not work in an script running outside an implementation.

And you cannot really remove the _dict
Wait at least a year to remove deprecated methods. Only Superpol uses defcon UFO3 for now...

I guess the beginUndoGroup/endUndoGroup is also related to an implementation, then we could just make it a thin API with canUndo/undo/canRedo/redo.

Updated.

Added docstrings.


Pipelines / Transform functionality discussion

We're planning to have some kind of recommended pipeline / transform system(s) for Frictionless toolkit.
@roll i (rufus) am opening this issue so we have a place to discuss plans for transform / pipelines functionality. I think this is something worth discussing a bit and maybe even speccing in an RFC.
This is an It's something @rufuspollock has now been involved with implementing several times (including dataflows and a new system called AirCan https://github.com/datopian/aircan) - we may want to think whether we directly reuse this or build anew. It is also an area where there is a lot of existing open source tooling so it is worth thinking what we reuse vs what we build ourselves.
Tasks

[ ] Summarize existing work
[ ] Identify the gaps / opportunities / challenges
[ ] Describe plan of action in e.g. RFC
[ ] Sign off

Analysis
Existing work (related to frictionless)

https://github.com/datahq/dataflows
frictionless-py @roll perhaps you could summarize what is happening here as i saw some issues e.g. https://github.com/frictionlessdata/frictionless-py/issues/389
https://github.com/datopian/aircan - frictionless-oriented data pipeline system built on top of AirFlow (i.e. pipelines are DAGs in AirFlow). See also https://tech.datopian.com/flows/
(Deprecated) Data Factory in DataHub

Existing work (non frictionless) - see also https://tech.datopian.com/flows/research/

AirFlow
Luigi
Petl
...

Please preserve this line to notify @lwinfree (lead of this repository)

@roll @risenW ^^^

@rufuspollock @risenW
Frictionless for Python now supports:

core transform functionality powered by PETL (ATM ported around 100 PETL operations)
a plugin that can run DataFlows pipelines

https://colab.research.google.com/drive/1C4dFWDExyxzGIwLUovrDQZghZK4JK2PD

@roll great and i've read through that. What do you think of doing a bit of syncing / planning before we proceed much further?
Also some questions:

Looking at the code it looks like you export the resource to be used by PETL (which seems very sensible) rather than wrapping. Is that right?
Does it run in streaming or non streaming mode?

@rufuspollock
Yes, of course.
Another question is that in my opinion, I don't think it should be driven by some kind of committee :smiley:. Honestly, I don't know is it a solution good enough or not until real people start using it in real projects and give us feedback.
For example, as far as I can remember, this project https://github.com/datasets/covid-19 was driven by DataFlows initially and it uses Pandas. I guess you ran into some shortcomings of the pipeline and had to switch. Also, it's very interesting whether we will be able to prototype something like this using Frictionless Transform.

Looking at the code it looks like you export the resource to be used by PETL (which seems very sensible) rather than wrapping. Is that right?
I use a pretty simple adapter that makes Resource to be compatible with PETL Table Container interface - https://petl.readthedocs.io/en/stable/intro.html#conventions-table-containers-and-table-iterators. It allowed to just re-use all their battle-tested processors for data and actually write only metadata updates in our processors. Although we fully wrap PETL as a project so our users don't need to go to their documentation once we have finished ours.

Does it run in streaming or non streaming mode?
It's streaming

hi @roll do you think we can close this? has it been resolved in frictionless-py?


Advanced Recipes Overview page is listed as a source for how to setup a custom authentication method with a private docker registry, but does not include such information

File: registry/recipes/index.md, CC @joaofnfernandes
I'm working on setting up a private repo with advanced authentication. I made my way to the recipes file, which had then provided several advanced configurations of the docker repository, but none of them include an example of how to set up an authenticator for repo logins. All the linked examples only provide basic username and password auth.
I'll be working through the problem and will create a PR with updated documentation if no one else can get to it first :)

Closing this ticket due to its age, and the impending refactor. If you think this is in error, feel free to reopen. Thanks!


Reduce size of testLargeMethods2 test

Purpose

Describe the problems, issues, or needs driving this feature/fix and include links to related issues.

Since we only have 120 seconds allocated to run a test, testLargeMethods2 failed in the Github runner because it took more than that giving a timeout exception. Hence reduced the size of the test, thus, time taken.
Approach

Describe how you are implementing the solutions along with the design details.

Samples

Provide high-level details about the samples related to this feature.

Remarks

List any other known issues, related PRs, TODO items, or any other notes related to the PR.

In the local machine following were the time taken for CodegenErrorsTests.

After the change testLargeMethods2 take less than 5 seconds.

Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

@HindujaB please note the change.


Add Semigroup instance for ActionT

This adds a Semigroup instance for ActionT to make eve compile with newer base versions.
I hope it does't break things on older ghc versions (>=8.*).

Thanks for merging :-)


Unable to stream Nest doorbell  without Hass.

Hi all,
i try to stream my Nest doorbell 2 wired without Hass in my Frigate in docker
I followed this: https://developers.google.com/nest/device-access/get-started
to obtain refresh_token and devide id
and i used this code in my frigate/config.yml
streams:
nest-doorbell: nest:?client_id=&client_secret=&refresh_token=&project_id=&device_id=***
but i received a error

2023-10-12 10:33:06.767427016  10:33:06.767 ERR github.com/AlexxIT/go2rtc/cmd/streams/producer.go:53 > error="unsupported scheme: nest:?client_id=8320
Thanks for your help

You need to use go2rtc with nest support. Probably you using very old version