narrow_mode: Add option to hide user list on wide screens.

We store the css required to hide user list on narrow screens in
narrow_width_styles and apply them on wise screens if
narrow_mode is enabled in user_settings.
Declaring narrow_width_styles requires addition of
postcss-mixins packages. This helps keep codebase clean
of repeated code.
To apply the JS required to hide user list in wide screens we
make the app behave as if we were in narrow width on wide
widths by making is_narrow True in message_viewport.
narrow_mode field is added to UserProfile which is used
to toggle this option by the user.

There is some issue with scrollbar width, which I am working on. Need to run this both on mac and window to properly test it.

This should be ready for review. Since the changes only apply if narrow_mode is enabled, should be a safe deploy. It is important this gets tested on different OS and browsers.

Having a blank right sidebar looks funny to me. Maybe we should put the gear just to the right of the search and make the right sidebar really narrow?

I think we should also drop extra options from the gear menu. Maybe like this?

I would also drop the keyboard in the right sidebar. I think it's kind of weird/confusing, the tooltip gets messed up as it currently is, and people can find the info in the menus if they need it.

I find it kind of confusing that there's no indication that the compose buttons won't work until you click on them. Maybe they should just be disabled?
Regardless, perhaps the "Log in to send messages" button should probably be more visually salient.

This PR is actually for the logged in user experience right now. I will do a small commit to enable it permanently for spectators. To see what this PR does:

Run provision
enable hide user list option  and reload.

I have noticed the rest of your points and will either act upon them or open #design topics for them as I get to them.

What do you think?
Oh yeah! I thought I fixed it, let me see.

I center aligned it. It is hard to do more radical changes in the navbar because it is so carefully crafted for every width / browser / OS and different modes it supports.

I recall now this is actually a scrollbar thing. Let me do another fix.

Okay, I fixed it on windows and mac (with or without scrollbar).

As I scroll up and down when testing this, I notice the right hand border next to the "users" button appearing and disappearing, which looks glitchy.
I think it's better without the border, to match the left side of the top bar.

Since I cannot reproduce it, we will get more details on that once it is test deployed.

As I scroll up and down when testing this, I notice the right hand border next to the "users" button appearing and disappearing, which looks glitchy. Whether or not there's a border there also seems to be width-dependent.
I think it's better without the border, to match the left side of the top bar.

Fixed it. Thanks for finding this.

I merged the first commit and pushed back here.  I was able to get into these states by resizing the window while viewing settings and then toggling the checkbox (one screenshot for each direction):

My guess is we're missing a resize.js or server_events_dispatch handler to recalculate something after changing this setting value.

I think you didn't reload the page.

Instead of writing more complicated css inserting JS in this code path, it is best that we tell user to reload to avoid a rabbit hole of bugs.

Related issues: #14160, #301

Let me push another change. I forgot to make this work with fluid_layout. It's working, I just need to polish it a bit.

Let's try to merge #20699 first.

There is one issue that I am aware of, that there is extra space on top of the expanded right-column. I can easily fix it, but I don't understand what change is causing it, so I am trying to figure that out.

@alya this should be ready for your review. Thanks!

Cool, thanks! A few observations:

Turning on this

@alya this is ready for another review. I fixed a lot of cases related to the three modes that I missed earlier.

Thanks!
I guess playing around with this combination of settings:

It looks like we end up always showing the list in the left sidebar (including in wide windows). That doesn't quite seem to match what the settings say, but I also don't want us to spend much time on tuning this, since I expect the left sidebar option to be deprecated.

Thinking about the new option, should it just be called "Hide user list", or perhaps "Hide user list (right sidebar)" for clarity? Since the user list is hidden with all window sizes when this option is selected, it seems confusing to specifically mention wide windows.

Makes sense for the name change

@timabbott , we've done some iteration on this one -- I think the next step is a review from you.

Sorry to poke @timabbott but it would be fantastic to get this through the finish line.
Appreciate all the hard work! <3

Latest discussion - https://chat.zulip.org/#narrow/stream/101-design/topic/hide.20userlist.20on.20wide.20screens.20.2321638

Closing as this work was completed with #21638.


Add OTLP receiver to collector

A proof-of-concept of adding an OTLP receiver (#3625) to Jaeger collector/all-in-one.

Imports OTLP receiver from otel-collector
Imports OTLP->Jaeger Proto translator from otel-collector-contrib
Minimal & dirty initialization of the OTEL collector runtime environment (like component.Host)
Receiver apparently implements internal tracing, I am passing the default TracerProvider (which should be no-op), we may need to revisit it later
There are NO external configuration options for this receiver, it only works with defaults. However, internally it does have a config struct which can be in the future PRs hooked up with CLI flags.

Testing with this little program https://gist.github.com/yurishkuro/c60011b9b3b3e6b5c11f0852f877a6c3 produces a trace like this:

@albertteoh care to review?

thanks for the review @albertteoh


Benchmark performances compared to Bref 1 layers

See https://github.com/brefphp/benchmarks/pull/5


Add verbose, _print, and _warn args to PhotosDB.init

verbose: enable verbose output (e.g. copying database, processing faces ,etc.)
_print: specify print function to use for verbose output (default to print but CLI could for example pass click.echo)
_warn (same as _print but for warning, default to logging.warning but CLI could pass wrapper func around click.echo(err=True) )
All methods of PhotosDB would then have access to these

Also add --verbose as common option for all CLI functions

Make --verbose a multiple option so -V -V is equivalent to --verbose 2 etc.

Also add a _debug option


chore(*): rename solution

closes #8

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Provide diagram description fix

Summary

Add a description for each diagram, so that on the main threat model page it is obvious which diagram is associated with which activity. Fixes #553
Description for the changelog

#553 - Add description field on thread model edit page and replace the threat model image with the newly added description.
Other info

Nothing else to add.

Hello @jgadsden,
I needed to apply more changes to the code for the main branch, so I did not rebase the v2-development branch. Instead I applied the changes to the main branch and created a new PR. I will close this one and we will handle it from #555.
If you need anything else, just let me know!


Not possible to specify quoted arguments in tasks

eg
"C:\Program Files\Git\bin\bash.exe" -c "ls -a"
It is impossible to encode "arguments" to resolve to -c "ls -a" as far as I can tell. Every attempt to escape quotes or put spaces in an argument string fail.

care to show some code on how you're trying to do it?

Good point  - I'll add to https://github.com/Microsoft/vscode/issues/1028
For some reason this comment is not appearing there
Steve Lee
OpenDirective http://opendirective.com
On 5 December 2015 at 23:36, Angel Leon<EMAIL_ADDRESS>wrote:

care to show some code on how you're trying to do it?
—
Reply to this email directly or view it on GitHub
https://github.com/Microsoft/vscode/issues/1028#issuecomment-162257053.

So I set up a new task and commented out all the others (the docs dont say this but it seems what is needed). There is a good reason I'm using bash but even so other commands can have similar options. The ls -a' is just something easy to test. The point is the arg is the switch -c` followed by a quoted value string with spaces in it.

{
	"version": "0.1.0",
	"command": "C:\\Program Files\\Git\\bin\\bash.exe",
	"isShellCommand": true,
	"showOutput": "always",
	"args": ["-c ","ls a"],
	"problemMatcher": "$tsc"
}

I realise the problem matcher is not right but for now need to ge the command to run
Here's the various options I tried for args

As -c "ls a" is a single arge we try it as a array element and escape the quotes- ["-c \"ls -l\""] This is difficult as code keep adding extra " characters. Had to copy and paste a single " Get error

'"C:\Program Files\Git\bin\bash.exe"" -c  "ls' is not recognized as an internal or external command,
operable program or batch file.
It seems to be missing every following the space after ls.

Ok, so how about making it 2 strings incase the arge parsing is a little borked  ["-c","\"ls -a\""] but get error

'"C:\Program Files\Git\bin\bash.exe"" -c "ls' is not recognized as an internal or external command,
operable program or batch file.

Again it seems to break at that space

lets try 2nd string without quotes in case they get added.["-c","ls -a"] again get error

'"C:\Program Files\Git\bin\bash.exe"" -c   "ls' is not recognized as an internal or external command,
operable program or batch file.
I tried even more esoteric options like trying to escape the space, but could find nothing that works.
For what it's worth thing rings a vague bell. Many moons ago I did a lot of 'system' OS programming for developer tools involving launching command sub shell processes and handling env and args etc.. in C. I seem to recall there were to APIs for running the cmd shell; one with an array of args and one taking a string for all args. I also seem to recall similar problems handling all possible args.
If I get time in coming weeks I'll take a look at the source

This got addressed by the new terminal runner and task 2.0.0. To make use of the new terminal runner please see: https://code.visualstudio.com/docs/editor/tasks#_convert-from-010-to-200
Tasks 2.0.0 support to specify a command line for shell tasks which give full control over quoting and space escaping.


feat: Meltano manifest

Todo:

[ ] Unit tests (aside: maybe we should run coverage on Meltano during the integration tests, and include that data our coverage stats - I'll open another issue for this if there's any interest)
[ ] Misc schema updates
[ ] Maybe some easy way to mute the schema warnings?

Closes #6876
Closes #7147

Compiling the manifest for Squared has resulted in many schema violations, and a crash with a useless error message following the schema check. We likely need additional schema updates.

Am I understanding right that this isn't meant to resolve the existing longstanding inheritance and env expansion issues?

@cjohnhanson The manifest feature is, among other things, meant to provide us with a foundation with which we can address the longstanding inheritance and env expansion issues by providing a single source of truth for their values. It is not currently being used for this purpose, but hopefully that next step will come relatively soon after this PR has been merged.

In testing this locally with target-jsonl I configured its destination_path to ${DST_PATH}/dir in the plugin-level config and then in the environment-level env (in dev) set DST_PATH: some_path. Then in the manifest, the plugin-level env is resolved as "TARGET_JSONL_DESTINATION_PATH": ""/dir". This seems reasonable to me, if the idea is that we need to resolve these issues upstream in the relevant SettingsService and that the scope here is simply to persist env vars and config as they are currently expanded/inherited--but wanted to confirm that that's the case since I know scope for this has changed over time and you've run up against some bugs in the current env var logic.

Good catch. That sounds like a bug. I'll look into that tonight. I expect it'll be difficult to fix if the problem is what I think it is.

In testing this locally with target-jsonl I configured its destination_path to ${DST_PATH}/dir in the plugin-level config and then in the environment-level env (in dev) set DST_PATH: some_path. Then in the manifest, the plugin-level env is resolved as "TARGET_JSONL_DESTINATION_PATH": ""/dir". This seems reasonable to me, if the idea is that we need to resolve these issues upstream in the relevant SettingsService and that the scope here is simply to persist env vars and config as they are currently expanded/inherited--but wanted to confirm that that's the case since I know scope for this has changed over time and you've run up against some bugs in the current env var logic.

Good catch. That sounds like a bug. I'll look into that tonight. I expect it'll be difficult to fix if the problem is what I think it is.

@cjohnhanson I just replicated this, and it's working (incorrectly) as expected. I was originally thinking that you were describing a problem with expansion happening in the config in the manifest, but it's only in the env section (which is what you said :facepalm:). This will be fixed later on, as more things (e.g. PluginSettingsService) come to depend on the manifest as the source of truth, and the manifest is updated to handle more of the config processing logic itself (i.e. no longer calling .as_env on various SettingsService objects).

@WillDaSilva I may be missing a build step but I'm seeing the following error when trying to compile on my dogfood project:
$ meltano compile
Compiling Meltano manifest for environments: no environment, dev, staging, prod
Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

[Errno 2] No such file or directory: '/Users/edgarramirez/meltano/meltano/src/meltano/schema/meltano.schema.json'

This was easily solved by copying schema/meltano.schema.json to src/meltano/schema/meltano.schema.json but wanted to call out if this may not work during development.

I'm also seeing the options key of settings of kind: option may not be registered in the schema
/Users/edgarramirez/edgar/meltano-dataops/.meltano/manifests/meltano-manifest.prod.json::$.plugins.extractors[0].settings[5]: Additional properties are not allowed ('options' was unexpected)

and another warning I'm not sure how to interpret
/Users/edgarramirez/edgar/meltano-dataops/.meltano/manifests/meltano-manifest.prod.json::$.plugins.extractors[0]: Unevaluated properties are not allowed ('description', 'docs', 'env', 'label', 'logo_url', 'name', 'namespace', 'pip_url', 'plugin_type', 'repo', 'settings', 'settings_group_validation', 'variant' were unexpected)

I may be missing a build step but I'm seeing the following error when trying to compile on my dogfood project:
$ meltano compile
Compiling Meltano manifest for environments: no environment, dev, staging, prod
Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

[Errno 2] No such file or directory: '/Users/edgarramirez/meltano/meltano/src/meltano/schema/meltano.schema.json'

@edgarrmondragon Yeah, this is annoying. It's the same problem as when one tries to do UI development before building the UI. The build process copies the schemas over, but it doesn't get run when doing an editable install (as poetry install does). I'd like to find a solution to this more elegant than something like "run nox -rs build before doing dev work, and after every schema or UI asset change".
Regarding the schema validation issues: thanks for reporting them. I expect many more will follow, as we're now going to be forced to make our schema accurate.

@edgarrmondragon Aside from the warnings about env_aliases, which is deprecated, you should no longer see any warnings when compiling schemas.
We're also able to compile every environment for Meltano Squared without any warnings, plus a few other misc projects I had available.

Many of the settings from settings.yml that could be set in meltano.yml weren't included in meltano.schema.json. I've added them, but have run into a problem.
Settings with periods in the name (e.g. snowplow.collector_endpoints, ff.strict_env_var_mode, etc.) can be specified in the yaml with the period in the name, or expanded (e.g. "snowplow": {"collector_endpoints": ...}, etc.). The schema can only represent one of these 2 representations, and ideally it would be the expanded one so that each level in the schema can have additional details/requirements added.
As such, when a name is used with a period in it it won't match the schema. To work around this we'll have to expand all of these keys into nested dictionaries. This is probably how it should be stored in the manifest files too, rather than exclusively using this representation during the schema check.

@WillDaSilva - I will put some comments here based on my review of the output files.
1. We may want to pull back on the number of env vars populated via setting defaults, especially at the global level.
I don't see this as a blocker for merging, but wanted to call it out.
The benefit of including a feature flag expansion for something like MELTANO_UI_BIND_HOST and MELTANO_UI_BIND_PORT presumably would be extra stability/portability. However, this creates an inference that these are specifically requested or declared in the project, whereas they are actually injected based on those settings' defaults.
That said, since there are no dependencies on these undeclared, unreferenced env vars, we should be able to decide (if we choose this) to not expand them. Since these (specifically Meltano global settings and feature flags) are generated by Meltano and only consumed by Meltano, and we should be able to could on Meltano to re-assert those defaults when needed if the manifest does not declare them.
2. Consider popping include_paths (and similar) from the declaration , or nesting under annotations: { meltano: {...} }
Since include_paths is made obsolete/stale by the generation of the manifest file that necessarily consumed those paths, we should consider removing or nesting include_paths rather than keeping it as a top-level artifact in the file.
My preference would be to move these down under top-level "Meltano" annotations, as {..., annotations: {meltano: {include_paths: [...]}}. A parser of the manifest should not be re-consuming the include_paths, although they might want to present this as an "FYI" for documentation to human or machine-readers.
For other cases as well where some interesting information is valuable to preserve, but shouldn't be operated on directly from the context of the compiled manifest, I do lean in favor of these documentation/audit functions living under a annotations['meltano']. Note this is distinct from annotations['meltano.cloud'] which generally would be providing instruction to Meltano Cloud as an orchestrator.
3. Duplicate entries under env and config when plugin settings are injected as env vars
In the example here, tap-cloudwatch has
"config": {
    "aws_region_name": "us-west-2",
    "log_group_name": "API-Gateway-Execution-Logs_i32s35df22/prod",
    "query": "fields @timestamp, @message",
    "start_date": "${DEFAULT_START_DATE}"
},

and also these twice declared dupes, once as env vars under MELTANO_EXTRACT_* and once under TAP_CLOUDWATCH_*:
"env": {
    "MELTANO_EXTRACT_AWS_REGION_NAME": "us-west-2",
    "MELTANO_EXTRACT_LOG_GROUP_NAME": "API-Gateway-Execution-Logs_i32s35df22/prod",
    "MELTANO_EXTRACT_QUERY": "fields @timestamp, @message",
    "MELTANO_EXTRACT_START_DATE": "2022-12-08",
    "MELTANO_EXTRACT__LOAD_SCHEMA": "tap_cloudwatch",
    // ...
    "TAP_CLOUDWATCH_AWS_REGION_NAME": "us-west-2",
    "TAP_CLOUDWATCH_LOG_GROUP_NAME": "API-Gateway-Execution-Logs_i32s35df22/prod",
    "TAP_CLOUDWATCH_QUERY": "fields @timestamp, @message",
    "TAP_CLOUDWATCH_START_DATE": "2022-12-08",
    "TAP_CLOUDWATCH__LOAD_SCHEMA": "tap_cloudwatch",
    // ...
},

Most likely, this is actually just exposing an existing issue, which we shouldn't try to solve here. That is, we probably should not create new env vars for config that we're sending within a config file or via another pre-defined env var for the plugin.
Explicit is better than implicit, and we should move towards requiring plugin definition files to declare an env: key on settings that need to be passed via env var. This does not imply we won't auto-capture settings by env var name patterns - just that we decouple the "receiving" of user config with the "sending" of what the plugin actually needs.

@aaronsteers Thanks for the review! I've addressed point 2 in 8f3af30 by removing include_paths from the manifest. If a use-case for knowing what it is emerges, we can add it back using annotations as you describe. For now let's keep it simple.
Regarding point 3, I think that's the desired behaviour. The duplication is not elegant, but we do want all of those env vars when that plugin is being run.
As for the env vars autogenerated from config being present at all, that's tricky. Stuff like "start_date": "${DEFAULT_START_DATE}" is eagerly resolved to 2022-12-08, and then at run time if we use those env vars it won't respect $DEFAULT_START_DATE.
If we remove the config derived env vars then the manifest will have a more incomplete description of the env vars active within that context. Currently it only lacks info about the env vars from lower levels, such as .env, the shell env vars, or env vars from jobs and schedules. Even so, that may be the best course of action.

@aaronsteers I've created the following issue to address your first and third points:

https://github.com/meltano/meltano/issues/7306

🎉🎉🎉


Tried running it with q5_1 ggml files but it is interpreting them as q6_k

KeyNotFoundException: The given key 'Q6_K' was not present in the dictionary.
System.Collections.Generic.Dictionary2[TKey,TValue].get_Item (TKey key) (at <e40e5a8f982c4b618a930d29f9bd091c>:0) GGMLTraitTypeExtensions.GetTraits (GGMLType type) (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/GGMLTypes.cs:189) GGMLLoader.CalculateTensorSize (TensorMeta tensor) (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/GGMLLoader.cs:341) GGMLLoader.ReadTensorMetadata (System.IO.BinaryReader reader, GGMLFileVersion fileVersion) (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/GGMLLoader.cs:331) GGMLLoader.LoadMetadata (System.String path) (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/GGMLLoader.cs:45) GGMLLoader+<>c__DisplayClass6_0.<LoadModelImpl>b__0 () (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/GGMLLoader.cs:90) System.Threading.Tasks.Task1[TResult].InnerInvoke () (at :0)
System.Threading.Tasks.Task.Execute () (at :0)
--- End of stack trace from previous location where exception was thrown ---
GGMLLoader.LoadModelImpl () (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/GGMLLoader.cs:90)
Rethrow as AggregateException: One or more errors occurred. (The given key 'Q6_K' was not present in the dictionary.)
System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) (at :0)
System.Threading.Tasks.Task1[TResult].GetResultCore (System.Boolean waitCompletionNotification) (at <e40e5a8f982c4b618a930d29f9bd091c>:0) System.Threading.Tasks.Task1[TResult].get_Result () (at :0)
ModelLoaderBase.Update () (at D:/01_Unity/2023-09-29_PerroPastor-master/UnityPackage/Loaders/ModelLoaderBase.cs:40)

Thanks for writing up the issue.  I think this is related to the fact that ggml introduced a breaking change to the file format like, days after I wrote the ggml file loader.  Could you try syncing llama.cpp to dadbed9 and re-exporting the model as q5_1?  I think that should work.
If that doesn't work, then if you could link me to the model I can take a look at it.

When I get some time I definitely would like to update the loader to use their new format, since it seems like they took some effort this time to make the format extensible and a bit less fragile.

It happened 5 different models by TheBloke. https://huggingface.co/TheBloke
Im new at this so ("try syncing llama.cpp to dadbed9 and re-exporting the model") not sure how to do that

ok, no worries, i'll make updating the loader a priority, and try to get it done in the next week or two, and ping this thread when it's done :).   in the meantime, i will put up a llama7b model i've been using to my own huggingface that you can try if you're interested

The models you provided worked! this is so cool!!


Feature/fiat

This allows users to configure their FIAT currency and then passes that along to CoinMarketCap.
I have my portfolio in EUR, so watching this as EUR is a must-have for me. Hence this PR.
The test-coverage did not improve much, since I only added tests for this new feature. There were no tests around checking output so that is still not covered. Nor were there any tests for the telegram-bot, so that too is untested, still.
I did evaluate the output manually on the CLI. I did not manually test the Telegram-bot, since I don't have Telegram myself.

Coverage increased (+0.4%) to 85.33% when pulling 8867abb89cb86f62425c7652420c6eb68a69c6e6 on berkes:feature/fiat into 4d25d25da5e8b245b393bba56dba16df4acf6237 on bugaiov:master.

Coverage increased (+0.9%) to 85.854% when pulling 42e1667bfd5f1c602cbe66f2a151134074700544 on berkes:feature/fiat into 4d25d25da5e8b245b393bba56dba16df4acf6237 on bugaiov:master.

Coverage increased (+0.9%) to 85.922% when pulling e18484a78a6a380509f40dd1486c0bb164d0f1a4 on berkes:feature/fiat into 4d25d25da5e8b245b393bba56dba16df4acf6237 on bugaiov:master.

@berkes Cool! thanks! Great Job!
I'm using euro myself to buy/sell, but I was too lazy to convert it each time so it become easier for me to adapt to USD :-)


simplify and reduce custom html

remove network icons custom html
add github url custom html (simple version - thanks Austin)
simplify firmware reset code

@austinscreations can you have a look at this and test please? I know you won't like it, but this is much cleaner and easier for this web-dev noob to follow! ;)


Make it clear that customer should not use POA and AutomaticOSUpgrades

@christopherpcosta : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@christopherpcosta : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

@mamccrea: I'm sorry - only the author of this article, @sukanyamsft, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.


make sure script is not None when adding as an attachment in Python client

Coverage remained the same at 88.304% when pulling 5980768f28d7ecfd05add5c676049aa4a15c9e1e on irontable:develop into d9f44eda5f681ad8e95495ad4a42daa4520398ab on Netflix:develop.


Add missing license files for third party

Looking at some of third party and saw that we were just missing the license files for two of the directories.

LGTM :+1:

cc @erickhan

LGTM. thanks Patrick & Quinton!


Simplify navigation item reordering

Since we're storing navigation items in an ordered data structure, we can use the start index and new index from the Sortable plugin to do the reordering instead of indirectly from an "order" metadata property.

<3 nice work


Bulk PR

Hi Fabian + Martha-
I am sending the first PR for the changes I've made so far. Note- I am focusing my work + testing on the western interconnect wildcard and will move to eatern/ercot after more progress is made for the WECC model.
Main Changes:

Changing cluster shapes to quasi- US Balancing Authorities
Adding options for OSW shapes (BOEM, Weather Gov)
Removing pypsa-eur subworkflow
Refactoring

Kamran

Hi Fabian,
As per (#4)

Reorganized Snakemake Rule structures to split apart aspects of network creation (similar to pypsa-eur)
Integrated data from EIA Grid Monitor to pull historical load data into network (2015-present). Mapped to network via BA's
Integrating WECC ADS Load forecasts for 2030 and 2032

Best,
Kamran

Very nice @ktehranchi

Thanks, @FabianHofmann.
If it is okay with you, I will start working directly on the main pypsa-usa repository instead of my fork. I will continue with using PRs for merging from a development branch to the master


Feature upgrade request

The Tekton plugin is fantastic and it has all the features that helps a developer to check the status of pipelines and task runs, but one thing that can be improved is  show logs for taskrun, it redirects to some non-formatted text (Logs).
also, there is no back button so it's not a good user experience.
Please provide logs in the structured format as we see in the tekton dashboard.
please look into it. @rannox @n-kostadinov @jizusun

Thank you very much for the feedback @Meta-sploit! Will try to implement it the following days.

In stepLog.tsx
For the back button
Either page can be reloaded like this
<Button onClick={() => window.location.reload()}>Go back</Button>
or it can be rendered to the previous route
<Button onClick={() => navigate(-1)}>Go back</Button>
Box can be changed to something like this
  const text = props.text;
  const newText = text.split('\n').map(str => <p>{str}</p>);
  return (
    <>
      <Paper
        elevation={3}
        style={{
          padding: 16,
          margin: 'auto',
          maxWidth: 1000,
          backgroundColor: 'black',
          color: 'white',
        }}
      >
        <Typography variant="body1">{newText}</Typography>
      </Paper>
    </>
  );
}```

Also added a fullscreen mode for the logs dialog

It's great to see the improvements. I'm looking forward to utilizing this feature on my backstage instance once it's released under the new version. Thank you for all your hard work.

You are welcome, and thank you for your valuable feedback :)
Just released the new version, you can close the issue if everything is okay :)

Everything is operating smoothly, and I am closing the issue with this comment


Some members of my team are not getting messages - the actual messages or notifications.

@stefanmaco and @veemex
I'm unable to message 2 people on our team. I thought the issue was due to being on invisible status by accident but issue continues. It's like my messages just don't send.
I'm on the mac app

Issue resolved - somehow i was set to invisible
There should be a note that messages don't seem to send right when on invisible and no prompt comes up reminding the user that this is happening. I lost a week trying to figure out how come I coudln't contact people correctly

This happens only via the desktop app? Does accessing it via browser work normally?

It seems it suddenly started working again . I received 40+ notifications at once . And after that we started to receive notifications again ( IOS ) . Desktop app did not refresh correctly . There were times when i received the message after several hours , or had to open/close the app multiple times. In the last couple of weeks it started to work normally.


New API version script

Updating the Copy-apiversion script to update the Readme tags and improve the parameters.
Most relevant changes are:

If the user does not provide a base version to copy the files from, the latest preview version will be used by default.
Data-plane is no longer the default for this script, this parameter is now mandatory.
Added argument-completion for parameters
New tag version section added to the Readme file
The main tag version in the Readme is updated only if the new version is stable or if there are no stable versions for that provider with the last stable or preview version.

This script is intended to replace the "New OpenAPI specs" tool in OpenAPI Hub

@JonathanCrd Do you have any specs PR that was created for a new API version added using this script?

No, do you want me to create a PR in this repo for testing purposes?
I tested this script on my local machine but didn't push the commits.

@JonathanCrd how did you test this? We should try to figure out how to include some tests for this as there is a lot of assumptions with the regex's happening here.


AggregateProfile error when trying to view brightness profiles after generating PSF corrected images

ValueError                                Traceback (most recent call last)
 in
----> 1 srcs[0].view_brightness_profile('r500')
~/code/PycharmProjects/XGA/xga/sources/extended.py in view_brightness_profile(self, reg_type, central_coord, pix_step, min_snr, figsize, xscale, yscale, back_sub, lo_en, hi_en)
797                 self.update_products(psf_sb_profile)
798
--> 799             sb_profile += psf_sb_profile
800
801         draw_rads = {}
~/code/PycharmProjects/XGA/xga/products/base.py in add(self, other)
1994             raise TypeError("You may only add 1D Profiles, 1D Aggregate Profiles, or a list of 1D profiles"
1995                             " to this object.")
-> 1996         return BaseAggregateProfile1D(to_combine)
1997
1998
~/code/PycharmProjects/XGA/xga/products/base.py in init(self, profiles)
2033         bounds = [p.energy_bounds for p in profiles]
2034         if len(set(bounds)) != 1:
-> 2035             raise ValueError("All component profiles must have been generate from the same energy range,"
2036                              " otherwise they aren't directly comparable.")
2037
ValueError: All component profiles must have been generate from the same energy range, otherwise they aren't directly comparable.

This is happening after its been read back in from the disk, because I use a trick of sets to check if all the profiles have the same energy bound - I remember something very similar happened in the same check, reading back in the astropy quantities alters them somehow.


[BUG] no country color when all countries have the same value

When all the countries have the same value, the world map is blank
it seems that you need to have a "minimum" value with at least 1 country at first render
How to reproduce:
import { WorldMap } from "react-svg-worldmap";
import React, { Component } from 'react';

export default class App extends Component {
  constructor(props) {
    super(props);
    this.state = {
      data: []
    }
  }

  addCountry(code) {
    const { data } = this.state;
    this.setState({
      data: [
        ...data,
        { country: code, value: 1 }
      ]
    })
  }

  render() {
    return (
      <div className="App">
        <WorldMap
          color="red"
          title="Top 10 Populous Countries"
          value-suffix="people"
          size="lg"
          data={this.state.data}
          onClickFunction={(country) => { this.addCountry(country.countryCode) }}
        />
      </div>
    );
  }
}

now, click a country. it will be white, instead of red, this error will be displayed:
Warning: `NaN` is an invalid value for the `fillOpacity` css style property.
path
Region@http://localhost:3000/static/js/bundle.js:39998:7
g
svg
figure
WorldMap@http://localhost:3000/static/js/bundle.js:40921:7
div
App@http://localhost:3000/main.68d55a7b532f973d3b34.hot-update.js:27:5

to solve it, you can simply add a country like: {country: '', value: 0}

Hi! This has already been fixed in #109. Unfortunately, @yanivam hasn't got time to release a new version yet. @yanivam Could you take a look? Maybe eventually give me publish access?

@warenbe In the meantime, you can use the styleFunction API to override the default.

Hi
sure i can, and i did
that's more an issue when you discover the library and start to play with it than when using it "for real".

Heh, indeed 😄


Middleware or something to allow "magic" (e.g. undo, time travel)

Here goes one idea of how it could be done (not sure if it's good enough):

When creating a Fluce instance you can provide a middleware that will be used
to change default fluce behavior regarding update stores state. A middleware is
a function that returns another function:
const myMiddleware = (replaceState) => {
  // The returned function will be called when fluce want to update its state.
  // It is called with the state object that contains state of all stores,
  // (the same object that is available as `fluce.stores`).
  return (newState) => {
    // Now you're in charge, and can decide whether the state will be changed
    // and to what value. To set the new state you should call the
    // `replaceState` function. You can not call `replaceState` on a request
    // from fluce, and you can also call it at any time you want. For instance,
    // you can delay all changes like this:
    setTimeout(() => replaceState(newState), 1000);
  }
}

A no op middleware is looks like this (replaceState) => replaceState, it won't change the default behavior.
To set a middleware you need to pass it to createFluce() function:
const fluce = createFluce(myMiddleware);

This feature allows you to implement advanced stuff like "time travelling" or "undo" from
this prototype by @gaearon.

Perhaps we should also allow hooks on before action dispatch...

If you go that road (it might be fun!) the API might need to be a bit smarter. I haven't figured it out myself yet so for now I'm just dumping everything I need right into my dispatcher, but at the very least middleware must be powerful enough to express the concept of transactions.
Here's a working implementation of transactions (built into dispatcher—but I want to somehow turn this into middleware later on). Call dispatcher.transact() at any moment, and the actions occurring after it can be committed by calling commit(), or cancelled by calling rollback() on the returned object. This is not the most interesting part. If a hot reload occurs while a transaction is active, the staged actions will be replayed on top of the last committed state. Sorry if this doesn't make sense—as redux grows more mature, I'll make some demos. But for now you can

npm start
Click the button a few times, see the counter increment (e.g. to 3)
Change store logic to do +10 on increment action
Click  the button a few times, see the counter increment with +10 from 3 (e.g. to 33)
Now comes the interesting part! Save a reference to dispatcher somewhere in a global object and call tx = dispatcher.transact() from DevTools console
Click the button a few times, see the counter increment (e.g. two times—from 33 to 53)
Change the store logic back to +1 on increment
This time—because the transaction is active!—it will reapply your actions with the new code, and it will "go down" automatically to 35

Transactions let you replay a few last actions on each hot reload so that you can fine-tune your Store code until the actions are handled correctly. I'm positive that any middleware system needs to have all the necessary hooks to make this possible.

Hm, I see how transactions work, but don't quite understand the use case. Where one will call dispatcher.transact() and transaction.commit() (in stores, action creators, app code?), and what kind of actions should be done in a transaction?

It is meant to be done from a devtool. Imagine a devtool (like Chrome DevTools extension) that lets you go back to arbitrary state, or to mark a certain state as "committed" and have hot reload only re-execute a few actions you're currently debugging.

Oh, I see, if we do something like this http://www.youtube.com/watch?v=Fo86aiBoomE Yes, transactions could be useful. Need to think about it.

Have an idea, the API may look like this:
const fluce = createFluce(({dispatch, replaceState, reducer}) => {
  // In middleware you're given an object with functions,
  // and must return an object with same shape. If you are not modifying
  // a function, put the original one to the output object.
  //
  // This middleware doesn't do anything, it only shows interface of the object.

  return {

    // This is called when someone call `fluce.dispathc(type, payload)`
    // But you can also call given `dispatch` function whenever you want.
    dispatch({type, payload}, curState) {
      dispatch({type, payload}, curState)
    },

    // This is called after `dispatch` and replaces `fluce.stores`
    // object with a new one and notifies subscribers of changes.
    // Again you can call given `replaceState` whenever you want.
    replaceState(newState) {
      replaceState(newState)
    },

    // This is used to apply an action to all stores to get new state.
    // It is a pure function, you can use it to modify state object when needed.
    // You can also change its behavior, but make sure it stays a pure function.
    reducer(curState, {type, payload}) {
      return reducer(curState, {type, payload})
    }

  }
})

This is how transactions could be implemented using it (totally untested):

function createTransactMiddleware() {
  let _replaceState
  let _reducer

  let inTransaction = false
  let capturedState
  let capturedActions

  return {
    transact() {
      if (inTransaction) {
        throw new Error('can\'t nest transactions')
      }

      inTransacrion = true
      capturedActions = []

      return {
        commit() {
          inTransaction = false
          capturedState = undefined
          capturedActions = undefined
        },
        cancel() {
          if (capturedState) {
            _replaceState(capturedState)
          }
          inTransaction = false
          capturedState = undefined
          capturedActions = undefined
        },
        replay() {
          if (capturedState) {
            _replaceState(capturedActions.reduce(_reducer, capturedState))
          }
        }
      }

    },
    middleware({dispatch, replaceState, reducer}) {
      _replaceState = replaceState
      _reducer = reducer

      return {
        dispatch(action, curState) {
          if (inTransaction) {
            if (!capturedState) {
              capturedState = curState
            }
            capturedActions.push(action)
          }
          dispatch(action, curState)
        },
        replaceState,
        reducer
      }
    }
  }

}

const transactionMw = createTransactMiddleware()
const fluce = createFluce(transactionMw.middleware)

// ...

let tr = transactionMw.transact()

// ...

tr.repaly()
tr.cancel()

I'm going to play with it more after I finished with React bindings (<Fluce />, and connectStores). Also going to implement fluce.optimisticDispatch as a middleware.

This sounds sensible!

https://github.com/gaearon/redux/issues/6#issuecomment-108634552


Where is Tanzanian shilling currency?

I cannot find TZS Tanzanian shilling) in .NET 4.7.2.
https://docs.microsoft.com/en-us/dotnet/api/system.globalization.regioninfo.isocurrencysymbol?view=netframework-4.7.2
https://en.wikipedia.org/wiki/Tanzanian_shilling
It is available in UWP but not in .NET 4.7.2.
https://docs.microsoft.com/en-us/uwp/api/windows.globalization.currencyidentifiers.tzs

Tanzanian shilling is already supported in the .NET framework. You just need to create a RegionInfo object from the Tanzanian culture name and then use currencies properties from the RegionInfo object to get what you want.
Here is some code showing all Tanzanian cultures and printing the currency of each of them:
            foreach (CultureInfo ci in CultureInfo.GetCultures(CultureTypes.SpecificCultures))
            {
                if (ci.DisplayName.Contains("Tanz"))
                {
                    RegionInfo ri = new RegionInfo(ci.Name);
                    Console.WriteLine($"{ci.Name,-10} {ci.DisplayName,-30} {ci.EnglishName,-30} {ri.CurrencyEnglishName,-20} {ri.CurrencySymbol,-20} {ri.ISOCurrencySymbol,-20} {ri.CurrencyNativeName,-20}");
                }
            }

running this code will produce the following output:
asa-TZ     Asu (Tanzania)                 Asu (Tanzania)                 Tanzanian Shilling   TSh                  TZS                  shilingi ya Tandhania
bez-TZ     Bena (Tanzania)                Bena (Tanzania)                Tanzanian Shilling   TSh                  TZS                  Shilingi ya Hutanzania
en-TZ      English (Tanzania)             English (Tanzania)             Tanzanian Shilling   TSh                  TZS                  Tanzanian Shilling
jmc-TZ     Machame (Tanzania)             Machame (Tanzania)             Tanzanian Shilling   TSh                  TZS                  Shilingi ya Tanzania
kde-TZ     Makonde (Tanzania)             Makonde (Tanzania)             Tanzanian Shilling   TSh                  TZS                  Shilingi ya Tanzania
ksb-TZ     Shambala (Tanzania)            Shambala (Tanzania)            Tanzanian Shilling   TSh                  TZS                  shilingi ya Tanzania
lag-TZ     Langi (Tanzania)               Langi (Tanzania)               Tanzanian Shilling   TSh                  TZS                  Shilíingi ya Taansanía
mas-TZ     Masai (Tanzania)               Masai (Tanzania)               Tanzanian Shilling   TSh                  TZS                  Iropiyianí e Tanzania
rof-TZ     Rombo (Tanzania)               Rombo (Tanzania)               Tanzanian Shilling   TSh                  TZS                  heleri sa Tanzania
rwk-TZ     Rwa (Tanzania)                 Rwa (Tanzania)                 Tanzanian Shilling   TSh                  TZS                  Shilingi ya Tanzania
sbp-TZ     Sangu (Tanzania)               Sangu (Tanzania)               Tanzanian Shilling   TSh                  TZS                  Ihela ya Tansaniya
sw-TZ      Kiswahili (Tanzania)           Kiswahili (Tanzania)           Tanzanian Shilling   TSh                  TZS                  Shilingi ya Tanzania
vun-TZ     Vunjo (Tanzania)               Vunjo (Tanzania)               Tanzanian Shilling   TSh                  TZS                  Shilingi ya Tanzania

Also, note that the UWP link is not really an .NET API but it is Windows Runtime API provided by Windows.
I am closing this issue but feel free to send any question if you have any.


Heading and context are mismatching

""KSAUDIOMODULE_PROPERTY structure (ksmedia.h)
Article
04/01/2021
2 minutes to read
The KSAUDIOMODULE_DESCRIPTOR structure describes the static, external properties of the audio modules.
Syntax
C++
Copy
typedef struct _KSAUDIOMODULE_PROPERTY {
KSPROPERTY Property;
GUID       ClassId;
ULONG      InstanceId;
} KSAUDIOMODULE_PROPERTY, *PKSAUDIOMODULE_PROPERTY;"""
the one highlighted text should be "PKSAUDIOMODULE_PROPERTY" or this is wantedly shown likewise if so why ??
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ae50d732-09e8-cc84-68b1-b0d84aabd239
Version Independent ID: db5f3028-e150-b8ac-09f2-e47cd1c53479
Content: _KSAUDIOMODULE_PROPERTY (ksmedia.h) - Windows drivers
Content Source: wdk-ddi-src/content/ksmedia/ns-ksmedia-_ksaudiomodule_property.md
Product: windows-hardware
Technology: wdk-api-reference
GitHub Login: @barrygolden
Microsoft Alias: bagold

Thank you Parihar, you are exactly right!  The description had a typo, for which I just pushed a fix.  The fix should appear on the live site within the next hour or so.  Thanks again for letting us know.


invalid terraform code Fails to compile

you have duplicate resource names in outline for file
variables.tf
current bad  _____________________________________
variable "resource_group_name" {
description = "Name of the resource group in which the Packer image  will be created"
default     = "myPackerImages"
}
variable "resource_group_name" {
description = "Name of the resource group in which the resources will be created"
default     = "myResourceGroup"
}
above fails during terraform plan ...   instead it should be __________________________________________
variable "managed_image_resource_group_name" {
description = "Name of the resource group in which the Packer image  will be created"
default     = "myPackerImages"
}
variable "resource_group_name" {
description = "Name of the resource group in which the resources will be created"
default     = "myResourceGroup"
}
Please fix this bug

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: b4371abd-f9cc-1ee9-1ed3-a5f0488740ce
Version Independent ID: 14c6cf0f-ec32-15ed-cb38-8eb5860d858e
Content: Create an Azure virtual machine scale set from a Packer custom image by using Terraform
Content Source: articles/terraform/create-vm-scaleset-network-disks-using-packer-hcl.md
Service: terraform
GitHub Login: @TomArcherMsft
Microsoft Alias: tarcher

@sekhemrekhutawysobekhotep Thanks for the feedback. I've removed the dupe from the variables.tf file. Once the product group merges the fix, it will go live (sometime next week).


Added rounding to shadow container and removed -1px.

PR Checklist
Check these before submitting your pull request:

[x] Visual and behavioral components are separated
[x] Exported components are documented with examples
[x] Props have JSDoc comments
[x] All relevant visual sub-components can be overridden via props
[x] Any extra props are spread into the outermost rendered element

Breaking Changes
Removed styling that points out its used for browser quirks. While working on these changes I didn't notice anything odd around the shadows rendering
Changes to Existing Components
Added border rounding and extended heading and separating lines to reach the end of the table.

The reason that hack was there was so that the right internal shadow would show up. Can you verify that the right internal shadow is present when there is more table content beyond the right border?

Table shadow still shows when there is content outside of the visible area

Looks good, but which browser are you using? Have you tried others?

Yea, just tested edge, IE and firefox (mac and windows) and all look good.


Questions: cloudMap Namespace Service deletion

external-dns policy is set to sync.
After deleting kubernetes service, external-dns de-register the instance from the cloudMap service.
My question is, why wouldn't it just delete the service inside cloudMap Namespace too? If there other flags that I need to set to delete the service inside a cloudMap Namespace?

external-dns policy is set to sync.
After deleting kubernetes service, external-dns de-register the instance from the cloudMap service.
My question is, why wouldn't it just delete the service inside cloudMap Namespace too? If there other flags that I need to set to delete the service inside a cloudMap Namespace?

@tchen-mdsol Have you find any solution for this, facing the same issue.


occludingBoundaryVertices.m file

Thanks for your code. I want to ask about In occludingBoundaryVertices.m file, the last statement
occludingVertices = intersect(occludingVertices,visiblevertices(FV, eye(3)));                                                                           why the transform maxtrix is eye(3) rather than R which was caculated in program above.

The reason is because on line 12, the rotation matrix (if supplied) has already been applied to the vertices. So, the call to visiblevertices doesn't need to pass the rotation matrix to be applied again.


Update Accordion styles and behavior

🙋 Feature Request
Accordion currently doesn't have up to date styles and behavior. Consult with Spectrum to make sure what is expected:
Styles:

Hover and focus states are different, no active state

Behavior:

Ability to have only a single item open at a time
Ability to enforce that an item is always open

🔦 Context

💻 Examples

🧢 Your Company/Team

🎁 Tracking Ticket (optional)

This will be tackled at a later date


[Bug] init react-table occur error

Version
develop
Link to Minimal Reproduction
none
Steps to Reproduce

Current Behavior
1
Expected Behavior
1
Environment
- OS:
- Browser:
- Framework:

Any additional comments?
No response

0.20.4 发版修复


create material-ui repo

Hi,
would you be so kind and create material-ui repo?
I plan to start working on such version for some time but I dont want to create it in separate namespace.
Or should I just create my own repo and notify you when it will be ready enough to initial cloning to meter-useraccounts namespace ?

@ShockiTV useraccounts:materialize works great, are you looking to create another material UI theme? You can also use the bootstrap useraccounts package with a material bootstrap theme.

@ShockiTV if it's for a new styled version different from useraccounts:materialize feel free to create your own repo (possibly starting from a fork of useraccounts:unstyled?) and work on it locally.
Once it'll be ready, I'll be happy to publish it under the useraccounts namespace after the transfer of ownership under meteor-useraccounts
Let we know! :)

I'd love a material-ui based useraccounts package too.... I prefer to use material-ui in my app (instead of materialize), and even though the appearance is similar, materialize is subtley different, so the buttons, font etc change between login and the rest of the app.


Feeds don't load at all

I'd like to try out your app (v1.9) but I can't seem to get it to work. Started it, waited for some time but the indicator just keeps on spinning and no feeds in the list would load. App's traffic log on the phone shows no traffic at all (mobile and wifi) and the app doesn't ask for any permissions either, which it should do on Android 6(+), at least for storage (I suppose). Using a (Huawei) Honor 6, Android 6. Help is appreciated! 😃

That sounds very strange to me. Maybe you have a special permission system or app. My app does not need any permission, which are dangerous and where it has to ask for. I only use normal permissions

If an app declares in its manifest that it needs a normal permission, the system automatically grants the app that permission at install time. The system does not prompt the user to grant normal permissions, and users cannot revoke these permissions.

To avoid from any Andriod 6+ permission asking problems on run time, I normaly set the target system to SDK 22 (Android 5.1.1). Setting the target to 22 ask for permissions only during the installation process. But actually I set it to 23 because there should not be any permission trouble. In the next version (2.0) I will change this.
Sometimes it is a problem to install the App on a real SD card. The app does not like it. I use a boot receiver to start feed requests after a reboot - this only works, if the App is installed internally. Normaly the installation process should force this.
Getting the first feeds takes some seconds. It is possible, that you close the app to fast and you have a special developer setting active to "really close the App and remove all their services".
It is hard to debug this problem. I feel sorry for that :-/

Because google play forbid downgrade the SDK from 23 to 22, you may try the v1.95 version from the project website APK file. Please uninstall the old version first to avoid from trouble.
Hint to target SDK in google play

Still doesn't load. Other network dependent Apps, including e.g. Flym or Aggregator load feeds almost instantly

Please download and build my app in debug mode and post the content of the debugging log. Did you read my comment above? Did you install the app on SD-Card? Does the widget load any content? Did you grant AnotherRSS access to the network and to display notifications?
If your device manufacture needs a non-standard file access to hold the app database, you are in trouble :-( Flym and Aggregator needs these permission for import/export - maybe this could be the main problem.

I reopen the issue and I try to compare my code with flym and Aggregator to fix the problem.
The first, big differences:

both Apps need file-access (AnotherRSS does not)
flym uses okhttp3.OkHttpClient: AnotherRSS uses build in java functions
flym store the images in a cache directory: AnotherRSS stores a thumbnail direct to the database
Fail: I did not find open source code of the Aggregator App and no private policy or impress
flym uses installLocation="auto": AnotherRSS uses "internalOnly" to force a RSS refresh on device boot

I think, I will build a debugging version of AnotherRSS - but I am bussy: I have an important exam in a couple of weeks :-(
The widget questions (above) is very important: the widget has its own refresh and request code and works a bit different.

A very very early Version of AnotherRSS is "LowTaz" with only one feed source:
LowTaz apk
Does LowTaz work on your device?
A very old and different/strange version of AnotherRSS is "LowFeed". It has the same Image/Database and permissions like AnotherRSS, but works I bit different:
LowFeed apk
Does LowFeed work on your device?
best regards,
Jochen

Hi Jochen, yes AnotherRSS is installed on internal mem as the code forces ito be installed there, no option to moved it to external mem either. The widget doesn't load any feeds either.
I checked both the apks you linked in your last comment and they load all the entries without a problem, LowTaz and LowFeed apprently work well.
Take your time, it's not that important, exams are though. I wish you the best of luck!


Icons are missing chevron_up_down

I noticed the chevron-up-down icon is missing from the set. I believe it's because it was accidentally missing from the available svg's in the heroicons repo's optimized folder.
I noticed the optimized folder has change structure and so I think the heroicons mix task will need to be updated to handle the change.

Hi @ynnadrules thanks yeah Heroicons recently updated to v2 while we are still on v1.
There is a plan to remove our mix task and rely on another Heroicons hex library that has updated to v2 (https://github.com/petalframework/petal_components/issues/77).


deprecate

Hi,
If I use the open-api deprecated attribute in a method or in a model, I want to have a @Deprecated annotation in the generated code.

depends on openapi-processor/openapi-processor-core#10

openapi-processor/openapi-processor-core#10 fixed with ticket


Support tsc --build --noEmit

Suggestion

🔍 Search Terms

tsc --build --noEmit
build noEmit
List of keywords you searched for before creating this issue. Write them down here so that others can find this suggestion more easily and help provide feedback.
✅ Viability Checklist

My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion

Allow tsc --build --noEmit, the relevant types can be emitted to the tsbuildinfo for incremental building. This is very useful to allow type checking only for projects using project references and working with a bundler. e.g. A create-vue project, which currently has to resort to hackery such as vue-tsc --noEmit -p tsconfig.vitest.json --composite false which is not as accurate as type checking each project individually.
📃 Motivating Example

tsc --build --noEmit is now supported, allowing to only run type checking for projects using project references that don't also build types using tsc.
💻 Use Cases

Type checking only, for projects using other build tools than tsc but still using project references for each of configuration.

I encountered this problem as well recently. You can specify noEmit: true in the tsconfig.js, but you can't pass it via the command-line

I got here because I am attempting to run tsc --noEmit for type checking since my build tooling (vite) explicitly transpiles only and does not validate type consistency.
I find that tsc --noEmit does what I want but emits no progress while it's running. when I tried to add --verbose it says that the progress update is only supported with --build but that --build and --noEmit are not permitted together
tsc --verbose --noemit
error TS5093: Compiler option '--verbose' may only be used with '--build'.

tsc --build --verbose --noemit
error TS5094: Compiler option '--noemit' may not be used with '--build'.

We are handling a sizable project where approximately 30% of the time is spent on transform and emit tasks. This duration could be decreased if this functionality is incorporated.
Projects in scope:                        100
Projects built:                            95
Aggregate Files:                       415465
Aggregate Lines of Library:           3727607
Aggregate Lines of Definitions:      25912137
Aggregate Lines of TypeScript:        2453924
Aggregate Lines of JavaScript:            844
Aggregate Lines of JSON:               111484
Aggregate Lines of Other:                   0
Aggregate Identifiers:               37850747
Aggregate Symbols:                   33949727
Aggregate Types:                      3646787
Aggregate Instantiations:            14245043
Aggregate Memory used:               3977596K
Aggregate Assignability cache size:   1965511
Aggregate Identity cache size:          96331
Aggregate Subtype cache size:          121867
Aggregate Strict subtype cache size:   116815
Aggregate I/O Read time:                1.57s
Aggregate Parse time:                  16.50s
Aggregate ResolveModule time:          14.52s
Aggregate ResolveTypeReference time:    0.51s
Aggregate ResolveLibrary time:          0.24s
Aggregate Program time:                56.92s
Aggregate Bind time:                    8.03s
Aggregate Check time:                 227.75s
Aggregate transformTime time:          33.99s
Aggregate commentTime time:             2.28s
Aggregate Source Map time:              2.16s
Aggregate printTime time:              59.12s
Aggregate Emit time:                   59.28s
Aggregate I/O Write time:               4.87s
Config file parsing time:               0.64s
Up-to-date check time:                  0.00s
Build time:                           371.55s

I am equally finding a need for this option. I am working on a large codebase with project references, and I am having issues related to this. We have had refactors that cause typechecking issues due to old dist files being left behind.
This could be solved with --build --noEmit, since we only need the typechecking signal from tsc. The files are otherwise all bundled via webpack

We're looking for something similar but for type checking a single composite project without generating any output. Currently a composite project can't turn off declarations.


v1.14 Backports 2023-11-15

[ ] #28557 (@dylandreimerink)

:warning: Conflicts in the lbipam unit test, as it did not have any leak check before in v1.14

[x] #28959 (@julianwiedmann)
[ ] #28264 (@aspsk)
[ ] #29007 (@mhofstetter)

:warning: Conflicts in the following files
operator/pkg/gateway-api/tlsroute.go Conflicts due to Gateway resoruce version (v1 vs v1beta1)
operator/pkg/gateway-api/httproute.go Conflicts due to Gateway resoruce version (v1 vs v1beta1)
operator/pkg/gateway-api/grpcroute.go Conflicts because the file does not exist in v1.14

[ ] #27062 (@gentoo-root)

:warning: Conflicts due to "cilium-dbg" and "pkg/time" rename

[ ] #29077 (@meyskens)

PRs skipped due to conflicts:

#28896 (@mhofstetter)
#29115 (@sayboras)

Once this PR is merged, a GitHub action will update the labels of these PRs:
 28557 28959 28264 29007 27062 29077

Seems that the fixup got lost? It's still probes.Jiffies() there.

This is v1.14, which does have probes.Jiffies!

/test-backport-1.14

ci-ginkgo hit a vm provisioning error. Restarting.


Possible example?

Would it be possible to publish an example case such that we could see how to run this pipeline practically? For example just a folder containing a 3D data set, where when you run the pipeline the segmentation maps are produced? Would help a lot with user adoption.

I agree that this would help, but for now we don't have any datasets that we could share. We will think about generating a volunteer or phantom dataset.


Extremely high page size of links shared

Simple typical links like this when opened are too heavy. Close to 15MB.

Link to webpage-size-checker-tool

This seems to have been resolved after the last deployment.

Link to test again

This seems to have been resolved after the last deployment.

Link to test again


First pass at making snakemake workflow for innovation model

This PR implements a Snakemake workflow for provisioning sequence counts with similar methods used in forecasts-ncov. This is my first time working with Snakemake, so any suggestions, comments, or questions are appreciated.
Major changes include:

[x] Specifying several analysis periods with different pivot variants which growth advantages are estimated relative to. (See config.yaml for specification here)
[x] Provisioning data sets for each analysis period (See workflow/snakemake_rules/prepare_data.smk)
[x] Running the innovation model with the specified period for all qualifying locations (See scripts/run-innovation-model)

This to be accomplished still:

[ ] Checking Python implementation of mlr-fitness/data/pango-relationships.nb for correctness
[ ] Testing of workflow in general.
[ ] Implementing phenotype prediction using DMS data / snakemake.

Note: I've borrowed the  files scripts/prepare-data.py and scripts/collapse-lineages directly from forecasts-ncov. Let me know if there's a better way of doing this.

I've tried to convert compare_natural notebook to a Python script scripts/compute-phenotype which can be used to generate phenotype files in specified in prepare_data.smk (corresponding rule is compute_phenotype).
There's a couple of remaining tasks for this section:

[ ] Ensuring pairs for predictor differences are generated with the parent-variant relationships in pango_variant_relationships
[ ] Figure out best way of specifying which phenotypes to generate. I imagine a section in config.yaml would be ideal?
[ ] Processing phenotypes to a single (or collection of) predictors files.
[ ] Testing in general

Thanks so much for diving in here @marlinfiggins. I'm sorry that I didn't notice this PR before you pointed it out to me last month. I just tried working from rewrite and I managed to fix some initial Snakemake bugs that were throwing errors. However, I now encounter
nextstrain build --cpus 1 . data/gisaid/pango_lineages/global/xbb15/prepared_cases.tsv
Building DAG of jobs...
MissingInputException in rule prepare_clade_data in file /nextstrain/build/workflow/snakemake_rules/prepare_data.smk, line 43:
Missing input files for rule prepare_clade_data:
    output: data/gisaid/pango_lineages/global/xbb15/prepared_cases.tsv, data/gisaid/pango_lineages/global/xbb15/prepared_seq_counts.tsv
    wildcards: data_provenance=gisaid, variant_classification=pango_lineages, geo_resolution=global, analysis_period=xbb15
    affected files:
        data/cases/global.tsv.gz
        data/gisaid/pango_lineages/global.tsv.gz

I think that you're assuming that local files like data/gisaid/pango_lineages/global.tsv.gz exist? Can you please add to the README.md what steps need to be taken to provision data before running Snakemake?
I just did almost exactly this over here: https://github.com/blab/fitness-dynamics?tab=readme-ov-file#provision-metadata-locally.
I can continue review once I know how to provision local data.

Also, separate question: can we just drop data/{data_provenance}/{variant_classification}/{geo_resolution}/{analysis_period}/prepared_cases.tsv? I don't think cases feed into any of the MLR analyses.

Hey @marlinfiggins. I was able to provision data locally and fix some errors to get me to
data/gisaid/pango_lineages/global/xbb15/collapsed_seq_counts.tsv
data/gisaid/pango_lineages/global/xbb15/pango_variant_relationships.tsv

However, I can't figure out how to even begin with phenotypes. From prepare_data.smk I see I should be able to ask for predictors/{analysis_period}/{pheno}_clade.csv. I tried this with nextstrain build . -j 1 predictors/xbb15/EVEscape_clade.csv and got the error:
InputFunctionException in rule compute_phenotypes in file /nextstrain/build/workflow/snakemake_rules/prepare_data.smk, line 163:
Error:
  AttributeError: 'Wildcards' object has no attribute 'phenotype'
Wildcards:
  analysis_period=xbb15
  pheno=EVEscape
Traceback:
  File "/nextstrain/build/workflow/snakemake_rules/prepare_data.smk", line 165, in <lambda>

I'm pretty sure this was an issue with line 165 of:
input_data = lambda wildcards: phenos_compare_natural.get(wildcards.phenotype),

I just updated this to
input_data = lambda wildcards: phenos_compare_natural.get(wildcards.pheno),

(continued...)
However now calling nextstrain build . -j 1 predictors/xbb15/EVEscape_clade.csv is giving the error
WorkflowError in rule compute_phenotypes in file /nextstrain/build/workflow/snakemake_rules/prepare_data.smk, line 163:
Function did not return str or list of str.

Can you please take a look at this? Please add instructions in README.md for how to proceed with provisioning local phenotypes.

This can be resolved via https://github.com/blab/evofr/pull/44.
@marlinfiggins: I'd suggest doing the following:

Merge https://github.com/blab/evofr/pull/44 and start the process of getting the Nextstrain docker image updated to the latest evofr
Update ncov-escape README.md with current installation instructions. This will be nextstrain update + running nextstrain shell and updating evofr to the latest version ahead of things cascading down to the default Docker image. The readme should have installation instructions in any case.

@marlinfiggins ---
After the update to the Nextstrain docker image, I'm now longer getting the above jax error. However, if run attempt to run the repo in it's current state without touching config.yaml I get the following error:
MissingInputException in rule process_metadata in file /nextstrain/build/workflow/snakemake_rules/run_models_over_period.smk, line 2:
Missing input files for rule process_metadata:
    output: data/xbb15_windowed/sequence_counts_by_submission.tsv
    wildcards: analysis_period=xbb15_windowed
    affected files:
        data/gisaid_metadata.tsv.gz

The config looks like it should just use what's in analysis_period.xbb15. However, if I take config.yaml and delete other entries in analysis_period and keep only xbb15 I get different behavior with a different error after it runs through a number of steps:
rule download_phenotypes:
    output: data/xbb15/phenotypes/mutation_phenotypes.csv, data/xbb15/phenotypes/mutation_phenotypes_randomized.csv, data/xbb15/phenotypes/lineage_phenotypes.csv, data/xbb15/phenotypes/lineage_phenotypes_randomized.csv
    jobid: 7
    reason: Missing output files: data/xbb15/phenotypes/lineage_phenotypes.csv
    wildcards: analysis_period=xbb15
    resources: tmpdir=/tmp

        python ./scripts/provision-phenotypes.py             --config-path config.yaml             --base-url https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main             --output-path data/xbb15/phenotypes

No `output` key in config. Falling back to default files.
Downloading https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/mutation_phenotypes.csv...
Saved to data/xbb15/phenotypes/mutation_phenotypes.csv
Downloading https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/mutation_phenotypes_randomized.csv...
Saved to data/xbb15/phenotypes/mutation_phenotypes_randomized.csv
Downloading https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/lineage_phenotypes.csv...
Failed to download https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/lineage_phenotypes.csv: 404 Client Error: Not Found for url: https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/lineage_phenotypes.csv
Downloading https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/lineage_phenotypes_randomized.csv...
Failed to download https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/lineage_phenotypes_randomized.csv: 404 Client Error: Not Found for url: https://raw.githubusercontent.com/jbloomlab/SARS2-spike-predictor-phenos/refs/heads/main/results/lineage_phenotypes_randomized.csv

Marlin, this doesn't have to be a correct / complete analysis, but I really would like the default analysis and what's specified in the readme to at least run. Can you take another look at this? Please try working from a fresh repo to make sure things at least run.
Noting here that if I swap xbb15 to the provided jn1 analysis period, the workflow completes successfully. I'd be 100% of okay merging this PR if you can just reduce scope to something like this and then we'll get phenotypic predictors working in a subsequent PR.

Sorry about this, I'm made some changes.
The bug you were running into was with running the regression prior model. It was attempting to fetch the phenotypes from the wrong file.
The issue here was that I had to switch the code to use the old version of the predictors clade_phenotypes.csvpresent on themainbranch of [SARS2-spike-predictor-phenos](https://github.com/jbloomlab/SARS2-spike-predictor-phenos). There were several differences in that version of the phenotypes and the one that lives on the [lineage phenotypes PR](https://github.com/jbloomlab/SARS2-spike-predictor-phenos/pull/1) which islineage_phenotypes.csv`.
Previously, I just grabbed this by hand from the main branch and renamed the relevant columns to be consistent since I expected this PR to be merged by the time we tested this, but I'll just revert these changes once this is properly merged.

I've also removed the window analysis for now until I fix the location of the gisaid_metadata file for the windowed analysis.

I've also added the window analysis back and fixed the instructions for provisioning the metadata.

@marlinfiggins --- Thanks so much for humoring me with the detailed instructions. I was able to run this workflow for different analysis periods and everything worked as expected. I like the set up for data/{analysis_period}/ for sequence counts and variant relationships and the corresponding results/{analysis_period}/ for model outputs.


updates including verified id settings config

@ajburnle

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

NOTE: the validation comment has 2 warnings about broken links. these will need to be addressed before we can merge the PR.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Changes present in another PR.

#please-close


stencil needs to be manually enabled starting from threejs 0.163.0 for Mask to work

Hey, I noticed the Mask component suddenly stopped working for me so I did some digging. Turns out the stencil buffer, starting from Three.js 0.163.0, is now disabled by default, breaking the Mask component.
To re-enable it, the appropriate renderer parameter needs to be set to true like so:
<Canvas rendererParameters={{ stencil: true }}>
  <Mask id={1}>
...

Creating this issue for two reasons:

Hope that others having the same issue come across it.
To find an approach how to deal with it in Threlte.

Personally I see two possible solutions.

Add a hint in the docs that tells users to make sure that the stencil buffer is enabled in the renderer.
The Mask component automatically taking care of enabling it, e.g. in the onMount method.

Not sure if there could be some unwanted side effects from enabling the stencil buffer when going with approach 2 though.
Let me know what you think! :slightly_smiling_face:
References:

Three.js 0.163.0 Release Notes
PR setting stencil to false by default

Thank you for the update! I'm happy to jump into option number 1. We could also add some warnings to the component ☺


CI: Consolidate and blanket-ignore kernel definitions on non-glibc

Skip definitions from the kernel on non-glibc Linux targets.
They're libc-independent, so we only need to check them on one
libc. We don't want to break CI if musl or another libc doesn't
have the definitions yet. (We do still want to check them on
every glibc target, though, as some of them can vary by
architecture.)

r? @Amanieu
(rust-highfive has picked a reviewer for you, use r? to override)

This also experimentally removes P_PIDFD from the ignores on glibc. Let's see if that builds now; if not, I can re-add it.

FYI: https://www.githubstatus.com/incidents/s654n76c1bwr
Let's wait until it's resolved.

Looks reasonable to me.

This also experimentally removes P_PIDFD from the ignores on glibc. Let's see if that builds now; if not, I can re-add it.

Let's check it, @bors r+

:pushpin: Commit db07dbc578827e23d292edad08d347b77a17ee1c has been approved by JohnTitor

:hourglass: Testing commit db07dbc578827e23d292edad08d347b77a17ee1c with merge ce7783a937193a4ec9a4b44a1fa6302a7bc24bf9...

:broken_heart: Test failed - checks-actions

Actually, failed on mips-unknown-linux-gnu.

Fixed.
@bors r=JohnTitor

:pushpin: Commit 0a9aec7bafafde798d9655f5be15b08c269daff5 has been approved by JohnTitor

:hourglass: Testing commit 0a9aec7bafafde798d9655f5be15b08c269daff5 with merge 56c85494a82021c0831b2330e4c053dc39c3510f...

:sunny: Test successful - checks-actions, checks-cirrus-freebsd-11, checks-cirrus-freebsd-12, checks-cirrus-freebsd-13
Approved by: JohnTitor
Pushing 56c85494a82021c0831b2330e4c053dc39c3510f to master...


Prevent deadlock when Windows Forms are loaded in the PowerShell process

PR Summary
Must use ConfigureAwait(false) on any awaited methods to prevent use of windows forms in the PowerShell process
from deadlocking the process. This was the cause of the CI failure - test was added specifically for this issue that causes the CI test to deadlock and time out the build stage.

@CarloToso - hopefully this patch fixes the build failures. I wasn't aware this was an issue but found a patch by seeminglyscience that must have been found post-release of pwsh 7.4. The test included in that patch is what was failing/stalling the build process.


TZif parser

Fully parse TZif, to support #
Useful links:

https://www.ietf.org/rfc/rfc9636.pdf

Try to use the TZDIR env variable


”Buznea”, ”Dădești”, ”Gănești” are listed in the drop-down menu

Description:
When user writes ”Buznea”, ”Dădești”, ”Gănești” the locations appear in the drop-down menu next to the comune and the county which is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Buznea"

Expected results
The location appears in the drop-down menu as "Sat Buznea, IAȘI (Ion Neculce)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Dădești"

Expected results
The location appears in the drop-down menu as "Sat Dădești, IAȘI (Ion Neculce)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 5
Write in the search bar "Gănești"

Expected results
The location appears in the drop-down menu as "Sat Gănești, IAȘI (Ion Neculce)".


use LAN_ENGLISH for windows api FormatMessage

#5338

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

This is okay to test.

@ctiller, I'm actually fine with that change, given that the whole of gRPC is currently English only, without i18n support.

Can one of the admins verify this patch?

My mistake, I should not execute git push -f :sob:


legacy-postgis not working ?

The property 'xxx' is of type 'PostgisGeometry' which is not supported by current database provider.Either change the property CLR type or ignore the property using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.
follow instructions here:  https://github.com/npgsql/npgsql/blob/dev/doc/types/legacy-postgis.md ,seems not working.
versions:
nuget Microsoft.AspNetCore.All == 2.1.0-preview2-final preview
nuget Npgsql.EntityFrameworkCore.PostgreSQL  == 2.1.0-preview2 preview
nuget Npgsql.EntityFrameworkCore.PostgreSQL.Design == 2.0.0-preview1 preview
nuget Npgsql.LegacyPostgis

The Npgsql.LegacyPostgis sets up support for the old postgis types at the ADO.NET level, but not at the EF Core level. That package isn't ready yet, but it will be by the time the next version is released.


Add a similarity score threshold

A great library, making it possible for me to work on 100K embeddings on my laptop!
I have a couple of suggestions.
Other than top_k, is it possible to return all items having a cosine similarity score above a given threshold, e.g., 0.99?
Currently, it is not possible to return all pairwise similarities, due to the following restriction:
if abs_top_k >= n_rows_right:
    raise ValueError(
        f"The number of requested similar items (top_k={abs_top_k}) must be less than the "
        f"number of items available for comparison ({n_rows_right})"
    )

I think "abs_top_k >= n_rows_right" can be changed to "abs_top_k > n_rows_right".  Or alternatively, when abs_top_k  >= n_rows_right, set abs_top_k to  n_rows_right and provide a warning.
If the other_embeddings has only one row, an error will be thrown currently.
After this change, it will be possible to return all pairwise similarities and users can screen them based on a threshold.
This is not possible currently.

@VictorXQWang thanks a lot for your suggestions.

Is it possible to set a min similarity threshold?

The problem that this library addresses is to deal with the memory constraint of keeping all similarity values. This works by knowing in advance how to partition the matrix given the wanted size of the result "top_k". If instead a "min_similarity_reshold" is used I cannot know in advance how many similarity matrix entries are non-zero therefore I cannot calculate in advance the partitions that I need to make for the calculations to fit in memory. It might be that it works if not many similarities are bigger than the threshold but it might be that it doesn't. It is an improvement over other methods where for sure they won't fit though, but it just feels itchy to add a functionality that I won't be sure it will succeed.

not able to return all similarities

The logic is that I would advice to use other methods (like sklearn cosine similarity), as ChunkDot does not produce any advantage in this scenario. It would slow it down probably.
I understand that is not the best for a user experience to be juggling between different functions depending on the case. Perhaps in this scenario I could just use sklearn cosine similarity function, but I don't like to add dependencies as heave as that just for a function.
Really good points though... Let me think about it.
And please let me know what you think and/or if you would like to contribute!


REST API in Deno

I want to share basic REST APIs (GET, POST, DELETE, PUT) in TypeScript on Deno
Languages

TypeScript

@YashSaxena9 your call on this?

@JigyasuPrakash Assigned to you.


Proper mask shape for a multiclass task

Hello,
I'm trying to use custom_unet for a multiclass task. In my case all masks inputs have a following shape
(BATCH_SIZE, NUMBER_OF_CLASSES, IMAGE_WIDTH, IMAGE_HEIGHT, 1).
Since I have 4 different colours on the black background and my images're 256x256 pixels and I use batch size of 4, I end up with the following shape:
(4, 4, 256, 256, 1).
Unfortunately, custom_unet doesn't like this shape giving me this error:
<...>
  File "C:\Users\E-soft\Anaconda3\envs\Explorium\lib\site-packages\keras_unet\losses.py", line 44, in jaccard_distance
    intersection = K.sum(K.abs(y_true * y_pred), axis=-1)
  File "C:\Users\E-soft\Anaconda3\envs\Explorium\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 899, in binary_op_wrapper
    return func(x, y, name=name)
  File "C:\Users\E-soft\Anaconda3\envs\Explorium\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 1206, in _mul_dispatch
    return gen_math_ops.mul(x, y, name=name)
  File "C:\Users\E-soft\Anaconda3\envs\Explorium\lib\site-packages\tensorflow_core\python\ops\gen_math_ops.py", line 6698, in mul
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [4,4,256,256] vs. [4,256,256,4] [Op:Mul] name: loss/conv2d_18_loss/mul/

It seems that I should reshape my input for masks, Should it be
(BATCH_SIZE IMAGE_WIDTH, IMAGE_HEIGHT, NUMBER_OF_CLASSES, 1)?

Hi @EtagiBI !
For multiclass semantic segmentation tasks you need to reshape your masks into (BATCH_SIZE IMAGE_WIDTH, IMAGE_HEIGHT, NUMBER_OF_CLASSES).
And to explain why behind it: with this shape you should treat NUMBER_OF_CLASSES as a number of layers in your output masks where each layer represents each class you are trying to predict. In other words your output mask will be a volume of binary masks where each mask represents predictions for specific class.
To give you some visuals on a dummy example:
print(x.shape, y.shape, y_pred.shape)
(1, 256, 256, 3) (1, 256, 256, 3) (1, 256, 256, 3)

utils.plot_imgs(
    org_imgs=np.vstack(
        (x, x, x)),
    mask_imgs=np.stack(
        (y[0][:,:,0],  # first layer of mask is the first class gt
         y[0][:,:,1],   # second layer of mask is the second class gt
         y[0][:,:,2])),   # third layer of mask is the third class gt
    pred_imgs=np.stack(
        (y_pred[0][:,:,0],  # first layer of mask is the first class predictions
         y_pred[0][:,:,1],   # second layer of mask is the second class predictions
         y_pred[0][:,:,2]))   # third layer of mask is the third class predictions
)

Thanks for your reply!
Here's one of my masks:

I use a following code to create an encoded array for learning:
COLOR_PACK = (60, 30, 16, 120)
masks = np.zeros((4, 512, 512)).astype('float')
image_hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
for i, color in enumerate(COLOR_PACK):
    hsv_min = np.asarray([color - 5, 100, 100])
    hsv_max = np.asarray([color + 5, 255, 255])
    mask = cv2.inRange(image_hsv, hsv_min, hsv_max) / 255.
    masks[i] = mask
masks_pack = masks.swapaxes(0, -1) # The result of this operation is an array of shape (512, 512, 4)

Is it a valid way to encode multicoloк masks?

@EtagiBI
I quickly checked your code and I don't think it's correct:

swapaxes is rotating your output image (I advise to always visualize outputs with matplotlib or something to verify if everything is ok)
COLOR_PACK values seems to be a bit inaccurate (at least in my experiment it didn't pick up red color

Code below works for me (notice inline comments):
import cv2

image = cv2.imread("../../dumpster/multiclass_segm.jpg")

COLOR_PACK = (60, 30, 5, 120)  # notice I changed the third value here (it was not picking up red color with previous setup)
masks = np.zeros((512, 512, 4)).astype('float')  # we go straight with to the target shape and use slicing
image_hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
for i, color in enumerate(COLOR_PACK):
    hsv_min = np.asarray([color - 5, 100, 100])
    hsv_max = np.asarray([color + 5, 255, 255])
    mask = cv2.inRange(image_hsv, hsv_min, hsv_max) / 255.
    masks[:,:,i] = mask  # using slicing to assign correct values to correct layers
# no need to swap axes anymore
# masks_pack = masks.swapaxes(0, -1) # The result of this operation is an array of shape (512, 512, 4)

import matplotlib.pyplot as plt

for i, color in enumerate(['green', 'yellow', 'red', 'blue']):
    plt.imshow(masks[:,:,i])
    plt.title(color)
    plt.show()

green
yellow
red
blue

Or:
from keras_unet import utils

utils.plot_imgs(
    org_imgs=np.stack(
        (image, image, image, image)),
    mask_imgs=np.stack(
        (masks[:,:,0],  # first layer of mask is the first class gt
         masks[:,:,1],   # second layer of mask is the second class gt
         masks[:,:,2],   # third layer of mask is the third class gt
         masks[:,:,3])),   # third layer of mask is the third class gt
    nm_img_to_plot=4,
    figsize=10
)

Thank you!
Axes manupulations still confuse me a lot... What other changes should be done in custom_unet for a multiclass task? I assume that softmax is more appropriate output activation function. Anything else?

@EtagiBI no problem.
I honestly haven't used U-NETs for multiclass (in case of multiple classes I just trained multiple single class models) so not much to advise here other than custom_unet offers variety of different hyperparameters to play with so it makes sense to experiment with different settings and check what works best for your problem space.

@EtagiBI no problem.
I honestly haven't used U-NETs for multiclass (in case of multiple classes I just trained multiple single class models) so not much to advise here other than custom_unet offers variety of different hyperparameters to play with so it makes sense to experiment with different settings and check what works best for your problem space.

Oh, I see. That's a good workaround indeed if you have few classes.
One last question, though. What's the correct way of transforming prediction into mask images in a multiclass model?
At the momet I use a following chunk of code:
test_image = cv2.imread("test.jpg") / 255.
ready_image = np.expand_dims(test_image, axis=0)
prediction = model.predict(ready_image).round()
prediction = prediction.squeeze(axis=0)
masks = np.array_split(prediction, 4, axis=-1)
for i, mask in enumerate(masks):
    ready_mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2RGB)
    ready_mask = ready_mask * 255
    cv2.imwrite(f"mask_{i}.jpg", ready_mask)

Results look messy, so I assume there's something wrong in my code.

@EtagiBI the code you shared looks fine although I prefer array slicing to extract specific layers. Also GRAY2RGB conversion is unnecessary unless you really need it. Grayscale image is perfectly fine especially for segmentation mask.
If your prediction results look messy your model probably failed to learn anything.
In my dummy example (training on a single image..) I used code below:
from keras_unet.models import custom_unet

model = custom_unet(
    (256,256,3), num_classes=3,
    use_batch_norm=False,
    dropout=0.0, dropout_type='standard',
    output_activation='sigmoid')

from tensorflow.keras.optimizers import Adam
from keras_unet.metrics import iou

model.compile(
    optimizer=Adam(),
    loss='binary_crossentropy',
    metrics=[iou]
)

model.fit(x, y, epochs=200)

When I tried building the network with softmax it couldn't learn anything.

Thanks for your answers!