Iterating over ArgMatches

 Issue by neysofu
Friday Mar 09, 2018 at 12:04 GMT
Originally opened as https://github.com/clap-rs/clap/issues/1206

Hi, I am using Clap in a little side project and I would like to suggest a minor feature. I am ready to implement it myself and send a PR if needed, I just need a few pointers. The feature at hand is an ordered iterator over ArgMatches: as far as my understanding goes, Clap doesn't allow to freely iterate over the parsed arguments. What if my application APIs are dependent upon the order of the arguments? GNU find comes to my mind. Calling index_of for every argument can be expensive if you have a lot of them.
If this feature were to be implemented, what would the return type and name of the method be?

 Comment by kbknapp
Monday Mar 19, 2018 at 17:58 GMT

Sorry for the wait, I've been on vacation with my kids this past week :)
Do you mean iterating over only positional values, or over all args that have been used? Also, do you envision this iterating over non-matched args as well?
If you'd like to take a stab at implementing I'd be happy to mentor you! This is how I'd attack it:

Draw out how you see this working, including the questions I listed above (including any edge cases you can think of)
I start with some usage code (such as in a test) to make sure the ergonomics are correct/comfortable
Write some tests in tests/matches.rs for the various uses you've drawn up in step one
I then just test that file alone with cargo test --test matches or can further test just a single function in that file with cargo test --test matches -- name_of_function
Start the implementation (this will vary greatly depending on how you've designed it)

For the implementation once we have a good design I can give you some pointers as to where the changes you'll want to make are located / any particulars associated with those changes.
Once all your new tests pass, I'd run a full test suite cargo test --features "yaml vec_map wrap_help unstable" and make sure nothing broke.
If you need to see the debug output while you're building testing a single function use cargo test --test matches --features debug -- name_of_function

 Comment by neysofu
Monday Mar 19, 2018 at 19:54 GMT

First of all, thanks for taking the time to address this! I hope you enjoyed your vacation.
The API I had in mind wouldn't be constrained only to positional arguments, but to all the arguments that have been used. Currently ArgMatches allows you to know which arguments have been used and their index in respect to all other arguments, yet it lacks a way to actually iterate over them.
I mean adding an API to access the order in which options are given. Ex.:
cmd --abc --xyz
cmd --xyz --abc

Let's say my command's behavior is specific to the order in which the options abc and xyz are given. As long as it's only two arguments I can get away with calling index_of for one of the two and figuring out which comes first. But what if I provide many another options? My idea is offer a method over ArgMatches that gives an iterator over the arguments in an ordered fashion:
arg_matches_1.iter_raw() == vec!["abc", "xyz"];
arg_matches_2.iter_raw() == vec!["xyz", "abc"];

This example is just off the top of my head and I am sure it could be improved.
Please correct me if I talked nonsense at any point because I am not very familiar with the API.

 Comment by kbknapp
Monday Mar 19, 2018 at 20:25 GMT

To make sure it's not an XY problem, how does your CLI work with the order of args? I've always seen conflicts/overrides as the main driving factor for when order matters. Both of which handled in the parsing logic.
The only other form I've seen is for things like include/exclude logic where there aren't really hard conflicts/overrides at the parsing level, but further up the execution stack there are. I've only seen this between a handlful of args at the most, and hence iterating over the indices isn't an issue.
This is of course not to say my list is exhaustive, I'm sure there are other use cases out there!
This actually wouldn't be a huge change for clap to implement this, but it would introduce an additional dep (such as indexmap ) and thus would need to be behind a non-default cargo feature.
The change you'd make is using cfg statements to change the hashmap to an indexmap and provide an ArgMatches method to simply return an Iterator of the requested info, perhaps as a tuple? (Such as (name, &[vals])?) This could be similar to how Arg::values_of works.
For the iterator itself, it'd fine to use the Values impl as a guide.

 Comment by kbknapp
Monday Mar 19, 2018 at 20:28 GMT

Another more lengthy option (but arguably more correct) is instead of returning an Iterator of some custom data type tuple, returning the MatchedArg itself. I say more lengthy because you'd have to document MatchedArg and provide the appropriate methods to get at it's internals such as values, occurrences, and indices. However, this is more flexible in the long run as you wouldn't need to add more ArgMatches methods in order to deal with different types of iteration...you'd just let the calling code deal with it has it wants.

 Comment by neysofu
Tuesday Mar 20, 2018 at 15:24 GMT

This actually wouldn't be a huge change for clap to implement this, but it would introduce an additional dep (such as indexmap ) and thus would need to be behind a non-default cargo feature.

That's a pity. I thought it would be as simple as adding a method to ArgMatches, with no fundamental changes to the actual program.

 Comment by neysofu
Tuesday Mar 20, 2018 at 15:22 GMT

My application has a custom version subcommand that works this way:
// Prints human readable version if no options are given
$ cmd version
Cmd 2.42.1-rc1 (fj23o83b)

// Switches to line-by-line format if options are given, to facilitate machine readability. The requested data is printed in the specified order.
$ cmd version --tags --major
rc1
2

However there are many other use cases. Tasks execution order in a Gulp-like application, apply filters in a specific order in GNU find, ecc.

 Comment by kbknapp
Wednesday Mar 21, 2018 at 00:33 GMT

Right now, clap uses a HashMap to hold the matches, which is inherently not in order. Switching to an IndexMap would be basically a drop in replacement, but because it's another dep I'd want people to have to opt-in to it.
So it basically would be just a cfg statement which adds the method on the matches struct and uses an indexmap instead of hashmap.  And I'd create a struct to act as the iterator, and impl Iterator for it.
Something like (details left out for brevity):
#[cfg(not(feature = "indexmap"))]
struct ArgMatches {
    args: HashMap
}

#[cfg(feature = "indexmap")]
struct ArgMatches {
    args: IndexMap
}

impl ArgMatches {
    #[cfg(feature = "indexmap")]
    fn iter(&self) -> MatchIter {
        MatchIter { iter: self.args.iter() }
    }
}

#[cfg(feature = "indexmap")]
struct MatchIter {
    iter: // whatever IndexMap::iter returns
}

#[cfg(feature = "indexmap")]
impl Iterator for MatchIter {
    type Item = // whatever args.iter().next() returns

    fn next(&mut self) -> Self::Item {
        self.iter().next()
    }
}

So it's a pretty straight forward change.

 Comment by neysofu
Wednesday Mar 21, 2018 at 04:44 GMT

Thanks for the tips, I'll start working on a prototype then. 👍

 Comment by neysofu
Saturday Mar 24, 2018 at 23:19 GMT

You can take a look at the prototype here. There is some more work to do, specifically:

write documentation for both the feature and the iter method;
remove the feature from the default set (it was a quick hack to make it build every time without passing arguments);
add some more tests.

Please let me know how you like the API and if you want any changes. Congrats on the codebase BTW, I know this is a small change but still, it is a pleasure to dive in.

 Comment by neysofu
Saturday Mar 24, 2018 at 21:49 GMT

I feel hesitant adding an iter method to ArgMatches; it feels like a duplicate of indexmap's functionality. ArgMatches.args: IndexMap is already public, so what's wrong with the user doing matches.args.iter()?

 Comment by kbknapp
Sunday Mar 25, 2018 at 03:17 GMT

I feel hesitant adding an iter method to ArgMatches; it feels like a duplicate of indexmap's functionality. ArgMatches.args: IndexMap is already public, so what's wrong with the user doing matches.args.iter()?

My current policy is public, but undocumented internals are effectively private and should not be relied on. Sure, I can't stop anyone from using them directly, but there's the caveat that if I change them without warning it's considered a breaking change. Hence the #[doc(hidden)]
Most of the time public but undocumented is because it needs to be public because other mods (or eventually crates) need to touch it, but those mods (or crates) are maintained by myself or other clap members and thus it's essentially just a private API.

 Comment by kbknapp
Sunday Mar 25, 2018 at 03:22 GMT

Looks like a good start! It's a small nit, but let's change the feature name to iter_matches because iter is somewhat ambiguous and may conflict with other types of iteration we may do in the future.
Let me know if you have any other questions :+1:

 Comment by neysofu
Sunday Mar 25, 2018 at 05:18 GMT

I'll create a PR for discussing further details, sounds good?

 Comment by kbknapp
Sunday Mar 25, 2018 at 19:21 GMT

Yep, I'd actually prefer it that way because it's easier to comment and review :smile:  :+1:

 Comment by XX
Saturday Apr 14, 2018 at 20:07 GMT

I also need this functionality

 Comment by bradwood
Sunday Apr 26, 2020 at 19:57 GMT

Hi all, I'm also looking to use something like this. I'd like to process my args in a functional fashion. The PR, however, seems to have died a death. Given that 3.0 is on the horizon how much of a bad idea is it for me to just use matches.args.iter() in 2.x?  Also, will this make it into 3.0?
PS, I don't need ordering.

 Comment by CreepySkeleton
Monday Apr 27, 2020 at 07:59 GMT

@bradwood Not as bad as you'd think: 2.x branch is frozen. I'd be really surprised if we change something regarding publicity of those fields because there is plenty of code out there relying on those, ugh. So, if you aren't planning on switching to 3.0 and you're OK with the fact that every time you use private API god kills a kitten, you can do that, yes.
Beware: those will be truly private in 3.0.
I suspect it won't make it into 3.0. The main question is: what are you going to iterate through? You can't have &Arg from ArgMatches because it stores Ids, basically a hash of those. On doesn't simply restore the Arg from the hash.

 Comment by bradwood
Monday Apr 27, 2020 at 12:32 GMT

i am trying to call map() on each arg I find to invoke the appropriate handler function based on the argument passed...
And, as regards the kittens, I'm cool with that!  :)

 Comment by CreepySkeleton
Monday Apr 27, 2020 at 12:39 GMT

clap has Arg::validator that is called on every value passed in. And of course, you can always iterate over std::env::args() and map it as you please; you don't really need clap for that.

 Comment by pksunkara
Monday Apr 27, 2020 at 14:48 GMT

I think I can see a certain use case for this where people want to do dependency injection pattern for evaluating their args.

 Comment by CreepySkeleton
Monday Apr 27, 2020 at 17:02 GMT

I want my CLI to accept options yes|no but I want the value held my context struct to be some concrete type

Resolved by derive (and partially by value_of_t* methods).
I'm not sure I get your "handlers" idea, I imagine it's something akin https://github.com/clap-rs/clap/issues/1185:

a single Arg::custom_parse which accepts a function that takes the arg/value currently being parsed, and returns an Option where None is skip this value (i.e. parse it as a different arg), or Some(T) is "save this to the matches" it could be the value as passed in, or it could be something different entirely (i.e. a map).

Some thoughts:

When is it ran, exactly? Before default values/environment variables were populated or after? What about required/conflicts stuff? How does it interact with old-style validators? With groups? Possible values?
I'm not sure I'm ready to answer these questions.

We've already got the flush-related bug. The problem here that clap exits with sys::exit() which implies that no destructors are being accounted for, no finallizers are being run, the text still awaiting to be processed by IO facilities is lost, etc... When I'm reading the description above, I'm given impression that "heavy" side-effect functions are OK, so will be users.

In nutshell: I think this kind of tool is too vague and too powerful for a command line parsing library to provide. I don't think clap should be trying to accommodate every single CLI in existence, the number of well-supported designs is quite enough already.

 Comment by bradwood
Monday Apr 27, 2020 at 17:38 GMT

FWIW, I'm not needing these map()s to run during arg parsing, I'm happy to pass an iterator of matched arguments into a function pipeline. I'll take a look at the validator, as I've not really looked properly at that. It might work for my use case.

 Comment by CreepySkeleton
Monday Apr 27, 2020 at 17:47 GMT

@bradwood Out of morbid curiosity, what is the core problem you're trying to solve? Why do you think map is  a good way to approach this kind of problem?
I suspect you're just "doing it wrong".

 Comment by kbknapp
Monday Apr 27, 2020 at 18:48 GMT

Resolved by derive (and partially by value_of_t* methods).

Exactly. I'm describing a problem/issue that existed prior to derive macros. The "handlers" was just a common theme that popped up every so often, especially when people were moving from Python based CLIs which offer a "when this argument is used, run this function." Although that's a totally different way to build/design a CLI and thus I don't think it's really applicable to how clap runs.

I don't think clap should be trying to accommodate every single CLI in existence, the number of well-supported designs is quite enough already.

100% agree. I'm just providing some historic context :wink:

@bradwood I agree with @CreepySkeleton it sounds like you may be approaching the problem from a particular design angle and we may be seeing it as an XY problem.
Some of the principal issues with a feature mapping over arguments (and their values) is:

For us (clap) to provide it, it needs to be general enough to fit all (or most) use cases, and with feature in particular there are a lot of things to consider and edge cases
If we're talking about true map (i.e. T->U), that can essentially already be done easily as @CreepySkeleton mentioned
If we're talking about handlers (i.e. perform some action for each argument); unless these actions are only blind side affects they:

need some sort of context passed to/from them (other args, maybe application context, etc.)
cannot rely on deterministic order or hierarchy
most likely need the concept of fallibility (what happens on failure)

Those final three bullets are some of what making this feature a general clap feature difficult. If you don't have any of those requirements and are truly only wanting blind side affects, I'd question if there may be a better way to represent the CLI.
If you can give some examples of what you're trying to do, we may be able to either point you in a direction or give you a way to accomplish that task with existing features.

 Comment by bradwood
Monday Apr 27, 2020 at 19:02 GMT

Heh. Given that I'm only 3 weeks in and coming from Python, I'm almost certainly doing it wrong! :smile:
Here's the situation:

I have a subcommand with 1 positional and about 20 non-positional arguments (some are just flags, and some take 1 or more values).
I want to drive another builder API with arguments passed in via clap. There is (almost) a 1-to-1 relationship between clap argument and the other API's builder methods
I could do a massive chain of if statement (although maybe a match would be better) that goes something like the below pseudo-code

if arg_n passed then:
     builder = builder.fn_n(arg's value[s]);
 else if..
(repeat 30 times)

Instead, more as a academic exercise than anything else, I'm trying to figure out if there's a more elegant way of doing in in a functional style... something like:
    // NOTE: args.args is a public, but _undocumented_ field in clap.rs
    args.args.iter()
        .map(
            |(k,v)| println!("{} {:?}",*k ,v) //replace this with a mechanism that I can use to call the appropriate builder method
            ).count(); // I'm using this to drive the function pipeline, purely for the side effects, rather than for the resulting value (I accept that this is probably not idiomatic)

What the aforementioned mechanism is, I'm not quite sure (yet)... but I was hoping to figure out a way to look up the other API's builder method that I need to call using the name of the clap argument passed...
I wait with bated breath for you to give me some beautiful idiomatic pattern that'll make this into anything other than 30 if statements!
Thanks!

 Comment by kbknapp
Monday Apr 27, 2020 at 19:17 GMT

Yours is not a super common case. Essentially, your CLI is a facade for a builder API? A few things about what you mention stood out to me (emphasis mine):

There is (almost) a 1-to-1 relationship between clap argument and the other API's builder methods

Unless you have a way of mapping those non 1:1 arguments, you'll end up with errors. So dealing with those errors is the fallible requirement I mentioned above.

In your example you use println! which kind of hides the problem, but in your pythonesque example you have builder = builder.fn_n which means each of these maps/handlers/whatever need to receive context, mutate it, and send it to the next fn (in this case its the builder). This is the  context sending I mentioned above.

Although it's not the 2 line map you're hoping for, the most idiomatic way to deal with this type of situation is via the clap derive macros. Your CLI would evaluate to some type, lets call it MyRawOpts, and your other builder would then impl From<MyRawOpts>. And by using some combination of macros you could decrease the verbosity of the From<..> impl to essentially a loop and map.

 Comment by bradwood
Monday Apr 27, 2020 at 19:27 GMT

I'm still reeling from everything you just wrote below the horizontal rule 2 comments above... :thinking:.  I'm going to have to do a whole lot of research on this, which is no bad thing... Thanks for your help (and patience) here!
As a final question, if I may, where would I look for examples of Clap and derive macros. Is this documented somewhere? I'm digging through the docs but not finding much on this topic (but I might not be looking in the right place).

 Comment by kbknapp
Monday Apr 27, 2020 at 19:21 GMT

Although it's a more advanced topic, and probably defeats the purpose of what you're trying to do, you could combine your own Custom Derive macro, and do something like #[derive(Clap, IntoMyBuilderApi)] which would instead impl Into<MyBuilderApi> for MyRawOpts automatically, and using things like the syn crate loop through the struct fields of MyRawOpts simply applying them to your builder.

 Comment by kbknapp
Monday Apr 27, 2020 at 19:39 GMT

The clap derive macros are the new big feature of clap 3.x which is still in beta and thus not formally released yet. So the documentation is still lacking.  The source is under the clap_derive crate in the root of this repository. You can look at the examples directory of clap_derive for some ideas.
If you'd prefer, you can use structopt which is well documented, until clap 3x is released. structopt uses clap 2.x internally, and all the clap 3.x macros are based on structopt (i.e. for clap 3.x we combined efforts between the two crates :wink:).
For info on writing Custom Derive macros:

Rust book section on macros
clap_derive source
structopt source
You'll need to reference the docs for the following two crates which are pretty critical to procedural macros:

syn
quote


Does noms have a story for storing files?

Git has git-lfs, SQL Server has FILESTREAM. Does noms have (or will have) a story for storing files?

i dont get it. It stores files on disk in its own format as it is right now.

Yes. Store files as Blob. See https://github.com/attic-labs/noms/tree/master/samples/go/blob-put


protoc-gen-bufmonkey location

"Download protoc-gen-bufmonkey and extract the files into the same" where can I find the protoc-gen-bufmonkey, referenced in the ReadMe?
Thanks

I was able to get this to work. Open the project and run the Grandle task createAllExecutables. Then you should be able to find protoc-gen-bufmonkey in /generator/build/artifacts/.
I then renamed the executable to protoc-gen-bufmonkey.exe and moved it to a folder in my $PATH.
Finally, I generated my files using protoc .\my_proto_file.proto --bufmonkey_out=./my_output_dir


How to reduce buffer size?

Buffer size is related with bitrate and iframe. If you reduce bitrate and increase iframe you will get less buffer size.


Support for NumPy > 2.0.0

In PR #107 NumPy was constrained to <2.0.0 in env.yml but not in pyproject.toml, so when sleap-io is installed via pip, the latest NumPy is installed.
This led to the following errors and deprecation warnings (in the tests):

AttributeError: np.string_ was removed in the NumPy 2.0 release. Use np.bytes_ instead.
sleap-io\sleap_io\io\video.py:151: DeprecationWarning: array_wrap must accept context and return_scalar arguments (positionally) in the future. (Deprecated NumPy 2.0)
is_grayscale = bool(np.all(test_img[..., 0] == test_img[..., -1]))

Thanks @lochhh!
Merged and available in v0.1.8. Up on PyPI and will be up soon on conda-forge.


6p 6sp full screen apps  Status bar overlay invalid

6p 6sp  full screen ，i add   in ，but when I run it， i found in  ，dont have  "with-statusbar-overlay" class，
other size device is normal！

Sorry, didn't understand much. Pleas provide more information, better with screenshots

Closing due to inactivity


npm WARN deprecated<EMAIL_ADDRESS>

I got the following warning while installing through npm:
npm WARN deprecated<EMAIL_ADDRESS>This package is deprecated and will no longer work with React 16+. We recommend you use CSSTransitionGroup from 'react-transition-group' instead.
<EMAIL_ADDRESS>/Users/thirumaleshc/PlayWing/playwing-publisher-portal-ui/playwing-ui
└─┬<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>  └──<EMAIL_ADDRESS>

Fixed in v1.6.1!


BUG: Fixed adapter channel name if several channels in use

Fixed adapter channel name if several channels in use and adapter created from container definitions:
use the same way as default adapter below.

Is that connected? https://github.com/yiisoft/queue/actions/runs/9244214835/job/25429257187?pr=204#step:9:100

Thank you @holdmann!
I used your changes and fixed correspoding tests in #213.


Create general DappIcon component that handles fallbacks

There is a lot of repetitive use of handleImageFallback functions all over the app that needs to be consolidated into a component.

Seems like fixed. Recently refactored all fallback images with the new one. Closing


Concept: projectiles / throw a bone

New Concepts:

Controller button events
projectiles
(Bonus) drawing a bone

Actually, I'll throw the next one in this pr too

It will introduce

on game update interval
random
screen height (properties)


Need a routine to respike aia images.

In GitLab by @markcheung on Apr 28, 2020, 09:12
AIA level 1 images are distributed with spikes removed, but can be restored with spikes.
This is sometimes used by researchers who care about bright points at the pixel level.
This is the IDL version of aia_respike
https://hesperia.gsfc.nasa.gov/ssw/sdo/aia/idl/calibration/aia_respike.pro
We need this function in the calibration submodule of aiapy as well as a jupyter notebook for the example gallery.

In GitLab by @wtbarnes on Apr 28, 2020, 09:20
Just a quick clarification: the example gallery entries are just .py files. Our documentation on Read the Docs will automatically generate the notebook for us.

In GitLab by @wtbarnes on Jul 14, 2020, 10:13
closed via merge request !62

In GitLab by @wtbarnes on Jul 14, 2020, 10:13
mentioned in commit 4df4e6d57e01051b27ca4449822168688611c2f2


v2 search is not working

Describe the bug
Search is not working throwing 404 when any search is made in version 2 docs page.
To Reproduce
Steps to reproduce the behavior:

Go to https://react-query-v2.tanstack.com/docs/overview
Search something like "QueryCache"
Hit enter
See error

Expected behavior
Should take user to search results
Screenshots

Desktop (please complete the following information):

OS: IOS
Browser:  chrome
Version: 89

sorry for that, it's a known issue with algolia, see: https://github.com/tannerlinsley/react-query/issues/1420#issuecomment-745008257

Search unfortunately will not work until algolia docsearch supports multiple subdomains .


Add .jsproj extension

https://github.com/search?utf8=✓&q=jsproj+extension:jsproj&type=Code&ref=searchresults=
Sample: https://github.com/MicrosoftEdge/JSBrowser/blob/master/JSBrowser/JSBrowser.jsproj

:+1:
For reviewers: A search on github.com showing more examples.

:+1: thanks @thejameskyle. This will be live on GitHub later this week when we cut a new version of the Linguist gem.


Refactoring file and folder names

This is a pretty big refactor of file names and folders, along with a little cleanup.
dynamax/hmm -> dynamax/hidden_markov_model

dynamax/generalized_gaussian_ssm/conditional_moments_gaussian_filter.py -> dynamax/generalized_gaussian_ssm/inference.py
dynamax/generalized_gaussian_ssm/conditional_moments_gaussian_filter_test.py -> dynamax/generalized_gaussian_ssm/inference_test.py
dynamax/generalized_gaussian_ssm/generalized_gaussian_ssm.py -> dynamax/generalized_gaussian_ssm/models.py

dynamax/linear_gaussian_ssm/linear_gaussian_ssm.py -> dynamax/linear_gaussian_ssm/models/standard.py
dynamax/linear_gaussian_ssm/linear_gaussian_ssm_conjugate.py -> dynamax/linear_gaussian_ssm/models/conjugate.py
dynamax/linear_gaussian_ssm/lgssm_test.py -> dynamax/linear_gaussian_ssm/models/models_test.py

dynamax/nonlinear_gaussian_ssm/nonlinear_gaussian_ssm.py -> dynamax/nonlinear_gaussian_ssm/models.py
dynamax/nonlinear_gaussian_ssm/extended_kalman_filter*.py -> dynamax/nonlinear_gaussian_ssm/inference_ekf*.py
dynamax/nonlinear_gaussian_ssm/unscented_kalman_filter*.py -> dynamax/nonlinear_gaussian_ssm/inference_ukf*.py

looks promising


test: spi: update fast test frequecny for em_starterkit

Async test failed in fast test due to too fast spi frequency.
Now it is lowered to 2500000Hz.
Signed-off-by: Siyuan Cheng<EMAIL_ADDRESS>

@tbursztyka @teburd May I ask you to take some time to review this one, thanks!


Can't see the summary line

Tools -> Options -> Fonts and Colors -> Editor Tooltip -> Item foregrounp: White

Tools -> Options -> Fonts and Colors -> Editor Tooltip -> Item foregrounp: White

@JobAbout  thanks for your report: even if a user can change the colors from the IDE, the fact that the theme (in its dark versions) doesn't allow to read the tooltip of the method summary is my mistake. Therefore I reopen the ticket to fix it.

@JobAbout thanks for your report: even if a user can change the colors from the IDE, the fact that the theme (in its dark versions) doesn't allow to read the tooltip of the method summary is my mistake. Therefore I reopen the ticket to fix it.

I really like this theme, thank you for porting the theme


api_token pass into either Repo or in the Issue class

There was issue #20 looking at adding the api_token into the Repo dataclass.
There is also a bug where the api_token is not passed when calling Issue.get_content_issues()
Problem
Even if your initial call for the data is authenticated then the others are no longer authenticated.
Recreating Problem

Expire your free non-authenticated API Calls
Create a issue with an attribute referencing other issues.
Create an issue object from that issue
Call issue.get_content_issues()

Solution
Either:

Pass api_token into Repo like #20 suggests and change the htmx call to pull the api_token from the Repo dataclass
Pass api_token into Issues. You will need to ensure that it is passed in Issue.from_issue_number() and wherever in code that is called.

cc @jonafato


Add ClientStore connector support

Description
This PR moves away from SimpleConnector to ClientStore, in sync with remaining juju modelcmd commands.
It also adds support for the user to specify the controller name, which changes the schema.
Fixes:  #421
Type of change

Change existing resource
Bug fix (non-breaking change which fixes an issue)

Environment

Juju controller version: v3.4

Terraform version: v1.7.4

QA steps
Manual QA steps should be done to test this PR.
provider juju {}
...

@phvalguima schema changes require separate approval. https://github.com/juju/terraform-provider-juju?tab=readme-ov-file#planning--design When you file a GitHub issue, use the schema change type.  Thank you.

Hi @hmlanigan okay, no worries.
Agreed, I need to build the AccountDetails object. I left unset only because it makes my own testing easier :)
Now, it is not passwordless connection. AFAIU if the user leaves it unset, then it will try to load the details from the Juju CLI. Granted, we may have or not it available. Now, if you have it available, you will also have access to ~/.local/share/juju/ folder of that given user anyways.

Closing this PR. The change appears to work because the client uses the juju store written to the local host. If the client finds a good ip address when connecting, it will write that ip address to the top of the list and continue to use until it doesn't work. Therefore added fan networks won't be tried unless things are going wrong, and this change appears to solve the problem described.
The fan networks will eventually be added by juju in any case. There are configs in the wild which depend on this behavior by juju as well.


Why is Microsoft telling me to install an unsigned app and give it admin priveleges?

This app requires admin to install and is not signed.  Let's fix that.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2483813a-3c16-7ddd-3160-43e6843b55d4
Version Independent ID: 25f7fb5b-92b0-a45b-57ab-fdceab571884
Content: Get started with NSwag and ASP.NET Core
Content Source: aspnetcore/tutorials/getting-started-with-NSwag.md
Product: aspnet-core
Technology: aspnetcore-tutorials
GitHub Login: @zuckerthoben
Microsoft Alias: scaddie

What unsigned app? What's the instructions?

Hi Rick, the link to install NSwagStudio:
https://github.com/RicoSuter/NSwag/wiki/NSwagStudio
On this page:
https://docs.microsoft.com/en-us/aspnet/core/tutorials/getting-started-with-nswag?view=aspnetcore-3.0&tabs=visual-studio

Does Microsoft officially support/offer this? Or is this just a 3rd party windows form application created by independent software developer?

The application is a 3rd party application. Our documents mention its use but Microsoft does not support it.
@RicoSuter can speak to why it's not signed.

@danroth27 please review. Seriously, you need to review this.

On the release page you can also download an xcopy version which can be started without installation and admin privileges. But  I agree that the app should be signed or provided via app store (better).

@glennc

On the NSwag releases page you can also download an xcopy version which can be started without installation and admin privileges.

Thanks @RicoSuter  I'll add that to this doc and close the issue.


Will this Circuit works as an Active PoE or Passive PoE Device?

It is active PoE, the TPS2375PW chip in the design has detection, classification and undervoltage lockout (UVLO),  also these controllers include an adjustable inrush limiting feature.


remove duplicated max delay check

Are they duplicated? Already a Math.max before if check.

Thanks


Feature onboard developer form b2

Pull Request details
I was experimenting with prisma migrations on the old branch, so I created a new branch for this PR.
I removed the DeveloperDetails model and added all fields to the User model instead.
npx prisma migrate dev && npx prisma migrate reset
I removed the splash page so the route is now:
/alpha/additional-details
I left the summary page in for now.
Any Breaking changes
NB
I removed @unique from the Membership and RSVP tables. Otherwise, I could not migrate.
I have to go out now, but will try my best to figure this issue later.

This was my first time to reconcile a branch divergence. It was only the prisma file, so I think it should be fine.

It was only the prisma file, so I think it should be fine.

Actually, I am really starting to doubt myself, so please check this  to be sure :)


better return type for collect

I’ve recently needed to add an assert in my code because of the old return type of collect.

Can one of the admins verify this patch? (see README)

By the way, you guys might need to change the return type of List.collect to [Element+]|[].

By the way, you guys might need to change the return type of List.collect to [Result+]|[].

Okay, I’ve just done that myself.

I suppose this is a binary-compatibility-breaking change, right? There was a hack we did to the backend to make the return type of sequence() BC, and we would probably need to do the same thing for collect(), I imagine.

I've made the suggested change, and hacked the backend in order to preserve BC.

Cool. I think the same could be done to .sort, and even maybe to top-level sort if you were willing to change its signature backwards-incompatibly.

OK, done.

@gavinking
Okay, cool. What about the top‐level sort? May I open an issue about it so you guys can do it in the next non‐patch release (1.X, as opposed to 1.3.X), or is it something you don’t expect to change even then?

Sure, I guess.


Azure ACI with MSI fails with Restart Policy of either Never or OnFailure

I've been able to get a Azure Container Instance working with a UserAssignedIdentity, and successfully retrieve secrets from the KeyVault. This works either using the "az container create" command, and also with a YAML file. For some reason the container group stays in the "Waiting" state, even though the application in the container executed successfully, without errors.
Since I need my container to run on demand (e.g. once a day, started by a Logic App), and don't want to pay for the rest of the time, I need to specify a RetryPolicy of "Never" or "OnFailure". But when I make only this single change in either my YAML file or my az container create statement, my container fails to get my secret from the keyvault, giving an error of "Connection String: RunAs=App, Resource: https://vault.azure.net, Authority: https://login.windows.net/<tenantID>. Exception Message: Tried to get token using Managed Service Identity. Unable to connect to the Managed Service Identity (MSI) endpoint. Please check that you are running on an Azure resource that has MSI setup."
So, 2 questions:

Why does the Container Group stay in the Waiting state after successfully running the container once? Does it assume a RetryPolicy of Always?
Why does changing the Restart Policy to either Never or OnFailure cause the Managed Identity (either system or user assigned) to fail?

Notes: I'm using C#, .NET Core 3.0, with Microsoft.Azure.Services.AppAuthentication, running in a Linux container, not using a Virtual Network.
Thanks in advance!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f4bb823e-1c72-5777-bdd6-e89942c4f470
Version Independent ID: eb046e79-f39c-dcca-fbd4-519c0a320201
Content: Use a managed identity with Azure Container Instances
Content Source: articles/container-instances/container-instances-managed-identity.md
Service: container-instances
GitHub Login: @dlepow
Microsoft Alias: danlep

@LouisDK
Thanks for the question! We are investigating and will update you shortly.

@LouisDK
I tried to reproduce your issue.
First i created a ACi with the below command.
az container create --name jakaruna-container --image microsoft/azure-cli --assign-identity  --command-line "sleep 300"
This command created am ACI and assigned an identity to it.  i gave the permission for that identity to access my key vault.
Then i logged in to the container and i am able to access the secret.(first got the token , Then used to token to make an api call)
Since the command used is sleep 300, The container sleeps for 5 minutes and then goes into terminated state.
By default the restart policy is Always, So it restarted my container.
I logged in again and i am able to access the secret.
Now If we want to change the restart policy, Then we need to delete the existing container group and create a new one. Because changing the restart policy is not allowed.
So i deleted the existing one and created a new one with restart policy as shown below.
az container create --name jakaruna-container --image microsoft/azure-cli --assign-identity --restart-policy  OnFailure --command-line "sleep 300"
Since old one is deleted and new once is created, The identity also changes. So i need to get the ACi sp id and add permissions to read the key vault secret.
After that i was able to get the secrets as usual.
If you are using user assigned id, Then you dont need to assign permissions to access key vault again.
After 5 minutes, The container went into terminated state and the overall status is "succeeded"
The script we used is sleep 300 and it exited with exit code 0.
When i started the container again, I was able to access the secret in key vault.
Now lets go to your question.

Why does the Container Group stay in the Waiting state after successfully running the container once? Does it assume a RetryPolicy of Always?

By default the restart policy is "Always".  So if the container terminates, It will try to restart. In that time overall status will be waiting.

Why does changing the Restart Policy to either Never or OnFailure cause the Managed Identity (either system or user assigned) to fail?

We can change the restart policy only when we delete the old one and create a new container group. If you are using a system assigned managed identity, you need to give the permissions again.  If you are using user assigned identity, Then assigning the permissions again is not required.
I tried with system assigned identity and i have to give permissions again to get it work.
Let me know if you are using user assigned identity

@LouisDK is there any upadate

Hi,
I am using a User Assigned identity.
If I create a new container (using "az container create", with either
various parameters, or a YAML file), it works fine (able to retrieve
secrets from the KeyVault) as long as the Restart Policy is unspecified
(defaulting to Always).
I then delete the container group, wait a few minutes, then deploy a new
container - this time specifying a Restart Policy of 'Never' (with either a
parameter, or in the YAML file) (that is the only thing that change from
the first, working deployment), the container group and container is
successfully created, runs successfully, terminates after it ran - but it
fails to retrieve secrets from the KeyVault.
If you like - I could provide you with my YAML file, as well as a sample
project (I'll put something in a public GitHub repo). Let me know if I
should do this.
Thanks in advance!
-Louis
On Wed, Oct 23, 2019 at 6:23 AM jakaruna-MSFT<EMAIL_ADDRESS>wrote:

@LouisDK https://github.com/LouisDK is there any upadate
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/41107?email_source=notifications&email_token=AARI5F7XXD5OJ6H5KOLCJY3QP7GTFA5CNFSM4JCZEDDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB777TQ#issuecomment-545259470,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AARI5F6GMTNTLCT22OIZOF3QP7GTFANCNFSM4JCZEDDA
.

@LouisDK yes please share your yaml file :)

@LouisDK Both Jagadeesh and myself attempted to reproduce this but we found the keyvault was accessible regardless of the restart policy. So I am assuming there is something specific with your deployment or yaml file.
I would suggest you simply create the container with the restart policy you want the first time and avoid having to delete, redeploy then reconfigure your containers.
If you still want help digging in deeper, you can email me at<EMAIL_ADDRESS>and provide me with your SubscriptionID and link to this issue. I can then enable your subscription for a support request so you can work with the engineering team to help you debug further.

Hi,
I found the problem - there seems to be a time delay from when the
container starts, and when the Managed Service Identity endpoint becomes
available.
My container failed to connect to the Managed Service Identity endpoint
when it tried to immediately connect to it when starting up. When I
introduced a 1 second delay, I had intermittent success (1/5), with
occasional failures. When I increased the delay to 3 seconds, I had
consistent success (5/5).
I will now consider the Managed Service Identity endpoint as something for
which I should consider Transient Errors, and will use something like
Polly.NET to implement a retry policy.
I also suspect that a Restart Policy of Never have some impact on the time
it takes to start the container, since I had 0/5 successes with a restart
policy of Never, but a 3/5 success rate with no Restart Policy.
You probably did not notice anything wrong since your way of testing was
interactive - and only tried to connect to the Managed Service Identity
endpoint many seconds (>30?) after the container was started.
My scenario (as would 99.99% of other people's) was that an application
would do the work, and not a human typing commands in a shell.
I'm speculating that this problem will typically only present itself when
the application in the container attempts to immediately connect to the
Managed Service Identity endpoint - so mostly console applications / micro
services. ASP.NET applications takes a little longer to start up, and would
see this less often / not at all.
Thank you for your help - I would not have found the problem if you did not
prove to me that the Interactive way works, and that my automated way
always failed - leading me to consider transient conditions / timing. My
YAML file was fine by the way.
Best Regards,
-Louis de Klerk
On Sat, Oct 26, 2019 at 2:04 AM Micah<EMAIL_ADDRESS>wrote:

@LouisDK https://github.com/LouisDK Both Jagadeesh and myself attempted
to reproduce this but we found the keyvault was accessible regardless of
the restart policy. So I am assuming there is something specific with your
deployment or yaml file.
I would suggest you simply create the container with the restart policy
you want the first time and avoid having to delete, redeploy then
reconfigure your containers.
If you still want help digging in deeper, you can email me at
<EMAIL_ADDRESS>and provide me with your SubscriptionID and
link to this issue. I can then enable your subscription for a support
request so you can work with the engineering team to help you debug further.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/41107?email_source=notifications&email_token=AARI5FZ65LTQP53QKRKFY73QQOCQLA5CNFSM4JCZEDDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJ2T5Y#issuecomment-546548215,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AARI5F4HAXLNQPQF6KHUQM3QQOCQLANCNFSM4JCZEDDA
.

I have struggled with Azure Container Instances and Service Principals (both System- and User Assigned Identities).
Turns out, as a last resort, that i followed @LouisDK suggestion with adding a sleep before adding the KeyVault configuration.
Before adding a sleep of for now 30 sec., I kept receiving these messages that indicates that MSI is not running (either by using default implementation, where Microsoft tries MSI, CLI or current user) or by specifying a connection string (for either App or App and ClientId). Given the size of the company I work for, there was a risk that this was actually true by some policy/setting unknown to me.
 I ended up sleeping for 30 seconds - less will probably do, but tired of testing this bug.
Hooooooooray! It is now working as expected. The good thing about this (IMO bug) is, that i know alot more about what is going on behind the scene; the bad thing is, that this should really not be an issue - it should have been thought into the KeyVault Nuget package.
Microsoft, please do something about this issue .. the error message is not clear enough in stating what could be wrong.
Thanks.


BROWSER environment variable handling doesn't handle multiple browsers or %s escapes

Thanks for adding the fallback to the $BROWSER environment variable.
However, unlike $PAGER and $EDITOR, $BROWSER doesn't always consist of a single command name that you can run with an URL argument.  Quoting man 1 man (one of a few sources fully documenting $BROWSER):

If $BROWSER is set, its value is a colon-delimited list of commands, each of which in turn is used to try to start a web browser for man --html. In each command, %s is replaced by a filename containing the HTML output from groff, %% is replaced by a single percent sign (%), and %c is replaced by a colon (:).

See the documentation for the Python webbrowser module for another explanation of $BROWSER.
So:

Split $BROWSER on ':' and loop over each, continuing to the next on failure
For each one, replace '%s' with the URL, '%c' with a colon, and '%%' with a percent.  If and only if it didn't contain '%s', append the URL.

Sorry about that. I presumed a certain behaviour for $BROWSER. Will fix.

Looks great, thank you!


JWT-based authentication using local SQLite3 database

Chris: Here is the local authentication implementation I outlined at BayHac last month. Without additional configuration, CodeWorld should continue to work exactly as it does now, using Google authentication as the standard authentication method. With some configuration, as described in README.md, the local authentication method (accounts stored in SQLite3 database and JWT-based authentication) will be enabled.
-Richard

Thanks for the fixes.  I definitely don't want to block you for too long on changes, if the current code works correctly.  I was just trying to understand what was going on and match it to my understanding of HTTP authentication and related specs.  It may well be that it's my understanding that needs fixing, rather than the code!

I'm going to rework the code so it doesn't need the Authorization header in the response. I'm also going to implement a proper refresh token workflow. The code will be better for it! Thanks!


AttributeError: 'DatasetInfo' object has no attribute 'task_templates'

Describe the bug
Hello,
I'm trying to add dataset and contribute, but test keep fail with below cli.
 RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<my_dataset>
Steps to reproduce the bug
It seems like a bug when I see an error with the existing dataset, not the dataset I'm trying to add.
 RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<any_dataset>
Expected results
All test passed
Actual results
                # check that dataset is not empty
                self.parent.assertListEqual(sorted(dataset_builder.info.splits.keys()), sorted(dataset))
                for split in dataset_builder.info.splits.keys():
                    # check that loaded datset is not empty
                    self.parent.assertTrue(len(dataset[split]) > 0)

                # check that we can cast features for each task template
>               task_templates = dataset_builder.info.task_templates
E               AttributeError: 'DatasetInfo' object has no attribute 'task_templates'

tests/test_dataset_common.py:175: AttributeError

Environment info

datasets version: 1.6.2
Platform: Darwin-20.4.0-x86_64-i386-64bit
Python version: 3.7.7
PyTorch version (GPU?): 1.7.0 (False)
Tensorflow version (GPU?): 2.3.0 (False)
Using GPU in script?: No
Using distributed or parallel set-up in script?: No

Hi ! Can you try using a more up-to-date version ? We added the task_templates in datasets 1.7.0.
Ideally when you're working on new datasets, you should install and use the local version of your fork of datasets. Here I think you tried to run the 1.7.0 tests with the 1.6.2 code


WARNING:root:NaN or Inf found in input tensor when training UNETR network.

Describe the bug
Nan popped up while I was training the UNETR network.
WARNING:root:NaN or Inf found in input tensor.
To Reproduce
Steps to reproduce the behavior:

Go to 'UNETR/BTCV'
Run commands:
CUDA_VISIBLE_DEVICES=1,2,3 python main.py --distributed --feature_size=32  --batch_size=4 --logdir=unetr_test --optim_lr=1e-3 --lrschedule=warmup_cosine --infer_overlap=0.5  --save_checkpoint --data_dir=/data/BTCV/Abdomen/RawData/Training/ --workers=12

Screenshots

Environment (please complete the following information):

OS
Python version  3.8
MONAI version [e.g. git commit hash]
CUDA/cuDNN version CUDA11.1
GPU models and configuration. V100

Hi @mjq93
I am sorry you are facing this difficulty. We never ran into a NaN while training the UNETR. It is most likely caused by DiceCELoss as the values of denominator become very small.
One quick fix it to set squared_pred = False in DiceCELoss. This prevents the value of predictions, which are between 0 and 1, to further become smaller -- although we observed slightly better performance by squaring the predictions.
Another fix would be to pass a larger value such as smooth_dr = 1e-5 to prevent division by zero. I believe first fix will work out of the box and is more preferred.
Thanks

I have met the same issue that  the loss become Nan when training,  both  the two fixs  are invalid. How do you solve them in the end?

@mjq93
I have met the same issue that the loss became Nan when training. Both the two fixs are invalid, how do you solve them in the end?

I have met the same issue that the loss became Nan when training. Both the two fixs are invalid, how do you solve them in the end?

Thank you, I  have solved it


Update Gulp to 4.0.0

This PR does a couple of things surrounding the upgrade to Gulp 4.0. This PR closes #52.

Update the gulp package to 4.0.0
Rename the gulpfile to gulpfile.babel.js to be able to utilize newest ECMAscript features
Reorder and restructure gulp tasks and to better utilize the new gulp serial and parallel methods
Remove unused build task
Remove unused npm packages (gulp-gh-pages, gulp-load-plugins)
Update npm lock file

I'm happy to continue optimizing, or to pull back and limit these changes a little. One thing to note is that the clean, sw and critical tasks all run with the default task now, whereas they had tobe run individually before. Even with these additions, the gulp parallel method means that it still runs just as fast (~8s before to ~5s now)
Further improvements could be splitting the gulpfile into a folder with modules and includes. (i.e. gulpfile.babel.js/index.js)

Hey @lachieh ,
Thanks a lot for this, this is great! 🎉
If you would be interested in splitting the gulpfile into separate modules, as you mentioned, just feel free to make a PR 🙂


Remove "Photoshop Record Action json" item from Barkin's docs

Per mention here:
https://forums.creativeclouddeveloper.com/t/cannot-see-the-record-action-json/2123/2
The doc page is here:
src/pages/ps_reference/media/advanced/batchplay.md

Assign to Barkin once he's onboarded to GH team.


Add learn/docs for Groth16

Including an example Move smart contract.

done in https://github.com/MystenLabs/sui/pull/11450


Less Support

I'd love to add this to our internal styleguide, but we use Less instead of SASS.
I see it's on your roadmap, I just thought I'd add an Issue for visibility and so anyone else can voice any opinions or that they would also like this.

There is a pull request open for adding less support:
https://github.com/animade/frontend-md/pull/9

There's already a pull request for this! :)

I checked and it hadn't gone in, but excellent. :+1:

+1

Closed by https://github.com/animade/frontend-md/commit/310d0d81bd68bf3bd30563ada219e568e4c31c17. Thanks @crismanNoble for the patch!


Add ability to use GCP Cloud Storage as ModelRepository

Is your feature request related to a problem? Please describe.
If at all possible, we should be able to Upload, Download, Deploy models using GCP Cloud Storage as a provider.

Absolutely, great idea @willvelida I think that should be very doable.


Avatar optimizer 3

Replaces https://github.com/webaverse/app/pull/3409.
Depends on:

THREE.js r143 update (based here).
https://github.com/webaverse/totum/pull/127

Ok, this should be done. All 3 avatar optimizers produce correct results, and are running off the main thread now.

This was merged in the labs!


Yolo models on Qualcomm X Elite Dev Box

Hi, I use your quantized YOLO model Model_Yolo_v9c_f16.onnx downloaded from googledrive to get Yolo running on the Qualcomm X Elite Hexagon NPU with DirectML. You state that these are v9 but Netron shows they are v10. At least I need to use function detectYOLOv10() from here to get the bboxes and classifications from the output tensor.
Can you explain?

I'm sorry for the confusion, but it does show version v9c at METADATA-description. Additionally, we have optimized the original v9 code (in the folder: modeling_modified/tasks.py, lines 148-172), changing the output of the v9 model to [x, y, w, h, max_score, max_indices]. This output format is indeed similar to the results of version v10.

Hi, it works now. I'm using one of your models over here:
https://github.com/fobrs/yolov9_npu


Update Modules “partner_fax”

Automatically generated by Decap CMS

@florenciafrigieri, @dreispt, @TumbaoJu It worked!
This was created by Decap CMS, rendered by Hugo (finally not mkDocs due to technical restrictions), and deployed by Netlify.
The integration is nice and as you see it creates automatically a comment in https://github.com/moduon/oca-docs/pull/1#issuecomment-2386129511 with a preview link that you can use to review the PR functionally.
The main branch is deployed to https://moduon-oca-docs-prototype.netlify.app/ but that doesn't include this PR until merged. You can try using decap cms clicking on the link from the contribution guide there. Give it a try and open some PR.

@florenciafrigieri, @dreispt, @TumbaoJu It worked!
This was created by Decap CMS, rendered by Hugo (finally not mkDocs due to technical restrictions), and deployed by Netlify.
The integration is nice and as you see it creates automatically a comment in #1 (comment) with a preview link that you can use to review the PR functionally.
The main branch is deployed to https://moduon-oca-docs-prototype.netlify.app/ but that doesn't include this PR until merged. You can try using decap cms clicking on the link from the contribution guide there. Give it a try and open some PR.

Yeah!!!  I will try it.


Add CREDITS file

We should add a credits / contributors file where we will thank each person that has commited / created an issue in this repository for their valuable input.

Done.


y-axis tick mark showing up after the label text after upgrading to v2.7.2

Expected Behavior
The tick marks should be before the label.
See below screenshot from v2.7.1

Current Behavior

the tick marks are rendering after the labels
Context

Environment

Chart.js version: v2.7.2
Link to your project: https://jsfiddle.net/fh1ydwbt/

@bhavin36 sorry about that. I opened a PR fixing it in #5401


Lambdakiq jobs no longer work on rails 7.2

The job enqueuing fails with
undefined method `enqueue_after_transaction_commit?' for an instance of ActiveJob::QueueAdapters::LambdakiqAdapter

I have created the following pull request to address this: https://github.com/rails-lambda/lambdakiq/pull/35

Thank you! Merged, going to do a gem update now too.

Amazing! thank you so much @metaskills

@metaskills I have just upgrade my app to 2.3.0 and can confirm it works with rails 7.2


After I run GSEA, I got the heatmap like the below one: [KD, KD, control, control]. I initially thought this indicated that these genes were repressed in my KD samples.

But when I checked these genes in my input files and did scatter plot I am confused... It is indeed that these genes show down-regulated tendency in KD samples( X axis: KD, Y axis: control) but not as significant as they did in previous heatmap. Could you please explain what the heatmap value stands for? Why the scale is -1~1?
I used normalized read counts as input. The scatter plot is log2(counts + 1)

Thanks for your help in advance!

for heatmap visualization, data is scaled by standard normalization, that's to said, for each data point x, the normalized values = (x - row_means)/row_standard_deviation.


CLOUDP-179041: unify go version for ci/cd

Description
CI/CD process is using some old versions of Golang.
Unifying everything to run on 1.20.

why does our go.mod say 1.18 and our CI uses 1.20? is it intended?


Add list of available time zones.

Pull Request Test Coverage Report for Build 314

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 300:
0.0%

Covered Lines:
270

Relevant Lines:
270

💛  - Coveralls


Install Fails on Ubuntu 16.10

I do realize the .deb is for 16.04.1 but I thought someone might want to know.
Steps to reproduce
ransom@claw:~/Downloads$ sudo dpkg -i powershell_6.0.0-alpha.14-1ubuntu<IP_ADDRESS>_amd64.deb
Selecting previously unselected package powershell.
(Reading database ... 221717 files and directories currently installed.)
Preparing to unpack powershell_6.0.0-alpha.14-1ubuntu<IP_ADDRESS>_amd64.deb ...
Unpacking powershell (6.0.0-alpha.14-1ubuntu<IP_ADDRESS>) ...
dpkg: dependency problems prevent configuration of powershell:
powershell depends on libicu55; however:
Package libicu55 is not installed.
dpkg: error processing package powershell (--install):
dependency problems - leaving unconfigured
Processing triggers for man-db (2.7.5-1) ...
Errors were encountered while processing:
powershell
ransom@claw:~/Downloads$ sudo apt-get  install -fReading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following packages will be REMOVED:
powershell
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 130 MB disk space will be freed.
Do you want to continue? [Y/n] n
Abort.
ransom@claw:~/Downloads$ powershell
Failed to initialize CoreCLR, HRESULT: 0x80131500
ransom@claw:~/Downloads$
Expected behavior
Install success
Actual behavior
Install failed to work
Environment data
ransom@claw:~/Downloads$ cat /proc/version
Linux version 4.8.0-32-generic (buildd@lcy01-34) (gcc version 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) ) #34-Ubuntu SMP Tue Dec 13 14:30:43 UTC 2016
ransom@claw:~/Downloads$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.10
Release:	16.10
Codename:	yakkety

> $PSVersionTable

Duplicate of #2735


Project names in factory file should be strings, not names

Occasionally, names contain odd symbols which may break things. Simply change the project factory name line to this:
$name = $faker->unique()->word;

v0.9.9


X.L.SimplestFloat contains an unsafe getWindowAttributes

getSize :: Rectangle -> Window -> X (Window,Rectangle)
getSize (Rectangle rx ry _ _) w = do
  d  <- asks display
  bw <- asks (borderWidth . config)
  wa <- io $ getWindowAttributes d w
  let x  =  max rx $ fi $ wa_x wa
      y  =  max ry $ fi $ wa_y wa
      wh = fi (wa_width  wa) + (bw * 2)
      ht = fi (wa_height wa) + (bw * 2)
  return (w, Rectangle x y wh ht)

I guess nobody actually uses this layout, or we'd have heard about layouts suddenly becoming Full because they'd thrown exceptions by now.

Or getWindowAttributes never fails :)

Pretty sure it does, and that's why safeGetWindowAttributes is in Prelude.hs. (I have certainly seen apps create and withdraw windows before we can manage them,)


Adding all GPU Tests

Awesome!

Thanks! Still on its way, there's so many operations already, it is exciting tho

"Houston, we have lost control of the Diego rocket, we can't stop it anymore" 😆
👍 👍 👍

To skip GPU tests on Travis I wasn't able to pass any params to the Unity app, so I had to create a script to delete the file directly. Though the application supports sending arguments through command line, they don't work.
More info:
https://answers.unity.com/questions/1436131/command-line-test-runner-option-editortestscategor.html?childToView=1440854#answer-1440854

Tests can be filtered by category: "CPU" or "GPU".

Make sure to select the OpenMinedMain asset for the GPU tests to work!

Whew!!!! What a monstor!!!


Push Notifications with RNN V2

Have you guys been able to get react-native-navigation V2 working with push notifications?

local or remote? I can double check today, we usually use Firebase CFM for remote.

Nevermind, I managed to get it work. Local/remote using GCM.


Tag releases

Hello,
Could you tag releases to make packaging rainbow stream easier?

Hello @kyrias. I am releasing RainbowStream through pip - the official Python package manager. The version on pip contains the CONFIG KEY, but this repo on Github is not. So packaging this repo on a tag release will not provide a runnable version of RainbowStream.

Fair enough, though you can also upload release tarballs/zips to GitHub.

Anyway, tags make making a -git package nicer too, since git-describe will give sensible output, and will also help working with the history to find eg a bug introduced in some commit since the last release.


Support map and set

Changes
Close #1564
Checklist

[ ] An issue to be fixed by this PR is listed above.
[ ] New tests are added to ensure new features are working. Please refer to this page to see how to add a test.
[ ] ./frb_internal precommit --mode slow (or fast) is run (it internal runs code generator, does auto formatting, etc).
[ ] If this PR adds/changes features, documentations (in the ./website folder) are updated.
[ ] CI is passing. Please refer to this page to see how to solve a failed CI.

Remark for PR creator

./frb_internal --help shows utilities for development.
If fzyzcjy does not reply for a few days, maybe he just did not see it, so please ping him.

Depend on: https://github.com/shekohex/allo-isolate/pull/54/files

the failed ci is because of temporary change of cargo.lock

Will merge after https://github.com/shekohex/allo-isolate/pull/54 and upstream release is done


[Site] Home page IA

As a consumer, I want an attractive, useful homepage to greet me when visiting the site so that I feel like Mineral UI is a professional product and can quickly find what I need.
AC:

Survey peer design system site homepages
Create IA doc
Possibly create wireframe

IA doc is here: https://docs.google.com/document/d/1whPfwGyavHGwwUQHue3Yz-IjA-0Ngyk7gL4_95xQ-W0/

Basic wireframes showing a couple different layouts.


standard library suppress instead of try...except...pass

Code readability :)
Just thought it would be nice to use the suppress instead of the except...pass usage.

Codecov Report

Merging #4858 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #4858      +/-   ##
=========================================
+ Coverage   66.89%   66.9%   +<.01%
=========================================
  Files          15      15
  Lines        1577    1571       -6
=========================================
- Hits         1055    1051       -4
+ Misses        522     520       -2

Impacted Files
Coverage Δ

requests/sessions.py
76.42% <100%> (-0.34%)
:arrow_down:

requests/models.py
77.7% <100%> (+0.34%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c501ec9...d3d21af. Read the comment docs.

I'm -1ish on this change because it seems to purely be a refactoring that isn't necessary which can make archaeology on bugs significantly more difficult. I'm not a fan of change for change's sake when it comes to requests.

I think I’m in agreement with @sigmavirus24 here. While this does make a small simplification to the code it’s adding additional overhead to ensure we continue using this going forward.
I think we’ll pass on this change for now, @hyaxia, but thank you for putting the idea forward!

Thanks for replying.
Just out of curiousity, if there was no need for the python 2.7 compatibility, and you could use the "suppress" method from the standard library, would your opinion on the matter be different?

We’ve discussed removing Python 2 support as part of Requests 3.0 but I don’t know if that decision has been finalized. We may revisit this at that point but until then I think we’re unlikely to merge a change like this.

Cool thanks!


Get.back() not working in Get.Dialog using GetMaterialApp builder property

Describe the bug
I have used the Default layout using material App for all pages, so i have a Get.Dialog() in which I have used a container to show a dialog box, but on the click of the button Get.back() is not working due to which i am not able to close dialog box on the button click.
Please refer to the below examples for the details with Screenshots and error description
Example:
main.dart
import 'package:flutter/material.dart';
import 'package:flutter_application_1/defaultview.dart';
import 'package:get/get.dart';

void main() {
  runApp(GetMaterialApp(
    initialRoute: '/task',
    builder: (context, child) => DefaultView(child: child),
    getPages: [
      GetPage(
        name: '/task',
        page: () => Container(
          child: Center(
            child: RaisedButton(
              child: Text('Open Dialog'),
              onPressed: () => Get.defaultDialog(
                title: 'Login Failure',
                radius: 5.0,
                middleText: "result",
                actions: [
                  RaisedButton(
                    onPressed: () {
                      print("test");
                      Get.back();
                    },
                    child: Text(
                      "OK",
                      style: TextStyle(color: Colors.white),
                    ),
                    color: Colors.black,
                  )
                ],
              ),
            ),
          ),
        ),
      )
    ],
  ));
}

defaultview.dart
class DefaultView extends StatelessWidget {
  final Widget child;
  DefaultView({@required this.child});
  final GlobalKey<ScaffoldState> scaffoldKey = new GlobalKey<ScaffoldState>();
  Color _colors;
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      debugShowCheckedModeBanner: false,
      home: Scaffold(
        appBar: AppBar(
          title: Text('Task'),
        ),
        body: Row(
          children: [
            SizedBox(
              width: 60.0,
              child: NavigationRail(
                selectedIndex: 0,
                onDestinationSelected: (index) {
                  _colors = Colors.amber;
                },
                destinations: [
                  NavigationRailDestination(
                    icon: Container(
                      child: Icon(
                        Icons.home,
                        color: Colors.red,
                      ),
                      color: Colors.red.withOpacity(0.50),
                    ),
                    label: Text("Home"),
                  ),
                  NavigationRailDestination(
                    icon: Icon(Icons.supervised_user_circle),
                    label: Text("Persons"),
                  ),
                ],
              ),
            ),
            Expanded(
              child: Container(
                child: child,
              ),
            ),
          ],
        ),
      ),
    );
  }
}

Error Description in logs:
[GETX] "GetMaterialController" has been initialized
[GETX] GOING TO ROUTE /task
test

To Reproduce
Steps to reproduce the behavior:
Use the above example with builder property issue is easily reproducible
Expected behavior
Get.back() should close the dialog on click of OK button.
Actual behavior
Dialog box does should get closed on OK button using Get.back()
Screenshots
Dialogbox with OK button

Flutter Version:
Flutter 1.23.0-18.1.pre • channel beta • https://github.com/flutter/flutter.git
Framework • revision 198df796aa (4 months ago) • 2020-10-15 12:04:33 -0700
Engine • revision 1d12d82d9c
Tools • Dart 2.11.0 (build 2.11.0-213.1.beta)
Getx Version:
Using the latest get package as specific version is not specified in the yaml file
dependencies:
get: 3.24.0
Describe on which device you found the bug:
Flutter web and Mi A1
Minimal reproduce code
Use the Above provided example issue is easily reproducible

Why you use MaterialApp inside GetMaterialApp?

I had referred the issue raised by one of my colleague for Use of Popupbutton which requires a material App in which he suggested we can use Material App inside a material app which does not create the Material App twice.
https://github.com/jonataslaw/getx/issues/729

@ghprod  Any updates..?

Try this:
Get.back(closeOverlays: true);

Tested on get: 4.2.0

Do not use Material App in GetMaterial App, and try to create a scaffold before creating dialog


Add --batch option to workflow_add_task

It should return a set of IDs (especially including the one of the new WorkflowTask), so that it can be used in scripts.

This is not urgent since we are already fixing #346 (in PR #347), and it would require a change to the server endpoint (which currently returns the entire Workflow object, rather than the created WorkflowTask one). Closing now, and we can re-open whenever it is needed.
If it comes up again, after modifying the server endpoint the client fix is as easy as https://github.com/fractal-analytics-platform/fractal/pull/347/commits/f8eb2912ebaa978e101af9d1d7cec00056d50a71


[CIVP-18964] Update notebook package to 5.7.8

This PR updates the notebook package version in requirements.txt

@leanne73 Could I get your thoughts on whether this is the right way of going about updating the package?


HTML in Radio or Checkbox Values

I need help.  I am trying to put the  tag within a checkbox option within a group of checkboxes.  I tried using $sce.trustAsHtml but that didn't work, neither did just putting the html in the name value.  Code below.  No matter what I do, the HTML inside of the name doesn't evaluate.  Any ideas?
{
	key: 'applicable_parcel_exemptions',
		type: 'customMultiCheckbox',
		className: 'col-md-12',
		templateOptions: {
    	label: 'Select exemption(s) applicable for this parcel:',
		options: [
		{
				 	"name": "IFT (P.A. 198) Exemption", "value": "01"
			},
			{
				 	"name": "Michigan Strategic Fund (MSF) Resolution", "value": "02"
			},
			{
				 	"name": $sce.trustAsHtml('P.A. 328 Exemption'), "value": "03"
			},
			{
				 	"name": "Renaissance Zone Reduction Exemption", "value": "04"
			}
		]
	}
}

Hi @austinstudio! Thanks for your interest in angular-formly. I'm sure that angular-formly can do this for you. The best way to get help is by following the instructions here: http://help.angular-formly.com
I'm going to go ahead and close this issue because the github issues for this project are reserved for bug reports and feature requests. See you on chat! Thanks!


Endpoint to send notification

We have the need to notify data curation/ingest staff of events during the data ingest process.  The call to this endpoint will require a Globus Auth token with general write access (commons auth_helper.has_write).  Add a POST endpoint that takes the following information:

base message
channel name
And posts a message to the specified HuBMAP Slack channel.  The posted message should be the concatenation of "From : " + .  Where  is extracted from the user's credentials via the  auth token and the  is from the request.

curl -X POST -H 'Content-type: application/json' --data '{"text":"This is a testing message via Webhooks"}' https://hooks.slack.com/services/xxx/yyy/zzz


Search by product Tags

Hello. Tags  needs to be in bagisto for search enhancement.  I think  currently search is made by product names only. If add search by tags, finding products will be much easier for customers. They search by generic keywords like books, mobile .... more than product names.

This is not planned as of now but you can implement this feature in a seperate module


Consistency with team-member titles

#First GH issue!!  Starting out small...
Comparing titles for Dave & Diana, I see differences in:
Format: "title1 and title2" versus "title1 (title2)"
Tone: "serious and silly" versus "silly (serious)"
Not sure if these are intentional?  I think the format for Dave is preferable.

Fixed. See https://github.com/e-lo/fast-trips-project/commit/637c4a1d0d011892a8085b6f7b240ffe92baf415


The lap numbers are not aligned

When writing the <length> as <lengthAsLaps> then the resulting HTML file has some layout error.
The error is visible in the squared-square-program.html

The numbers should be right aligned.

added fix to main

There are some random numbers at the end of the program now.

Here is an example of the random numbers at the bottom. It's probably just some leftover debug code.

forgot to remove after testing
should be all good now

tested and works. Great work!


Accidentally went off dev mode and it restore my chromebook and deleted chroot

Please paste the output of the following command here: sudo edit-chroot -all (Dont have chroot anymore, havent tried to install due to the following issue)

Please describe your issue:
My chromebook died and when plugged back on the screen with "OS verification is off - press space to re-enable" came up (which i understand is normal on developer mode) and somebody pressed space, it immediately said that it was deleting all my files and bringing it back to factory. The big problem is that i had very important and sensitive files saved on the linux chroot that i had installed. Is there a way to restore those files?? or bring back the chroot normally?
Thanks in advance.
If known, describe the steps to reproduce the issue:

@DavidVUSA,
Please refer to the response I gave you on Crouton Central -

Accidentally went off dev mode and it restore my chromebook and deleted chroot

Hope this helps,
-DennisL


'With well' routes being scheduled but not followed

I've noticed over the last few weeks that the trolley routes have frequently been the 'the well' variant, but the trolleys aren't actually going over there, they're just going straight down main. Who can we let know about this? I think we've imported the schedule for the arena into our back-end under the assumption that the trolley will be going over there for every event. Are we sure this is the case? Or do we have rouge drivers?
It could be that I was checking the tracker when the event was over/before it had begun/etc... but if it's confusing enough to confuse me, it's confusing enough to confuse our users :)

I pulled the routes for Saturday and found that they did go over to the well at times, but I didn't break those out to see when that was.    They must be following the event schedules more closely than what is on the schedule.
Officially, the trolley cannot make a direct path from the parking lot to The Well, and I see that the drivers are now doing that by running the entire route when they go to the Well.
Officially, the trolley is supposed to skip 'The Towers' stop when it goes to The Well, but they all seem to circle back around and pick it up.


[BUG] Supbixel Rendering does not work on Linux

Description
Subpixel rendering does not work on Linux. The equivalent C++ does work as expected.
Code
C#
var imageInfo = new SKImageInfo(
    width: 100, 40,
    colorType: SKColorType.Bgra8888,
    alphaType: SKAlphaType.Premul);
var props = new SKSurfaceProperties(SKSurfacePropsFlags.None, SKPixelGeometry.BgrHorizontal);
var surface = SKSurface.Create(imageInfo, props);
var canvas = surface.Canvas;

var paint = new SKPaint();
var font = new SKFont {Subpixel = true, Edging = SKFontEdging.SubpixelAntialias, Size = 12};
canvas.Clear(SKColors.White);
canvas.DrawText("Hello World", 10, 20, font, paint);
using (var image = surface.Snapshot())
using (var data = image.Encode())
{
    File.WriteAllBytes("b.png", data.ToArray());
}

C++
auto info = SkImageInfo::Make(100, 40,
                              SkColorType::kBGRA_8888_SkColorType,
                              SkAlphaType::kPremul_SkAlphaType);
SkSurfaceProps props {0, SkPixelGeometry::kBGR_H_SkPixelGeometry};
auto surface = SkSurface::MakeRaster(info, &props);
auto *canvas = surface->getCanvas();

SkPaint paint;
SkFont font;
font.setSubpixel(true);
font.setEdging(SkFont::Edging::kSubpixelAntiAlias);
font.setSize(12);
canvas->clear(SK_ColorWHITE);
canvas->drawString("Hello World", 10, 20, font, paint);

auto img = surface->makeImageSnapshot();
auto data = img->encodeToData();

std::ofstream fout;
fout.open("a.png", std::ios::binary | std::ios::out);
fout.write(static_cast<const char *>(data->data()), data->size());

Expected Behavior
Output from from C++ (with subpixel AA):

Actual Behavior
Output From C# (no subpixel AA):

Basic Information

Version with issue:  2.80.3-preview.40
Last known good version:  Not known; tested also 1.68.3, which had the same problem

Works correctly on Windows

IDE:  Rider
Target Devices:  Arch Linux PC

Code for 1.68.3:
paint.SubpixelText = true;
paint.LcdRenderText = true;
paint.IsAntialias = true;
canvas.Clear(SKColors.White);
canvas.DrawText("Hello World", 10, 20, paint);

After further investigation, the issue must lie in the freetype2build inside skia.
When building skia with skia_use_system_freetype2=true, subpixels are rendered correctly (mono/skia requires the skia.patch.txt).

Ah it looks like mono/skia does not yet have the patch for harmony subpixel rendering (https://skia.googlesource.com/skia/+/f25787b72c20e97cdeb74e037dc1ff56a88b45c6).
For compiling, one could also enable cleartype (FT_CONFIG_OPTION_SUBPIXEL_RENDERING), but that apparently is using patented technology. The simplest fix would be to update skia or use the system freetype2.

Updating freetype2 should be a one-liner: https://github.com/mono/skia/blob/173debd238b1a55ab12f5994c5dc074fd141ccd6/DEPS#L15

Yes, but the patch linked above (https://skia.googlesource.com/skia/+/f25787b72c20e97cdeb74e037dc1ff56a88b45c6^!/#F0)  also needs to be applied. However, it apparently is now fine to enable cleartype (https://www.phoronix.com/scan.php?page=news_item&px=Fedora-ClearType-Subpixel-Font) so that is what I will do for myself.


Vincenty.rs and Graph Search Algorithms

This repository features the Vincenty inverse formula for precise geodesic calculations in Rust, along with implementations of DFS, BFS, and A* search algorithms for graph exploration and pathfinding.

dont forget to add hacktoberfest-accepted label


Make output properties TRAPI compliant

Right now the p-values and stuff returned from AC I think are not in full trapi compliance?
{'node_bindings': {'n1': [{'id': 'NCBIGene:3778'}],
  'n0': [{'id': 'PUBCHEM.COMPOUND:5994',
    'coalescence_method': 'property_enrichment',
    'p_values': [5.515487725071365e-06],
    'properties': ['molecule_type:Small molecule']},

i will correct.

The TRAPI 1.1 spec does not specify the "attributes" property for result node bindings. the spec is generous is generous in this case and will allow basically anything you put in there.
however, node normalization currently fails when a non-hashable type (in this case the attributes dict) is encountered while merging node data. in addition, node norm does not normalize any curie values in the results. Node normalization will have to be modified if this change is implemented.
it is not known at this time if any other downstream components would have an issue here as well.

I think it makes sense to robustify nn to handle properties of this type, and to update the binding properties to match other trapi properties, even if it is not required.

understood, thanks!

What we're doing is TRAPI compliant, but we would like to make TRAPI require us to include standards around this part of the message. Phil is creating an issue in the TRAPI repo to add standards around attributes in this part of the message.


update xstreams to 1.4.15 (from 1.4.14)

Description
within the current xstream version 1.4.14 two more vulnerabilities were found. These are fixed with the update to 1.4.15.

CVE-2020-26258 (Server-Side Forgery Request)
CVE-2020-26259 (arbitrary file deletion)

Motivation and Context
Fix potential security problems
How Has This Been Tested?
run gradlew check, first run failed with one library (xstream) having changed as expected, rerun with -PupdateExpectedJars switch.
The following executions of gradlew check and gradlew test succeeded now.
Screenshots (if appropriate):
none
Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist:

[x] My code follows the code style of this project.
[x] I have updated the documentation accordingly.

Codecov Report

Merging #639 (0e8c999) into master (53c6db8) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #639      +/-   ##
============================================
- Coverage     55.42%   55.42%   -0.01%
+ Complexity    10132    10131       -1
============================================
  Files          1041     1041
  Lines         63951    63951
  Branches       7226     7226
============================================
- Hits          35447    35446       -1
  Misses        26009    26009
- Partials       2495     2496       +1

Impacted Files
Coverage Δ
Complexity Δ

...n/java/org/apache/jmeter/reporters/Summariser.java
84.73% <0.00%> (-0.77%)
17.00% <0.00%> (-1.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53c6db8...0e8c999. Read the comment docs.


Fix a link to leadingTrivia

I noticed that the link to TokenSyntax/leadingTrivia on https://swiftpackageindex.com/apple/swift-syntax/main/documentation/swiftsyntax/working-with-swiftsyntax is incorrect.

@swift-ci Please test


[SPARK-30178][ML] RobustScaler support large numFeatures

What changes were proposed in this pull request?
compute the medians/ranges more distributedly
Why are the changes needed?
It is a bottleneck to collect the whole Array[QuantileSummaries] from executors,
since a QuantileSummaries is a large object, which maintains arrays of large sizes 10k(defaultCompressThreshold)/50k(defaultHeadSize).
In Spark-Shell with default params, I processed a dataset with numFeatures=69,200, and existing impl fail due to OOM.
After this PR, it will sucessfuly fit the model.
Does this PR introduce any user-facing change?
No
How was this patch tested?
existing testsuites

Test build #115013 has started for PR 26803 at commit 9ddad92.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19833/
Test PASSed.

Test build #115015 has started for PR 26803 at commit 3807027.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19835/
Test PASSed.

Test build #115013 has finished for PR 26803 at commit 9ddad92.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115013/
Test PASSed.

Test build #115015 has finished for PR 26803 at commit 3807027.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115015/
Test PASSed.

Test build #115073 has started for PR 26803 at commit dab3d3a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19889/
Test PASSed.

Test build #115073 has finished for PR 26803 at commit dab3d3a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115073/
Test PASSed.

Test build #115099 has started for PR 26803 at commit 51f833c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19913/
Test PASSed.

Test build #115099 has finished for PR 26803 at commit 51f833c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115099/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19958/
Test PASSed.

Test build #115147 has started for PR 26803 at commit 5283294.

Test build #115147 has finished for PR 26803 at commit 5283294.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115147/
Test PASSed.

@srowen I impl a simple treeAggregateByKey here, and made several local tests like:
val rdd = sc.range(0, 10000, 1, 100)
val rdd2 = rdd.map{i => (i % 10, i)}
val rdd3 = rdd2.treeAggregateByKey(0.0, new HashPartitioner(3))(_+_, _+_, 2)
rdd3.collect

and ran successfully.

BTW, it is reasonable to call compress before reduce tasks, so maybe a aggregateByKeyWithinPartitions is needed? Then we can call compress after locally aggregation within each partition. I guess they maybe common functions and we add this method in RDD/PairRDD?

Test build #115185 has started for PR 26803 at commit a94d5d6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19997/
Test PASSed.

Test build #115185 has finished for PR 26803 at commit a94d5d6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115185/
Test PASSed.

test code:
import org.apache.spark.ml.linalg._
import org.apache.spark.ml.feature._
import org.apache.spark.storage.StorageLevel

val rdd = sc.range(0, 10000000, 1, 100)
val df = rdd.map(i => Tuple1.apply(Vectors.dense((i % 1000).toDouble / 1000))).toDF("features")
df.persist(StorageLevel.MEMORY_AND_DISK)
df.count

val scaler = new RobustScaler().setInputCol("features")

val start = System.currentTimeMillis; Seq.range(0, 100).foreach{_ => val model = scaler.fit(df)}; val end = System.currentTimeMillis

end - start

Master: 243493
This PR: 285341
I test an edge case with only numFeatures=1, and existing impl is about 17% faster then this PR.
That is to say this PR will support medium/large (>1000) numFeatures at the cost of some performance regression on low-dim cases.
Or we check the numFeatures at first, and decide which method to use?

Moreover, for existing impl, we can use reduceByKey/aggregateByKey with feature index as a key to generate a RDD[QuantileSummaries] and then compute and fetch the range/median instead of the large Array[QuantileSummaries], as @srowen suggested.

Test build #115438 has started for PR 26803 at commit a4d4ef4.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20241/
Test PASSed.

Test build #115438 has finished for PR 26803 at commit a4d4ef4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115438/
Test PASSed.

Test build #115486 has started for PR 26803 at commit 8ef2b1b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20289/
Test PASSed.

Test build #115486 has finished for PR 26803 at commit 8ef2b1b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115486/
Test PASSed.

Test build #115536 has started for PR 26803 at commit 6c37c06.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20339/
Test PASSed.

Test build #115536 has finished for PR 26803 at commit 6c37c06.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115536/
Test PASSed.

Test build #115562 has started for PR 26803 at commit 4572b34.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20362/
Test PASSed.

Test build #115562 has finished for PR 26803 at commit 4572b34.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115562/
Test PASSed.

Test build #115610 has started for PR 26803 at commit 03b9bd0.

this still kind of has the 'skew' problem for few features.

I think we can resolve it by adding new RDD ops, maybe my previous treeAggregateByKey is a choice.
I will test it in more practice cases, if it help improve perfermance, I will try to add it to PairRDDFunctions

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20409/
Test PASSed.

Test build #115610 has finished for PR 26803 at commit 03b9bd0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115610/
Test PASSed.

@srowen If you do not object, I will merge this in one or two days.
I will try to optimize the performance by including new ops for PairRDD.

@srowen OK, I will revert the last commits and add a testsuite for high-dim dataset.

Test build #115686 has started for PR 26803 at commit be1051b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20482/
Test PASSed.

Test build #115686 has finished for PR 26803 at commit be1051b.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115686/
Test FAILed.

retest this please

Test build #115724 has started for PR 26803 at commit be1051b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20518/
Test PASSed.

Test build #115724 has finished for PR 26803 at commit be1051b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115724/
Test PASSed.

Merged to master, thanks @srowen for reviewing!


serde_json documentation linked in README out of date

It looks like the documentation is for an older version of serde_json.
At least one difference are the variants of "Error" between the documentation:
https://serde-rs.github.io/serde/serde_json/serde_json/error/enum.Error.html
and the latest version of serde_json (0.6.0):
https://github.com/serde-rs/json/blob/master/json/src/error.rs#L102

Hello! Oops, that link to serde_json is quite old. I'll remove it. Try this one instead.


chore(metadata-service): add more projects to db (BS-39)

resolves BS-39
First 2 commits are about previous mobile first task - few minor things needed to be updated.
I've went through all projects data we have and added all of them only to the projects endpoint. These which I was able to get the JSON only were added to metadata endpoint - we don't want to see JSON-LD format here right? ;)
Btw. I think both endpoints we have could be easily combined into one. Do you agree?

I agree, the two endpoints could easily be combined. However, I'm not sure they should be combined.
I kept them separate in order to mimic the API as closely as possible, that we'll eventually have to query (without really knowing how that API will look like). From my understanding, we'll eventually have to get the projects from the v2/projects endpoint, but the metadata itself from the newly created metadata service.
That's why I kept them separate. But I'm not sure that makes any sense... What do you think? And maybe @subotic has an opinion on that too?

I agree, the two endpoints could easily be combined. However, I'm not sure they should be combined.
I kept them separate in order to mimic the API as closely as possible, that we'll eventually have to query (without really knowing how that API will look like). From my understanding, we'll eventually have to get the projects from the v2/projects endpoint, but the metadata itself from the newly created metadata service.
That's why I kept them separate. But I'm not sure that makes any sense... What do you think? And maybe @subotic has an opinion on that too?

I think about merge only in the scope of our fake BE, which is the temporal solution. I think we will not use both endpoints and soon we will have the real BE implementation, so I would like to simplify it.

I agree, the two endpoints could easily be combined. However, I'm not sure they should be combined.
I kept them separate in order to mimic the API as closely as possible, that we'll eventually have to query (without really knowing how that API will look like). From my understanding, we'll eventually have to get the projects from the v2/projects endpoint, but the metadata itself from the newly created metadata service.
That's why I kept them separate. But I'm not sure that makes any sense... What do you think? And maybe @subotic has an opinion on that too?

I think about merge only in the scope of our fake BE, which is the temporal solution. I think we will not use both endpoints and soon we will have the real BE implementation, so I would like to simplify it.

Fine by me. The simpler, the better!

I agree, the two endpoints could easily be combined. However, I'm not sure they should be combined.
I kept them separate in order to mimic the API as closely as possible, that we'll eventually have to query (without really knowing how that API will look like). From my understanding, we'll eventually have to get the projects from the v2/projects endpoint, but the metadata itself from the newly created metadata service.
That's why I kept them separate. But I'm not sure that makes any sense... What do you think? And maybe @subotic has an opinion on that too?

A bit late to the party, but here is my opinion on this. Define the endpoints in such a way, so that it is easiest to be used by the frontend. It doesn't matter from where what data will come from.


[BEAM-5486] GCSIO: Allow empty object prefix in list_prefix().

Allows listing a whole bucket, e.g.,
Filesystems.match(['gs://bucket/*'])

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.


How to disable mouse?

Hi, I'd like to completely disable mouse input. Using Linux/Mac, I believe with ncurses driver. Is it possible?

Done for all drivers.

Works perfectly, thank you :)


Chapter1/homework/3

Code refactored, tests passed.
I'm sure there must be a better way to get user input than having one method for each variable type, but would that be overcomplicating things at this stage?

Made requested changed to the code, have also added comments to clarify some things.