Binary size could be smaller

 Issue by smklein
Friday Oct 19, 2018 at 21:59 GMT
Originally opened as https://github.com/clap-rs/clap/issues/1365

Rust Version
rustc 1.31.0-nightly (e7f5d4805 2018-10-18)
Affected Version of clap
2.32.0
Bug or Feature Request Summary
I compared the binary size of a "hello world" application, with and without clap, and came up with the following data:
Using a Cargo.toml:
  [profile.release]
  opt-level = 3
  lto = true
  panic ='abort'

And a main.rs (of either this, or a version which only calls println!("Hello world"):
#![feature(alloc_system)]
extern crate alloc_system;
extern crate clap;
use clap::{App, Arg};

fn main() {
    let matches = App::new("Hello")
                    .arg(Arg::with_name("foo")
                         .short("f")
                         .takes_value(true))
                    .get_matches();
    let foo = matches.value_of("foo").unwrap();
    println!("Hello, world: {}", foo);                                                                                }

I ran the following:
$ cargo rustc --release -- -C link-args=-static && strip target/release/hello
The "Non-clap" version of hello world resulted in the following output from bloaty:
     VM SIZE                      FILE SIZE
 --------------                --------------
  78.3%   283Ki .text            283Ki  78.3%
   8.2%  29.8Ki .rodata         29.8Ki   8.2%
   6.0%  21.7Ki .eh_frame       21.7Ki   6.0%
   3.7%  13.6Ki .data.rel.ro    13.7Ki   3.8%
   1.1%  4.16Ki .bss                 0   0.0%
   0.2%     568 [ELF Headers]   2.37Ki   0.7%
   0.6%  2.16Ki .dynsym         2.16Ki   0.6%
   0.0%       0 [Unmapped]      1.91Ki   0.5%
   0.4%  1.36Ki .dynstr         1.36Ki   0.4%
   0.3%  1.17Ki .rela.dyn       1.17Ki   0.3%
   0.2%     712 .got               712   0.2%
   0.2%     672 .rela.plt          672   0.2%
   0.2%     576 .dynamic           576   0.2%
   0.1%     534 [12 Others]        525   0.1%
   0.1%     464 .plt               464   0.1%
   0.1%     416 .gnu.hash          416   0.1%
   0.1%     320 .gnu.version_r     320   0.1%
   0.1%     272 .data              272   0.1%
   0.0%       0 .shstrtab          250   0.1%
   0.0%     184 .gnu.version       184   0.0%
   0.0%     152 .plt.got           152   0.0%
 100.0%   362Ki TOTAL            362Ki 100.0%

Where the clap version resulted in the following:
     VM SIZE                      FILE SIZE
 --------------                --------------
  82.6%   620Ki .text            620Ki  82.7%
   7.0%  52.6Ki .rodata         52.6Ki   7.0%
   5.4%  40.3Ki .eh_frame       40.3Ki   5.4%
   3.2%  23.8Ki .data.rel.ro    23.9Ki   3.2%
   0.6%  4.29Ki .bss                 0   0.0%
   0.1%     568 [ELF Headers]   2.37Ki   0.3%
   0.3%  2.27Ki .dynsym         2.27Ki   0.3%
   0.2%  1.41Ki .rela.dyn       1.41Ki   0.2%
   0.2%  1.38Ki .dynstr         1.38Ki   0.2%
   0.0%       0 [Unmapped]      1.12Ki   0.1%
   0.1%     760 .got               760   0.1%
   0.1%     576 .dynamic           576   0.1%
   0.1%     576 .rela.plt          576   0.1%
   0.1%     542 [12 Others]        501   0.1%
   0.1%     476 .gnu.hash          476   0.1%
   0.1%     400 .plt               400   0.1%
   0.0%     320 .gnu.version_r     320   0.0%
   0.0%     272 .data              272   0.0%
   0.0%       0 .shstrtab          250   0.0%
   0.0%     194 .gnu.version       194   0.0%
   0.0%     184 .plt.got           184   0.0%
 100.0%   751Ki TOTAL            750Ki 100.0%

(Diff for visibility)
     VM SIZE                    FILE SIZE
 --------------              --------------
  +119%  +336Ki .text         +336Ki  +119%
   +77% +22.8Ki .rodata      +22.8Ki   +77%
   +85% +18.6Ki .eh_frame    +18.6Ki   +85%
   +75% +10.2Ki .data.rel.ro +10.2Ki   +75%
   +20%    +240 .rela.dyn       +240   +20%
  +3.0%    +128 .bss               0  [ = ]
  +5.4%    +120 .dynsym         +120  +5.4%
   +14%     +60 .gnu.hash        +60   +14%
  +6.7%     +48 .got             +48  +6.7%
   +21%     +32 .plt.got         +32   +21%
   +24%     +32 .tbss              0  [ = ]
  +1.9%     +26 .dynstr          +26  +1.9%
  +5.4%     +10 .gnu.version     +10  +5.4%
 -28.9%     -24 [LOAD [RX]]      -24 -28.9%
 -13.8%     -64 .plt             -64 -13.8%
 -14.3%     -96 .rela.plt        -96 -14.3%
  [ = ]       0 [Unmapped]      -808 -41.2%
  +107%  +388Ki TOTAL         +388Ki  +107%

The amount of .text being generated here doubles the size of the binary.
Clap is really useful, and I'd love if I could use it on more constrained environments, where binary size might be expensive.
Chatting with @kbknapp , it seems that there are some changes to clap-rs underway to allow more flexible usage of the crate, potentially disabling some features at compile-time to shrink the binary, but I figured I'd file this bug to track.
Are there currently other ways of toggling clap features, to compare binary sizes for this evaluation?

 Comment by kbknapp
Saturday May 02, 2020 at 01:19 GMT

Re-energizing this topic. Not necessarily for action prior to 3.0 release, but something to keep in our mind and begin to tackle as available.
My comment from #1891

Along those same lines but also a separate issue entirely, at some point I would like to have the discussion about aggressive cargo feature use. Ideally, all non-essential sections of clap could be disabled (but enabled by default) to allow small binary sizes when desired. I've had quite a few people / companies approach me about clap's binary size being their only issue. In fact, I believe that is the number one factor that led Google to work on argh, was binary size is their top concern (for Fuschia) and don't need a lot of the "extra" features that clap provides.

@CreepySkeleton's comment from that same issue:

On the other hand, people from this anti-dependency camp likely see clap as bloat-full and having too much allegedly unneeded deps irrespective of term_size. Clap already pulls 7 crates in --no-default-features mode, 22 in default-features mode, and 27 in all-features mode; one extra crate - very tiny and very useful btw - won't change their opinion. It's kind of all-or-nothing situation, 8 deps (including transitive) kind of count as "as small as possible". I really doubt any of them would become our user if we feature-gate one or two tiny crates.
That's not to mention that clap itself takes as much time to build as (almost) all of the deps combined, and the compilation time is a much more common complaint among users (in my view). Along with code bloat, which would only decrease without this feature.
Regarding modularization of clap.
I've been thinking about it too, for some time, and I've come to conclusion that, while it's doable - to some extent, it would require full-scale refactoring. And by full-scale I mean "rewrite the parser and the validator entirely" which is quite an endeavor.

Finally, @Dylan-DPC's comment from that issue:

Given how popular clap is, the number of dependencies is definitely a huge concern. I've been planing of tackling a lighter clap in some form or the other post the 3.0 release. That discussion is beyond the scope of this issue.

 Comment by kbknapp
Saturday May 02, 2020 at 01:42 GMT

Just so my thoughts are known, I don't believe we need to remove all dependencies and features for the smallest possible size. What I do think is that we should know which dependencies have a low "weight to functionality" ratio.
"Weight" can assessed in two manners, code size added to clap, and compile time.
"Functionality" should be self evident.
I am definitely not in the camp of "make sure there are as few deps as possible." However, I am in the camp of I don't want to take on a dep that doesn't provide intrinsic value, or is a maintenance burden.
The harder crates to evaluate will be ones that don't add a lot to clap functionally, but at the same time don't add much if any compile time or code size either. I would propose for those style crates we look at either pulling the code internal, or dropping the crate.
I'm most concerned with the --no-default-features case, that case should be as small as possible, with as few high ratio deps as possible. I'm not concerned with the default features, or even --all-features cases, because at that point the consumer as chosen to go a heavier weight route, while there are lighter weight options.
This would be a great first issue, as it's not a lot of code changing, and doesn't require too much in depth knowledge of clap. At first, it will primarily be measuring the default crates, or seeing how widespread their use inside clap is.

Taking a quick look at our default crates I see:

Crate (+ transitive deps)
Functionality
Notes

bitflags
Underpin of AppSettings

unicode-width
Heavily used in Help Messages
If we made auto generated help optional, this could become an optional feature

textwrap
Used for Help Messages
same as unicode-width

indexmap (autocfg)
used in ArgMatcher/ArgMatches
Perhaps we could use vec_map instead

os_str_bytes
used in parsing
clap fundamentally messes with OsStrs or bytes heavily...so this is probably well worth the dep

vec_map
Used throughout Arg and usage parser
Used to be an optional feature. Has perf gains over BTreeMap...maybe those gains need to be evaluated again to see if they're worth it

 Comment by kbknapp
Saturday May 02, 2020 at 02:03 GMT

Doing a --no-default-feautres --features std build on a simple:
App::new("foo").get_matches();

Gives us this output via cargo bloat  in release mode:
cargo bloat --crates --release
[..]
    Analyzing target/release/fake

 File  .text     Size Crate
 9.3%  56.5% 285.5KiB clap
 4.8%  29.3% 148.1KiB std
 1.9%  11.7%  59.2KiB [Unknown]
 0.1%   0.5%   2.6KiB indexmap
 0.1%   0.3%   1.6KiB textwrap
 0.0%   0.1%     307B fake
 0.0%   0.0%      41B os_str_bytes
16.5% 100.0% 505.5KiB .text section size, the file size is 3.0MiB

Note: numbers above are a result of guesswork. They are not 100% correct and never will be.

Discounting clap and std, it looks like textwrap and indexmap need a close eye. If either has better alternatives, or is simple enough to not use/pull internal I'd say those may be on the chopping block.
Also, if we did something drastic like make auto help messages optional, that alone would cut a huge portion of the code along with size. Additionally and oddly enough, that may the easiest thing to modularize first.
"But wait!" I can hear you say. "Auto help generation is like THE thing clap is used for, right?" Sure. It's one of the things, but I'd argue not the most important at all. In many of the contexts where code size matters the most, help generation is  one the things they don't care about because it's super easy to just write a help string manually for a small CLI.

 Comment by CreepySkeleton
Saturday May 02, 2020 at 02:07 GMT

Before we more any further, would you please test it in release mode 😄? Nobody cares about debug binaries, release the  ones of concern here.

 Comment by kbknapp
Saturday May 02, 2020 at 02:08 GMT

Literally just edited the comment with that info :laughing:

 Comment by pksunkara
Saturday May 02, 2020 at 06:27 GMT

Did you enable lto? I think we should assume with that.

 Comment by CreepySkeleton
Saturday May 02, 2020 at 13:47 GMT

main.rs
use clap::*;

fn main() {
    App::new("app").get_matches();
}

With lto enabled (release build, no default features)
CreepySkeleton@builder:~/workspace/probe$ cargo bloat --crates --release
    [...]

 File  .text     Size Crate
19.4%  60.3% 277.7KiB clap
 9.0%  27.9% 128.5KiB std
 3.0%   9.2%  42.3KiB [Unknown]
 0.3%   1.0%   4.7KiB probe
 0.2%   0.6%   2.5KiB indexmap
 0.0%   0.1%     394B textwrap
32.2% 100.0% 460.3KiB .text section size, the file size is 1.4MiB

What interesting here is that the size of the main.rs crate increased from 307B to 4.7KiB. Basically, some code from std has been inlined and this is the biggest difference.
The overall difference is less than 10Kb (clap and its deps only) and it will thin as more features are actually used. All the further runs will be performed without LTO.
See also https://stackoverflow.com/a/52297790. TL;DR: LTO is not about reducing code size. z opt level is about it.
Another interesting read: https://internals.rust-lang.org/t/rust-staticlibs-and-optimizing-for-size/5746. No TL;DR here, go read, it's worthy.

Kevin, just to make sure we're on the same page: I wasn't counting you as a no-deps guy, I was just thinking aloud "how would we sell clap to those guys" and I'm pretty sure the answer is - we wouldn't.
If their wariness comes from concern that the deps affect binary size, we can righteously tell them that all the deps put together (in --no-default-features mode) take less than 5KiB, the next rustc release will likely change much more of that. The same can be said about compilation time: ~2sec difference, essentially a rounding error.
At the very least, if 5KiB do mater to you, you're working on embedded, and clap will likely never suit you. Take a look at getopts/argh or write it yourself.
I agree regarding help messages. It also drags error messages along.
What not clear to me is your logic about "either pulling the code internal, or dropping the crate". It sounds like "Let's copy-paste code from third party crates (we can't just drop them) in order to decrease our maintenance burden". But from that point on, we will be the ones who maintain this copy-pasted code. A contradiction. Could you please elaborate on that?

 Comment by pksunkara
Saturday May 02, 2020 at 15:27 GMT

At the very least, if 5KiB do mater to you, you're working on embedded, and clap will likely never suit you. Take a look at getopts/argh or write it yourself.

I don't think we should be saying that. We should be using cargo feature flags to make sure all the fields can use clap.

 Comment by TeXitoi
Saturday May 02, 2020 at 15:43 GMT

To make it work on embedded, you'll need do be alloc free (usable wIthout, String, Vec and such kind of collection).

 Comment by CreepySkeleton
Saturday May 02, 2020 at 16:44 GMT

To clarify my statement about embedded:

On something like Rabspery PI clap will fly marvelously; you won't even need no-std there. It's close to a real PC after all.
If you have a chip with megabytes of ROM on board, you may be able to squeeze clap in. no_std, alloc, and accurate fine tuning (see modularization) will be required.
128 KiB is all you've got? Too bad! Take a look at getopts (and you probably won't be able to afford even that).

To summarize all of that, you either have an extra hundred of KiBs at your disposal (and this is why I think we shouldn't optimize every single KiB) or clap is not for you.
And a bit of personal opinion: as someone with a (very limited) experience with embedded, I can say that allocations are almost tabu in there. Given that clap relies on Vec and stuff very heavily... I won't even try it if I need a CLI parser for embed. I'll go grab getopts.

 Comment by Dylan-DPC
Saturday May 02, 2020 at 15:57 GMT

There's a post 3.0 plan for that already which is why added the no_std feature

 Comment by TeXitoi
Saturday May 02, 2020 at 17:04 GMT

Even getopts requires Vec and String. I personally don't see why I would need clap in embedded, that's a very niche usecase.

 Comment by TeXitoi
Saturday May 02, 2020 at 17:05 GMT

(embedded = bare metal on microcontroller)

 Comment by kbknapp
Saturday May 02, 2020 at 23:36 GMT

A lot to reply to here :) I don't think we need to be concerned with embedded (bare metal) right now. Maybe one day, but there are many things we can do that will improve clap before we worry about embedded.
I'm more concerned with embedded like, i.e. resource constrained devices, like small arm based (Pi, SBC, phone, etc.). I spoke with some Google employees who were looking at clap for that exact use case (presumably Fuschia), and binary size was their concern. Binary size isn't everyone's concern. To be clear, I'm not trying to supplant argh, but I do think we should be cognizant of that use case while building clap.
We're in a little bit of a catch-22 scenario. Many people assume argument parsing is such simple task that the deps should be ultra small and no transitive deps. That's understandable why one would think that. However, as all of us know, it turns out argument parsing is somewhat of a hidden beast. There are edge cases EVERYWHERE, and a near endless amount of features to implement.
It turns out argument parsing is complex.
Having said that, the issue we're discussing conflates a few concerns that we should probably be more explicit about:

Binary size of clap
Compile times of clap
Adoption and Perception of clap

Both binary size and compile times are affected by:

Deps
clap features and code structure
Rust and LLVM compile time features (LTO, dead code elimination, monomorphization, etc.)

While adoption and perception can be heavily influenced by binary size, compile times, and the dep graph as it's own construct.
So long as we're doing our due diligence when it comes to crates we take on as dependencies to ensure they're worth the price we pay in terms of binary size and compile times, we're good. Alternatively, if they negatively affect those areas, we should do our best to ensure they're optional.
Meanwhile, we should be thinking about the dep graph as it's own construct because right, wrong, or indifferent people do care about how many crates they pull in. Unfortunately, everyone's reason for caring can be different, for some it's binary size, for some it's compile times, for others it's the perception of risk (what happens if a crates has a security flaw, or goes away over night?).
We're addressing the first two concerns already, but the third (perception of risk) is primarily addressed by either using ultra popular, well supported crates, or crates related to/owned by the parent (which is usually signalled by the name, i.e. clap_* for us).

"how would we sell clap to those guys" and I'm pretty sure the answer is - we wouldn't.

You're correct in that part of the answer is, "we don't." But I think we mean different things by this. I mean we "We don't try to sell them on clap, we just do the best we can to care about that scenario."
We can do things like making clap modular enough, or structured in a way to turn off features the end user doesn't care about. This has an added benefit of potentially being able to make most of our deps optional.
And where they can't be option, oh well. At least we're trying. clap isn't going to fit every situation and that's fine. But we should at least do what we can to be an option for even those resource constrained situations.

If their wariness comes from concern that the deps affect binary size, we can righteously tell them that all the deps put together (in --no-default-features mode) take less than 5KiB, the next rustc release will likely change much more of that. The same can be said about compilation time: ~2sec difference, essentially a rounding error.

I totally agree with you. However, the problem is many times we don't get the chance to explain ourselves. They see the dep graph front-and-center and many times simply make assumptions from there. That's the part about the "no dep" crowd I'm not a fan of. I can't count the number of reddit comments, or blog posts I've seen that say, "Can you believe [program X] pulled in 170 crates?! That's absurd!!!" with no backing evidence as to why that's absurd. Were those 170 crates not required? How much compile time did they add, etc.? If the compile times were the EXACT same, but [program X] just copy/pasted the code from those crates, you probably wouldn't hear a peep. In fact, you'd probably hear how amazing it was that [program X] has no dependencies!
...I'm not a fan of that. But it's the world we live in.

What not clear to me is your logic about "either pulling the code internal, or dropping the crate". It sounds like "Let's copy-paste code from third party crates (we can't just drop them) in order to decrease our maintenance burden"

I should have been more clear :stuck_out_tongue_winking_eye: Decreasing our maint burden and decreasing our dep graph are separate concerns that get intertwined. Also taking the code internal is only in the case where we're using a dep for a small well defined  subset of what the crate offers, and we aren't concerned with maint burden part, just the raw number of deps part. This should only be a last resort for us.
This is one of those things that's counter intuitive; copying and pasting a subset of a crate into clap directly, (or even into a new sub crate like clap_foo may actually not make any difference in compile times or binary size. But it will still make many people "feel" better about using clap.
Is it dumb. Yep. But again, this should be a last resort for us and not something I'm actually looking at doing seriously. The only crate that comes to mind for this type of thing is term_size which is technically already a clap owned crate, but again the name doesn't signal that so when people compile clap they just assume it's some rando crate we pulled in as a dep. Am I suggesting re-naming term_size to clap_foo? Nope. Not unless we were in drastic territory and have exhausted all other options. We're nowhere near that! :smile:

 Comment by pksunkara
Tuesday May 12, 2020 at 13:56 GMT

There are some good arguments regarding inclusion of deps in this reddit thread today, https://www.reddit.com/r/rust/comments/gi7v2v/is_it_wrong_of_me_to_think_that_rust_crates_have/

 Comment by BurntSushi
Tuesday May 12, 2020 at 15:18 GMT

@kbknapp invited me to participate in this issue, but I have to say, from reading the comments, it seems like there's an undercurrent of counter-productive "us vs them." If I'm coming from (your perspective) the "anti-dependency" camp and you already believe that you can't sell clap to me

Kevin, just to make sure we're on the same page: I wasn't counting you as a no-deps guy, I was just thinking aloud "how would we sell clap to those guys" and I'm pretty sure the answer is - we wouldn't.

then is it really worth my time to participate here if I'm treated as a lost cause in the first place?
reddit and low effort comments are a pit, and people love to make sweeping generalizations that makes the world look more extreme than it is. In reality, I think most people exercise good judgment and try to straddle this line as best as they can. Sometimes I'm on the "anti-dependency" side of the fence and take steps to reduce my dependency tree. But other time, I'm on the "yeah dependencies are great" side and try to resist the urge to remove dependencies.
I think @kbknapp has the right attitude here. The struggle is the important part. Doing due diligence on dependencies before bringing them in, and really weighing their costs and benefits, is what's important. And to me, that's what I see myself doing when I notice that a new dependency is about to come into my tree. What I see is a prior state in which I could opt out, but now I can't, and I don't know why. The code that was deleted looked pretty small, its commit history suggests it was very lightly edited and its performance was more than good enough for me. Moreover, when I first dropped the vec-map dependency from my use of Clap, I noticed some really crazy compile time improvements. I suspect there was a bug somewhere or some weird case, but still, the flexibility to drop dependencies was really nice.
I'm not familiar with Clap internals, and as someone who has written more than one argument parsing library, I am definitely someone who appreciates their complexity. I never take a good argument parser for granted. They are super hard to build because they need to balance so many competing concerns. IMO, Clap does a great job of this already.
So with that said, I'll give some opinions on Clap 3's required dependencies:

bitflags - I would personally just hand write any bit flags logic that I needed. In my view, this is a crate that optimizes for writing code, which I generally think isn't the right thing to be doing in widely used fundamental crates. So there's a bit of a philosophical opinion involved here.
textwrap - I would make this optional but enabled by default. I write all of my help strings and documentation wrapped to 79 columns anyway, which is probably good enough.
unicode-width - I would make this optional but enabled by default. If my help text is all ASCII, then an assumption that each byte is one column is reasonable.
indexmap - I don't have enough context to know what to do here, but it looks like it was recently added. I assume this is justified somehow through a major performance win, but otherwise, I would drop this.
os_str_bytes - This is definitely better than using unsafe code. If I knew more about why Clap needed this, then perhaps there is a way to get rid of this too. But yeah, if Clap needs to be mucking around with OsStrs, then the lack of an OsStr string API probably makes this really hard to the point where you pretty much have to do what os_str_bytes is doing. (I still wish std would just expose its internal WTF-8 representation, but there are good arguments against doing that.)
vec_map - I don't know the performance story here, but keeping it optional looked easy? But maybe there are some issues I don't know about.

 Comment by kbknapp
Tuesday May 12, 2020 at 15:25 GMT

Thank you for the detailed response! Before I can finish reading the comment and add any thoughts, let me address something real quick as a moderation note:

from reading the comments, it seems like there's an undercurrent of counter-productive "us vs them." If I'm coming from (your perspective) the "anti-dependency" camp and you already believe that you can't sell clap to me

I want to be very clear with everyone reading these messages (especially including the clap team) that this is not what I want to portray or the tone I want to encourage. Internet communications via text are hard, especially when we start considering native languages and such. (Edit: to be even more clear. I don't think @BurntSushi is portraying this tone. He's reading and noticing this tone from previous comments we made. And I should say thank you for pointing this out! :smile: )
I just want to be clear that the clap team is on the same page of, there is no "us vs them" and all opinions are welcome! We have differences of priorities and how to achieve those priorities, but all opinions are valid within this discussion. Ultimately the clap team will have to make a decision on which priority and implementation strategy to go with, but through discussions and issues like these my goal is to take all opinions in, consider them carefully, and make a decision that is best for clap and all consumers (current and future).

...now back to reading :smile:

 Comment by BurntSushi
Tuesday May 12, 2020 at 15:35 GMT

Also, one other note: IIRC, cargo bloat has an option to list binary size by function. That might also be useful to look at for decreasing binary size. Sometimes you can get big wins by tweaking the inlining strategy used, for example. Or using trait objects is sometimes possible, although I've found it difficult to do in practice in many cases because of object safety rules.

 Comment by pksunkara
Tuesday May 12, 2020 at 15:44 GMT

I just want to be clear that the clap team is on the same page of, there is no "us vs them" and all opinions are welcome!

I strongly agree with this.
Unfortunately, I only started using Rust this year (maybe Dec last year), so I missed all the deps related discussions and am still trying to catch up on understanding the pros and cons related to rust ecosystem. That is why I was asking you the recent question on deps in termcolor @BurntSushi.
In Javascript before, while I do try to reduce our usage of deps, I still try to prefer commonly used packages and using them as deps instead of writing all of them myself which was the convention all across the ecosystem.
If we take the case of vec_map, my question is why do we want to make it optional? Shouldn't it be either including it completely or removing it completely? I think this is basic question we want to answer first.

 Comment by kbknapp
Tuesday May 12, 2020 at 16:28 GMT

If we take the case of vec_map, my question is why do we want to make it optional? Shouldn't it be either including it completely or removing it completely? I think this is basic question we want to answer first.

My understanding is there are several concerns that can also overlap with each other, and not everyone has the same concerns. As mentioned above, some of those include:

Binary Size
Compile Time
Transitive Deps (which repeats points 1-2 for each dep)

Adding a dep can have negative impacts on points 1 and 2, or both. Some people care about point 1, some people about point 2, and some about both. So if we consider either point a deps "cost" we have to weigh that cost against it's benefits (usually performance, or lower maintenance burden for us directly).
This is what I was trying to say earlier, we have to weigh any cost (including the subjective ones) against the objective benefit.
The reason for making it optional and not a binary include or not include is because people have various requirements. Some may not care about the binary size, or increased compile time if the feature provided is worth the cost, but since some do care we need a way to allow our consumers the same choices we're making at the library level.
For vec_map which is a self contained crate and no transitive deps, I would want to look at exactly what perf gains we get, compared with binary size/compile times. Add to that, since we pulled in IndexMap, perhaps we can use that instead and remove one or the other entirely? I haven't looked lately, so that may not be feasible.

To address the comments per dep from @BurntSushi :

bitflags - I hadn't really considered just pulling in the logic of bitflags to be honest. ...but now that you say it I'm sure it's actually simple subset since we don't particularly care about the macros, etc. just the functionality. Would probably worth a look at least.
textwrap - the hard part with this one is this dep helps us wrap at word boundaries intelligently. But perhaps making it optional for those that will manually wrap their help strings anyways.
unicode-width - I also like the idea of providing a, "I only use ASCII args/help strings (values excluded), so turn off all extra deps related to handling Unicode logic" flag
indexmap - Might be able to combine with vec_map, or relook at our exact use case. I need to familiarize myself with the code again.
os_str_bytes - This is probably staying for the time being. Clap is pretty much written with idea we use OsStr(ing) internally everywhere, and only assume/use UTF-8 at the consumer demarcation point. I've toyed with the idea of bstr since all we really care about is bytes, but not done enough research to see what overlap, if any, bstr and os_str_bytes have or which would be the best fit.
vec_map - Addressed in other comments. I just want to make sure we're getting a good cost:benefit ratio otherwise we may drop it entirely.

 Comment by BurntSushi
Tuesday May 12, 2020 at 17:49 GMT

I've toyed with the idea of bstr since all we really care about is bytes, but not done enough research to see what overlap, if any, bstr and os_str_bytes have or which would be the best fit.

You probably want to stick with os_str_bytes. os_str_bytes re-implements WTF-8 and gives you 100% correctness on both Unix and Windows. The only help bstr gives you with OS strings are utility functions for converting between &OsStr and &[u8], which is zero cost on Unix but does lossy decoding on Windows, so it sacrifices some correctness. (bstr does not re-implement WTF-8.)
I don't know how hard it would be to avoid doing the equivalent of os_str_bytes while maintaining correctness. It could be impossible. I could imagine some possibilities, like lossily decoding each argument for parsing only, which would work since the "significant" characters in the arg parser I believe are all ASCII. But then making sure you hand back the original OsString to the caller. But this is complex and could potentially be a performance regression. Unless std decides to change its tune and expose an OS string's encoding, os_str_bytes is probably the right answer.

 Comment by pksunkara
Tuesday May 12, 2020 at 17:51 GMT

I think using indexmap in place of vec_map would make the performance worse. And we need indexmap for ordered map IIRC.
There was a PR in the v2 commit history which removed bitflags but then you reverted it back @kbknapp. Not sure what happened but it was around 2 yrs ago.

 Comment by kbknapp
Tuesday May 12, 2020 at 17:57 GMT

I think using indexmap in place of vec_map would make the performance worse. And we need indexmap for ordered map IIRC.

That's entirely possible. I haven't done any investigating on the matter yet.

There was a PR in the v2 commit history which removed bitflags but then you reverted it back @kbknapp. Not sure what happened but it was around 2 yrs ago.

That was switching bitflags for a HashSet which had a significant negative performance impact which was the reasons for the revert.

 Comment by CreepySkeleton
Wednesday May 13, 2020 at 07:50 GMT

Also, one other note: IIRC, cargo bloat has an option to list binary size by function. That might also be useful to look at for decreasing binary size. Sometimes you can get big wins by tweaking the inlining strategy used, for example. Or using trait objects is sometimes possible, although I've found it difficult to do in practice in many cases because of object safety rules.

I actually looked at the list long time ago and found that

there are about 5 big (10KiB - 33KiB) functions, they hold ~90KiB of clap. Maybe we can shrink them a bit, but I wouldn't expect any breakthrough here.
there are a few dozens of pretty small but non-negligible functions (500B - 5KiB) that are responsible for the most of the bloat. This is where I'd expect we can get a win.
clap doesn't use generics internally at all. All (or most of) the generics are placed on public builder's functions and end up inlined.

textwrap - I would make this optional but enabled by default. I write all of my help strings and documentation wrapped to 79 columns anyway, which is probably good enough.
unicode-width - I would make this optional but enabled by default. If my help text is all ASCII, then an assumption that each byte is one column is reasonable.

Agreed.

bitflags - I would personally just hand write any bit flags logic that I needed. In my view, this is a crate that optimizes for writing code, which I generally think isn't the right thing to be doing in widely used fundamental crates. So there's a bit of a philosophical opinion involved here.

Counterpoints:

If you're suggesting to use a set of raw u64 constants, like
const REQUIRED:     u64 = 1;
const MULTIPLE_OCC: u64 = 1 << 1;
const EMPTY_VALS:   u64 = 1 << 2 | TAKES_VAL;
const GLOBAL:       u64 = 1 << 3;
const HIDDEN:       u64 = 1 << 4;
const TAKES_VAL:    u64 = 1 << 5;
const USE_DELIM:    u64 = 1 << 6;

Then there's a very significant downside to it: you can accidentally OR it with something that is not a bitflag, especially if names are similar and the types happen to match.
bitflags prevents this by creating a separate type for it that is incompatible with everything else. Of course, you could create a separate type yourself, by hand, but then again, you'll be left to implementing OR, And, Not... by hand - and you better make sure you aren't making more errors when you'll be copy-pasting the impls!

This crate is de-facto standard for bitflags in rust world, judging by download count.

I don't really have very serious objections to importing the macro into clap codbase, but I'd rather avoid it nonetheless ("reinventing the wheel" argument).
As an argument for importing the macro, I'd say that bitflags looks mostly unmaintained at this point (four months without updates and a couple of unanswered PRs).

indexmap - I don't have enough context to know what to do here, but it looks like it was recently added. I assume this is justified somehow through a major performance win, but otherwise, I would drop this.

This is justified by the need of a specific data structure, and IndexMap satisfies it. We need:

A key-value-like representation with O(1) lookup. Probably even O(log(n)) would be fine, number of arguments (n) rarely exceeds a hundred or so. Both HashMap and BTreeMap could be used here.
Iteration order must match the insertion order. Neither HashMap nor BtreeMap fit.

Yes, we could use a HashMap coupled with a Vec to trace the insertions, but why reimplement the wheel?

os_str_bytes - This is definitely better than using unsafe code. If I knew more about why Clap needed this, then perhaps there is a way to get rid of this too. But yeah, if Clap needs to be mucking around with OsStrs, then the lack of an OsStr string API probably makes this really hard to the point where you pretty much have to do what os_str_bytes is doing. (I still wish std would just expose its internal WTF-8 representation, but there are good arguments against doing that.)

Well, either std decides to expose the wtf-8 representation - which is very unlikely - or we do what os_str_bytes does simply because we need these methods. And I'd rather use one, well-audited implementation (os_str_bytes appears to be one) instead of spawning our own unicorn impl with lots of unsafe.
This is core functionality. It breaks, we break. It shines, we shine, too.
bstr is great, but not quite what we need.

vec_map - I don't know the performance story here, but keeping it optional looked easy? But maybe there are some issues I don't know about.
vec_map - Addressed in other comments. I just want to make sure we're getting a good cost:benefit ratio otherwise we may drop it entirely.

90% sure we can simply replace it with BTreeMap or even plain Vec. I'll be looking into it.

 Comment by bb010g
Friday Jun 12, 2020 at 07:47 GMT

@kbknapp

"But wait!" I can hear you say. "Auto help generation is like THE thing clap is used for, right?" Sure. It's one of the things, but I'd argue not the most important at all. In many of the contexts where code size matters the most, help generation is one the things they don't care about because it's super easy to just write a help string manually for a small CLI.

Would also exposing static help generation via a build script and/or macro be feasible?

 Comment by therealprof
Thursday Jul 02, 2020 at 13:48 GMT

The main reason why performance matters for CLI parsers is when a lot of arguments are passed to the command, or when the command is repeatedly executed. This can happen pretty easily with things like xargs.

Sure, I wasn't implying that abysmal performance was acceptable. But you don't need superfancy order-preserving hashmaps to get perfectly fine CLI parsing speeds for 99+% of all applications, a regular Vec would do just fine. And for the rest we can always have a separate opt-in performance option which uses highly optimized external crates to cater for crazy fast 2 MB CLI arguments parsing needs.

 Comment by BurntSushi
Thursday Jul 02, 2020 at 13:55 GMT

I see, I didn't realize you were talking at that level of detail. I think at that point, you need a benchmark. I do share your skepticism on that specific point, but would definitely want to subject it to testing.


Make init_logging more easily usable

Instead of requiring the LogRotationOpt, it now requires an
Option<LogRotationOpt>. This makes it much more easy to use the
interface when someone isn't interested on the LogRotationOpt's
polkadot-companion: https://github.com/paritytech/polkadot/pull/1386

bot merge


Add documentation for Variant queries

DeepRank core generally allows you to predict 2 different things: protein-protein-interactions (PPIs) or something about variant residues.
All documentation in README is focussed on PPIs. It is unclear what the purpose of the Variant queries is and how to use this. We should add an explanation of this to the README.
Also, development of the package is done (almost) exclusively with PPIs in mind, given that it is unclear what the purpose of the variant queries is. It would really help if this was clarified so that both types of projects can be taken into account during development.

Blocked by #377

We will keep PPIs only on the readme and getstarted, but in #459 we're developing detailed tutorials for variants.

We will keep PPIs only on the readme and getstarted, but in #459 we're developing detailed tutorials for variants.

Not sure whether that is already happening in #459 (I think yes), but we should at least mention in the readme that this is the case.
I also think that means we can close this issue once #459 is closed.

We will keep PPIs only on the readme and getstarted, but in #459 we're developing detailed tutorials for variants.

Not sure whether that is already happening in #459 (I think yes), but we should at least mention in the readme that this is the case.

Yep, we do it here


New position of added item?

How can i retrieve the new col and row position right after adding item with push(item) ?
When i add item, the col and row are undefined, and at this time i need record the right positions on DB.
I tried the 'gridster-item-initialized' event, but i'm getting null.
Watching the col and row vars on item controller is the only aproach that worked , but seems to be like a hack.
An 'added-item' event with new the positions or even a addItem() function would be interesting in this case.

Do you use angular to control itens of gridster?
When I add a new item by the angular the "row" and "col" attributes is set correctly.
Dont need to use any event to this... its only save in database what is in angular model.

Yes i'm using angular, but it takes some time to update the right positions internaly in the directive after the push(). Now, i am using a timeout right after the push() and then i can get the new positions.

One thing I noticed is that the gridster library sets the row/col values as NULL when adding to the gridster items array. When saving this to the database, how do you get around this fact?

Do you use two-way-data-binding with the controller?


⚠ Email has degraded performance

In b6ea66a, Email (https://mail.google.com/a/stjohn.edu.et) experienced degraded performance:

HTTP code: 200
Response time: 9630 ms

Resolved: Email performance has improved in 50b0315 after 10 minutes.


WIP: Add support for right-to-left messages

Follow up on my previous PR #4535 to implement full support for right-to left messages.

@Shayan-To Thanks for working on this!  You were correct on chat that this change is surprisingly simple for such an important/useful feature.
I made one comment above about extracting a function.
There are some build errors here:
https://circleci.com/gh/zulip/zulip/8830#tests/containers/0
Let us know if you need help resolving any of them.  For the find-add-class error, you can fix that by adding 'rtl' to the GENERIC_KEYWORDS list.
Do you have time to create a few node tests for the JS changes?

@showell And it would be far simpler if css logical layout properties were supported by the browsers. You can see how simple by looking at #2758 on the mobile project.
Fixed the CI errors.
And I actually have no idea how the changes I made can be tested.

@Shayan-To I added #9914 for adding node tests.  Here are docs on the node tests:
https://zulip.readthedocs.io/en/latest/testing/testing-with-node.html
We can find somebody else to write these, but let us know if you're interested.

@timabbott This LGTM, but I only tested this insofar as it doesn't break the English left-to-right experience (and that seems fine).


Update text on index html redirect page

This PR resolves #49.

I added the indentation and line wrap.

I added the indentation and line wrap.

Thanks @ShrillShrestha!

Thanks @ShrillShrestha!

Thanks to you too, @andyk :)

Thanks to you too, @andyk :)


Modify the workflow to allow quantitative traits

Although the is_binary option is already implemented, the current workflow throws an error when is_binary=false. This PR fixes that error.

Thanks @Lipastomies!


Drop Microsoft from namespaces in the programming model

I suggest that .NET Interactive simply drop Microsoft from the namespaces in the programming model.
Current: Microsoft.DotNet.Interactive.*
Suggestion: Just use just DotNet.Interactive.*.
Why:

The use of Microsoft for a namespace qualifier for neutral, open source, collaborative  .NET technologies and tooling is inappropriate in 2020.   THis is an open source technology, not a Microsoft proprietary technology.

The use of Microsoft as a namespace qualifier on DLLs and namespaces is a long and ugly legacy of a rather unfortunate set of guidelines that got created long before .NET was open source tech. We should not be trapped by decisions that capture the mindset of 2001.  If necessary policies can change.

If any other company were contributing and making this tech as a .NET core technology we would not expect Adobe. or Facebook to go on the front.

In the open source world the neutral technology is the primary qualifier in matters of naming.  Not the company.  That's now well-established practice across the industry.

I'm not sure of current "policy" (though it should change whatever it is), but there are now hundreds of examples where .NET is a neutral lead identifier, e.g. .NET is in https://github.com/dotnet and so on.

As background, we went around and around this for F#.  In the end the F# programming model uses just FSharp.* (though there is an unfortunate backward compat issue where the namespaces in the actual DLL use Microsoft.FSharp, though we would drop that if we could).  .NET has taken the same journey.   To repeat: in the open source world a neutral technology acts as the primary qualifier in matters of naming.
I personally think it's really essential to make this change to properly position .NET Interactive as a neutral, broad technology and also to stand up for what's right for the future of the .NET ecosystem)
If necessary we can get opinions from @richlander, @terrajobst and others?

We really need to do this. This would look so much better than with the company prefix.
open DotNet.Interactive.FSharp.FSharpPocketViewTags

Why does it even need DotNet.

@davidfowl In my eyes the top namespace should be:

a neutral organisation moniker (DotNet for .NET Foundation matching "dotnet", FSharp for F# Foundation matching "fsharp")
a company (Microsoft)
a technology (OxyPlot)

Using the name of the technology here would lead to "DotNetInteractive.*"
Using the name of the organisation would lead to "DotNet.*"
Using the name of the company would lead to "Microsoft.*"


Try out some old Neuon software

https://web.archive.org/web/20141011212633/http://neuon.com/downloads/

Come to think of it, perhaps it would be nice to start collecting these locations together on the website?

I've now added a link to the website, so I'm going to close this issue as 'done'. Hopefully that's enough to encourage us to try stuff out.


Translation issues for Science 5 (EN) graphs

Reposted from https://github.com/corona-warn-app/cwa-website/issues/3065 where the issue was mistakenly closed.
Where to find the issue
https://www.coronawarn.app/en/science/2022-03-03-science-blog-5/
Describe the issue
These are issues with the graphs included after merge of the Science 5 article translation into English. See PR https://github.com/corona-warn-app/cwa-website/pull/3058.
Fig 1: Tests provided and accessed via the CWA and their results.

Typo: Possitive test results (accessed)
should be "Positive"

Fig 3: Number of individuals who received a red and green warning per individual providing a warning in chronological sequence.

Missing closing bracket on y-axis Number of warning individuals (per individual providing a warning

Multiple years shown with no year specified

Figure 4: Positivity rate after risk notification in chronological sequence (PCR)

y-axis is labelled Percentage of individuals that tested positive and Percentage is also used to label graph data, however the numbers on the y-axis are a fraction not a percentage. In German the term "Positivenanteil" is used.

Multiple years shown with no year specified

Figure. 5: Positivity rate after risk notification. The number of new infections in chronological sequence (RAT).

Similar comment to above regarding the incorrect use of the term "Percentage" for the y-axis.

Multiple years shown with no year specified

Figure 6: Percentage of users, for whom the risk determination is not possible (CWA data donation).

Similar comment to above regarding the incorrect use of the term "Percentage" for the y-axis.

Multiple years shown with no year specified

Typo: White tail should be White tile

Figure 10: Estimation of the number of active users resulting from the percentage of the individuals donating data.

Untranslated label Mittelwert der letzten 30 Tage is in German

Figure 11: Comparison of download numbers and active users or active apps, respectively, for Corona-Warn-App, and SwissCovidApp.

Untranslated label Aktive Nutzende is in German

Figure 12: Active devices in the narrower sense) via downloads of daily key files (TSI).

Dates are in German format e.g. 6. Dec. English dates do not use a "."

Figure 13: Active devices (in the narrower sense) after days since last accessing the daily key files (TSI)

Untranslated x-axis label Tag

Figure 15: Information relating to the installation (Apple).

Untranslated term Hochrechnung is in German

Figure 16: Various estimations of active users (in the broader sense).

Untranslated terms Mittelwert, Heuristik and Warnende are in German

Suggestion
Correct translation issues in graphs and republish.

Internal Tracking ID: EXPOSUREAPP-14097

@MikeMcC399 thanks for your report. I have provided a fix for this in #3129


Firefox simulcast

This PR intrduces the possibility to do simulcast also from Frirefox.
There is an unresolved bug that still have to be managed.
Refer to https://github.com/lynckia/licode/issues/1076
It worked well before the Sdp JSON refactor
@jcague wait for your help!
[X] It needs and includes Unit Tests

At this point firefox simulcast is working but there's a bug that inverts the spatial layer order, so che base (0) layer is 640x480 and the layer (1) us 320x240.
I think that maybe is due to a wrong sdp generation. In fact when SdpInfo process the offer created by browser, the result is a duplicate entry (not a bad thing consider the RFC allowing it)

a=ssrc:3872236851 cname:{750bc0de-2f2f-454b-8bad-47c4654dc244}
a=ssrc:2229183214 cname:{750bc0de-2f2f-454b-8bad-47c4654dc244}
a=ssrc:2229183214 cname:{750bc0de-2f2f-454b-8bad-47c4654dc244}
a=ssrc:2229183214 msid:{87566d06-d440-6c41-9ca6-82b7dcd20d01} {83919928-7494-d549-80d2-6cf793d632da}

I made it working commenting two lines in SDPInfo.js but I think this is not the right solution.

if (!track) {
  track = new TrackInfo(media, trackId);
  // track.setMediaId(md.mid);
  // track.setEncodings(encodings);
  stream.addTrack(track);
}

Interestingly, commenting these 2 lines the output now includes another msid mapping for the other simulcast track that previously was not present.

a=ssrc:2729708755 cname:{2908d91d-ac91-584b-a736-bbb5e2674d66}
a=ssrc:2729708755 msid:{1b6a9807-366f-f84c-a5dc-85cee9be48ee} {66011479-01dc-de4a-a37a-fd35435d187b}
a=ssrc:3385189830 cname:{2908d91d-ac91-584b-a736-bbb5e2674d66}
a=ssrc:3385189830 msid:{1b6a9807-366f-f84c-a5dc-85cee9be48ee} {66011479-01dc-de4a-a37a-fd35435d187b}
a=ssrc:3385189830 cname:{2908d91d-ac91-584b-a736-bbb5e2674d66}
a=ssrc:3385189830 msid:{1b6a9807-366f-f84c-a5dc-85cee9be48ee} {66011479-01dc-de4a-a37a-fd35435d187b}

Hope this helps the debug process, I'm stuck.
But why this library was introduced? I think it may be another point of failure debugging new browsers such as edge and safari since sdps are in a way "manipulated"

@kekkokk I updated this PR, can you check whether it works or not for you now please?

Simulcast and layers seems to work finally!
But I noticed some strange behaviour due to p(probably) the padding generation algorythm.
It is extremely slow to reach the upper layer (also in localhost) and sometimes it never happens.
This is the chrome receiving side:

Seems to work better with firefox as subscriber. A bit slow to reach the upper layer but is more consistent

@kekkokk I updated the code and it now uses the higher layers by default (640x480 por higher spatial layer and 320x240 for lower spatial layer).


How to Correctly Send Signals to Workers?

Hey Guys,
I have had a look at the demo's and done some extensive research and can't find an exact method or example with how to send a signal the a worker.
When a job runs, I store the PID of the PHP script in a database via

getmypid()

I then have tried to find a way to send the commands listed in the README to the PID... I have found via my search in Google

posix_kill( PID, SIGNAL )

This allows me to send a signal to the PID... which is exactly what I want.. however none of the accepted values match the values supplied in the readme.

SIGUSR1

It wouldn't accept SIGUSR1 as a parameter, so on my setup I ran

kill -l (this gave me the int value of the command which was 10 in my case)

This seems close to what I need, to I am using this -> however the command posix_kill fails..
I have a "health" script which runs every few mins to check things are progressing and if something times out and gets stuck... which is possible, then I can send a signal to free up  the worker..
What is the general accepted method in sending signals to workers via the PID?
I'm using Ubuntu 12.04 and using php 5.3+
Thanks heaps

First off, you probably don't want to send the signal to the job's PID - you'll usually want the worker's PID, which is different because Resque forks itself before each job.  SIGUSR1, especially, won't work when pointed at the job's PID, since the signal handler is set up to terminate a child process, which won't exist in the job's fork.  There are a couple of ways to get this - you could use posix_getppid() in your job instead of getmypid(), for example.  Or you can grab the worker PIDs from Redis, where they are all stored in the resque:workers set as {$hostname}:{$pid}:{$queues}.  Since you didn't specify how your health script iterates through your system, I can't tell whether you already have those PIDs available in other ways.
Second, SIGUSR1 is highly preferred over hard-coding the integer value.  Check that the pcntl extension is installed and enabled in CLI mode on your system (most systems have a different config file for each SAPI), as that's where these constants are defined.
Third, you say posix_kill() fails, but you don't say how.  It could be considered a failure, for example, if sending the signal simply doesn't kill the job as expected (which it wouldn't, as explained above).  I suspect that isn't the case, here, but I can't confirm that without more information.
You've got the gist of it, we just need to figure out a few of the details.

Hey
Thanks so much for your detailed response, upon further digging, I found I did not enable the pcntl extension on this particular server, with that enabled the SIGUSR1 doesn't raise any exceptions.
I modified my query to save the PID via  posix_getppid() and can confirm when I run
  ps -e -o pid,command | grep [r]esque it shows the PID there.
I still can't seem to get posix_kill to work... it always returns false even with the correct PID shown and the actual SIGUSR1  command.
My health script operates over a cron job which executes every minute > I will bring over to a redis job when it's all working.... if it detects with parameters I set that a job is hung, then it sends the command via the above to the processed which is meant to end the job and free up the worker


Write permission for Github Actions TOC generator

I added a table of content (TOC) generator, however it is failing due to missing write permission (see logs). Currently I do not have the priviliges to update the permissions for the workflow.

Log

==================================================
Version<EMAIL_ADDRESS>          3ba472e2cc699e92e6dba3f5ce758e1cde43fbc9
Event:    push
Action:   undefined
sha:      a8bd89710b295b0728835372eeb8de3bd2ac3901
ref:      refs/heads/main
owner:    Greeley-Robotics
repo:     conventions
==================================================
git config --global 'init.defaultBranch' main
Fetching...
Switching branch to [main]...
Running commands...
Checking diff...
Committing...
Pushing to<EMAIL_ADDRESS>git push origin main:refs/heads/main
>> remote: Write access to repository not granted.
>> fatal: unable to access 'https://github.com/Greeley-Robotics/conventions.git/': The requested URL returned error: 403
Error: command [git push origin main:refs/heads/main] exited with code 128. message: remote: Write access to repository not granted.
fatal: unable to access 'https://github.com/Greeley-Robotics/conventions.git/': The requested URL returned error: 403
  at Command.value [as execAsync] (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:89037:31)
  at processTicksAndRejections (node:internal/process/task_queues:96:5)
  at async GitHelper.value [as runCommand] (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:89118:45)
  at async GitHelper.value [as push] (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:89552:17)
  at async push (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:96169:5)
  at async createCommit (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:96[62](https://github.com/Greeley-Robotics/conventions/actions/runs/7771466343/job/21192794048#step:2:63)2:5)
  at async execute (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:96816:28)
  at async main (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:96845:5)

Error: command [git push origin main:refs/heads/main] exited with code 128. message: remote: Write access to repository not granted.
fatal: unable to access 'https://github.com/Greeley-Robotics/conventions.git/': The requested URL returned error: 403
  at Command.value [as execAsync] (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:89037:31)
  at processTicksAndRejections (node:internal/process/task_queues:96:5)
  at async GitHelper.value [as runCommand] (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:89118:45)
  at async GitHelper.value [as push] (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:89552:17)
  at async push (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:96169:5)
  at async createCommit (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:9[66](https://github.com/Greeley-Robotics/conventions/actions/runs/7771466343/job/21192794048#step:2:67)22:5)
  at async execute (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:9[68](https://github.com/Greeley-Robotics/conventions/actions/runs/7771466343/job/21192794048#step:2:69)16:28)
  at async main (/home/runner/work/_actions/technote-space/toc-generator/v4/lib/main.js:96845:5)

Error: command [git push origin main:refs/heads/main] exited with code 128. message: remote: Write access to repository not granted.
fatal: unable to access 'https://github.com/Greeley-Robotics/conventions.git/': The requested URL returned error: 403

Changed workflow permissions


display price multipliers for selected item

when bringing up the detailed information on an item, it'd be nice to be able to compare its sell multipliers/price between the rest of the places it can be bought/sold from.

Would appreciate it if you explained in more detail or even gave visual examples. Or do you just want me to include those multipliers in the item info table?

currently to see the buy/sell price of an item, one would go to the wiki, or click on every permutation.
pretty much like the price table here:
https://barotraumagame.com/wiki/Underwater_Scooter
shows buys/sell price on each place.
Additionally, a little visual indicator of lowest/highest prices would be nice


How to actually persist?

I have added the --persist to my goaccess command. However, if I stop goaccess and run it again (say with new options), I see the report only for the current day (what's available on the log).
Is there any database settings I need to perform for persistent data reporting? Or is my understanding of the --persist option totally wrong?

Hi @notakoder
Yeah... off course, do you need more option.
At manual page at here at Persistence Storage Options can read about this.
So, at first running only --persist option is ok. But, for next do need specific --restore.
And if you need save data processing at specific location, a place different from LOGs location, use --db-path.
I hope be clean.

--restore does the job. Sorry I missed that in the documentation. Thank you for pointing it out.


I can not start netplugin

Hi, I can not start netplugin, and got the log message:
9月 29 10:41:23 10-57-18-102 netplugin[30892]: time="Sep 29 10:41:23.655921833" level=fatal msg="OVS switch vxlan Failed to connect"
9月 29 10:41:23 10-57-18-102 systemd[1]: netplugin.service: main process exited, code=exited, status=1/FAILURE
9月 29 10:41:23 10-57-18-102 systemd[1]: Unit netplugin.service entered failed state.
9月 29 10:41:23 10-57-18-102 systemd[1]: netplugin.service failed.
what should I do?

@kongbo1987 : We have seen these issues most of the times due to unsupported ovs installation / vxlan udp port being used by docker kind of cases. What is the ovs version being used ?

@abhinandanpb  ovs version 2.6.0,should I down downgrade to 2.3.1?

@abhinandanpb I found that the ovs controller failed  to listen on port 6633

@kongbo1987 we would suspect the ovs installation may not have gone right. Can you let us know how you went about deploying contiv ? Did you use one of our installers ? What is the OS you are trying to use this on ?


CSI Plugin Not Registering on Nodes

This is my first time working with DRBD and Linstor. Following the README, I can get most everything up and running. However, the CSI plugin fails to register on the nodes. I can provision PVCs from Kubernetes but pods won't attach to them, which is strange.
Here are logs from the csi-node pod:
I0108 15:00:47.982707       1 main.go:113] Version: v2.2.0
I0108 15:00:47.983302       1 main.go:137] Attempting to open a gRPC connection with: "/csi/csi.sock"
I0108 15:00:47.983328       1 connection.go:153] Connecting to unix:///csi/csi.sock
I0108 15:00:47.983742       1 main.go:144] Calling CSI driver to discover driver name
I0108 15:00:47.983775       1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo
I0108 15:00:47.983787       1 connection.go:183] GRPC request: {}
I0108 15:00:47.987578       1 connection.go:185] GRPC response: {"name":"linstor.csi.linbit.com","vendor_version":"v0.13.1"}
I0108 15:00:47.987648       1 connection.go:186] GRPC error: <nil>
I0108 15:00:47.987657       1 main.go:154] CSI driver name: "linstor.csi.linbit.com"
I0108 15:00:47.987714       1 node_register.go:52] Starting Registration Server at: /registration/linstor.csi.linbit.com-reg.sock
I0108 15:00:47.987884       1 node_register.go:61] Registration Server started at: /registration/linstor.csi.linbit.com-reg.sock
I0108 15:00:47.987938       1 node_register.go:83] Skipping healthz server because HTTP endpoint is set to: ""
I0108 15:00:49.379501       1 main.go:80] Received GetInfo call: &InfoRequest{}
I0108 15:00:49.412651       1 main.go:90] Received NotifyRegistrationStatus call: &RegistrationStatus{PluginRegistered:false,Error:RegisterPlugin error -- plugin registration failed with err: rpc error: code = Unknown desc = failed to retrieve node topology: failed to get storage pools for node: 404 Not Found,}
E0108 15:00:49.412696       1 main.go:92] Registration process failed with error: RegisterPlugin error -- plugin registration failed with err: rpc error: code = Unknown desc = failed to retrieve node topology: failed to get storage pools for node: 404 Not Found, restarting registration container.

Here is a message from a pod attempting to mount a provisioned PVC:
AttachVolume.Attach failed for volume "pvc-de8fe93c-7431-4e62-981f-49b2541a95de" : CSINode rke1w does not contain driver linstor.csi.linbit.com

However, I can plainly see the storage was provisioned:

What might I be doing wrong? When installing from helm I'm pretty much using your example values.
Thanks.

Hi, could you provide also output of linstor n l and linstor sp l commands?
Is linstor-satellite daemonset running on rke1w?
kubectl get pod -n linstor --field-selector spec.nodeName=rke1w -l app=linstor-satellite

check the configurator logs:
kubectl logs -n linstor <pod_name> -c configurator


Renamed gem

Replaced use of master/slave language with leader/replica.

:clap:

@dandrabik @dandrabik @dandrabik

@samuelcole That @dandrabik is currently deep in the wilds of South America.

:100:
i'll rebase my pull request after!

Style!


allow having multiple context views up at once

There are instances when we need to have two separate context views visible at one time.
See https://github.com/microsoft/vscode/pull/189074 for a hack that worked around the issue. Ideally, the views could have separate IDs and therefore be able to coexist.
cc @Tyriar

@meganrogge the context view service and the hover service use separate context views now. Does this solve this issue?


Features of autograd.record

Hello!
What could be the problem?
I have saved gluon model after 250 epochs like (model.save_parameters(...)).
After that I try to get output from load_params model:
First way is:
with autograd.record():
    outputs = [model(X) for X in data]

Second way is just:
outputs = [model(X) for X in data]
First way gives better predictions.
What could be the reason?

If you have e.g. batchnorm in your model, the first way will use training mode of it (using statistics from your batch), whereas the second way will call inference mode (which uses statistics gathered during training).

@ptrendx
But default use_global_stats is False in all BN in my model. It's meaning that global moving statistics is not agreagating.

No. use_global_stats=True would mean that you don't calculate statistics during training, False calculates statistic from a current batch. In inference mode you never calculate statistics, and instead always use the global mean and variance.

@ptrendx thanks!

@two-names Please feel free to re-open the issue if you have further questions, closing for now.


Why fairseq-interactive and fairseq-genetate generate results so different?

❓ Questions and Help
Why fairseq-interactive and fairseq-genetate generate results so different?
Before asking:

search the issues.
search the docs.

What is your question?
I load the same model and use these two methods to produce completely different results. Is it my setting wrong?
Code

train:
GPU_ID=0,1,2,3
DATA_BIN_DIR="asr_dataset/bin/"
OUT_DIR="asr_dataset/models/"
BATCH_SIZE=64
MAX_LEN=400
MAX_TOKENS=4096
SEED=1
MAX_EPOCH=100
CUDA_VISIBLE_DEVICES="${GPU_ID}" fairseq-train
${DATA_BIN_DIR}
--save-dir ${OUT_DIR}
--ddp-backend=no_c10d
--task translation_lev
--criterion nat_loss
--arch levenshtein_transformer
--noise random_delete
--share-all-embeddings
--optimizer adam --adam-betas '(0.9,0.98)'
--lr 0.0005 --lr-scheduler inverse_sqrt
--min-lr '1e-09' --warmup-updates 10000
--warmup-init-lr '1e-07' --label-smoothing 0.1
--dropout 0.3 --weight-decay 0.01
--decoder-learned-pos
--encoder-learned-pos
--apply-bert-init
--log-format 'simple' --log-interval 100
--fixed-validation-seed ${SEED}
--max-tokens ${MAX_TOKENS}
--save-interval-updates 10000
--max-epoch ${MAX_EPOCH}
--fp16

fairseq-genetate
DATA_BIN_DIR="asr_dataset/bin/"
MODEL="asr_dataset/models/checkpoint_best.pt"
BEAM=2
BATCH_SIZE=64
fairseq-generate
${DATA_BIN_DIR}
--gen-subset valid
--task translation_lev
--path ${MODEL}
--iter-decode-max-iter 9
--iter-decode-eos-penalty 0
--beam ${BEAM} --remove-bpe
--print-step
--batch-size ${BATCH_SIZE}

fairseq-interactive
DATA_BIN_DIR="./asr_dataset/bin/"
MODEL="./asr_dataset/models/checkpoint_best.pt"
BEAM=2
BATCH_SIZE=64
input=$1
CUDA_VISIBLE_DEVICES=0 fairseq-interactive
${DATA_BIN_DIR}
--task translation_lev
--path ${MODEL}
--iter-decode-max-iter 9
--iter-decode-eos-penalty 0
--beam ${BEAM} --remove-bpe
--print-step
--buffer-size ${BATCH_SIZE}
--input ${input}
--fp16

$1:
cat input.txt:
想 吃 海 鲜
我 很 快 乐

output by fairseq-generate:

想 吃 海 鲜
我 很 快 乐

output by fairseq-interactive :

我 的
我

This result is very strange
What have you tried?
My issue is different with #568
I also tried:
fr2en = TransformerModel.from_pretrained(
model_name_or_path=model_path,
checkpoint_file='checkpoint_best.pt',
data_name_or_path=user_dir,
task='translation_lev',
iter_decode_max_iter=9,
iter_decode_eos_penalty=0,
beam=2,
buffer_size=64,
fp16=True,
print_step=True
)
fr2en.eval()
fr2en.translate('想 吃 海 鲜')
but the result is same with  fairseq-interactive
What's your environment?

fairseq Version (e.g., 1.0 or master): 0.9.0
PyTorch Version (e.g., 1.0): 1.3.1
OS (e.g., Linux): Ubuntu
How you installed fairseq (pip, source): pip
Build command you used (if compiling from source):
Python version:  python3.6.9
CUDA/cuDNN version: 10.0
GPU models and configuration: GeForce RTX 2080 Ti
Any other relevant information:

@myleott Excuse me? Could you do me a favor, please?

I have tried this code:
CUDA_VISIBLE_DEVICES="${GPU_ID}" python -m fairseq_cli.interactive
--path ${MODEL}
--beam ${BEAM} --nbest ${NBEST}
--task translation_lev
--iter-decode-max-iter 9
--iter-decode-eos-penalty 0
${DATA_BIN_DIR} < ${INPUT_FILE} \
and :
CUDA_VISIBLE_DEVICES="${GPU_ID}" python -m fairseq_cli.generate
--gen-subset valid
--path ${MODEL}
--beam ${BEAM} --nbest ${NBEST}
--task translation_lev
--iter-decode-max-iter 9
--iter-decode-eos-penalty 0
${DATA_BIN_DIR}
The ${MODEL} is same, But the result is very different

You have a fr->en translation task that translates zh->zh? 🤔

You're training a translation task from zh->zh?

@erip Yes， I am doing Chinese spell correction，but the result should have nothing to do with what task I do, right?

It shouldn't, no. Have you tried your task with what's on the master branch?

Now is Dev2020, we still do not have something that one can git clone easily and doing fairseq-generate INSTEAD of Fairseq-Interactive for Chinese to English as attempted before back in 2017

For levenshtein transformer, the preprocessing step requires adding bos token to the source which is not done in the interactive mode and hence the difference in results.
In my experiments, just adding bos token to the source tokens when in interactive mode fixes this issue.

For levenshtein transformer, the preprocessing step requires adding bos token to the source which is not done in the interactive mode and hence the difference in results.
In my experiments, just adding bos token to the source tokens when in interactive mode fixes this issue.

hello, I also want to train a chinese spell correction model, how was your result?

For levenshtein transformer, the preprocessing step requires adding bos token to the source which is not done in the interactive mode and hence the difference in results.
In my experiments, just adding bos token to the source tokens when in interactive mode fixes this issue.

where can i add bos token to the source tokens in the source code?  thanks


Update README.md - Ticketed events Box office

Box office with listings and calendar invite/ticket sales/ etc for timed events and donations to the program

Cool, good, thanks!


fix(staging.volumes.fibre_channel): update documentation reference link

This PR fixes the link to fcvolumesource
What was the issue?

The link in the specific README.md is broken

What has been done?

Update the fcvolumesource link generated document to the latest and correct one -- https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#fcvolumesource-v1-core --

How to review?

Old one is broken
New one is fine

/assign @idvoretskyi


.flayignore isn't working

Hello,
I've just tried to setup .flayignore in my project's repository with the content below:
spec/**/*

and running it with the command flay spec/ it works and doesn't throw any warnings, but when I run pronto runit says about duplications. Am I missing something? Anyone else has the same issue?

Skips files matched via patterns in .flayignore (subset format of .gitignore).

@giedriusr it works as expected on Pronto side. .flayignore has its own way for pattern matching. For example, we use *_spec.rb. I couldn't find decent documentation, you can try to figure out more from the source code: https://github.com/seattlerb/flay/blob/master/lib/flay.rb#L153

@mmozuras their documentation says it works same way as .gitignore, but thanks.

subset format of .gitignore

@giedriusr no problem :bow:


Should all js be bundled when deployed to Heroku?

I have a problem with a site that loads slowly and after asking on Stackoverflow everyone said that there is far too many js files
There is 71.
What i noticed is that its loading all the non third party js files seperately. Is this wrong? i.e routes, views etc.
Heres the URL.
http://saveme.ie/

You can concatenate those files and see if that helps.

Well then my question is how do i bundle them?

Check out gulp-concat and see if concatenating into 1 script will load faster.

@Ilanbala Gulpfile should actually have a build task that's suitable for production env.

Confusing...
Build task:
// Lint project files and minify them into two production files.
gulp.task('build', function (done) {
  runSequence('env:dev', 'wiredep:prod', 'lint', ['uglify', 'cssmin'], done);
});

Odd to see both env:dev and wiredep:prod?
Wiredep task
...isn't used anywhere. Was this meant for development?

prod I think uses minified files.

@ilanbiala prod task also runs the app, so it's not really suitable for building the prod version.

Sorry, I meant wiredep:prod uses the minified files for wiredep.

On HEROKU you need to go to env vars in settings and add
NODE_ENV production
Then use grunt build before deploying to Heroku.
Cuts loading time to about 1/5th on my site.

@ChrisMaher can you make a PR to add that to our documentation for Heroku?

@ilanbiala I think we actually need a better production build maybe?


feat: enable techdocs by default

Description
Please explain the changes you made here.
Which issue(s) does this PR fix

Fixes #?

PR acceptance criteria
Please make sure that the following steps are complete:

[ ] GitHub Actions are completed and successful
[ ] Unit Tests are updated and passing
[ ] E2E Tests are updated and passing
[ ] Documentation is updated if necessary (requirement for new features)
[ ] Add a screenshot if the change is UX/UI related

How to test changes / Special notes to the reviewer

rebased to reflect update to backstage 1.23
adding lgtm back
/lgtm


Don't run docker-exec interactively, as docker is sad about this when run via cron.

This change is

If you want the output sent to the same place all cron output goes, I recommend using docker -t but not with '-i', because then the output of the cron job will get sent to the right place but docker won't get upset that it can't get stdin from the cron invocation.

Review status: 0 of 1 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

In this case, I don't  think we're worried about the output. The cronjob was failing before, yet nobody noticed anyway, until I happen to login for some other reason and noticed it.


Would you share the detection result json file on ActivityNet?

Dear Author:
I am reading your papers:

A Pursuit of Temporal Accuracy in General Activity Detection
Temporal Action Detection with Structured Segment Networks

Thanks for your great works!
Currently, I am evaluating my detection result on one out of five subsets of ActivityNet 1.2 validation set. However, on one has ever release the results on these five subsets respectively.
I am wondering would you share your detection result json file of these two works on ActivityNet 1.2 or 1.3 validation set with me?
If yes, can you email the file to<EMAIL_ADDRESS>or just paste down below? That means a lot!
Thanks a lot!

You can generate the detection results by following the instructions and using the provided pre-trained models.


Fix gluten test util getExecutedPlan.

current function is more readable.
previous version has a bug for plans like:

CommandResult <empty>
   +- AppendData
      +- AdaptiveSparkPlan isFinalPlan=true
         +- == Final Plan ==
            VeloxColumnarToRow
            +- AQEShuffleRead coalesced
               +- ShuffleQueryStage
                  +- ColumnarExchange

The nodes of both descendants of CommandResult and AdaptiveSparkPlan cannot be collected. (for example, the ColumnarExchange above).

@marin-ma


Don't support youtube play list

don't support youtube play list(public) ????

https://www.youtube.com/playlist?list=PLTRkBgepZZEWwZe6EKN_BDVMzctVA_j7S the link

Fixed with youtube api: https://developers.google.com/youtube/v3/getting-started

Fixed with https://developers.google.com/youtube/v3/getting-started


Remove merge dependency

It is a temporary dependency until #4297 is not implemented.

as far as I see, this is no longer relevant


sched: fix compile error when SCHED_CRITMONITOR is enabled

Summary
Compilation error occurs after SCHED_CRITMONITOR is enabled
sched/sched_critmonitor.c:315: undefined reference to `serr'
Impact
Testing

@Gary-Hobson it is a good idea to include the info from this Summary in the commit log message, this way people will see it in the git local repository.


Gender written Female instead of male in Vaccine certificate

I was vaccinated today, my certificate shows gender as female instead of male

I was vaccinated yesterday, My DOB is incorrect actual DOB is 1976 its showing 1967
certificate.pdf


[2Q2024] QA Distorted Datasets

Context:
For UXR datasets, we discovered that many datasets actually have a lot of white spaces surrounding the actual slices, which creates image distortion in Explorer
https://docs.google.com/document/d/17BGkBSIX-jYMmDvoBTiwDcVJ8dzv5egqpwWmT0PQWnU/edit#bookmark=id.jhtqe2q5b4no
DoD:

Coordinate with the curators to crop the image and reprocess spatial data to realign with the image, and re-upload the datasets?

Distorted image Slack:
https://chanzuckerbergteam.slack.com/archives/C06FM5ZQE3Y/p1710803742849079?thread_ts=1710803560.861729&cid=C06FM5ZQE3Y

Actually even with white spaces the image ratio shouldn't be distorted! Ronen likely has a fix already


Binary FBX Export

Blender cant import ASCII FBX, so converting the FBX to binary produces mangled models like this:

This issue has nothing to do with Unity Studio.

It was not an issue really, a suggestion to add binary fbx export. But yes, the mangled models may or may not be a unity studio, rather a blender or fbx converter issue, which could be alleviated through adding a binary fbx export.

@MichaelPriebe Great suggestion!

You can always just use http://richwhitehouse.com/index.php?content=inc_projects.php to convert to dae/Binaryfbx and many other formats.

This mangling happens because of incorrect skin processing in UnityStudio. By default, UnityStudio applying that every vertex has 4 bone influences. But that is almost always not true. Compressed mesh could contains 2200 vertices, 1900 bones influences and 1850 bones weights, so there is no definite correspondence between them. This algorithm should be improved in order to correctly export fbx.

⬆ This guy knows his stuff @Perfare

@MichaelPriebe I don't have any knowledge of the 3D model, even if someone tells me where the problem is, I can't do anything.

@Perfare :-( I hope there is someone will help you on 3D model. Its needed for Unity Studio

My implementation of AssetBundle handler has correct skin processing, but it hasn't released yet. I'm going to publish  it within a few weeks.

My implementation of AssetBundle handler has correct skin processing, but it hasn't released yet. I'm going to publish it within a few weeks.
I'm really looking forward to it. Because of what you mentioned, the fbx obtained — while losslessly exported in a Unity environement point of view — are almost unusable elsewhere

Ok, I wasn't quite correct. Skin still has incorrect bone index influence issue. But at least visually it is looking good. I am working on improvement. If I wont find the solution for this problem, I release my current work as pull request for this repo.

It does sound promising.

I'va made pull request with correct skin reading. Other issue still actual. I hope someone who had worked with something similar can solve this problem: when UnityStudio export FBX it changes quaternion rotations to euler, but beside that it also make some math tricks to convert rotation from unity's world space to 3DSMax(?) world space. So, after those transofrms final euler rotation values dont directly correspond to initial quaternion values. But when Unity import that FBX, it convert those value back to inner format, so rotation value look fine. But!
Skinned mesh renderer read values from transformation matrices, not from euler value list. Unity Studio export FBX with fixes matrices. Transformation look like this:
ob.AppendFormat("{0},{1},{2},{3},", m[0, 0], -m[1, 0], -m[2, 0], -m[3, 0]);
ob.AppendFormat("{0},{1},{2},{3},", -m[0, 1], m[1, 1], m[2, 1], m[3, 1]);
ob.AppendFormat("{0},{1},{2},{3},", -m[0, 2], m[1, 2], m[2, 2], m[3, 2]);
ob.AppendFormat("{0},{1},{2},{3}", -m[0, 3], m[1, 3], m[2, 3], m[3, 3]);0},
It is just changing some signes. And it is work for transform which has uniform (like Vector3(1, 1, 1) or Vector3(2, 2, 2)) scale value. But for non uniform scale those sign transformations are not enough. It is required some more transformation to correctly export such matrices.
I tried much cases, but non or them work for me. So i hope someone can figure it out how correctly export those matrices.

Those matrices are bind poses for skinned mesh renderer.

Just the problem described by the title, AS now supports, but for other problem I suggest opening a new issue.


only python grpc client can cancle request,  when will  support  with golang grpc

the doc descript only python support request cancel.we use golang grpc.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/request_cancellation.html?highlight=cancel#

Please ask this question in tritonserver repo.


🐞 [Bug]: some nodes have broken flag images

Is there an existing issue for this?

[X] I have searched the existing issues

which package/s did you face the problem with?
Dashboard
What happened?
some nodes have broken flag images
#2852
Steps To Reproduce
No response
which network/s did you face the problem on?
Dev
version
2.6
Twin ID/s
No response
Node ID/s
No response
Farm ID/s
No response
Contract ID/s
No response
Relevant screenshots/screen records

Relevant log output
na

Verified, Devnet db4d515.
The broken flag was changed to NA for country with no flag (wrong countries from fake nodes).


code release

Your work is very good. When can you release the code and try it out

Thank you very much for your interest in our work! Feel free to contact me directly at jamessealesmith at gatech dot edu


Don't allow user to create multiple product lists with same name

Summary:
Don't allow user to create multiple product lists with same name
Screenshots:

How do you want to manage backward compatibility on devices with multiples lists and already existed Database ?
I though about adding @Index on listName, but it will crash older app.
If you want to manage that in a resilient way, I'll just add a check on list name before adding.
That's less maintainable.


fix python 3+ setup.py

Somehow I missed this pull request. Your specific changes conflicted with the ones I made because I didn't merge yours first so won't be merging this pull request. However, I have fixed the Python 3 support (v3.2+).


Define styles in css

Coverage remained the same at 77.305% when pulling 47797b575a30593d13391c03cd167cb74cb61703 on laurpulk:define_styles_in_css into 607fe827d1f2539f447fcdbbaf24f2faea5fed4b on tdd-pingis:master.

Coverage remained the same at 74.376% when pulling cc3b5ebbbbee90009d364d44ab079e6fcdc09953 on laurpulk:define_styles_in_css into a743c4367d28aef6ebf788ad90591cdd9f0f7a1f on tdd-pingis:master.


Client connect trigger instead of min threshold

I got everything working great, but is there any way to make it spin up the docker container based on when a client connects or sends an http request to the port?
I am using it with motioneye, and motioneye doesn't ever really settle down to 0 packets and is impossible to gauge the threshold correctly.  It does, however, seem to register the clients connected just fine.   Also, you cannot send enough traffic to motioneye to get it to trigger back on because of the login screen it uses, I think.
TIA!

I agree some containers are always communicating with one another and it's almost impossible to get a good baseline of how many packets are too little or too much to start a container.

This is an incredible idea for game servers among friends! I'm looking at using this for a Satisfactory server (very high memory utilization) but am running into the same issue described here. The container is nearly immediately restarting based on received packets despite no players being logged in. I suspect this is some sort of keep alive traffic potentially coming from Steam and/or Epic? I'll look to mitigate this by watching averages of rx packets in the logging.
Is a clients connected metric for something like this even feasible for restarting a container? Clients connected will always be zero when stopped, and they aren't "connected" until the container is running and they're logged in.
Excellent work here either way!

I've been hacking on this project a bunch over the last couple of days.
Something I've come across (and would love to be proven wrong on) is that in order to make a client connect trigger work, all traffic would need to be something where we could determine what a "client" is.
Because this was purpose built to be a lazy loading front for containers as a whole and not for something like web servers specifically, or game servers specifically, I don't know that it will be possible to determine "client connection".
Again, if you know a way to determine that for generic traffic, please let me know, but I just don't know if this issue is solvable without compromising the purpose of lazytainer or a TON of custom code.
That said, I'm actively developing port specific packet thresholds which should help with some of the "spammy traffic hit lazytainer and now your other container is awake" stuff that was happening.


API 500 Support for Server Hardware

Description
Adding Server Hardware for HPE OneView API500 support.
Issues Resolved
N/A
Check List

[x] New functionality includes testing.

[x] All tests pass ($ rake test).
[x] New resources and/or actions have are included in the matchers.rb and matchers_spec.

[x] New functionality has been documented in the README if applicable.

[x] New functionality has been thoroughly documented in examples (please include helpful comments).

[x] Changes documented in the CHANGELOG.

I guess it would be better to add here as soon as possible o avoid adding code that we know it will be removed right after merge.
Also, you can still open an issue to track this enhancement and also fix it in this PR.

@tmiotto
There is not a new code to test the refresh action, because this test already was created to API200, so there is not code of this PR to be removed after.
But there is no problem for me make the shared example in this PR, this will be very good. =)


getting a ukvm-bin with the gdb module

At the moment, using the "normal" build for Solo5 or Mirage, we'll end up with a ukvm-bin that doesn't have the gdb module in it (which makes sense).  However, it's a bit cumbersome to attach gdb if you do want to debug it via gdb.  I've been editing the generated Makefile.ukvm (https://github.com/Solo5/solo5/blob/master/ukvm/ukvm-configure#L38) by hand if I want to attach gdb through that ukvm-bin, e.g.:
UKVM_MODULE_OBJS=$(addsuffix .o,$(addprefix _build-ukvm/ukvm-,)) _build-ukvm/ukvm-gdb.o
UKVM_MODULE_FLAGS=$(addprefix -DUKVM_MODULE_,$(shell echo | tr '[:lower:]' '[:upper:]')) -DUKVM_MODULE_GDB

Perhaps it makes more sense to have a ukvm-bin-gdb target or similar be generated in the Makefile that already specifies the gdb module?  What do others think?

What about having the Makefile always generate both versions of ukvm: ukvm-bin and ukvm-bin-gdb. Otherwise, telling mirage to generate the gdb versions might involve changes to the mirage tool.

Fixed in mirage/mirage#847.


Dont see update snapshot option on jest watch

🐛 Bug Report
The cli option to update snapshot option interactively i dont see when i run jest --watch
To Reproduce
Steps to reproduce the behavior:
yarn
yarn test:watch
Expected behavior
Per the docs I was hoping to see in the console the snapshot options. If i change one of the components for example, i dont see the diff in the console or the option to update snapshot
Link to repl or repo (highly encouraged)
https://github.com/shavo007/react-boilerplate
Issues without a reproduction link are likely to stall.
Run npx envinfo --preset jest
Paste the results here:
  System:
    OS: macOS High Sierra 10.13.3
    CPU: x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
  Binaries:
    Node: 10.10.0 - ~/.nvm/versions/node/v10.10.0/bin/node
    Yarn: 1.10.1 - ~/.yarn/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v10.10.0/bin/npm
  npmPackages:
    jest: 23.6.0 => 23.6.0

You're snapshotting shallowly rendered App component. You'll see the failing snapshot (and the option to update) when you change something that affects its rendering, e.g. switch the ordering of these two components.
However, if you try to change these components contents, shallow snapshot will not catch it, because shallow renders only 1 level deep into the component tree.

@thymikee  i dont see the option here to interactively update the snapshot?

ignore, i see it now
Watch Usage
 › Press f to run only failed tests.
 › Press o to only run tests related to changed files.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press u to update failing snapshots.
 › Press i to update failing snapshots interactively.
 › Press q to quit watch mode.
 › Press Enter to trigger a test run.

Thanks,
Shane.

because you only have 1 failing snapshot


Templates box does not update automatically to show new templates

Only after window refresh. (Unlike the rest of the application.)

I found the only situation when the template box was not updated after an relevant update and fixed it (when the statement label was changed in the Editor).
I am closing this for now, and if you find any other use case when this happens, please reopen and provide more details.


Add Fortran interfaces for OpenMP library functions

Is your feature request related to a problem? Please describe.
The LOMP library does not, yet, have interfaces for Fortran code to call into the user-visible OpenMP library functions (such as integer omp_get_thread_num).
Describe the solution you'd like
The runtime should provide the relevant functions/subroutines with the appropriate Fortran argument interfaces and mangled names (if you count "adding a trailing '_' to the name" as mangling...)
Describe alternatives you've considered
Not supporting Fortran is possible, but undesirable, given that this is not a huge amount of work.
Additional context
None.

I'll take this task. One of the main things will be to figure out how to best enable a Fortran compiler in build system.

If we want compatibility with the existing runtime libraries, we need to provide the same interface functions as they do, and not require recompilation of Fortran codes.
For GCC, at least, that includes functions with the trailing '_'
...
0000000000012470 T _omp_destroy_lock
0000000000012f50 T _omp_destroy_lock_
00000000000124d0 T _omp_destroy_nest_lock
0000000000012f80 T _omp_destroy_nest_lock_
0000000000022a40 T _omp_display_affinity
0000000000013440 T _omp_display_affinity_
...

Sure.  Let me have a look at this.


feat: add eventbridge schemas

Current dependencies on/for this PR:

11-29-feat_add_apigateway_schemas

PR #9   👈

This comment was auto-generated by Graphite.

Graphite Merge Job
Current status: ⏳ Queued to merge
This pull request is currently queued to be merged as part of a stack.
This comment was auto-generated by Graphite.

Job Reference: j7Bu8pCEUHqFlKED3zbX


implement embedded-hal-async traits using maybe-async-cfg

Fixes #30
To do:

[ ] Add usage example in README
[ ] Add usage example in rustdoc
[x] Add example for microbit v2
[x] Fix errors caused by #[doc = include_str!("delay.md")]'
[ ] Test async stuff

@eldruin I'm having some trouble getting the tests to run for async. Here's my current attempt: https://github.com/hdoordt/lsm303agr-rs/commit/91505ceeb56fa22f49c80b2ac922c5ebbeb5edfe
Sadly, the attributes don't seem to work together all that well. Can you maybe have a look?

The tests themselves in https://github.com/hdoordt/lsm303agr-rs/commit/ded90ca154ded9da145b4c36bc31c163cf7e9759 work fine for me after adding a single .await in accel_mode_and_odr.rs:297 and running cargo test --features='async' --lib.
However, the doctests and example cannot be compiled due to linux-embedded-hal not supporting async. That is expected and I would not try to do anything about it.
It is very interesting to be able to run async tests but honestly I would not have put the effort into it since the code is so far the same.

The tests themselves in hdoordt@ded90ca work fine for me after adding a single .await in accel_mode_and_odr.rs:297 and running cargo test --features='async' --lib. However, the doctests and example cannot be compiled due to linux-embedded-hal not supporting async. That is expected and I would not try to do anything about it. It is very interesting to be able to run async tests but honestly I would not have put the effort into it since the code is so far the same.

Problem is that now the sync tests don't work anymore. I'll leave the async testing out if it's all the same to you

Done and done.
Could you create a release that includes this feature to crates.io? I'd like to use the driver in a training.

I have released this in version 1.1.0.
I have also removed the microbit-example feature, since we are not running the tests with async and async is a requirement for the example. Let me know if that is a problem for you.


Fixed dev-dependencies

See #65 to address the CS issue.

@GrahamCampbell can you consolidate everything in one PR?

@GrahamCampbell I had to force pushed master rolling back to 6.0.3 as the tag 6.0.4 had one issue, could you please rebase your PR with master? thanks

can you consolidate everything in one PR?

The changes are unrelated though, and I couldn't be sure which ones you'd actually want.

@GrahamCampbell it's fine, could you please just rebase the PRs with master? thanks

Done.

Just rebased again.

@GrahamCampbell I see two commits

Just pushed a few fixes.

NB, it's important the lock file isn't committed so that php 5.4/5.4 can use phpunit 4, and later versions can use phpunit 5, since, officially, phpunit 4 is marked as not compatible wit php 7.

:+1:

could you squash in one commit? thanks

Done.

Done, and squashed again.

thanks a lot!
Tomorrow I will probably cherry-pick the good commits from the deleted 6.0.4 tag and create a new release.

In terms of the cs stuff, I'd not recommend pulling in php-cs-fixer and all that stuff. I'd recommend just setting the config on styleci's web interface, then forgetting about it, and let styleci handle it all.


[FEATURE REQUEST] Relate channel parameter to payload field

Is your feature request related to a problem? Please describe.
AsyncAPI 2.0.0/next version supports parameters in channel names which is great. But, there is no context as to where the value comes from. One of the things I see often is that the parameter is populated from the runtime event payload. If we supported noting this context, code generation could do this lifting and make the experience easier for developers.
Can't it be tackled using specification extensions?
The specification needs some sort of a context field that is optional.
Describe the solution you'd like
See below:
{
  "user/{userId}/signup": {
    "parameters": [
      {
        "name": "userId",
        "description": "Id of the user.",
         "context": "payload" <-- add something like this, maybe an enumeration of fields
        "schema": {
          "type": "string"
        }
      }
    ],
    "subscribe": {
      "$ref": "#/components/messages/userSignedUp"
    }
  }
}

Describe alternatives you've considered
N/A
Additional context
N/A

This is a great idea. We can probably use Runtime Expressions as we did with Correlation IDs: https://github.com/asyncapi/asyncapi/blob/master/versions/next/asyncapi.md#correlationIdObject

In case you use "context", "schema" is not needed because it will be taken directly from the definition of the payload field.
Also, pointing to a payload field makes this field mandatory in the payload or we should provide a default value.

I think mandatory is the best option, In my opinion there will be much cases where no default value exists the example above shows it good. if we subscribe or publish to a channel where a "userId" is a parameter a default value would be crazy :). But, in my experience there will be definitly cases where a default value makes sense. So why not both?
parameters:
  userId:
    type: String
    default: 4

If a default value is provided it will be used if nothing was provided, if the default value definition does not  exists, the parameter becomes mandatory, maybe but only maybe a definition of optional: true without a default value can make the parameter optional?

Note to myself: check out #255 before starting to work on this issue.

+1 using a runtime expression here would be a great idea. We're having similar discussion on runtime expressions for selecting entire payloads. Applying them to parameter extraction within a Channel key would be great.


init rushstack zh translation

@chengcyber PLAT, thanks.

Thanks for @chengcyber , I found a lot of the original article forgot to delete, I'm so stupid😅

Thank you @bvanjoi for doing this work!