Add support for DNX

Because the PCLs built here are not System.Runtime PCLs, they cannot be used in DNX. There are projects using Common.Logging and Common.Logging.Core that I would like to port to DNX, but cannot do that without either System.Runtime PCLs, or DNX-specific libraries.
I have a quick prototype (https://github.com/bradwilson/common-logging) using the existing portable source code which seems to imply that it should be fairly simple to go the DNX-specific route.
Thoughts?

This would be a welcome enhancement to common-logging

This makes total sense to me; let me take a look @ this tonight/tomorrow in more detail, but I'm 100% behind this goal...

Merged in https://github.com/net-commons/common-logging/pull/110 ; will review shortly.

+1 for this. Trying to use Microsoft.Framework.Logging and it's a total mess.  Would much rather stick to one dependency than switch between Common.Logging (for full .net) and MS.Logging (for Core CLR).
https://github.com/bchavez/RethinkDb.Driver/issues/12
Hoping for a release soon! :+1:

I've actually already started to look into this and it looks like its entirely feasible to do.  Right now my immediate blocker is actually some arcane failure of the DNX Beta 8 installation that is keeping me from completing my review of this.  Assuming I can solve this in the next few days, I'd be planning a release of this within about a week (preferably before we head into the Thanksgiving Holiday here in the States)...

Great! ... just today,  at connect 2015 conference they just announced .NET Core RC1:
http://blogs.msdn.com/b/dotnet/archive/2015/11/18/announcing-net-core-and-asp-net-5-rc.aspx
And has GoLive licence. :+1:

Yeah, thanks.  I'm already contemplating abandoning troubleshooting whatever is wrong with my beta 8 install and just falling over to RC1 to more easily facilitate my review of this...
-Steve B.
-----Original Message-----
From: "Brian Chavez"<EMAIL_ADDRESS>Sent: 11/18/2015 3:32 PM
To: "net-commons/common-logging"<EMAIL_ADDRESS>Cc: "Steve Bohlen"<EMAIL_ADDRESS>Subject: Re: [common-logging] Add support for DNX (#108)
Great! ... just today, at connect 2015 conference they just announced .NET Core RC1:
http://blogs.msdn.com/b/dotnet/archive/2015/11/18/announcing-net-core-and-asp-net-5-rc.aspx
And has GoLive licence.
—
Reply to this email directly or view it on GitHub.

Curious if there was any progress here.
-- Just checking, not trying to pester

No worries; its a fair question.  I've been doing a crazy amount of traveling but have the week betw XMAS and New Years off with the intent to finish on this during that coming week.
-----Original Message-----
From: "John Fly"<EMAIL_ADDRESS>Sent: 12/22/2015 2:17 PM
To: "net-commons/common-logging"<EMAIL_ADDRESS>Cc: "Steve Bohlen"<EMAIL_ADDRESS>Subject: Re: [common-logging] Add support for DNX (#108)
Curious if there was any progress here.
-- Just checking, not trying to pester
—
Reply to this email directly or view it on GitHub.

Hello, joining the thread of :+1: on really wanting this.  We've traditionally used Common.Logging in the Highway Framework as our general abstraction for logging, and are working on updating to support CoreCLR ourselves right now, and this is a stumbling block.

@trayburn I'd actually put this effort to the side owing to all the recent churn in .NET Core fundamentals (everything from naming conventions to APIs to the right NuGet feeds to get the packages from).  Now that its all (more or less) stabilized with the effort reaching RC2 recently, I can come back to this and hopefully finish it in short order (next few weeks at the latest, I'd be hoping!)

Really looking forward to this, as many of our libs depend on Common.Logging (just like @trayburn ), which blocks our transition to .NET Core.

Agreed; its definitely coming!

OK, I've now pushed the recent changes to MASTER in https://github.com/net-commons/common-logging/commit/78c3b1b19d9e7c6d9e44dd57375483abe368f977 and https://github.com/net-commons/common-logging/commit/2d4eaf0ce36e7f6ddee110f60c70b56c4f6b6c54.  I'd appreciate it if people would take a look at this and comment before I go much farther with it.
Its only an initial implementation, but AFAICT it should (?) work.  A caveat: its missing any ability to configure it via any external config file of any sort.  I'm still exploring the options here (e.g., don't support config-by-file at all, support via some sort of JSON construct that mimics what's in the XML-based config for the non-.NET Core implementation) and would be interested in any feedback/input anyone cares to offer on this point.
As usual, all feedback welcome; hopefully I can devise a config-by-file story for this in the next week or so and push an initial release of it to NuGet subsequently.
Thanks!

@sbohlen
I'm also interested in the DNX support for Common.Logging, as I'd love to implement it for the Couchbase .Net SDK.  We've recently started working moving towards .Net Core support in the SDK with a variety of changes, and this is one of the blockers.
If it's of any assistance, I found an approach for dealing with configuration files that works across .Net 4.5 and .Net Core, though it does require a some compiler directives.  You can see the current work in progress here if it's of any help: http://review.couchbase.org/#/c/63815/
The basic premise is to have both old-style XML Section and Element classes, as well as new style POCOs.  Then let them both share an interface, and let your higher level classes accept the abstract interface.  For the .Net Core build, just exclude the old-style Section and Element classes from the build using compiler directives, so that the POCO configuration approach is the only option.
Brant

Thanks for the suggestion; I'll definitely take a look at that approach shortly.
FWIW, in my research into this on and off over the past several weeks I've also found an MSDN Article that suggests that there is a pretty strong concept in .NET Core around configuration, even if its presently (like much of .NET Core) quite poorly-documented.  For example, there's little (no?) documentation re: just what the XML config would look like under this model.  And while there's at least a small amount of indication re: what the JSON might look like, there's no guidance (at least that I can find) re: just what conventions might be around even simple things like the name of the .JSON file that should be loaded/read/parsed.
Based on what I've seen in samples it seems like it should just be config.json, but that means that Common.Logging settings could (potentially) collide with other settings and also that the consumer of this config would have to ensure their object model of their config properly accommodated Common.Logging's settings.  Right now I'm sort of leaning towards a common.logging.config.json approach where unlike the app.config in 'full' .NET where all settings are jammed into a single file, in .NET Core Common.Logging would be configured entirely separately from anything else in the consuming app (and this would permit Common.Logging to have its own config object model that the consuming app needn't concern itself with accommodating when it reads its 'main app' config out of some other .json file).
Common.Logging does have its DefaultConfigurationReader and its LogSetting classes that do already provide some abstraction points for reading config settings and reporting them to the remainder of the library.  You can even see where the (potential) tie-in to this is already stubbed out here.
Right now my tentative plan is to implement a .NET Core-specific IConfiguration/ConfigurationBuilder-based version of the settings reader that would be "in line" with the proposed way that config seems to want to be handled for .NET Core.  I'm still experimenting with both the feasibility of this approach and the degree to which it offers sufficient flexibility, but initial investigation seems to be (quasi-)conclusive that this should be a viable strategy.

@sbohlen
Yes, that's the article I based most of my work on.  Based on my reading, I think the lack of standards you are referring to is actually a part of the design.  It seems like the premise is to allow the user to structure the file however they like.  Meaning they could put the Common.Logging configuration at any spot in the file they want, in any format they want (XML/JSON/etc) so long as they know where it's at and it follows your POCO class structure.
So, if they want to use "config.json" they can do something like this:
{
  "someConfig": {},
  "logging": { "settings": "here" },
  "someOtherConfig": {}
}

Then they could load your POCOs like this to pass in for configuration:
// Load all their configuration stuff, might be any filename
var builder = new ConfigurationBuilder();
builder.AddJsonFile("config.json");
_jsonConfiguration = builder.Build();

// Do other stuff with configuration

// Send configuration
Common.Logging.SomeInitFunction(builder.Get<Common.Logging.ConfigPOCO>("logging"));

Alternatively, they can define their own POCOs that include the Common.Logging ones:
public class MyConfiguration
{
  public SomeConfigPOCO SomeConfig { get; set; }
  public Common.Logging.ConfigPOCO Logging { get; set; }
}

// To send configuration
Common.Logging.SomeInitFunction(builder.Get<MyConfiguration>().Logging);

Of course, the downside to all of these is that it requires some action on application startup to do the logging configuration.  Because the definition is so flexible, it can't be automatically loaded.  The upside is that the configuration can be done however you like, even built in memory from a Dictionary, which is great for flexibility and testing.
One option might be to expose the ability to configure like this, but if it isn't configured during startup then look for a common.logging.json file as a fallback.
Brant

Interesting (and quite helpful!), thanks!
I completely agree that the flexibility provided by the various ways to 'override' the reading of the config values offers benefits in re: testing, etc.  I also like the concept of offering some sort of 'formal front door' for the consumer to pass the config data into Common.Logging and then for us to also provide a fall-back 'default' file should the system be called absent some sort of 'config method' having first been invoked.
We'll explore an approach sim. to that and see how far it takes us...

@sbohlen -
Any update on when there will be a Common.Logging .Net Core package available?
Thanks!
-Jeff

My plan is to get this configuration story sorted out by approx. mid-month (AUG).
This feature (along with https://github.com/net-commons/common-logging/issues/127) are the two remaining blockers to our being able to release 3.4 GA so I'd expect this to follow shortly after that.  FWIW, the 3.4.Alpha1 release should be functional for .NET Core 1.0 now assuming you config your logging in code instead of via a config file.

@sbohlen -
Awesome, thanks for the update!
-Jeff

@sbohlen
One issue we're having is that the Common.Logging.Portable nuget doesn't appear to support one of the netstandard versions.  This is requiring us to import the old PCL profile instead, which I think may give problems for some consumers.  Any plans to include a netstandard compatible PCL?
https://docs.microsoft.com/en-us/dotnet/articles/standard/library
Thanks,
Brant

@brantburnett
The plan (at least at this time) isn't to have the Common.Logging.Portable package support netstandard.  In our (near-term) estimation, this isn't feasible b/c of the differences in e.g., project system, tooling, etc. between netstandard and all other PCL targets.
Instead, we've already spun up a NetCore-specific project here https://github.com/net-commons/common-logging/tree/master/src/Common.Logging.DotNetCore to address this target.  As of now, this project is functional from a logging PoV but it presently offers support only for configuration via code (no support for config-via-external-file as of yet).
We're working on an external config-by-file story now, but the fact that this isn't (yet) complete is the reason that this package isn't available on NuGet as part of the 3.4-Alpha1 push.  Would you be willing to pull the source, build this assembly for yourself, and perform some tests with it to help ensure that its indeed headed in a direction that will support your needs --?

@sbohlen
I'll give it a shot and see what we can do.  We're actually building a netstandard compatible library, so I'm not sure if it will work or not.  Depends on how we can make the dependencies work and vary them based on the target.

@brantburnett
Our objective/goal for that assembly is that it properly support NetStandard, so if you discover otherwise, please do let us know ASAP.  This is still in active development so we've not (yet) done full compat-testing, but my expectation is that in its present form it should work for you (minus externalized/file-based configuration, which as mentioned is coming but not yet complete).

If there's any way you can publish a pre-release package that contains the NetStandard binaries, that would be awesome.

@sbohlen
I think I may have gotten Common.Logging.Portable to support a netstandard1.3 distribution as part of the nuget package.  I've uploaded a proof-of-concept here: https://github.com/brantburnett/common-logging/commit/293598e03222a5da8ad8a9d0e13c227b08b1b190
As it turns out, you can actually create PCLs that target Net Standard without switching to the new (and soon to be obsolete) xproj system.  It can be done the same old way as before, using a csproj for each framework target.  Add to that a dependency on System.Reflection.TypeExtensions, and most of the reflection issues go away.
This POC probably isn't fully integrated into the build system, but hopefully it's a big step in the direction of Net Standard support without changing your process up too much.
Brant

@sbohlen
Just checking to see if you had a chance to look at the proposed netstandard approach?  If you'd like, I can work on making a pull request from it, I just wanted to make sure you thought the approach would work before I go further.
Thanks,
Brant

@brantburnett
Apologies for the delay -- I'd been traveling on business and didn't see your comment re: your PoC.  Let me review shortly and let you know my thoughts before you spend the time to craft a PR from it.
Standby (and thanks for the input)!
-Steve B.

While checking what it would require to port Spring.NET towards .NET Core I found Common.Logging to be the first show-stopper. I've created a pull request https://github.com/net-commons/common-logging/pull/130 to address the immediate need of having Common.Logging.Core and Common.Logging to contain also the netstandard1.3 artifacts.
Might this be something to sneak into prerelease feed for testing?

@lahma
I'll merge this all in and create a pre-release NuGet package based on it during this coming weekend.

Thanks, sounds good. I'll check if I have time to add Serilog and Nlog building support before that too. They also seem to have .NET Core support already.

As you can see from the CI output from this merge, the build breaks due to the missing .snk file.  Any ideas how best to address that --?  It will build locally on my system where I have the .snk file present, but of course I don't want to commit that to the repo just to make the CI output GREEN.
Any thoughts --?  The only that that immediately comes to mind would be for us to put a 'dummy' .snk file in the repo and then have the build script copy it from somewhere 'special' on my HD into the expected path when I'm building locally so its all signed correctly.  But that's both brittle and error-prone: once the .ignore is tweaked to include .snk files, any commit after a local build where I 'forget' to revert the .snk contents from the 'official' values back to the 'dummy' values would result in the not-for-public .snk being pushed to GitHub :-/
Any ideas how others are addressing this seeming conflict between needing the .snk for a .NET Core build like this and not wanting the .snk to be public in the repo?

@sbohlen
What about using build configurations to control whether it looks for the file or not?  The example segment in project.json below should work if you use a special "Publish" configuration only when building to make the nuget packages.
{
   ...
    "configurations": {
        "Publish": {
            "buildOptions": {
                "keyFile": "../../Common.Net.snk"
            }
        }
    }
   ...
}

https://docs.microsoft.com/en-us/dotnet/articles/core/tools/project-json

Yes this was the pitfall I mentioned... @brantburnett 's solution might be the way. I decided to make Quartz's key file public as it really offers no special protection to keep it secret, anyone can build a "valid" copy now. Even Microsoft has stated that whole strong-naming is a bit obsolete: https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/strong-name-signing.md .

@lahma yeah, I recalled that in your earlier message to me, so I wasn't surprised to see the failure TBH.
FWIW, I'm not sure I agree re: checking the .snk into the repro -- that post you reference is CORRECT that strong naming is used for identity rather than security, but IMO allowing any binary someone arbitrarily creates to 'pretend' to be Common.Logging (or NHibernate, or anything else for that matter) is one more step down a road to untrustable binaries.
Once the .snk is public, what keeps someone from compiling and signing their own Common.Logging.dll that contains all of the functionality of Common.Logging plus phones home to share all PII that might appear in the logs?  Even if its not PII, logs often contain info that would be valuable to me as a hacker (e.g., IP addresses, DB conn string).  I'd argue that validating the identity of a binary from an OSS project is even more important than for a closed-source one because once the code is open/available the above becomes trivial to create and more challenging to defend against (without using a decompiler to inspect the IL to see exactly what the binary might be doing).
Imagine this starts with a nefarious hacker submitting a 'Commn.Logging' package (note the intentional misspelling) to NuGet.  Some subset of people will mistype the name and get that package erroneously installed and I'm off to the races as a bad guy :(
All of this is a round-about way of my saying that I think @brantburnett is probably right in his recommendation .... perhaps I'll try using the .snk file only on RELEASE builds and not DEBUG...that way I needn't specficy a whole other build configuration and CI builds DEBUG builds anyway (IIRC).
Will try that approach next ....

@sbohlen sure, I understand the reasoning and I'm happy that you take the extra mile with protecting the signing key :)
If I reflect to Java land there's no concept of signing the jar files, still they seem to cope with the fact. Information security is getting harder and harder and I guess any measures that can be done to protect malicious acts is good.
I hope that you can tweak the build to work with and without the key. Just something I took a shortcut with.
In the long run it might be easier to use project.json/the-forthcoming-replacement for all building. Now it's a hack and hopefully works. I'm not familiar with creating PCL packages but might be something to consider in v. 4 to just build all with .NET Core tooling.

Yeah, agreed; my anticipation is that with Common.Logging 4 we'll both do that as well as drop most (all?) of the older VS version support so that we've got a much cleaner build system and a (hopefully?) lighter maintenance burden for it.
I just pushed updated project.json incorporating @brantburnett 's suggestion -- FAICT it "works-on-my-machine"; let's see if it gets the CI build to GREEN too -- FINGERS CROSSED :)

OK, as of https://github.com/net-commons/common-logging/commit/91244f1da8110aed530f89d10696d4275dbc8fd5 we're only building DEBUG (and not also  RELEASE) on the CI build, meaning I've opted us out of needing to add the snk file to the public repo for at least the time being.
We're GREEN on the CI server again and I've just packaged and pushed 3.4-Alpha2 to NuGet.
We still need to address the (unresolved) configuration story for Common.Logging in DotNetCore, but I've got that on my backlog for prior to the 3.4GA (unless someone else wants to take stab at in the interim...)

Great, thank you!

@sbohlen
Just wanted to let you know that NetStandard binaries in 3.4.0-alpha2 appears to be working great.  I have seen successful logs using the DebugLogger in ASP.Net Core on Windows, ASP.Net Core on Linux in Docker, and in Xamarin using MonoAndroid.
Now we just need more adapters supporting NetStandard. For that, we need more loggers on NetStandard.  Looks like NLog and log4net are both close.  :)
:+1:
Brant

@sbohlen
I also took a swing at a configuration system that supports .Net Core.
https://github.com/net-commons/common-logging/pull/133
Brant

Closing this issue now that we have both direct support and a viable configuration story for .NET Core.


开放开发分支

Coverage remained the same at 100.0% when pulling 94ab9e74e50258c13b2f0bce74ab71d7e9d8665a on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.

Coverage remained the same at 100.0% when pulling 05713a0208549fb5057df9bcc4ffb3fd64f72301 on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.

Coverage remained the same at 100.0% when pulling 5a4985341101ef7a1823a82ec0e1fd31e70d14c5 on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.

Coverage remained the same at 100.0% when pulling 1126dc2059e133c6f11d291314f422d89aa65e6d on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.

Coverage remained the same at 100.0% when pulling 9eccbf858f6b9530a072fa2b8ff48c78e8e0b25d on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.

Coverage remained the same at 100.0% when pulling 593bb023728b7f249e30741f1c14b50533125e4e on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.

Coverage remained the same at 100.0% when pulling<PHONE_NUMBER>51db067a76baed947dea3a184f0bfc on dev into 36aa9ebef2e476ec942dd01e177dfcec2dc43480 on master.


[BUG] DeepSpeed non-deterministic inference with HF GPT2 when replace_with_kernel_inject=True

Describe the bug
https://github.com/microsoft/DeepSpeed/issues/1950 describes a bug by which running inference twice on the same input leads to different outputs. It was supposedly fixed in version 0.6.5, but I am encountering a similar bug (for Huggingface's GPT2, on an NVidia A10G) in every deepspeed version after including 0.6.3 when running long sequences. My current fix is to use version 0.6.1.
Note: When running too short a sequence this bug does not appear. When running too long a sequence, I am rather seeing another open bug (https://github.com/microsoft/DeepSpeed/issues/2062) which prohibits inference.
Perhaps related bug: https://github.com/microsoft/DeepSpeed/issues/2229
To Reproduce

Install packages

!pip uninstall -y torch deepspeed transformers
!pip install --upgrade pip
!pip install --upgrade torch==1.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
!pip install --upgrade deepspeed==0.7.0 transformers==4.21.1

Run code

import os
import deepspeed
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

os.environ["CUDA_LAUNCH_BLOCKING"] = "1"

model = AutoModelForCausalLM.from_pretrained("gpt2").to("cuda")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

model = deepspeed.init_inference(model, dtype=torch.half, replace_method='auto', replace_with_kernel_inject=True)

long_sequence = "asdfjk **[][] 890 889288 =-0=- 888***&*&#*$&*(#$ &*#$ &*( *(&))  lf  ds890342888977889888***&*&#*$&*(#$ &*#$ &*( *(&))890234908 fdS 809d890342888977889888***&*&#*$&*(#$ &*#$ &*( *(&))fs<PHONE_NUMBER>&*(#$ &*#$ &*( *(&)))"
complex_input = tokenizer(long_sequence, return_tensors="pt").to("cuda")

for _ in range(3):
    outputs = model(**complex_input)
    token_id = torch.argmax(outputs.logits.squeeze()[-1]).item()
    print(tokenizer.decode(token_id), outputs.logits.mean().item())  # we should always see the same output, but we don't

Observe that the output of the last print statement is different each time, although the input was always the same. Last time I ran it, I got e.g.

 Season -125.25
sp -170.25
 A -82.25

Expected behavior
I expected to see the same output each time, i.e.
 Season -125.25
 Season -125.25
 Season -125.25

ds_report output
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
sparse_attn ............ [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-devel package with yum
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
utils .................. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/torch']
torch version .................... 1.12.1+cu116
torch cuda version ............... 11.6
torch hip version ................ None
nvcc version ..................... 11.1
deepspeed install path ........... ['/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/deepspeed']
deepspeed info ................... 0.7.0, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.12, cuda 11.6

System info (please complete the following information):

OS: Amazon Linux 2
GPU count and types: one NVidia A10G (AWS g5.xlarge)
Python version: 3.8.12

Launcher context
inside a Python notebook

Hi @trianxy ,
I think I know where this issue is coming from. It is due to reducing the max-tokens to 128 here. We have a PR to fix this issue. We will merge this soon to resolve this issue.
Thanks,
Reza

Okay, I verify that by changing the MAX_OUT_TOKES to a large enough #tokens, the problem goes away. We will merge the PR soon to resolve this issue.
cc: @cmikeh2

Hi @trianxy,
I'm sorry for the lack of updates on this, but with latest master (should be released as 0.7.5 in the next few days) I believe the issue you're observing here is fixed. Would you mind testing this on your end to verify if this is true?
Thanks!


File Loaders Missing

Function to handle json and xml format loading in models.py are not present.

they have now fixed the xml file handler but not the json yet


[WIP] Use store_true for bool args

Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?
Overview
Fixes #1168
This PR edits pl.Trainer.add_argparse_args to allow users to pass flags instead of key value pairs for bool type arguments with a default of False.
For example, instead of:
python trainer_main.py --fast_dev_run True
You can now do:
python trainer_main.py --fast_dev_run
Notes

Could write this up a bit differently to where we only make a single parser.add_argument call, but I felt it started to look messy when I did that.
Docs not updated. Please point me in right direction and I'll cover that too.
I am completely lost as to why there is a nested function called allowed_type(x) within the loop that gathers the flags. (CC: @alexeykarnachev from git blame)

PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

I am completely lost as to why there is a nested function called allowed_type(x) within the loop that gathers the flags.

@nateraw I'll try to clarify this point.
We want to automatically assign an appropriate type for the argument (parser.add_argument). So, we check the type hint of a specific Trainer argument and in the simplest case we just pass this type to the parser. Then, when we pass the argument value through the command-line interface, it'll be automatically converted to the thing we need.
And now, imagine, that we have some argument, which has a bool type, but it can be passed to the command-line interface like a string: "True", "true", "false", "no", "yes", and so on. So, in such a case, we can't just put bool type for this argument in parser.add_argument method, because in casting time every string will be converted to True, because bool("False") equals True. It means that our argument type is not really a bool, it is a tricky type that casts "True" to True and "False" to False. That's why the argparser can receive not just primitive types, but functions which determine the casting logic. And that's how the allowed_type(x) has appeared. A would say, that allowed_type(x) is not a function, it's a type.

Finishing in #1822


Remove un-needed } in sample code.

The removed } is not required in the block. Results in JSON parse error.

@BenMcGarry : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@BenMcGarry You are correct!  Thanks for noticing and fixing this.  Looks like it's in the same file in two other spots, so I'll go ahead and also fix those instances.
#sign-off


Support for gain and range settings

The "gain" for the SI1145 is different than many other Lux sensors.  While the datasheet labels the settings as "gain", it appears to be implemented as "integration time".  A gain of 32 just means the integration time will be 32x the gain 1 setting.  The response should be approximately linear.

@aaronwmorris if you set up pre-commit as described in the guide page here: https://learn.adafruit.com/creating-and-sharing-a-circuitpython-library/check-your-code you can run the same checks that the actions are doing locally and in the case of code formatting it will resolve it for you automatically.
It'll show you any pylint issues locally as well which will also cause the actions to fail.

@FoamyGuy Thank you for the pointers.  I have corrected all of the issues.

@FoamyGuy I have updated the copyright notices.  I am hoping everything looks good now.


Free libxml2 doc in Opml tests

Fixes #621.
I'll merge this in three days unless someone stops me for a review.

Coverage increased (+0.07%) to 39.212% when pulling 9c5e3c016b5ace0ddc5b4ebd6850279aa63243c1 on bugfix/621-opml-memleak into f5225b8333185a11c6ef7d9d32475c0b66b104c4 on master.


adding a staticwebapp.config.json results in a failed build

The following staticwebapp.config.json results in an error when uploading artificts:
{
  "navigationFallback": {
    "rewrite": "/index.html",
    "exclude": ["/*.{png,webp,eot,svg,ttf,woff,woff2,ico}", "/main.css"]
  }
}

Hi, I'd be curious to see the exact error with which this navigationFallback section errored. Would you mind linking your github repo if public or sharing the github action logs here?

@mkarmark

Full logs here @mkarmark

I'm not able to access that link nor am I able to repro this issue myself. I'll need to take a look at the full logs and the full staticwebapp.config.json to be able to proceed further in this investigation.

logs_453.zip
Does that help? @mkarmark .
In the config I only had:
{
  "navigationFallback": {
    "rewrite": "/index.html",
    "exclude": ["/*.{png,webp,eot,svg,ttf,woff,woff2,ico}", "/main.css"]
  }
}

Where in the directory is the staticwebapp.config.json file located, so we can try to repro as closely as possible?

The build output is in dist/release and that's where the config is.

Hey @sfabriece we think we know what's going on. We should have a fix for this in the next couple of days. In the meantime, our assumption is that the staticwebapp.config.json file has been placed in your 'assets' folder and thus is copied to the output. This is not required for this new config file (one of the changes from routes.json). As long as the staticwebapp.config.json file is in the app_location folder, we will find it and include it in the upload. You should be unblocked if you remove it from 'assets' or remove the mechanism that is copying it to dist/release.

Thanks for the support @miwebst @mkarmark  😄

Hey @sfabriece, our fix for this should be out. Do you mind checking if your project is still hitting this error?

Hi @miwebst , I'll try to get to it in the weekend.

@miwebst for clarity's sake: does the folder have to be named app_location ?

Hi @mmisztal1980, I think you may have this confused with the static site offering from Azure Storage. There is no such blob container in static web apps, these are different products with unfortunately similar naming.

Hi @miwebst , I did that. A few months ago I was working on a static website in blob storage and the .netconf had all this amazing stuff about SWA which I confused for blob sites so had a little trouble with routes.json until I realised my mistake.
Just FYI as I've seen others confused by the similar names too. Might be something to highlight in the docs

that's quite possible...

Closing this one as the initial issue should be fixed, as for the issue with the $web, seems like that is a separate service


report.save not accepting "standalone" as a parameter

System Information

OS: Ubuntu
Python version: 3.10.1
Python environment: pip
Using jupyter: true
Datapane version: 0.15.1

Bug / Issue
When passing standalone to report.save, I get the following error:
TypeError: Report.save() got an unexpected keyword argument 'standalone'
Code snippet:
report.save("path_to_html.html", formatting=dp.ReportFormatting(width=dp.ReportWidth.FULL<EMAIL_ADDRESS>standalone=True )

It looks like https://datapane.github.io/datapane/report/core.html#datapane.client.api.report.core.Report is showing docs for master branch, and not 0.15.1, our most recent release - the standalone parameter has only been added very recently, and isn't released yet.
@shahinrostami will know most about this I think.

It looks like https://datapane.github.io/datapane/report/core.html#datapane.client.api.report.core.Report is showing docs for master branch, and not 0.15.1, our most recent release - the standalone parameter has only been added very recently, and isn't released yet.
@shahinrostami will know most about this I think.

Looping in @mands who knows more about our API reference docs!

Hey @codelancers-sd - just to keep you in the loop, we are looking at a better way to manage our API docs moving forward. As @spookylukey said, the API reference is generated against master which contains changes that haven't made it into the release (yet). The standalone parameter will be available shortly!

I've moved this to our roadmap software (https://datapane.nolt.io/27), so we can close this issue for now!


🐞 : 2048 Game Bug

:red_circle: 2048 Game Bug :
:red_circle: Spell Error for 'Failure' :
:red_circle: Changes :  'Fialure' to 'Failure'
Screenshots 📷
N/A
✅ Details to Include When Taking the Issue:
Name : ArpitCS
Participant Role: GSSOC

Hello @iamrahulmahato, Thank you for assigning the issue to me. I have already made a PR in this regard. Kindly, check and suggest changes as required and if there are no issues please merge it under GSSOC-EXT.
PR:  #1639


Pinpoint with SpringBoot with embedded Tomcat 8 captures only first request

Description
I'm trying to integrate pinpoint with my Spring Boot 1.5.2 and 1.5.4 application deployed on embedded tomcat. As per the different post and issues, I've set profiler.tomcat.conditional.transform=false and also profiler.applicationservertype=TOMCAT. Setting this property allowed my application to be shown in the pinpoint web UI, but what I noticed is only the first request to my application (REST API) is traced and subsequent calls are not traced/collected. Below screenshot shows my application vm args on how javaagent is configured

Also the call stack for the first and only capture request doesn't include the Spring component/resources involved int the call.  (see below screenshot)

Is there anything which I'm missing in configuration or is this a known bug or something?
Environment

Pinpoint version: 1.6.2
OS: macos Sierra & Ubuntu 16.04
Additional details (e.g. how your collector/web is set up): Deployed pinpoint-web (as ROOT) and pinpoint-collector on Tomcat 8

Additional Info

hi jaykhimani
please check you're profiler.sampling.rate option in pinpoint.config
profiler.sampling.rate option governs how often the agent traces transactions. If you set this option to 2 the agent will trace once every 2 transactions (50% sampling), if you set it to 10 it'll trace once every 10 transaction (10% sampling) and default is 20. (5% sampling).

Awesome, Thanks. It worked like a charm.

I'm trying to integrate pinpoint with my spring boot application but I'm getting some error please guide me.

Hi @koo-taejin
I'm trying to configure the pinpoint with my spring boot application but I'm getting some error. P

lease help me

Hi @Ajaya1997
Error logs show it has problem with connecting <IP_ADDRESS>:9991.
Please check whether collector works well or not.
Thanks :)


Add summary of MOVE for mounts to readme

Current coverage is 79.70%

Merging #1159 into master will decrease coverage by -0.01% as of 6223cb0

@@            master   #1159   diff @@
======================================
  Files          162     162
  Stmts         8828    8828
  Branches       263     263
  Methods          0       0
======================================
- Hit           7037    7036     -1
  Partial          0       0
- Missed        1791    1792     +1

Review entire Coverage Diff as of 6223cb0

Powered by Codecov. Updated on successful CI builds.

This looks good to me but assigning @mossprescott for a quick sanity check.
Technically, we need a changelog file in order to merge this. @sellout and I had a heated discussion about whether a change to the README warrants a changelog or not and I lost 😄 .

@garyb: Thanks for the contribution :)
And @jedesah's right, please add a file CHANGELOG/revision.md with a single line like:
- add missing documentation

That's what our merge script uses to figure out how to bump the version.

Ah ok! I'll add that once the 409 conflict description is confirmed.

How's this look? I assume we needed a new changelog file since 6.1.1 has been released?

That wording looks good to me.
The changelog file actually has to be called, literally, revision.md. Our merge script figures out what the new version is based on that and whatever happens to be on master and bumps the actual version for the build. See https://github.com/quasar-analytics/quasar/wiki/Versioning if you're curious.

I see, neat!
I think I've got it this time... at least I should know what I'm doing next time I attempt to contribute!

@garyb: One more thing for the script... this branch needs to be rebased on latest master so the script can merge it with --ff-only. Sorry about all the fuss.

No problem, it's rebased now.

Merged as 7.0.3. Thanks again!


Switch off prefixes for all locales in the client

It would be nice to add toggle of removal locale codes in URLs exactly for the client. And automatically redirects to "clean" version of url in the client only.

This question is available on Nuxt.js community (#c45)

Agree. For example if you use i18n for web-app, it would be nice to remove locale prefixes from URLs.

this might be useful. https://nuxt-community.github.io/nuxt-i18n/routing.html#module-s-configuration-2

Any plan to add this feature @paulgv? It could be very useful 🙂
Maybe a prefix: false, // or null or pages: false, would do the job!

Yep, if you make app with interface localization only, there is no need to prefix language in routes.

@imShara I think it is necessary for Server Side Rendering part. Otherwise search engine bots will not be able to index multi-language sites. May be it should be neglected dynamically on the client side. Add some triggers, you know, some state, that remember current language(preferred locale).

@adetbekov E.g app is forum. Interface is multilangual, content - mixed. What should be in url? What should happens with locale when you send link to thread?


libidn: 1.36 -> 1.37

Motivation for this change
See https://lists.gnu.org/archive/html/info-gnu/2021-05/msg00012.html
for release information.
This release is announced as API and ABI compatible.
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[X] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 123330 at ef3582ce run on aarch64-linux 1

1011 packages marked as broken and skipped:

Literate
OVMF-CSM
adoptopenjdk-hotspot-bin-13
adoptopenjdk-hotspot-bin-14
adoptopenjdk-jre-hotspot-bin-13
adoptopenjdk-jre-hotspot-bin-14
agdaPackages.iowa-stdlib
aldor
ams-lv2
amuleDaemon
...

16032 packages skipped due to time constraints:

AusweisApp2
CHOWTapeModel
DisnixWebService
EBTKS
EmptyEpsilon
Fabric (python38Packages.Fabric)
MIDIVisualizer
MMA
OVMF
OVMF-secureBoot
...

9 packages built successfully:

curl
libidn
libmysqlclient (libmysqlclient_3_1 ,mariadb-connector-c ,mariadb-connector-c_3_1)
lua51Packages.luarocks
luarocks (luaPackages.luarocks)
lua53Packages.luarocks
luajitPackages.luarocks
passExtensions.pass-checkup
slack-cli

1 suggestion:

warning: maintainers-missing
Package does not have a maintainer. Consider adding yourself?
Near pkgs/development/libraries/libidn/default.nix:45:5:
   |
45 |     maintainers = [ ];
   |     ^

@SuperSandro2000 I have done the changes as requested.
I have also re-enabled tests that where disabled for ages (there where report of test failure with gcc-7), and build it successfully. I however have not been able to test the build on darwin (no box available), I’ll see if hydra reports an error.
I have also added myself as maintainer as the package was maintainer-less.


The AOVs from render layer nodes always connected to the Output File Node

Changelog Description
This PR is to make sure the AOVs from render layer nodes always connected to the Output File Node.
Resolve https://github.com/ynput/ayon-blender/issues/37
Additional review information
n/a
Testing notes:

Add some render settings in Ayon server
Create Render Instance with your camera
Publish

Ideally it should be behaving like this mockup I created manually:

@moonyuet it still does not work and to me it seems that it does not take the ViewLayers present into account at the collecting phase as it should propagate AOVs set in the blender addon settings and turn those on automatically but this just happens on the default first ViewLayer and not any other present e.g, Layer2 in my case...
Ideally we should first collect any ViewLayers, then check if AOVs set according to addons setting and lastly connect those to the file output node.
Im really not aware how much of this we already have in place (in terms of code) and which parts we miss completely atm...


Upload session example

For the file size greater than 150 MB we use the upload session. But any example for the same would be great because I am not able to implement it correctly after so many trials.
Thanks!

Thanks for the request! We don't have a sample of using upload sessions in this JavaScript SDK, but I'll be sure to pass this along as a request for one.

Just one more thing. When I am uploading an image using .filesUpload() the image is uploaded to the dropbox without any issues but when I try to open it, it says the image empty. Any other way of uploading images?
Thanks!

The filesUpload method should work fine for images. Please open a new issue with the code you're using and the output you're getting and we'll be happy to look into it and help.

Thanks! I got it wrong. I just completely missed the encoding while reading the file. Was using utf-8.

+1 For this example being added - I can't find a single example of how to use upload sessions! Been struggling with this for a couple hours

It would be nice to also include a batch upload example, as the documentation is a bit lacking and confusing.

I would like to see an example of monitoring upload progress.  The previous SDK had an onXhr member where I could call clientObj.onXhr.addListener() and then use xhr.upload.addEventListener().  Is there access to the XMLHttpRequest object in this new SDK?

Thanks for the requests all!
@kennellink I don't believe this SDK exposes that, but I'll be sure to pass this along as a feature request.

@greg-db Any update on this? Example of using sessions and/or batch upload would be really helpful as the docs are a bit sparse about this (or it's just me that finds understanding code more easy than api docs).

Okay, to be more specific… I read the docs here: https://www.dropbox.com/developers/reference/data-ingress-guide
I struggle with the third step since I don't know what the argument for filesUploadSessionFinishBatch(arg) should be. the api docs only talk about sending a big file in multiple chunks, but not about batch uploading a lot of small files at once. therefore I don't know what cursor, etc. should be.
here's my example code so far with the missing piece commented
batchUpload(files) {
     return Promise.all(files.map((file, index, array) => {
      return this.dbx.filesUploadSessionStart({
        contents: files[0],
        close: (index === (array.length-1)) ? true : false,
      });
    }))
    .then(() => {
      // call this.dbx.filesUploadSessionFinishBatch(args)
      // but got no idea of the args
      // the dropbox docs only talk about uploading one big file in multiple chunks
      // but not about uploading a lot of small files
    })
    .catch((ex) => {
      console.log(ex);
    });
  }

Hi @florianbepunkt I don't have an update on a sample for this right now unfortunately.
As far as the documentation is concerned though, filesUploadSessionFinishBatch is documented as taking a FilesUploadSessionFinishBatchArg argument. That FilesUploadSessionFinishBatchArg itself is a link to documentation for that, which indicates its entries should contain an array of FilesUploadSessionFinishArg, which is itself linked to specific documentation for that. That likewise has information on cursor, and commit.
If you're using filesUploadSessionFinishBatch to upload multiple files, each single entry in the FilesUploadSessionFinishArg array should correspond to one file that you're uploading. You make the cursor using the FilesUploadSessionStartResult.session_id from filesUploadSessionStart. The commit is whatever options you want to set for the upload.
Here's a contrived example that should make this clearer:
testFile1Data = "test file data 1";
dbx.filesUploadSessionStart({
  contents: testFile1Data,
  close: true,
})
.then(function (response) {
  file1Start = response;
})
.catch(function (err) {
  console.log(err);
});

testFile2Data = "test file data 2";
dbx.filesUploadSessionStart({
  contents: testFile2Data,
  close: true,
})
.then(function (response) {
  file2Start = response;
})
.catch(function (err) {
  console.log(err);
});

...
dbx.filesUploadSessionFinishBatch({entries: [
    {cursor: {session_id: file1Start.session_id, offset: testFile1Data.length}, commit: {path: "/testFile1.txt"}},
    {cursor: {session_id: file2Start.session_id, offset: testFile2Data.length}, commit: {path: "/testFile2.txt"}},
  ]})
.then(function (response) {
  finishBatch = response;
})
.catch(function (err) {
  console.log(err);
});

...
dbx.filesUploadSessionFinishBatchCheck({async_job_id: finishBatch.async_job_id})
.then(function (response) {
  finishBatch = response
})
.catch(function (err) {
  console.log(err);
});

Hi @greg-db , thank you for the example. In your example the sequential execution of the promises is not guaranteed, right? So basically testFile2 could finish the uploaded before testFile1.
Is the order of the entries in dbx.filesUploadSessionFinishBatch not relevant? Let's say, testFile2 is uploaded before testFile1 but the entries array order is testFile1, testFile2.

That's correct, in my sample the filesUploadSessionStart call for testFile2 could complete before the one for testFile1.
For the sake of making the example more readable, I separated the different pieces. In a real app, you'd need to make sure you have all of the filesUploadSessionStart calls completed (as well as any filesUploadSessionAppendv2 calls if you need them) before calling filesUploadSessionFinishBatch. The order of the filesUploadSessionStart calls doesn't matter though. The files aren't committed until filesUploadSessionFinishBatch completes anyway.
You can give the entries you supply to filesUploadSessionFinishBatch any order you want, but make sure you store that, as that's the order the results will be returned by filesUploadSessionFinishBatchCheck.

Another request for a working example to be added.  It would be very helpful.

@greg-db Thanks for the example, but:
A) What is the recommended file chunk size (lets say i want to upload 3GB movie to my dropbox)
B) Why is the filesUploadSessionAppendV2 endpoint so ridiculously slow (appending 64KB in ~5sec)

Hi @AlesMenzel :
A) There isn't a particular recommended chunk size, as the optimal value will depend on various factors, such as the network connection itself. A bigger size is more efficient in general, but it increases the risk of any particular request failing. And if a request fails, you need to re-upload all of the data for that request again. A smaller size reduces how much you would need to re-upload for any failure, but it also increases the overhead due to the increased number of HTTPS requests required.
That all said, a 8 MB is probably a good place to start.
B) Using a 64KB chunk size is generally not recommended, as you need to open and close an HTTPS connection very often, increasing the overhead.
Even so, 5 seconds still sounds longer than expected. I just tried it myself and it only took around .6 seconds though, so it sounds like the network connection is likely a factor in this case.

@greg-db
Thank you for your response.
I have made a repository to demonstrate.
Summary:
I have a readable stream that goes through a transformation stream (which buffers up data, /src/transform.js) and then passes it to dropbox upload stream (/src/dropbox.js) that runs the session dropbox API internaly.
Repository: https://github.com/AlesMenzel/dropbox-session-test
IMPORTANT prerequisites:

Include a file to be read into a stream (see index.js - './samples/sample-10MB.mp4')
Please set DROPBOX_TOKEN in the /src/dropbox.js
Please set your DEBUG  env to stream-to-dropbox:* to see whats happening.

@AlesMenzel Have you tried with a larger chunk size, e.g., 8 MB? (You last mentioned 64 KB.)
In any case, this is now getting off topic for this issue, so if you're still seeing very slow performance for these calls, please open a new issue with this information. (That way we don't keep spamming everyone else here.) Thanks!

Yes, the repo is sending 8MB chunks which takes ~45s to upload.

Moved the issue to a separate thread at #141.

Need some help running this code in AWS Lambda function. Can any on help me please?
I tried it and just got an error "Handler 'handler' missing on module 'index'" on the cloud watch.

@anishtimila It sounds like this isn't an issue with the Dropbox library, so I'm afraid I can't be of help. Perhaps someone else here happens to know. Otherwise, you may be better aided on another forum, e.g., for AWS or node in particular. There's a post on StackOverflow that may help though.

The @greg-db sample code looks right to me.  If you want to see some working code, our app includes an upload function that handles single or multiple files, and does multi-part (session-based) uploads on files larger than a specified max chunk size.  It does these operations in serial, allowing you to show progress and cancel if desired.  If interested, take a look at: https://github.com/SynchroLabs/CloudStashWeb/blob/f828bec5d81e3dd5b6784f359cf4bb75d82eb89a/public/script.js#L186-L376

An example would be great too. I've started just today using the DB API today and can do files < 150mb fine... however I'm building a website for a recording studio with 800mb+ files. Will post results if I succeed.

Hi,
I created PR for with example(thanks to @BobDickinson). Need review.
https://github.com/dropbox/dropbox-sdk-js/pull/187

The issue should be closed

Thanks for the reminder! Yes, there is a sample for this now.

here's a solution if you need to stream the file
async function uploadStream(filePath, dropboxFilePath) {
  console.log(`upload stream file '${filePath}' to '${dropboxFilePath}'`)

  const fileChunker = new FileChunker({filePath, chunkSize: 10000000})
  let contents = fileChunker.getNextChunk()

  const fileUploadSession = await dropbox.filesUploadSessionStart({contents, close: false})

  let response
  while (true) {
    contents = fileChunker.getNextChunk()
    const offset = fileChunker.getLastChunkOffset()

    if (fileChunker.isFinished) {
      response = await dropbox.filesUploadSessionFinish({
        contents,
        cursor: {session_id: fileUploadSession.session_id, offset},
        commit: {path: dropboxFilePath, mode: 'overwrite'}
      })
      break
    }

    await dropbox.filesUploadSessionAppend({
      contents, offset,
      session_id: fileUploadSession.session_id
    })
    console.log(`uploaded ${prettyBytes(offset)}`)
  }

  console.log('upload stream finished')
  console.log(response)
  return response
}

const fs = require('fs')
const assert = require('assert')

class FileChunker {
  constructor ({filePath, chunkSize}) {
    assert(typeof chunkSize === 'number', `chunkSize ${chunkSize} is not a number`)
    assert(typeof filePath === 'string', `filePath ${filePath} is not a string`)
    this.filePath = filePath
    this.chunkSize = chunkSize
    this.fileDescriptor = fs.openSync(filePath)
    this.fileSize = fs.statSync(filePath).size
    this.nextChunkNumber = 0
    this.isFinished = false
  }

  getChunk (chunkNumber) {
    assert(typeof chunkNumber === 'number', `chunk number ${chunkNumber} is not a number`)
    const chunk = Buffer.alloc(this.chunkSize)
    const offset = 0
    const length = this.chunkSize
    const position = this.chunkSize * chunkNumber
    const bytesRead = fs.readSync(this.fileDescriptor, chunk, offset, length, position)
    if (bytesRead !== this.chunkSize) {
      this.isFinished = true
      return chunk.slice(0, bytesRead)
    }
    return chunk
  }

  getNextChunk() {
    return this.getChunk(this.nextChunkNumber++)
  }

  getLastChunkOffset() {
    if (this.nextChunkNumber === 0) {
      return 0
    }
    const lastChunkOffset = (this.nextChunkNumber - 1) * this.chunkSize
    if (lastChunkOffset > this.fileSize) {
      return this.fileSize
    }
    return lastChunkOffset
  }
}

const prettyBytes = (bytes) => {
  var sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB']
  if (bytes == 0) return '0 Byte'
  var i = parseInt(Math.floor(Math.log(bytes) / Math.log(1024)))
  return Math.round(bytes / Math.pow(1024, i), 2) + ' ' + sizes[i]
}


Fix Episode Selections

This PR fixes the issue where episode selection buttons stretch out. This problem occurs in anime films or shows with very few episodes.

Thanks @yumiris


[dashboard]Omnibar issues and enhancements

Issues with the current implementation
There are several issues currently affecting the experimental Omnibar functionality:

The Omnibar only opens up on a Dashboard page but not on the Dashboards' list
The Omnibar does not reset the input when closed and opened back. Currently, it keeps the previous search
Selecting an option in the dropdown is not possible with the arrow keys due to the autocomplete
Closing the Omnibar is only possible through reloading. The Omnibar should be closable with the ESC key (this PR introduces this ability https://github.com/apache/superset/pull/16168). I suggest it should close also when clicking outside of the modal.

(thanks to @michael-s-molina for spotting some of these)
Solutions

Make the Omnibar appear on the Dashboards' list with cmd + k or ctrl + k
Clear up the search when the Omnibar is closed
Disable autocomplete
Close the Omnibar when clicking outside

Looking forward
I believe that the Omnibar does not provide much value right now as its only ability is to search for Dashboards. Also, it is only available on the Dashboard page (soon on the Dashboards' list) and nowhere else.
It would make more sense if the Omnibar was available everywhere and would allow searching for the main items in Superset, including the Charts.

I'll start working on the solutions that I have proposed up there as soon as priorities will allow

cc @hughhhh as well who I believe did some of the Omnibar work

@junlincc FYI

@geido @michael-s-molina thank you for spotting it and taking it on. feel free to work on any low hanging fruit to uplevel superset, while I am planning for Q4. :)

And let's remove the feature flag once all the enhancement goes in :)  @rusackas sgty?

I believe the omnibar has been removed as of 2.0, and this is no longer relevant. If I'm taking crazy pills, let me know.


RabbitMQ requests

Hi
Is it possible to work with RabbitMQ queues via httpYac in the same way as https://github.com/novotnyr/rabbitmq-script-idea-plugin ?

@QuAzI Rather not. The syntax is different. If that is not the problem, there should already be a basis. However, I haven't tested it that extensively since my use case disappeared (kafka rules). So you might have to write me a few bugs.


Removes Syndicate headset from uplink and changes TC prices around to be slightly viable

Removes the uplink headset as suggested, changes around TC prices to make a few things more viable

About the PR

Multiple items have had their stock prices changed and the syndicate headset was removed from the uplink as requested, input is desired alongside helpful criticism
Why / Balance

I've seen many an item on the uplink not be viable or just be too bad to actually use as a syndie or nukie, this hopefully adds viability to certain items and encourages some use (.45 armor piercing was increased due to armor piercing actually being a bit fucked)
Technical details

Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog
 :cl:

tweak: The Syndicate has changed around their stock prices and gotten rid of some old dusty headsets

<!--

Do syndie reinforcements get a headset when they spawn in?

They do iirc, I'll check

They do iirc, I'll check

It would be about the only singular use case for the headset that I can think of, as their name makes them valid anyway so having the valid headset wouldn't make much of a difference

They do iirc, I'll check

Traitor syndicate reinforcement does not.

Alright so since it's only traitor it's not problematic?

Ah... So that makes them viable for one use only?

Ah... So that makes them viable for one use only?

Basically, I can imagine a situation in which you buy a reinforcement, and want to get them a headset but can't, but I can't think of a situation outside of that that it would be worth the buy.

I'll wait to see if a maintainer wants it or not, thanks Eclipse

If I can, I'll give them the headset, there's no reason for them not to have it

This removes headsets but not encryption keys, right?

Yes it only removes the headset

Binary encryption key should be 1 tc , it is more than useless otherwise since it doesn't provide much value for its price.


Is MSF broken in latest Apollo master?

I see the default localization module in all Dreamview modes has been changed from MSF to RTK. For example:
https://github.com/ApolloAuto/apollo/blob/master/modules/dreamview/conf/hmi_modes/ipc1_mkz_standard_debug.pb.txt#L29
https://github.com/ApolloAuto/apollo/blob/master/modules/dreamview/conf/hmi_modes/mkz_standard_debug.pb.txt#L41
Does it mean the MSF is not working now?

no, if you need,  you can change the localization mode to MSF.

Hi @rongguodong , it seems that your issue is resolved. And I'm going to close this issue now. With you success in your new career.


Port hecs EntityBuilder::add_bundle

This method adds a convenient way to build an entity from a bundle. Introduced in https://github.com/Ralith/hecs/commit/5d29b318a09b7878dcf84ff2e920760e1b54a100

From the link:
    /// Add all components in `bundle` to the entity.
    ///
    /// If the bundle contains any component which matches the type of a component
    /// already in the `EntityBuilder`, the newly added component from the bundle
    /// will replace the old component and the old component will be dropped.

This seems redundant to Commands::insert. Am I missing some nuance?

Yeah, I believe this is not actual anymore, thanks. EntityCommands provides all the needed functions.


MAINT: Bump version to 0.43.0

Getting ready for new release: #3289

I'll take an "approve" on this PR as a signal that you're happy for us to proceed with making the release :)


Feat(compiler-base-session): Add some methods for Session.

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[ ] N
[x] Y

issue #115
2. What is the scope of this PR (e.g. component or file name):
compiler-base-session
3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[x] Contains experimental features
[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Other

'emit_stashed_diagnostics_and_abort' to emit all the diagnostics and abort.
'emit_stashed_diagnostics' to emit all the diagnostics but not abort.
'add_err' to add error to diagnostics.
'add-warn' to add warning to diagnostics.
'diagnostic_count' to get the count of diagnostics.
4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

[x] N
[ ] Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

[x] Unit test
[ ] Integration test
[ ] Benchmark (add benchmark stats below)
[ ] Manual test (add detailed scripts or steps below)
[ ] Other

6. Release note

Please refer to Release Notes Language Style Guide to write a quality release note.
None

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

57 of 57 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.5%) to 89.341%

Totals

Change from base Build<PHONE_NUMBER>:
0.5%

Covered Lines:
1316

Relevant Lines:
1473

💛  - Coveralls

The reason for code modification should be clearly described in PR or code comments.


New vision/eye MP FO-wave implicit time

If this is a request for a new term, please provide us with the following information.

Preferred term label: abnormal FO-wave implicit time

Synonyms: n/a

Textual definition (the definition should be understandable even for non-specialists. Include a PubMed ID or URL(s) to refer to any relevant article(s) that provides information about the suggested term).

anomaly in the time from the onset of light stimulus to the peak of the FO-wave

Any additional notes to clarify the meaning or use of the term. Please provide a PubMed ID or URL(s) for the source of this information.

Suggested parent term (Please look in the hierarchy of the MP using the MP Browser or other browsers such as OLS and find the place in the MP where the new term best fits)

abnormal ERG implicit time (MP:0012033)

If you would like a nanoattribution, please indicate your ORCID id or provide a label such as MGI:sbello (organization, first initial-last name)

abnormal - MP:0020978
increased - MP:0020980
decreased - MP:0020979

abnormal - MP:0020978
increased - MP:0020980
decreased - MP:0020979


Wine 7 not working as intended on macOS 13.3

macOS 13.3 broke something in Wine 7 that is causing it to be unable to create windows for graphical apps (like Genshin). This is resolved in Wine 8.

It turns out that wine 8 is a solution. CrossOver nightly still use 7.7.

hi @3Shain
i update to latest release 0.0.34
still seeing the same error
▶ cat GenshinImpact_dxgi.log
info:  Game: GenshinImpact.exe
info:  DXVK: v1.10.3-47-gb144ae28-async (macOS)
info:  Found built-in config:
info:    dxgi.customDeviceDesc = AMD Radeon Pro 5300M
info:  Effective configuration:
info:    dxgi.customDeviceDesc = AMD Radeon Pro 5300M
info:  Built-in extension providers:
info:    Win32 WSI
info:    OpenVR
info:    OpenXR
info:  OpenVR: could not open registry key, status 2
info:  OpenVR: Failed to locate module

Library/Application Support/Yaagl OS
▶ cat logs/game_1680151965686.log
esync: up and running.
[2023-03-30 12:52:49.758] MultiThreadStackTrace init success!
GCGMAH active
013c: thread_get_state failed on Apple Silicon - faking zero debug registers```


 string values

We've seem to have encountered either a regression in v2.12.0 or at least some behaviour that was recently changed.
Our values file contains the following:
some_key:

And we're using those values in our environment variables:
env:
  - name: SOME_KEY
    value: {{ .Values.some_key | quote }}

In v2.11.0 this would evaluate to:
              - name: SOME_KEY
                value:

In v2.12.0 however it shows the following diff if we upgrade our helm deployments:
              - name: SOME_KEY
-               value:
+               value: "<nil>"

This is causing quite a few  of our deployments to fail since we're trying to pass the string "<nil>" as a configuration option instead of an empty string value.
Is this a bug or a new feature where empty values are interpreted as <nil>?
Output of helm version:
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.7", GitCommit:"0c38c362511b20a098d7cd855f1314dad92c2780", GitTreeState:"clean", BuildDate:"2018-08-20T10:09:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.2-gke.18", GitCommit:"5796233393d7bc034428de15191ad3d2eaff95fb", GitTreeState:"clean", BuildDate:"2018-11-08T20:49:08Z", GoVersion:"go1.10.3b4", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): GKE

I am able to reproduce this. The cause looks to be a change in how nil values are handled in the text/template package between version 1.10 and 1.11 of golang. Building Helm 2.11 with go 1.10 results in value:, but building Helm 2.11 with go 1.11 results in value: "<nil>".
I'm working on a fix.

This is caused by a change in Go 1.11, where untyped nil values are now passed in to template functions when using the text/template package as described here. Unfortunately, some of the template functions in the sprig library are not expecting nil values and that is why you are seeing the "<nil>" output.
I think this needs to be fixed in sprig, and I have a PR that at least addresses this for some of the string functions https://github.com/Masterminds/sprig/pull/133.

Does anyone came up with workaround for this? Downgrade is a bad option for us. Thanks

@gillesdemey Can you test if this is now fixed? @mortent provided a fix https://github.com/Masterminds/sprig/pull/133, which was pulled into Helm via #5300.

@gillesdemey Can you test if this is now fixed? @mortent provided a fix https://github.com/Masterminds/sprig/pull/133, which was pulled into Helm via #5300.

I think we can safely close this issue for now as this issue was referenced directly in the report. Please re-open if this persists. Thanks!


Added a DomainLocaleGuesser

As discussed in #141, here is the PR to include a DomainLocaleGuesser.

Released 2.4.1
Maybe backporting to 2.3 also

Thanks for the merge.

very cool @coudenysj !
@lunetics i would not backport new features to 2.3, once there is a 2.4 the 2.3 should only see bugfixes to minimize the risk of breaking things (and to reduce workload)

yep, you are right :) we should rather encourage updating to 2.4 :)


[libllbuild] Drop back to API version 4, for now.

Nice, I like the partial approach.  We'll get to 5 one of these days. 🤞

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test


Update SecretStore API documentation to account for Features feature.

Thank you for helping make the Dapr documentation better!
Please follow this checklist before submitting:

[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)
[x] Read the contribution guide
[x] Commands include options for Linux, MacOS, and Windows within codetabs
[x] New file and folder names are globally unique
[x] Page references use shortcodes instead of markdown or URL links
[x] Images use HTML style and have alternative text
[x] Places where multiple code/command options are given have codetabs

In addition, please fill out the following to help reviewers understand this pull request:
Description
Updates secret stores data/components and partial HTML fragment to account for this new feature, updating the table of supported secret stores.
Updates Secret Store API page to refer to this feature in the secret retrieval API documentation.
Updates HashiCorp Vault documentation to explicitly mention this feature is supported and used by HashiCorp Vault.
Issue reference
Please reference the issue this PR will close: #2787

@tmacam, This helps describe which secret stores support the features feature, but it's not clear how you use then use the feature once enabled?

@greenie-msft :

@tmacam, This helps describe which secret stores support the features feature, but it's not clear how you use then use the feature once enabled?

To be clear: for local files, there is a flag that enables this feature but for HashiCorp vault this is a description of its behavior. I can perhaps add a sample output in the API docs to make this clearer.

@tmacam - You may not have seen this ask above ^^
Also change the multivalued property in the secret filestore  table to say
| multiValued | N | 'true' sets the multipleKeyValuesPerSecret behavior. Allows one level of multi-valued key/value pairs before flattening JSON hierarchy. Defaults to "false" | "true" |

@tmacam - You may not have seen this ask above ^^
Also change the multivalued property in the secret filestore table to say
| multiValued | N | 'true' sets the multipleKeyValuesPerSecret behavior. Allows one level of multi-valued key/value pairs before flattening JSON hierarchy. Defaults to "false" | "true" |

Oh! I missed it was in the file secret store. One sec...

Validate-links is blocked due to https://github.com/dapr/samples/pull/120, which is not merged but is referred in dapr/docs v1.9 already.


Loading state

Version

react-native-router-flux v3.30
react-native v0.28

Is possible to create a loading state in overlay (with semi-transparent background, so it's possible to see the scene that triggered the loading)?
I'm playing with the modal but even if I add a transparent background I can't see the scene behind it. Am I doing something wrong?
Thanks

Figured it out.... just follow the Example!

@zelphir How did you go about this? I'm trying to get a transparent background as well, but can't seem to get it. I've managed getting my Scene like a modal but not the transparent part. Please share! 😄

Same issue.
@zelphir, please share your code.
@believer, have you solved that?

@believer and @Daniellt1 , as @zelphir mentioned, just follow the Example


Please kindly help answer : at the first layer how to distinguish low-frequency and high-frequency feature maps

Please kindly help answer : at the first layer how to distinguish low-frequency and high-frequency feature maps

You don't have low-frequency feature maps before the first layer. The one-octavte-lower feature maps are created by the OctConv layer as a low-frequency feature extraction method. Any feature maps prior to the first OctConv layer are considered high-frequency.

You don't have low-frequency feature maps before the first layer. The one-octavte-lower feature maps are created by the OctConv layer as a low-frequency feature extraction method. Any feature maps prior to the first OctConv layer are considered high-frequency.

You don't have low-frequency feature maps before the first layer. The one-octavte-lower feature maps are created by the OctConv layer as a low-frequency feature extraction method. Any feature maps prior to the first OctConv layer are considered high-frequency.

So,

You don't have low-frequency feature maps before the first layer. The one-octavte-lower feature maps are created by the OctConv layer as a low-frequency feature extraction method. Any feature maps prior to the first OctConv layer are considered high-frequency.

Thanks for your reply first.Do you mean that the first OctConv randomly selects a certain percentage of input feature maps, reducing their resolution as  low-frequency feature maps?

You don't have low-frequency feature maps before the first layer. The one-octavte-lower feature maps are created by the OctConv layer as a low-frequency feature extraction method. Any feature maps prior to the first OctConv layer are considered high-frequency.

Thanks for your reply first.Do you mean that the first OctConv randomly selects a certain percentage of input feature maps, reducing their resolution as low-frequency feature maps?

Short answer, no.
Here's the TL;DR: as is explained in the paper:

We set α_in = α_out = α throughout the network, apart from the first and last OctConv of the network where α_in = 0, α_out = α and α_in = α, α_out = 0, respectively.

In the paper α_in = α_out = α = 0.5 is set across the whole network, except for the first and the last layers. So to answer your question, all channels are used in the layer to go thru two convolutions separately, one with W_HH as weights and one with W_HL as weights, where the number of the kernels of W_HH are set to (1 - α_out) * c, and W_HL to α_out * c, respectively.
As a result, the first layer will produce two outputs, the high-frequency feature maps with (1 - α_out) * c channels, and the low-frequency feature maps with α_out * c channels.
The operation in the last layer is similar just going the other way around.
Hope this helps.


Supplement Panels

[ ] neither fixed 5050, cupy

[x] rough in
[ ] caption
[ ] clean up

[ ] neither fixed denovo, cupy

[x] rough in
[ ] caption
[ ] clean up

[ ] neither fixed, wse 2048-a-tile

[x] rough in
[ ] caption
[ ] clean up

[ ] neither fixed, wse 32-a-tile

[x] rough in
[ ] caption
[ ] clean up

[ ] neither fixed, wse 256-a-tile, denovo

[x] rough in
[ ] caption
[ ] clean up

[ ] neither fixed, wse 256-a-tile, 5050

[x] rough in
[ ] caption
[ ] clean up

[ ] unaccrued 5050, cupy

[x] rough in
[ ] caption
[ ] clean up

[ ] unaccrued denovo, cupy

[x] rough in
[ ] caption
[ ] clean up

[ ] fixprob heatmap 5050, cupy

[x] rough in
[ ] caption
[ ] clean up

[ ] fixprob heatmap denovo, cupy

[x] rough in
[ ] caption
[ ] clean up

[ ] fixprob heatmap, wse 2048-a-tile

[ ] rough in
[ ] caption
[ ] clean up

[ ] fixprob heatmap, wse 32-a-tile

[ ] rough in
[ ] caption
[ ] clean up

[ ] fixprob heatmap, wse 256-a-tile, denovo

[ ] rough in
[ ] caption
[ ] clean up

[ ] fixprob heatmap, wse 256-a-tile, 5050

[ ] rough in
[ ] caption
[ ] clean up

need site explicit panels


How can i get "_type" value in custom Rule

Hello
I'm trying to make a custom Rule where i need to compare value from several documents.
Each document have is own '_type' and i want to base my comparison on that.
I saw in the code that elastalert get only '_source' fields but i saw also that portion of code in elastalert.py :
if 'doc_type' not in rule and len(hist):
rule['doc_type'] = hits[0]['_type']
return hits
So my question is : how can i get/use "_type" value in custom Rule ?
regards

I finally found what to do. I added hit['_source'].setdefault('src_type', hit['_type']) as last instruction in "process_hits" static method


Fixed the issue of could not update user role on the Manage Users page.

I found a issue of we colud not update user role on the "Manage Users" page.
It seems we can fix it with using setRole instead of getRole in the function update of UserController.php.
I'm happy if the fix could be useful.

Thanks!


jwt isValidIssuer check not strict enough?

Hi, while perusing the code I noticed this function:
https://github.com/clerk/clerk-sdk-go/blob/685118cf63abd0ff4269af0ef924d9bebff0b91d/jwt/jwt.go#L127-L133
It appears to accept any issuer whose first domain segment is clerk.. Is this intentional? While it may not lead to a vulnerability on its own, it seems like it would have to be undesirable to accept an issuer like https://clerk.example.com (or substitute any attacker domain suffix).
Again I'm not saying this is itself a vulnerability, but if this check has any purpose at all then it would seem to be not doing a good enough job of serving it.

Hello @bgentry,
You are right, the issuer validity check is incomplete. I would say that it's a compromise though.
A more robust and complete check would be to verify that the JWT's issuer is the same as the instance's Frontend API.
Here's an example. If your domain is github.com then most likely your FAPI lives in clerk.github.com, and that's what the JWT's iss claim should be set to.
Unfortunately, the Go SDK currently has no way of knowing your domain without contacting Clerk's servers.
The problem could be easily solved if users provided their FAPI (or domain) when setting up the SDK, but we wanted to avoid having to pass too many configuration options.
The check (and its rationale) was carried over from v1. It provides a simple check leveraging just enough information, but it's by no means complete.
Hope the above explains the rationale behind this function.
Now, where do we go from here? I guess at some point we'd want to provide more configuration options to the SDK, especially if we want to support cookie based authentication too.
We should definitely revisit the iss claim validation. Is this currently a blocker for you?


HV-1472 Update our WildFly integration to 11.0.0.Final

https://hibernate.atlassian.net/browse/HV-1472

For now, we keep the 10.1 patch, waiting for the next development cycle
to start. Once it's started, we will switch to the next version.

waiting for the next development cycle to start.

Which one do you mean? HV 6.1 or WF 12? +1 for keeping the 10.1 file for the time being, it should be useful.

I mean the first alpha of WF 12.


Problem with showing a modal window while another window is focused

Hi,
If a modal window is shown though clicking a button in another window, the modal window is not focused and the other window is shown above the backdrop, even though I call focus() on the modal window. When the modal window gets focus via clicking, the other window goes behind the backdrop and everything becomes OK. A workaround is doing that in a setTimeout(() => modalWindow.show().focus(), 0), but of course this is not desirable.
Edit: This is not specific to modal windows.
Also, there is no way to blur a window... maybe adding a blur() method would be nice.
TIA

Same problem here

This should be fixed now. Please try again.

Call with modal:true don't display modal window (in result div: display:none)

For me there is a conflict if you are also using bootstrap 4/5. Setting modal:true in the config section has the following effect:

winbox .modal class is overridden by the bootstrap .modal class.

If I rename all instances of "modal" in the winbox .js and .css files to something else, ie 'modals' then I see the window pop correctly as a shown on the examples site.


[Feature Request] Immersive mode

Hi! First of all, thank you for this amazing app!
Would an "Immersive mode" be possible? By that I mean an option that when activated, would hide the notification bar at the top when reading. I find the bar quite distracting when reading, especially because the text crosses it when scrolling.

I have already added it. Should be there in next release.
https://user-images.githubusercontent.com/57822170/120883994-1800c200-c5fe-11eb-9bc9-591454bdc1b1.mp4

Wow, so fast. Thank you very much!


CMS Upgrades version 4.0.x to 4.1.1

Automated CMS upgrades will run upgrading customers from CMS version 4.0.x to 4.1.1 where they have the option to automatically upgrade to new minor versions enabled.
As each CMS is upgraded, there will be a brief period (normally a few minutes) where the CMS may be unavailable.
Upgrades are carried out at off-peak times for the region that your CMS is hosted in.

Update: We will be postponing automatic upgrades from v4.0.15 to v4.1.1 for customers who have interactive actions setup in their CMS since a problem has been found with previewing these layouts in the CMS. We will schedule further automatic upgrades once that problem is resolved. Other customers with v4.0.15 CMS instances will see their CMS automatically upgrade as planned.

Complete: All automatic upgrades have been completed.


MSID: 96433 Version: 1 DOI: 10.1101/2<IP_ADDRESS>6666

MSID: 96433
Version: 1
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>6666
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=96433
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>6666

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/96433v1
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>6666
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/96433/v1/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

Hi @fred-atherden, as this is press, it can be QC, but I did notice Figure S1 captures Figure S1 and S2 in the same image, with the S2 figure caption captured as normal article text below the figure. This is how its captured on the preprint so maybe best to leave as its been captured.

Hi @rkcook312 - thanks for flagging. I reckon we should fix that. I've uploaded a corrected package. Should be ready to check in a few hours.


Windows installer improvements (working)

Move manual init cmd to before ExecuteAction in InstallUISequence to make it work properly.
Move dependencies out to separate file and add script to update dependencies.
Allow for setting config file path during install instead of running init.

I signed it!


Meta variables with duplicate name are lost

It is common to add hashes to test rules on in the meta variables like so:
rule test
{
    meta:
        // The files that should match this rule. It will be tested against these.
        hash = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
        hash = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
    condition:
        true
}

The current API loses the first hash, because the result is a map keyed by the variable name.
Possible fix, which will put []interface{} into the metas map[string]interface{} map:
diff --git a/rule.go b/rule.go
index e0b7e7d..ef0966c 100644
--- a/rule.go
+++ b/rule.go
@@ -132,15 +132,30 @@ func (r *Rule) Metas() (metas map[string]interface{}) {
        for _, m := range mptrs {
                var id, str *C.char
                C.meta_get(m, &id, &str)
+               goId := C.GoString(id)
+
+               var val interface{}
                switch m._type {
                case C.META_TYPE_NULL:
-                       metas[C.GoString(id)] = nil
+                       val = nil
                case C.META_TYPE_STRING:
-                       metas[C.GoString(id)] = C.GoString(str)
+                       val = C.GoString(str)
                case C.META_TYPE_INTEGER:
-                       metas[C.GoString(id)] = int(m.integer)
+                       val = int(m.integer)
                case C.META_TYPE_BOOLEAN:
-                       metas[C.GoString(id)] = m.integer != 0
+                       val = m.integer != 0
+               default:
+                       continue
+               }
+
+               if oldval, prs := metas[goId]; prs {
+                       if sl, ok := oldval.([]interface{}); ok {
+                               metas[goId] = append(sl, val)
+                       } else {
+                               metas[goId] = []interface{}{oldval, val}
+                       }
+               } else {
+                       metas[goId] = val
                }
        }
        return

I'm not too happy with it though, too many interface{} :/

Hi @Tasssadar,
thank you. I wasn't aware of this problem. I think you're right about an overuse of interface{}...
What do you think about adding a separate method? Let's call it Metas2 that returns a map[string][]interface{} where the individual interface{} values would be guaranteed to be a nil, string, integer, or boolean value just like with the existing Metas method.

For what it's worth, the Python bindings show the same behavior:
#!/usr/bin/python3
import pprint
import yara

rs = yara.compile(source="""
rule test
{
    meta:
        hash = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
        hash = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
    condition:
        true
}
""")
ms = rs.match(data="")
pprint.pprint(ms[0].meta)

Here's a pull request on the python implementation. They basically added a second Meta() method that can return lists: https://github.com/VirusTotal/yara-python/pull/74
Perhaps add an api that would return map[string]MetaEntry, and MetaEntry would have nice methods to work with:

IsString, IsInt, IsBool, IsList....
String() string, Int() int (panic when the type is wrong?)

It's a little bit unfortunate that each of the variables in list can have a different type. If we'd restrict this artificialy in go-yara, we could have nice methods like:

Strings() []string, Ints() []int...

I don*t like the combination IsString() bool / String() string (with panic). You end up having more explicit library code compared to passing back interface{}s with a documented selection of 3 or 4 types. And the burden of checking for the correct type is still left with the caller -- without allowing for language features such as type switches.

@Tasssadar I'd like to put the named identifier and interface{} value into a new Meta type and have a MetaList method that returns a list of Metas, in the same order that the meta variables were defined (and how the YARA C API returns them). What do you think?

That sounds good to me, probably better than to return []interface{} for each meta variable name.

Hello, any progress on this? I think the solution with MetaList/MetaMap looks pretty good, could you merge it to master please?

Oh, yeah, I had totally forgotten about that...


PasswordStrengthMeter added on registration

I Added Very Basic Password Meter  Based On My knowledge.  If It is Okay please merge

@abhiimali  The strength detector should come below confirm password .

i am done with required changes . please check it again and confirm


rabbitmq source - does not recconect?

We switch to rabbitMQ as message broker.
When I define rabbitMQ as source and restart rabbitMQ broker, wso2/siddhi app will not reconnect.
I try to define heartbeat, durable, etc... but every time when rabbitMQ is restarted app will not reconnect and pull new messages. We need to restart worker also.

@Idriel sorry about the delay. We'll check on this and get back to you.

Was able to reproduce the issue and raised a PR to fix that.
https://github.com/wso2-extensions/siddhi-io-rabbitmq/pull/47

Raised an issue to verify the retry logic for other Siddhi source implementations - pull mode.. https://github.com/siddhi-io/siddhi/issues/1407

We have released the siddhi-io-rabbitmq version 1.1.2 with this fix..


Build scripts may not play nicely with dummy sources

Discussed in https://github.com/ipetkov/crane/discussions/115
Seems like cargo may be looking at the timestamp of the build script (or at the timestamp of the artifacts) to determine when it is being run. Since nix strips out the timestamps, cargo may not be (re)running the actual build script with the real files.
One potential solution is to make the dummy build script have a unique name (i.e. not build.rs and not whatever build script the Cargo.toml file may set), though I'm not sure what implications this may have on build artifact caching, if any

I'm not sure what implications this may have on build artifact caching, if any

Now that I think about it - why does crane even produce fake build.rs scripts in dummySrc?

why does crane even produce fake build.rs scripts in dummySrc?

To get cargo to build any build-dependencies so we can cache their artifacts


Wallet > History: Rename Recipient Address to Sender Address

Hello,
In the transaction details in History, as I understand it, the Inputs are the senders UTXOs which are used as inputs to the transaction and the Outputs have 2 outgoing addresses - one is the address of the receiver (recipient) and the other is the sender's address used for the returning utxo.
The confusing thing is having "Recipient Address" in the input section, when it is not the recipient's address but the senders.
Perhaps the labelling can be renamed:
Inputs: Sender Address
Outputs:  Recipient/Outgoing Address and Sender/Change/Return Address
Sending 10 Minima to the address:  MxYYUPDWVQKFOJQL3JGQKWIEMAVB3QTXLT

Thanks,
Jazmin

IMHO the fields in history should be completely unambiguous
Senders address
Recipients address
Change address
currently Recipient is mis labelled, it is referring to the sender which is the opposite!!

This is now changed in v1.1.4,
Input address is renamed to just Address.
Output address now recognises which address is your change address, and what address is going to the recipient/receiver

Tested and confirm the above - Change address is always my address and outgoing address is the counter party's address.


Allow a device DID to use a delegated grant as an owner to store any third party authored message in owner's DWN

Note both delegated grant as an author and delegated grant as an owner can coexist in a message since they authorize different things.

Done by #707.


How to obtain the d.t.s file in 2.3.0 ?

Hi,
I have been using your library in version 2.0.0 for about 2 weeks, and I saw this morning that you upgraded to version 2.3.0. Everything seems fine except that I cannot find the declaration file.
Here is how I use ts-events :
/// <reference<EMAIL_ADDRESS>/>
import {SyncEvent} from "ts-events";

Is there a new way to obtain it ?

Typescript should now automatically take the dist/index.d.ts file so you
should be able to remove the reference to ts-events.d.ts .
On Feb 21, 2016 10:02 AM, "Pierre"<EMAIL_ADDRESS>wrote:

Hi,
I have been using your library in version 2.0.0 for about 2 weeks, and I
saw this morning that you upgraded to version 2.3.0. Everything seems fine
except that I cannot find the declaration file.
Here is how I use ts-events :
/// import {SyncEvent} from "ts-events";
Is there a new way to obtain it ?
—
Reply to this email directly or view it on GitHub
https://github.com/rogierschouten/ts-events/issues/4.

I see that you added a "typings" key in package.json, but the dist/ directory is not indexed in git.

That is correct. The Javascript output is in NPM only, not in Git

Oh dear, I see that the latest version doesn't work at all after publishing to NPM.  It worked when I 'npm link'ed it to a test module. I'll revert and publish the old version under a new number so everyone can continue. Thanks for reporting.

v2.4.0 published on NPM, all changes reverted. I hope it works for you, otherwise please just clamp your version to 2.0.0 in the meantime. I'm out for the day.

Ok, i'll just clamp my version to 2.0.0 for now. Thanks for your quick answers !

Version 3.0.0 is out and should work. @hnb2 can you please try and report back? Thanks!


Shorter way to add extra data to Sentry message

Hi guys. Can you suggest how to write the following into one line:
$eh = new \Sentry\EventHint();
$eh->extra = ['data' => ['a', 'b', 'bla-bla' => 1]];
\Sentry\captureMessage('egisz_callback', Severity::info(), $eh);

Older versions of this library allowed to pass an array argument for extra. Can you please help to figure out a shorter notation?

A EventHint can be created from an array, this might be a little shorter?
\Sentry\captureMessage(
    'egisz_callback',
    \Sentry\Severity::info(),
    \Sentry\EventHint::fromArray([
        'extra' => ['data' => ['a', 'b', 'bla-bla' => 1]],
    ])
);

You can also include a helper in your project to make it even shorter:
function sentry_message(string $message, ?array $extra = null, ?\Sentry\Severity $severity = null): void
{
    \Sentry\captureMessage(
        $message,
        $severity ?? \Sentry\Severity::info(),
        \Sentry\EventHint::fromArray([
            'extra' => $extra,
        ])
    );
}

and use it like:
sentry_message('egisz_callback', ['data' => ['a', 'b', 'bla-bla' => 1]]);

Hope that helps!

A EventHint can be created from an array, this might be a little shorter?
\Sentry\captureMessage(
    'egisz_callback',
    \Sentry\Severity::info(),
    \Sentry\EventHint::fromArray([
        'extra' => ['data' => ['a', 'b', 'bla-bla' => 1]],
    ])
);

You can also include a helper in your project to make it even shorter:
function sentry_message(string $message, ?array $extra = null, ?\Sentry\Severity $severity = null): void
{
    \Sentry\captureMessage(
        $message,
        $severity ?? \Sentry\Severity::info(),
        \Sentry\EventHint::fromArray([
            'extra' => $extra,
        ])
    );
}

and use it like:
sentry_message('egisz_callback', ['data' => ['a', 'b', 'bla-bla' => 1]]);

Hope that helps!

Hi, I implemented my code same way, but I am not able to see any extra data in Sentry web interface. I am not sure if implementation is correct or I need to make some additional setup to allow extra data or something? Many thanks

Yes I am sorry, I misunderstood our own code 🤦
This will work (tested it myself this time):
function sentry_message(string $message, ?array $extra = null, ?\Sentry\Severity $severity = null): void
{
    \Sentry\withScope(function (\Sentry\State\Scope $scope) use ($message, $extra, $severity) {
        $scope->setExtras($extra);

        \Sentry\captureMessage($message, $severity ?? \Sentry\Severity::info());
    });
}

Sorry for the confusion!


A new doc for the JWT based SSO

Describe the improvement
Please create an official doc for the doc [3] https://mifrazmurthaja.medium.com/jwt-sso-with-wso2-identity-server-cf486ffba1bf

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


new parser: doesn't manage correctly default value

Example
Nor
parser.add_option("-s", "file", "segmentation", False, "spgr10_crop_seg.nii.gz", default_value='')
neither
parser.add_option("-s", "file", "segmentation", False, "spgr10_crop_seg.nii.gz", default_value=[])
neither
parser.add_option("-s", "file", "segmentation", False, "spgr10_crop_seg.nii.gz", default_value=0)
work if we don't specifiy something in -s flag, although it is defined as an optional flag.

This is not really an issue actually, because you shouldn't be allowed to take a empty value for an input file... I would say that no default value should be available for "file", "image_nifti" and "directory" options