window.jwt_decode is undefined

https://github.com/auth0-blog/angular2-authentication-sample
I took the code for above link and I am applying jwt related code to this project but I get this error.
browser_adapter.js:77 ORIGINAL EXCEPTION: TypeError: window.jwt_decode is not a function
And I included this code to custom-typings.d.ts
interface Window {
jwt_decode(jwt: string): any;
}
Any idea?

look in their inde.html
https://github.com/auth0/jwt-decode

i don't see inde.html nor index.html file in this repo

https://github.com/auth0-blog/angular2-authentication-sample/blob/master/index.html
https://github.com/AngularClass/angular2-webpack-starter/blob/master/src/index.html

i see this line in index.html file.

But when I include that in my index.html
It can't load the file.
http://localhost:3000/node_modules/jwt-decode/build/jwt-decode.js (Not found)
And I am trying to import this file in vendor.browser.ts but I can't proceed because this library doesn't have .ts file
Do you have any suggestion for this?

hi @gdi2290 do you have any opinion?
I appreciate thanks.

in custom-typings include
declare module "jwt-decode" {
  function decode(token: string): any;
  export = decode;
}

then in vendor
import 'jwt-decode';

then in your app
import * as jwt_decode from 'jwt-decode';

let jwt = 'wat'; // your jwt
let result = jwt_decode(jwt);
console.log(result);

hi @gdi2290 it works and when I debug 'let result=jwt_decode(jwt)' it actually calls the function from node_modules/jwt-decode.
But I can't understand how it calls the node_modules function?
Because "import 'jwt-decode'" in vendor will look up custom-typings file
can you explain a little bit please?

hi @gdi2290 it works and when I debug 'let result=jwt_decode(jwt)' it actually calls the function from node_modules/jwt-decode.
But I can't understand how it calls the node_modules function?
Because "import 'jwt-decode'" in vendor will look up custom-typings file
can you explain a little bit please?

@gdi2290 the code is working it shows the error around this code.
import * as jwt_decode from 'jwt-decode';
[default] /home/georgismall/Project/closetbox-new-admin/src/app/app.component.ts:11:29
Module '"jwt-decode"' resolves to a non-module entity and cannot be imported using this construct.
But it works, any idea?

@georgi-kovachev can you update the type definitions to
declare module "jwt-decode" {
  function decode(token: string): any;
  namespace decode {}  // notice how we have to create a namespace that is equal to the function we're assigning the export to
  export = decode;
}

there needs to be a namespace

Thanks it works.

I'm not getting this to work. I still get Message: jwt_decode is not a function as an error. I get no errors from typescript or webpack so it seems like the imports, type definitions and webpack bundling works. But I still get the error in runtime and I do not know how to debug it. Anyone?

declare function jwt_decode(token: string): any;

is different from
declare module "jwt-decode" {
  function decode(token: string): any;
  namespace decode {}  // notice how we have to create a namespace that is equal to the function we're assigning the export to
  export = decode;
}

the declare you have is saying jwt_decode is a global function while the other one is saying the type returned from the "jwt-decode" module is a function. With the latest changes to typescript 2 you shouldn't need to do this anymore


Handle overflow in number parsing

In some areas of number parsing, e.g. ++dig_ or ++sig_ we need to check for overflow / wraparound and handle it. This will also need tests.

The only variable that has potential to overflow/underflow is bias, and this will only happen in extreme cases (e.g. numbers > 2.4 billion digits long).


Use onbuild

If we use an ONBUILD instruction this container can be automatically build via docker.io
example https://registry.hub.docker.com/u/sstarcher/hound/
Once that is happening instead of cloning this repo and adding a config.json locally you could do the following.
Dockerfile
From sstarcher/hound
config.json
Whatever your config is
Those 2 files would be enough to run hound.

Be patient with me. Sadly, I know very little about Docker. How do the commands in the README need to be adjusted to use this Dockerfile?

So every time I run the image, it is unable to find /hound/config.json. When does ONBUILD run?

On build runs when you reference the image that has the onbuild instruction.

So you would push etsy/hound to docker.io
Someone would reference etsy/hound at the top of their Dockerfile
When this person builds their docker image based off of etsy/hound the ONBUILD instructions will now run.

The difference in the workflow would be you would not clone this repo.  You would create your own Dockerfile and reference etsy/hound along with dropping your config.json into the directory with your Dockerfile.

@kellegous  let me know if you have any other questions or want any more concrete examples.

@sstarcher Thanks.
The biggest issue is that the instructions in the REAME need to work with the Dockerfile. Not being much of a Docker user, I'm not sure what the norm is for distributing an image. I have been sitting on this PR until I figure out what other projects are doing here.

I updated the documentation on this branch to reflect one option.
To test this out you would have to replace etsy/hound in docker run with sstarcher/hound as I have the image on Docker IO.
On my branch the current quick start reflects the following.  This using docker's volume capability to mount the config.json file into the container.

Docker is required
Add the config file and run docker

The option I talk about in this pull request at the top is similar, but instead of mounting a docker volume.  We build a new container and use the etsy/hound aka sstarcher/hound docker container as the base.  When the container builds it uses the ONBUILD instruction to add the local config.json into the container image itself.
Both of these options have the added benefit that we don't have to have go installed locally.

Sorry for the delay, I'm just getting around to setting up the etsy/hound repository on DockerHub. The process is incredibly flaky but I think I finally have an automated build working. I'm about to merge this.


Feature - Data containment levels

Fixes the issue #1979.

I like this as an idea, but I would like to talk with the gnome software maintainers about this, would you be open to creating an issue on their bug tracker and reaching out?

Thank you for the feedback, @razzeee!
I have created an issue on Gnome GitLab.

Closed due to a lack of interest from the GNOME community.


⚠ Bofrost.at has degraded performance

In 429492b, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1457 ms

Resolved: Bofrost.at performance has improved in 0790a7a.


[New Product] Add RDS MySQL

If you are wondering why you need another mysql release page. This proposed page would be for the AWS RDS releases. Which follow about a month behind and have their own end of life dates.
There is no AWS API to get End of life dates for RDS databases.
I wrote a script to scrape the dates on this page.
I also have another PR for postgres here

Looks great 🎉
I was able to add a latest column.

🤔 A consideration
I do have the data scraped for minor versions. I used markdown table format to add it to the page. It does end up looking nice

However,
I don't want to be making a PR every other month to update this minor table. So, I think I will omit this section since it's not even added as an API.
If there is a way to work this data into an API then I would be interested in maintaining it. I could set up some kind of automation for creating PRs with you guys. Since I do have the scraping script generate the markdown file for me already.
Let me know what you guys think of what to do with the minor version data. I personally would use it for my work if it exists. There are probably other people that would enjoy the available data. But I understand a /rds-postgres-minor page might be confusing as well as more overhead.

@captn3m0, I would like to have your opinion on this PR. Appart from the various Kubernetes distros we don't have much products like that.

Side note on our integration efforts wrt identifiers: While we're relying on PURLs to identify packages in SBOMs, cloud services have no such similar identifier.
So if a tool (and there's many) scans a cloud environment (say AWS), it will generate a report listing down installed versions of various products, such as RDS/EKS/AKS/GKE/Looker etc.
While we publish EOL data of these, there's no easy way for such tools to use our dataset to link back.
I'd raised this with PURL, and it seems that PURLs will stick to "installable packages", which these cloud services are not. So we need a better way.
CycloneDX wants to standardize SaaSSBOMs so my hope is to use whatever identifiers they settle on. (See https://github.com/CycloneDX/bom-examples/tree/master/SaaSBOM for current progress).
If this doesn't work out, I'd like to figure out a cloud-service:// identifier, similar to PURLs with a simple conversion against AWS ARNs, and other similar IDs.

Enabled the releaseDate column, merged. Thanks a lot for your contribution @CodaBool 🎉.
Depending on feedback, I'd open to switching to listing multiple minor versions as well, if that's more helpful.


HttpWebRequest/HttpWebResponse Not Working!

HttpWebRequest request= with Headers ....etc
var httpWebResponse = (HttpWebResponse) await request.GetResponseAsync()
never passed this code >fail fast!! idk what to put in the rd.xml or if event i should put anything there!

what error do you get? you need provide more info; corert version, dotnet version, stacktrace etc

what error do you get? you need provide more info; os, corert version, dotnet version, stacktrace etc

Problem Caused by the Response is not being Decoded When clrcompression.dll is missing
Solved By adding clrcompression library Based on this https://github.com/dotnet/corert/issues/5496/

I'll leave it open For The Other Dudes!


Can't sync to GDrive

When I try to sync a document to Google Drive, it works at first but then doesn't sync any updates. I get this notification:

I don't get this particular error, though when I open up the Synchronize > Open from Google Drive menu, although all my google drive folders are listed, they are empty of all documents.  Can't seem to get any files to synchronize over for opening

I find the same error. Have been able to do the first sync, but thereafter it doesn't work. Also happening on Dropbox.

The same. Only the frist saving to GDrive works correctly.

@Dzenly Versioning is a part of the way Google Drive functions. (You can always access previous versions of a file, and whenever you upload a new document with the same name & type as a previous file, it will replace the existing as the latest version.)

Closing as too old.


Dependency Analytics not working as expected for file explorer

Trigger Dependency Analytics via file explorer on package.json inside submodule with file explorer run's for root package.json, not for submodule one.

Closing as fixed


Cover ErrorHandler component with tests

That's not an easy task since the handler uses some tricks to, for example, handle out of memory errors.

But at least straightforward parts of it could be covered.


Added verifySignature

Steam signs some data like apptickets with the same RSA key that's used for client-side encryption.

Can you describe the use case in more detail? Is this going to be used in conjunction with the other functions provided by this module?
Does it ever use any other algorithm?

The use case is verifying stuff that was signed by Valve's key. App tickets and I believe install scripts (the latter being less useful for bots) are signed.
No, it won't be used in conjunction with any of the other functions in this module.
I don't know if it ever uses any other algorithm. I only checked app tickets.

I don't know if it ever uses any other algorithm.

In that case, let's hard-code "RSA-SHA1". It won't be a problem to add the ability to use another algorithm later if needed.
Also, can you provide some code that could be used to test this functionality?

I don't think hardcoding a specific algorithm because we haven't observed another algorithm in my one single observation is a great idea. It already defaults to "RSA-SHA1" if you omit the algorithm, so the effect is the same as it not being there at all anyway.
Here's some code to test it:
var rawTicket = new Buffer("3800000004000000f10ff40801001001b801000006e2f7830df0adba0000000080fb0b5700ab275701005a2100000100cb01000000000000c7746b2098a01ff26624e0a56353927ae9f3d40ecf297c13ce2d54111c3933f2098e2e53c6c356bf4ef51115c999c1377bb347f7042b84e3e19b56383f8e2f26b8626a564744770bf0aec4f914a1ba3c015e9520f4d5317e938790de657292e0d8ad7e6458b947e2fe086566380e434bfd0147e1d57c965dbc247276b1f2a3fc", "hex");
var length = rawTicket.readUInt32LE(0);
var ticket = rawTicket.slice(0, length);
var signature = rawTicket.slice(length);

var SteamCrypto = require('./index.js');
console.log(SteamCrypto.verifySignature(ticket, signature));

It already defaults to "RSA-SHA1" if you omit the algorithm, so the effect is the same as it not being there at all anyway.

It still needlessly increases the API surface.

Here's some code to test it:

I meant code that actually receives an app ticket (or anything else with a Steam signature)

It still needlessly increases the API surface.

How are you so sure it's needless?
This may be what you're looking for: https://github.com/DoctorMcKay/node-steam-user/blob/master/components/appauth.js

How are you so sure it's needless?

Again, it won't be a problem to add it later if needed. There is nothing to be gained by adding it preemptively.

This may be what you're looking for: https://github.com/DoctorMcKay/node-steam-user/blob/master/components/appauth.js

Ok, I'll try to figure this out, but a complete example would be preferable.

Time will potentially be saved later by adding it now. There's nothing to be lost by adding it now.
I don't have a complete example using node-steam since I haven't used node-steam in ages, but in a nutshell it's:

Send ClientGetAppOwnershipTicket with the app_id of some game you own
The response will contain a ticket
The first 4 bytes of that ticket are the length of the ticket portion, including the actual length field (little-endian)
The last 128 bytes of that ticket (which is the portion starting at the offset defined by the length field) are the signature of the preceding bytes

There's nothing to be lost by adding it now.

It increases the API surface area. If that does not convince you, imagine that in the future we'll need to add another string argument to this function.
Thanks for the steps, I'll take a look. Are all apptickets signed?
I'm still not sure what the usefulness of verifying the signatures for bots is, though.

The chances of ever having to add another string argument to a function to verify a signature are very low. Future-proofing is better to me than potentially having to add an unlikely argument after the algorithm argument.
As far as I know, all apptickets are signed. They wouldn't be very useful without being signed.
I'm implementing code to both generate and authenticate apptickets, which is why I want to verify signatures. I'm not sure about how incredibly useful this is, but it may prove to be useful at some point.

The chances of ever having to add another string argument to a function to verify a signature are very low.

Perhaps, but the risk is more significant to me than the negligible potential time-saving.

The risk of adding an unlikely new argument and having it not in the position you want is more significant than much more likely to be required future-proofing?

It's not about the position, but the fact that it would be necessary to specify the algorithm to use the new argument.
Also there is no basis to state that Valve using multiple different signing algorithms is more likely to happen than some other change that would require a new argument.
On a less related note, at least in my experience, future-proofing usually refers to making sure the architecture allows adding new functionality without significant and/or breaking changes, rather than implementing every single thing that could potentially be needed.

RSA has been around since 1978. It's far more likely that Valve will be inconsistent (as Valve is very good at being inconsistent) and use different signing algorithms than RSA changing somehow. Being required to specify the algorithm in order to use some other argument is really not a big deal. Could just make it accept null and default to "RSA-SHA1" (indeed, that's how it would work currently anyway).
My sample size for Valve using RSA-SHA1 is one. I have only checked one specific instance of Valve signing stuff. It's not unlikely that someday I'll come across something else that Valve signed that I want to verify where they didn't use SHA1, and I'll have to wait on yet another PR to get scrutinized.

It's far more likely that Valve will be inconsistent (as Valve is very good at being inconsistent) and use different signing algorithms than RSA changing somehow.

RSA doesn't need to change. Valve could as well add some weird post-processing to the signature or god knows what else.

Being required to specify the algorithm in order to use some other argument is really not a big deal.

Maybe for you, but I find it ugly.

and I'll have to wait on yet another PR to get scrutinized.

I don't consider that a valid factor to affect API decisions.

The stock crypto module requires the algorithm to be specified, is that ugly?

If you mean crypto.createVerify, then it doesn't have an optional argument followed by another argument, so no.

I beg your pardon? Where does my code have an optional argument followed by another argument?

Not when they have no known use. But if you scroll up you'll see that my actual argument is keeping the API surface area as small as possible. The optional argument thing is just one special case of this.
Anyway, I don't want to continue this conversation. I've already explained my opinion. If you're so opposed to the way I do things, it might be better for you to just use your own fork.

@DoctorMcKay It is common sense to not write any code without a clear use case. Since there is no evidence and no testable use cases of other algorithms, there should be no option to choose them.

What's the clear use case for SteamRichPresence? How is it useful in a bot? What's the usefulness for setting a non-one universe in a SteamID? Are you expecting to connect to non-public universes? SteamKit even has a property for the connected universe. SteamKit sets the client's OS in the logon message. What's the clear use case for that? Is SteamKit a horrible library because it doesn't have an itty bitty API surface?
Any halfway decent library or application has "code without a clear use case". At a certain point you just implement stuff because it's there and has good potential to be useful in the near future.

Here's an example of Valve using SHA256.

 PublicKey = Reader.ReadBytes(publicKeySize);

I have no idea how ValveResourceFormat works, but this line makes me suspect that it doesn't necessarily use Steam's public key.

Even if it doesn't, it's proof that Valve doesn't exclusively use SHA1.

It doesn't prove that Valve doesn't exclusively use SHA1 with Steam's public key.

You're splitting hairs.

If you're so opposed to the way I do things, it might be better for you to just use your own fork.

Thanks for the help.


Need to be able to spin up an agent without it accepting other jobs

If you have an army of CI agents running builds targeting *, and you want to set up a CI agent on a special machine with --meta-data deploy=true, there's no way to have it only accept deploy jobs… it'll just start accidentally picking up the other CI jobs (and failing them) upsetting existing team members.
We need some mechanism to force an agent to only accept jobs with matching meta-data, so you can safely set up a new set of agents and have a play. Something like:
buildkite-agent start --only-jobs-with-meta-data ci=true

:hand: we're in exactly this position. Switching to a white-list of acceptable jobs/metadata would be huge.

Perhaps another way of doing this, is instead of defining * in the Buildkite UI, you define: deploy!=true, so it'll match any agent where deploy isn't there?

@keithpitt sure, but what if you forget? It's far safer to be able to whitelist acceptable jobs, rather than to accidentally run full builds which may not even be possible on this agent.
What you've got is a system where agents can :hand: and make the representation that they are capable of building jobs requiring specific properties and all other default jobs. I'd like to exclude the defaults.

The same could be said for the --only-jobs-with-meta-data approach. You could forget to add that.
 If you have a fleet of these things, then it becomes even more hard. If you've got some sort of provisioning stuff in the mix, then you'd have to do all that jazz as well.
Or you go: "doh!", edit the step to be deploy!=true, which would match every other agent but deploy=true then carry on!
I'm not saying that -only-jobs-with-meta-data  is a bad approach, just throwing another one out there from fresh eyes.

I could forget, true, but when setting up a build agent I'm far more aware of what its capabilities are. I'm in a better position to rationally evaluate what types of jobs can run on that agent. On the other hand, when setting up a build job, you need to know every possible piece of metadata that you have to exclude, you're bound to make a mistake.
This is really a situation for larger groups of teams, or for much longer-running projects that build up assumptions over time. So I can see how it doesn't make much sense when seen through the eyes of a new or small team, but it makes a ton more sense if you think about it from the perspective of a team where most people do not know the majority of the ins-and-outs of what other teams are up to.

With Dave he would need to edit every step of every existing project just to boot up a new deploy agent for a new job :(

Oh, we shipped this! Let me know if you want more details.


Reset state upon load if showPreviousViewOnLoad

Fixes #5648

I'm sorry, but given the asynchronous nature of the code involved I don't understand how we can guarantee that this will work (even if it currently does).
For it to work reliably, the code in the patch needs to be executed before viewer.js#L869. Note how both firstPagePromise and the preference fetching are async.
If these async operations happen in the wrong order, not only will the code in this PR not work, but it would also cause breakage in the history.
I'm instead suggesting that we move Preferences.get('showPreviousViewOnLoad') up to the rest of the prefs near the top of the file, and then place the code in this patch immediately before viewer.js#L869.
This would also, for free, prevent any possible issues with the browser not supporting replaceState.
Edit: Yes, I'm aware that this means that the pref won't be read if a new file is opened in the viewer. However, I regard that as an edge case (given the main use-case for the default viewer), and I'm more worried about this patch causing unnecessary and easily preventable regressions in the future.

If you worry about race conditions, what about:
        showPreviousViewOnLoadPromise.then(function() {
            PDFHistory.initialize(self.documentFingerprint, self);
        });

That would introduce a completely unnecessary dependency between the history and the preferences.
If we fail to initialize the preferences, this means that the history wouldn't work either, which would be bad.
All in all, since this is really an edge case I think that we should complicate this as little as possible, hence why I'm suggesting to move the preference fetching code to one central spot.

@Snuffleupagus Done.
I guess that by the same reasoning, we should move defaultZoomValue up.
I've noticed that we're using var self = this; in various places, where this is just the singletion PDFViewerApplication. To make it easier to search for the declarations and uses of certain properties, shouldn't we get rid of self.foo and use PDFViewerApplication.foo directly?

Seems fine, but please also add preferenceShowPreviousViewOnLoad: true here: https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L112.

I guess that by the same reasoning, we should move defaultZoomValue up.

Yes, I think that would be best.
The reason that this code is spread out, is that the various pieces was added at different points in time, and no-one got around to moving it yet (hence the TODO).
(The recent effort, as far as refactoring the viewer code, has been (and still is) on preparation work to be able to release embeddable viewer APIs, see e.g. the viewer components.)

I've noticed that we're using var self = this; in various places, where this is just the singletion PDFViewerApplication. To make it easier to search for the declarations and uses of certain properties, shouldn't we get rid of self.foo and use PDFViewerApplication.foo directly?

It's fairly common in the PDF.js codebase to use self as an alias for this of the surrounding scope, to avoid the need to create unnecessary closures with bind(this).
Considering the length of e.g. PDFViewerApplication vs self, spelling it out actually feels slightly unwieldy to me. (Anyway, I think this discussion is out of scope of the current PR.)

@Snuffleupagus Done. Moving the other parameter can be done in a separate commit.

Thanks for the patch!


Segmentation fault (core dumped) when I accept the LICENSE and install the meshes and URDFs

###########################################################
| You must accept the license to install the robot meshes. |
###########################################################
Accept (y/n)? y
Segmentation fault (core dumped)

@xiejialong Any updates on the above comment? Did you manage to solve your problem?

@xiejialong I will close the issue for now, don't hesitate to reopen it if you still encounter the problem


Remove expect and unwrap from library code

(closes #1)
Signed-off-by: Nikos Filippakis<EMAIL_ADDRESS>

@nikofil thanks for this, looks like you need to run make checks you're failing a format check.

Yep, didn't notice, sorry. :) Fixed now.

Awesome work thanks again!


!notThisTag

I have a small suggestion. Would it be possible to implement an exclusion tag when sorting molecules? Namely, by adding an exclamation mark before the tag, is it possible to exclude the molecules containing this tag (!notThisTag)? This would make sifting through the data much faster :D

Yes I totally agreed we should have this! I just added this but it might take a little while before it makes it to the update site. Other changes need a bit more testing.
I have not yet added this to tag search of metadata records. I would like that tag search to have all the same features eventually.

Perfect! I guess that means that it doesn't yet work for the virtual store, but it works for the archive?On Jun 23, 2023 16:44, karlduderstadt @.***> wrote:
Yes I totally agreed we should have this! I just added this but it might take a little while before it makes it to the update site. Other changes need a bit more testing.
I have not yet added this to tag search of metadata records. I would like that tag search to have all the same features eventually.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

No it should also work for virtual archives. The UI code is the same for both. All the differences live in mars-core. I will work on getting it released to the update site so you can test it. The other changes are the inclusion of cloud support for N5 datasets. This will include a new jar mars-minio. I will add everything to the update site over the weekend.


Autocomplete filter

Typeahead filter inspired by Console's AutocompleteInput.
Key features:

match the search pattern in the entire string (not only prefix)
reset the filter after selection - the selected item is visible in
the chip list

Contract:

enum values passed as supportedValues param have id prop the same as
the label prop
values in supportedValues are unique (no de-duplication similar
to EnumFilter)

Reference-Url: http://v4-archive.patternfly.org/v4/components/select#typeahead
Reference-Url: https://github.com/openshift/console/blob/cf961054415464ba90959b614517903fc9b808a3/frontend/public/components/autocomplete.tsx#L22
Screencast from 2023-11-27 13-17-46.webm

Fixes: #790


Extracting decision path

Hi all,
I was wondering if it is possible to extract the decision path for a specific sample.
If this is not possible. Does anyone know how I can extract the splits used for a specific sample?
Thanks in advance!

Closed in favor of being in #2302. We decided to keep all feature requests in one place.
Welcome to contribute this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.


[📖] Docs/tutorials expect extra renders - signals are too efficient!

Suggestion
If a component inserts store values directly into the JSX for example, Qwik can propagate signal updates directly into the DOM without re-running the component's render function. This seems to have made a number of tutorials/docs out of date.
Discussed a bit on discord: https://discord.com/channels/842438759945601056/1060999298671718540/1060999298671718540
These are the sorts of inaccuracies noticed (please feel free to add more)

[ ] https://qwik.builder.io/docs/concepts/reactivity/#counter-example "downloads the rendering function and re-runs the OnRender method"
[ ] https://qwik.builder.io/tutorial/reactivity/template/ "a++ button will only cause the re-rendering of "
[ ] https://qwik.builder.io/tutorial/projection/basic/ "Notice that the <App> is re-render on button click"
[ ] https://qwik.builder.io/tutorial/props/store/ "The re-rendering of <App> is needed to update the props of "

Additionally, is there anything we can to to assist the mental model when describing this behavior - is there a word for the "rerender without rerender"?
If there is appetite here to make some changes I can read through all the docs/tutorials and make sure we have the exhaustive list.

I think it'd be a good idea

@cmbartschat Thanks for your report. Would you like fix the docs for this particular topic?

Happy to! I just need a little bit of direction from a terminology standpoint. Is there a word the core team uses to describe the "rerender without rerender"?
How would these two processes be labelled?

A component's render function gets run again in response to a signal change
The DOM structure receives targeted updates in response to a signal change, without invoking component code

Thanks, this would make a significant community issue. Is anyone up for a PR?

@mhevery I'm working on it ✏

Thanks @wtlin1228 💪

Discussion for the last task: https://discord.com/channels/842438759945601056/1147859577170317376

Closing this issue as PRs have landed...


LA County

Update LA county data

# us-ca-los_angeles-1552024872208.geojson.gz
ok - 2748247/2764973 (99.3%) valid addresses
ok - 0 outside of boundary (dropped)
ok - 346 have an unsupported number format
ok - 1363 are missing street names
ok - 0 could not be parsed
ok - 15017 suffered misc failures


Doubled gas mask action message

Description

https://github.com/space-wizards/space-station-14/assets/58439124/6ae0dbb1-2e53-4e2b-b421-7c27413fa6f3
Reproduction

Screenshots

Additional context

this has been broken for so long that i've gotten used to it


How can I customize the model arch crnn.cfg

There seem to be possible to modify the number in crnn.cfg
But how do I add more layers in crnn.cfg
I am newbee for this type of implementation
REPEATx2 is predefined keyword in Pytoch or hardcoded variable name in your code?
If I would like to make 10 CNN before LSTM then, how can I modify?
Aldo if I would like to use different input size, where should I change?
[convs_module]
    [conv2d]
        out_channels=16
        kernel_size=3
        stride=1
        padding=valid
    [batchnorm2d]
    [elu]
    [maxpool2d]
        kernel_size=3
        stride=3
    [dropout]
        p=0.1

    REPEATx2
        [conv2d]
            out_channels=32
            kernel_size=4
            stride=1
            padding=valid
        [batchnorm2d]
        [elu]
        [maxpool2d]
            kernel_size=4
            stride=4
        [dropout]
            p=0.1
    END

[moddims]
    permute=[2,1,0]
    collapse=[1,2]

[recur_module]
    [lstm]
        hidden_size = 64
        num_layers = 3
        bidirectional=True

[moddims]
    permute=[1]

@takeofuture so some of the details of how the cfg file works will be in https://github.com/ksanjeevan/torchparse. Basically, you can do something like:

  [conv2d]
      out_channels=16
      kernel_size=3
      stride=1
      padding=valid
  [batchnorm2d]
  [elu]
  [maxpool2d]
      kernel_size=3
      stride=3
  [dropout]
      p=0.1
[conv2d]
      out_channels=16
      kernel_size=3
      stride=1
      padding=valid
  [batchnorm2d]
  [elu]
  [maxpool2d]
      kernel_size=3
      stride=3
  [dropout]
      p=0.1
...

And keep adding convolutions. At no point do you have to pre-define a input size as depending on the model you've defined, multiple input dimensions can be valid.
If using torchparse is confusing, or you'd rather define your model some other way, you can just write your own in net/, or modify https://github.com/ksanjeevan/crnn-audio-classification/blob/f09449b87ed61d28b82931c7055d033a7b69160b/net/model.py#L29 with whatever post processing you want to do after the stft. Hope this helps!


Return 404 when public gpg key is not exist

I think it's better than return an empty string.

Codecov Report

Merging #12916 into master will decrease coverage by 0.03%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master   #12916      +/-   ##
==========================================
- Coverage   43.07%   43.04%   -0.04%
==========================================
  Files         658      658
  Lines       72451    72453       +2
==========================================
- Hits        31211    31185      -26
- Misses      36183    36217      +34
+ Partials     5057     5051       -6

Impacted Files
Coverage Δ

routers/api/v1/misc/signing.go
0.00% <0.00%> (ø)

modules/util/timer.go
42.85% <0.00%> (-42.86%)
:arrow_down:

services/pull/check.go
31.53% <0.00%> (-16.16%)
:arrow_down:

modules/indexer/stats/queue.go
64.70% <0.00%> (-11.77%)
:arrow_down:

modules/indexer/stats/db.go
43.47% <0.00%> (-8.70%)
:arrow_down:

models/unit.go
46.57% <0.00%> (-2.74%)
:arrow_down:

models/pull.go
54.54% <0.00%> (-0.63%)
:arrow_down:

services/pull/pull.go
40.78% <0.00%> (ø)

modules/log/event.go
59.43% <0.00%> (+1.88%)
:arrow_up:

modules/process/manager.go
75.00% <0.00%> (+2.50%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2dbca92...755ff73. Read the comment docs.

It would be better to return something like:
-----BEGIN PGP PUBLIC KEY BLOCK-----
Note: This user hasn't uploaded any GPG keys.

=twTO
-----END PGP PUBLIC KEY BLOCK-----

@zeripath why? Imho that would be confusing

@a1012112796 what use case for this PR?

When writing any API you should ask yourself who is expected to be calling this API.
I'm going to go out on a limb and say the user of *.gpg urls is for gpg to use them when importing keys.
Therefore the output of this should be something gpg will be able to use.
A 404 page which is not in armored key output won't work.
A blank response does work as does my suggested response.

When writing any API you should ask yourself who is expected to be calling this API.
I'm going to go out on a limb and say the user of *.gpg urls is for gpg to use them when importing keys.
Therefore the output of this should be something gpg will be able to use.
A 404 page which is not in armored key output won't work.
A blank response does work as does my suggested response.

I see, Now it will return empty with 404 status, How about this way?

Well take a look at what GH returns for say: https://github.com/go-gitea.gpg

Then it's not so necessary also.

@zeripath But this was for API not Gitea web route


arduino-cli: 0.25.1 -> 0.27.1

Description of changes
https://github.com/arduino/arduino-cli/releases
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 191642 run on x86_64-linux 1

2 packages built:

arduino-ci
arduino-cli

Result of nixpkgs-review pr 191642 run on x86_64-linux 1

2 packages built:

arduino-ci
arduino-cli

Thank you!


Show bot name below message if available. Fallback to id.

-Show from.name below incoming chat messages if it was provided. Fallback to from.id otherwise.
-Also deleted some dead css classes (were leftover from the debugview).

ping. :)

WHAT I AM WIDE AWAKE


[BUG] Closing camera in batch mode leads to data lost

Version
1.8.4 Build 75
What platform are you using?
Android
OS Version
/e/OS 2.0 (based on Android 12)
What happened?

I open the camera in the app with activated batch mode.
I take photos of several pages of longer document.
As I was finished I accidentally tap on the cross in the upper left corner instead of the check mark in the lower right corner.
All photos are lost.

If you already have taken photos there should be some confirmation dialog when closing with the cross that warns before the photos are lost.
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@BaumiCoder sorry that you lost everything... Will add the confirmation

@BaumiCoder i added the confirmation in latest. Hope you wont get into issues anymore!

@farfromrefug Thank you for the fast bugfix.


X-Forwarded-Method is missing from request

Do you want to request a feature or report a bug?
Bug
What did you do?
The header 'X-Forwarded-Method' is missing from request received by backend server. It's missing also when
using ForwardAuth.
I'm trying to get Traefik with ForwardAuth to work with my authentication server. The server needs all information from the original request including all headers, URI, method, body. While working on this, I noticed that the method is not received by the authentication server.
I reduced my setup to a minimal setup and still the header is missing. Tested this behavior with Chrome, Firefox, HTTPie.
What did you expect to see?
Expected to see 'X-Forwarded-Method' header in the request. Also should 'X-Forwarded-URI' be present?
What did you see instead?
Received request by the backend server
# http https://api.localhost.local/api/foobar

HTTP/1.1 200 OK
Content-Length: 298
Content-Type: text/plain
Date: Tue, 30 Jun 2020 05:23:57 GMT
Server: openresty/<IP_ADDRESS>

GET /api HTTP/1.1
Host: api.localhost.local
User-Agent: HTTPie/0.9.8
Accept: */*
Accept-Encoding: gzip, deflate
X-Forwarded-For: <IP_ADDRESS>
X-Forwarded-Host: api.localhost.local
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: afdadc467921
X-Real-Ip: <IP_ADDRESS>

Output of traefik version: (What version of Traefik are you using?)
Version:      2.2.1
Codename:     chevrotin
Go version:   go1.14.2
Built:        2020-04-29T18:02:09Z
OS/Arch:      linux/amd64

Problem exists also when using Docker images v2.0 and v2.1.
What is your environment & configuration (arguments, toml, provider, platform, ...)?
TLS certificates are created using mkcert.
traefik/traefik.yml
global:
  checkNewVersion: false
  sendAnonymousUsage: false
accessLog: {}
log:
  level: DEBUG

entryPoints:
  web:
    address: ':80'

  web-secure:
    address: ':443'

api:
  dashboard: true

providers:
  file:
    directory: /etc/traefik/conf
    watch: true

  docker:
    endpoint: unix:///var/run/docker.sock
    exposedByDefault: false

traefik/conf/services.yml
http:
  routers:

    portal-backend-https:
      rule: "
        Host(`api.localhost.local`) &&
        (PathPrefix(`/api`))
        "
      entrypoints:
        - web-secure
      service: portal-backend@docker
      tls:
        domains:
          - main: "api.localhost.local"

traefik/conf/tls.yml
tls:
  certificates:
    - certFile: /certs/localhost+5.pem
      keyFile: /certs/localhost+5-key.pem
      stores:
        - default
  stores:
    default: {}

  options:
    default:
      minVersion: VersionTLS12
      sniStrict: true
      preferServerCipherSuites: true

traefik/conf/dashboard.yml
http:
  routers:
    api-dashboard:
      rule: "Host(`traefik.localhost`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      middlewares:
        - auth
      service: api@internal

  middlewares:
    auth:
      basicAuth:
        users:
          # => user:user
          - "user:$2y$10$ItlennzhImvIU9mOrh/Gq.p7upNwzLUUIB3m899lpFKBB9MfXSlq2"

docker-compose.yml
version: '3'
services:

  reverse-proxy:
    image: traefik:v2.2
    ports:
      # The HTTP port
      - "80:80"
      # The HTTPS port
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - "./traefik:/etc/traefik"
      - "./certs:/certs"

  portal-backend:
    image: brndnmtthws/nginx-echo-headers
    expose:
      - 8080
    networks:
      default:
    labels:
      - traefik.enable=true
      - traefik.http.services.portal-backend.loadbalancer.server.port=8080

  autsi:
    image: brndnmtthws/nginx-echo-headers
    expose:
      - 8080
    networks:
      default:
    labels:
      - traefik.enable=true
      - traefik.http.services.autsi.loadbalancer.server.port=8080

networks:
  default:
    external:
      name: traefik_bugs

If applicable, please paste the log output in DEBUG level (--log.level=DEBUG switch)
time="2020-06-30T05:44:51Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/api/foobar\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip, deflate\"],\"Connection\":[\"keep-alive\"],\"User-Agent\":[\"HTTPie/0.9.8\"],\"X-Forwarded-Host\":[\"api.localhost.local\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"afdadc467921\"],\"X-Real-Ip\":[\"<IP_ADDRESS>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.localhost.local\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<IP_ADDRESS>:39126\",\"RequestURI\":\"/api/foobar\",\"TLS\":null}"
time="2020-06-30T05:44:51Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/api/foobar\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip, deflate\"],\"Connection\":[\"keep-alive\"],\"User-Agent\":[\"HTTPie/0.9.8\"],\"X-Forwarded-Host\":[\"api.localhost.local\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"afdadc467921\"],\"X-Real-Ip\":[\"<IP_ADDRESS>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.localhost.local\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<IP_ADDRESS>:39126\",\"RequestURI\":\"/api/foobar\",\"TLS\":null}" ForwardURL="http://<IP_ADDRESS>:8080"
time="2020-06-30T05:44:51Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/api/foobar\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip, deflate\"],\"Connection\":[\"keep-alive\"],\"User-Agent\":[\"HTTPie/0.9.8\"],\"X-Forwarded-Host\":[\"api.localhost.local\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"afdadc467921\"],\"X-Real-Ip\":[\"<IP_ADDRESS>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.localhost.local\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<IP_ADDRESS>:39126\",\"RequestURI\":\"/api/foobar\",\"TLS\":null}"

Hi @Shemeikka ,
Thanks for your interest in Traefik :)
I can confirm that Traefik is not sending X-Forwarded-Method header when sending the request to the backend, or an authentication server (when using the ForwardAuth middleware).
Thus, if i understand well what you were expecting, from the Traefik point of view, the method is never "changed" along the request lifecycle.
Accordingly, sending the X-Forwarded-Method header will not make sens because it will be the same method as the method of the request received by the backend.
WDYT ?

Hi @rtribotte
I now noticed that I had pasted wrong config to my issue description above. I have now updated it. It was completely missing the forward authentication part.
Yes, in normal situations there's no reason to send the X-Forwarded-Method header to backend server. However, there are use cases for sending this X-Forwarded-Method header to the authentication server when using ForwardAuth middleware and currently this header isn't being send as you said.
If this isn't a bug, then I would like to propose that this X-Forwarded-Method header and X-Forwarded-URI header are both sent to the authentication server when ForwardAuth middleware is used. It's not necessary to sent those to the request's backend server, just to the authentication server.

Hello @Shemeikka ,
I apologize, i think my previous answer was not clear enough.
Traefik is not transmitting forwarded headers unless the insecure mode is set to true or trusted IPs is not empty and the origin match one those IPs.
Regarding the internal behavior of Traefik, the X-Forwarded-Method header is not added because the method is never modified, but as stated previously Traefik will pass this header if it was already present on the request and allowed to pass it.
Also regarding the Forward Auth middleware it's currently passing the X-Forwarded-Method headers in both cases:

if the trustForwardedHeaders is set and the request has X-Forwarded-Method headers set, it will copy the X-Forwarded-Method headers from the original request to the request to the authentication backend.
OR if the request method is not empty it set X-Forwarded-Method to the request method value

But otherwise the X-Forwarded-Method headers is not transmitted to the authentication backend.
I hope it's more clear.
Thus, in your case, could you check that your are matching those conditions ? (I cannot reproduce a potential bug)

Hi! I'm Træfiker :robot: the bot in charge of tidying up the issues.
I have to close this one because of its lack of activity :disappointed:
Feel free to re-open it or join our Community Forum.


🐛 Mobile | Temporary authentication issues

Cc: @adamcogan, @matt-goldman, @tkapa, @zacharykeeping

Anton
Bug - Any idea on either platform?
⚠ FYI, both Iphone & Ipad bugs – all ok now.

Figure: iPhone.
Error text: "Login Failure There seems to have been a problem logging you in. Please try again. Arg_KeyNotFoundWithKey, code"

Figure: iPad.
Error text: "There seems to have been a problem logging you in. Please
try again. Error Domain=com.apple.Authenti-
cationServices.WebAuthenti-
cationSession Code=2
"Cannot start ASWebAuthenticationSession without providing presentation
context. Set presentationContextProvider
before calling -start."
UserInfo={NSDebugDescrip-
tion=Cannot start ASWebAuthenticationSession without providing presentation
context. Set presentationContextProvider
before calling - start.}"
Expected Behavior
The user can login.
Tasks

[x] Investigate
[x] Fix

More Information
The issue happened on Sunday 25th of Feb around 4pm.
iPad issue log: https://appcenter.ms/orgs/SSW.MobileApps/apps/RewardsMAUI-2/crashes/errors/1761151820/overview
Environment

Device: iPhone, iPad, Android phone.
OS: iOS, Android

Thanks!

Matt's fix had to resolve this issue.
If anyone comes across it on version 3.0.9 and above, feel free to re-opem.

The error on the first screenshot is still an issue in v3.0.9. Just had it on my Android device. (cc: @tkapa)

Figure: Auth error

Blocked - @AntPolkanov can this be closed with #781

The issue should be fixed now.
Once the users open the app, they have to re-login to get a new token. After that they shouldn't be prompted to login for at least 6 months.


Support DNS over TCP

In pillar we use customized dialers for DNS resolvers to enforce the
source IP address to use, thus effectively choosing the output network
interface for a given DNS request.
However, currently we expect that DNS servers always respond on the UDP
port, but they may in fact fallback to use TCP (same port number).
This happens for example when the response message size is too large.
In that case we get "mismatched local address type" error.
I observed this with Azure datastore and a private DNS server, so it
make sense to support DNS over TCP at least for Downloader, but for
completeness I also added this for the controller name resolution.
Marking as draft until I find a way of reproducing and testing this locally.
Right now I can observe the "mismatched..." error only in customer logs.
Signed-off-by: Milan Lenco<EMAIL_ADDRESS>

It seems we should pull changes of zedUpload into pillar with make proto-vendor.

It seems we should pull changes of zedUpload into pillar with make proto-vendor.

Correct, I forgot that it is a separate Go package.

@milan-zededa I was assuming there would be some test DNS server which returns a ton of A or AAAA records, but I've yet to find one. What I did find so far is a test setup which returns lots of NS records, but I suspect that will not have any impact on the A/AAAA lookup done here.
FWIW the NS lookup fallback to TCP can be seen with
dig -t ns toomanyns-eth.test.globnix.net @nlns.globnix.net

@eriknordmark It turned out that the underlying customer issue was different (https://github.com/lf-edge/eve/pull/2800). Device was missing one of two DNS servers. The other one is not able to resolve controller domain (only private destinations) and God knows what truncated response it returns, which causes Golang resolver to try TCP.
Since we haven't yet actually seen any realistic scenario in which A/AAAA lookup needs TCP to succeed, I would suggest to not introduce this change. There is already a "XXX" comment, so we may just leave it at that until we find a case where support for DNS over TCP is really needed.

I'm reopening this PR - the problem with the DNS server truncating UDP response and expecting the client to switch to TCP has been observed again in the customer deployment. We tried to build the latest EVE with this PR included and it indeed fixed their problem, so we will need to merge it and prepare an LTS point release with the patch.

Do we have some test setup where e.g., there is a ton of A or AAAA records configured for some FQDN which we can use to regression test this?

I will prepare something using Eden-SDN, it should be possible.


Please explain Step 3 iOS -Swift.h file not found

The project is always failing. Not able to configure successfully . -Swift.h file not found. @HwangTaehyun

Hi @vinayn-mend Could you follow this?
https://github.com/HwangTaehyun/react-native-lottie-splash-screen/issues/4#:~:text=I think I,Thank you!


Avoid O(N) string allocations in unescapeContent

I have a file with ≤1000 lines, and the longest line is ≤400 characters long. Unfortunately, I have seen my poor AWS Lambda box time out (10 sec+) while processing this file. Profiling showed unescapeContent as the costliest function. None of my strings have escape characters in them, so this function should be able to return very quickly. However, whatever version of Node I'm using is not optimizing out the string concatenation in a tight loop, and this is causing a significant amount of pointless allocations and GC. Using a global RegExp allows this function to run very quickly, with a minimum of string concatenations.

This is great @InPermutation - thanks for your pull request. I will do a release and the new version including your optimization will be available shortly.

Thank you for the incredibly rapid release!
I found some more performance bottlenecks and filed issues #17 #18 #19 to track them. I will try and address them as I find the time, but I think I'm going to stop using PropertiesEditor for now instead. On my local machine, time for that particular file went from
1.14s user 0.05s system 95% cpu 1.243 total to
0.04s user 0.01s system 26% cpu 0.174 total, a 96% speedup!

This is really great. I never did profiling or ran into performance issues so fixing those issues will definitely be valuable.

The PropertiesEditor-less solution succeeded on my puny Lambda in ≤102ms, but now I've found a legitimate application for PropertiesEditor so I'll be back 😀


undefined symbol: _Py_Dealloc

when i ran:
python tools/test_net.py
got this error:
ImportError: /home/work/hcx/Detectron.pytorch/lib/model/roi_pooling/_ext/roi_pooling/_roi_pooling.so: undefined symbol: _Py_Dealloc
python 3.5
pytorch 0.4

Have you compile the code ?

i have compiled @roytseng-tw

Did you use roi_pooling? If not, you can try comment out corresponding lines for roi_pooling in lib/modeling/model_builder.py

I pulled your latest code and it ran ok,thank you @roytseng-tw

But actually I didn't change anything related to this. Anyway, glad to hear that.

hi @hcx1231
have u solved this problem?


Cosmic S2 - second batch of changes.

:sunny: The build tests passed at 654bef5603b90fd0c86480018e84ea30efc1145a.

Test
Result
Details

test with
:white_check_mark:
Command did not list any other PRs to include

merge
:white_check_mark:
Merged 654bef5603b90fd0c86480018e84ea30efc1145a at dea81d6fcbc9556c56bb637da94feab725c52afb

build (prof)
:white_check_mark:
Log file. Build time: 11 min 37 sec

ceSimReco
:white_check_mark:
Log file.

g4test_03MT
:white_check_mark:
Log file.

transportOnly
:white_check_mark:
Log file.

POT
:white_check_mark:
Log file.

g4study
:white_check_mark:
Log file.

cosmicSimReco
:white_check_mark:
Log file.

cosmicOffSpill
:white_check_mark:
Log file.

ceSteps
:white_check_mark:
Log file.

ceDigi
:white_check_mark:
Log file.

muDauSteps
:white_check_mark:
Log file.

ceMix
:white_check_mark:
Log file.

rootOverlaps
:white_check_mark:
Log file.

g4surfaceCheck
:white_check_mark:
Log file.

FIXME, TODO
:white_check_mark:
TODO (0) FIXME (0) in 0 files

clang-tidy
:white_check_mark:
0 errors 0 warnings

N.B. These results were obtained from a build of this Pull Request at 654bef5603b90fd0c86480018e84ea30efc1145a after being merged into the base branch at dea81d6fcbc9556c56bb637da94feab725c52afb.
For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

There are also 'low' in the POMS scripts.
I think it would be easier to understand if a single capitalization were
used everywhere.
On Mon, Apr 29, 2024 at 1:22 PM oksuzian @.***> wrote:

@.**** commented on this pull request.
In Scripts/gen_CosmicStage2.sh
https://github.com/Mu2e/Production/pull/313#discussion_r1583701394:

@@ -82,10 +87,16 @@ if [[ ${NJOBS} == 0  || ${NEVTS} == 0 ]]; then
exit_abnormal
fi

+if [[ ${LOW} != "Low" && ${LOW} != "All" ]]; then
The original code had "--low Low" option.
I've added "--low All".
Do you want to change it to "--Low Low"?
—
Reply to this email directly, view it on GitHub
https://github.com/Mu2e/Production/pull/313#discussion_r1583701394, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABAH573CHUK75NB6TUJCZ33Y72TZRAVCNFSM6AAAAABG6XWHH2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDAMRZGU2DEOBWGY
.
You are receiving this because you are on a team that was mentioned.Message
ID: @.***>

--
David Brown            @.***
Office Phone<PHONE_NUMBER>
Lawrence Berkeley National Lab
M/S 50R5008 (50-6026C)  Berkeley, CA 94720

On Mon, Apr 29, 2024 at 1:26 PM oksuzian @.***> wrote:

@.**** commented on this pull request.
In Scripts/gen_CosmicStage2.sh
https://github.com/Mu2e/Production/pull/313#discussion_r1583706481:

@@ -31,6 +32,7 @@ usage() {
[ --low Resample 'Low' S1 output ]

I don't think All is the default.
--low is the required option.

Oh, I see:  --low "All" is a valid command.  That's unintuitive to me.
Either the parameter name should change (--S1Type or something) or --low
should an optional (Boolean) with 'All' as default.

—
Reply to this email directly, view it on GitHub
https://github.com/Mu2e/Production/pull/313#discussion_r1583706481, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABAH574ACYVIQNK3E35AZLTY72UH5AVCNFSM6AAAAABG6XWHH2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDAMRZGU2TQNJVGM
.
You are receiving this because you are on a team that was mentioned.Message
ID: @.***>

--
David Brown            @.***
Office Phone<PHONE_NUMBER>
Lawrence Berkeley National Lab
M/S 50R5008 (50-6026C)  Berkeley, CA 94720

On Mon, Apr 29, 2024 at 1:26 PM oksuzian @.> wrote: @.* commented on this pull request. ------------------------------ In Scripts/gen_CosmicStage2.sh <#313 (comment)>: > @@ -31,6 +32,7 @@ usage() { [ --low Resample 'Low' S1 output ] > I don't think All is the default. > --low is the required option.
Oh, I see: --low "All" is a valid command. That's unintuitive to me. Either the parameter name should change (--S1Type or something) or --low should an optional (Boolean) with 'All' as default.
…
— Reply to this email directly, view it on GitHub <#313 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAH574ACYVIQNK3E35AZLTY72UH5AVCNFSM6AAAAABG6XWHH2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDAMRZGU2TQNJVGM .  Office Phone<PHONE_NUMBER> Lawrence Berkeley National Lab M/S 50R5008 (50-6026C) Berkeley, CA 94720

--low is the required option. The argument is also used in POMS configs.
I can't have the default, undefined option.
I'll do --S1Type
Should I change the following to --S1Gen?
[ --S1 Cosmic S1 name (CRY, CORSIKA, ...) ]

On Mon, Apr 29, 2024 at 1:43 PM oksuzian @.***> wrote:

On Mon, Apr 29, 2024 at 1:26 PM oksuzian @.*> wrote: @.** commented on
this pull request. ------------------------------ In
Scripts/gen_CosmicStage2.sh <#313 (comment)
https://github.com/Mu2e/Production/pull/313#discussion_r1583706481>: >
@@ -31,6 +32,7 @@ usage() { [ --low Resample 'Low' S1 output ] > I don't
think All is the default. > --low is the required option.
Oh, I see: --low "All" is a valid command. That's unintuitive to me.
Either the parameter name should change (--S1Type or something) or --low
should an optional (Boolean) with 'All' as default.
… <#m_-8912663168129233673_>
— Reply to this email directly, view it on GitHub <#313 (comment)
https://github.com/Mu2e/Production/pull/313#discussion_r1583706481>, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABAH574ACYVIQNK3E35AZLTY72UH5AVCNFSM6AAAAABG6XWHH2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDAMRZGU2TQNJVGM
. You are receiving this because you are on a team that was
mentioned.Message ID: @.
> -- David Brown @. Office Phone<PHONE_NUMBER> Lawrence Berkeley
National Lab M/S 50R5008 (50-6026C) Berkeley, CA 94720
--low is the required option. The argument is also used in POMS configs.
I can't have the default, undefined option.
I'll do --S1Type
Should I change the following to --S1Gen?
[ --S1 Cosmic S1 name (CRY, CORSIKA, ...) ]

Sure if you prefer.

—
Reply to this email directly, view it on GitHub
https://github.com/Mu2e/Production/pull/313#issuecomment-2083628424, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABAH574ANJZCZFEQWFDX2P3Y72WHRAVCNFSM6AAAAABG6XWHH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBTGYZDQNBSGQ
.
You are receiving this because you are on a team that was mentioned.Message
ID: @.***>

--
David Brown            @.***
Office Phone<PHONE_NUMBER>
Lawrence Berkeley National Lab
M/S 50R5008 (50-6026C)  Berkeley, CA 94720


jsParensError with stage-3 "Class field declarations"

Sorry if this is out of scope or maybe to soon to implement, for reference look: tc39/proposal-class-fields: Orthogonally-informed combination of public and private fields proposals, with a minimal init.vim:
call plug#begin('~/.config/nvim/plugged')
Plug 'yuezk/vim-js'
call plug#end()

And the following JavaScript file:
class Foo {
  foo = async x => {
    try {
      console.log(x)
      return x || {}
    } catch (e) {
      console.error(e)
    }
  }
}

There are some jsParensError syntax errors:

Just letting you know in case you want to check it, maybe we can wait until this gets in a official ES release, although currently is a wide supported feature:

Babel 7.0+
Node 12
Public fields are enabled by default in Chrome 72 / V8 7.2
Private fields are enabled by default in Chrome 74 / V8 7.4
Public instance fields are enabled by default in Firefox 69
Public static fields are enabled in Firefox Nightly 75 as of Febraury 22, 2020
Public instance fields are shipped behind a flag in Safari Technology Preview 101
Moddable XS
QuickJS
TypeScript 3.8

Thanks!

@davidsierradz It seems to be a flaw caused by the anonymous async function. I have fixed it in the latest code and didn't break my test files. Let me know if you have any questions.


adjust look for viz_leaf_samples_by_class

hi @parrt,
I adjusted the look for the viz_leaf_samples_by_class().
Is there a reason why you chose the previous method name to ctreeviz_leaf_samples()? Should I change the name of viz_leaf_samples_by_class to ctreeviz_leaf_samples_by_class ?

This is how it looks now.

Dang. I messed up. You're right. I'll change back after merge. I meant to do ctreeviz_leaf_samples for viz_leaf_samples_by_class, not for the other function. ctreeviz_leaf_samples is consistent with the other names and ctree implies leaf samples would be shown per class.

Fixed names in https://github.com/parrt/dtreeviz/commit/aca1aed0611b2029ff9498fed6de4481d6457593


Automate connecting and disconnect for clients

I threw together a simple auto connect/disconnect batch file. All its doing is using adb connect to the desired client, executing scrcpy wth the -s option. Then it watches tasklist.exe until you close scrcpy which triggers the adb disconnection of the client. Just thought it may be helpful to others.... I have a template and a example attached.
Example.txt
auto-connect.txt

Just updated ethernally script which facilitates wireless mirroring
https://github.com/fusionneur/ethernally
Let me know if you encounter any issue. Thank you


Support asynchronous JSON RPC client with Tornado

Fixed #16.

Looks good thanks

This is now on pypi. The class name TornadoServer is kind of confusing, I'd be keen to hear your thoughts on changing the names (issue #19)

That's good.


1.0.3 - added automatic selection of iface for subscription server

Hi,
I have added a automatic selection of the interface where the subscriber server resides on.
This solves the problem of 2 active NIC's wherby the UPNP-device resides on the "second" nic and the subscriber server would use the first NIC for listening to the subscription events. (Will never get the events in this case)
It works only if the HOSTNAME of the client is an ip. The resolving of the name is not implemeneted. If its no ip then the code should behave as it is like now and use the first nic.

I have added another bugfix on subscriptions.
The response to a subscription event has to be an 200 OK, otherwise a proper developed UPNP device will kick us out of its subscription list


Service Fabric Deploy Task should allow to take SF connection from publish profile instead of taking from Cluster Connection Service endpoint

Environment

Server - VSTS or TFS on-premises?

VSTS

Agent - Hosted or Private:

Private

Issue Description
ServiceFabricDeploy:
Service Fabric Deploy Task should allow to take SF connection from publish profile instead of taking from Service endpoint. This would allow to automate the deployment from a generated publish profile from script. The creation of service endpoint in VSTS is a manual process and for a highly automated release process for big project, we do not want to create that Cluster Connection endpoint manually for multiple environment and rather would prefer to generate the publish profile and populate the endpoint and credential taking output from cluster bootstrapping and then feed it to this deploy task since it already has a provision of providing publish profile path.

@krish-gh
Thanks for reaching out.
SF endpoint creation requires information such as certificate and password. Wanted to hightlight that and check if you are ok to maintain that as part of code ?

That should be fine..

@krish-gh we haven't heard this requirement from any other customer. Most of the people actually want the connection to be defined in service endpoint as it enables them to create a secure(proper permissions set on service endpoints), well-defined(deployment to a particular prod environment will happen only through a particular release environment) deployment workflow.
FWIW you can create service endpoint programmatically using REST API.


Tensorboard works with terminal&browser but not vscode

Environment data

VS Code version: 1.84.2
Python Extension version (available under the Extensions sidebar): v2023.20.0
OS (Windows | Mac | Linux distro) and version: Windows 10 family
Anaconda version: 23.3.1
Type of virtual environment used: conda
Jupyter server running:  N/A

Expected behaviour
Show data.
Actual behaviour
"No dashboards are active for the current data set"
Steps to reproduce:
If I first launch tensorboard in terminal and then open vscode everything is fine. But if I don't prelaunch tensorboard and make it running under the hood, vscode can't show dashboard. Even worse, If I did prelaunch it but after open vscode, I close the terminal process, then I can never make tensorboard works except I close vscode.

The problem is clearly caused by vscode extensions.

Please could you enable logging as follows:

Open settings & go into Tensorboard->Logging
Change the value to verbose
Reload VS Code,
Attempt to repro the issue & then copy the output from the Tensorboard output panel.


Checkbox to Radiobutton?

Is this intended?
VB.Net :
 Dim Checkbox As CheckBox = New RadioButton("")
C# :
   CheckBox Checkbox = new RadioButton("");
I couldn't understand why my checkbox couldn't uncheck itself, it's because i didn't change the new radiobutton to checkbox, when i was messing around with it.

Radiobutton inherit from checkbox because they share some basic logic. However they are entirely different entities and should not be used polymorphically.
Checkbox should not uncheck themselves since that's not what check boxes do.
Hope this answers your question.


finishing up v2.4 docker to singularity conversion

hey @singularityware/singularity-admin @chrisfilo here is a first shot at docker2singularity conversion for 2.4. We do the following:

use an empty build sandbox to construct the image from docker export
use the folder to build a squashfs image

I had some trouble with the check for busybox to determine if -executable should be used or not. For an ubuntu test image, it triggered the case that failed. If it's the case that not using executable works for both busybox and other flavors, I just removed that step. But I'm not familiar with the addition.
Chris - squashfs can be converted back to a writable thing, if needed. I will leave it up for open discussion to decide how we should maintain the branches (e.g., there are many benefits to squashfs that would make it an appealing default). Take a look at the changes and let me know your thoughts! I'm likely closing up shop soon, I think it's 10pm here, and I haven't gone home yet... oups :P

I am a bit busy right now - please give me a few days to thoroughly test
this PR.
Best,
Chris
On Wed, Oct 18, 2017 at 6:54 PM, Vanessa Sochat<EMAIL_ADDRESS>wrote:

hey @singularityware/singularity-admin
https://github.com/orgs/singularityware/teams/singularity-admin
@chrisfilo https://github.com/chrisfilo here is a first shot at
docker2singularity conversion for 2.4. We do the following:

use an empty build sandbox to construct the image from docker export
use the folder to build a squashfs image

I had some trouble with the check for busybox to determine if -executable
should be used or not. For an ubuntu test image, it triggered the case that
failed. If it's the case that not using executable works for both busybox
and other flavors, I just removed that step. But I'm not familiar with the
addition.
Chris - squashfs can be converted back to a writable thing, if needed. I
will leave it up for open discussion to decide how we should maintain the
branches (e.g., there are many benefits to squashfs that would make it an
appealing default). Take a look at the changes and let me know your
thoughts! I'm likely closing up shop soon, I think it's 10pm here, and I
haven't gone home yet... oups :P
You can view, comment on, or merge this pull request online at:
https://github.com/singularityware/docker2singularity/pull/27
Commit Summary

finishing up v2.4 docker to singularity conversion

File Changes

M Dockerfile
https://github.com/singularityware/docker2singularity/pull/27/files#diff-0
(10)
M README.md
https://github.com/singularityware/docker2singularity/pull/27/files#diff-1
(12)
M docker2singularity.sh
https://github.com/singularityware/docker2singularity/pull/27/files#diff-2
(81)

Patch Links:

https://github.com/singularityware/docker2singularity/pull/27.patch
https://github.com/singularityware/docker2singularity/pull/27.diff

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/singularityware/docker2singularity/pull/27, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOkp-xCJ2lW8KGMkXl6aiwe4VWtJ93Jks5stqvMgaJpZM4P-mAe
.

No worries! Please take the time that you need. Happy Friday! :ice_cream:

I tested it on the following images:

poldracklab/mriqc
ubuntu
continuumio/miniconda
bids/cpac

Conversion did not error for any of them, but the bids/cpac image does not work correctly after conversion using this branch:
PS D:\drive\workspace\docker2singularity> docker run -ti --rm --read-only bids/cpac
usage: run.py [-h] [--pipeline_file PIPELINE_FILE]
              [--data_config_file DATA_CONFIG_FILE]
              [--aws_input_creds AWS_INPUT_CREDS]
              [--aws_output_creds AWS_OUTPUT_CREDS] [--n_cpus N_CPUS]
              [--mem_mb MEM_MB] [--mem_gb MEM_GB] [--save_working_dir]
              [--participant_label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]]
              [--participant_ndx PARTICIPANT_NDX] [-v]
              bids_dir output_dir {participant,group,test_config,GUI}
run.py: error: too few arguments

With old converter (using .img)
[chrisgor@sh-ln02 login! ~]$ singularity run -c -e /share/PI/russpold/singularity_images/bids_cpac-2017-03-29-b91418919662.img
usage: run.py [-h] [--pipeline_file PIPELINE_FILE]
              [--data_config_file DATA_CONFIG_FILE]
              [--aws_input_creds AWS_INPUT_CREDS]
              [--aws_output_creds AWS_OUTPUT_CREDS] [--n_cpus N_CPUS]
              [--mem_mb MEM_MB] [--mem_gb MEM_GB] [--save_working_dir]
              [--participant_label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]]
              [--participant_ndx PARTICIPANT_NDX] [-v]
              bids_dir output_dir {participant,group,test_config,GUI}
run.py: error: too few arguments

with new converter
[chrisgor@sh-ln02 login! ~]$ singularity run -c -e /share/PI/russpold/singularity_images/bids_cpac-2017-03-29-c780e38ca1e9.simg
Traceback (most recent call last):
  File "/code/run.py", line 5, in <module>
    import yaml
ImportError: No module named yaml

Great news! I'll test out that last one this weekend. Thanks for testing these out, I'll keep you updated if I figure out the last one.

Great!
With regards to image format I would at least provide an option for people
to chose between .img and .simg. Not all systems have updated singularity.
Best,
Chris
On Fri, Oct 27, 2017 at 4:10 PM, Vanessa Sochat<EMAIL_ADDRESS>wrote:

Great news! I'll test out that last one this weekend. Thanks for testing
these out, I'll keep you updated if I figure out the last one.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/singularityware/docker2singularity/pull/27#issuecomment-340118430,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkp6VU0R3i2Oln5jnZJBN3AdBLoscnks5swmL6gaJpZM4P-mAe
.

Good idea. I'll have the default build a folder (sandbox) to simg, and then the user can choose to either keep the folder (for development) OR to produce instead an ext3 image. I will think more about the specific commands and include them in my notes after testing.

hey @chrisfilo ! A quick answer for you. The reason for the missed import is that yaml is installed under miniconda at /usr/local/bin/miniconda and the runscript specifies to use python from the environment (#!/usr/bin/env python) and the active python (after building the buggy container) is:
$ which python
/usr/bin/python

and we can see yaml is where it should be, with miniconda
$ /usr/local/bin/miniconda/bin/python
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import yaml

and this all stems from the fact that for the new build command, when we build from a sandbox folder it doesn't include the typical metadata bits (bootstrap did). I should be able to fix this up.

Okay, you can check out the updated README to see the usage, and I'll show examples here!  First, some general comments:

we don't need to worry about figuring out a pre-specified size, ever again
the default format is much smaller than previous
users have options to choose any format, and all formats can be converted to others later if needed

The user has the option to build:

squashfs (no arguments specified) gives you a squashfs (*.simg) image. This is a compressed, reliable, and read only format that is recommended for production images.
sandbox (-f) builds your image into a sandbox folder. This is ideal for development, as it will produce a working image in a folder on your system.
ext3 (-w) builds an older format (ext3) image (*.img). This format is not recommended for production images as we have observed degradation of the images over time, and they tend to be upwards of 1.5x to 2x the size of squashfs.

Build into Sandbox Folder
I'll start with a folder, because we can easily peek inside to see some changes. By specifying -f the user can build a sandbox folder (very useful for development)
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /home/vanessa/Desktop:/output --privileged -t --rm docker2singularity -f bids/cpac

Image Format: sandbox
Inspected Size: 3689 MB
(1/9) Creating a build sandbox...
(2/9) Exporting filesystem...
(3/9) Adding run script...
(4/9) Setting ENV variables...
(5/9) Adding mount points...
(6/9) Fixing permissions...
(7/9) Stopping and removing the container...
(8/9) Building sandbox container...
(9/9) Moving the image to the output folder...
  4,830,419,466  99%  200.87MB/s    0:00:22 (xfr#103037, to-chk=0/121041)
Final Size: 3606MB

let's go look at it...
cd -
/home/vanessa/Desktop
$ singularity run bids_cpac-2017-03-29-c8816111946c/
usage: run.py [-h] [--pipeline_file PIPELINE_FILE]
              [--data_config_file DATA_CONFIG_FILE]
              [--aws_input_creds AWS_INPUT_CREDS]
              [--aws_output_creds AWS_OUTPUT_CREDS] [--n_cpus N_CPUS]
              [--mem_mb MEM_MB] [--mem_gb MEM_GB] [--save_working_dir]
              [--participant_label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]]
              [--participant_ndx PARTICIPANT_NDX] [-v]
              bids_dir output_dir {participant,group,test_config,GUI}
run.py: error: too few arguments

Note that it runs without error. Also note that it's a folder,  and we can see that this updated docker2singularity has a proper metadata folder (.singularity.d)
tree -L 1 bids_cpac-2017-03-29-c8816111946c/
bids_cpac-2017-03-29-c8816111946c/
├── bin
├── boot
├── code
├── cpac_resources
├── dev
├── environment -> .singularity.d/env/90-environment.sh
├── etc
├── home
├── lib
├── lib64
├── local-scratch
├── media
├── mnt
├── oasis
├── opt
├── proc
├── root
├── run
├── sbin
├── scratch
├── singularity -> .singularity.d/runscript
├── singularity.json
├── srv
├── sys
├── tmp
├── usr
└── var

instead of having /environment we mirror the practice of docker import and add it to 10-docker.sh
tree -L 1 bids_cpac-2017-03-29-c8816111946c/.singularity.d/env/
bids_cpac-2017-03-29-c8816111946c/.singularity.d/env/
├── 01-base.sh
├── 10-docker.sh
├── 90-environment.sh
├── 95-apps.sh
└── 99-base.sh

and the runscript
cat bids_cpac-2017-03-29-c8816111946c/.singularity.d/runscript
#!/bin/sh
/code/run.py $@

Build Squashfs
Now let's look at the default, which is to build a squashfs image:
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /home/vanessa/Desktop:/output --privileged -t --rm docker2singularity bids/cpac

Image Format: squashfs
Inspected Size: 3689 MB

(1/9) Creating a build sandbox...
(2/9) Exporting filesystem...
(3/9) Adding run script...
(4/9) Setting ENV variables...
(5/9) Adding mount points...
(6/9) Fixing permissions...
(7/9) Stopping and removing the container...
(8/9) Building squashfs container...
Building image from sandbox: /tmp/bids_cpac-2017-03-29-274df13a156c.build
Building Singularity image...
Singularity container built: /tmp/bids_cpac-2017-03-29-274df13a156c.simg
Cleaning up...
(9/9) Moving the image to the output folder...
  1,402,847,263 100%  275.90MB/s    0:00:04 (xfr#1, to-chk=0/1)
Final Size: 1338MB

Look at the difference in size!! 3689MB down to 1338MB and more robust! Woot! Let's test it out...
singularity run bids_cpac-2017-03-29-274df13a156c.simg
usage: run.py [-h] [--pipeline_file PIPELINE_FILE]
              [--data_config_file DATA_CONFIG_FILE]
              [--aws_input_creds AWS_INPUT_CREDS]
              [--aws_output_creds AWS_OUTPUT_CREDS] [--n_cpus N_CPUS]
              [--mem_mb MEM_MB] [--mem_gb MEM_GB] [--save_working_dir]
              [--participant_label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]]
              [--participant_ndx PARTICIPANT_NDX] [-v]
              bids_dir output_dir {participant,group,test_config,GUI}
run.py: error: too few arguments

Legacy Format
If the user wants the legacy format, that is available with -w:
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /home/vanessa/Desktop:/output --privileged -t --rm docker2singularity -w bids/cpac

Image Format: writable
Inspected Size: 3689 MB

(1/9) Creating a build sandbox...
(2/9) Exporting filesystem...
(3/9) Adding run script...
(4/9) Setting ENV variables...
(5/9) Adding mount points...
(6/9) Fixing permissions...
(7/9) Stopping and removing the container...
(8/9) Building writable container...
Building image from sandbox: /tmp/bids_cpac-2017-03-29-189989307b50.build
Creating empty Singularity writable container 3606MB
Creating empty 4507MiB image file: /tmp/bids_cpac-2017-03-29-189989307b50.img
4507+0 records in
4507+0 records out
Formatting image with ext3 file system
Image is done: /tmp/bids_cpac-2017-03-29-189989307b50.img
Building Singularity image...
Singularity container built: /tmp/bids_cpac-2017-03-29-189989307b50.img
Cleaning up...
(9/9) Moving the image to the output folder...
  4,725,932,063 100%  259.65MB/s    0:00:17 (xfr#1, to-chk=0/1)
Final Size: 4508MB

and test...
singularity run bids_cpac-2017-03-29-189989307b50.img
usage: run.py [-h] [--pipeline_file PIPELINE_FILE]
              [--data_config_file DATA_CONFIG_FILE]
              [--aws_input_creds AWS_INPUT_CREDS]
              [--aws_output_creds AWS_OUTPUT_CREDS] [--n_cpus N_CPUS]
              [--mem_mb MEM_MB] [--mem_gb MEM_GB] [--save_working_dir]
              [--participant_label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]]
              [--participant_ndx PARTICIPANT_NDX] [-v]
              bids_dir output_dir {participant,group,test_config,GUI}
run.py: error: too few arguments

This image format is less ideal because it's larger, more prone to corruption over time, and likely to not have future support.
Conversion
Importantly, any format can be converted to another format easily, for example, let's convert the ext3 image to squashfs:
sudo singularity build production.simg bids_cpac-2017-03-29-189989307b50.img
Building from local image: bids_cpac-2017-03-29-189989307b50.img
Building Singularity image...
Singularity container built: production.simg
Cleaning up...

$ du -shm production.simg
1338	production.simg

and that would work for the sandbox folder too. Still not recommended, because if there were changes not recorded, this is not reproducible.

huh, I wonder if I can get the labels from the image too? Let me try that.

To keep a record in our discussion, here is how the modified generation looks, showing an added step to generate metadata labels
docker run -v /var/run/docker.sock:/var/run/docker.sock \
-v /host/path/change/me:/output \
--privileged -t --rm \
singularityware/docker2singularity
ubuntu:14.04

Image Format: squashfs
Inspected Size: 188 MB

(1/10) Creating a build sandbox...
(2/10) Exporting filesystem...
(3/10) Creating labels...
(4/10) Adding run script...
(5/10) Setting ENV variables...
(6/10) Adding mount points...
(7/10) Fixing permissions...
(8/10) Stopping and removing the container...
(9/10) Building squashfs container...
Building image from sandbox: /tmp/ubuntu_14.04-2017-09-13-3e51deeadc7b.build
Building Singularity image...
Singularity container built: /tmp/ubuntu_14.04-2017-09-13-3e51deeadc7b.simg
Cleaning up...
(10/10) Moving the image to the output folder...
     62,591,007 100%  340.92MB/s    0:00:00 (xfr#1, to-chk=0/1)
Final Size: 60MB

and now the container can be inspected in various ways, default shows labels:
 singularity inspect ubuntu_14.04-2017-09-13-3e51deeadc7b.simg
{
    "org.label-schema.singularity.build": "squashfs",
    "org.label-schema.docker.version": "17.06.2-ce",
    "org.label-schema.schema-version": "1.0",
    "org.label-schema.singularity.build-type": "docker2singularity",
    "org.label-schema.docker.id": "sha256:dea1945146b96542e6e20642830c78df702d524a113605a906397db1db022703",
    "org.label-schema.build-date": "2017-10-28-17:19:18",
    "org.label-schema.singularity.version": "2.4-dist",
    "org.label-schema.docker.created": "2017-09-13"
}

and then with different args (-e and -r) you can see the environment and runscript, respectively
singularity inspect --json -e -r ubuntu_14.04-2017-09-13-3e51deeadc7b.simg
{
    "data": {
        "attributes": {
            "environment": "# Custom environment shell code should follow\n\n",
            "runscript": "#!/bin/sh\n/bin/bash $@\n"
        },
        "type": "container"
    }
}

ping!

Sorry for the late reply. I'm running into the following problem:
PS D:\drive\workspace\docker2singularity> docker run -ti --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v D:\singularity\:/output singularityware/docker2singularity:dev poldracklab/fmriprep

Image Format: squashfs
Inspected Size: 10760 MB

(1/10) Creating a build sandbox...
(2/10) Exporting filesystem...
tar: lib64: Directory renamed before its status could be extracted
tar: lib/systemd/system/timers.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/sysinit.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/sockets.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/sigpwr.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/resolvconf.service.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/rescue.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/reboot.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/poweroff.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/multi-user.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/local-fs.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/graphical.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/getty.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system/busnames.target.wants: Directory renamed before its status could be extracted
tar: lib/systemd/system: Directory renamed before its status could be extracted
tar: lib/systemd: Directory renamed before its status could be extracted
tar: lib: Directory renamed before its status could be extracted
tar: etc/xdg/systemd: Directory renamed before its status could be extracted
tar: etc/xdg: Directory renamed before its status could be extracted
tar: etc/systemd/system/timers.target.wants: Directory renamed before its status could be extracted
tar: etc/systemd/system/sysinit.target.wants: Directory renamed before its status could be extracted
tar: etc/systemd/system/multi-user.target.wants: Directory renamed before its status could be extracted
tar: etc/systemd/system/getty.target.wants: Directory renamed before its status could be extracted
tar: etc/systemd/system: Directory renamed before its status could be extracted
tar: etc/systemd: Directory renamed before its status could be extracted
tar: etc/sysctl.d: Directory renamed before its status could be extracted
tar: etc/ssl/certs: Directory renamed before its status could be extracted
tar: etc/ssl: Directory renamed before its status could be extracted
tar: etc/rcS.d: Directory renamed before its status could be extracted
tar: etc/rc6.d: Directory renamed before its status could be extracted
tar: etc/rc5.d: Directory renamed before its status could be extracted
tar: etc/rc4.d: Directory renamed before its status could be extracted
tar: etc/rc3.d: Directory renamed before its status could be extracted
tar: etc/rc2.d: Directory renamed before its status could be extracted
tar: etc/rc1.d: Directory renamed before its status could be extracted
tar: etc/rc0.d: Directory renamed before its status could be extracted
tar: etc/modules-load.d: Directory renamed before its status could be extracted
tar: etc/ld.so.conf.d: Directory renamed before its status could be extracted
tar: etc/fonts/conf.d: Directory renamed before its status could be extracted
tar: etc/fonts: Directory renamed before its status could be extracted
tar: etc/alternatives: Directory renamed before its status could be extracted
tar: etc: Directory renamed before its status could be extracted
tar: dev: Directory renamed before its status could be extracted
tar: bin: Directory renamed before its status could be extracted
tar: Exiting with failure status due to previous errors

This happens with -w as well, but does not happen when using current master.

Thanks @chrisfilo ! A few quick questions to help debug, since I don't have windows.

Previously you indicated these images functioned as expected:

poldracklab/mriqc
ubuntu
continuumio/miniconda
bids/cpac

Did these all have the same error as you reported above?

For the image poldracklab/fmriprep does it have issue with the whiteout files when used with regular singularity?

For the entire list of images we've looked at:

poldracklab/mriqc
ubuntu
continuumio/miniconda
bids/cpac
poldracklab/fmriprep

Do you get the same error and failure on a linux host? I think this might be related to the filesystem or overlay fs, see these issues:

https://github.com/coreos/bugs/issues/1095

We also might want to consider the version of Docker running on the host:

https://github.com/nodejs/docker-node/issues/379

We are having a release of 2.4.1 soon, and we might have a solution for the whiteout files, although not sure yet. Huge thanks for helping with the testing, the most important/relevant environment is Windows, and I don't think I'd know where to find it if I tried. And again please take the time you need to properly test!

Out of the list this is the only image I get this error for. I did not have a chance to try it on Linux.
What makes you think this is a Windows related issue? Have you tried to replicate it on your machine? The two issue you linked to were reported for Linux hosts.
As for .wh files I tried this but I run into another error:
[chrisgor@sh-101-58 ~]$  SINGULARITY_CACHEDIR=/tmp singularity pull -s 15307 --name /home/groups/russpold/singularity_images/poldracklab_fmriprep_test.img docker://poldracklab/fmriprep
WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull. Use Singularity Registry
WARNING: (shub://) to pull exactly equivalent images.
WARNING: Using the --size option when building from Docker Hub is deprecated.
WARNING: Container will be sized appropriately for Docker image.
Docker image path: index.docker.io/poldracklab/fmriprep:latest
Cache folder set to /tmp/docker
[48/48] |===================================| 100.0%
Importing: base Singularity environment
Importing: /tmp/docker/sha256:b3e1c725a85f0953e81815b7c7aabfad9ebfd90af53f99248981282b8045d787.tar.gz
Importing: /tmp/docker/sha256:4daad8bdde31f5969d6a44800e993a911bed25853b2f89417b07ce7f0ac67714.tar.gz
Importing: /tmp/docker/sha256:63fe8c0068a811103c702452985368fc99014f83b5f0151cf631644f13f383b2.tar.gz
Importing: /tmp/docker/sha256:4a70713c436f2350e8a9f1c1e6c823290f331254c6c21d5657056499d8575608.tar.gz
Importing: /tmp/docker/sha256:bd842a2105a862f62ba4ad726531029be49046c290307e9c07a688c89f4606e6.tar.gz
Importing: /tmp/docker/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4.tar.gz
Importing: /tmp/docker/sha256:67664e735a306bf2beff64a5f8b3ffb22a3f48684cdf412fd93e8866f211fe0a.tar.gz
Importing: /tmp/docker/sha256:7f609af92f3a7ceb8134c5f278612a84f9558872ee2102fb0499266a1a6f5b4c.tar.gz
Importing: /tmp/docker/sha256:4649e39f3fb4cd5e4ebf9750a9b324fa0005056ebf076f70f9845f64adb2c5b9.tar.gz

gzip: /tmp/docker/sha256:4649e39f3fb4cd5e4ebf9750a9b324fa0005056ebf076f70f9845f64adb2c5b9.tar.gz: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

Is there any update on this?

I think these bugs were present in earlier versions of 2.4 and may be resolved now (but in development branch) I think the best thing to do is to keep it in a branch for now (and @0xaf1f you should clone the branch and build the image freshly) to do your conversions (with the development version of singularity). When the next release goes out, we will make a corresponding branch (and only merge to be the "default" given no errors). It is  my preference to only create the branches with official releases of some flavor, so having 2.4 here represent some stage of development doesn't seem ideal.

hey @0xaf1f I just threw together a docker2singularity that has the updated 2.4 (development) installed if you want to wait for it to build, and then try. --> https://hub.docker.com/r/vanessa/singularity/builds/ and probably faster to build on your own --> https://github.com/singularityhub/docker/blob/master/docker2singularity/Dockerfile.dev I'm going to be afk a bit but will be back if you have questions or want to work together, I didn't put much time into writing docs or testing.... :)

I really appreciate it, @vsoch. Though I went on the route of creating a local registry and pulling from that, and it looks like it produced a good image.

hey, that works too! :)

Version 2.5 is released, so this PR is being closed as it wouldn't be ideal to merge into master. For those wanting docker2singularity for version 2.4, use the 2.4 branch: https://github.com/singularityware/docker2singularity/tree/v2.4
I will prepare a v2.5 branch soon.


Automated cherry pick of #9076: Notify via contact skip receiver

Cherry pick of #9076 on release/3.5.
#9076: Notify via contact skip receiver

Codecov Report

Merging #9078 (803d718) into release/3.5 (6a15e02) will increase coverage by 1.84%.
The diff coverage is 12.00%.

@@              Coverage Diff               @@
##           release/3.5   #9078      +/-   ##
==============================================
+ Coverage         5.98%   7.82%   +1.84%
==============================================
  Files              796     247     -549
  Lines           125693   33743   -91950
==============================================
- Hits              7522    2641    -4881
+ Misses          117504   30841   -86663
+ Partials           667     261     -406

Flag
Coverage Δ

aFlag
7.82% <12.00%> (+1.84%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/cloudcommon/db/jointbase.go
0.00% <ø> (ø)

pkg/cloudprovider/objectstore.go
2.57% <ø> (ø)

pkg/hostman/hostinfo/hostovn.go
27.45% <0.00%> (-2.03%)
:arrow_down:

.../monitor/alerting/conditions/suggestrulereducer.go
0.00% <0.00%> (ø)

pkg/multicloud/esxi/host.go
0.77% <0.00%> (-0.01%)
:arrow_down:

pkg/multicloud/esxi/manager.go
0.70% <0.00%> (-0.01%)
:arrow_down:

pkg/util/netutils2/netutils.go
21.62% <100.00%> (+0.35%)
:arrow_up:

pkg/util/seclib2/seclib.go
86.04% <100.00%> (ø)

pkg/keystone/models/regionresource.go

pkg/appsrv/ring.go

... and 525 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 13834c9...803d718. Read the comment docs.


Feature/22 report listener

Hey, I had a look at the code. The XMLParser looks good and I think its good you decided to work with XML-Files instead of writing a custom Resultslistener. Just for the Unit-Tests we should put them in a separate 'Test'-folder but for now it is fine.
You can just merge the branch in main :)