[RFE] run image as part of running source-to-image

It came up in conversation that it might be nice if the sti "program" accepted a flag that would: cause the resulting built Docker image to stay running instead of actually being a created image.
There may be cases where I want to test the result of my STI process locally without having to push the Docker image somewhere.
I guess this flag would essentially cause "run" to be executed immediately after "assemble" inside the instance of the builder image.
It's not a huge time savings, but it could potentially make a few other automation tasks easier... maybe?

I would actually describe it as immediately running the output image, as opposed to running the run script after the assemble script.  (among other things, that's much easier to implement).

I guess I don't understand how it's easier.
If there's an env-var for "run immediately", assemble calls run?
I'm probably missing something.

what you're proposing would require every single assemble script be updated to respect that env var and invoke run.
in addition, even if you want to run immediately, we're still going to want to commit the output of your assemble script.  so polluting that container by running the run command in it makes that hard/impossible.
it's just cleaner to run the assemble script, commit the result, and then if desired, launch that image.

Well, the point of this RFE was to improve time and make for more rapid development cycles. Having to stop the result of the build, commit it, and then run the new image takes time. Sure, it's a trivial amount of time, but still time.
You can kind of look at it both ways. Either we "pollute" the running container and then might have to rebuild, or we "pollute" the local user's Docker with lots of old junk images.
The other thing to think about is that, if we're only using this sti tool to "test" before sending code along somewhere else, it's going to end up getting rebuilt anyway.
I don't think I'm being very coherent.

there will be junk images either way, since even with your suggestion you'll get a new image everytime you run STI.
you will have fewer junk containers with your approach (half as many) if we don't run with --rm
anyway i take your point, but it also seems like there is value in producing a re-runnable image when you build, even if you want to run it immediately initially.  Which means paying the cost of commit.

Yes, I think there's value in both sides. Sounds like we're on the same page.

One important question is, should we publish the exposed ports to host by default when doing this run? I assume yes, but what if port is already in use, should we increment number to find first free and tell the user? As for the first question I'm positive, the latter I'm quite reluctant for doing so. Thoughts?

I would think that if the port is already in use we should probably just fail?

One other question that just popped in my head, what if running image requires passing certain vars, see our DB images (I know these are not the STI case, but I can imagine such requirement). What then? We'll just print the image usage (if one is provided, like in our DB images) and finish. I guess that's acceptable, we tried, but it's up to the user to provide those.

If the tool can accept the vars to pass to the container, then yeah.
If the container needs vars but we don't offer a way to get them into the container, then the "run" flag becomes slightly less useful...

We do support providing env vars, but only for build process itself. I think OOTB we should provide simple run with exposing the EXPOSE port to host but without any vars.

@soltysh if you provide env vars to the build process, they get fed into assemble but then committed into the image so they'd be available for run as well, right?  So i think we're sort of ok there.
For ports, yeah i guess we should try to expose it on the port the image declares and fail if we can't get it, rather than exposing on a random port and making the user figure it out.... though we probably could run it on a random port mapping and print out that mapping as part of the whole flow.

@bparees yeah, but I think I wouldn't want to have an image (even locally) with commited secrets or other env variables, which supposed to be configurable at run time. I'm not sure if docker will let you override those though. But generally speaking this is a big security whole IMHO.

@soltysh I think you're beyond the local dev use case now.

@bparees I think that applies to local dev flow as well.

@soltysh why do i care if my secrets/config are embedded in my image if i'm not pushing that image anywhere and i'm repeatedly rebuilding the image?

@bparees maybe that's one of my obsessions about security :wink:

So I've got a prototype of the form @bparees suggested working.  A few devil in the details type items (or perhaps opportunities for folks to educate me on Docker and Go ;-) ) arose when I looked at attaching to the container and piping stdout/stderr to glog so the users could see the output of their build in real time as a way to determine image validity.

If the resulting image from the sti invocation results in a long running container, attaching and waiting for the container to exit will not occur, and the sti invocation will not exit.  A specific example is the image produced by sti build git://github.com/bparees/openshift-jee-sample openshift/wildfly-8-centos test-jee-app

I believe the relevant code in s2i wrt attaching to the container, capturing stdout/stderr, waiting for the container to exit, is here:   https://github.com/gabemontero/source-to-image/blob/master/pkg/docker/docker.go#L420 -L473
Best as I can tell, the WaitGroup API from the sync package ONLY reacts to the counter state, with no timeout based bail out.  If I bypass the WaitGroup call in the above code snippet, the sti invocation exits, and the image is running in a new container, but we don't get the images stdout/stderr output.  Reworking this logic to somehow employ a wait(timeout) approach is currently not deemed desirable for this item.

If I leave the "hanging" behavior in, and ctrl-c out of sti, the newly running image is still running, and you have to explicitly stop it with docker stop <container id>

So a (at least partial) list of options we could pursue:
have some flavor of --run options that allows you to choose between i) no run (default) ii) run with wait to exit, pipe stdout/stderr, iii) run with no wait and no stdout/stderr
we on iii), and printing out the resulting container id and reminding the user they can use docker cmds to inspect the container/logs/imgaes and stop the resulting container as needed, but solicit feedback.

At after some initial discussion, we don't plan on getting into capturing arguments specific to the docker run command.  The user can already set ENV vars with the -e option.  The current plan is to leave it at that.

given that even in (ii) the user ultimately has to docker kill the container that got launched, I think we should go with (iii) in which we explicitly tell them "we launched this for you, but you need to go inspect/manage it yourself"  (along with (i) as the default behavior)

@thoraxe this was your request so please weigh in.

It appears there is a fly in the ointment with iii) from the earlier comments that I missed last night.  With either avoiding the docker wait equivalent call altogether, or doing it in a separate thread, the container is exiting prematurely and not staying up.  I'm able to grab the exit code on the container when waiting on a separate thread, and it appears to be 137.  There are some google hits on this, but nothing with a solution per se.  What is weird is the process does not even show up as having started then exited in docker ps -a.
Like it never started unless I call wait in-line.
I'll experiment some more, but for images/containers that stay up, we may be left with
a) printing a banner like reminder to tell them to ctrl-c and inspect the image like before, don't pipe stdout/stderr to glog
b) or do pipe stdout/stderr for glog, and hope our banner reminding them to ctrl-c does not get lost in the shuffle as their image output comes out.
Pros and cons with each ...a) might be the lesser of the evils here.
Needless to say, this is getting a bit messier for images that stay up.

definitely prefer (a) but we ought to be able to launch the container and then exit.  I wonder if the s2i process is simply exiting before the call to the docker daemon is complete.
you said you're getting a 137 rc?  i'm confused about the circumstances under which you're getting that given that docker ps -a doesn't even show the container having run.

Yep, that was one of my theories.  The sti process exiting has something to do with it, but it appears that the way it exits matters.  In other words, ending "naturally" as the last instruction completes results in different behavior than a Ctrl-C.
Here's what I've tried:

First call "InspectContainer" on the docker client and confirm the container was in Running state before moving on.  That did not help.  And it exited too quickly to see stdout/stderr from the image.

What I was trying to avoid ... just adding sleeps.  With a sleep of 5 seconds, I saw what looked like the complete set of stdout/stderr from the image.  But when I run docker ps -a after sti completes, the container is gone - not even listed as exited.

With a sleep of 60 seconds, I have time to run docker ps -a while sti is still up.  I see the container.  When the 60 seconds is up, and sti exits, then docker ps -a no longer shows the container - does not even indicate that it exited.  Saw the same set of image stdout/stderr show up via glog.

As a reminder, employing Ctrl-C once sti has invoked the docker wait.. equivalent leaves the container up.

So some form of configurable version of 3) would perhaps/conceivably be more usable than prior options.
I'll start looking at how origin/k8s starts containers, and see if there are any differences in how the fsouza docker client is leveraged.  Perhaps since openshift stays up, this behavior is masked ?  Curious to see if that is the case.
But I'll probably reduce the cycles on this one for the next little bit and dive into extended tests.

sounds like you're running the container with the equivalent of "docker run -rm" which removes the container after it terminates.  that would explain it not showing up in docker ps -a later.  A quick survey of the client library doesn't show me a way to control that behavior, unfortunately, but we don't necessarily have to change it as long as we're convinced the container is being run when it's supposed to be run.
if you cntrl-c while s2i is sleeping in (3) does that immediately kill the running container?  i'm guessing not, but might be worth a test.  Seems like there's some cleanup/signal propagation that only happens when s2i exits cleanly, not when it gets cntrl-c'd.

Ctrl-c while s2i is sleeping leaves the container up - i.e. the same as when we ctrl-c during the docker wait .. call.
And my interpretation of the fsouza client library is the same as yours, in that they do not expose the -rm switch.  RE: https://github.com/openshift/source-to-image/blob/master/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go#L405-L447
And the google hits I  got with the exit code 137 that I mentioned earlier spoke to signal handling, so concur there as well wrt your point about the cleanup/signal handling differences with clean exit vs. ctrl-c.
My search through k8s revealed no differences in the low level starting of containers either via the fsouza Godeps.
I'll spell out the approach I referenced in today's scrum (granted your initial reaction was not exactly exultant, though I'm not thrilled either):

change --run= to accept integers instead of boolean
--run=0 means we wait forever (i.e. docker wait) and we both doc in the usage help and online doc, and print a banner during execution, stating that you have to ctrl-c to exit sti
otherwise, --run= specifies the number of seconds we sleep before exiting
we explain in docs that ctrl-c leaves the container up, clean exit does not

I feel dirty even spelling that out, but pending a better suggestion or just punting on this, that is where I'm at on this one.

@gabemontero maybe you can do something with this to stop the container:
http://nathanleclaire.com/blog/2014/08/24/handling-ctrl-c-interrupt-signal-in-golang-programs/

I'm fine with @bparees suggestion from https://github.com/openshift/source-to-image/issues/180#issuecomment-128103661
Although I'm wondering now about being able to edit the filesystem in the running image, since you can't do that in the "built" image later... but since the build needs to be able to write, perhaps this is the way we can achieve the "hot edit" stuff that we wanted? I forget which issue that is.

since this is running via s2i directly, not via openshift, the container we ultimately run (the one running your application code) will be running as the same user who did the assemble/build steps.  so you'll be able to exec in and modify code in this scenario.  But it will have no impact on the ability to do so when running in openshift.

@bparees the signal handling got us the consistent behavior we wanted - thx again for the tip.
So now, we can pipe stdout/stderr to glog, but the customer can also use docker commands to inspect the container running the image (the code prints out a easy to spot message citing the container id).
Once the customer ctrl-c's, sti exits and cleans up the running container.
Aside from scrubbing the code prior to submitting the PR, I've got to tweak the origin doc.  Assuming that is all, I should have a PR late AM / early PM on Wednesday.
An initial scan of the existing unit tests in source-to-image did not reveal any useful suggestions of note there - nothing specific to start up options, and this change is outside the existing algorithm.  I'm leaning toward no unit test, but will take any suggestions.

@gabemontero can we see your POC as a PR, I'd like to give it a try and then provide some useful feedback.

@gabemontero sounds good to me, nice work!

Absolutely @soltysh I will be doing that later today (have an appt this AM so it may be after lunch US EST).  Among other things, I'm curious about your take on where I hooked in the running of the container (among other things, I decided NOT to use the PostExecute phase).
Thanks @bparees

(among other things, I decided NOT to use the PostExecute phase).

Perfect, I'd expect this to be completely separate from the Build process withing S2I, that's why I'd like to see the code behind it.

OK cool.


[oneDNN] UT failure fix: //tensorflow/python/kernel_tests/nn_ops:pooling_ops_3d_test_cpu

With oneDNN enabled //tensorflow/python/kernel_tests/nn_ops:pooling_ops_3d_test_cpu got triggered, a sanity check of empty tensor on avgpool3d_grad is needed. This PR will resolve //tensorflow/python/kernel_tests/nn_ops:pooling_ops_3d_test_cpu test failure

This PR will resolve this issue. https://github.com/tensorflow/tensorflow/issues/58446

@penpornk, I have modified the requested change.

@penpornk , do you have any update on this PR?

@cantonios , it's been pending for some time, any update on merging this PR?


Fixing PR Comments

Please don't merge this PR until all fixes are done.

Marked as on hold as requested


Update error 3.2.0 => 3.4.0

The error I get when clicking the update prompt in code-server is:
https://github.com/cdr/code-server/releases/download/v3.4.0/code-server-v3.4.0-linux-arm64.tar.gz: 404
And yeah, that URL is a 404. Because the release is actually at:
https://github.com/cdr/code-server/releases/download/v3.4.0/code-server-3.4.0-linux-arm64.tar.gz
Note the missing "v" before the version number in the file name.
So my hope is, you guys could fix this simply by uploading a correctly named archive?

Sorry about this.
Should be fixed.
See https://github.com/cdr/code-server/issues/1746
Please let me know!


fix broken link typo

📝 Summary
Fix link in readme
⛱ Motivation and Context
Was looking through the readme and found that this link was broken.
✅ I have run these commands

[X] make lint
[X] make test-race
[X] go mod tidy

Codecov Report

Merging #463 (69abd51) into main (19a5221) will decrease coverage by 0.33%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main     #463      +/-   ##
==========================================
- Coverage   67.69%   67.37%   -0.33%
==========================================
  Files           8        8
  Lines        1226     1226
==========================================
- Hits          830      826       -4
- Misses        347      351       +4
  Partials       49       49

Flag
Coverage Δ

unittests
67.37% <ø> (-0.33%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

server/service.go
74.85% <0.00%> (-0.76%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

ty 🙏


Integrate User Management

It’s not absolutely clear yet, what this will include.
Login, registration, changing the password, email etc. will probably be served by the middleware. Integration in the 2nd frontend may not be necessary.
Now it seems that the middleware will even serve vcs, user defined queries etc. So the user management for the second frontend needs excellent caching mechanisms to make this work without a database close by. Search properties however will be served by the frontend.
(This was copied from the GDoc "[KorAP] Necessary Tasks for the 2nd Frontend to become potentially production ready" and the Trac Ticket #216)

As this issue is wide in scope, I will close it for now. Not all parts of user management are integrated yet, but further integration will refer to specific features.


Language Detection: Nim

https://nim-lang.org

Definitely a language we'll look into once we release a Planner Framework to improve language support. See #95 for discussion and more details

Basic Nim-lang support can be found here: https://www.jetpack.io/devbox/docs/devbox_examples/languages/nim/
Since we deprecated build, we will no longer ship the Planner detection for it.


Example for tailwind directives

I have added
<style>
  :root {
   ...
   // some default css vars
  }

  .form-input,
  .form-select {
    @apply block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-gray-900 focus:border-gray-900 sm:text-sm;
  }
</style>

to a _layout.svelte file and somehow the classes are not taken into account. Any idea why? Could you update the sapper example as well? 🤗
I also tried <style global>...</style>

check your terminal logs. Rollup shows you why it has not taken them into account. Seems that it removes unused styles. You also see that windicss is parsing the classes. So, check if you can change the rollup config.
Also, I am not sure what to do with Sapper and SvelteKit, one is going to be replaced the other one needs some more time to be production ready, so I do not feel safe to suggest any example for that.

Yes, I'd like to switch to SvelteKit ASAP (from Sapper) but it's just not ready and has major breaking changes on a nightly bases. So we have to hold our Sapper App as long as possible.
You are right, windicss does parse the classes
src/components/Globalstyle.svelte
Unused CSS selector ".form-select:focus"
37: }
38: .form-input,
39:   .form-select:focus {
      ^
40:   outline: 2px solid transparent;
41:   outline-offset: 2px;

but rollup somehow removes them? Any idea, where I could look?

rollup.config.js and used svelte plugin (rollup-plugin-svelte) for rollup. Svelte removes unused styles, I guess. But somehow you can change, might found config for rollup-plugin-svelte

I was able to wrap the global styles with :global() and it looks like, its working as expected!

It should wrap global styles with :global()
I might have an idea where to fix. Let me check later today

@wiesson can u test with latest master and latest rollup example. Should include global styles if using
<style global>

</style>

Hmm, it did not work for me. I have created an example repo -> https://github.com/wiesson/svelte-sapper-windicss/blob/main/src/components/Styles.svelte (but still wrapped with :global())

@wiesson the change is not released as a package yet. However, you gut try it with this repo and the examples here. Next release is scheduled later tonight.


Continuous mode enabled, restarting bug in Artyom

Good day,
I am using the Artyom 1.0.6 version. When I'm initializing the script. it always prints at the console the message, " Continuous mode enabled," and suddenly my machine hangs because of the instance.

let check = () => {
gieselle.initialize({
lang:"en-GB",
continuous:true,
debug:true,
listen:true
}).then(() => {
gieselle.say("Hello, I'm Gieselle. Your Virtual Travel Buddy");
}).catch((err) => {
gieselle.say("I couldn't be initialized, please check the console for errors");
console.log(err);
});
};
check();

I had this issue just now, and noticed it was because another tab was using artyom (or probably more specifically speech recognition). Your code is fine, its just that multiple instances cannot be running on the same device, i think...


Amazon Business Documentation Needs Updating

This support document is broken. I have been unable to successfully integrate Amazon Business with Azure AD following this guide.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4b612c20-6995-2ae5-5d73-ef54ccf4afa2
Version Independent ID: cfda3f69-819a-8e44-718e-e718e8eb0806
Content: Tutorial: Azure Active Directory integration with Amazon Business
Content Source: articles/active-directory/saas-apps/amazon-business-tutorial.md
Service: active-directory
Sub-service: saas-app-tutorial
GitHub Login: @jeevansd
Microsoft Alias: jeedes

@theonemcdonald
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

@theonemcdonald
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

@chetansriv Can you please reach out to Amazon Business team and update the document here?

@chetansriv Can you please reach out to Amazon Business team and update the document here?

@v-rant Please reach out to Amazon Business team and validate the instructions again.

Now, we can successfully be able to integrate the application by following the documentation.
#please-close


Configuring Private Networking

Hello,
I have used this module to successfully launch a PostgreSQL instance and I want to secure communication within the VPC by enabling private networking. The relevant variable block:
variable "ip_configuration" {
  type = object({
    authorized_networks = list(map(string))
    ipv4_enabled        = bool
    private_network     = string
    require_ssl         = bool
  })

Can anyone advise what is wrong with the value of "private_network" below and if possible, share an acceptable example value?
variable "ip_configuration" {
  type = object({
    authorized_networks = list(map(string))
    ipv4_enabled        = bool
    private_network     = string
    require_ssl         = bool
  })
  default = {
    authorized_networks = [
      {
        name = "Ralph"
        value = "<IP_ADDRESS>/32"
      }
    ]
    ipv4_enabled        = true
    private_network     = "projects/${var.project_id}/global/networks/${var.vpc_name}"
    require_ssl         = false
  }
}

It's not clear why using variables as part of the value for "private_network" failed, it worked using only text:
private_network     = "projects/webapp-prod-341021/global/networks/default"
Now I get a different error:
Error: Error, failed to create instance webapp-apps-3: googleapi: Error 400: Invalid request: Incorrect Service Networking config for instance: webapp-prod-341021:webapp-apps-3:NETWORK_NOT_PEERED., invalid
with module.webapp-sql-db.google_sql_database_instance.default
on .terraform/modules/webapp-sql-db/modules/postgresql/main.tf line 44, in resource "google_sql_database_instance" "default":

Thanks for your help @morgante.


[Feature Request] Add a new function to InAppEventListener that would allow customization of message action

I have the same issue reported in https://github.com/customerio/customerio-ios/issues/205, which is Universal Links when triggered from In-App won't open the app but the system's browser - and I completely understood the technical reasons behind this, as explained there.
That said, I thought about an option that would allow the message action event to be intercepted and then customized if needed.
My suggestion to address this is to add a new method to the InAppEventListener delegate - messageActionWillExecute, like this:
public func messageActionWillExecute(message: CioMessagingInApp.InAppMessage, actionValue: String, actionName: String) -> Bool {
    // Having this delegate method would allow message to be inspected beforehand then we could choose to:
    // - Let it continue,
    // - Cancel its execution,
    // - Or perform some customized action

    // Returning true -> The message action will execute
    // Returning false -> The message action won't execute
}

And example usage scenario: processing universal links from In-App:
public func messageActionWillExecute(message: CioMessagingInApp.InAppMessage, actionValue: String, actionName: String) -> Bool {
    // Run a custom deep linking handling mechanism, cancelling default message action behavior
    if let url = URL(string: actionValue) {
        aCustomDeepLinkingHandlerInstance.handle(url: url)
        return false
    }

    // Resume with the default message action behavior
    return true
}

Let me know if you need additional information.

Hey @glauberl-ciandt, thank you for writing to us and for a great proposal. We are currently evaluating our options to me the universal-link experience smoother for in-app messages. It's on our near roadmap and we hope to send something out soon. Thank you for your patience and proposal.

Hey @glauberl-ciandt, thank you for being very patient with us while we worked on this. I have good news for you and everyone who asked for it.
We have released the support for universal links from in-app, you can find more details about it here.
Feel free to re-open this, in-case you encounter any issues.


[WEBSITE] Add Link for CWiki

Closing pending website refactor, and this was the wrong way to do it anyway.


Renaming Guzzle to GuzzleHttp since it has changed

When running composer install/update:

Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Package guzzle/guzzle is abandoned, you should avoid using it. Use guzzlehttp/guzzle instead.
Writing lock file
Generating autoload files

Reference:
https://packagist.org/packages/guzzle/guzzle

This package is abandoned and no longer maintained. The author suggests using the guzzlehttp/guzzle package instead.

https://packagist.org/packages/guzzlehttp/guzzle
(also, composer.lock has been updated)

+1

@rscafi Thank you, but guzzlehttp is fo version 5. Unfortunately Guzzle decided to move forward with versions without bringing things like services and commands along, so we are waiting for those to be stable in v6 to make an upgrade.
For now we will remain on v3.

Are you sure? If you check the lock file, it used v3.8.1 of guzzlehttp.
Also, all tags are listed here:
https://packagist.org/packages/guzzlehttp/guzzle#v3.8.1

@rscafi yes, it is possible to use that package, but the other one is not going away, and either way, v3 is deprecated, so there is not much gain either way.
Both packages point to the same repo as it is.


Flask g persisting across requests in test suite

g seems to be persisting longer than it should.  My understanding is that g only exists during a request's context.  It shouldn't be persisted across multiple separate requests.
My auth handling
When you visit the resource a token is used to determine who you are.  Once we get the user_id from that token we persist it like the below code.
if '_auth' not in g:
    g._auth = Auth(user_id=user)
return g._auth

Test Setup
def test(self):
    get = self.client.get('users', headers=make_headers())
    raise Exception(g._auth.user_id) # returns the user id outside the request context

Any ideas as to why g would be available?  Let me know if you need more information.  I wasn't sure how much to include.

It's stored on the app context. During a normal request both contexts have roughly the same lifetime.

@ThiefMaster Ah, so g will persist for the lifetime of that individual test?

Depends on when you push/pop the app context

In my case, it isn't popped until cleanup.  I'll look into that.  Thanks for the help dude!


Gr/feat/bako safe

Changes:

Create Bako safe connector
Add new packages on project: axios, socket.io-client
Add ignore to panpm-lock

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Repository status

Hey @corymsmith and @startupthekid what is the status of this repository? Is it usable? If not, I'd be inclined of making it private since there hasn't been a lot of activity lately and we don't want to cause false hopes ;)

I'd say not usable at this point, @startupthekid had mentioned the other day that he has some thoughts in how to sort some of this out but I agree that maybe we should make it private in the interim

I say go for it, I already have a fork going. I can ping you all when it's ready

Perfect, I was just going to ask. I'll do that then and we'll revisit once there is more to work with.


SingleCellPrediction should instead be labled Perturb-based prediction task for scperturb datasets

*SingleCellPrediction should instead be labled Perturb-based prediction task for scperturb datasets

scperturb contains datasets with very different type of perturbations, e.g. genetic perturbation (which can be knock-out, inhibition, or activation in DNA or RNA level using diverse platforms such as Cas9, Cas12, or Cas13), drug perturbation, and etc. Also, the perturbation can be in higher orders, e.g. inhibition of two genes at the same time. So the "prediction" tasks can go in many directions in my opinion.

It is with this understanding that the loader is kept to this level of detail rather than segmenting into single instance, multi instance etc.
Sent from Proton Mail for iOS
On Fri, Apr 5, 2024 at 6:11 PM, Abolfazl (Abe) @.***(mailto:On Fri, Apr 5, 2024 at 6:11 PM, Abolfazl (Abe) < wrote:

scperturb contains datasets with very different type of perturbations, e.g. genetic perturbation (which can be knock-out, inhibition, or activation in DNA or RNA level using diverse platforms such as Cas9, Cas12, or Cas13), drug perturbation, and etc. Also, the perturbation can be in higher orders, e.g. inhibition of two genes at the same time. So the "prediction" tasks can go in many directions in my opinion.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

It is with this understanding that the loader is kept to this level of detail rather than segmenting into single instance, multi instance etc. This decision was made in consultation with fellow ML researchers at our lab.

Cool. As a user, I strongly agree that ML tasks with perturbation datasets could be seen as a new "problem" rather than being part of current problem definitions.

Feel free to provide a suggestion for a better api as a feature request or create a pull request.

Will do, thanks.

closed with https://github.com/mims-harvard/TDC/pull/252 thanks @kexinhuang12345 !


Graphql schema with a non-model child type (of an auth'd model) gets the error Parameters: [authRoleName] must have values

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
16.6.2
Amplify CLI Version
5.3.0
What operating system are you using?
Mac
Amplify Categories
auth, api
Amplify Commands
push
Describe the bug
When deploying a Graphql API with a model that has a non-model child type and the parent model has an auth rule, an error is returned complaining that [authRoleName] must have values.
I have checked team-provider-info.json, parameters.json and cloudformation-template.json and all files seem to either correctly define or reference authRoleName.
Expected behavior
The API should deploy without error.
Reproduction steps

Define a schema with two types, one using the @model and @auth directives (the parent) and the other without either (the child).
On the parent type, define a field that is the child type (do not create a model-to-model relationship via @connection).
Deploy.

GraphQL schema(s)

type Parent @model @auth(rules: [{ allow: private, operations: [read] }]) {
  name: String
  child: Child
}

type Child {
  name: String
}

Log output

UPDATE_FAILED      api      AWS::CloudFormation::Stack Tue Aug 17 2021 09:36:25 GMT+0100 (British Summer Time) Parameters: [authRoleName] must have values

Additional information
We've noticed this issue only arises when using @auth on the parent model. Our current workaround is to use amplify api gql-compile with the auth rules in place to generate the resolvers correctly, and then migrate the built resolvers into our resolvers folder. Following this, we replace the usage of @auth with @aws_iam  @aws_cognito_user_pools  @aws_api_key.
I believe this gives us auth coverage as AppSync will do the high-level auth checks via the latter directives, with the granular control being on the resolvers.
I also wonder if this issue could stem from the fact that our default authorisation mode is AWS_IAM.

@adam-nygate Are you using imported auth by any chance? If so, this commit should fix the issue.

Thanks for that and yep we are - weird that we haven't had this issue before (until introducing this unusual way of defining the types)


[FEATURE]: [Stax] Use connector_request_reference_id as reference to the connector

:memo: Feature Description

We transmit our reference id from Hyperswitch to the connector, enabling merchants to subsequently locate the payment within the connector dashboard.
Presently, there is a lack of uniformity in this process, as some implementations employ payment_id while others utilize attempt_id.
This inconsistency arises from the fact that payment_id and attempt_id correspond to two distinct entities within our system.

:hammer: Possible Implementation

To mitigate this confusion, we've introduced a new field called connector_request_reference_id in our RouterData.
This id should replace them as a reference id to the connector for payment request.
One might need to have exposure to api docs of the connector for which it is being implemented.
You can check this PR for further reference https://github.com/juspay/hyperswitch/pull/2052

:bookmark: Note : All the changes needed should be contained within hyperswitch/crates/router/src/connector/
:package: Have you spent some time checking if this feature request has been raised before?

[X] I checked and didn't find a similar issue

:package: Have you read the Contributing Guidelines?

[X] I have read the Contributing Guidelines

:sparkles: Are you willing to submit a PR?

Hey @sccalabr, sure! I've assigned this to you.
In case you have any queries, you can ask them on this issue thread, or on our discord server, or on slack whichever you are comfortable with

Hello @sccalabr
Yes, that is a part of solution which in itself is sufficient for the PR👍, but if you are interested, you can go through Stax's api docs to check if a field is present where passing our connector_request_reference_id will help us to retrieve the payment using this id

Hello @sccalabr, it seems like this issue was resolved by the PR #2415, I will close this assign you the issue of your preference. Sorry for the inconvenience


Adding a replication controller for all monitoring pods.

Added a grafana service
Removed auth for Grafana to facilitate usage via service proxy on the api-server.
Removed elasticsearch dependency for monitoring - faster startup times.
Fixes #2408

@lavalamp addressed your comments and I improved the bash script also in the process. PTAL!

A great idea to use replication controllers here and I plan to do the same for the cluster logging setup although I might wait for run to be implemented.

@satnam6502 PTAL


Add comparison links to decide page

It might be preferable to have some links to compare pages on the Decide view.
I think it might actually be best to see if we can move some parts of the ReviewDetailView into a seperate template, and include them on both of those pages.

Fixed in #173


Failed to execute git on project folders which are not git repos

VSCode Version: Version 1.2.1 (1.2.1)
OS Version: OSX 10.11.4 (El Capitain)

My workflow is like this:
Steps to Reproduce:

$ mkdir gitcodetest && cd gitcodetest && code .
Create a new file in the visual studio code window that opens up.
Save it with some content.

Get:

Workaround:
run git init on the repo
By contrast this does not happen if I open a folder that is a git repo.
This popup is a royal pain, why should I have to be using git on the repo? I'm just checking out projects which I'm not actually developing. It's seriously annoying.
Opening the git panel results in an endless loading bar under these circumstances:

Console.log:

I assume you mean without a git repository initialised instead of without git installed.
Plus, from the error it looks like you do have a strange setup: fatal: Invalid gitfile format: .git\n. Are you sure there is no .git folder or file in the cwd or any of its ancestors?

I had a ~/.git file for some unknown reason (it had node_modules as it's sole contents, so a .gitignore gone wrong perhaps). Removing this file resolves the issue. I had not thought at all about ancestor search paths, thank you sir!

Thanks for confirming!


[FP]: Wrongly reporting vulnerability CVE-2022-45378 on geronimo-saaj_1.3_spec-1.1

Package URl
<EMAIL_ADDRESS>CPE
cpe:2.3:a:apache:soap:::::::: versions up to (including) 2.3
CVE
CVE-2022-45378
ODC Integration
{"label"=>"Maven Plugin"}
ODC Version
8.3.1
Description
Per CVE, affected versions:
cpe:2.3:a:apache:soap:::::::: versions up to (including) 2.3
Reported 3PP<EMAIL_ADDRESS>is NOT packing/using the vulnerable 3PP component "Apache Soap", even they are NOT packed as indirect dependency in the environment.   But tool is reporting this vulnerability on geronimo-saaj_1.3_spec-1.1, which is wrong.
From Dependency Check tool team, we need confirmation on these false positives. Could you please validate and confirm?

approved


A performance problem about PredictionIO

When I import data in batch (about 100 million samples) into MyApp, the commands are as follows:
cd MyRecommendation
python data/import_netflixdata.py --access_key $ACCESS_KEY
However,  the speed is very slow, could you please explain why?

it's because it's sending 1 record per HTTP request. The event server is better for recording real time event data.
You could try using batch import tool instead:
https://docs.prediction.io/datacollection/batchimport/

Using pio import is the best way to speed up.
Another option is to use asynchronous method.
http://pythonhosted.org/PredictionIO/predictionio.html#batch-import-data-with-eventclient

I imported 13GB data in batch by "pio import", it took about 50 minutes. was this normal?

This should be taken to the user group
https://groups.google.com/forum/#!forum/predictionio-user
How many executors in total in your Spark cluster and are you using HDFS

one executor and I am using Hbase, not HDFS

One executor generally means one read operator so this does not take advantage of Spark parallelism. Also you must use HDFS to import from multiple executors on multiple machines, HBase uses HDFS so you already have it running. In general horizontal scaling (distributed operations) relies on multiple executors, which can be set to the number of cores on your machine.
How much memory, what template?
Please move this to This should be taken to the user group
https://groups.google.com/forum/#!forum/predictionio-user

OK, Thanks


--ids not working for az backup item set-policy command

This command needs vault_name, container_name and item_name and my plan is to fetch all these 3 from the --ids parameter. I've declared the params like so:
    with self.argument_context('backup item') as c:
        c.argument('vault_name', vault_name_type, id_part='name')
        c.argument('container_name', container_name_type)

    with self.argument_context('backup item show') as c:
        c.argument('name', item_name_type, options_list=['--name', '-n'], help='Name of the backed up item. You can use the backup item list command to get the name of a backed up item.')

    with self.argument_context('backup item set-policy') as c:
        c.argument('vault_name', vault_name_type, id_part='child_name_3')
        c.argument('container_name', container_name_type, id_part='child_name_1')
        c.argument('item_name', item_name_type, options_list=['--name', '-n'], id_part='name', help='Name of the backed up item. You can use the backup item list command to get the name of a backed up item.')
        c.argument('policy_name', policy_name_type, help='Name of the Backup policy. You can use the backup policy list command to get the name of a backup policy.')

When I run the command, I get the following error:
(env) d:\MabOneSdk\cli>az backup item set-policy --ids "/Subscriptions/da364f0f-307b-41c9-9d47-b7413ec45535/resourceGroups/vsarg-sea-RS-1606/providers/Microsoft.RecoveryServices/vaults/vsarg-sea-RS-1606/backupFabrics/Azure/protectionContainers/IaasVMContainer;iaasvmcontainer;canarytestvm1;vsarg-1705-li/protectedItems/VM;iaasvmcontainer;canarytestvm1;vsarg-1705-li" -p DefaultPolicy -debug
az backup item set-policy: error: argument --ids: invalid ResourceId value: '/Subscriptions/da364f0f-307b-41c9-9d47-b7413ec45535/resourceGroups/vsarg-sea-RS-1606/providers/Microsoft.RecoveryServices/vaults/vsarg-sea-RS-1606/backupFabrics/Azure/protectionContainers/IaasVMContainer;iaasvmcontainer;canarytestvm1;vsarg-1705-li/protectedItems/VM;iaasvmcontainer;canarytestvm1;vsarg-1705-li'

Environment summary
Install Method (e.g. pip, interactive script, apt-get, Docker, MSI, edge build) / CLI version (az --version) / OS version / Shell Type (e.g. bash, cmd.exe, Bash on Windows)
Install Method: dev_setup.py
CLI Version: azure-cli (2.0.29)
OS Version: Windows 10 Enterprise
Shell Type: cmd.exe

There are a couple issues here:

It looks like the is_valid_resource_id method (which is now in msrestazure.tools) does not recognize your ID format.  You would want to open an issue there.

However, even if it did you might run into another problem.  You can create a dummy parameter to test this.  Just call parse_resource_id on that ID. If it works, it would likely parse your container name as "IaasVMContainer;iaasvmcontainer;canarytestvm1;vsarg-1705-li" and your item name as "VM;iaasvmcontainer;canarytestvm1;vsarg-1705-li".  However, per issue #4502 the command will not accept the namely formatted name.

@tjprescott : is this issue fixed with merge mentioned above?

Closing this issue since I didn't receive a response. Can be re-opened if the issue still persists.


picking cache is not invalidated when camera.setView was used

Behaviour I'd expect: When changing the camera, the picking cache is invalidated so that the new depth values are used.
Behaviour I currently see: It seems that even after calling setView on the camera, the pickPosition is still getting old depth values.
How to use the Sandcastle:
The blue/green canvas lines mark the center of the screen.
After loading, use Double Click on a point cloud point (e.g., the roof). This will add 3 point entities:

Red marks the point picked from the center of the screen from the current camera position/orientation.
Green marks the point picked from the current mouse position from the current camera position/orientation.
Blue marks the point picked from the center of the screen from a camera positioned at the green point looking towards earth center.

However, the blue point is placed exactly on the same spot as the red point (which I consider a bug).
Now click the button to remove all point entities. Now use Shift + Double Click on a point cloud point (e.g., the roof again). Now the red point is visible at the screen center, because the blue point is placed somewhere on the ground (or within the Earth). Blue should be exactly below green ("dropped a perpendicular").
The difference between DoubleClick and Shift+DoubleClick is that when pressing Shift, the code manually updates the picking (and thus invalidating the picking cache) after the camera is positioned at the green mark.
My initial goal was that the blue point is positioned somewhere in the church (church ground), but I guess it misses a picked point because of low precision, points not being rendered yet, etc. However, having to manually update the picking cache is something that a user shouldn't be required to do; it's not part of the public API.
Sandcastle example:
https://sandcastle.cesium.com/#c=vViNb9o4FP9XfD2dGiQaCCSFfuo6yla0rkOFrjoJqQqJKb4zduQ4dGzq/37PdhJICB3TdFdpGsn7+r3n9+UEnMUSLQl+wQJdIIZfUA/HJFnYX/Q7a3IQ6OceZ9InDIvJQR19nzCEJBYC3gwFX5IQi9NMMBDYl/iRCxqODYtVq0/Ya+1swias0RhywiTqUZ6EaLpCID+z0RCDOnRFKWZ11OM0WUyJjx4YWWIRE7lC93zKJQlidOtPbTQKfMYIe1YKrij+22eh4GgseBBwShA8ok8AgqAe4YEvKMH2hAXaV0kojrEsOmv+a1+PDdHSDiaC5k4NOLvHMU9EgO2Z4IurGNgGoeUcuy1HuYdQpPzSbo3mfgjgTk2cEPKlxCzxJeHsFEmR4Lp5z7hY+DTnnvk0NqTXLF7mYOw4wAzbkSALIiEise2HoZU6YsI6ApcDP5YUK9qYczr1xbtESs4spfFQ4AVfYoSZBBU4PtR2ZgkLFCpkqP2UaNUy5Kn9TMo2fHBMljILQCdsT/vPHEmOCAM9PoVYwaGC4T1hqFhryZ6/wMIfptJlELmenfxGYSGqgWaxQURlvJVaDHEsCUuPbDNRfCHhl8/almFEyHWbTsd17WPXdVyv7bj1jNJqeU3PsV2n2emeOE7Ta+Uk1zvx3FbTbnfcTtd1221DqKUMXBCI+bb5G6xzZUhkML/ncAyZPg/MH3fa7W7Ta7ZPOl4nt3TUtFud4+Nmx/Ha3fZJt3uck47tVtcBXN2O5zabza63CQLyDyFIQxXYt+JfCHxEgn9GMpnNrIDCz/4SvKijhQ/ZT+nqIQqhKIdAAB96fjDH6YGYyoyhc2CWKS8VaBb3Frg8kaUTZEs/tsEiWHskoZyjBlKhfovvBpPnuUwZwWftSYZEuYHDIRbXOAJtF8WMUdQ8BEXQJiHVPzJDVlFNLU2tKhM65VdFl827LCHVn+4weVfJ35KvmI7IN3yKvHqRxBNJoWfrmJyiVjUVui1fd2/9ZD/eDMb9EntQwXffvy5xhST2pxRrr8ZQRNcklj4LANwd9HSI4fDzaDAefOk/De7eD+4G4782FLymv19NHFFllOxoR4ZQ6DlwMoJHWEDgoNovLksa6qbJZurL7U211SqTeZ9Zn+0Pc7rUZp4iw2InWsAqqTRJoYsGjK8d3J146wLLA2J0KnS/pUEJ8QxOOLRKims5RmWXQ9N+8QWzDq9JCENJojmRMENXcg6AD7NgCSwTwdawM1HKn63DGU9g5oJAnPJvUsvWz3Z6vE8dVFXBRg047UJGvlUCbxTAh/t+/67AHOxVJr+Q/q9Z29XRqYrLz2V+SUUh9asSv9Jk1hcbDTVU4YAxMgNTTXP9ZKTyia53L5WoSK1JikVgFPIXBhkWxhsHb7RcVA3jQi/mNEynDrgXg0B68JnB0+3RDC2eM2ylkz2vjjTSIRE42Esy58xEk+iHMkmUjs+CEyoAAD3LaEpJFHMS2o8fRl3XfsY8xLDcjhIx8wN8p9fC6qrZts7ws+onykJd29nJqNVCjZR41yiTaI3xDf82ylfoIVopJHgcp5aSqL5jh6rtAmvEtXqDVGnJ0VaubKWlrZz+GVNhtfperN08MbTFTZo6eXCjolZ3bQ2tn14btleGVm2NsNLGfnvDrs3hrb754/Xhzf757vahv8Uf7Llp/OIWsd4jNjYJVB26X1kmWqVtYp99olVYKBAG8fIo1vO0nEsk1oM5neeH+faw0ZvTjqpg710kxeaaB+PniqWkJKdsFVCJsbqetFN54agqGUXQFPWmcwPDhar36j4SJfJK27Hy6wdn1zyBvOmp2rfyOapl4ZJoPKi+o+Qn+VpH+/5NWJouoxLO8SrC9m3//fjp+vPDu9v+U+920PtYuND/b46p7w3/rV/1rGo+4tWUw5jXvJ94SGYEfBrdDN6PFYKD+sHkIL8gmaM/j+WK4sssD/4ki4gLqT69WLbdkHgRUZhucWOaQD1IO4jjrNrOG5ui5yFZIhJeVHyuglD5cQyUWUJ1r5scZEJzirQSIH47IizEX6EPnm2sFv4USjKR+Ay9mCboNJt/nMGFVjwTIDezn0cUz6C/Am3Khf4YxmBmniHzdDTlUvIFSEdwy+UUtuzfnaDjTYOcQ/Kokqzfe6AXQJ835jRDvtwf+VzfdHdD1yZ2ITd+FYBNAVpOF6nubQYjmUNf5tAbcFRbp0a5/rDxeYkF9VdKYu5c3pqXtm2D685ltaQ035q0kTX9Xw
Browser: Firefox, Chrome
Operating System: Windows

Thanks for the detailed write-up @anne-gropler!
@lilleyse Can you confirm this behavior is unexpected? And is it related to any other recently reported pickPosition issues, such as  https://github.com/CesiumGS/cesium/issues/9846?

I forgot to mention that we use Cesium 1.92, I hope this helps tracking down the bug, in case you confirm that this is unexpected behaviour.

This is expected behavior, at least with how pickPosition currently works.
pickPosition gets the position from the depth buffer of the last rendered frame, so you'd need to manually render the scene if you want to get a new value.
const firstPosition = scene.pickPosition(screenPosition);
camera.setView(...); // set camera to new position
scene.render();
const secondPosition = scene.pickPosition(screenPosition);
camera.setView(...); // set camera back

The danger of doing this with 3D Tiles is that not all the tiles will be loaded during the second render and you'll get different results depending on where the camera is and what tiles are currently loaded.
pickFromRayMostDetailed may be helpful here? Though it's also private API.

I guess that pickFromRayMostDetailed won't be public soon. But we really rely on it currently, so I just hope it will make it to public API.
I'll close this issue because the original issue was answered as "expected behaviour", and my underlying problem is fixed using pickFromRayMostDetailed  👍


Support for quic 46

1. What would you like to have changed?
Yes... quic seems to have updated again to version 46...
Latest chrome don't love quic version 44 anymore and is not using it...
So can caddy add support for quic version 46?
2. Why is this feature a useful, necessary, and/or important addition to this project?
Since support for quic is an really attractive feature of caddy as an web server, I think it would be great and important to continue supporting & maintaining quic.
3. Please link to any relevant issues, pull requests, or other discussions.
Since quic standard is quickly evolving and git repository of quic's rfc is updating every half day , I would argue maybe it's better to wait for the rfc to finalize and then implement it afterwords, then chrome maybe willing to communicate with host even if it's one / two version behind.
(Sorry for my bad english orz

Which version are you testing with?
Chrome Canary (75.0.3732.0) still speaks QUIC 43.

Chrome 73.0.3683.75 (Official Build) (64-bit)

Then you're probably on an experiment. We won't support that.

fine.......................

sorry...........


webapp:Unable to set WebApp Slot logging using az webapp log config --ids [webapp slot id]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
The command "az webapp log config" when used with the parameter "--ids [appServiceSlotID]" configure the main AppService instead of the specified AppService Slot.
To Reproduce

Create a Resource Group, say "resourceGroupName".
Create an AppService in the newly created Resource Group, say "appServiceName".
Add an AppService Slot to the newly created AppService, say "appServiceSlotName".
Run the following command:

az webapp log config \
--docker-container-logging "filesystem" \
--ids "/subscriptions/[subscriptionID]/resourceGroups/[resourceGroupName]/providers/Microsoft.Web/sites/[appServiceName]/slots/[appServiceSlotName]"

Expected behavior
The specified AppService Slot/AppService Logs is enabled with the following default parameters:

Application Logging: filesystem
Quota (MB): 100
Retention Period (Days): 3

Observed behavior
The AppService/AppService Logs is enabled with the following default parameters:

Application Logging: filesystem
Quota (MB): 100
Retention Period (Days): 3

Environment summary
Install Method: apt
CLI version: 2.27.
OS version : Ubuntu 20.04.2 LTS
Shell Type: bash
Additional context
The following equivalent command works as expected:
az webapp log config \
--docker-container-logging "filesystem" \
--resouce-group [resourceGroupName] \
--name [appServiceName] \
--slot [appServiceSlotName]

route to service team

I am unable to repro the issue. please share the --debug log to analysis further

I think I misunderstood the use of the --ids parameters. I guess since the az webapp log config command is at the webapp level, the --ids parameter ignores the slot name when specified. Thus we have to explicitly specify the slot name using the --slot parameter.
RIGHT:
az webapp log config \
    --docker-container-logging "filesystem" \
    --ids "/subscriptions/${subscription}/resourceGroups/${resource_group}/providers/Microsoft.Web/sites/${app_service_name}" \
    --slot "${app_service_slot_name}"

WRONG:
az webapp log config \
    --docker-container-logging "filesystem" \
    --ids "/subscriptions/${subscription}/resourceGroups/${resource_group}/providers/Microsoft.Web/sites/${app_service_name}/slots/[app_service_slot_name}"

Given the above, it would be nice if the az webapp log config command - and those alike - could give a warning in such circumstances.

This is the problem with all the commands when using --ids parameter  for slot. We will look in to it.
@panchagnula - FYI.

--ids argument will work only for the webapp. to work with slots we must pass --slot argument. This is the common behavior across all the cli  commands.

As mentionned on Sept 10th, I now understand that. My point is that - based on the command documentation on the [--ids] parameter - "It should be a complete resource ID containing all information of 'Resource Id' arguments. You should provide either --ids or other 'Resource Id' arguments." make me (and probably some other people) believe that we can either provide a full resource ID (an App Service Slot has a Resource ID) or the other resource Id arguments which are namely the [--subscription], the [--resource-group], the App Service [--name], and the [--slot] name. In other words, we should not mix [--Ids] and [--slot] parameters.
A meaningfull warning or error message would have been appreciated. That's all.

@PatrickMessierGoC Agree with your suggestion. But this is not the problem with only webapp this is the same behavior with all the other azure resources too. So we will take it into CLI core team to handle this at root level instead of individual module level.


quick fix page size override

A specificity conflict was forcing pages to the letter size, from this commit: Add default page size to brewrenderer.less
This PR lowers specificity for that selector using :where.

Thanks @5e-Cleric !


Godot 4 beta 17 update

This PR makes the project work in Godot 4 Beta 17.
I made the following changes:

Inputs were broken so I re-set them. I don't have a copy of G3 so I best-guessed it.
The character's AnimationNodeStateMachineTransition's needed their advance_modes set to Auto in various places. Also I connected the Start node up to Idle

Since we have a nice example controller here I'd love to see it hooked up to a state machine for cleaner code.

Rather than accepting this PR, it might be best to createa  godot-4 branch and let me PR to that instead. That way you'll have a working project for people using both versions.

This PR makes the project work in Godot 4 Beta 17.
I made the following changes:
* Inputs were broken so I re-set them. I don't have a copy of G3 so I best-guessed it.

* The character's `AnimationNodeStateMachineTransition`'s needed their `advance_mode`s set to _Auto_ in various places. Also I connected the _Start_ node up to _Idle_

Since we have a nice example controller here I'd love to see it hooked up to a state machine for cleaner code.

As you said, i'm hesitant to merge this request. I'd rather leave this repo focused on just Godot 3. A lot of things broke when i first moved it to a while back which is why i made the alternate Godot 4 controller repo at that time:
https://github.com/pemguin005/Third-Person-Controller--SoulsLIke-Godot4
Have you already tried that one out? I'd rather Godot 4 code stick with that repo. I've let these projects sit for a while once the beta releases started coming so fast. So i appreciate the assistance maintaining the code during their rapid development.

Ahh, had no idea there was already a G4 version. My bad!


Issues with using a config file specified part without command line part argument.

When I run the same command using --dry-run it executes without the part positional argument added.
Also running with --verbose option it works fine.
Sample command
bumpversion --allow-dirty  --list --config-file /path/to/.bumpversion-local.cfg patch /path/to/setup.py
My sample config file
[bumpversion]
current_version = 1.0.21
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)
serialize = {major}.{minor}.{patch}
commit = True
tag = False
message = "{now:%Y-%m-%d %H:%M} - Increased version: {current_version} to {new_version}"

[bumpversion:part:patch]

[bumpversion:file:/path/to/setup.py]
search = version="{current_version}"

Raises
Traceback (most recent call last):
  File "/usr/local/bin/bumpversion", line 11, in <module>
    sys.exit(main())
  File "/Library/Python/2.7/site-packages/bumpversion/__init__.py", line 912, in main
    f.should_contain_version(current_version, context)
  File "/Library/Python/2.7/site-packages/bumpversion/__init__.py", line 202, in should_contain_version
    if self.contains(serialized_version):
  File "/Library/Python/2.7/site-packages/bumpversion/__init__.py", line 214, in contains
    with io.open(self.path, 'rb') as f:
IOError: [Errno 2] No such file or directory: 'patch'

I ran into this problem too, and found that -n and --dry-run are not equivalent, even though the usage message says they should be:
  --dry-run, -n         Don't write any files, just pretend. (default: False)

With -n I get the error your report.  With --dry-run it works as advertised.   @jackton1 - Which are you using?


Feat/optimize the query session

Codecov Report

Merging #387 (93c4d65) into main (25bcae8) will decrease coverage by 0.10%.
The diff coverage is 62.39%.

:exclamation: Current head 93c4d65 differs from pull request most recent head 136b390. Consider uploading reports for the commit 136b390 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main     #387      +/-   ##
==========================================
- Coverage   62.65%   62.56%   -0.10%
==========================================
  Files          67       68       +1
  Lines        9516     9848     +332
==========================================
+ Hits         5962     6161     +199
- Misses       3554     3687     +133

Flag
Coverage Δ

rust
62.56% <62.39%> (-0.10%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/error/src/lib.rs
100.00% <ø> (ø)

src/node/src/abci_impl.rs
0.50% <ø> (+<0.01%)
:arrow_up:

src/node/src/auth_storage.rs
0.66% <0.00%> (-0.02%)
:arrow_down:

src/node/src/storage_node_impl.rs
0.37% <0.00%> (-0.02%)
:arrow_down:

src/storage/src/lib.rs
100.00% <ø> (ø)

src/cmd/src/command.rs
49.51% <11.86%> (-2.04%)
:arrow_down:

src/sdk/src/store_sdk.rs
76.94% <60.00%> (-1.22%)
:arrow_down:

src/storage/src/db_store.rs
90.29% <71.94%> (-2.76%)
:arrow_down:

src/storage/src/db_owner_key.rs
92.07% <92.07%> (ø)

src/crypto/src/db3_signer.rs
98.38% <100.00%> (+0.01%)
:arrow_up:

... and 2 more

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


nltk.corpus.ptb.fileids() is empty？

Why when I run  nltk.corpus.ptb.fileids() in data_ptb.py, I got an empty list of fileids.
nltk.corpus.ptb.fileids()
[]

You need to get the Penn Treebank dataset

You need to get the Penn Treebank dataset

I've downloaded ptb corpus by nltk.download('ptb'), what else I need to do?

You need to get the Penn Treebank dataset

I've downloaded ptb by nltk.download('ptb'), what else I need to do ?

Sorry. Due to the copyright policy of PTB, I can't directly send you the data.

I'm sorry, do I have to pay for this data set？and how do you get it?

You need to register at LDC. Then you can check whether you need to pay or not.

ok，thank you

@cuijiekun You could also get a subset of the data through NLTK: http://www.nltk.org/nltk_data/

@tyj1997 Hello, I can't get data either, Have you resolved this issue?


Stream and control multiple games?

In its current form, can GamingAnywhere be used to stream and control multiple games running simultaneously on one system?

It depends. The current GA is able to run and control multiple games in one
system if the games are:

writing in SDL, and
Use GA's hook mode to interact with the game.

Chun-Ying
On Wed, Mar 9, 2016 at 10:22 PM, SMUsamaShah<EMAIL_ADDRESS>wrote:

In its current form, can GamingAnywhere be used to stream and control
multiple games running simultaneously on one system?
—
Reply to this email directly or view it on GitHub
https://github.com/chunying/gaminganywhere/issues/23.

May I ask why does the multi client feature is restricted for SDL games only?
Is it technically possible and what's the issues we can face to make it work on any other type of games ?

You have to simulate inputs correctly for other types of games. On windows input can be pretty much perfectly simulated with SendInput but that will work only for currently active game process/window.
Some other generic technique which would allow you to hook input at lower level for current process only would simply solve the issue.
That is probably the only major issue with current GA.

Yes, you are right.
Currently we have complete hook implementations (control, audio, and video)
for SDL games. For non-SDL games, now we only have hook implementations for
audio and video, but not control.
Thanks,
Chun-Ying
On Thu, Mar 10, 2016 at 9:34 PM, SMUsamaShah<EMAIL_ADDRESS>wrote:

You have to simulate inputs correctly for other types of games. On windows
input can be pretty much perfectly simulated with SendInput but that will
work only for currently active game process/window.
Some other generic technique which would allow you to hook input at lower
level for current process only would simply solve the issue.
That is probably the only major issue with current GA.
—
Reply to this email directly or view it on GitHub
https://github.com/chunying/gaminganywhere/issues/23#issuecomment-194840034
.

Another method is to use Multiple Sessions on Windows Server. Windows Server support multiple concurrent users. So to stream and control multiple games, each game will executed on different windows login. Each user will be able to use the OS independently so SendInput only should work fine for any number of games running in full screen.
However to keep multiple sessions active concurrently, Remote Desktop Licensing setup is required.


WPTO Hindcast

Adding new WPTO hindcast function to MHKiTMatlab

@ssolson this is ready for review, even though the CI has failed. It fails because it uses pip to install MHKiT-Python, and since the WPTO function is not tagged in a pip release- it cannot run the function. That being said, we will not merge this PR until MHKiT-Python has been tagged with pip, but it would be good to get it reviewed so we can merge it quickly after the pip tag.
I will look into if we can change how the Matlab CI gets MHKiT-Python so the tests will work in the future.

This was re-done in https://github.com/MHKiT-Software/MHKiT-MATLAB/pull/61 allowing for Windows utilization


operator instana-agent-operator (v2.0.32)

/pipeline restart operator-release-pipeline

/pipeline restart operator-release-pipeline

/pipeline restart operator-release-pipeline

/pipeline restart operator-release-pipeline


DOC : UPDATION README.md

Describe the bug
README.md file does not contain any featured in section
No repository stats table
no forkers or stargazers section
No instructions for contribution or link to code of conduct
No good website preview
Expected behavior
Displaying repository stats (forkers, stargazers) to showcase popularity.
Providing contribution guidelines, encouraging community involvement.
Adding a Code of Conduct, fostering a respectful environment.
Improving the website preview, attracting more users.
These improvements boost professionalism and engagement.
Screenshots

Additional context
A visual representation can also be added by the owner of the repository using repobeats analytics
@ANSHIKA-26  kindly assign this issue to me.

Kindly assign this issue to me.


Lazy loading implemented

Implemented Lazy loading for all the page

Closes #68
Proposed Changes
Implemented Lazy loading for all the pages in this project due to which whenever the website loads initially then only the pages that are visible to the user are loaded instead of the whole website. This reduces the initial loading time making the user experience better.
Check List (Check all the applicable boxes)

[ ] Bug(s) fixed
[ ] Enhancement done
[x] New Feature(s) added
[ ] Documentation changed
[x] All new and existing tests passed.
[x] My code follows the code style of this project.

@LakshaySK106 Any update on this please?

@Avinash905  This fails in this check ! pls go through it


Inertia bounce interferes with camera flights in 2D

Start Cesium Viewer
Switch to 2D
Zoom out with the wheel to get a good camera "bounce" at max distance
Quickly enter a location in the Geocoder and fly there

The flight will happen but then you will get sent back to where you started.  I can reproduce this easy enough, but it is timing dependent, so if you have problems let me know.
CC @bagnell

I took a quick look at this and while the bug manifests differently, the end result is that bounce inertia will end up interfering with camera flights.  I'm not sure there's an easy to fix this because the SceenSpaceCameraControlleris what knows about inertia while the Camera itself and the flights know nothing.  We should probably take a hard look at camera architecture for Cesium 2.0

There's no bounce back animation so I can't reproduce the original issue. Does it happen with inertia as well?

@mramato I can't reproduce this.  Can this be closed?

Looks like we got rid of the inertial bouncing at some point (probably when we added infinite 2D) either way I can no longer reproduce this either.


sumologic: document fluentdUserConfig in readme

The fluentdUserConfig param is documented with a comment in values.yml. I copied that comment into the readme table.

/assign @scottrigby

@scottrigby Any chance you can approve this?


Add [ACCORDION], [DONE] and [VALIDATE] tags

Why did you revert the syntax to v1 parser?

One of the validation question is not showing up so I looked at https://developers-qa.sap.com/tutorials/docs-tutorial-3-adding-validation.html how to fix it. This only works without specifying a parser version. I'll go back to v2 and investigate further what the issue might be


export/import_nir__Adding-Conv2D_and_AvgPool2d

fixes #304
Adding support for the following torch.nn modules [Conv2d, AvgPool2d and Flatten] to export_nir.py and import_nir.py
Create example notebook in examples/tutorial_snntorch_to_nir.ipynb for snntorch model with layers [Conv2d, AvgPool2d,LIF, Linear,LIF] and export/import it to NIR
Include this model's tests in tests/test_nir.py

Yep, this would be really practical to have (pls review) :)

There are a couple of conflicts in snntorch/functional/loss.py.

Line 100: arguments are different; population coding is missing.
Line 114: weights are now an instance variable; I'm not totally sure if these are inherited properly.

Conflicts in snntorch/functional/loss.py where resolved by merging the changes from master branch to Adding-Conv2D_and_Pooling branch because the original changes for export/import_nir__Adding-Conv2D_and_AvgPool2d were not not intended to affect the loss function logic.

The tests haven't passed as there is now a dependency on sinabs.layers.
I don't think we should add dependencies on external libraries to enable NIR exports to function correctly.
How easy would it be to remove that requirement?

The sinabs.layers dependencies for sumpooling layers were easily removed, as this feature was used for a custom model conversion to NIR and indeed shouldn't be added to the snntorch NIR functions.

Thank you! I ran the tests and it oddly worked with 3.9/3.10, but it didn't work on Python 3.8.
One of the tests throws module 'nir' has no attribute 'AvgPool2d'.
I'm not totally sure why this wouldn't be compatible with 3.8...
@stevenabreu7 : have you observed anything like this among the other libraries?
I would rather not eliminate compatibility with 3.8 over this, but a non-ideal option is to modify the test if we're not sure why this error persists.

I  checked the NIR repo and in the pyproject.toml file it indicates in line 35 requires-python = ">=3.9" and I think that's why the tests in snntorch for python 3.8 fails, so I guess if it is the case, then the snntorch compatibility with 3.8 should be eliminated.

thanks for checking that. Yep, let's remove Python 3.8 from the test matrix. I tried to commit to your fork but I don't have the necessary permissions. All that needs to be changed are the following files:

.github/workflows/build-tag.yml
CONTRIBUTING.rst
setup.py

If you update those, the tests should pass and I'll merge the PR.

I removed python 3.8 dependencies from the appointed files.


Simplify BloomFilterBuilder.filter_bytes

It looks like the only call-site only provides values from self that the function filter_size_bytes itself has access to.

Thanks!


Ensure operator version labels

Towards https://github.com/giantswarm/giantswarm/issues/14156
This PR ensures the azure-operator and cluster-operator labels are set on create in the MachinePool and AzureMachinePool CRs.
This removes the need to know the release version on kubectl-gs when creating a node pool.

Doesn't Cluster contain the components' labels? I was wondering if it would be easier to just read the versions from the labels in the Cluster CR like you do to find out the Release version, instead of fetching the Release.

Why cluster-operator version is added to CAPI or CAPZ CRs? The version we can use doesn't know anything about those types. It only reconciles AzureClusterConfig CR. I think it shouldn't be there. It doesn't do anything effectively, but creates noise in the logs.

Doesn't Cluster contain the components' labels? I was wondering if it would be easier to just read the versions from the labels in the Cluster CR like you do to find out the Release version, instead of fetching the Release.

And fail if they're not there?
Might be a good idea.

Why cluster-operator version is added to CAPI or CAPZ CRs? The version we can use doesn't know anything about those types. It only reconciles AzureClusterConfig CR. I think it shouldn't be there. It doesn't do anything effectively, but creates noise in the logs.

Can't answer for sure.
kubectl-gs was doing that and I mimicked the same exact behaviour

@fiunchinho pushed the change to meet your suggestion


Increase performance of many DuplicateRow calls

Description
File.DuplicateRow is a useful method for keep format and styles, but it can be worst if you need to work/duplicate more than 1k rows.

Describe the results you expected:
Maybe it's time to create a File.BunchRow(sheetName string, row int, n int) method. This new method can be create a single deepCopy of row and apply it for n rows inserted!
Remember: File.InserRow (adjustHelper internaly) is absurd faster than that.

Sorry for my late reply. If you want to set the styles for multiple rows to keep format and styles, using the SetRowStyle function will get better performance.


Story: Inline edit of params from experiments webview

We want to be able to edit params values inline within the experiments webview.
🐉s:

Parameters files can be TOML, JSON, YAML or Python files: https://dvc.org/doc/user-guide/glossary#parameter-dependency.
Data type of each parameter is currently unknown.

Note: Studio team has already been through a couple of iterations of this. Look at what they have done.
From initial designs:

Closing placeholder and we have implemented a very basic solution. Will come back to it when we are ready to implement.


Transparency in the Interoperability of the National Electronic Toll(ETC) network

Checklist for Hyperledger Lab Proposal PR

[ ] Read README.md carefully
[ ] Make sure your changes are committed with the proper sign-off
[ ] Use the labs name as the PR title
[ ] Copy the short description of the proposal as the description of the PR
[ ] Remove this checklist section
[ ] Submit your PR and the labs stewards will review your proposal
[ ] Ask your sponsor to confirm sponsorship by stating so in a comment to the PR

Please restore the proposal-template.md file. This file should not be deleted.

Hi, @nikoturin - Thank you for the PR and suggested lab. It looks like there is currently a DCO issue with this PR. Can you follow the steps listed under DCO details to fix?
I am not sure that I understand what specifically you are using from the Hyperledger projects. Can you help us understand how this relates to the work that is being done at Hyperledger other than a blockchain application?

Please restore the proposal-template.md file. This file should not be deleted.

@tkuhrt Do you have a template you copy/paste for this one? ;-)

need requested changes in place to approve.

Hi @nikoturin, if you would like to move forward with this lab, please re-propose it after addressing the feedback from Lab Stewards.


Conflict: Constraint<EMAIL_ADDRESS>is not satisfied by vulcan:lib 1.4.0

I am currently on 1.8.6 and I am trying to deploy to Galaxy.
I am getting this error:
While selecting package versions:
error: Conflict: Constraint<EMAIL_ADDRESS>is not satisfied by vulcan:lib 1.4.0.
Constraints on package "vulcan:lib":
*<EMAIL_ADDRESS><- vulcan:core 1.8.6
*<EMAIL_ADDRESS><- vulcan:users 1.8.6 <- vulcan:core 1.8.6
*<EMAIL_ADDRESS><- vulcan:routing 1.8.6 <- vulcan:core 1.8.6
*<EMAIL_ADDRESS><- vulcan:i18n 1.8.6 <- vulcan:core 1.8.6
*<EMAIL_ADDRESS><- vulcan:debug 1.8.6 <- vulcan:core 1.8.6

I deleted .versions but can't deploy. I am not sure how to solve this dependency issue.
To add to this list. I don't think it is necessary to include vulcan:debug in vulcan:core. I believe it is better to include debug in my .meteor/packages when I need it.

Hello!  Sacha, MHerszak.
I experienced same trouble.
Please give me some advices.
(I'm using latest version Vulcan.)

@nisshii0313 until all packages are updated, you need to update all packages to Meteor 1.6.1 and run them in your project. As soon as you use the currently published packages<EMAIL_ADDRESS>you will get a mix of versions of Meteor and this seems to be a problem for Meteor. This isn't a very in-depth explanation but it helped in my case. I hope that helps.

I understand. Could you tell me how this is done specifically?

When I try to update my all packages, the error massages interrupt me.
How did you resolve this problem ?

One approach is to delete your .meteor/versions file and start over.

I deleted my ".meteor/versions" file but new error message appear.

Do you have any ideas how to solve this error ?

please let me know another way.

Ploblems are resolved. thanks !

This worked for me:

delete ./.meteor/local folder
remove all pined versions in your projects package.js
METEOR_PACKAGE_DIRS="../vulcan-base/packages" meteor update --all-packages

Hello!
Anyone to help me with the solution or how to fix this...
`=> Started proxy.
=> Errors prevented startup:
While selecting package versions:
error: Conflict: Constraint<EMAIL_ADDRESS>is not
satisfied by vulcan:lib 1.3.2.
Constraints on package "vulcan:lib":

<EMAIL_ADDRESS><- vulcan:core 1.16.0
<EMAIL_ADDRESS><- vulcan:users 1.16.0 <- vulcan:core
1.16.0
<EMAIL_ADDRESS><- vulcan:i18n 1.16.0 <- vulcan:core
1.16.0
<EMAIL_ADDRESS><- vulcan:ui-bootstrap 1.10.1 <-
sidebar2020 0.0.0

Conflict: Constraint<EMAIL_ADDRESS>is not satisfied by email
2.0.0.
Constraints on package "email":

<EMAIL_ADDRESS><- top level
<EMAIL_ADDRESS><- vulcan:lib 1.3.2 <- vulcan:core 1.16.0
<EMAIL_ADDRESS><- vulcan:accounts 1.3.2 <- sidebar2020 0.0.0

Conflict: Constraint<EMAIL_ADDRESS>is not satisfied by
vulcan:lib 1.3.2.
Constraints on package "vulcan:lib":

<EMAIL_ADDRESS><- vulcan:core 1.16.0
<EMAIL_ADDRESS><- vulcan:users 1.16.0 <- vulcan:core
1.16.0
<EMAIL_ADDRESS><- vulcan:i18n 1.16.0 <- vulcan:core
1.16.0
<EMAIL_ADDRESS><- vulcan:ui-bootstrap 1.10.1 <-
sidebar2020 0.0.0

Conflict: Constraint<EMAIL_ADDRESS>is not satisfied by email
2.0.0.
Constraints on package "email":

<EMAIL_ADDRESS><- top level
<EMAIL_ADDRESS><- vulcan:lib 1.3.2 <- vulcan:core 1.16.0
<EMAIL_ADDRESS><- vulcan:accounts 1.3.2 <- sidebar2020 0.0.0

=> Your application has errors. Waiting for file change.`
Please I need help.


Contentful mods

Added a contentful template along the lines of what was done for airtable.
accidentally deleted airtable's readme.md, so when merging we have to make sure that we do not accept that.

We should run the formatting script on this to be consistent. It's in https://github.com/steprz/experimental/tree/master/darren/schema-formatter ... happy to help w this if you want, just say the word.


Failed to open a zarr file

Dear All,
I am trying to visualize ome.zarr file in napari on Windows 10.
I converted a dataset to zarr from the https://camelyon17.grand-challenge.org/Data/
I downloaded it using the following script
import argparse
import ftplib
import os

def download_camelyon16_image(filename):
    filename = filename.lower()
    if os.path.exists(filename):
        print(f"The image [{filename}] already exist locally.")
    else:
        print(f"Downloading '{filename}'...")
        prefix = filename.split("_")[0].lower()
        if prefix == "test":
            folder_name = "testing/images"
        elif prefix in ["normal", "tumor"]:
            folder_name = f"training/{prefix}"
        else:
            raise ValueError(
                f"'{filename}' not found on the server."
                " File name should be like 'test_001.tif', 'tumor_001.tif', or 'normal_001.tif'"
            )
        path = f"gigadb/pub/10.5524/100001_101000/100439/CAMELYON16/{folder_name}/"
        ftp = ftplib.FTP("parrot.genomics.cn")
        ftp.login("anonymous", "")
        ftp.cwd(path)
        ftp.retrbinary("RETR " + filename, open(filename, "wb").write)
        ftp.quit()

if __name__ == "__main__":
    arg_parser = argparse.ArgumentParser()
    arg_parser.add_argument(
        "filename",
        type=str,
        help="The image name in Camelyon16 dataset to be downloaded.",
    )
    args = arg_parser.parse_args()
    download_camelyon16_image(args.filename)

I then converted it into zarr using the following sccript:
import numpy as np
import zarr
import os
#Just to load the openslide dll
os.add_dll_directory('openslide-win64-20171122/bin/')
from openslide import OpenSlide

#Require pip install openslide-python
slide = OpenSlide('data/tumor_001.tif')
file_name = 'data/tumor_001.zarr'

root = zarr.open_group(file_name, mode='a')

for i in range(0, 3):
    print(i, 10)
    shape = (slide.level_dimensions[i][0], slide.level_dimensions[i][1], 4)
    z1 = root.create_dataset(str(i), shape=shape, chunks=(300, 300, None),
                             dtype='uint8')

    # image = np.asarray(slide.read_region((0, 0), i,
    #                    slide.level_dimensions[i])).transpose(1, 0, 2)
    # z1[:] = image

    for j in range(slide.level_dimensions[i][0]//1528):
        print(j, slide.level_dimensions[i][0]/1528)
        image = np.asarray(slide.read_region((j*1528*(2**i), 0), i,
                           (1528, slide.level_dimensions[i][1]))).transpose(1, 0, 2)
        z1[j*1528:(j+1)*1528] = image

I installed the plugin for napari ( pip install ome-zarr )and tried to open it but get the following error:
napari "data\tumor_001.zarr"
10:46:57 ERROR exception calling callback for <Future at 0x2173e8e27c0 state=finished returned ChunkRequest>
Traceback (most recent call last):
  File "c:\gbw_myprograms\anaconda3\envs\napari-env2\lib\concurrent\futures\_base.py", line 328, in _invoke_callbacks
    callback(self)
  File "c:\gbw_myprograms\anaconda3\envs\napari-env2\lib\site-packages\napari\components\experimental\chunk\_pool.py", line 162, in _on_done
    self._on_done_loader(request)
  File "c:\gbw_myprograms\anaconda3\envs\napari-env2\lib\site-packages\napari\components\experimental\chunk\_loader.py", line 225, in _on_done
    self.cache.add_chunks(request)
  File "c:\gbw_myprograms\anaconda3\envs\napari-env2\lib\site-packages\napari\components\experimental\chunk\_cache.py", line 108, in add_chunks
    self.chunks[request.location] = request.chunks
  File "c:\gbw_myprograms\anaconda3\envs\napari-env2\lib\site-packages\napari\_vendor\experimental\cachetools\cachetools\lru.py", line 19, in __setitem__
    cache_setitem(self, key, value)
  File "c:\gbw_myprograms\anaconda3\envs\napari-env2\lib\site-packages\napari\_vendor\experimental\cachetools\cachetools\cache.py", line 47, in __setitem__
    raise ValueError('value too large')

Any tips will be appreciated
Benjamin

Hi Benjamin. Your conversion has produced a Zarr file but not an OME-Zarr image. You can see a similar thread under https://forum.image.sc/t/manually-creating-ome-zarr-from-a-folder-of-tiffs-pngs/59806/2
Support for opening raw Zarr files is provided directly from the napari team. Alternatively, we can help you to format the Zarr according to https://ngff.openmicroscopy.org/ in which case this would be the right repository.


Update to Apache 2.0 license

Per https://github.com/uber/jaeger/issues/252

Coverage remained the same at 85.689% when pulling 7aae636fea35e7fb3476a08b6ac7de802de70b16 on apache-license into aa5016eb72fdc6904f35847fd40379b34b66a256 on master.


read comments of a post of a user

how to real latest  comments of latest post of a user

are you able to do that on app?


CommandLineUtils: Match options using case-insensitive comparison

Currently, CommandLineUtils uses a case-insensitive comparison when matching commands:
https://github.com/aspnet/Common/blob/dev/shared/Microsoft.Extensions.CommandLineUtils.Sources/CommandLine/CommandLineApplication.cs#L367
But case-sensitive comparison when matching options:
https://github.com/aspnet/Common/blob/dev/shared/Microsoft.Extensions.CommandLineUtils.Sources/CommandLine/CommandLineApplication.cs#L152
I think these should both use case-insensitive.

I kind of disagree.
While case-insensitivity is common for long options it is a stopper for short options.
I know of quite a few applications out there where "-v" stands for "--verbose" while "-V" stands for "--version".
So my suggestion would be the following:
Support case-insensitivity independently for short options and long options and make it default for long options.

I agree @Brar's point. Also, this a potentially a breaking change for apps on 1.x. I suggest we make case insensitive comparison an option, but not the default.

Agree with short options, but I would make case-insensitive the default for long options.  Breaking changes are OK in 2.0.

We have a few command line things I think we need to discuss for triage... I'll send an email on this (incl. you @mikeharder and @natemcmaster ).

When discussing command line things, please consider that compatibility of command line interfaces has a pretty high value in the Unix world. Some applications are written as drop in replacements for other (older) ones but maintain the same options and arguments. (cc -> gcc; lex -> flex, yacc ->bison, ...).
While it would always be possible to wrap new applications with a batch or a shell script, a high level of flexibility in the CommandLineUtils would be helpful for writing command line compatible replacement-applications.

Some things that might be useful:
http://perldoc.perl.org/Getopt/Long.html#Command-Line-Options%2C-an-Introduction
http://perldoc.perl.org/Getopt/Long.html#Case-and-abbreviations

Discussed in triage; there is no pressing need to change this, so closing it.

 Clearly there are both sides to this coin, so why not make it a developer option? I can't believe this was dismissed so rapidly.

@brobichaud adding features has cost, and we felt that it wasn't worth the cost, given that this library has limited usage. One of the engineers on the team forked the project at https://github.com/natemcmaster/CommandLineUtils and has added various features, so perhaps this option is available there.

@brobichaud unfortunately cost is quite often the main reason. We'd love to add lots of features to lots of areas but we have to prioritize our efforts. Even features/fixes contributed by community members have a non-zero cost (the code has to be reviewed, maintained, etc.).
In this specific case the original issue was an "internal find" (as in, not customer complaint), and it remained a fairly quiet issue for over a year.

it remained a fairly quiet issue for over a year

I think that this is rather because you've internalized the whole api than because of lack of interest.
I'm glad @natemcmaster forked the project and maintains a public command line argument api that Microsoft can only maintain internally (separately in several different projects with slight modifications)

FYI:  the API you're looking for is  McMaster.Extensions.CommandLineUtils.CommandLineApplication.OptionsComparison


Everything worked great until today : telnet on port 25 times out

I don't know what happened. But yesterday postal worked great on my VPS. Today it doesn't.
$ telnet mysmtpserver.com 25
Trying <IP>
telnet: connect to address <IP>: Operation timed out
telnet: Unable to connect to remote host

Here is postal status:
|| web
|| Quantity            1
|| Command             bundle exec puma -C config/puma.rb
|| Respawning          5 every 3600 seconds
|| Restart mode        usr1
|| Log path            none specified
|| Address/Port        none
|| => web.1            Running      10:38        pid:1883     respawns:0      port:-        tag:-

|| fast
|| Quantity            1
|| Command             bundle exec rake postal:fast_server
|| Respawning          5 every 3600 seconds
|| Restart mode        term-start
|| Log path            none specified
|| Address/Port        none
|| => fast.1           Running      10:38        pid:1886     respawns:0      port:-        tag:-

|| worker
|| Quantity            1
|| Command             bundle exec ruby script/worker.rb
|| Respawning          5 every 3600 seconds
|| Restart mode        start-term
|| Log path            none specified
|| Address/Port        none
|| => worker.1         Running      10:38        pid:1888     respawns:0      port:-        tag:-

|| cron
|| Quantity            1
|| Command             bundle exec rake postal:cron
|| Respawning          5 every 3600 seconds
|| Restart mode        term-start
|| Log path            none specified
|| Address/Port        none
|| => cron.1           Running      10:38        pid:1890     respawns:0      port:-        tag:-

|| smtp
|| Quantity            1
|| Command             bundle exec rake postal:smtp_server
|| Respawning          5 every 3600 seconds
|| Restart mode        usr1
|| Log path            none specified
|| Address/Port        none
|| => smtp.1           Running      10:38        pid:1892     respawns:0      port:-        tag:-

|| requeuer
|| Quantity            1
|| Command             bundle exec rake postal:requeuer
|| Respawning          5 every 3600 seconds
|| Restart mode        term-start
|| Log path            none specified
|| Address/Port        none
|| => requeuer.1       Running      10:38        pid:1894     respawns:0      port:-        tag:-

netstat -antup:
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 <IP_ADDRESS>:8080          <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:80              <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:4369            <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:11443         <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:443             <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:5000          <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:25672           <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:*               LISTEN      -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42458         ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42454         ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:42466         <IP_ADDRESS>:3306          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42468         ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:42462         <IP_ADDRESS>:3306          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:55280         <IP_ADDRESS>:5672          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:42460         <IP_ADDRESS>:3306          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:4369          <IP_ADDRESS>:41756         ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:42468         <IP_ADDRESS>:3306          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:42458         <IP_ADDRESS>:3306          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42462         ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:42454         <IP_ADDRESS>:3306          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42466         ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:55284         <IP_ADDRESS>:5672          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:4369          <IP_ADDRESS>:43046         TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:41756         <IP_ADDRESS>:4369          ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:22       <IP_ADDRESS>:59804     ESTABLISHED -
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42460         ESTABLISHED -
tcp6       0      0 :::80                   :::*                    LISTEN      -
tcp6       0      0 :::4369                 :::*                    LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -
tcp6       0      0 :::25                   :::*                    LISTEN      -
tcp6       0      0 :::443                  :::*                    LISTEN      -
tcp6       0      0 :::5672                 :::*                    LISTEN      -
tcp6       0      0 <IP_ADDRESS>:5672          <IP_ADDRESS>:55280         ESTABLISHED -
tcp6       0      0 <IP_ADDRESS>:25       <IP_ADDRESS>:38244    TIME_WAIT   -
tcp6       0      0 <IP_ADDRESS>:5672          <IP_ADDRESS>:55284         ESTABLISHED -
udp        0      0 <IP_ADDRESS>:68              <IP_ADDRESS>:*                           -

Also, on the server if I run the telnet command, it works.
It definitely sounds like a firewall problem from my VPS provider but I called them and everything looks good from their point of view. So, Am I missing something?

try with:  "sudo tcptraceroute mysmtpserver.com 25"
Possible reasons:

route issue
port access blocked
remote service ACL / limits
remote service down

anyway,, looks like your problem has nothing related to Postal itself and it close to a network issue.

tcp6       0      0 <IP_ADDRESS>:25       <IP_ADDRESS>:38244    TIME_WAIT   -

Can this line explain the problem perhaps? <IP_ADDRESS> is the server IP

Maybe. Try to disable IPv6 by using tcp flags on /etc/sysctl.conf and try again. As I said it has nothing related to Postal.

if you ran netstat as sudo it may reveal which program has which port?

charlie@charlie-XPS-15-9550:~$ telnet <IP_ADDRESS> 25
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
220 mail.typebot.io ESMTP Postal/TNSDKD
quit
221 Closing Connection
Connection closed by foreign host.

Works for me :slightly_smiling_face:

It's possible that the place you are trying to connect from does not allow SMTP connections.

charlie@charlie-XPS-15-9550:~$ telnet <IP_ADDRESS> 25
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
220 mail.typebot.io ESMTP Postal/TNSDKD
quit
221 Closing Connection
Connection closed by foreign host.

Works for me 🙂

Interesting!

I installed Spark Mail client. Could this cause SMTP connections to be blocked??

if you ran netstat as sudo it may reveal which program has which port?

I had this issue a while back on my Mac with the connection timing out.
Try this: run Terminal and enter:
$ sudo sysctl -w net.inet.tcp.always_keepalive=1
The interval might need to be tweaked, I think it's something like 20 minutes, but I'm not sure.

It also times out when I'm connecting from a google cloud function (europe-west servers).

if you ran netstat as sudo it may reveal which program has which port?

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 <IP_ADDRESS>:8080          <IP_ADDRESS>:*               LISTEN      4531/[postal] fast.
tcp        0      0 <IP_ADDRESS>:80              <IP_ADDRESS>:*               LISTEN      1552/nginx -g daemo
tcp        0      0 <IP_ADDRESS>:4369            <IP_ADDRESS>:*               LISTEN      1596/epmd
tcp        0      0 <IP_ADDRESS>:11443         <IP_ADDRESS>:*               LISTEN      4531/[postal] fast.
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN      1443/sshd
tcp        0      0 <IP_ADDRESS>:443             <IP_ADDRESS>:*               LISTEN      1552/nginx -g daemo
tcp        0      0 <IP_ADDRESS>:5000          <IP_ADDRESS>:*               LISTEN      1883/[postal] web.1
tcp        0      0 <IP_ADDRESS>:25672           <IP_ADDRESS>:*               LISTEN      1320/beam
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:*               LISTEN      1699/mysqld
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42458         ESTABLISHED 1699/mysqld
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42454         ESTABLISHED 1699/mysqld
tcp        0      0 <IP_ADDRESS>:4369          <IP_ADDRESS>:59790         TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:43664         <IP_ADDRESS>:3306          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:42466         <IP_ADDRESS>:3306          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42468         ESTABLISHED 1699/mysqld
tcp        0      0 <IP_ADDRESS>:42462         <IP_ADDRESS>:3306          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:55280         <IP_ADDRESS>:5672          ESTABLISHED 4534/[postal] cron.
tcp        0      0 <IP_ADDRESS>:42460         <IP_ADDRESS>:3306          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:43664         ESTABLISHED 1699/mysqld
tcp        0      0 <IP_ADDRESS>:4369          <IP_ADDRESS>:41756         ESTABLISHED 1596/epmd
tcp        0      0 <IP_ADDRESS>:42468         <IP_ADDRESS>:3306          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:42458         <IP_ADDRESS>:3306          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42462         ESTABLISHED 1699/mysqld
tcp        0    332 <IP_ADDRESS>:22       <IP_ADDRESS>:28301     ESTABLISHED 4073/sshd: ubuntu [
tcp        0      0 <IP_ADDRESS>:42454         <IP_ADDRESS>:3306          ESTABLISHED 4532/[postal] reque
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42466         ESTABLISHED 1699/mysqld
tcp        0      0 <IP_ADDRESS>:55284         <IP_ADDRESS>:5672          ESTABLISHED 4521/[postal] worke
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:43052         ESTABLISHED 1699/mysqld
tcp        0      0 <IP_ADDRESS>:43052         <IP_ADDRESS>:3306          ESTABLISHED 4524/[postal] smtp.
tcp        0      0 <IP_ADDRESS>:41756         <IP_ADDRESS>:4369          ESTABLISHED 1320/beam
tcp        0      0 <IP_ADDRESS>:3306          <IP_ADDRESS>:42460         ESTABLISHED 1699/mysqld
tcp6       0      0 :::80                   :::*                    LISTEN      1552/nginx -g daemo
tcp6       0      0 :::4369                 :::*                    LISTEN      1596/epmd
tcp6       0      0 :::22                   :::*                    LISTEN      1443/sshd
tcp6       0      0 :::25                   :::*                    LISTEN      4524/[postal] smtp.
tcp6       0      0 :::443                  :::*                    LISTEN      1552/nginx -g daemo
tcp6       0      0 :::5672                 :::*                    LISTEN      1320/beam
tcp6       0      0 <IP_ADDRESS>:25       <IP_ADDRESS>:58178      TIME_WAIT   -
tcp6       0      0 <IP_ADDRESS>:5672          <IP_ADDRESS>:55280         ESTABLISHED 1320/beam
tcp6       0      0 <IP_ADDRESS>:25       <IP_ADDRESS>:21386      TIME_WAIT   -
tcp6       0      0 <IP_ADDRESS>:5672          <IP_ADDRESS>:55284         ESTABLISHED 1320/beam
tcp6       0      0 <IP_ADDRESS>:25       <IP_ADDRESS>:49380      TIME_WAIT   -
udp        0      0 <IP_ADDRESS>:68              <IP_ADDRESS>:*                           1047/dhclient

Postal is running on port 25.

Ok I just saw #864

Ok I just saw #864

Yeah sorry about that, some ISPs just get too much abuse and don't support email :slightly_frowning_face:


Update to use fixed CMI

Right after you included my changes, Wuzzy pointed out that CMI does not use the proper entity field conventions, so I have updated CMI to use them.

Updated.