Developer mode requirement on Windows

The current snapshot_download and hf_hub_download methods currently use symlinks for efficient storage management. However, symlinks are not properly supported on Windows where administrator privileges or Developer Mode needs to be enabled in order to be used.
We chose to take this approach so that it mirrors the linux/osx behavior.
Opening an issue here to track issues encountered by users in the ecosystem:

https://github.com/huggingface/transformers/issues/19048

Thanks for making this issue. Responding to Lysandre's comment here https://github.com/huggingface/transformers/issues/19048#issuecomment-1249642049:
No, to use WSL you do not need to enable developer mode [since 2017](https://devblogs.microsoft.com/commandline/developer-mode-no-longer-required-for-windows-subsystem-for-linux/. I have been running WSL2 with CUDA and never had Developer Mode activated. I also only ever use WSL2 if I really cannot get software to run natively on Windows. WSL is great but it has its pitfalls/quirks. Like the interaction with PyCharm of setting up/managing different environments, but also IO which is really slow between /mnt/ and the distro's native drive. I prefer native Windows when I can.
So for most of my transformers projects, I've used Windows, mostly without issue or I posted a PR to fix the problem. Windows users seem plenty. And in my case, I am especially worried for the class room. At the start, I use Colab because it's easy for beginners. But after a while we switch to a local installation and running scripts instead of notebooks so they at least know about those things. I cannot require them to enable developer mode (liability) and WSL will take too much time of a class to get working for everyone.
Outside of the classroom, you can think about anyone who are just getting started with transformers and want to play around with it on their own system - gamers, hobbyists, enthusiasts, or anyone with an interest in AI-related things. Windows users are still the majority (even among developers) and they are not all hardcore PC enthusiast. Even if they are (like me), they may not be eager to have this additional step to allow access to transformers. So my issue here is that instead of making access to AI easier (as Hugging Face has done in the past, for which I am of course grateful), this new change actually makes it harder for a portion of users. I am not sure if the advantages outweigh the disadvantages because everything worked fine before.

For our frontend and many others as well this will be a big issue, it is very hard to explain to users as to why they need developer mode or administrative permissions in order to be able to download a model. This seems very suspicious to a user since they will not be aware of the technical limitation. It would be much nicer if we have control over this, such as being able to use the old cache system if desired.

Posting here to keep an eye on this. Fully agree with the sentiment. Requiring developer mode will limit the ability to use transformers in programs that are given to non-developers. An average user will not want to enable developer mode when it's full of warnings.

I'm curious if this affects many many Windows users, or not. Let's see if more users post about this in the coming days (to see how we could fix it)? Will also ask on Twitter=)

I'm curious if this affects many many Windows users, or not. Let's see if more users post about this in the coming days (to see how we could fix it)? Will also ask on Twitter=)

As @henk717, @ebolam and my posts suggests, the problem is not just with developers. If they really need to and you do not give them a choice, I guess they will enable developer mode or be forced to switch to WSL (but again: I was not eager to enable dev mode on my machine either; it comes with its own risks).
A major problem is the consequence for non-developers: hobbyists, students, end-users who try to run products/demos/tools  that use transformers under the hood. Those won't know what to do or will not want to start enabling things in Windows that give you a big security warning. Those are exactly the new people you'd want to get interested, not scare away. The results of both your Twitter poll and the comments on Github will therefore be very biased towards developers.
As @henk717 suggests, keeping the option to use the old cache mechanism would be great.

The results of both your Twitter poll and the comments on Github will therefore be very biased towards developers.

Yes you're right @BramVanroy

Adding my comment here (as suggested by @julien-c). Activating developer mode is asking for trouble. On top of that, I had to drop support for WSL because my developer PC simply cannot handle it (running it will take 75% of available RAM + impossible to shut down once activated, not to mention that I cannot even run dropbox on my windows machine without constant hanging). Turning on WSL and developer mode not only requires users to run a beefy pc, but it also pushes them to security warnings and dialogues that I Pavlov'ed normal users into to pressing "no" for security reasons.

Following this closely as I faced the same issue I posted in the HF forum here:
https://discuss.huggingface.co/t/symlink-error-when-importing-ortseqclass-model-via-pipeline/20706
I am not a developer and was not too comfortable with running PC in dev mode, tried experimenting to see if that would solve it but got the same error after switching dev mode on and doing a full restart.

A major problem is the consequence for non-developers: hobbyists, students, end-users who try to run products/demos/tools that use transformers under the hood. Those won't know what to do or will not want to start enabling things in Windows that give you a big security warning. Those are exactly the new people you'd want to get interested, not scare away. The results of both your Twitter poll and the comments on Github will therefore be very biased towards developers.

Very much this, transformers is a dependency of multiple user friendly or end user experiences. KoboldAI in our case, but on steam there are games like AI Roguelite and AIDventure that also use transformers as a dependency. Explaining a user who buys an end user product in a store front that they manually need to change Windows settings for the program to work correctly is a step to far and should not be automated away since you are then changing security settings.
So a solution that does not require system wide changes is very much desired for applications that are either portable, or installed trough automated means.

Another thing: It's windows now, but what if you would force the user to use root to install the package under Linux? Would you force people to use "sudo" to run transformers, knowing that it uses pytorch which has an exploit in their system that you are currently scanning for?

Hey, thanks all for your comments. We agree that it shouldn't behave this way under Windows and we're working on a solution.

Hi everyone and thank you for the feedback.
Here is a PR to implement a cache-system variant that do not need symlinks if there are not available: https://github.com/huggingface/huggingface_hub/pull/1067. See description for more details. Any feedback is very welcomed ! :)

https://github.com/huggingface/huggingface_hub/pull/1067 is now merged and will be available in next release of huggingface_hub. For now one can pip install from the main branch to run on Windows.
I'll keep this thread updated once the release is made.

@BramVanroy and others in this thread, would you mind please updating your install to use the updated main branch and let us know if it works out for you?
You can update with the following:
pip install git+https://github.com/huggingface/huggingface_hub

Thank you!

@BramVanroy and others in this thread, would you mind please updating your install to use the updated main branch and let us know if it works out for you?

The implementation works within our software, I did notice a performance regression compared to the old cache however since it takes time (and space) to copy the files to the snapshot folder. Moving the files, or referencing the blob location trough text files (effectively fake symlinks) would be a leaner solution that would impact users with slow storage less.

I did the pip install but I am now getting a different error:

I have the following env variables set:
HF_HOME: D:\nickm\Transformers
TRANSFORMERS_CACHE: D:\nickm\Transformers

Hi @nickmuchi87 , thanks for reporting this bug. I created a PR (https://github.com/huggingface/huggingface_hub/pull/1077) to make the are_symlinks_supported() more reliable. This should solve your problem. Could you try to install from the branch and confirm ?
pip install git+https://github.com/huggingface/huggingface_hub@1062-check-symlink-supported-in-cache-dir

@henk717 We understand your concern. To be fair, we hesitated between two workarounds when symlinks are not supported. Both solutions (fake symlinks or duplicated files) have their own pros and cons. Please let me share the elements on which we based the decision to go for duplicated files:
Fake symlinks
With fake symlinks, a text file containing only a path to the actual blob file is stored under the snapshot path. This way the user has to read the file before going to the blob path to actually get the content for the file.
Pros:

minimal impact on storage
minimal impact when downloading (no files copied)

Cons:

extra logic when reading the file content

Duplicate files
In this approach, we do not use the blob directory anymore but directly copy the files in the snapshot directory.
Pros:

no extra logic required once the file is cached

this is particularly important for us as huggingface_hub is a package meant to be re-used in a lot of external libraries. We currently have a helper snapshot_download() that downloads a snapshot of a full repo and return the path of the directory in the cache. Once this path is returned, the user expects all the files to be under this directory path. In particular, one can do  snapshot_path.glob("*") to iterate over all the files of the snapshot. With fake symlinks we cannot guarantee to the user that the files are "actual" files and not symlinks.
having to handle fake symlinks in downstream libraries is really something we want to avoid as we don't have a hand on all of them. Besides, it would be very confusing/frustrating for maintainers to have to debug/investigate a fake-symlink-issue of one of their users if they are not even aware of the fake-symlink specificity (I don't expect everyone to carefully read the documentation on this topic).

compatibility:  if a Windows user starts using huggingface_hub without developer mode and then enable it after a bit of time, the cache structure and usage will be exactly the same. In particular there will be no need to handle both real symlinks and fake symlinks on the same machine/same cache folder. Caching will be optimized for newly downloaded models but previously downloaded files will still work fine.

Cons:

more disk usage when downloading several revisions of the same model
more execution time when downloading if a file is copied locally

however this is only a corner case if a previous blob file existed in the blob folder (that is then copied to the snapshot folder). This can only happen if the same model was previously downloaded using dev mode/admin rights. If the cache is clean (never used symlinks before), the downloaded file is moved to the snapshot folder using os.replace() which should be instant.

Conclusion
As you may have understood, we really favored an approach that will avoid as much as possible the friction to integrate huggingface_hub everywhere instead of a fully optimized cache-system on all platform/machines. This decision is also motivated by the fact that it is always possible to activate the developer mode on Windows to support symlinks. The comments in this thread were mentioning a lot the necessity to support any Windows machines, especially to allow anyone to use huggingface_hub/transformers for teaching, for experimenting, for studying, in games,... All those use-cases are legit and now supported with no downside. Only users with a more intensive usage (e.g. iterating over revisions of the same model) will be affected and that's where we advise to enable the developer mode.
Please also notice that we provide a delete-cache command line to make it easy for users to clean up their disk.
I hope this message helps everyone here (and in the future) understand the workaround we chose to implement and the reasons behind it. Please let me know if you have any further questions.
(Note: for any questions on how the cache-system is working, you can refer to this documentation page)

Hi @nickmuchi87 , thanks for reporting this bug. I created a PR (#1077) to make the are_symlinks_supported() more reliable. This should solve your problem. Could you try to install from the branch and confirm ?
pip install git+https://github.com/huggingface/huggingface_hub@1062-check-symlink-supported-in-cache-dir

I tried installing the above:

then restarted my jupyter-lab but getting the same error, not sure what I am doing wrong here:

@Wauplin Thanks for the elaborate write up! While I do not have time now to investigate the details (and noting that @nickmuchi87 is already testing your changes), I am extremely satisfied about the quick PR to make transformers tools fully/openly accessible to Windows users again. While I can understand @henk717's comment about disk usage, I am thankful for your elaborate motivation. The decision seems sensible to me.
One question. You write the following:

Only users with a more intensive usage (e.g. iterating over revisions of the same model) will be affected and that's where we advise to enable the developer mode.

Does this mean that specifically only revisions of the same model will be duplicated? Or does this also affect regular models? While I do not use revisions of the same model often, I do switch between a lot of different models often. Does that mean that they will all be duplicated?
(This is more of a hypothetical questions because I ended up enabling developer mode to continue my current work with transformers.)

@BramVanroy Thank you for your feedback. Here are some more detailed explanations:

Does this mean that specifically only revisions of the same model will be duplicated?

What I mean by "revisions of the same model" is if you download multiple versions of a single repo.
Let's consider the repo distilgpt2. If you download the model from main, you'll have the model only once in your cache, the same way as other users. If you want to test the model from branch refs/pr/6 (let's say for reviewing the PR), then you will end up with duplicate files on your system. That's what I meant by having "a more intensive usage" than just experiencing around.
(for the record, I mix a lot "repo" and "model" in the wording. In transformers the user only sees "models" but in the background all models are stored as git repositories on the HF Hub)

Or does this also affect regular models? While I do not use revisions of the same model often, I do switch between a lot of different models often. Does that mean that they will all be duplicated?

I not sure what you mean by "regular models". If you switch between different models (e.g. gpt2, distilgpt2, gpt2-large, xlnet-base-cased,...), each model will be individually downloaded in your cache-system as any user. Blob files are never shared between repositories in the cache, no matter the platform. Once they have been downloaded once, you can switch between them as many times as you want without redownloading anything.
Hope this makes it clearer for you now :)

@Wauplin that worked, so I uninstalled and re-installed using Anaconda Prompt this time instead of Jupyter, thanks for that! Appreciate the efforts!

@nickmuchi87 Perfect ! :tada:

@Wauplin If I understand correctly, this means that indeed if I download many different repos I won't get duplicated data and that this only happens if you download different commits/revisions of the same model repo. So that's a non-issue for me.
Thanks again for your work!

@BramVanroy exactly :)


do not regenerate every file each time

When writing documentation with Sphinx we often need to rebuild the output to verify if the page layout etc.. are as we expect.
This isn't usually problematic and is relatively quick. However, when we enable exhale it forcefully re-generates the entire API documentation each time make html is invoked.
I've seen in the backlog of issues that the project basically would need a rewrite to support partial re-generation of docs, but I haven't found a way to temporarily disable it so that only the non-generated docs get rebuilt.

Never mind, I had to comment both breathe and exhale out of the conf.py extension list and do a clean.

Would still be nice to prevent regenerating the docs while keeping a version so that the links still work though.

Re-opening, part of #10.  Not regenerating is also a component of #94.  Added some more notes there.
I think if I skip regenerating things that have the same hash then that could help a lot.  It may produce false no-rebuilds though, if a given document is not reprocessed and the underlying documentation for the construct is updated (doxygen comment changed).  That said, for iterative development what exhale does (spew new documents every time) is extremely inconvenient.

Re-opening, part of #10.  Not regenerating is also a component of #94.  Added some more notes there.
I think if I skip regenerating things that have the same hash then that could help a lot.  It may produce false no-rebuilds though, if a given document is not reprocessed and the underlying documentation for the construct is updated (doxygen comment changed).  That said, for iterative development what exhale does (spew new documents every time) is extremely inconvenient.

It may produce false no-rebuilds though, if a given document is not reprocessed and the underlying documentation for the construct is updated (doxygen comment changed).

What might work is a local cache of hashes for each XML file? Or even the .rst files? And then only overwrite the files who changed relative to the currently existing .rst files. That way it would "only" be an extra processing step at the end of the normal breathe/exhale pipeline instead of needing to rewrite how exhale works
This would prevent sphinx from writing output, which, as far as I can tell, is the slowest part of the build-process.

Wanted to add a quick follow-up, this is a good suggestion. I'll need to learn how to interface with the sphinx environment caching to implement this. But once I've got a handle on that the scheme you propose seems like it should be straightforward.

just stumbled across this: https://docs.python.org/3/library/filecmp.html
So what might work:

Put all of exhale / breathe output in a temporary folder.
dircmp/filecmp the directories to check if the XML is updated.
i.  if they're identical stop the flow here since nothing has changed
ii.  if they're not identical, generate the necessary .rst files in a temporary directory. (optionally dircmp /filecmp this step too)
copy the .rst files (optionally only those that need copying) to the actual output directory, overwriting the old files, triggering sphinx to regenerate the necessary output.

Thins I'm not sure about:

If a new .rst file is generated, that could mean links to it break and/or it links to files that might not be available.
If a .xml/.rst file from the output disappears, should this trigger a full-copy to trigger a full-rebuild from sphinx?
Same goes for a new file, if a new file is detected, it might replace links or something in other files, so maybe a full-rebuild should be triggered.

but just adding steps 1&2 would speed up the builds considerably whenever you're not changing the C/C++ code/docstrings.

This one is going to have to wait since commenting out exhale from extensions list is a viable workaround.  The changes here are substantive and would need testing.


Encode private keys to PKCS1/PKCS8 as requested by user

What this PR does / why we need it:
Currently when a user uses experminetal certificates controller only, the private keys are always encoded in PKCS8. This PR is to correctly handle the Spec.keyEncoding field to encode private keys to PKCS1 or PKCS8 as requested by the user.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #2863
Special notes for your reviewer:
Release note:

Experimental certificate controllers encode private keys according to specification of user.

/kind bug
/lgtm
/approve


Center instead of enlarge hero image

I want to use a square image as a hero image.  Something with a side length of about 600px.  However, it doesn't work very well now.  The Hero component enlarges the provided image until the width matches the width of the browser window.  I would prefer if the image is left at its original size and is instead horizontally centered.  However, if the image were too large, then I think I would want the image shrunk to fit though.  I don't think I ever want an image to be made larger.
Every hero image I have seen @loreanvictor use comes from unsplash, which includes an API for specifying what size the image should be.  See this for an example.  This has put the image-sizing burden on unsplash, so the Hero component hasn't needed to deal with images that are not perfectly sized.
Can the Hero component center the image given to it instead of enlarging it?
If you want to keep the current behavior, can options be added to the Hero component to support both the current behavior and the behavior like I want?

seems like a good addition, at least as an option (though a default behavior supporting all current use cases might also be possible, in which case it would obviously be preferable).
just a quick question though: why don't you use a normal image instead? the main point of a hero image is having an image that is expanded beyond the normal content container, which is not what you want. in other words: which features of hero images do you miss on normal images for this particular use case?

The hero image is also used as the image for an article card.
I could put this 600px square image in the body of the post, but I also want it to be the image for the article card.
Maybe if there is no hero image, then the article card can use the first image in the body of the post.

As can be seen here, the ArticleCard component should pick up images picked as social graph images. As it seems from this, normal images with absolute URLs.
This leaves images with relative URLs. In that case, I suspect one could simply add the target base URL of the blog in the config, similar to how RSS is configured. In fact, since RSS also requires this base URL, it could be provided on a level higher than that of the RSS configuration and the RSS post-processor could simply read and utilize that config as well. In other words, with this change we would enable using relatively addressed images both for ArticleCards and social cards / previews.
@TysonMN if that satisfies what you need for the outlined use-case, I will change the title of the issue so that it reflects the intended broader change.

I don't completely follow, but it sounds like a good idea.  You can change the title and make the change you describe.  Then I could create another issue if I am still having a problem.

This leaves images with relative URLs.

When testing, I wasn't able to link to a certain image with an absolute URL.  The request to obtain the image was blocked by Google Chrome because of Cross-Origin Read Blocking (CORB).  I don't know why that image was blocked but images from unsplash are allowed.

I don't completely follow, but it sounds like a good idea. You can change the title and make the change you describe. Then I could create another issue if I am still having a problem.

Basically, I would make changes that would allow images with relative paths to be used for social cards (Twitter / Open Graph). This means you could use a normal image for this use case instead of a hero image, and it would still be picked up by ArticleCard component.

This leaves images with relative URLs.

When testing, I wasn't able to link to a certain image with an absolute URL. The request to obtain the image was blocked by Google Chrome because of Cross-Origin Read Blocking (CORB). I don't know why that image was blocked but images from unsplash are allowed.

That depends on the source of the image. The source should allow cross origin requests, and unsplash does (for example I am not sure if coding.blog does, which means you couldn't get the images locally but would be able to get them when published since then you would be on the same origin).

Basically, I would make changes that would allow images with relative paths to be used for social cards (Twitter / Open Graph). This means you could use a normal image for this use case instead of a hero image, and it would still be picked up by ArticleCard component.

Yes, that is my understanding.  I don't completely follow how you are going to do that, but I don't I need to know either.  I am confident that you know what you are doing :)

The source should allow cross origin requests, and unsplash does...

Oh, I didn't know that.  Thanks for telling me.

I don't completely follow how you are going to do that, but I don't I need to know either. I am confident that you know what you are doing :)

Quite simple actually: with a config value determining the absolute base URL to use, similar to how the RSS post-processor figures out absolute URL of an article from the relative paths that go through a post-processor.

FYI: this is the issue that I am most eager to see fixed.  I have a blog post done, but I really want the first image in the article to be used when I share a link to the post on Twitter and Facebook.

Makes sense, will prioritize this.


Update/correct metadata in Datacite via API/query.

Update or correct metadata via API call for an already published nmrXiv dataset.
Related to #1064

Dear @tilfischer,
This is fixed, which fixes the issue #1064. Can you please check and confirm once?
https://api.datacite.org/dois/10.57992/nmrxiv.p55
Thanks,
Nisha

Dear Nisha,
looks fine now. Thank you for that!
What I really still do not like is that nmrXiv Team members are automatically added as contributors. This is not obvious and I would leave the peower to the people i.e. researchers who upload dataset should be allowed to select who is a contributor. This will otherwise lead to scientific misconduct following the DFG guidelines of good scientific practice. Please see https://github.com/NFDI4Chem/nmrxiv/discussions/948
nmrXiv need a separate section for contributors. nmrXiv team members which be prefilled, but should be removable and movable i.e. the order should be able to change, as with the Creators/Autors. I guess this is work in progress, but I still want to highlight this here, as this might lead to, as mentioned, scientific misconduct if people are added, which had nothing to do with a submission except being part of the same nmrXiv team.
For the dataset of Mr. Cicek, I would, for instance, have prefered to have him listed as a contributor first, as he did upload the data as a member of the Lead-by-Example team, which was required that I could help him for issues he had.
Best,
Tillmann


Apply Deadlines RFC

Reference: https://github.com/EventStore/architecture-and-planning/pull/52

Done


fix: context level should execute before variant level

Added sorting keys by its length, so that the shortest length keys execute first.

hotfix-v1.6


specify tornado version for python2.6

fix #84

Coverage remained the same at 88.451% when pulling<PHONE_NUMBER>db284b2854c17e8b5604b9961da210 on yukimori:fix-84 into e084dba523afbee2395d132a29ef7852bac46da9 on jubatus:develop.

Coverage remained the same at 88.451% when pulling<PHONE_NUMBER>db284b2854c17e8b5604b9961da210 on yukimori:fix-84 into e084dba523afbee2395d132a29ef7852bac46da9 on jubatus:develop.

Coverage remained the same at 88.451% when pulling<PHONE_NUMBER>db284b2854c17e8b5604b9961da210 on yukimori:fix-84 into e084dba523afbee2395d132a29ef7852bac46da9 on jubatus:develop.

Coverage remained the same at 88.451% when pulling<PHONE_NUMBER>db284b2854c17e8b5604b9961da210 on yukimori:fix-84 into e084dba523afbee2395d132a29ef7852bac46da9 on jubatus:develop.

I have advised to specify tornado version on jubatus-python-client.
PR jubatus/jubatus-python-client#78
So, ths PR closed


Please invite me to the GitHub Community Organization

Please invite me to the GitHub Community Organisation.

Name: Aman Kumar

Discord Username: kr_aman#5243

Additional Context: Discord

I'm trying to get more into open source and this community is helping a lot :)

:nerd_face:

Invitation sent for the GitHub Organisation. Welcome to the community 🎉
Don't forget after accepting to make it public so it appears on your GitHub profile for everyone else to see, you can do this by finding your name in the GitHub organisation list and change the dropdown to public https://github.com/orgs/EddieJaoudeCommunity/people
Tips for practising:
Customise your GitHub profile, here is a live stream on it https://www.youtube.com/watch?v=cT6GXCuS0Zo
Practice repo, instructions on how to add your name to the README in the README https://github.com/EddieJaoudeCommunity/hacktoberfest-practice
Remember contributing to open source is not just about code, its about collaboration, communication and adding value
I hope that helps

Invitation sent for the GitHub Organisation. Welcome to the community 🎉
Don't forget after accepting to make it public so it appears on your GitHub profile for everyone else to see, you can do this by finding your name in the GitHub organisation list and change the dropdown to public https://github.com/orgs/EddieJaoudeCommunity/people
Tips for practising:
Customise your GitHub profile, here is a live stream on it https://www.youtube.com/watch?v=cT6GXCuS0Zo
Practice repo, instructions on how to add your name to the README in the README https://github.com/EddieJaoudeCommunity/hacktoberfest-practice
Remember contributing to open source is not just about code, its about collaboration, communication and adding value
I hope that helps


Add page-action icon to Firefox

The status icon in Chrome which shows when devtools is available or not does not show in Firefox.
See #36 for more detail on what the icon does.

That seems to be working for me.
This is a route in an app that does not have urql as a HOC.

And here is a route that has, icon not greyed out.

The other icon there was just an experiment with pageAction API to see how chrome and firefox API differs.

@r281GQ thanks for checking this - I'll close it in that case :+1:


doc(README+Quickstart): Update examples to include ASGI

@vytas7 lots of good feedback... I think I'll circle back on this once #1716 is merged.

Nevermind, I went ahead and made the updates without waiting. 😄

@kgriffs mujson errors in python 3.6 and 3.7 when dumps is called without kwargs. Not sure why.
I've added a simple fix here https://github.com/falconry/falcon/pull/1701/commits/32ca2ac55d7b7744470adf8ff92cec456ea030e5 that seems to make it happy

@vytas7 @CaselIT OK, I think this should be ready to go now.


Rename Next steps to "Related resources"

This aligns with how these items will be referred to in templates.
Testing
This is what users should see:

This is just a label change; merging.


can`t run rancher agent on docker with storagedriver is overlay

Version: docker-1.9.1    agent-0.9.0
[root@test482 docker]# docker info
Containers: 0
Images: 18
Server Version: 1.9.1
Storage Driver: overlay
Backing Filesystem: xfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.1 (Maipo)
CPUs: 24
Total Memory: 62.73 GiB
Name: test482
ID: HKUW:HMQV:AOG3:MBG2:2KXQ:HRDI:POJ5:BKAP:46G3:I3MI:YNHZ:H3MK
Steps:
run rancher agent on docker server
sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher stargate.lufax.tool/rancher/agent:v0.9.0 http://<IP_ADDRESS>:8080/v1/scripts/D8411D6DCA17E45CE30E:1455804000000:4zxJTziV47nQKQC0xrWqySi1szw
Results:
[root@test482 docker]# sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher stargate.lufax.tool/rancher/agent:v0.9.0 http://<IP_ADDRESS>:8080/v1/scripts/D8411D6DCA17E45CE30E:1455804000000:4zxJTziV47nQKQC0xrWqySi1szw Error response from daemon: mkdir /data/docker/overlay/ea1047215e2fa4578ddbe04007db2372e3df69bea2efc55cfae9f5b24ff1a93a-init/merged/dev/shm: invalid argument
Expected:
docker config
INSECURE_REGISTRY="--insecure-registry=stargate.lufax.tool" OPTIONS="-s overlay" DOCKER_NETWORK_OPTIONS="-H tcp://<IP_ADDRESS>:2375 -H unix:///var/run/docker.sock" DOCKER_STORAGE_OPTIONS="-g=/wls/docker"

Did any people have the same problem ?

What Rancher server version are you using?
What is the rancher/agent version you are using? It doesn't look like the default one provided by Rancher and we've never released v0.9.0 with any release.

Rancher server : v0.59.0
My fault.I update rancher/agent to default version for rancher/server(v0.9.2).But it`s still not working.
[root@test481 ~]# sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher stargate.lufax.tool/rancher/agent:v0.9.2 http://<IP_ADDRESS>:8080/v1/scripts/D8411D6DCA17E45CE30E:1456394400000:KKNGWKWXorWIfYBRiGb2AKlCmA
docker: Error response from daemon: mkdir /data/docker/overlay/fa2810f5c62cdfc174482b8f23142f9b210169b1b6e97f322521a17a2e3efbb8-init/merged/dev/shm: invalid argument.
See 'docker run --help'.

I realised that docker can not work prefectly on RHEL/Centos.
The problem is gone when I change node to Ubuntu.

What version of RHEL/CentOS were you trying?

Operating System: Red Hat Enterprise Linux Server 7.1 (Maipo)

I believe overlay on XFS does not work well.  We have seen overlay on ext4 work on RHEL 7.2.

I've experience the same problem today.  I'd be happy to change to Ubuntu, but we've invested a fair amount into Centos 7.2.  This is NOT using XFS.  I've done minimal security measures.  No SELinux yet.
See '/usr/bin/docker-current run --help'.

With all changes in both RHEL and Rancher, please file a new issue if you still experience issues.


Look into rewrite with postcss-nested bubbles

If we bubble @responsive we might simplify some selector walking

For instance unnest with bubble @responsive at the start of pipeline
Then unnest at the end to bubble @media.


Add new broker query point for querying multi-stage engine

Per #11251 , create two new Broker API endpoints for multi-stage query engine.
GET /query
POST /query

Make Integration tests to use new endpoint for testing multi-stage engine.

Please add these new APIs to the pinot doc


Increment version for keyvault releases

Increment version for keyvault releases

Can one of the admins verify this patch?


[Scope Change] codeunit 226 "CustEntry-Apply Posted Entries" procedure SetCheckDim

Please remove the scope OnPrem for procedure SetCheckDim in codeunit 226 "CustEntry-Apply Posted Entries".
thanks

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


feat: make templ fmt, format writen go code in templates

This PR solves the problem as written in the Issue #208 that go templ formats Go code which can be written alongside the templ keyword.
I'm still new to the go ecosystem but saw what was done in PR #102 with the go/format package.
I think my little modification to format the code with this package when generating the clean string version is good.
Of course I'm open to any improvement.

Hey, this looks great. I noticed this the other day, but hadn't had a chance to look into it.
I would like to see a unit test of the behaviour so that we don't accidentally remove this feature and break it in the future.
It shouldn't be much effort to that, just need to write some badly formatted Go code and check that it gets properly indented, some invalid Go code to show that it is left as-is, and some already well formatted Go code to show that it's not adjusted.

This may result in not needing writeIndent execution and tracking. They could just all just be changed to Write.

This may result in not needing writeIndent execution and tracking. They could just all just be changed to Write.

I just finish to add unit tests.
If I understand what you said, you propose I write this ? :
func (exp GoExpression) Write(w io.Writer, indent int) error {
	data, err := format.Source([]byte(exp.Expression.Value))
	if err != nil {
		return writeIndent(w, indent, exp.Expression.Value)
	}

	_, err = w.Write(data)
	return err
}

If that's it, then you were right, writeIndent is more need if we successfully format with the builtin formater !
It's not necessarily obligatory I would say to make this change but I'm still going to make a commit since theoretically it's a performance gain.
(At least I think so? Afterwards it's not necessarily very important to be a little slower or fast)

I was merely making an observation. I defer to @a-h on what they would want.

I was merely making an observation. I defer to @a-h on what they would want.

It may have just been an opinion but I actually think it's a good idea.
The reason I started writing Go was to write simple code in a simple language.
So it seems simpler and faster to me to do a Write directly than to have to apply a function that no longer makes sense.
In addition I want to add but unit tests are also designed to test and refactor things that we didn't think about at the beginning ^^

In any case I'm just going to ask a quick question here on Github since I'm not yet used to contributing that much.
For the small modification I did not want to create an additional commit because it was a fix of a commit which is not part of the main branch. (And also I don't know what message I could write)
So I did a git rebase to modify the commit.
This changed the IDs so my commits created everything new.
To put it on Github I had to git push force since I altered the history.
My question is does this kind of thing make sense in a PR or not?
Is it a bad thing ?
Is it more important in a PR to have a clean list of modification with multiple commit or have multiple commit that represent the evolution of the suggestion ?
In any case I'm just going to ask a quick question here on Github since I'm not yet used to contributing that much.
For the small modification I did not want to create an additional commit because it was a fix of a commit which is not part of the main branch. (And also I don't know what message I could write)
So I did a git rebase to modify the commit.
This changed the IDs so my commits created everything new.
To put it on Github I had to git push force since I altered the history.

My question is does this kind of thing make sense in a PR or not?
Is it a bad thing ?
Is it more important in a PR to have a clean list of modification with multiple commit or have multiple commit that represent the evolution of the suggestion ?
I don't think there is one way to think about that and both are valid but I never have the ocasion to just ask.

In a PR, you can make as many commits as you like. I find it helpful to see the evolution of the PR.
However, when I merge the PR, I'll "squash" them into a single commit using a "conventional commit" message so that only one single commit is made to the main branch. The commit message history helps to form the release notes, hence standardising them.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges#squash-and-merge-your-commits
https://www.conventionalcommits.org/en/v1.0.0/

Thanks, that looks great. I've tested it with the LSP and didn't see any issues there.


TronWeb does not have a method called getContract

via: https://docs.sun.io/developers/swap/sunswap-overview
tronWeb does not have a method called getContract at all：

Original request: Use the tron web SDK to implement Sunswap trading.

"dependencies": {
"@noble/secp256k1": "^1.7.1",
"tronweb": "^5.3.2"
}

The documentation gives wrong api. Please use let contract = await tronWeb.contract([], 'TXk8rQSAvPvBBNtqSoY6nCfsXWCSSpTVQF')


Add content explaining technical flow of verify process

fixes #15

I might try and make that a bit more visually clear tomorrow, but if you could make sure that's technically accurate, that would be great @gdsamit


Adding support for Bootswatch.com themes for b2 and b3

Optionally if defined
app.config['FLASK_ADMIN_SWATCH'] = 'cerulean'

available swatchs in http://bootswatch.com
License of swatches is MIT (the same as bootstrap)

Core flask admin got static/swatch folder with all the swatch files
Changed base.html to use the defined swatch, if not defined it uses swatch/default which is pure bootstrap.min
Examples swatch and swatch3 created

@mrjoes @pawl if you think it is not nice to include all swatch files, maybe we can create a plugin system to be installed. pip install flask-admin-swatch and a form of the plugin to override the base.html block.
cerulean

cosmo

united

slate

More 10 themes

Couldn't this be distilled down to a layout.html example with this?:
{% extends 'admin/base.html' %}

{% block head_tail %}
  {{ super() }}
    {% set bootswatch_theme = 'cyborg' %}
    {% set bootswatch_version = '3.0.0' %}
    <link href="//netdna.bootstrapcdn.com/bootswatch/{{ bootswatch_version }}/{{ bootswatch_theme }}/bootstrap.min.css" rel="stylesheet" >
{% endblock %}

And this when the admin object is declared:
admin = admin.Admin(app, 'Example: Bootswatch', base_template='layout.html', template_mode='bootstrap3')
If someone is changing the styling of flask-admin, they're likely going to be overriding base.html anyway.

:+1: @pawl  I will convert in to a layout example

@rochacbruno Thanks a ton! please watch out for my email :)

Would be super dupa awesome if this could be merged in :)


Adds some padding to CDB-Shape-threePoints so that is easier to click on the button

Buttons are now 10px wide (or tall if the component .is-horizontal).

NOTE: Red outline in ☝ is not present in the new CSS. It's only there so that you can see how tall/wide the component will be.
@piensaenpixel anything to say about this? thx!

yes one thing, "puto amo" 💃
Merge when tests are ok.

Thanks @piensaenpixel!


Rabin-Karp: How was this constant determined?

More of a question than an issue, really. I'm implementing my own version of this for binary searching and wonder if this constant was selected with the assumption that the search string would just be text.
https://github.com/raywenderlich/swift-algorithm-club/blob/898db6fd3f266812be8387a2b2478d5f072cf872/Rabin-Karp/rabin-karp.swift#L24

That was a typo.  The hash multiplier is used to help reduce the chance of a hash collision while calculating the hashes.  It is typically a large prime number, but not so large as to overflow your data structures.  My original intent was for that value to be 69061.

Oh awesome, I was wondering why it wasn't prime.  I guess this is a defect ticket then!

Yup, ^ PR updates the code.  Thank you for catching this error!


Can not sign a normal transaction with getting error cross-origin

Currently I am working on a project called Sigma wallet. I have a input where user would paste the base64 format transaction and sign the multisig transaction.
Now I am heading on making multisig transaction work with myalgo. But I have test if I paste a normal transaction and sign that transaction using myalgo wallet. It appear that window would spinning till the end of the universe with errors:
Uncaught (in promise) DOMException: Blocked a frame with origin "http://localhost:8080" from accessing a cross-origin frame.     at isRef (webpack-internal:///./node_modules/@vue/reactivity/dist/reactivity.esm-bundler.js:1329:20)     at Object.get (webpack-internal:///./node_modules/@vue/reactivity/dist/reactivity.esm-bundler.js:622:9)     at Proxy.signTransaction (webpack-internal:///./node_modules/@randlabs/myalgo-connect/lib/main.js:378:29)     at async Proxy.signTransaction (webpack-internal:///./node_modules/@algo-builder/web/build/lib/myalgowallet-mode.js:69:12)     at async Proxy.sign (webpack-internal:///./node_modules/@vue/cli-plugin-typescript/node_modules/babel-loader/lib/index.js!./node_modules/ts-loader/index.js??clonedRuleSet-41.use[1]!./node_modules/vue-loader/dist/index.js??ruleSet[0].use[0]!./src/pages/MsigUI.vue?vue&type=script&lang=ts:91:27)
Link to my source code: https://github.com/scale-it/algobuilder-msig/blob/815ea0a93504e0d00fff38bc40f67915a4b4eb46/src/pages/MsigUI.vue#L169

Is there any way that I can fix this?
Edited: I can access the the window myalgo wallet connect like below

I have try to enable vue.config.js like this but it still did not work

making https also not help me to solve this issues

Looking around for some similar issues like this. I have found this link: Link
It related to the ref https://github.com/vuejs/core/blob/3ca83179d1a798f65e4e70215c511e2f1b64adb6/packages/reactivity/src/ref.ts#L65-L67

@M-Picco can you check this?

I think I have fixed the issues by myself now.
It seems that myalgo struggle with vuejs and therefore, we need to use shallowref to containt myalgo object!


Virus warning

Hi,
On my Windows 10 device, Smartscreen reports malware. I understand this as it's such a new exe that maybe it doesn't recognise it yet.
Avast shows the following:

Also, virustotal.com shows this:

I hope this is helpful

Hi! The Smartsceen warning happens because the code is self signed. This is expected and normal in this case.
It's super weird that you're getting virus reports. I'll have a look tonight why this could be happening. Thanks for reporting this!

I scanned the file that reported as a virus here and got nothing positive. What exactly did you upload and where did you get it from?

Thank you for taking the time to write this! I uploaded setup.exe, which I got from...
https://hassworkstationstorage.z6.web.core.windows.net/publish/setup.exe
https://www.virustotal.com/gui/file/e1b28927a0a3ed377a0c88b3e33d9859faac93d0c516898693a95a80faa741f3/detection

Thanks for the extra information. This is so strange. If you put the URL to the file in virustotal.com it shows negative but if you download the file and then upload it, it's positive.
This is most likely a false positive as there's only one scanner that finds anything and well, I wrote this thing myself but you probably shouldn't take my word for it. I'll do more research tonight.

Thanks for spending the time and sorry these are my first comments. This must be quite far away from the work you'd rather be doing - the dev itself!

No worries, I appreciate it.

MetaDefender shows the same results with the Zilya! engine finding a threat. VirScan gives the same result.
For me this confirms this as a false positive but others should not take my word for it. I want to be transparent about this so here's what I think is needed.

Building and releasing the app should be automated and transparent. Right now I create builds on my own computer and upload the binaries to GitHub and Windows Azure.
Releases should be signed by a Authenticode codesigning certificate. This would ensure you are using builds created by the automated build process.
There will always be some level of plain trust associated with downloading software from the internet. I'm open to ideas about how to do this well.

Before coming out of beta this should be set up and documented.

Thanks a lot for the update. I was reading earlier about the below, which
may be another way to sign the code.
https://sigstore.dev/what_is_sigstore/
Does GitHub actions allow you to build and sign the code centrally?
Matt
On Tue, 16 Mar 2021, 20:28 sleevezipper, @.***> wrote:

MetaDefender
https://metadefender.opswat.com/results/file/71896573256e6fc746a6819c21701779/hash/multiscan?lang=en
shows the same results with the Zilya! engine finding a threat. VirScan
https://r.virscan.org/language/nl/report/b07b0fd2a41593a56bddc7df393f9d01
gives the same result.
For me this confirms this as a false positive but others should not take
my word for it. I want to be transparent about this so here's what I think
is needed.

Building and releasing the app should be automated and transparent.
Right now I create builds on my own computer and upload the binaries to
GitHub and Windows Azure.
Releases should be signed by a Authenticode codesigning
certificate. This would ensure you are using builds created by the
automated build process.
There will always be some level of plain trust associated with
downloading software from the internet. I'm open to ideas about how to do
this well.

Before coming out of beta this should be set up and documented.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sleevezipper/hass-workstation-service/issues/52#issuecomment-800581388,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACRRJ3M2WM2HRJTQ3KKZW4DTD65NPANCNFSM4ZGVZ4ZA
.

Same for me with Avast

Does GitHub actions allow you to build and sign the code centrally?

I think it does! I'm not sure if it supports the installer we currently use (ClickOnce) but we don't have to use that.

Looks like another AV decided to mark it as a false positive

I've gone ahead and dispatched an email to Zillya and Gridinsoft, requesting that this program signature be removed from their databases.

Same issue my side with Symantec Endpoint Protection..... Very sad I was looking forward to using this with automation to manage office power and lights when my workstation is offline.   I shall find another way.


MATCH keyword callbacks

I am facing some problems in MATCH keyword callbacks. According to our design ,  we can have multiple custom workflows. So if i click on that button workflow summary and all other details should be visible. That i have achieved with pattern matching callbacks.  And i also have to open that function window with selected functions of particular workflow. And whenever someone click on that functions i have to render those graphs. So basically i am using nested match keyword and it is not working properly.  Could you suggest something please ?

I'm looking at this now. If you could include a minimal example of the corresponding layout, that would be helpful. is_open is not a valid attribute for dcc.Tab, dcc.Tabs, dbc.Tab or dbc.Tabs. The closest thing I could find was html.Details, which has an open attribute.

here, when i split things in different callbacks

Thanks @asherp for suggestion ,
But When I splitting things up in multiple callbacks like below,
  collapse_workflow:
    input:
      - id:
          type: custom-workflow-btn
          id: MATCH
        attr: n_clicks
    output:
      - id:
          type: custom-workflow-summary
          id: MATCH
        attr: is_open
    callback: callbacks.workflow_callbacks.show_workflow_summary

open_function_area:
    input:
      - id:
          type: custom-workflow-btn
          id: MATCH
        attr: n_clicks
    output:
      - id:
          type: function-area
          id: MATCH
        attr: style
    callback: callbacks.workflow_callbacks.show_workflow_function

when i tried this its not working for the all callbacks its only going for "show_workflow_summary" callback but its not calling the other callback  "show_workflow_function" that displays functions of clicked workflow .
and i have to make another match for the plotting that is the graph plot according to the checkbox are checked so the problem will come in that also like i mentioned below,
  plot_graph:
    input:
     - id:
          type: workflow-check
          id: MATCH
       attr: checked
    output:
     - id:
         type: graph-parent
         id: MATCH
       attr: children
    state:
     - id: workflow-list-store
       attr: data

    callback: callbacks.workflow_callbacks.show_graph

So could you please suggest something on this?


write Software Management Plan

See "ELIXIR Software Management Plan for Life Sciences": https://biohackrxiv.org/k8znb/

Written a first software development plan based on the eScience-NL template at https://doi.org/10.5281/zenodo.1451750

There is also this checklist: https://doi.org/10.5281/zenodo.1422656

And there is this: https://doi.org/10.37044/osf.io/k8znb


Doesn't work on the newest versions of Atom

It just doesn't work.

Hi @Coconaut11
terminal-plus doesn't seem to be maintained any longer. I maintain a working fork if you'd like to try that instead:
apm install termination
Another fine alternative is platformio-atom-ide-terminal.

Thanks for replying!
De: Fred Barclay
Enviado: jueves, 15 de junio de 2017 21:04
Para: jeremyramin/terminal-plus
CC: Roger Esteve; Mention
Asunto: Re: [jeremyramin/terminal-plus] Doesn't work on the newest versionsof Atom (#488)
Hi @Coconaut11
terminal-plus doesn't seem to be maintained any longer. I maintain a working fork if you'd like to try that instead:
apm install termination
Another fine alternative is platformio-atom-ide-terminal.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Remove label alerts

I don’t think these add very much value…

@tysongach I agree. @kylefiedler what do you think? What's the reason for keeping them?

:+1:


Add initial traefik support

Adding initial traefik support to start wrapping our web services
with a reverse proxy to keep the number of ports being exposed much
simpler.
Closes #93

Currently a WIP, please do not merge.

Hi Leif,
I'm wondering whether traefik makes network easier or not in our docker-compose case.
Currently our docker-compose scenario is mainly for one container host hence nothing is different between with traefic and without traefic. Without traefic, docker-compose expose works fine. Traefic is very useful in case of multi-host container environment, such as docker-swarm, k8s and mesos, so your plan is good for future use case scenario.
So how about to make separate docker-compose.yaml for

one container host scenario  (without traefic)
multi container-host scenario (with traefic)?

@s1061123 (I know we talked today and think we're on the same page, but for historical reference, I'll comment below to your questions above.)
I think for now traefik doesn't really add any complexity that makes it harder to deploy TOAD than it is now, in fact I think it makes it bit simpler since you don't need to know the specific ports for connecting to the services. Right now, you would connect to Jenkins via:  http://toad.tld/jenkins and everything would just route.
If you don't want to use a hostname (specified via your /etc/hosts file for example), then you could override the .env file with your own PROXY_DOMAIN that points at the IP address that you want to listen on (untested).
Overall, I think while it doesn't add a lot to the system as it stands, it is a good tool for helping to future proof a dynamic scaling scenario, and also for the cases where we want to deal with life cycle management, allowing for two containers to live simultaneously behind the proxy, in order to upgrade the software running in that container.


Error when 'Adding Container Support' to existing project

Hi,
I'm trying the use of AKS for the first time and I am trying to continer support to an existing asp.net MVC solution. I have installed the extension but whenever i right click a project -> Add -> Container Orchestration Support and then select Kubernete/Helm i get the following generic error:

I dont know whats causing the error or how to troubleshoot it.
Selecting Docker Compose works fine and if i create a new project using the 'Container Application for Kubernetes' template it works fine.
Any help appreciated,
Andy

Kia ora @Ivers07 - I don't think that message is coming from the VS Code Kubernetes extension.  The commands you're describing aren't part of the VS Code Kubernetes extension, and your screenshot looks like 'full' VS rather than VS Code.  Are you in VS Code, or Visual Studio 20xx?

I'm trying to get this routed to the appropriate team within Microsoft.  Bear with us!

@Ivers07 - This issue belongs at https://github.com/Azure/dev-spaces/issues.  Can you please close this one and open a new issue there?  Ping me on the new issue you create as well.  We're working on making this error message more descriptive.  In the mean time, you can get more detailed info about the underlying error by attaching another instance of VS as a debugger and breaking on exceptions.  Just attach at the point where you're about to click the OK button on the dialog to add the orchestrator support.

Closing this issue, since we found the right home for it.


how to get 12 map？

I run all.sh and get nothing. so I use detetron demo predict code to generate the coco result and the resulte is to low.loading annotations into memory...
Done (t=0.50s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.38s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE (t=11.88s).
Accumulating evaluation results...
DONE (t=2.87s).
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.011
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.020
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.010
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.006
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.023
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.027
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.032
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.032
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.010
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.071

Could send me your training log?

Could send me your training log?

I do not train the model  I just test it on coco val use the weight you provided.

Could send me your training log?

the code I run has been send to you email.

What's your detectron2 version?

What's your detectron2 version?

0.3

Please try this one to install detectron2 0.1 version
python -m pip install detectron2 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

Please try this one to install detectron2 0.1 version
python -m pip install detectron2 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

YOU install MD said that recommend the Pre-Built Detectron2 (Linux only) version with pytorch 1.5. I use the Pre-Built Detectron2 with CUDA 10.1 and pytorch 1.5. and  I found detectron2  V0.1 just for pytorch 1.4 and  use python -m pip install detectron2 -f
https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html can just install V0.3.

Please try this one to install detectron2 0.1 version
python -m pip install detectron2 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

I can just use V0.1.3

/workspace/FewX-master/fewx/modeling/fsod/fsod_roi_heads.py in ()
8
9 from detectron2.config import configurable
---> 10 from detectron2.layers import ShapeSpec, nonzero_tuple
11 from detectron2.structures import Boxes, ImageList, Instances, pairwise_iou
12 from detectron2.utils.events import get_event_storage
ImportError: cannot import name 'nonzero_tuple'

Please try this one to install detectron2 0.1 version
python -m pip install detectron2 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

/workspace/FewX-master/fewx/modeling/fsod/fsod_roi_heads.py in ()
8
9 from detectron2.config import configurable
---> 10 from detectron2.layers import ShapeSpec, nonzero_tuple
11 from detectron2.structures import Boxes, ImageList, Instances, pairwise_iou
12 from detectron2.utils.events import get_event_storage
ImportError: cannot import name 'nonzero_tuple'

I tried the evaluation again and I can get the reported number.
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Saving results to ./output/fsod/finetune_dir/R_50_C4_1x/inference/coco_instances_results.json
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 10.44 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 1.24 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.056
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.029
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.031
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.052
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.047
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.059
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.114
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 2.989 | 5.592  | 2.948  | 0.724 | 3.057 | 5.165 |
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 11.95
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 22.37
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 11.79
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 2.89
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 12.23
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 20.66

I tried the evaluation again and I can get the reported number.
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Saving results to ./output/fsod/finetune_dir/R_50_C4_1x/inference/coco_instances_results.json
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 10.44 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 1.24 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.056
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.029
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.031
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.052
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.047
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.059
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.114
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 2.989 | 5.592  | 2.948  | 0.724 | 3.057 | 5.165 |
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 11.95
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 22.37
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 11.79
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 2.89
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 12.23
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 20.66

you env is pytorch 1.5 . detectron2 v0.31 ? just run all.sh can get the result?

I tried the evaluation again and I can get the reported number.
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Saving results to ./output/fsod/finetune_dir/R_50_C4_1x/inference/coco_instances_results.json
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 10.44 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 1.24 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.056
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.029
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.031
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.052
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.047
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.059
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.114
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 2.989 | 5.592  | 2.948  | 0.724 | 3.057 | 5.165 |
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 11.95
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 22.37
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 11.79
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 2.89
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 12.23
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 20.66

If I run all.sh to evaluate the model all I get just this and end.
[12/30 22:20:19 fewx.modeling.fsod.fsod_rcnn]: =========== Offline support features are generated. ===========
[12/30 22:20:19 fewx.modeling.fsod.fsod_rcnn]: ============ Few-shot object detetion will start. =============

I tried the evaluation again and I can get the reported number.
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Saving results to ./output/fsod/finetune_dir/R_50_C4_1x/inference/coco_instances_results.json
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 10.44 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 1.24 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.056
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.029
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.031
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.052
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.047
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.059
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.114
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 2.989 | 5.592  | 2.948  | 0.724 | 3.057 | 5.165 |
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 11.95
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 22.37
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 11.79
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 2.89
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 12.23
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 20.66

in your code fewx/modeling/fsod/fsod_rcnn.py 314   there is  sys.exit(0)   I don't understand why. so I just delete it and meet
[12/31 01:24:21 fewx.modeling.fsod.fsod_rcnn]: ===========inference call===========
Traceback (most recent call last):
File "fsod_train_net.py", line 121, in
args=(args,),
File "/opt/conda/envs/21de/lib/python3.6/site-packages/detectron2/engine/launch.py", line 59, in launch
daemon=False,
File "/opt/conda/envs/21de/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 200, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/opt/conda/envs/21de/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
while not context.join():
File "/opt/conda/envs/21de/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 119, in join
raise Exception(msg)
Exception:
-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/opt/conda/envs/21de/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap
fn(i, *args)
File "/opt/conda/envs/21de/lib/python3.6/site-packages/detectron2/engine/launch.py", line 94, in _distributed_worker
main_func(*args)
File "/workspace/FewX-master/fsod_train_net.py", line 103, in main
res = Trainer.test(cfg, model)
File "/opt/conda/envs/21de/lib/python3.6/site-packages/detectron2/engine/defaults.py", line 516, in test
results_i = inference_on_dataset(model, data_loader, evaluator)
File "/opt/conda/envs/21de/lib/python3.6/site-packages/detectron2/evaluation/evaluator.py", line 141, in inference_on_dataset
outputs = model(inputs)
File "/opt/conda/envs/21de/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/workspace/FewX-master/fewx/modeling/fsod/fsod_rcnn.py", line 128, in forward
return self.inference(batched_inputs)
File "/workspace/FewX-master/fewx/modeling/fsod/fsod_rcnn.py", line 350, in inference
for cls_id, res4_avg in self.support_dict['res4_avg'].items():
File "/opt/conda/envs/21de/lib/python3.6/site-packages/torch/nn/modules/module.py", line 772, in getattr
type(self).name, name))
torch.nn.modules.module.ModuleAttributeError: 'FsodRCNN' object has no attribute 'support_dict'

I tried the evaluation again and I can get the reported number.
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Saving results to ./output/fsod/finetune_dir/R_50_C4_1x/inference/coco_instances_results.json
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 10.44 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 1.24 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.056
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.029
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.031
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.052
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.047
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.059
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.114
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 2.989 | 5.592  | 2.948  | 0.724 | 3.057 | 5.165 |
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 11.95
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 22.37
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 11.79
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 2.89
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 12.23
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 20.66

I find the problem..... In your all.sh you do delete the ./support_dir/support_feature.pkl which inference needed ... plz delete it in your all.sh

I tried the evaluation again and I can get the reported number.
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Saving results to ./output/fsod/finetune_dir/R_50_C4_1x/inference/coco_instances_results.json
[12/30 18:13:39 fewx.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 10.44 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 1.24 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.030
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.056
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.029
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.031
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.052
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.047
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.066
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.059
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.114
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 2.989 | 5.592  | 2.948  | 0.724 | 3.057 | 5.165 |
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 11.95
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 22.37
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 11.79
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 2.89
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 12.23
[12/30 18:13:51 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 20.66

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.019
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.035
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.019
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.009
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.041
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.036
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.041
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.041
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.013
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.096
[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for bbox:

AP
AP50
AP75
APs
APm
APl

1.939
3.469
1.948
0.015
0.889
4.124

[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP  : 7.75

[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP50: 13.88

[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> AP75: 7.79

[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APs : 0.06

[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APm : 3.56

[12/31 01:46:01 fewx.evaluation.coco_evaluation]: Evaluation results for VOC 20 categories =======> APl : 16.50

I know that the model generates 100 BBoxes for each class, but when I predicted that only dozens of BBoxes were generated for each class. Why not 100 and how to solve this problem?


[Question]: Ingesting Powerpoints with graphs and Images

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
HI there!
I am curious on how to handle PowerPoints that contain images and graphs besides texts. What is the most efficient way to embed the information.
I know we can use a multimodal model that can analyze the images and provide descriptions of what it sees, but is there a better more efficient way to do this in llama-index?
Or is it better to convert the pptx into a pdf and ingest it that way? Do PDF images get handled better?
Thanks!

Have you seen this example?
https://github.com/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_rag_slide_deck.ipynb

@logan-markewich  I hadn't! Thanks for sharing it :) I will take a look now :)

@logan-markewich  Hi there! I read through it but I am curious... Is there a way to not have to use Llama-parse?


feat: adding requiresTime type

💔 Breaking Changes
🏆 Enhancements
Adds typing for the requiresType property, on ControlPanelConfig, in the chart-controls pkg.
📜 Documentation
🐛 Bug Fix
🏠 Internal

I see this property defined for a lot of charts, but couldn't find it in the actual code---bother frontend or backend. Could you point me to the code piece where it was used if there's any? (Otherwise maybe cleanup?)

You're right, I'm not seeing its purpose either. Maybe @villebro has some insight here.

I think you may be right... it seems like the plugin works just fine without the requireTime param. I'll just leave this PR hanging out here for a little while, press on with the generator PR, and maybe we can go through and I'll open a PR deleting requireTime everywhere, to see how that goes.

I've never seen it being used anywhere, so AFAIK it's probably ok to remove it.

Per discussion, removed it instead


Introduce endpoint that returns possible reasons a job isn't scheduled

@mforsyth can you rebase when you get a chance?

@mforsyth, when you get a chance, can you rebase this and add a note about the feature in the changelog? I think this is ready to go

@wyegelwel rebased.

This LGTM, @pschorf, any additional comments?


Manifest updates

need to rebase and merge on https://github.com/splunk/splunk-connect-for-kubernetes/pull/46

Fixed


Image URLs without file extensions break search views

Parent issue: https://github.com/creativecommons/cccatalog-api/issues/509, introduced by https://github.com/creativecommons/cccatalog-api/pull/460
The thumbnail proxy expects a file extension when there is none to be found for some images from certain upstream sites (like digitaltmuseum). We should not include the file extension in thumbnail URLs at all; browsers can figure out that it's an image based on the content type headers.
To fix this:

Update _post_process_results to not parse file extensions.
Update /v1/thumbs to not expect URLs to have a file extension.

Internal Server Error: /v1/images
Traceback (most recent call last):
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/rest_framework/views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/rest_framework/views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
    raise exc
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/rest_framework/views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "./cccatalog/api/views/image_views.py", line 89, in get
    page=page_param
  File "./cccatalog/api/controllers/search_controller.py", line 349, in search
    filter_dead
  File "./cccatalog/api/controllers/search_controller.py", line 135, in _post_process_results
    'identifier': "{}.{}".format(res["identifier"], ext)
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/urls/base.py", line 90, in reverse
    return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
  File "/var/www/cccatalog-api/.venv/lib/python3.7/site-packages/django/urls/resolvers.py", line 673, in _reverse_with_prefix
    raise NoReverseMatch(msg)
django.urls.exceptions.NoReverseMatch: Reverse for 'thumbs' with keyword arguments '{'identifier': '4b17bb11-e2f8-4210-ad64-c6aeb29e6ff4.org/image/03347xEG5oUk?dimension=1200x1200'}' not found. 1 pattern(s) tried: ['v1/thumbs/(?P<identifier>[^/]+)$']

Hello, I am working on this issue. How should I test if the changes I made are working?


[TW#12186] LWIP socket loopback support

This one is going to take a little explaining.  I am working with a 3rd party open source project (Mongoose).  That uses an interesting signaling technique.
Imagine we have two sockets that are connected to each other within ESP32.  In principle, if we write on one, we can read on the other.  Now imagine two FreeRTOS tasks... call one "reader" and the other "writer"
The reader does:
void reader() {
   recv(sockReader, buffer, sizeofbuffer, 0);
}

while the writer does:
void writer() {
   send(sockWriter, buffer, size, 0);
}

Again, these execute on distinct tasks and we have ensured that reader() is running before writer().
When we run the code, we find that the writer() sends the request but reader() doesn't wake up/un-block.  It is as though the message is not delivered.
The sockets are connect to each other with the following initialization:
 sockReader = sockWriter = -1;

  (void) memset(&sa, 0, sizeof(sa));
  sa.sin_family = AF_INET;
  sa.sin_port = htons(0);
  inet_pton(AF_INET, "<IP_ADDRESS>", &sa.sin_addr.s_addr);

  sockReader = socket(AF_INET, SOCK_DGRAM, 0);
  rc = bind(sockReader, (struct sockaddr *)&sa, len);
  rc = getsockname(sockReader, (struct sockaddr *)&sa, &len);
  sockWriter = socket(AF_INET, SOCK_DGRAM, 0);
  rc = connect(sockWriter, (struct sockaddr *)&sa, len);
  rc = getsockname(sockWriter, (struct sockaddr *)&sa, &len);
  rc = connect(sockReader, (struct sockaddr *)&sa, len);

The real world code snippet within Mongoose can be found here:
https://github.com/cesanta/mongoose/blob/master/mongoose.c#L3706
Attached to this issue is my full main.c that I used to test.
main_c.txt

http://www.nongnu.org/lwip/2_0_0/group__lwip__opts__loop.html

That's a good thought ... thank you kind sir.  I tried compiling lwip by adding:
CFLAGS += -DLWIP_NETIF_LOOPBACK=1

But that seemed to cause an almost immediate crash.  However I do think you are on to something.  If we didn't have a <IP_ADDRESS> address, that would produce a story similar to the experienced results in the initial problem.

I think you need at least LWIP_LOOPBACK_MAX_PBUFS and possibly other configuration

I think that puts me out of my depth ... hopefully someone with LWIP skills can assist.

Well if you look here those two might be enough:
https://github.com/espressif/esp-idf/blob/d467eabadbc90aa0703d3e131e5751f9d09ebdf8/components/lwip/include/lwip/port/lwipopts.h#L330

I tried setting those two by adding the following into my component.mk:
CFLAGS += -Wno-address -Wno-unused-variable -Wno-unused-but-set-variable -DLWIP_NETIF_LOOPBACK=1 -DLWIP_LOOPBACK_MAX_PBUFS=8

For the lwip component.  Unfortunately it still resulted in a crash at startup as opposed to when I remove them and no crash.  I don't disagree at all that these may be part of the solution, and hopefully someone will get to this issue to comment on it who has internals knowledge and experience.

Solution that worked for me, this needs to be added into the main project Makefile
## LWIP settings
CFLAGS += -DLWIP_HAVE_LOOPIF=0
CFLAGS += -DLWIP_NETIF_LOOPBACK=1
CFLAGS += -DLWIP_NETIF_LOOPBACK_MULTITHREADING=1

This actually enabled me to do

bind to <IP_ADDRESS>
connect to self using device local IP address

@nathanjel Howdy ... I tried setting these values in my component.mk file for the lwip component ... unfortunately it resulted in a panic crash during startup.

@nkolban As far as I am concerned it should not go to component.mk for LWIP. I have it in the main Makefile  in my project's root folder. My esp-idf is left unmodified. Could You try the same and then re-build Your project.

@nathanjel
Thank you kind sir.  Your original instructions were clear and I had messed up.  I changed my main component.mk and all now compiles and runs correctly.  I haven't yet had the opportunity to test the loopback logic ... but so far so good.  Thank you again.

@nathanjel One more thought ... if I may ... in your post, you bound to <IP_ADDRESS> and the devices local IP address ... should I read this to mean that you did NOT test with <IP_ADDRESS>?

@nkolban
This is how i bind:
char * s_address = "<IP_ADDRESS>:1883";
netfsm->mqtt_server_cn = mg_bind(&mgr, s_address, mg_mqtt_broker);

and then this is how I connect - first i get the local IP from the event, and later i use it to connect locally
esp_err_t network_event_handler(void *ctx, system_event_t *event)
{
	switch (event->event_id) {
	case SYSTEM_EVENT_STA_GOT_IP:
		MN_Status.myipinfo = event->event_info.got_ip.ip_info;

sprintf(s_address, "%s:%d", inet_ntoa(localmq),
		        MQTT_INSECURE_PORT_NUMBER);
cptr = mg_connect_opt(&mgr, s_address, mqtt_client_event_handler, opts);

I tried multiple approaches before, don't remember exactly how <IP_ADDRESS> did behave

Is LOOPBACK available now?  After reading all the comments above, I am messed up. Who can make a summary about how to use this feature. Thanks.

@young-mu it works for me very well as per the settings and code above (using Moongose). I have not tried it with lwip sockets directly.

I am getting compiler error.
error: implicit declaration of function 'mg_broadcast' on esp32.
on windows platform it build. but mg_broadcast does do nothing.

namespace
{
	static void ev_handler(struct mg_connection *nc, int ev, void *ev_data)
	{

		switch (ev) {
		case MG_EV_POLL:
			break;
		case MG_EV_SEND:
			break;
		case MG_EV_ACCEPT:
			break;
		case MG_EV_CONNECT:
			break;
		case MG_EV_RECV:
			break;
		case MG_EV_CLOSE:
			break;
		default:
			break;
		}
	}

}

void sendUDPMessages()
{
	ACDTRACEF(ES_NoError, "Enter sendUDPMessages \n");

	struct mg_mgr mgr;
	mg_mgr_init(&mgr, NULL);

	struct mg_connect_opts opts;

	memset(&opts, 0, sizeof(opts));
	opts.flags = MG_F_ENABLE_BROADCAST;
	opts.iface = nullptr;

	const char *port1 = "udp://1457", *port2 = "udp://<IP_ADDRESS>:1457";
	const char *port3 = "udp://<IP_ADDRESS>:1457";

	struct mg_connection * nc_client = mg_connect_opt(&mgr, port3, ev_handler, opts);

	bool stop = false;
	std::thread thread = std::thread([&]() {
		while (!stop) {
			mg_mgr_poll(&mgr, 1);
		}
	});

	for (int i = 1; i <= 10; i++)
	{
		std::string  send_msg = "hello";
		mg_broadcast(&mgr, ev_handler, (void*)send_msg.c_str(), send_msg.size());
	}

	std::this_thread::sleep_for(std::chrono::seconds(3));

	stop = true;
	if (thread.joinable()) {
		thread.join();
	}

	mg_mgr_free(&mgr);

}

Here is my example code.

I have created a simple example. It may be helpful for others visiting this thread.
https://github.com/ddomnik/ESP-localhost-loopback-example/tree/main


Allow Navbar to accept collapseOptions = false

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Fixed issues
https://github.com/yiisoft/yii2-bootstrap5/issues/62

Could you please add an unit test?

Done @simialbi!


Netif PPP

Allow non-static callbacks for EspNetifDriver
Change T: Borrow<EspNetif> to T: BorrowMut<EspNetif> which is the correct one, as the driver mutates the netif
Remove the pesky got_ip_event_id / lost_ip_event_id from the constructor functions; these are extracted directly from the wrapped EspNetif
Add explicit start/stop/is_started methods to make the API more flexible, and to match more closely the API of EspEth(Driver) and EspWifi(Driver)
Expose EspNetifDriver::netif and EspNetifDriver::netif_mut
Map PPP event loop events to PppEvent
Implement PppConfiguration as well as EspNetif::set_ppp_conf / EspNetif::get_ppp_conf
Remove new_slip and new_ppp constructor function variants now that there is a safe set_ppp_conf / get_ppp_conf API
Convert got/lost event IDs to Option<NonZeroU32> which is equivalent to u32 where 0 means no event
Implement PppAuthentication as well as EspNetif::set_ppp_auth

Thanks for working on it! Looks good now.
@DaneSlattery since you started the complete PPP inclusion feel free to also have a look.

Dane is in the root of this PR. :) My changes are on top of his ones (PPP events mapping and non-static netif driver).
I'll merge. I think the new EspNetifDriver is strictly more powerful than the previous version, so I don't anticipate API drama. If there are any bugs, we'll fix later.


OPA gatekeeper customize button color makes it look like disabled.

What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):

Create a cluster, navigate to dashboard

Result:
Customize button appears as it is disabled.

Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): master-head - 1575d706a
Installation option (single install/HA): Single

Button color needs to be addressed not just for OPA but all create pages where Cancel button also looks like this (Looks disabled) in both light and dark.

@vincent99, @lvuch  and I discussed and decided to change the focus color so we can use the blue border. Sending back to lauren for color decision and implementation.

Fixed