Import Topic with CorrelationId Rules throws exception. - fixed

I exported a topic with multiple subscriptions with CorrelationId filter rules. On importing it throws exception.

:white_check_mark: Build ServiceBusExplorer 4.2.0-PullRequest0395.122.build.608 completed (commit https://github.com/paolosalvatori/ServiceBusExplorer/commit/08628180c1 by @androdd)

@androdd , due to the merging of another PR you need to resolve a merge conflict in a file for this PR.

I don't have write access so I can't resolve the conflicts.

@androdd conflicts should be resolved by re-basing your branch on top of the updated develop.

I don't have a branch. I forked the repo to be able to create a pull request.
The bug is fixed by changing one letter in a string - from "i" to "I".
This whole stuff is taking too much time now. If somebody wants the import to work  please fix it. I don't need my name to be on this fix.
Best regards

When you forked the repo, you have created a branch under your repo called develop.
. The link is https://github.com/androdd/ServiceBusExplorer.
Alternatively, you could delete the clone/close this PR and redo the change to avoid conflicts in case rebasing is complicating matters. This is not about fame but helping. If you're still in, great. Otherwise, would you mind raising an issue instead of this PR so someone else could pick it up? Thank you, @androdd.

@androdd , the ImportExportHelper.cs file has been moved to another project and thus another location within the repo. If you want to I can create a PR with the changes you did.

Yes
Please, do. If this fix is still relevant at all.
Thank you.
On Mon, Dec 16, 2019 at 10:02 AM Erik Mogensen<EMAIL_ADDRESS>wrote:

@androdd https://github.com/androdd , the ImportExportHelper.cs file
has been moved to another project and thus another location within the
repo. If you want to I can create a PR with the changes you did.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/paolosalvatori/ServiceBusExplorer/pull/395?email_source=notifications&email_token=ANYTKUKIYM6UECXL7AWPRYTQY4Y3BA5CNFSM4JYZ6Z2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG53EAI#issuecomment-565948929,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANYTKUJV4HMWSVTYHX4CCH3QY4Y3BANCNFSM4JYZ6Z2A
.

Replaced by #398


Simplify unit test display names

Closes #82
There is already a class DisplayNameGenerator.ReplaceUnderscores.class handling underscores but this case is more complex than replacing underscore with space.

[x] Implement core logic
[ ] Make String replaceCamelCaseAndUnderscoreAndNumber(String input) private
[ ] Use static final DisplayNameGenerator INSTANCE = new ReplaceCamelCaseAndUnderscoreAndNumber()
[ ] Update tests
[ ] Update TESTING-GUIDELINE

Ok thanks @KyleAure for your reviews.  I will apply them.

Thank you for this PR!
I have an uncomfortable, but honest question: what's the main benefit of this PR?
The @DisplayName method sometimes will have the same method name, and its intent is to provide better naming for reports and explanatory purposes.

Thank you for this PR! I have an uncomfortable, but honest question: what's the main benefit of this PR?
The @DisplayName method sometimes will have the same method name, and its intent is to provide better naming for reports and explanatory purposes.

Thanks @eliasnogueira for sharing your thought. As you said, sometimes there is a pattern, instead of duplicating @DisplayName, we will just use @DisplayNameGeneration. at class level. Then the developer  will only use @DisplayName in case of a naming not matching the pattern or wanting to customize display name.
About method name transformation, sure we could use the method name itself but  readability may decrease.
Let's illustrate it better, shouldReturn5Errors is really easy to read but what about this one shouldReturnTheNumberOfErrorsAs_numberOfErrors_InferiorOrEqualTo5 ? Is it not better Should return the number of errors as numberOfErrors inferior or equal to 5 ?

To conclude,  we are just doing the same thing as inner  classes of DisplayNameGenerator.class do but with custom rules.

Thank you for this PR! I have an uncomfortable, but honest question: what's the main benefit of this PR?
The @DisplayName method sometimes will have the same method name, and its intent is to provide better naming for reports and explanatory purposes.

Thanks @eliasnogueira for sharing your thought. As you said, sometimes there is a pattern, instead of duplicating @DisplayName, we will just use @DisplayNameGeneration at class level. Then the developer will only use @DisplayName in case of a naming not matching the pattern or wanting to customize display name.
About method name transformation, sure we could use the method name itself but readability may decrease.
Let's illustrate it better, shouldReturn5Errors is really easy to read but what about this one shouldReturnTheNumberOfErrorsAs_numberOfErrors_InferiorOrEqualTo5 ? Is it not better Should return the number of errors as numberOfErrors inferior or equal to 5 ?

I got your point.
I personally like to write the test name output in the @DisplayName rather than write it in the method name itself.
Using your example, both are the same and readable:

shouldReturnTheNumberOfErrorsAs_numberOfErrors_InferiorOrEqualTo5
Should return the number of errors as numberOfErrors inferior or equal to 5

Thank you for this PR! I have an uncomfortable, but honest question: what's the main benefit of this PR?
The @DisplayName method sometimes will have the same method name, and its intent is to provide better naming for reports and explanatory purposes.

Thanks @eliasnogueira for sharing your thought. As you said, sometimes there is a pattern, instead of duplicating @DisplayName, we will just use @DisplayNameGeneration at class level. Then the developer will only use @DisplayName in case of a naming not matching the pattern or wanting to customize display name.
About method name transformation, sure we could use the method name itself but readability may decrease.
Let's illustrate it better, shouldReturn5Errors is really easy to read but what about this one shouldReturnTheNumberOfErrorsAs_numberOfErrors_InferiorOrEqualTo5 ? Is it not better Should return the number of errors as numberOfErrors inferior or equal to 5 ?

To be honest, this is kind of overengineeing. I really appreciate your time and effort added here, trying to contribute. On the other hand, you added something that's already a feature from JUnit 5.
We can achieve the same thing you implemented using the Setting the Default Display Name Generator from the official JUnit 5 documentation, where we can simply set a property to have the same result you developed.
In your opinion, would you go toward a custom implementation, that already exists as a supported feature from the library, or would you use the library feature?
Going with the first approach will generate more work and maintainability from the project side compared to using exactly the same feature that the framework already provides.
I really appreciate your time and effort, in trying to contribute, but I cannot see any tangible benefit in this PR.
@otaviojava @KyleAure any thoughts?

@eliasnogueira Thanks for your feedback.  I was hoping for this type of feedback when I opened issue #82.  I just thought it was weird that we were using @DisplayName to convey duplicate information that was already described within the test name itself.  So having some sort of way to reduce the redundancy was my goal in opening the issue.
Since I never got any feedback I figured I was the only one who was bothered by it :)
We have 3 options:

keep things the way they are now, some what redundant but not over engineered.
switch to using ReplaceUnderscores which is built-in to JUnit but makes test method names a bit ugly.
use the custom DisplayNameGenerator from @FanJups

I'm fine with any of these options.

@KyleAure somehow I missed the issue :(
I would go with item 2, adding the property to use the ReplaeUnderscors without adding any additional code, so we can provide both ways: explicitly adding the @DisplayName which will have the priority even with the feature enabled and letting JUnit generate it in the case we don't add.

I think we could keep this custom generator and the dev will use it or not. At least, the dev will have another option.

@KyleAure , please are you availabl these days for a call to help me figuring out how to solve setup issues.  I sent you an email last week.

The custom generator is not ready to be used because it should be registered first by updating JUnit config.
This error occurs when it is used:  TestEngine with ID 'junit-jupiter' failed to discover tests
I Will figure out how to solve this new issue.

To be honest, this is kind of overengineeing. I appreciate your time and effort added here, trying to contribute. On the other hand, you added something that's already a feature from JUnit 5.
We can achieve the same thing you implemented using the Setting the Default Display Name Generator from the official JUnit 5 documentation, where we can simply set a property to have the same result you developed.
In your opinion, would you go toward a custom implementation, that already exists as a supported feature from the library, or would you use the library feature?

@eliasnogueira  sorry for replying late, the main goal is not to make this custom generator the default one, I am thinking about a way to let the developer choose to use it or not.
Yesterday, I noticed that my tests were always failing because I was using that annotation.  I got this message so many times:
TestEngine with ID 'junit-jupiter' failed to discover tests
Commenting that annotation made tests pass.
After searching, I also got that JUnit link you shared helping me to understand that I have not yet registered this new custom generator. That's a good point but I don't want  to make it default.  I just want JUnit to know there is a new generator available and not complain that it fails to discover tests.
I agree with you that there is no need to use that annotation after setting the default generator through properties.
About readability, it's a little bit subjective because I wear glasses, I feel like the longer the method name, the more complicated it is to read as display name.  We have our own preferences, this is another reason to not make it default.
To sum up, I am thinking about a way to register this new generator without making it default as if this generator is a JUnit built-in generator (like Standard, Simple, ReplaceUnderscores and IndicativeSentences) then not required to be set as default to be used.

After uncommenting annotation and paying attention to the logs, I am on the right path to avoid making it as default but have to solve this module issue
Internal Error occurred.
org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160)
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:132)
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:107)
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:78)
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:99)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
	at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: org.junit.platform.commons.JUnitException: MethodSelector [className = 'jakarta.data.displaynamegeneration.test.ReplaceCamelCaseAndUnderscoreAndNumberTest', methodName = 'shouldReturnMethodDisplayNamesForCamelCaseAndUnderscoreAndNumber', parameterTypes = 'java.lang.String,java.lang.String', classLoader = null] resolution failed
	at org.junit.platform.launcher.listeners.discovery.AbortOnFailureLauncherDiscoveryListener.selectorProcessed(AbortOnFailureLauncherDiscoveryListener.java:39)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:103)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.run(EngineDiscoveryRequestResolution.java:83)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolver.resolve(EngineDiscoveryRequestResolver.java:113)
	at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:46)
	at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:69)
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152)
	... 13 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make public jakarta.data.displaynamegeneration.ReplaceCamelCaseAndUnderscoreAndNumber() accessible: module jakarta.data does not "exports jakarta.data.displaynamegeneration" to unnamed module @7b69c6ba
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:188)
	at java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:181)
	at org.junit.platform.commons.util.ReflectionUtils.makeAccessible(ReflectionUtils.java:1770)
	at org.junit.platform.commons.util.ReflectionUtils.newInstance(ReflectionUtils.java:553)
	at org.junit.platform.commons.util.ReflectionUtils.newInstance(ReflectionUtils.java:528)
	at org.junit.jupiter.engine.descriptor.DisplayNameUtils.lambda$findDisplayNameGenerator$6(DisplayNameUtils.java:140)
	at java.base/java.util.Optional.map(Optional.java:260)
	at org.junit.jupiter.engine.descriptor.DisplayNameUtils.findDisplayNameGenerator(DisplayNameUtils.java:127)
	at org.junit.jupiter.engine.descriptor.DisplayNameUtils.lambda$createDisplayNameSupplier$5(DisplayNameUtils.java:117)
	at org.junit.jupiter.engine.descriptor.DisplayNameUtils.determineDisplayName(DisplayNameUtils.java:89)
	at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.<init>(JupiterTestDescriptor.java:69)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.<init>(ClassBasedTestDescriptor.java:101)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.<init>(ClassTestDescriptor.java:51)
	at org.junit.jupiter.engine.discovery.ClassSelectorResolver.newClassTestDescriptor(ClassSelectorResolver.java:119)
	at org.junit.jupiter.engine.discovery.ClassSelectorResolver.lambda$resolve$0(ClassSelectorResolver.java:71)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution$DefaultContext.createAndAdd(EngineDiscoveryRequestResolution.java:250)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution$DefaultContext.addToParent(EngineDiscoveryRequestResolution.java:213)
	at org.junit.jupiter.engine.discovery.ClassSelectorResolver.resolve(ClassSelectorResolver.java:71)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$resolve$2(EngineDiscoveryRequestResolution.java:135)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1602)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:189)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:126)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.access$100(EngineDiscoveryRequestResolution.java:58)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution$DefaultContext.resolve(EngineDiscoveryRequestResolution.java:228)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution$DefaultContext.addToParent(EngineDiscoveryRequestResolution.java:222)
	at org.junit.jupiter.engine.discovery.MethodSelectorResolver$MethodType.resolve(MethodSelectorResolver.java:208)
	at org.junit.jupiter.engine.discovery.MethodSelectorResolver$MethodType.access$300(MethodSelectorResolver.java:164)
	at org.junit.jupiter.engine.discovery.MethodSelectorResolver.lambda$resolve$0(MethodSelectorResolver.java:93)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.junit.jupiter.engine.discovery.MethodSelectorResolver.resolve(MethodSelectorResolver.java:97)
	at org.junit.jupiter.engine.discovery.MethodSelectorResolver.resolve(MethodSelectorResolver.java:75)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$resolve$2(EngineDiscoveryRequestResolution.java:150)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1602)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:189)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:126)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:92)
	... 18 more

Process finished with exit code -2

About readability, it's a little bit subjective because I wear glasses, I feel like the longer the method name, the more complicated it is to read as display name. We have our own preferences, this is another reason to not make it default.

@eliasnogueira , I just want to come back on readability by sharing this Igorski's article.

Test names usually tend to become very long and difficult to read. Depending on the naming convention you use, you could have some mandatory words like "should" or "test" or you could be using snake case. All of that contributes to longer names which makes the test results difficult to read in test reports, in IDEs and build tools.

Hi @KyleAure , please I need your help to Solve the module issue related to the usage of @DisplayNameGeneration(ReplaceCamelCaseAndUnderscoreAndNumber.class) - TestEngine with ID 'junit-jupiter' failed to discover tests

I was able to resolve the TestEngine with ID 'junit-jupiter' failed to discover tests by:

Moving the test classes to the same directory as the generator class
Removing @Disabled

Even with that the reporter did not use the generator.  The surefire plugin needs to be configured to use the new generated names:
    <build>
        <plugins>
            <plugin>
	          <groupId>org.apache.maven.plugins</groupId>
	          <artifactId>maven-surefire-plugin</artifactId>
	          <version>3.2.1</version>
	          <configuration>
		      <statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
		          <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
		          <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
		      </statelessTestsetReporter>
	          </configuration>
            </plugin>
        </plugins>
	</build>

It also seems like @ParameterizedTest tests are excluded from DisplayNameGeneration.
Since this @DisplayNameGeneration doesn't seem to be a viable, nor well supported way to accomplish this task I do not think it would be worthwhile exploring this any further.  I think updates to JUnit 5 would be more appropriate.

Ok thanks @KyleAure for your review. I will propose this custom generator to JUnit team.  I had the same idea weeks ago but I wanted to do it here instead of JUnit side. Finally, looks like the best solution.

This feature will be implemented by JUnit Pioneer https://github.com/junit-pioneer/junit-pioneer/pull/819


Account and API Keys: Although token for the current session is deleted an error is thrown

Setup

Rancher version: v2.10
Rancher UI Extensions:
Browser type & version: FF, Chrome

Describe the bug
Although the token for the current session is successfully deleted an error is thrown.

Cannot delete token for current session. Use logout instead"

To Reproduce

log in to dashboard
go to Account and API Keys
delete token for the current session
notice error in modal
refresh page
notice token is deleted

Result
Although the token for the current session is successfully deleted an error is thrown.
Expected Result
Either do not allow deletion of the token or allow it without the error.

Screenshots
https://github.com/user-attachments/assets/ac47724e-94a7-4b57-af0e-c239c903d1b1

Additional context

Just as a quick note for reproducing. If you don't see a ui session token just logout and log back in a couple times.


Add a classmethod to load a trained model

The current process of loading a trained model can be quite challenging for newcomers (cf), this can be both frustrating and discouraging for them.
SMIT should provide a classmethod that would automatically load a trained model given a path to a safe_tensor and the config file used to generate the model.
Ultimately it would great to not have to provide the config file by maybe storing everything into the safe_tensor file.

It appears that Trainer currently does not provide a way to add metadata to the saved model.safetensors file.
A work-around would be to load the saved model.safetensors file, add the metadata, and re-save it again. It will add some processing time at the end of each save_steps but it will result in a more friendly way to load a train model.
A user will be able to simply provide a tensor tensor and SMIT could automatically adapt the model (LoRA / Quant) depending the config file provided.
As a fallback in case the safetensor file does not contain the config file in its metadata, SMIT should also provide a method that will takes as input both the safetensor and a config as dict/file.


Private Projects

Is it possible to mark a uv project as private such that uv publish is completely disabled? For private code this is a desirable feature.
Something like:
[tool.uv]
private = true

NPM has a similar "private": true:

https://docs.npmjs.com/cli/v10/configuring-npm/package-json#private

If you set "private": true in your package.json, then npm will refuse to publish it.
This is a way to prevent accidental publication of private repositories.

In Cargo, this is controlled by the publish flag.

To prevent a package from being published to a registry (like crates.io) by mistake, for instance to keep a package private in a company, you can omit the version field. If you’d like to be more explicit, you can disable publishing:

[package]
# ...
publish = false

For some additional motivation, see a similar feature request for Poetry

https://github.com/python-poetry/poetry/issues/1537

This is mentioned over in https://github.com/python-poetry/poetry/issues/1537#issuecomment-1154642727 — we might want to include a private mode that avoids attempting to upload to the registry at all.
cc @konstin

@zanieb Yeah, I personally feel like the Private :: Do Not Upload classifier is not 100% ideal and having something built in at the package manager level like NPM or Cargo would be more reliable (and also probably easier to teach).

I'd love to support something like tool.uv.private = true. The only blocker for this is that currently, uv publish is independent of pyproject.toml: We will upload the files you hand uv publish and if you hand it a wheel, there's no pyproject.toml anymore. We also aren't allowed to set "Private :: Do Not Upload" for you when tool.uv.private = true is set, since we have to respect project.classifiers.
For now, we should start hinting users towards the Private :: Do Not Upload classifier in the docs. It's not the most elegant solution, but it's a reliable way to prevent accidental publishing.

The only blocker for this is that currently, uv publish is independent of pyproject.toml: We will upload the files you hand uv publish and if you hand it a wheel, there's no pyproject.toml anymore.

I understand that in some contexts we may not know where the wheels come from, but I feel like we should still be able to read project metadata and configuration when publishing artifacts for builds in a project. This seems like a fairly important base capability for uv publish.

We also aren't allowed to set "Private :: Do Not Upload" for you when tool.uv.private = true is set, since we have to respect project.classifiers.

How "not allowed" is this? Couldn't we append this when building the wheel and source dists?

I agree with @zanieb on this. It's fairly important to protect users against shooting themselves in the foot when they accidentally attempt to publish something that is marked as private.
As a user, I can provide the following perspective/expected workflow.

If a project is created with uv init, add the private = true by default. The vast majority of projects are not public. Being proactive here brings less harm than the contrary;
If a project has the "Private :: Do Not Upload" classifier, warn the user that it must be removed in order to continue with publishing;
If a project is marked with private = true and the "Private :: Do Not Upload" classifier is not specified, warn the user that they should add it, link to https://pypi.org/classifiers/, and do not publish;
If a project is marked with private = true and the "Private :: Do Not Upload" classifier is specified, warn the user that it is set and therefore, do not publish.

This respects the apparent rule of not setting the classifier automatically while still protecting the user as much as possible. I strongly encourage not adding any --override parameters either. Those usually are a bad idea for users that blindly use them without understanding the consequences and reasons for their errors.

There's two options to implementing private project: METADATA and pyproject.toml.
Metadata
One option is going through the METADATA file and making sure "Private :: Do Not Upload" ends up in the source dist and wheel.
The non-uv solution is adding classifiers = ["Private :: Do Not Upload"] to your pyproject.toml:
[project]
name = "foo-internal"
version = "0.1.0"
classifiers = ["Private :: Do Not Upload"]

This prevents publishing to PyPI.
(There is a separate question that alternative registries such as GitLab have options to host packages both as public and private, but I'm skipping over that here.)

How "not allowed" is this? Couldn't we append this when building the wheel and source dists?

PEP 621 is unambiguous that we must not append to static parts of [project] when building the wheel and source dists, this is required for the ability to read pyproject.toml without a build. There is an escape hatch that requires explicitly declaring specifiers as dynamic, we set the actual classifier on build depending on the value of tool.uv.private, i.e., if we see tool.uv.private = true we transform the metadata on build as if we had classifiers = ["Private :: Do Not Upload"]:
[project]
name = "foo-internal"
version = "0.1.0"
dynamic = ["classifier"]

[tool.uv]
private = true

Through dynamic, we are allowed to emit the following as METADATA:
Metadata-Version: 2.3
Name: foo-internal
Version: 0.1.0
Version: Private :: Do Not Upload

I prefer classifiers = ["Private :: Do Not Upload"] over the above for being self-documenting and more concise.
pyproject.toml
The other option is requiring pyproject.toml to be present.
To check for tool.uv.private, we need to require the pyproject.toml of the (forbidden to be) published packages to be present for all uv publish invocations, for workspace packages this means having all workspace pyproject.toml. If we were to only check if the respective pyproject.toml is present, we would risk ignoring a tool.uv.private.
The other, bigger downside is that this will be ignored when using a standard workflow such as python -m build && twine upload dist/*: Twine, @pypa/gh-action-pypi-publish and other tools will publish this package to pypi with no regards to custom uv configuration, while the classifier protects no matter the tooling mix.
There are other advantages too for requiring pyproject.toml for publish, e.g., reading configuration from it. In the current trade-off, that doesn't tip the scales for me towards switching to a poetry-like uv publish, given that the currently implemented solution is a universally supported self-documenting one-liner:
[project]
name = "foo-internal"
version = "0.1.0"
classifiers = ["Private :: Do Not Upload"]

I agree that cargo and npm have a clearly better state, and would favor the addition of project.publish or project.private to the pyproject.toml spec.

The other option is requiring pyproject.toml to be present.

Is there a third option where if pyproject.toml is present, then uv publish inspects it and if private = true  is there, it prevents publishing? So normal uv projects will respect it, but if you're using uv publish without a pyproject.toml, then it's back on you to set the classifiers and it works more like twine?

Yeah I agree, I don't think we need to require it. We could always add a flag that requires we can find it; for people that are concerned about accidentally ignoring their configuration.

I know packaging tool authors are generally against putting UX details into PEPs, but maybe this is a case where it would make sense to propose an optional private = true flag in the project table that fails the build if classifiers = ["Private :: Do Not Upload"] is missing from the classifier list? (@konstin considered a flag above, but in the context of reading it at publication time, rather than using it as a build time cross check on the classifier list)
That way as long as the project developers are running their CI builds with a version that respects the flag, publication tools will still see the classifier in the metadata.

an optional private = true flag in the project table that fails the build if classifiers = ["Private :: Do Not Upload"] is missing from the classifier list?

That is an interesting idea. The downsides I see to this are:

Simplicity for the user. Now we need to teach users that ideally they need to state their intention that a project is private twice
The "Private :: Do Not Upload" still seems like a non-ideal solution in the first place because it's only after your code is shipped through an HTTPS connection over the network somewhere that then it's blocked from being saved. I get that this is a bit pedantic, but in the presence of SSL proxies, etc it's possible that your code is revealed somewhere
What I like about private = true is that (just like NPM and Cargo) it can prevent any transmission of the code from package managers that understand it

But I do like the overall idea of proposing that private = true be in a real PEP eventually. That could resolve some of the concerns that "well Twine will upload the .whl anyway". As Twine, Poetry, etc could also be updated to look for this common field in the current directory (or, bikeshedding, could look upwards for a project.toml in upper directories too) and then all refuse to upload.

What I like about private = true is that (just like NPM and Cargo) it can prevent any transmission of the code from package managers that understand it

Yeah, that's one of the two main advantages I see in defining project.private formally. The other advantage is this:

That could resolve some of the concerns that "well Twine will upload the .whl anyway".

If the invalid classifier is explicitly present in the metadata, even old versions of Twine (or other tools) won't successfully upload the package. So even though people (or their dev tools) would need to express "this is private" twice, they'd be doing it for a specific backwards compatibility reason. (and if uv init accepted a --private flag, the dev themselves would still only be marking the package as private once)

Use-case here!
I maintain private version of my open-source projects (sponsorware strategy). In the past, I accidentally uploaded a private version to PyPI, which I then deleted when I noticed (a few weeks later IIRC 😱).
I recently learned about Private :: Do Not Upload (classifiers starting with Private ::, really), and wondered if I could use it to prevent future accidents.
The thing is: although the private versions of my open-source projects must not end up on PyPI, my users must be able to publish them to their private registries. I myself upload these private packages to a locally-served pypiserver instance.
The official docs (packaging guide, PEP 301) do not mention whether Private :: is a PyPI thing only, or if other registries should or should not (must or must not) support it.
Given my use-case above, I hope they explicitly don't support it, but cannot confirm that.
So, in my case, I think a project.private = true value in pyproject.toml would be ideal:

it prevents me from accidentally publishing a private project to PyPI from my own machine
it doesn't add the Private :: classifier to my artifacts, which could prevent upload on alternate/private registries
and therefore lets users (including myself) upload built sdists and wheels to alternate/private registries
...even with uv publish <artifact>, since at that point, uv won't be able to see pyproject.toml, only built artifacts (unless it searches for it in the sdist: don't do that 😅!)


fix: remove code List.of in java11 grammar

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


signal-cli: 0.8.5 -> 0.9.0

Motivation for this change
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 137569 at c1fe8aea run on aarch64-linux 1

2 packages built successfully:

scli
signal-cli


Get "display name" from Jenkins

We utilize the display name, not the project name, to designate our Jenkins jobs.

Fixed in e989ee637aa0ece59485f5ec4995bc7fe6de8022


strapi 4.14 - change source folder

Latest strapi version has change files location

src/common/schemas-to-ts/BeforeRunEvent.ts:3:23 - error TS2307: Cannot find module '@strapi/database/lib/lifecycles/index' or its corresponding type declarations.

import { Event } from '@strapi/database/dist/lifecycles/index';

I was just about to come post this here. Thanks @win-ru @mancku


ARROW-271: Update Field structure to be more explicit

This is a proposal. I have not updated the code depending on this yet.

This seems reasonable to me, and avoids issues around the flattening. It also means in very large schemas that you will not have to reconstruct the tree structure if you wish to examine the metadata in a particular node, which may have nice performance benefits.
I think when we originally drafted the flatbuffers IDL, it seemed like using the struct flatbuffers type would be more efficient / faster, but having the table indirection doesn't seem so bad for the improved flexibility.
Note also that tables permit more flexible schema evolution

@wesm Yes. I'm a bit concerned of using structs since they can not be changed ever.

I'm -1 on the change from struct -> table.
I am the one who pushed hard that the core DataHeader structures should be structs with predictable layouts. One of the targets from my perspective held for the data header structure is that I can always directly locate a buffer length and offset without indirection via the data header. This is why they are in a struct. The schema negotiation gives me an ability to set up all my code with this knowledge and without having to deal with rewriting those pointers.
I'd suggest a level of indirection in the apis if you're finding the structures difficult to use rather than moving to a table based structure. The use of structs in data headers is parallel to the use of direct memory layout in the vectors themselves. I agree that this is unchangeable in a particular format version and think that is okay.

@jacques-n: could you point to an example of this code?
The main thing I'm trying to achieve is a more explicit metadata for interoperability.
It is easy to produce the original flat record batch structure from the tree structure I'm proposing above.
Going back to the tree structure from the flat buffer list requires implicit knowledge of how many buffers we have for a type and in what order (possibly depending on the value count as well). Bugs in this interpretation will be hard to figure out since buffers are not labelled.
Another option:

FieldNode and Buffer are structs. nodes and buffers fields in RecordBatch are unchanged
add childrenCount and bufferCount to FieldNode
add type to Buffer

I would be in favor of structs for both FieldNode and Buffer, but putting the buffers in the FieldNode. Structs cannot be used recursively. Thoughts?

@wesm yes. Using structures requires to have flattened lists for nodes and buffers. adding childrenCount and bufferCount in FieldNode makes it explicit which buffers belong to which fieldNode. the type in Buffer makes it explicit what each buffer is for.

A RecordBatch relies on the Schema structure explicitly. It cannot be consumed without one.  The children/bufferCount values are all already implicitly defined by the nature of the schema structure. I don't think it necessary to store this explicitly. I also think it is unnecessary to duplicate that data in the RecordBatch structure. These values would be exactly the same for every RecordBatch corresponding to a particular Schema. If you concerned about validity and interoperability, it seems that this is simply validation utility that can be written in each C++ and Java. Given a particular schema, this utility could tell you the associated children count and buffer count expected. Perhaps I'm fully understanding the technical problem?

@julienledem: I'm not exactly sure what code you're asking me to point you to.

@jacques-n I was referring to: "The schema negotiation gives me an ability to set up all my code with this knowledge and without having to deal with rewriting those pointers."
Yes the children/bufferCount is already known implicitly. However it is not possible to validate against all errors (buffers in wrong order for example). Adding a little more information would go a long way to pinpointing exactly what's wrong when validation fails (instead of "wrong buffer count"). It is not necessary but it is very useful.

Just to clarify, the reason we chose to use flatbuf was to avoid extra indirection in the RecordBatch structure (and specifically fieldnode and buffer vectors). Without this, we would have chosen protobuf or another less memory-structured (but more common) standard. I continue to think that is important to maintain each of those structures as a struct rather than a table.
Making things less error-prone and more explicit is always a good goal. Your proposal comes at the cost of additional memory overhead. In high performance computing, I think we have to weight the value of that redundancy against the types of errors we'll be able to detect. I also continue to think that a good set of construction validation tools and APIs in C++ and Java will lead to the same thing without overhead. From my perspective, safety and redundancy is the realm of the schema, not the RecordBatch.
Staying with structs, can you make your proposal more specific? It sounds like you're wanting to add an additional 12 bytes to every fieldnode (type, childcount, buffercount)? Is that correct? Are you also proposing an additional 4 byte value per buffer of type? If so, what values would this select from? What specific errors do you think this will help improve?

@jacques-n: I have updated the PR to something that provides the same information but in the schema only. Each Field provides the list of buffers to be expected in each RecordBatch (and all RecordBatches look the same).
The RecordBatch is unchanged.
That should address your concerns.

In theory, the set of buffers associated with each field should be deterministic (with deterministic types), but having this additional metadata adds a small extra layer of validation / flexibility (for example: if the null count is 0, the validity / nullness buffer metadata could be omitted). I am okay with either solution.

@wesm With the vector info in the schema, the number of vectors has to be the same for every RowBatch. Since null count is in the RowBatch, we still need to have a buffer if it is all null (or at least metadata for it). However this Buffer can be empty (size = 0). Technically it is the same but just represented differently in the metadata.

Looks good to me +1

Will merge after rebase

As I was using this in the File format I realized that there's one thing missing which is the bit_width of a slot in a buffer.
I updated accordingly. With that the physical layout is fully explicit.

rebased

Could you explain why the bit_width is needed (the buffer lengths are indicated already in bytes, but this doesn't help you with the size of the slots)? I suppose otherwise you need to use the logical types specification to know the physical size of the slots.

@wesm yes. In some case the bit_width is obvious (ints), in others it is not (dictionary id width. Decimal width?)

I see. In theory the physical size would be a deterministic function of the metadata, but this may not be guaranteed in the implementation (for example, if the dictionary ids were int8_t -- though I wonder whether we should make the id storage more explicit in the metadata).
For decimals, is it true that the physical representation may not be consistent across systems for a given precision and scale?
In any case, I'm okay with having the bit width in the buffer metadata

I'm satisfied with the solution of putting the vector/buffer layout in the schema. @jacques-n?

Where are we at now that ARROW-264 is merged?

@wesm Here is a rebased PR (since part of this was used in ARROW-264).

+1


Segmentation fault with Levenshtein Transformer (build issue with libnat)

Hi, when I trained a new Levenshtein Transformer in my datasets, and it was processed by process.py by '--joined-dictionary', It will show this error, and I trained the model in two gpus.  Could you help me find the reasons, thanks very much.
File "train.py", line 343, in
cli_main()
File "train.py", line 335, in cli_main
nprocs=args.distributed_world_size,
File "/root/anaconda3/envs/len_transform/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 171, in spawn
while not spawn_context.join():
File "/root/anaconda3/envs/len_transform/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 107, in join
(error_index, name)
Exception: process 1 terminated with signal SIGSEGV
And when I trained the model in 3 gpus, the error is :
170848, 512Traceback (most recent call last):
File "train.py", line 343, in
cli_main()
File "train.py", line 335, in cli_main
nprocs=args.distributed_world_size,
File "/root/anaconda3/envs/len_transform/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 171, in spawn
while not spawn_context.join():
File "/root/anaconda3/envs/len_transform/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 107, in join
(error_index, name)
Exception: process 2 terminated with signal SIGSEGV
Traceback (most recent call last):
File "", line 1, in
File "/root/anaconda3/envs/len_transform/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "/root/anaconda3/envs/len_transform/lib/python3.6/multiprocessing/spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated

Similar issues:

https://github.com/pytorch/fairseq/issues/1308
https://github.com/pytorch/fairseq/issues/1305

Oh, thank you.

Similar issues:

#1308
#1305

hi, @sdll , the issues #1308  and #1305  do not solve the problem.

This should be fixed now, can you please try again?

This should be fixed now, can you please try again?

hi, thanks for you replay， and I download the latest version of fairseq， and processed the data ‘--joined-dictionary’， but when I trained a new model， the problem is：
Traceback (most recent call last):
File "train.py", line 337, in
cli_main()
File "train.py", line 329, in cli_main
nprocs=args.distributed_world_size,
File "/root/anaconda3/envs/len_transform/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 171, in spawn
while not spawn_context.join():
File "/root/anaconda3/envs/len_transform/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 107, in join
(error_index, name)
Exception: process 3 terminated with signal SIGSEGV
could you help me？

seems to be a multi-gpu error? Can you try running on single GPU?

seems to be a multi-gpu error? Can you try running on single GPU for debugging?

hi， I have used in single GPU， when the model loaded the trained data， the code is over, but it does not show anything, and I can the GPU is not used.

seems to be a multi-gpu error? Can you try running on single GPU for debugging?

hi， I have used in single GPU， when the model loaded the trained data， the code is over, but it does not show anything, and I can the GPU is not used.

@xiaoshengjun  just following up. Is the problem still there in the recent refactored codebase?

@MultiPath this still occurs in recent codebase.
Even without apex. With less no of max tokens and with higher gcc version and nvcc installed.
Installed fairseq via python setup.py build_ext --inplace.
Even on single GPU, it is segmentation fault.
Please recheck.

I just ran the example command in the README and it works fine.
Can you confirm that libnat is built properly? Please try running the following:
$ python -c 'from fairseq import libnat; print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))'

You should see something like: [[[0], [0], [0], [0], [5, 6], [0, 1, 0, 0]]]

python -c 'from fairseq import libnat; print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))'

Thanks for your reply, and I have run the command what you said above, and get :

from fairseq import libnat
print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))
Segmentation fault

so something is wrong in my environment?

seems to be a multi-gpu error? Can you try running on single GPU for debugging?

hi， I have used in single GPU， when the model loaded the trained data， the code is over, but it does not show anything, and I can the GPU is not used.

@xiaoshengjun just following up. Is the problem still there in the recent refactored codebase?

yes, I also meet the problem.

so something is wrong in my environment?

Yes, libnat has not been built properly. Can you please run python setup.py build_ext --inplace and share the output here?

so something is wrong in my environment?

Yes, libnat has not been built properly. Can you please run python setup.py build_ext --inplace and share the output here?

hi, the output is：
python setup.py build_ext --inplace
which: no nvcc in (/root/anaconda3/envs/fq09py12/bin:/root/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/ibutils/bin:/usr/bin:/usr/local/bin:/usr/libexec/git-core:/root/bin)
running build_ext
/root/anaconda3/envs/fq09py12/lib/python3.6/site-packages/torch/utils/cpp_extension.py:196: UserWarning:
                           !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++ 4.8.5) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 4.9 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.
See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 4.9 or higher.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                          !! WARNING !!

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))
skipping 'fairseq/data/data_utils_fast.cpp' Cython extension (up-to-date)
skipping 'fairseq/data/token_block_utils_fast.cpp' Cython extension (up-to-date)
copying build/lib.linux-x86_64-3.6/fairseq/libbleu.cpython-36m-x86_64-linux-gnu.so -> fairseq
copying build/lib.linux-x86_64-3.6/fairseq/data/data_utils_fast.cpython-36m-x86_64-linux-gnu.so -> fairseq/data
copying build/lib.linux-x86_64-3.6/fairseq/data/token_block_utils_fast.cpython-36m-x86_64-linux-gnu.so -> fairseq/data
copying build/lib.linux-x86_64-3.6/fairseq/libnat.cpython-36m-x86_64-linux-gnu.so -> fairseq

Please follow the instructions in the warning message :)

so something is wrong in my environment?

Yes, libnat has not been built properly. Can you please run python setup.py build_ext --inplace and share the output here?

My output:
running build_ext
skipping 'fairseq/data/data_utils_fast.cpp' Cython extension (up-to-date)
skipping 'fairseq/data/token_block_utils_fast.cpp' Cython extension (up-to-date)
copying build/lib.linux-x86_64-3.7/fairseq/libbleu.cpython-37m-x86_64-linux-gnu.so -> fairseq
copying build/lib.linux-x86_64-3.7/fairseq/data/data_utils_fast.cpython-37m-x86_64-linux-gnu.so -> fairseq/data
copying build/lib.linux-x86_64-3.7/fairseq/data/token_block_utils_fast.cpython-37m-x86_64-linux-gnu.so -> fairseq/data
copying build/lib.linux-x86_64-3.7/fairseq/libnat.cpython-37m-x86_64-linux-gnu.so -> fairseq
And the snippet gives Segmentation fault

Steps ::

conda create -n nat python=3.7 &&  conda activate nat
git clone https://github.com/pytorch/fairseq.git  &&  cd fairseq
conda install -c psi4 gcc-5
conda install libgcc -y
pip install torch && pip install cython
python setup.py build_ext
python setup.py install --user

Can you install gcc from the main channel instead? Something like:
conda install gcc_linux-64 gxx_linux-64

There should be versions for other platforms/architectures too.

Please follow the instructions in the warning message :)

The problem has been solved after I update the version of gcc, thanks.

Can you install gcc from the main channel instead? Something like:
conda install gcc_linux-64 gxx_linux-64

There should be versions for other platforms/architectures too.
Here's what I just ran and it works as expected:
1. conda create -n nat python=3.7 && conda activate nat

2. git clone https://github.com/pytorch/fairseq.git && cd fairseq

3. conda install gcc_linux-64 gxx_linux-64

4. pip install torch && pip install cython

5. python setup.py build_ext --inplace

6. python -c 'from fairseq import libnat; print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))'

Solved Thanks.

@myleott when I run scaling-nmt translation example, https://github.com/pytorch/fairseq/blame/master/examples/scaling_nmt/README.md#L40 in the above conda environment along with --encoder-layerdrop 0.2 the gpu utilisation amounts to 100 and training does not start and is stuck but with the --ddp-backend no_c10d it is working fine.
Whereas without --encoder/decoder-layerdrop it works with/without ddp-backend.
Is this intended behaviour?

Yes, no_c10d is required when some of the model parameters are not used in the forward pass, as is the case with LayerDrop.

Can you install gcc from the main channel instead? Something like:
conda install gcc_linux-64 gxx_linux-64

There should be versions for other platforms/architectures too.
Here's what I just ran and it works as expected:

conda create -n nat python=3.7 && conda activate nat
git clone https://github.com/pytorch/fairseq.git && cd fairseq
conda install gcc_linux-64 gxx_linux-64
pip install torch && pip install cython
python setup.py build_ext --inplace
python -c 'from fairseq import libnat; print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))'

great! It works!!!!!!!!

Can you install gcc from the main channel instead? Something like:
conda install gcc_linux-64 gxx_linux-64

There should be versions for other platforms/architectures too.
Here's what I just ran and it works as expected:

conda create -n nat python=3.7 && conda activate nat
git clone https://github.com/pytorch/fairseq.git && cd fairseq
conda install gcc_linux-64 gxx_linux-64
pip install torch && pip install cython
python setup.py build_ext --inplace
python -c 'from fairseq import libnat; print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))'

For me, after step 4, when I try to run step 5, I get
/opt/conda/envs/nat/lib/python3.7/site-packages/torch/package/_mock_zipreader.py:17: UserWarning: Failed to initialize NumPy: numpy.core.multiarray failed to import (Triggered internally at  /pytorch/to
rch/csrc/utils/tensor_numpy.cpp:67.)
_dtype_to_storage = {data_type(0).dtype: data_type for data_type in _storages}
running build_ext
Traceback (most recent call last):
File "/workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/core/init.py", line 22, in
from . import multiarray
File "/workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/core/multiarray.py", line 12, in
from . import overrides
File "/workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/core/overrides.py", line 7, in
from numpy.core._multiarray_umath import (
ImportError: /workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: failed to map segment from shared object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 268, in
do_setup(package_data)
File "setup.py", line 241, in do_setup
zip_safe=False,
File "/opt/conda/envs/nat/lib/python3.7/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/opt/conda/envs/nat/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/conda/envs/nat/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/opt/conda/envs/nat/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/opt/conda/envs/nat/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/opt/conda/envs/nat/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/opt/conda/envs/nat/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/opt/conda/envs/nat/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 709, in build_extensions
build_ext.build_extensions(self)
File "/opt/conda/envs/nat/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/opt/conda/envs/nat/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 265, in cython_sources
extension.include_dirs = list(extension.include_dirs)
File "setup.py", line 62, in include_dirs
import numpy
File "/workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/init.py", line 150, in
from . import core
File "/workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/core/init.py", line 48, in
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

The Python version is: Python3.7 from "/opt/conda/envs/nat/bin/python"
The NumPy version is: "1.21.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: /workspace/public_data/raven/fairseq/.eggs/numpy-1.21.1-py3.7-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: failed to map segment from shared object

Can you install gcc from the main channel instead? Something like:
conda install gcc_linux-64 gxx_linux-64

There should be versions for other platforms/architectures too.
Here's what I just ran and it works as expected:

conda create -n nat python=3.7 && conda activate nat
git clone https://github.com/pytorch/fairseq.git && cd fairseq
conda install gcc_linux-64 gxx_linux-64
pip install torch && pip install cython
python setup.py build_ext --inplace
python -c 'from fairseq import libnat; print(libnat.suggested_ed2_path([[1, 2, 3, 4]], [[1, 3, 4, 5, 6]], 0))'

For me, after finishing step 1-5, I get the following error at step 6.
Traceback (most recent call last): File "", line 1, in ImportError: /workspace/public_data/raven/fairseq/fairseq/libnat.cpython-37m-x86_64-linux-gnu.so: failed to map segment from shared object

I have figured out the solution. It is due to the permission problem of a mounted disk. If my git clone the fairseq repo inside the mounted disk, and do pip install --editable ./, then I will get the above mentioned error.
If I git clone the repo inside my system disk and do pip install --editable ./, this error is gone.
So there is some problem with the mounted disk. Now I am trying to follow this instruction to see if it works.
https://stackoverflow.com/a/64110234/9793316


Missing queue family ownership transfer in compute_nbody

The compute_nbody sample seems to be missing a queue family ownership transfer of compute.storage_buffer on platforms where a compute-only queue family is available.
Note how compute.storage_buffer is used as a storage buffer in compute.queue and as as vertex buffer in queue, and consider the following:

If the platform has a compute only queue family available, it will be preferred.

The compute dispatches use the  compute.queue queue: https://github.com/KhronosGroup/Vulkan-Samples/blob/55abaaea0ca34288045cd52093cb1060b891508a/samples/api/compute_nbody/compute_nbody.cpp#L629
compute.queue is set here: https://github.com/KhronosGroup/Vulkan-Samples/blob/55abaaea0ca34288045cd52093cb1060b891508a/samples/api/compute_nbody/compute_nbody.cpp#L437
The logic in Device::get_queue_family_index(VkQueueFlagBits queue_flag) will pick a queue family with compute and no graphics if available; https://github.com/KhronosGroup/Vulkan-Samples/blob/55abaaea0ca34288045cd52093cb1060b891508a/framework/core/device.cpp#L349
On my Radeon RX 590 queue family index 1 is picked here

The default queue's queue family is the first one available which supports presentation:

It's created here https://github.com/KhronosGroup/Vulkan-Samples/blob/55abaaea0ca34288045cd52093cb1060b891508a/framework/api_vulkan_sample.cpp#L57
This calls Device::get_queue_by_present which picks the first queue family which can present; https://github.com/KhronosGroup/Vulkan-Samples/blob/55abaaea0ca34288045cd52093cb1060b891508a/framework/core/device.cpp#L328
On my Radeon RX 590 queue family index 0 is picked here

compute.storage_buffer is created with VK_SHARING_MODE_EXCLUSIVE; see https://github.com/KhronosGroup/Vulkan-Samples/blob/55abaaea0ca34288045cd52093cb1060b891508a/framework/core/buffer.cpp#L32

In other words, compute.queue and queue can be from different queue families (and is on my system), which means compute.storage_buffer can not be shared between the two queues without having the queue family ownership transferred:
Resources created with a VkSharingMode of VK_SHARING_MODE_EXCLUSIVE must have their ownership explicitly transferred from one queue family to another in order to access their content in a well defined manner on a queue in a different queue family
The problem is the sample is missing pipeline-barriers to perform this ownership transfer currently. As such, the sample is only correct on systems where the code happens to pick the queues so they are from the same queue family.

We'll look into this. I tested this sample on a device with separate graphics and compute queue family indices and had no problems running it.
Are you getting any validation layer messages?

Are you getting any validation layer messages?

No, nothing (using Vulkan SDK <IP_ADDRESS>). Not that I'm surprised given synchronization validation still seems extremely lacking.

Can you check with this branch: https://github.com/SaschaWillems/Vulkan-Samples/tree/compute_nbody_sync
I have added proper acquire and release barriers on both queue families (if graphics and compute differ).
Note that this will trigger one initial validation message about a missing release barrier, though I'm not sure if that's my fault or if validation layers don't catch it.
Can you test if this version works fine for your setup?

Hi @SaschaWillems I ave integrated your fixes inside my 3DEngine, it workins fine! except only one one initial validation message that you mentioned above.

@SaschaWillems the intiial validation error is valid.  You create and populate the storage buffer on the primary queue in ComputeNBody::prepare_storage_buffers, but then the compute command buffer runs before the graphics command buffer, so there is an acquire barrier without a corresponding release barrier.
The solution is to copy the barrier from this line to between the copy and flush commands at this line.  This provides the initial release to correspond to the compute command buffer's acquire.  Subsequent releases will be provided normally by the graphics command buffer(s).

@SaschaWillems Agree with @jherico's comment on the commit here: https://github.com/SaschaWillems/Vulkan-Samples/commit/bf0c51c73d808eda268ab4585dbf126d651f98bd
But one wrinkle; when waiting on compute.semaphore on the graphics queue, the wait is in submit_info.pWaitDstStageMask = VK_PIPELINE_STAGE_VERTEX_INPUT_BIT (see here). So for a dependency chain to form, it seems like the acquire barrier must include that stage. (Actually, it can probably be any stage as long as it matches pWaitDstStageMask?)
However, after applying @jherico's comments, the graphics acquire barrier has srcStageMask = VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT -- which doesn't match the pWaitDstStageMask. So the queue ownership transfer barrier seems like it would not be blocked by the semaphore like it presumably needs to be.
Similar for the compute queue; semaphore wait is in compute_submit_info.pWaitDstStageMask = VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, but the acquire barrier has srcStageMask = VK_PIPELINE_STAGE_VERTEX_INPUT_BIT.

@cforfang, @jherico: Can you chime in? I'm still struggling with that one validation layer message left. And none of the Vulkan implementations I have access too is having troubles with the current state of the sample, so I'm not even able to test if this change makes things better or worse.

I'll try to take a look tomorrow.  Do you have a commit ID that I can use to repro the validation message?

It's this one from my fork. I just did the changes that you commented some time ago.

I've modified the code to resolve the validation error.  it runs successfully on my RTX 2080 without any validation errors.  The changes are here: https://github.com/SaschaWillems/Vulkan-Samples/pull/3


Create Critical_Service_Learning_Assignment

Homework for students about Critical Service Learning (based on the reading that we assigned).

We will get errors if PRs done from a fork. But that's ok... I can still check spelling afterwards


🐛 [firebase_messaging] FCMService: Invalid notification content

Bug report
i am getting this error about FCM after importing wakelock library in my flutter project:
D/FCMService: FCM received
D/FCMService: Invalid notification content
but if i remove the wakelock plugin from my dependencies list , it works well as it was working before!
what's wrong?
Steps to reproduce
Steps to reproduce the behavior:

only add the wakelock library into a your project dependencies when you have firebase_messaging too
run and send a notification to there
then you can see the logs

Flutter doctor
Click To Expand
[√] Flutter (Channel stable, 1.22.6, on Microsoft Windows [Version 10.0.19041.630], locale en-US)

Flutter dependencies
Click To Expand
- autocomplete_textfield 1.7.3 [flutter]
- awesome_notifications 0.0.5+4 [flutter platform intl]
- cached_network_image 2.5.0 [flutter flutter_cache_manager octo_image]
- cupertino_icons 1.0.0
- dio 3.0.10 [http_parser path]
- firebase_messaging 7.0.3 [meta flutter firebase_core]
- flutter 0.0.0 [characters collection meta typed_data vector_math sky_engine]
- flutter_audio_manager 0.2.3 [flutter]
- flutter_incall 1.0.0 [flutter]
- flutter_launcher_icons 0.8.0 [image args yaml]
- flutter_ringtone_player 2.0.0 [flutter]
- flutter_svg 0.19.1 [flutter meta path_drawing vector_math xml]
- flutter_typeahead 1.9.1 [flutter flutter_keyboard_visibility]
- flutter_webrtc 0.2.8 [flutter]
- fluttertoast 7.1.6 [flutter flutter_web_plugins]
- image_picker 0.6.7+17 [flutter flutter_plugin_android_lifecycle image_picker_platform_interface]
- numberpicker 1.3.0 [flutter infinite_listview]
- path_provider 1.6.24 [flutter path_provider_platform_interface path_provider_macos path_provider_linux path_provider_windows]
- persian_datetime_picker 1.0.7 [flutter shamsi_date]
- retrofit 1.3.4+1 [meta dio]
- sentry_flutter 4.0.1 [flutter flutter_web_plugins sentry package_info]
- shared_preferences 0.5.12+4 [meta flutter shared_preferences_platform_interface shared_preferences_linux shared_preferences_macos shared_preferences_web shared_preferences_windows]
- smooth_page_indicator 0.2.0 [flutter]
- socket_io_client 0.9.12 [logging socket_io_common]
- sqflite 1.3.2+1 [flutter sqflite_common path]
- tinode 0.0.1 [flutter]
- tus_client 0.0.4 [flutter http path]

dev dependencies:
- build_runner 1.10.11 [args async build build_config build_daemon build_resolvers build_runner_core code_builder collection crypto dart_style glob graphs http_multi_server io js logging meta mime path pedantic pool pub_semver pubspec_parse shelf shelf_web_socket stack_trace stream_transform timing watcher web_socket_channel yaml]
- flutter_test 0.0.0 [flutter test_api path fake_async clock stack_trace vector_math async boolean_selector characters charcode collection matcher meta source_span stream_channel string_scanner term_glyph typed_data]
- retrofit_generator 1.4.1 [dio source_gen built_collection code_builder tuple retrofit analyzer dart_style build]

transitive dependencies:
- _fe_analyzer_shared 14.0.0 [meta]
- analyzer 0.41.1 [_fe_analyzer_shared args cli_util collection convert crypto glob meta package_config path pub_semver source_span watcher yaml]
- archive 2.0.13 [crypto args path]
- args 1.6.0
- async 2.5.0-nullsafety.1 [collection]
- boolean_selector 2.1.0-nullsafety.1 [source_span string_scanner]
- build 1.6.0 [analyzer async convert crypto logging meta path glob]
- build_config 0.4.5 [checked_yaml json_annotation meta path pubspec_parse yaml]
- build_daemon 2.1.4 [built_collection built_value http_multi_server logging pedantic path pool shelf shelf_web_socket stream_transform watcher web_socket_channel]
- build_resolvers 1.5.1 [analyzer build crypto graphs logging meta path package_config pool pub_semver]
- build_runner_core 6.1.5 [async build build_config build_resolvers collection convert crypto glob graphs json_annotation logging meta path package_config pedantic pool timing watcher yaml]
- built_collection 4.3.2 [collection quiver]
- built_value 7.1.0 [built_collection collection fixnum quiver]
- characters 1.1.0-nullsafety.3
- charcode 1.2.0-nullsafety.1
- checked_yaml 1.0.4 [json_annotation source_span yaml]
- cli_util 0.2.0 [path]
- clock 1.1.0-nullsafety.1
- code_builder 3.5.0 [built_collection built_value collection matcher meta]
- collection 1.15.0-nullsafety.3
- convert 2.1.1 [charcode typed_data]
- crypto 2.1.5 [collection convert typed_data]
- dart_style 1.3.10 [analyzer args path source_span]
- fake_async 1.2.0-nullsafety.1 [clock collection]
- ffi 0.1.3
- file 5.2.1 [intl meta path]
- firebase_core 0.5.3 [firebase_core_platform_interface flutter quiver meta firebase_core_web]
- firebase_core_platform_interface 2.1.0 [flutter meta plugin_platform_interface quiver]
- firebase_core_web 0.2.1+1 [firebase_core_platform_interface flutter flutter_web_plugins meta js]
- fixnum 0.10.11
- flutter_blurhash 0.5.0 [flutter meta]
- flutter_cache_manager 2.1.0 [flutter path_provider uuid http path sqflite pedantic clock file rxdart image]
- flutter_keyboard_visibility 4.0.2 [meta flutter_keyboard_visibility_platform_interface flutter_keyboard_visibility_web flutter]
- flutter_keyboard_visibility_platform_interface 1.0.1 [flutter meta plugin_platform_interface]
- flutter_keyboard_visibility_web 1.0.1 [flutter_keyboard_visibility_platform_interface flutter_web_plugins flutter]
- flutter_plugin_android_lifecycle 1.0.11 [flutter]
- flutter_web_plugins 0.0.0 [flutter characters collection meta typed_data vector_math]
- glob 1.2.0 [async collection node_io path pedantic string_scanner]
- graphs 0.2.0
- http 0.12.2 [http_parser path pedantic]
- http_multi_server 2.2.0 [async]
- http_parser 3.1.4 [charcode collection source_span string_scanner typed_data]
- image 2.1.19 [archive xml meta]
- image_picker_platform_interface 1.1.1 [flutter meta http plugin_platform_interface]
- infinite_listview 1.0.1+1 [flutter]
- intl 0.16.1 [path]
- io 0.3.4 [charcode meta path string_scanner]
- js 0.6.2
- json_annotation 3.1.1
- logging 0.11.4
- matcher 0.12.10-nullsafety.1 [stack_trace]
- meta 1.3.0-nullsafety.3
- mime 0.9.7
- node_interop 1.2.1 [js]
- node_io 1.2.0 [file node_interop path]
- octo_image 0.3.0 [flutter flutter_blurhash]
- package_config 1.9.3 [path charcode]
- package_info 0.4.3+2 [flutter]
- path 1.8.0-nullsafety.1
- path_drawing 0.4.1+1 [vector_math meta path_parsing flutter]
- path_parsing 0.1.4 [vector_math meta]
- path_provider_linux 0.0.1+2 [path xdg_directories path_provider_platform_interface flutter]
- path_provider_macos 0.0.4+6 [flutter]
- path_provider_platform_interface 1.0.4 [flutter meta platform plugin_platform_interface]
- path_provider_windows 0.0.4+3 [path_provider_platform_interface meta path flutter ffi win32]
- pedantic 1.9.2 [meta]
- petitparser 3.1.0 [meta]
- platform 2.2.1
- plugin_platform_interface 1.0.3 [meta]
- pool 1.4.0 [async stack_trace]
- process 3.0.13 [file intl meta path platform]
- pub_semver 1.4.4 [collection]
- pubspec_parse 0.1.7 [checked_yaml json_annotation pub_semver yaml]
- quiver 2.1.5 [matcher meta]
- rxdart 0.25.0
- sentry 4.0.3 [http meta stack_trace uuid]
- shamsi_date 0.8.1 [flutter]
- shared_preferences_linux 0.0.2+4 [file flutter meta path path_provider_linux shared_preferences_platform_interface]
- shared_preferences_macos 0.0.1+11 [shared_preferences_platform_interface flutter]
- shared_preferences_platform_interface 1.0.4 [meta flutter]
- shared_preferences_web 0.1.2+7 [shared_preferences_platform_interface flutter flutter_web_plugins meta]
- shared_preferences_windows 0.0.1+3 [shared_preferences_platform_interface flutter ffi file meta path path_provider_platform_interface path_provider_windows]
- shelf 0.7.9 [async collection http_parser path stack_trace stream_channel]
- shelf_web_socket 0.2.3 [shelf web_socket_channel stream_channel]
- sky_engine 0.0.99
- socket_io_common 0.9.2 [logging]
- source_gen 0.9.10+1 [analyzer async build dart_style glob meta path pedantic source_span]
- source_span 1.8.0-nullsafety.2 [charcode collection path term_glyph]
- sqflite_common 1.0.2+1 [synchronized path meta]
- stack_trace 1.10.0-nullsafety.1 [path]
- stream_channel 2.1.0-nullsafety.1 [async]
- stream_transform 1.2.0
- string_scanner 1.1.0-nullsafety.1 [charcode source_span]
- synchronized 2.2.0+2
- term_glyph 1.2.0-nullsafety.1
- test_api 0.2.19-nullsafety.2 [async boolean_selector collection meta path source_span stack_trace stream_channel string_scanner term_glyph matcher]
- timing 0.1.1+3 [json_annotation]
- tuple 1.0.3 [quiver]
- typed_data 1.3.0-nullsafety.3 [collection]
- uuid 2.2.2 [crypto convert]
- vector_math 2.1.0-nullsafety.3
- watcher 0.9.7+15 [async path pedantic]
- web_socket_channel 1.1.0 [async crypto stream_channel]
- win32 1.7.4 [ffi]
- xdg_directories 0.1.2 [meta path process]
- xml 4.5.1 [collection convert meta petitparser]
- yaml 2.2.1 [charcode collection string_scanner source_span]

Would it not make more sense to raise this against the Wakelock library? We're not doing any side effects in this library so looks as though the issue isn't with us.

i have asked them and i have read their library code.
it does not seem like that

@benyamin218118 Did they give a reason for why this is coming from flutterfire? If you opened an issue in the maintainers repository could you provide a link to it?

they didn't sayd any thing about it is coming from flutter fire.
https://github.com/creativecreatorormaybenot/wakelock/issues/91
i just sayed maybe it is related to flutterfire , because there is a log like this :
D/FCMService: FCM received
D/FCMService: Invalid notification content
what this mean?

There have been two other issues about this before but in both cases, it seemed to be resolved by just using the latest version of firebase_messaging instead of 7.0.3.
Could you try upgrading to firebase_messaging: ^8.0.0-dev.14 or firebase_messaging: ^9.0.0-1.0.nullsafety.0?

At that time I needed to turn on the Lock screen on new notification Received, i solved that problem , i dont want to use wakelock package any more , but i will upgrade the version and let you guys know if it is fixed.
tnx for answering;

Since wakelock was the original cause I'm going to optimistically close this as solved for now, let me know if the issue persists.


proposal: restructure pkg/ingress

As support for the Gateway API progressed, we needed to redesign the structure of pkg/ingress.
This directory now has multiple types of resources mixed together: Kubernetes native resources, APISIX Ingress resources, and Gateway API resources. More types of resources may be added in the future.
In addition, the controller.go file has become increasingly large.
We need to redesign the controller so that supporting different resources doesn't require massive changes to the outer files, and to keep the changes in a smaller module as much as possible.

+1
I agree with you.
Do you have a more specific plan?
We can put this proposal into this year's roadmap

Perhaps we could create an interface called "(Feature)Provider". The provider holds several small controllers and associated clientet/listers/informers.
Also, we should separate the translators and only make the Kubernetes translator publicly accessible, since any other translator could potentially rely on it.
APISIX Ingress and any other CRD translators should be private.

RootController

KubernetesProvider (public)

Controllers: secret, service, pod, endpoint...
clientset/listers/informers (public)
KubernetesTranslator (public)

ApisixIngressProvider

Controllers: APISIX Ingress CRDs
clientset/listers/informers
ApisixIngressTranslator

GatewayProvider

Controllers: Gateway CRDs
clientset/listers/informers
GatewayTranslator

SGTM
I prefer to rename KubernetesProvider to CommonProvider.
When dealing with various resources(Ingress/CR/Gateway resource), we may need to deal with these basic resources of Kubernetes.


install with --user is not providing rspec command

Subject of the issue
I tried to install rspec but I had trouble to get rspec command.
Your environment

Ruby version: ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux-gnu]
rspec-core version: 3.9.2

Steps to reproduce
gem install --user rspec

resulted in
Successfully installed rspec-3.9.0
1 gem installed

without any complaints, but
rspec resulted in bash: /usr/local/bin/rspec: No such file or directory
Expected behavior
rspec command works or I get warning during install that my PATH is broken or at least hint what kind of thing went wrong (maybe --user install is unsupported?)
Actual behavior
rspec command was not working
sudo gem install rspec worked.

:wave: This sounds like an issue for Rubygems to work out why your --user gems are not accessible. Especially if root install works.

So it should be handled by https://github.com/rubygems/rubygems ? I have seen some messages "your path is not including XYZ, commands will not run" but I have no idea whatever it was emitted by rubygems, ruby or by gem itself.

Gem's are not responsible for their installation, they provide a manifest of files to rubygems (or some other package manager) which puts them in appropriate places.
The path error sounds like it's coming from rubygems because your system is misconfigured but I don't know that for certain.


-> Cannot close object, library is destroyed. This may cause a memory leak!

Bug description
I'm trying to run an example of doctr on my M1 Mac. Code is shown farther below.
Code snippet to reproduce the bug
from doctr.io import DocumentFile
from doctr.models import ocr_predictor
import os
# Let's pick the desired backend
os.environ['USE_TORCH'] = '1'

if __name__ == "__main__":

    model = ocr_predictor(pretrained=True)

    # PDF
    doc = DocumentFile.from_pdf("marco-test-2.pdf")

    # Analyze
    result = model(doc)
    # result.show(doc)

Error traceback
`-> Cannot close object, library is destroyed. This may cause a memory leak!`

Environment
DocTR version: v0.7.0
TensorFlow version: N/A
PyTorch version: 2.2.0 (torchvision 0.17.0)
OpenCV version: 4.9.0
OS: Mac OSX 14.3
Python version: 3.11.8
Is CUDA available (TensorFlow): N/A
Is CUDA available (PyTorch): No
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
Deep Learning backend

print(f"is_tf_available: {is_tf_available()}")
is_tf_available: False
print(f"is_torch_available: {is_torch_available()}")
is_torch_available: True

Hi @marcobornstein :wave:,
Could you please check if this also happens with images ?

Yes, it was not working still for images. However I found that result.synthesize() worked as expected. For example, this code snippet worked:
model = ocr_predictor(pretrained=True)

# PDF
doc = DocumentFile.from_pdf("marco-test-2.pdf")

# Analyze
result = model(doc)
synthetic_pages = result.synthesize()
plt.imshow(synthetic_pages[0]);
plt.axis('off');
plt.show()

@marcobornstein Mh the error disappeared ?
BTW: .show(doc) is now .show()

Seems to be solved -> Feel free to reopen if not :)


Cuda is Out of Memory. How to turn on CPU Offload?

Hi team,
Thanks for this great tool. I'm having the same issue with my 8GB card:
https://github.com/rsxdalv/tts-generation-webui/issues/56
Could I get some help with what to input into config.json to enable CPU offload? Got plenty of RAM and such (256gb) and 2x Xeon CPUs so would like to make use of it in case it runs out of memory

Whoops, just noticed as setting in option.

Hi, yes it's possible for Bark. For others - it's requires an improvement to the app. I usually just restart the UI, I use 8GB of VRAM myself.

How do we configure how much VRAM is allocated in Windows (if anywhere)?  Strange one, on 24gb 3090, it's telling me OOM allocating VRAM just trying to do a basic Tortoise TTS generation

How do we configure how much VRAM is allocated in Windows (if anywhere)? Strange one, on 24gb 3090, it's telling me OOM allocating VRAM just trying to do a basic Tortoise TTS generation

Yes that is strange. Did you launch any other models or any other AI apps? Tortoise should fit within 2, maybe 4gbs.