Make Plugin.getGroupId() NonNull

What's changed?
org.openrewrite.maven.tree.Plugin.getGroupId() is now non-null, defaulting to org.openrewrite.maven.tree.Plugin.PLUGIN_DEFAULT_GROUPID.
What's your motivation?
closes #4526
This ensures no more NullPointerExceptions can occur. Plugins can have a null groupId but not all places of the code take that situation into account. By automatically defaulting to org.apache.maven.plugins in the getter itself, there is no more need in the code to duplicate the nullability branching, and potentially forgetting it.
Anything in particular you'd like reviewers to focus on?
I was not able to provide a test for this case because the project being tested needs to use a parent pom that uses plugin management with plugins not explicitly setting <groupId>org.apache.maven.plugins</groupId>. I could not find a public pom file that I could use as a parent in the tests.
Locally there was 1, on first sight, non related test (MavenPomDownloaderTest.mirrorsOverrideRepositoriesInPom()) failing. I verified the changes by running our failing recipes again and now the NPE is gone.
Have you considered any alternatives or workarounds?
We bump into this issue because we use jgitver to manage out Maven versions. I tried to fix the issue in jgitver itself but it is using native maven to write the built pom file and it is in that native Maven code that the groupId for all plugins of org.apache.maven.plugins is not written to the new pom.xml file

Thanks for the suggestion @rob-valor ! I think this indeed more closely matches the default groupId handling in Maven. Let me tag @knutwannheden for a final check on serialization since I'm reviewing from mobile.

It’s strange that you would have to change org.openrewrite.maven.tree.Plugin because in #4507 I should have already ensured that Plugin.groupId is never null by initializing it with that same default value (see RawPom.java).
It’s true that the null check in RemoveRedundantDependencyVersions shouldn’t be needed anymore, but it shouldn’t cause a NPE.
Are you sure you are using the latest version of Rewrite? I have noticed with my own recipe library that sometimes the Maven Plugin does not pick the Rewrite version I want…

@DidierLoiseau, the issue is not in the (raw) pom of the project being rewritten. It's in the ' ResolvedPom' when its parent pom has such defaults.
But since you mention the logic you added, maybe that should also just depend on the default getter logic I added. That way, it's possible to detect a missing groupId and I could add a recipe AddMissingPluginGrouId that adds the groupId explicitly for plugins that haven't set it. The getters should get a proper name in that case. Adding a getOriginalGroupId or renaming the one with the default logic to getGroupIdOrDefault (but that can cause NPE bugs again). @timtebeek @knutwannheden your thoughts on this?

@DidierLoiseau, the issue is not in the (raw) pom of the project being rewritten. It's in the ResolvedPom when its parent pom has such defaults.

I don’t understand what you mean by such defaults – the default value for groupId is indeed org.apache.maven.plugins (as per the maven xsd, and since #4507 it should be always filled in. I had even added an explicit test on RemoveRedundantDependencyVersions for the case of a parent (and child) not specifying the groupId. (BTW this is also an example of test with parent and child, so you don’t need to find a public pom for such tests)

But since you mention the logic you added, maybe that should also just depend on the default getter logic I added. That way, it's possible to detect a missing groupId and I could add a recipe AddMissingPluginGroupId that adds the groupId explicitly for plugins that haven't set it. The getters should get a proper name in that case. Adding a getOriginalGroupId or renaming the one with the default logic to getGroupIdOrDefault (but that can cause NPE bugs again). @timtebeek @knutwannheden your thoughts on this?

If you want to add the missing <groupId> everywhere, you’re going to traverse the XML anyway, right? So at that point, you can just check if the <plugin> tags have it, and add it when missing, you don’t need to check the corresponding Plugin object…
In fact, the AddOrUpdateChildTag recipe I proposed in #4533 could be used to do that. I just added a test case to show it – the only drawback is that it does not allow you to choose the position of the new <groupId> tag, so it would be the last element. You would need to apply OrderPomElements afterwards… but it seems ordering the GAV for plugins is missing, I think it’s an oversight.

As a quick note in between holidays: there is already a recipe to make implicit groupIds explicit:
https://docs.openrewrite.org/recipes/maven/cleanup/explicitplugingroupid
It's also included with our Apache Maven best practices recipe:
https://docs.openrewrite.org/recipes/maven/bestpractices
Just to prevent any duplicate effort where existing recipes exist.
I am still chasing down a potentially related failure downstream in rewrite-jackson:
https://github.com/openrewrite/rewrite-jackson/actions/runs/11085613556/job/30802230458#step:5:1405
CodehausDependencyToFasterXMLTest > changeDependencies(String, String) > [1] pattern=2.12.x, expectedVersion=2.12.7 FAILED
    java.lang.AssertionError: Failed to parse sources or run recipe
        at org.openrewrite.test.RewriteTest.lambda$defaultExecutionContext$14(RewriteTest.java:631)
        at org.openrewrite.test.RewriteTest$$Lambda$513/0x00007fa1e8247618.accept(Unknown Source)
        at org.openrewrite.maven.MavenParser.parseInputs(MavenParser.java:100)
        at org.openrewrite.test.RewriteTest.rewriteRun(RewriteTest.java:280)
        at org.openrewrite.test.RewriteTest.rewriteRun(RewriteTest.java:130)
        at org.openrewrite.java.jackson.codehaus.CodehausDependencyToFasterXMLTest.changeDependencies(CodehausDependencyToFasterXMLTest.java:90)

        Caused by:
        java.io.UncheckedIOException: Failed to parse pom
            at app//org.openrewrite.maven.internal.RawPom.parse(RawPom.java:129)
            at app//org.openrewrite.maven.MavenParser.parseInputs(MavenParser.java:77)
            ... 3 more

            Caused by:
            com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Invalid definition for property 'plugins' (of type `org.openrewrite.maven.internal.RawPom$Build`): Could not find creator property with name 'plugins' (known Creator properties: [plugin, pluginManagement])
             at [Source: (org.openrewrite.internal.EncodingDetectingInputStream); line: 1, column: 1]
                at app//com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:62)
                at app//com.fasterxml.jackson.databind.DeserializationContext.reportBadPropertyDefinition(DeserializationContext.java:1883)
                at app//com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:636)
                at app//com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:277)
                at app//com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:151)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:471)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:415)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:317)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:284)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:174)
                at app//com.fasterxml.jackson.databind.DeserializationContext.findNonContextualValueDeserializer(DeserializationContext.java:659)
                at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:552)
                at app//com.fasterxml.jackson.databind.deser.std.DelegatingDeserializer.resolve(DelegatingDeserializer.java:60)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:347)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:284)
                at app//com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:174)
                at app//com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:669)
                at app//com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:5048)
                at app//com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4918)
                at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3897)
                at app//org.openrewrite.maven.internal.RawPom.parse(RawPom.java:123)
                ... 4 more

Any quick insight there @DidierLoiseau ? You've been diving deeper there lately and have provided a lot of helpful context already.

As a quick note in between holidays: there is already a recipe to make implicit groupIds explicit:
https://docs.openrewrite.org/recipes/maven/cleanup/explicitplugingroupid

Didn’t know about that one. I have removed the example test from the other PR then.

Any quick insight there @DidierLoiseau ? You've been diving deeper there lately and have provided a lot of helpful context already.

Could it be that the @AllArgsConstructor is confusing Jackson because there are two constructors? (see this SO answer)
It’s strange that it works fine in Rewrite itself. It’s not very clear for me what changed between the builds however.

Could it be that the @AllArgsConstructor is confusing Jackson because there are two constructors? (see this SO answer) It’s strange that it works fine in Rewrite itself. It’s not very clear for me what changed between the builds however.

Thanks a lot spotting that annotation and the linked context of the stackoverflow issue; I've pushed up a quick PR to remove it:

https://github.com/openrewrite/rewrite/pull/4530#issuecomment-2382971751

If that passes I'll see it through (assuming no harm) and see if that fixes rewrite-jackson.

@DidierLoiseau,

Are you sure you are using the latest version of Rewrite? I have noticed with my own recipe library that sometimes the Maven Plugin does not pick the Rewrite version I want…

Yes, I checked-out main branch of rewrite and rewrite-maven-plugin. In rewrite I added System.out.println() statements to find out which plugin was failing. I used that version of rewrite in my local version of rewrite-maven-plugin and ran the recipe with that version. Only then I found out that the NPE was caused by a plugin managed in our own company private version of Spring Boot parent. We have a project with a lot of auto-configuration and test libraries and Maven plugin configuration. It's for those plugins that we face the issue.

It’s strange that you would have to change org.openrewrite.maven.tree.Plugin because in https://github.com/openrewrite/rewrite/pull/4507 I should have already ensured that Plugin.groupId is never null by initializing it with that same default value (see RawPom.java).

Well, I can only say, it is null in the place where I get the NPE, so there is some code path not covered.
Since org.openrewrite.maven.tree seems to be the API in rewrite-maven to work with Maven, it's not that strange to put the logic only there. That way, anywhere in the code, the default fallback logic is always used.
Insufficient test setup?
I used the entire pom of our project in an existing test org.openrewrite.maven.UpgradeParentVersionTest#upgradeToExactVersion because that's the recipe we run when it fails. I needed to add a <repositories> section to our company repository, and once that was done... that test passes. It seems that when I run the recipe in the project, there are other code paths because it still gives me a NPE. I also added the system out prints and it's clear that there are null groupids, but not in tests:
Actual rewrite:
[INFO] Running recipe(s)...
plugins
org.apache.maven.plugins:maven-pmd-plugin:${maven-pmd-plugin.version}
...
null:maven-source-plugin:null
...
null:maven-enforcer-plugin:${maven-enforcer-plugin.version}
...

org.codehaus.mojo:versions-maven-plugin:${versions-maven-plugin.version}
org.codehaus.mojo:xml-maven-plugin:${xml-maven-plugin.version}
[ERROR] The recipe produced an error. Please report this to the recipe author.

Output during test:
org.apache.maven.plugins:maven-pmd-plugin:${maven-pmd-plugin.version}
...
org.apache.maven.plugins:maven-enforcer-plugin:${maven-enforcer-plugin.version}
...
org.apache.maven.plugins:maven-source-plugin:${maven-source-plugin.version}
...
org.codehaus.mojo:versions-maven-plugin:${versions-maven-plugin.version}
org.codehaus.mojo:xml-maven-plugin:${xml-maven-plugin.version}

null origin
I added a new Exception("groupId is null. " + groupId + ":" + artifactId + ":" + version).printStackTrace(); in the constructor of Plugin when groupId is null. I hope this is of any use for those knowing the what, why and how of the Maven processing stuff :wink: This is the result:
java.lang.Exception: groupId is null. null:maven-source-plugin:null
        at org.openrewrite.maven.tree.Plugin.<init>(Plugin.java:49)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
        at com.fasterxml.jackson.databind.introspect.AnnotatedConstructor.call(AnnotatedConstructor.java:126)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:291)
        at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:301)
        at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:532)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:576)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:446)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:576)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:446)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1493)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4905)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3909)
        at org.openrewrite.maven.cache.RocksdbMavenPomCache.deserializePom(RocksdbMavenPomCache.java:190)
        at org.openrewrite.maven.cache.RocksdbMavenPomCache.getPom(RocksdbMavenPomCache.java:148)
        at org.openrewrite.maven.cache.CompositeMavenPomCache.getPom(CompositeMavenPomCache.java:76)
        at org.openrewrite.maven.internal.MavenPomDownloader.download(MavenPomDownloader.java:542)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPom(ResolvedPom.java:504)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPropertiesAndRepositoriesRecursively(ResolvedPom.java:411)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPropertiesAndRepositoriesRecursively(ResolvedPom.java:421)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPropertiesAndRepositoriesRecursively(ResolvedPom.java:421)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPropertiesAndRepositoriesRecursively(ResolvedPom.java:421)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentsRecursively(ResolvedPom.java:369)
        at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolve(ResolvedPom.java:358)
        at org.openrewrite.maven.tree.ResolvedPom.resolve(ResolvedPom.java:176)
        at org.openrewrite.maven.tree.Pom.resolve(Pom.java:188)
        at org.openrewrite.maven.tree.Pom.resolve(Pom.java:171)
        at org.openrewrite.maven.MavenParser.parseInputs(MavenParser.java:113)
        at org.openrewrite.Parser.parse(Parser.java:59)
        at org.openrewrite.maven.MavenMojoProjectParser.parseMaven(MavenMojoProjectParser.java:552)
        at org.openrewrite.maven.MavenMojoProjectParser.listSourceFiles(MavenMojoProjectParser.java:145)
        at org.openrewrite.maven.AbstractRewriteBaseRunMojo.loadSourceSet(AbstractRewriteBaseRunMojo.java:236)
        at org.openrewrite.maven.AbstractRewriteBaseRunMojo.listResults(AbstractRewriteBaseRunMojo.java:152)
        at org.openrewrite.maven.AbstractRewriteRunMojo.execute(AbstractRewriteRunMojo.java:64)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:298)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:283)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:226)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:407)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:348)

Pff, now that I looked into that caching thing there, I eventually found the issue. The cache had the serialized pom files with the missing groupId. Clearing the cache (~/.rewrite-cache) solved it.
Please to consider putting the logic in the Plugin class. As you can see, the RawPom class is not the only place that creates instances of Plugin. But putting it only there, makes the rest of the code base more resilient to assumptions that were made when putting it in RawPom :wink:

Now that I looked into that caching thing there, I eventually found the issue. The cache had the serialized pom files with the missing groupId because of running rewrite-maven previously. So those cached serialized Poms were not handles by the new RawPom logic. Clearing the cache (~/.rewrite-cache) solved it.

Good catch! This means we know what bug we are fighting against, and it would potentially be possible to write a unit test for that.
I’m not sure what’s best approach here though. This would only affect people who have run the old version of Rewrite and have such resolved poms in their cache already.
Maybe the best would be to validate the values in the Plugin constructor instead? One option is to use Lombok’s @Nonnull on the applicable fields. This would also cover other potential future bugs.
Also maybe there is a mechanism to clear the cache automatically when such a change is detected? (didn’t check)

Well, I think, if it's all covered in Plugin, so accepting null but returning org.apache.maven.plugins when it is, nobody is impacted. Also, any code in the codebase will always be null safe, regardless of what the future brings us.
There also won't be any such logic dangling around anymore (if I would clean it all up :wink: )

Thanks for the detailed look here both! I'm hopping in and out, and will be away for the rest of the week before it's conference season starting next week. Let me know what the both of you settle on as the best way forward! 😅 I think handling a poisoned ~/.rewrite-cache might be best handled with an update to the docs for this one time change, as opposed to complicating handling.

Aha! Maybe I’ll meet you on Monday then @timtebeek :wink: Enjoy the rest of your holidays!

I think these changes look reasonable. Happy to continue to iterate as the discussion evolves. Thanks @rob-valor and @DidierLoiseau !


hide renders options menu

Kindly help me in the pivot table to hide renders menu as the attached screenshot  I want all the option but in the table format no any other options.

add the line below to your css file:
.pvtRenderer {display: none}


Update build with Android Studio 2.3 + NDK-r15

Removed app_dummy() from NativeActivity derived samples
Note:
NativeActivity derived samples will fail if NOT built with NDK-r15

Merging in since it is just build update: mainly to keep it for internal studio validation purpose.


topdoc (postcss) can't find index.pug

It's either a missing feature or a bug, but topdoc should be able to take a path to just the directory and sort it out from there.  If there is no index.js, it could all back to an index.pug and work it out.
... or maybe that's a bad assumption. Will discuss with @GarthDB

I can take a crack at this if you want to assign to me?  Just LMK.

This is fixed in the beta

IT'S ALIVE!!!


KITTI dataset details

I have a few questions regarding the KITTI dataset.

 I wonder which one was used.
When using the point cloud, did you remove the road and downsampled it to have 4096 points like the oxford and inhouse datasets? Moreover, if so, KITTI's point cloud seems to contain a wider area of information. Did you crop it?
How did you handle the northing and easting information? The distance was calculated only by northing and easting in the existing oxford and inhouse datasets without considering the altitude. Did you save and use it in the same way in KITTI? (for example, easting: x, northing: z) Or did you use the 3D distance?

I would be waiting for your reply. Thank you!

Hi,

We used color images only from Camera2, which corresponds to 'image_2' folder in the original dataset. The Camera2 was mounted in the center of the vehicle, which turns out to be sufficient for this task.
The results presented in the paper are the ones with no prior preprocessing since we want our method to operate on raw point clouds. Moreover, we did not observe any significant difference when removing the ground plane. However, I suspect cropping a cloud could slightly improve the results.
We used only x,z to calculate distances (2D).

Thank you for the comment. That was very helpful!

I have one more question. I'm trying to run it with the KITTI dataset, but if I put it as is, the number of points does not decrease with ME.utils.sparse_quantize. Is this correct? Or is it correct to shift and rescale to zero mean and [-1, 1] like the existing dataset?

No, there is no need for that. You can simply perform quantization without rescaling. For Kitti dataset we set quantization step=0.25.

Oh, I understand. Thank you again!


The performance of nginx drops a lot when using OpenSSL 3.0

Some parameters:
Cipher suite: TLS_AES_256_GCM_SHA384
TLS version: TLSv1.3
Kx=EC
Au=EC
ECDH_curve=prime256v1
CPS test result:

OpenSSL 1.1.1
OpenSSL 3.0.9
drop

4295.38
3132.76
27.07%

Flame Graph:
nginx with OpenSSL 1.1.1:

nginx with OpenSSL 3.0.9

Root Cause
1. EVP_PKEY_derive_set_peer_ex

I don't know if this part of the operation is necessary.
But it can be seen that it takes up a lot of CPU resources, so in order to eliminate its impact on performance, we can disable it in the source code.
diff --git a/crypto/evp/exchange.c b/crypto/evp/exchange.c
index d7a4ad142a..58eea73b80 100644
--- a/crypto/evp/exchange.c
+++ b/crypto/evp/exchange.c
@@ -499,7 +499,7 @@ int EVP_PKEY_derive_set_peer_ex(EVP_PKEY_CTX *ctx, EVP_PKEY *peer,

 int EVP_PKEY_derive_set_peer(EVP_PKEY_CTX *ctx, EVP_PKEY *peer)
 {
-    return EVP_PKEY_derive_set_peer_ex(ctx, peer, 1);
+    return EVP_PKEY_derive_set_peer_ex(ctx, peer, 0);
 }

Performance comparison after EVP_PKEY_derive_set_peer_ex disabled

OpenSSL 1.1.1
OpenSSL 3.0.9
drop

4295.38
3747.19
12.76%

The flame graph after EVP_PKEY_derive_set_peer_ex disabled

Now, it looks good, and the performance improved a lot.
Below are based on the flame graph that disabled the EVP_PKEY_derive_set_peer_ex
2. do_sigver_init
This function takes up ~4% of CPU resources.

3. EVP_xxx_fetch
These functions take up ~6.3% of CPU resources.
In OpenSSL, these functions are similar to engine_table_select, which consumes fewer CPU cycles.

Most of it is a lock operation.

4. Some other code changes
Question

If EVP_PKEY_derive_set_peer_ex is not necessary, can we disable it when config the project?
Will the function do_sigver_init and EVP_xxx_fetch be optimized in the future release?

The problem is that without the validate_peer parameter set the public key of the peer is not validated which means it can be a bogus key not providing security against eavesdroppers or mitm.

However perhaps EVP_PKEY_public_check_quick() would be sufficient instead of the full check. @slontis ?

Looks like the "quick" check is checking that the point isn't infinity, checking coefficients are in range, and checking if the point is on a curve. Other than potentially the first one (depending on your invariants), all these checks ought to have been performed at the time you constructed the EC_KEY. I.e. OpenSSL should not allow you to construct an EC_POINT that is off the curve... I thought you all fixed that in 1.1.x or so? But it's also not that expensive to redo wastefully.
The "full" check additionally checks that point * order = infinity. This one is expensive, as you're effectively doing twice as much ECDH as you need. It is also completely pointless if you're on a prime-order curve, where there are no small subgroups, and practically all curves used in this part of the code have prime order.

practically all curves used in this part of the code have prime order

Though, since OpenSSL supports arbitrary curves as well as a some built-in curves with a cofactor, keep in mind that when you aren't on a prime-order curve, you may need to take some care here.

Section <IP_ADDRESS>.2 of SP800-56Ar3 says you can do either the partial OR the full public key validation, so changing to the partial check should be sufficient.
It is also not helped currently by the code performing the point multiply twice..

Perhaps if we know that there is no way that an EC_KEY can exist without already having done most of the checks, then maybe we could remove some of the redundant checks (such as pointoncureve) and comment why they are removed (even if it was only for the partial case).

Though, since OpenSSL supports arbitrary curves as well as a some built-in curves with a cofactor, keep in mind that when you aren't on a prime-order curve, you may need to take some care here. (I don't know that part as well because BoringSSL just doesn't support EC_GROUP with cofactors.)

We could do the quick check for prime order curves, otherwise do the full check.

We've also run into the use of the expensive test instead of quick check at Akamai, in the course of our 3.0 migration. I don't however have a patch that does the right thing with non-NIST curves (although x25519 should bypass all of this).

For find_sig_alg optimization the problem is that the useable hash functions are not recorded in the EVP_PKEY, and so the calls are expensive. Optimizing by saying if you use a named curve you probably support the right hash is one possibility and is what I tried, but it would be better if the EVP_PKEY could keep that information around v. need an expensive provider call every time.

any updates? It seems worrisome.

Will this fix be back ported to 3.0.x for LTS releases?

Will this fix be back ported to 3.0.x for LTS releases?

I am sorry but no, we do not plan to backport any exisiting performance fixes to the 3.0 branch. The fixes are too invasive to be backported.


python310Packages.anyio: disable timing sensitive test

Description of changes
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.2.1, pluggy-1.0.0
rootdir: /build/source, configfile: pyproject.toml, testpaths: tests
plugins: anyio-3.6.2, hypothesis-6.68.2, mock-3.10.0, xdist-3.2.1
gw0 [917] / gw1 [917] / gw2 [917] / gw3 [917] / gw4 [917] / gw5 [917] / gw6 [917] / gw7 [917] / gw8 [917] / gw9 [917] / gw10 [917] / gw11 [917] / gw12 [917] / gw13 [917] / gw14 [917] / gw15 [917] / gw16 [917] / gw17 [917] / gw18 [917] / gw19 [917]m
.s...................................................................... [  7%]
..........s.......................................ss.................... [ 15%]
...........................s............................................ [ 23%]
........................................................................ [ 31%]
........................................................................ [ 39%]
.............F.......................................................... [ 46%]
........................................................................ [ 54%]
........................................................................ [ 62%]
......................................................................... [ 70%]
........................................................................ [ 78%]
........................................................................ [ 86%]
......................................................................... [ 94%]
...................................................                      [100%]
=================================== FAILURES ===================================
___________ TestRunAsyncFromThread.test_run_sync_from_thread_pooling ___________
[gw6] linux -- Python 3.10.12 /nix/store/bc45k1n0pkrdkr3xa6w84w1xhkl1kkyp-python3-3.10.12/bin/python3.10
tests/test_from_thread.py:81: in test_run_sync_from_thread_pooling
    run(main, backend="asyncio")
/nix/store/2lhv2r45jcjni9dsqjrs0ld69ghb5x00-python3.10-anyio-3.6.2/lib/python3.10/site-packages/anyio/_core/_eventloop.py:70: in run
    return asynclib.run(func, *args, **backend_options)
/nix/store/2lhv2r45jcjni9dsqjrs0ld69ghb5x00-python3.10-anyio-3.6.2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:292: in run
    return native_run(wrapper(), debug=debug)
/nix/store/bc45k1n0pkrdkr3xa6w84w1xhkl1kkyp-python3-3.10.12/lib/python3.10/asyncio/runners.py:44: in run
    return loop.run_until_complete(main)
uvloop/loop.pyx:1517: in uvloop.loop.Loop.run_until_complete
    ???
/nix/store/2lhv2r45jcjni9dsqjrs0ld69ghb5x00-python3.10-anyio-3.6.2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:287: in wrapper
    return await func(*args)
tests/test_from_thread.py:77: in main
    assert threading.active_count() == initial_count + 1
E   assert 4 == (4 + 1)
E    +  where 4 = <function active_count at 0x7ffff7471240>()
E    +    where <function active_count at 0x7ffff7471240> = threading.active_count
=========================== short test summary info ============================
SKIPPED [1] tests/test_fileio.py:124: Drive only makes sense on Windows
SKIPPED [1] tests/test_fileio.py:166: Only makes sense on Windows
SKIPPED [3] tests/test_fileio.py:340: os.lchmod() is not available
================== 1 failed, 911 passed, 5 skipped in 22.05s ===================

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Too many rebuilds for master. Picked into python-updates as b8539723e9cd7366837cdad6499c33fa4468818e.


Reset popup

Hello! What's the status of this? I am coming from Emacs and this feature is one I miss quite a bit, so it would be awesome if it were implemented. Are you planning to create another PR for this?
Thanks a lot for the hard work!

Hey! Forgot to open this back up. If you want, feel free to follow my fork for now until this can be merged here :) It's a little experimental at times, but I use it daily so 🤷

Couldn't find anything else so I made https://github.com/TimUntersberger/neogit/issues/490


chore: update cargo lock

reduce start up times for rust projects in local env (buck2 run //dev:up)

Its better to update each dev explicitly to keep the root and third-party in sync.


Cannot close opened folder

Issue Type: Bug
Once I open a folder (folder is a Office365 SharePoint Online site) I'm unable to close it and the VSC app will not close. I need to force it closed from the task manager.
VS Code version: Code 1.32.2 (e64cb27b1a0cbbc3f643c9fc6c7d93d6c6509951, 2019-03-13T02:01:00.888Z)
OS version: Windows_NT ia32 10.0.16299

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (8 x 2712)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Memory (System)
15.88GB (9.81GB free)

Process Argv

Screen Reader
no

VM
36%

Extensions (7)

Extension
Author (truncated)
Version

vscode-html-css
ecm
0.2.0

auto-close-tag
for
0.5.6

auto-rename-tag
for
0.0.15

beautify
Hoo
1.4.9

minify
Hoo
0.3.0

vscode-JS-CSS-HTML-formatter
lon
0.2.3

html-css-class-completion
Zig
1.18.0

Can you explain exactly what you open and happens when you try to close it? Is this a folder on a network drive?

Is this covered by one of the issues mentioned here? https://github.com/Microsoft/vscode/issues/70265#issuecomment-472732246

This could also be https://github.com/Microsoft/vscode/issues/70045


Weird cell proportions

Hi!
I'm testing linseed on RNAseq dataset. And I run into issue where linseed predicted proportions in summary gives >1, like:
                  sample1       sample2      sample3

Cell type 1    0.6073787   0.5740409    1.027164e+00
Cell type 2    0.5784655   0.6328196    7.704742e-06
Maybe you have any idea what could go wrong.
I'm using R 3.6.0, edgeR CPM matrix and there is only 2 cell types per sample.
I'm planning to try TPMs instead, will update if there is or there is not any changes.

Hi, I ran into the same thing. The proportions don't add to one and I have used TPM. Any idea why that would happen? Should I just normalize the samples to one?

@keremw, I got an answer from authors to my question via e-mail:
"this is not uncommon situation for predictors to give values that don't
sum to one - this constraint is not imposed in exact way in the DSA and
other similar algorithms.
I advise you to dig into the math of a process (both LinSeed or DSA etc)
a little deeper. "
I tried it with TPMs and got the same result. Also, I didn't have time to dig deeper into LinSeed/DSA math and just switched to another tool.

@pushtiks @keremw
sorry for the slow replies on my side
Weird proportions come from the fact that in both LinSeed and DSA we kinda try to solve sum-to-one constraint as good as we can, yet, we do not force the proportions to be exactly sum-to-one.
@keremw, if for your analysis sum-to-one is indeed required you can just try to force sum-to-one for the columns of the proportions matrix.
@pushtiks " Also, I didn't have time to dig deeper into LinSeed/DSA math and just switched to another tool.". Sorry to hear that, I will try to maintain the package better from now on.
Cheers,
Konstantin

@konsolerr
Konstantin, thank you for your reply! I'll also try to force sum-to-one of the proportion matrix then.
Could the issue be due to the "spill-over" effect, when the one cell type rise the proportions of another one?
Also, for example, we have cell-types A, B, and C. The C has sub-types C1 and C2, where C2 has a similar expression with type B. Is the "proportion" of C2 collapsed with both C and B cell-type proportions, or is it considered as ambiguous and thus removed from the analysis? So if the C2 is collapsed with two cell-types it could also make sum of proportions >1. These are just guesses of possible causes as I didn't dig into algorithms.
Cheers,
Vickie

@pushtiks
Vickie, the inability to fit sum-to-one constraint is rather technical: before we calculate the proportions we first find "pseudo-proportions" these are vectors changes in which correspond to proportions, however, they are found in different space, thus, pseudo-proportions won't fit sum-to-one constraint. After that we actually try to find such coefficients that would make our "pseudo-proportions" look more like actual proportions by solving linear equations to fit sum-to-one constraint. But these equations can not be solved accurately in most cases, so we "approximate" sum-to-one constraint but never force it.
I like your questions about cell types and subtypes!
Before answering your question I want to change a bit your perspective on the cell types and the signatures of the cell types.
When you have many cell types, it is easier to think in perspective of genes, signature genes and shared genes. Signature genes are defined by higher expression in one cell type while shared genes might have similar expression levels in several cell types.
Assume you have A, B, C1, and C2 (I assume that C2 is somewhat close to C1 but also have some expression signatures from B, if I read your example correctly). What LinSeed does is trying to identify linear subspace to put all the genes with respect to their expression in pure cell types. Imagine tetrahedron A-B-C1-C2 where we put all the genes with higher expression in A closer to vertex A (the same with B, C1, C2), all the housekeeeping genes will be in the middle of this tetrahedron. Now comes interesting case: we know that C1 and C2 are transcriptionally similar - it means that genes that are higher expressed in C will be somewhere in the middle of the edge C1-C2 of this tetrahedron, and if C2 has similar expression with B we will have some genes on the edge C2-B.
Now back to your question: if you have signature genes for C2 and C1, you will have marker genes in the corners of the tetrahedron, which means that you can just run the analysis with 4 cell types and you don't have to worry about "collapsing" C2 into C1 and B. However, if you run the analysis for 3 cell types it's hard to guess what the triangle projection will be. If C1 and C2 are very similar and C2 has just "some genes" shared with B than you will find you will find proportions of A, B, C1+C2. Otherwise it's kinda hard to tell.

@konsolerr
Konstantin, thank you very much for the answers! The picture is a lot clearer now, the "pseudo-proportions" part and importance of correct cell-type number are the pieces I missed. I think I have a plan on how to proceed further with my data and LinSeed.
Thank you and have a nice summer!
Vickie


Rework module macro to support targeting iota and cats coproduct types

Adds Iota 0.0.2 as a dependency
Introduces InjK to handle injecting to Iota and Cats coproducts
Reworks module macro to target both coproduct types. It would be possible to emit two "Op" types for each algebra, supporting both styles.
Adjusts docs so that they compile

There's a chance I missed something and the build will fail on CI 😢.
There's plenty of room for feedback on this. I could use some extra eyes to go through and help find the spots where documentation needs to change.

Codecov Report

Merging #261 into master will decrease coverage by 1.33%.
The diff coverage is 36.36%.

@@            Coverage Diff             @@
##           master     #261      +/-   ##
==========================================
- Coverage   74.75%   73.41%   -1.34%
==========================================
  Files          35       36       +1
  Lines         301      316      +15
==========================================
+ Hits          225      232       +7
- Misses         76       84       +8


Create task command and fix style

test

@albertarielw has this PR been superseded by https://github.com/AY2223S1-CS2103T-F12-2/tp/pull/52? If so, we can close this

Yes
On Tue, Oct 11, 2022, 23:02 Joe @.***> wrote:

@albertarielw https://github.com/albertarielw has this PR been
superseded by #52 https://github.com/AY2223S1-CS2103T-F12-2/tp/pull/52?
If so, we can close this
—
Reply to this email directly, view it on GitHub
https://github.com/AY2223S1-CS2103T-F12-2/tp/pull/50#issuecomment-1274841429,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYGL6ITUJOO5XTS6X7IKZD3WCV6OPANCNFSM6AAAAAARB2ZG6A
.
You are receiving this because you were mentioned.Message ID:
@.***>


Unit testing tmLanguage files [question]

Hi,
I've written a tool that allows to write assertions in a plaintext file against a textmate grammar, like so:
      def add(x:Int,y:Int) = x + y
          ^^^ entity.name.function.declaration

I use vscode-textmate to run the actual grammar internally. It is very similar to Sublime syntax tests but I've made a few deviations.
So, I thought maybe someone can take a look at it and if finds it useful maybe we can put some links in the documentation. What do you think?

:+1: Nice! a3f92e1396d1d1a43eb95a0c142c7966e50ba2dc


Exclude additional jackson core, databind, and annotations dependenci…

…es. Exclude event-hub-sdk which brings in jackson < 2.9.0. Predix auditing profile is temporarily commented out and is unusable.

approved by henry and I


Build with libitk5.3 support

Solution to issue cannot be found in the documentation.

[X] I checked the documentation.

Issue
The convert3d conda-forge package requires libitk 5.2, but does not declare this in its dependencies. Pinning 5.2 makes a conda environment that is incompatible with the ANTs package. It does not seem like you can have two versions of libitk simultaneously in conda. Would it be possible to update this package?
Overall, this does not feel great, as apps should not have to make new releases to support updated libraries, but I don't see a better short-term solution.
Installed packages
channels:
  - conda-forge
dependencies:
  - convert3d=1.3.0
  - libitk=5.2  # 5.3 fails with c3d
  - ants=2.4.4

Environment info
channels:
  - conda-forge
dependencies:
  - convert3d=1.3.0
  - libitk=5.2  # 5.3 fails with c3d
  - ants=2.4.4

The best option would be to figure out a patch to build with ITK 5.3 onwards, apply it here today and forward it upstream for inclusion later.
Is there an issue already filed upstream about the build issues with 5.3 that we could link to?

cc @pyushkevich for inputs

Any notion where to file an issue for conda to see how feasible supporting multiple versions of libraries in conda environments would be? Nowhere I can find so far seems suitable.

Sorry, by upstream I meant Convert3D directly.

I believe pinning only buy us some limited time, since eventually conda-forge will clear its cache from old build versions. I don't think they keep old versions indefinitely.

I doubt there will be any issues building c3d with itk 5.3
Paul
On Thu, Jul 20, 2023 at 4:36 PM Ghislain Vaillant @.***>
wrote:

Sorry, by upstream I meant Convert3D https://github.com/pyushkevich/c3d
directly.
—
Reply to this email directly, view it on GitHub
https://github.com/conda-forge/convert3d-feedstock/issues/1#issuecomment-1644043171,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJPEW2C6SD7GSMQQVDXZWTXRE66RANCNFSM6AAAAAA2ROYR64
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Paul Yushkevich

Thanks, Paul. Any chance the 1.4.0 tag could be pushed to GitHub?

I am attempting a build with the current tip of the main branch here.

I doubt there will be any issues building c3d with itk 5.3

@pyushkevich, there are a bunch of errors of the type: error: invalid new-expression of abstract class type.
See the logs for macOS and Linux builds here.

I also tried a build forcing the definition of ITKV4_COMPATIBILITY here just in case, which gives a different set of issues.


Major release supporting new API, subdomain, repo structure change and more [Do not merge]

This PR will upgrade all dependencies and upgrade Flutter SDK constraint to 2.8.
The branch will be used to release the following -dev versions
nhost_sdk               4.0.0-dev.x
nhost_flutter_auth       3.0.0-dev.x
nhost_gql_links          2.0.5-dev.x
nhost_graphql_adapter    3.0.0-dev.x
nhost_flutter_graphql    3.0.0-dev.x

Thank you!


[Functions] Fixes function worker get superuser role

Fixes #7879
Motivation
Function worker should use authorization service to check
a role if a superuser.
Modifications

Fix the isSuperuser method in the function

Verify this change

Adjust the original test case to verify it

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks


Special interfaces description length validation. Issue #9401

[ ] Redmine Issue: https://redmine.pfsense.org/issues/9401
[ ] Ready for review

If you make an OpenVPN interface name with 26 to 31 characters (in my case, I made them with exactly 26 and 31 characters), _VPNV4 and _VPNV6 are appended to them when making the gateways under System / Routing / Gateways. Trying to add a monitor IP is impossible because the 31 character limit is exceeded and attempting to change the gateway name to honor the limit is not allowed. Working around this requires picking shorter interface names.
This is caused by special suffixes added to some special interfaces:
_VPNV4 and _VPNV6 for OpenVPN and VTI interfaces
_TUNNELV4 and _TUNNELV6 for GIF and GRE interfaces

For example in the first block, a length of 25 would be rejected, but the error implies that should be valid.
strange issue - if I use (strlen($_POST['descr']) > 25)  this allows me to use names 26 characters long. End-of-line symbols?..

For example in the first block, a length of 25 would be rejected, but the error implies that should be valid.
strange issue - if I use (strlen($_POST['descr']) > 25)  this allows me to use names 26 characters long. End-of-line symbols?..

It shouldn't have anything extra unless the user accidentally added whitespace around it. Might try running it through trim() first.

trim() added
good now


Error POST http://localhost:5002/graphql 400 (Bad Request) query_blocks.js line 25

Pop up:
SyntaxError: "[object Object]" is not valid JSON

[x] Fixing error for loading query. So that we can save Blocks on the Database.

Now is Bad User Input

[ ] Fixing error with Register block now not correct inputs for blocks


Add resolverTemplate to allow for simple resolution without a custom function

Description
Right now, there are two image resolver options: id + '.tif' and the CloudFront preflight function. To add some flexibility, we should accept an optional printf-style template that the ID gets applied to. (e.g., '%s.tif', /path/to/images/%s.pyramid.tif).
Done Looks Like

[ ] Built-in resolver accepts a resolverTemplate environment variable
[ ] SAM templates updated to include resolverTemplate variable

Resolved by #78


Too many nodes and edges in family finding

Hi @Magdoll ,
One family finding command from generated batch commands in my project could not get through in a rather long time (roughly two days). It stuck at process_kmer_to_graph phase and took about 100G memory. Output information shows that graph in this cluster contains 49018 nodes and 79611730 edges , while corresponding numbers in other clusters are much smaller. Is this normal?

Hi @fantastycrane ,
Did you run run_preCluster.py? It is possible this is an unusually abundant gene, or minimap2 is too happy to pick up low similarity hits.
If the issue persists, I will come up with a way to resolve it.
--Liz

Hi @Magdoll ,
I did run run_preCluster.py, and preCluster.output.csv shows that 49643 sequences (67505 in total) were clustered into the same cluster. This number dropped to 27155 after I upgraded Cupcake. Still, it is too large for process_kmer_to_graph.
The same issue occurred in several different project.  It is noteworthy HQ sequences combined with corrected LQ sequences were used when running Cogent. Corrected LQ sequences had been filtered with accuracy above 0.99.
Results were relatively normal when only HQ sequences from IsoSeq were used.

Hi @fantastycrane ,
I'd like to take a look at it. I've requested a file upload to your email.
--Liz

Received and will work on it.

Working on it. Responding soon.

Hi Liz,
I met the same problem today. I have a family containing 43 K HQ and 141 K LQ sequences (corrected by Illumina reads), representing 65% of the total input sequences. I am not sure why this happened and how can I fix this problem? Can I just select those HQ sequences and use cd-hit to reduce the redundancy and then use the resulted sequences as the representative sequences for this family? Do you have a better suggestion? Thanks a lot.
Chen

Hi @Magdoll ,
I have learned previously that LQ sequences corrected by Illumina reads were not totally trustworthy. LQ was only included to gain more isoforms without requiring too much sequencing cells.
Hope that makes sense.
It would be much appreciated if your solution could work.

Hello @Magdoll ,
I have the same issue when running family finding of large dataset. After preCluster, I found one bin has 62124 contigs in the isoseq_flnc.fasta. When I run the family finding command for this bin, their is an error message of out-of-memory issue:

1.1
Expected output isoseq_flnc.fasta.s1000k30.dist already exists. Just use it.
WARNING: Output directory /storage/hpc/data/zzr25/pacbio_post_analysis/unalign_cogent/unaligned_total already exists
making weight graph from  /storage/hpc/data/zzr25/pacbio_post_analysis/unalign_cogent/preCluster_out/5818/isoseq_flnc.fasta.s1000k30.dist
graph contains 58165 nodes, 70780629 edges
performing ncut on graph....
slurmstepd: error: Exceeded job memory limit at some point.
srun: error: lewis4-r640-hpc5-node845: task 0: Out Of Memory
slurmstepd: error: Exceeded job memory limit at some point.

I am wondering what the problem is and appreciated if there are any solutions.

Hello @Magdoll,
the same appears on my dataset. But I have HQ isoforms ONLY coming from the isoseq3 pipeline (included smrtlink v6). The whole data set harbors 670,000 HQ isoforms (because we have generated 54 RSII and 15 Sequel SMRT cells for that project). It ended up with 210,000 isoforms within the largest bin. I aborted the process_kmer_to_graph process for that bin, as it ran more than two weeks and took more than 350 GByte of RAM at the end. Which parameters in which scripts does it make sense to play with?
Does it make sense to increase the number at the -f option in your hardcoded minimap2 implementation within your python script (AlignerRunners.py)? I ran it several times (0.00001, 0001, 0.001 and 0.02). At 0.02 it generates two bins with 50,000 and 46,000 isoforms. The number of orphans increases a little bit. Also the number of bins in total. The number of chimeras stays the same. Any help is appreciated.


Issues with broadcast folder

It might an harder problem to tackle but forge-deploy doesn't play well if you have files inside broadcasts/ that were deployed without forge-deploy, using barebone forge scripts.
Also, If I delete a deployment file so that it is forced re-deployed, forge-deploy sync is going to add back some of the old deployment back (if I didn't redeployed them yet).
Anyway, lots of small issues that prevents a fire-and-forget-about-it deploying experience.
My workaround for now is to rm -rf broadcast/ folder after every forge-deploy sync.

yes, I was foreseing this kind of issues
for issue 1, I am surprised it is causing an issue as I thought forge-deploy would skip things not matching its format. I ll investigate
for issue 2, I need to add a timestamp somewhere to not resync each time
Thanks again for reporting all these issues. this helps a lot


[BUG] codeunit 426 "Payment Tolerance Management" / IsCustBlockPmtTolerance

The function IsCustBlockPmtTolerance is Used in
local procedure SalesPmtTolGenJnl(var GenJnlLine: Record "Gen. Journal Line"): Boolean
var
NewCustLedgEntry: Record "Cust. Ledger Entry";
GenJnlPostPreview: Codeunit "Gen. Jnl.-Post Preview";
GenJnlLineApplID: Code[50];
begin
if IsCustBlockPmtToleranceInGenJnlLine(GenJnlLine) then
exit(false);
...
end;
This would mean that if the function gives TRUE as a result it would exit the function SalesPmtTolGenJnl.
local procedure IsCustBlockPmtToleranceInGenJnlLine(var GenJnlLine: Record "Gen. Journal Line"): Boolean
begin
CheckAccountType(GenJnlLine, GenJnlLine."Account Type"::Customer);
    if GenJnlLine."Bal. Account Type" = GenJnlLine."Bal. Account Type"::Customer then
        CODEUNIT.Run(CODEUNIT::"Exchange Acc. G/L Journal Line", GenJnlLine);

    **exit(IsCustBlockPmtTolerance(GenJnlLine."Account No."));**

end;
The Result is based on the result of function IsCustBlockPmtTolerance
Here is the problem. That function will always result in FALSE. It doesnt matter if the boolean "Customer."Block Payment Tolerance" is True or False. Because when it is on, it will exit the function with false. If not, it will exit the function and the result will also be FALSE.
local procedure IsCustBlockPmtTolerance(AccountNo: Code[20]): Boolean
var
Customer: Record Customer;
begin
if not Customer.Get(AccountNo) then
exit(false);
if Customer."Block Payment Tolerance" then
exit(false);
end;
Looking at the name of the function I would appear that the  function returns TRUE when the "Customer."Block Payment Tolerance" is TRUE. But then we have a problem with IsCustBlockPmtToleranceInGenJnlLine because that function should Return true when IsCustBlockPmtTolerance=TRUE. But when it's true it doesnt exit in SalesPmtTolGenJnl.
I think naming of the functions are mixed up.  Customer."Block Payment Tolerance" =false it should exit the function SalesPmtTolGenJnl. At the current state it of the naming and the IF's it continues.
For now I think this should be the change as a quick fix:
local procedure IsCustBlockPmtTolerance(AccountNo: Code[20]): Boolean
var
Customer: Record Customer;
begin
if not Customer.Get(AccountNo) then
exit(false);
if Customer."Block Payment Tolerance" then
exit(false);
   **exit(true);** //I Added this
end;

Same applies for the Vendor.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Cannot display the graph

Hi,
Everytime I try to display the graph using the code given in the examples I get this error "Uncaught graph-container is not available in this DOM or available multiple times". I feel dummy but I figure out where the problem come from.

What do you have in data-graph-container attribute of your  ?

The exact same thing shown in the examples ".. .. .highchart-container"

When you set data-graph-container to ".. .. .highchart-container", you set a relative DOM selector.
I means : search for .highchart-container element within my grand parent node.
Do you have the code online ? I can take a look at it :)

I put my code online http://help-isis.fr/projets/Stat/
Thx

You did not include javascript file from http://www.highcharts.com/download (that explains the "Highcharts is not defined" error).
We will change the documentation to explain it.
For the container, you have 2 options :

add a  in your page and set data-graph-container to "#highchart-container"
use data-graph-container-before="1"

Thx for the explaination. It works.

Cool :)
We are updating the documentation right now.

Hello there! How is the error fixed? I have the same error, I cannot figure what will be the solution

This is my error.


Why add a servant to the tars registry, which can find data in the database, but the registry does not?

E：Because the time of the tars registry is inconsistent with that of the database, the cache is always read and will not be updated. Kill the process of the registry, and the registry will restart.
C：因为Tars注册中心的时间和数据库的时间不一致，导致一直读的是缓存，不会更新。把注册中心的进程kill掉，注册中心重启后就有了。


The Debug derive macro does not need to be qualified

The windows-bindgen crate was inconsistently generating this derive macro due to some previous refactoring. This just switches to consistently using the unqualified form in all cases.

Build validation blocked by #3096


storage documentation / advice

This is semi-related to #246 about stabilizing more file operations.
With only putFile and getFile right now, how should storage be handled when 1) you need to access user specific data and 2) you need to access data for all users?
I've tried to look at other open-source Blockstack apps. Beacon seems to be in its early development or at least its changes are not public. Guild seems to be saving all the data in one file and iterating to find that of the correct user data (see here for source), but, unless I'm misunderstanding how storage works, this doesn't seem like an approach that will scale well.
I'll hack together an approach in the meantime but would prefer to know the proper way of handling storage. Thanks!

@alidcastano Hey thanks for starting this issue.
Can you explain what you mean by "access user specific data"? Do you mean that you have a central server and you want the locally stored data to be relayed back to your central server? Or do you mean that you have user A's data that needs to be seen by user B?
Regarding "access data for all users", can you elaborate on your use case? For fully decentralized apps this isn't possible because there is no central server or "mothership". That said, you can always include a call where user data is copied and synced to a central server. Highly depends on what you're trying to do.
Our ideal model for applications here is that we can eliminate the need for central servers at all. So apps like marketplaces and social networks need to be rethought. Very different approach and very different way of thinking.
Would love to know what app you're building. Happy to provide further guidance.

@shea256 Thanks for the quick response.
Based on your response, I need to rethink my approach which is anchored on a central server way of thinking.
I'm experimenting with a social note-taking app. The closest equivalent would be Quizlet.
So the basic functionality would be:

User A can write and access their data.
User B can see user A's data.

And vice-versa.
What's the recommended way of handling this via putFile and getFile?
Related question: so when you do puFile, user A's data is first being stored with all the other users' data in a decentralized fashion and but only user A's data is synced to a central location (e.g. his or hers Dropbox account)?

To add to the discussion I'll post relevant snippets of how current blockstack apps currently handle storage.
This is how the todos app handles storage (via vue component) :

 watch: {
    todos: {
      handler: function (todos) {
        const blockstack = this.blockstack
        return blockstack.putFile(STORAGE_FILE, JSON.stringify(todos))
      },
      deep: true
    }
  },
....
methods: {
    fetchData () {
      const blockstack = this.blockstack
      blockstack.getFile(STORAGE_FILE)
      .then((todosText) => {
        var todos = JSON.parse(todosText || '[]')
        todos.forEach(function (todo, index) {
          todo.id = index
        })
        this.uidCount = todos.length
        this.todos = todos
      })
    }
}

This is how the guild app handles storage:

export const saveBlogs = (blogs, dispatch) => {
  putFile(STORAGE_FILE, JSON.stringify(blogs)).then(isBlogSaved => {
    let user = loadUserData();
    if (isBlogSaved) { window.location = `/blogs/${user.username}`}
  });
}

export const fetchBlogs = dispatch => {
  var blogs = {}, blogIndex = 0;

  getFile(STORAGE_FILE).then(blogItems => { // eslint-disable-line
    blogItems = JSON.parse(blogItems || '[]');

    Object.keys(blogItems).forEach((id, index) => {
      blogItems[id].id = index+1;
      blogs[index+1] = blogItems[id];
      blogIndex = index+1;
    });

    dispatch(receiveBlogs(blogs, blogIndex));
  });

export const fetchUserBlogs = (user, dispatch) => {
  var userBlogs = {};

  getFile(STORAGE_FILE).then(blogItems => {
    blogItems = JSON.parse(blogItems || '[]');

    Object.keys(blogItems).forEach(id => {
      if (isBlogAuthor(blogItems[id], user)) {
        userBlogs[id] = blogItems[id];
      }
    });

    dispatch(receiveUserBlogs(userBlogs));
  });
};

They both store the core user content via a single file, e.g. todos.json or blogs.json. What I don't get is why does guild have to filter the user data, while the todo app doesn't. In other words,  is all user data being put in a single file or is it just the data of that specific user?
I'll be testing both approaches out soon but just laying down my thoughts here in the meantime.


Bayes is labeling everything as ham

I have this email address that I have moved into Modoboa and bayes has decided that everything gets labeled as ham with a bayes_00 flag. I have set up the spam folder to use sa-learn, but the --dump magic isn't increasing at all.
I wonder if I am training a second database by accident or if something is wrong. I used the installer script and am unsure how to clear and retrain the bayes database.
Thank you for any help

Update: I am closing this issue for the time being. I found that bayes was storing it under the --username=amavis user and am going to clear the database and re-teach it. I will see how that ends up.


fix(llm-markdown-code-rendering): Fixed markdown code processing

The issue with text processing when the LLM model returns incorrect Markdown code when ``` is immediately followed by other text has been resolved.
Here is an example of problem (replace ~ with `):
**Create a systemd service file:**
~~~sh
nano ~/.config/systemd/user/waybar.service
~~~

Unfortunately, RegExp is not suitable in some cases


ignore .babelrc on publish

According to https://github.com/loggur/react-redux-provide/issues/17:

React Native's made the migration to Babel 6, which has the unfortunate side effect of finding .babelrc files in node_modules and attempting to use those configurations (see this issue for more details).
As a result, Babel 5 configs break RN builds, and the team's official fix is to ask module authors if they would consider adding .babelrc to .npmignore.

See also https://github.com/facebook/react-native/issues/4062#issuecomment-179944792

Coverage remained the same at 100.0% when pulling b54aa27d562ed40b4e0cca691ddd9afe73806961 on audionerd:master into 1543910cfd23e01354039f3c659af5418d8728d2 on agraboso:master.

Fixed in v1.0.2. Thanks!


Move to storing state in ETS tables and execute in the calling process

This is a big win because we get to strip out the internal GenServer, which is awesome because sending the messages is like 90% of the actual overhead of Cachex.
This issue can't be resolved without either a minor (likely 1.2), or more likely a major (2.0), so no timeline on when this will be done. It's not yet guaranteed if this will actually be done, I'm just considering it.
The basic idea is to move all state into ETS itself so we read from ETS instead of storing in a GenServer. There are a lot of factors to this, and they're described in #39.
Reasons for doing this:

Any retrieval actions block the caller anyway, so this is no different.
A Cachex state changes only rarely, so we're pretty much just reading memory.
Huge performance boosts when moving to this model (right now you're looking at 9 microseconds, vs < 1 microsecond if using ETS).
The GenServer becomes an unnecessary bottleneck when using fallback functions.
It's much easier for the user to configure their own Tasks and Concurrency.
There should actually be very limited impact to the interface, only that of execute/3.

Things to consider:

Async behaviours are removed, because everything executes in the calling process. Not sure if this means a Major bump is required.
Timeouts are removed, so same as above because they do nothing.
We'll need Macros to avoid constantly getting/setting state.
What should the ETS table look like?
What happens if we crash? (I think this is redundant)
Synchronous hooks need to maintain their behaviour - they might clash with messages being passed by the user.

Discovered this can happen whilst maintaining a v1.2 label as we can keep interface compatibility.


Include styles.css in dist

Issue:
Related-to https://github.com/emilkowalski/sonner/issues/386
Granted, this should be seen as a workaround, since the css is already bundled in the js. The root cause of Remix's spotty support for sonner is unclear, but referencing the css directly works for me:
import sonnerStyles from 'sonner/dist/styles.css?url'

export const links: LinksFunction = () => [
    { rel: 'stylesheet', href: sonnerStyles },
]

What has been done:

[x] Copy the styles.css into the dist folder so it can be referenced by consumers as sonner/dist/styles.css

Screenshots/Videos:
N/A

Also see https://github.com/emilkowalski/sonner/issues/246

@emilkowalski could you please consider adding the styles to the final bundle? I'm not sure why anyone would be against that honestly, it's a very standard practice to include styles in the final output so the user has control over how to includet hem in their project.
E.g see rainbowkit: https://www.rainbowkit.com/docs/installation#import

Also requesting this @emilkowalski


updated diagram to reflect change to SNS event notification

Sorry for the lag - I didn't see a notification come thru.
:shipit:


What's the alternative for Service Fabric Cluster with managed disks?

What can we do if we want to move the service fabric cluster that contains the managed disks to csp one how to do that???

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 76c1c185-685f-27e0-e634-844b510b1834
Version Independent ID: 09375778-5354-ab44-c5c6-94c2aebe34bc
Content: Move Azure resources to new subscription or resource group
Content Source: articles/azure-resource-manager/resource-group-move-resources.md
Service: azure-resource-manager
GitHub Login: @tfitzmac
Microsoft Alias: tomfitz

@Anmolgan81 Good question. I have passed this on to the content owner to review and provide an answer. Please stay tuned.

sooner the better, we have a migration of vm scale set that includes service fabric cluster and managed vm scale set that has to be gone to csp, hence any help as quick as possible will be helpful, also can you updated the doc, and add a limitation for ADF V2 as well??

@Anmolgan81 - the article has been updated for ADF V2. It will get published this afternoon.

@tfitzmac Thanks for the update

@femsulu - I think this needs to be reassigned to service fabric or scale sets. Managed disks cannot be moved so there is no move solution. Service fabric or scale sets would know if there another work around.

@tfitzmac  - any insights on @Anmolgan81 question above?

@tfitzmac , ok let me dig into this further to see if there is a resource that can help.

@MicahMcKittrick-MSFT, any ideas who form the Compute side can help with @Anmolgan81's question?

Ack. Will Follow up on this :)

any updates in here??

This is not yet supported.

@Anmolgan81 - does it help to copy the managed disks to a new subscription? And then, create VMs from the copied managed disks?

@tfitzmac yeah I have been doing all the managed disks migration using the same method that you have described.
Its strange that 2TB of disks gets copied over in a matter of seconds

@tfitzmac when will we get the support for moving the managed disks??

@Anmolgan81 - Product announcements are made through the Azure roadmap. There are no announcements right now related to moving managed disks.

If support for moving managed disks is added, we will update this article. Until then, there is nothing related to this issue that we can add to the documentation.

#please-close


[5.6] Add table option to ModelMake command

Added option to the ModelMakeCommand to take optional table name to use when creating model.
I have worked on various projects where the table names are not plural or use other names. While it is not that difficult to edit the model and add protected $table = ... to the model, it would be nice to have an option to do it when creating the model at the command line.
php artisan make:model Foo table="foo" -m

No plans to add anymore options right now.


Remove small prop on select and text field

On both the select and text field, there is a small property toggle which is redundant and can be removed
Before conducting change, will need to create a new branch & double check with other designers that this wont have an impact on their work
Related to ticket #2335

Too many other components depend on this small property


Standardized mechanism for exposing load performance data to containers

In the container developed by the Fluid experiences team, it is critical for our team to obtain performance data on various pieces of the load path for end to end analysis. In particular, this container can be loaded on Teams, Office.com, Fluid Preview, or Outlook today, with ambitions to expand to other hosting apps in the future. Right now, load performance info is logged via the logger passed to the fluid runtime by the hosting app. Today, that logger happens to go to the FFX team, but we expect that plan to change over time.
From the perspective of the FFX team, the request is to have some standardized mechanism for a loaded container to access the data around how long various pieces of the load pipeline took. Today, this includes

Storage token fetch time
Snapshot fetch time
Whether the snapshot came from the network or a cache
Container code load

The goal here being that the container itself could log this data in their own telemetry pipeline in order to have the data to be fully accountable for load performance across all hosts.

@christiango, a lot of time has passed :) Is this issue still relevant?
If I understand the problem correctly, we are looking for something like performance counters.
I.e. driver & loader is essentially OS / kernel layer for container developer, so they should not have directly access to them, but some unified mechanism for all parties to expose perf data might be useful like it's done in win32.
Are there any other examples out there of how it may look like (API - wise)?

@vladsud  - to answer your question, yes I think the issue is still relevant as I think it is quite painful the way we have to extract runtime pef telemetry today. However, I don't think it's high priority and I'm not sure what the right API would be like

(This is an interesting case study for the position stated in #6976)


add jittered backoff

Based-on http://www.awsarchitectureblog.com/2015/03/backoff.html
Any thoughts?

this is very cool, thanks for contributing!

@mreiferson done

I guess my only other question is - should we make these backoff modes optional and configurable (i.e. expose an interface that the end-user can implement themselves plus provide default implementations for "exponential" and "full jitter")
@jehiah what do you think?

Maybe we can pick a reasonable default and expose the possibilities to overwrite the algorithm in the userland?

Exactly, I'm thinking something like"
type BackoffStrategy interface {
   Calculate(attempt int) time.Duration
}

type FullJitterStrategy struct {}
func (s FullJitterStrategy) Calculate(attempt int) time.Duration { ... }

type ExponentialStrategy struct {}
func (s ExponentialStrategy) Calculate(attempt int) time.Duration { ... }

And we would add a property to Config and default it to ExponentialStrategy, to maintain backwards compatibility.
What do you think?

Awesome. :+1:  Should I add something like a backoffConfig at L119 or just initiate a ExponentialStrategy in the composite literal?

Yea, add a new exported property somewhere around https://github.com/bitly/go-nsq/blob/master/config.go#L105?

This is looking really good, almost there, thanks!

closing in favor of #125


Code append for Azure AD B2C login - how?

I can contribute to the code which could benefit others, possibly. I have working code now to login to Azure AD B2C.
Don't feel confident to just push a new commit...

Is there any news on this? @JeroenBrouwers : Would love to use your code to work with OneDrive in VBA through the REST API...

Hi @JeroenBrouwers sorry for the delay. If you want to email it to me, I'm happy to take a look at it or you can just create a repo / gist and add a link to it here. Would love to see it.

In case @JeroenBrouwers is still active, I also have a need for this now and would love to review his work.


onChange fires twice

This is usually the case when the onChange is appended to the onClick and onChange handler.
I wanted to open a pull request but I don't have access.

I'm not really sure to understand the case you are describing. Can you provide an example ?

Since you have the onChange handler being called on both onClick and onChange. This is causing the callback to fire twice. So when I set a state to true it immediately sets it back to false.

Sorry, I'm not so sure anymore but it seems like the onChange handler fires twice. Whenever I set something to true it immediately sets it back to false.

Sorry nevermind


Search doesn't work and generate duplicated rows in table

react-bootstrap-table2 : 3.1.0
react-bootstrap-table2-toolkit: 1.4.0
I have a table only contains one column named "Class" and the value of the date is an array ['system', 'processer', 'system', 'processer', 'processer']
When I search on that column and type in processer, duplicated rows got generated and the result was messed up.
Repro:
export class Test extends React.PureComponent {
render() {
const { SearchBar } = Search;
const products = [
{
Class: 'system'
},
{
Class: 'processer'
},
{
Class: 'processer'
}
];
const columns = [
{
dataField: 'Class',
text: 'Class'
}
];
return (
  <ToolkitProvider
    keyField='Class'
    data={products}
    columns={columns}
    search
  >
    {props => (

Try to Search Bob, Cat or Allen instead of 0, 1 or 2

        <SearchBar {...props.searchProps} />

        <BootstrapTable {...props.baseProps} />

    )}
  </ToolkitProvider>
);

}
}

It seems the reason is the keyField is not unique, after adding one more unique column, it works.


Feat(variables-scss): Introduce export to javascript #DS-1437

[x] Add global index files
[x] Add formatting of js output
[x] Add functionality for exludedGroups (for other file)

@crishpeen points:

[x] @themes - please add a comment noting which theme is default. It can be basically anywhere in the file
[x] @global - please add empty line to the end of the file
[x] everywhere if the value is 0px, please remove the units
[x] in scss and js add comma after line-height in typography
[x] in js prettier expects spaces instead of tabs
[x] in js some objects can be simplified eg. focusRing: focusRing, -> focusRing,

Thing out of the scope will be solve in DS-1486

gradients and shadows color hexes can be shortened (in globals) - can be solved in the last issue (if you don't solve it here, add it to other issue, please)
can forwards be sorted in global/index.scss
in js eslint is failing on imports, probably not a problem of exporter, but I don't know what it is
I don't see all colors in theme-light-on-brand, i know not all colors are set, but even the set ones are not there

Description

Additional context

Issue reference

coverage: 74.468% (-4.3%) from 78.732%
when pulling d71c455712740f4a41bf4c2d966a511de00e538f on feat/ds-1437-export-into-js
into 4ab9f620ec62edddf5d8b7d64bb158bba371a603 on main.

Please copy locally the exported files into the design-tokens package to see how it deals with the linters.
My points:

@themes - please add a comment noting which theme is default. It can be basically anywhere in the file
@global - please add empty line to the end of the file
everywhere if the value is 0px, please remove the units
gradients and shadows color hexes can be shortened (in globals) - can be solved in the last issue (if you don't solve it here, add it to other issue, please)
in scss add comma after line-height in typography
can forwards be sorted in global/index.scss
in js eslint is failing on imports, probably not a problem of exporter, but I don't know what it is
in js prettier expects spaces instead of tabs
in js some objects can be simplified eg. focusRing: focusRing, -> focusRing,
I don't see all colors in theme-light-on-brand, i know not all colors are set, but even the set ones are not there

Thanks. I solved some things, some were outside the scope of this issue and I created a follow-up issue. See description above.

One little closing remark: I would prefer normalizing the themes directory content to the same casing, like darkmode or dark-mode, Dark theme to dark-theme or darktheme and same for the Lightmode, please.
Otherwise it is looking good. Great Job. Thanks 👍

One little closing remark: I would prefer normalizing the themes directory content to the same casing, like darkmode or dark-mode, Dark theme to dark-theme or darktheme and same for the Lightmode, please.
Otherwise it is looking good. Great Job. Thanks 👍

Sorry, but I don't understand what you mean. Themes are loaded dynamically, including the name. So it depends on how it is named in the supernova.

Sorry, but I don't understand what you mean. Themes are loaded dynamically, including the name. So it depends on how it is named in the supernova.

Ok, I see. So we need to standardize the naming of the themes or normalize them in JavaScript. Empty space can cause some troubles and I would like the directory structure to be normalized.
I will create an issue/long topic for this.


Add a macros

Add a macros or other preconfigured script to control wifi routers (like as vSSH)

Coverage increased (+0.2%) to 34.0% when pulling ebd0166840836b5346d0c87249cb60aa37a665ba on master into 13cb6b2dda983bc2af7c44280649e7a3b010cd33 on release.


SkiaStringSizeService and measuring strings in general seems off.

Apparently Skia is used in Chrome as well. However, it seems that the measuring of strings in Microsoft.Maui.Graphics.Skia.SkiaStringSizeService incorrectly uses the passed in font size as the height.
In Chrome or Edge
var el = document.createElement("div");
document.body.append(el);
el.innerText = "Hello, Maui.Graphics!"
el.style.fontFamily="Arial"
el.style.fontSize="13px"
el.clientHeight

15

15 is the correct height of the text when setting the font size to 13px for Arial. This value is returned by Chrome and Edge, both apparently using Skia.
I get the same correct value the good old gdi+ and libgdiplus setting the font size to 13 pixels, the height is 15.

But Microsoft.Maui.Graphics.Skia.SkiaStringSizeService always returns the incorrect height simply returning the size.
Is there a settings I'm missing or an additional way to measure a string correctly?

Use libgdiplus. this library is broken and incomplete

Yes we are. I’m planning for a feasible replacement. Ms seems determined to
block usage of libgdiplus at some point. Aspose has a fully managed
system.drawing library but the oem license is unworkable for us.
Another option is to simply take over libgdiplus code for our needs (which
are very modest). The c code is relatively small and well written and easy
to maintain for c/c++ developers. Cairo seems to work well too.
Otherwise I’d have to dig into chromium’s usage of skia to see how it
measures text blocks (with horizontal constraints, etc) since it seems not
to be implemented in skia by itself. But I may be wrong since I haven’t
spent time on this part yet.
--
Ice Tea Group
http://wisej.com
http://iceteagroup.com
+1-202-449-3778 x701


default values for elements

is there any ways to set default values for different elements?

@ManiAm If you enter the values in the form and save the Dashboard, it will be the default values.
If you use the Initial Request, the default value will be overridden with the returned value.
In the upcoming v3, you must save defaults using a separate button, as we are moving to the local states for updating values.


add flask socketio

Hi,
For a project I'm working on, we would like to do bidirectional streaming of information over the web for which we need WebSockets.
We tried using python web sockets, but it was too low level for our purposes. Also we are using Flask, so we want to use the Flask framework.
https://pypi.org/project/Flask-SocketIO/
https://flask-socketio.readthedocs.io/en/latest/

Thanks @hidmic for your help on this PR! We are closing it since we were able to find an alternative way to download the library we needed (via Dockerfile).


trigonometry

I'm tempted to add modulus, but I'm not sure whether int and double modulus are the same polymorphic operation, or if it should be mod and fmod
! @jystic @tranma

I'll just add some sanity tests in a sec

interesting tidbit: ages ago, when classifying double operations into whether they produced nans or not, atan, atan2, cos, sin, tan and tanh are all "clean". it feels more consistent to just say all double operations are possibly, though. any preference?

I'm not confident enough in my double testing code to say for sure that all tans are valid. I believe that tan (pi/2) = sin (pi/2) / cos (pi/2) = 1 / 0, so it should be able to produce infinity, even though for me cos (pi/2) = 6e-17.
I don't think we lose anything by making them Possibly, so I'll keep it.

I realised integer modulo is a somewhat larger job, because it would need to be in Possibly (1 mod 0). the machinery for this is already in place for doubles, but would be a bit different for integers.
so, given we can already encode modulo clumsily, I will leave both modulo operations for later

going to hold off on merging this, because if anybody is using any of these as variable names it will break. I had a look but couldn't see any.


Twilio component no longer loading

Running 0.28 and Twilio no longer loads. I see this error in the log.
16-09-21 10:01:51 homeassistant.bootstrap: Invalid config for [notify.twilio_sms]: expected string or buffer for dictionary value @ data['from_number']. Got<PHONE_NUMBER>2 (replaced my number)
Also this one.
16-09-21 10:05:00 homeassistant.core: Unable to find service notify/twilio
Problem-relevant configuration.yaml entries and steps to reproduce:
- platform: twilio_sms
  name: twilio
  account_sid: !secret twilio_acct
  auth_token: !secret twilio_auth
  from_number: !secret twilio_num

Add quotes around your your number.


Rebase with --autosquash

This would enable users to use --fixup and --squash, which makes for a convenient review workflow (no force-pushes required), but you still get a clean history afterwards.

Mostly implemented on the fixup branch, b94be82.
As part of future commit message validation, could fail integration if a commit message starts with fixup!.


Uncaught Error: ENOENT: no such file or directory, open '/Users/inigohernandez/.atom/packages/sav...

[Enter steps to reproduce below:]

...
...

Atom Version: 1.1.0
System: Mac OS X 10.11.1
Thrown From: Atom Core
Stack Trace
Uncaught Error: ENOENT: no such file or directory, open '/Users/inigohernandez/.atom/packages/save-session/projects//Applications/MAMP/htdocs/iWineBackend/project.json'
At fs.js:60

Error: ENOENT: no such file or directory, open '/Users/inigohernandez/.atom/packages/save-session/projects//Applications/MAMP/htdocs/iWineBackend/project.json'
    at Error (native)
    at EventEmitter.ipc.sendSync (/Applications/Atom.app/Contents/Resources/atom.asar/renderer/api/lib/ipc.js:21:31)
    at BrowserWindow.RemoteMemberFunction [as getSize] (/Applications/Atom.app/Contents/Resources/atom.asar/renderer/api/lib/remote.js:140:29)
    at Atom.module.exports.Atom.getWindowDimensions (/Applications/Atom.app/Contents/Resources/app.asar/src/atom.js:510:29)
    at Atom.module.exports.Atom.storeDefaultWindowDimensions (/Applications/Atom.app/Contents/Resources/app.asar/src/atom.js:542:25)
    at module.exports.WindowEventHandler.handleWindowBlur (/Applications/Atom.app/Contents/Resources/app.asar/src/window-event-handler.js:228:19)

Commands
  4x -1:47.9.0 core:move-down (atom-text-editor.editor.mini.is-focused)
     -1:47.1.0 core:confirm (atom-text-editor.editor.mini.is-focused)
     -1:47 project-manager:list-projects (atom-text-editor.editor)
  2x -1:45.4.0 core:move-down (atom-text-editor.editor.mini.is-focused)
     -1:44.5.0 core:confirm (atom-text-editor.editor.mini.is-focused)
 10x -1:42 click (div.header.list-item)
     -0:50.9.0 core:copy (atom-text-editor.editor.is-focused)
     -0:49.4.0 click (atom-text-editor.editor.is-focused)
     -0:48.8.0 editor:newline (atom-text-editor.editor.is-focused)
     -0:48.6.0 core:paste (atom-text-editor.editor.is-focused)
  3x -0:45.8.0 click (atom-text-editor.editor.is-focused)
     -0:42.6.0 core:backspace (atom-text-editor.editor.is-focused)
  4x -0:41.3.0 click (atom-text-editor.editor.is-focused)
     -0:14 core:paste (atom-text-editor.editor.is-focused)
  3x -0:11 click (atom-text-editor.editor.is-focused)
     -0:02.6.0 core:save (atom-text-editor.editor.is-focused)

Config
{
  "core": {
    "themes": [
      "native-ui",
      "monokai"
    ],
    "projectHome": "/Applications/MAMP/htdocs/",
    "disabledPackages": [
      "local-history",
      "cakephp-snippets",
      "live-archive",
      "goto"
    ],
    "destroyEmptyPanes": false,
    "excludeVcsIgnoredPaths": false
  },
  "editor": {
    "fontSize": 15,
    "invisibles": {},
    "showIndentGuide": true,
    "softTabs": false,
    "tabLength": 4
  }
}

Installed Packages
# User
atom-beautify, v0.28.17
atom-minify, v0.5.0
atom-pair, v2.0.10
autocomplete-php, v0.3.7
color-picker, v2.0.13
editor-stats, v0.17.0
file-icons, v1.6.11
laravel, v0.7.0
linter, v1.10.0
linter-csslint, v1.1.0
linter-php, v1.1.1
monokai, v0.18.0
native-ui, v0.7.1
php-debug, v0.1.3
php-extract-method, v0.2.0
pretty-json, v0.4.1
project-manager, v2.6.5
save-session, v0.15.7
symbols-tree-view, v0.10.1

# Dev
No dev packages

I think this exception has to do with the save-session package. Does it happen if you disable or uninstall that package?

Since there hasn't been a response to the request for more information, I'm going to close this. Please let us know if the issue reproduces for you.


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 509

Questions ?
Please refer to the Backport tool documentation


Housekeeping

Test failures are due to https://github.com/bpmn-io/bpmn-js/issues/2073 which either needs to be fixed here or (as suggested) at the core.

Updating to<EMAIL_ADDRESS>indeed fixes the issues.


DTU's coordinate system convention

Hi, thanks for the wonferful work!
I'm using a NeRF synthetic dataset, which follows OpenGL coordinate system convention (x-axis to the right, y-axis upward, and z-axis backward along the camera’s focal axis). When I apply the dataset to VolSDF directly, the computed ray_dir is incorrect.
I think the problem is in the rotation matrix, DTU/BlendedMVs might follow a different convention. But I couldn't find anything about the coordinate system convention of DTU dataset, I'm wondering could you give me a hint?
Thank you very much!

The coordinate system is: x right , y downward and z forward.


Follow up: Implement proper fix for android textinput selection workaround

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Context
In https://github.com/Expensify/App/pull/11684 there was a workaround introduced. See this conversation for more detail:
https://github.com/Expensify/App/pull/11684/files#r1013587630
It's about the android TextInput not updating its selection correctly.
Action Performed:
Select a range of text, then call textInput.setSelection(5, 5) on a text selection (for example).
Expected Result:
The android text input selection should just work as it does on web or iOS.
Actual Result:
Under some circumstances, the selection won't get updated actually on Android.
Workaround:
Already in place
Platform:
Where is this issue occurring?

Android

Version Number: 1.2.25-0
Reproducible in staging?: /
Reproducible in production?:  /
Email or phone of affected tester (no customers):  /
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
Expensify/Expensify Issue URL:
Issue reported by: @hannojg
Slack conversation: /
View all open jobs on GitHub

@roryabraham please reopen 🍀

Assigning to @hannojg to work on the upstream fix, but @hannojg let us know if you don't have time or get stuck

@roryabraham - do I need to create a job in Upwork too?

No, @hannojg will invoice us separately

@hannojg - any update? How are things going?

@hannojg - how's it going here?

Hey, I am currently stuck in trying to create a bug ticket on the android framework AOSP. Will reach out to my internal network and trying to reach an android engineer from google to help me with that ticket.
As far as I understand the bug, there is nothing we can really fix, but it has to be fixed in the android framework.

Chatted in slack here

@hannojg Let us know what issues you've run into creating the Android bug ticket, maybe we can crowdsource a solution in #expensify-open-source?

Hey @roryabraham the issue is that after creating the issue you don't seem to get access to it (but neither its in a review queue, its just gone it seems).
You can see screenshots of the issue here (also if you continue reading the comments you will find more screenshots):
https://github.com/facebook/react-native/issues/35350#issuecomment-1326184026
My idea now is to try to reach the google team (I already tried but weren't successful yet).

Here's my two cents on this issue: I think we should remove the Bug label and instead add the Planning and Weekly labels. The thing is, we already solved the "bug" in the linked issue, and this is effectively a research issue on how to potentially solve Android bugs. So yes, I think that's great, but I also don't think this is really within the scope of WAQ given that we have something that has resolved the bug for now.

Thanks for the feedback. I'm going OOO until January 3 and will catch up then to identify if there is any movement here.

I'm still OOO and plan to review next week when I'm back online.

I've asked for feedback here:
https://expensify.slack.com/archives/C01GTK53T8Q/p1673049231644019?thread_ts=1671463643.046229&cid=C01GTK53T8Q

No update, still in the planning stage

No update

Follow up: the react native team has recognised this issue and reopened the issue we opened for it:

https://github.com/facebook/react-native/issues/35350

I am pretty sure they will help forward the issue to the android engineering team, will keep this issue updated !

Thank you for this update!

Can we get a minimal repo that shows the issue? Then we can fix it.

@hannojg Do you still have your minimal reproduction in a native Android app? Any chance you can share that?

@hannojg when you get a chance this week, can you answer @roryabraham's question? Thanks!

@hannojg any update here?

Let's close this for now and move on to bigger and better things. I'll subscribe to the upstream issue and reopen this if it is ever resolved in Android itself.


How do you customize components? Like adding some user input text into a component.

That's out of the scope of this project.


Developed setting 'mouseOpensDefinitionInPeek' which controls whether the mouse click opens element definition in the peek widget.

This PR fixes #62571.
Now there is a setting to control whether the mouse click opens element definition in the peek widget, which is equivalent to Visual Studio configuration.

Default is false to maintain the same behavior as before (tipically opening the definition in a new window).

Oh, I'd like to point this out. When this setting is true I can open 'peek' inside 'peek' with ctrl+click.
This doesn't seem to be a problem because it worked really fine for me, but I'm not sure if this is the expected behavior.

This doesn't seem to be a problem because it worked really fine for me, but I'm not sure if this is the expected behavior.

That's unwanted behaviour

That's unwanted behaviour

Thanks for your feedback!
I will take a look on this as soon as I have free time.
I just noticed that on issue #6857 you mentioned:

we should have a way to invoke reference search/peek def in nested editors but without nesting peek inside peek

Maybe if I solve issue #6857, the behaviour of the current PR will be solved as well.
Anyway, I'll take a look on that! :thumbsup:

Thanks!

@jrieken Uhm, I noticed that 'peek inside peek' already happens today even in 1.41. If we use "Peek definition" menu and after that mouse+click an element which has multiple definition, a peek widget is open recursively.

What should we do?
Because according to @stevencl comment on #6857 it's not that simple to have 'breadcrumbs' like VS.

Can you try with insiders? There was something I have recently fixed, tho unsure if it also covers that. Otherwise, yeah file an issue please

Tested on Insiders and works fine :+1: