Proposal: Remove fixed Kotlin runtime version limitation in Gradle

Today Gradle bundles an embedded version of Kotlin. My understanding is that this is due to usage of private compiler APIs in order to support kotlin-dsl. This prevents all plugins on the classpath from using different (let alone newer) Kotlin versions though, even if both they and the consuming build are not using kotlin-dsl.
This is something that I feel is fundamentally holding back the ecosystem, because it negatively affects plugin developers in the following ways:

They must set arbitrary language settings on their build, like lowering the languageLevel setting to 1.3 today, otherwise risk accidentally writing incompatible code.
For much of Kotlin 1.4.x's release cycle, they could not use Kotlin 1.4 period. Kotlin 1.4 was not added until Gradle 6.8, and even then it didn't allow kotlin 1.4 language targeting.
Language level alone only covers their own sources, but then they must also meticulously test any Kotlin-using external libraries that may have updated to 1.4 and use 1.4 APIs.

A good example of where this comes up is Okio, OkHttp, or Coroutines. Even something as simple as a member reference becomes a breaking change when compiled in 1.4 and run in a 1.3 environment.

This would be less of an issue if Gradle updated its embedded Kotlin version regularly, but the judgement today is that it cannot target a new minor version of Kotlin (now on a ~6mo cadence) until a new major version of Gradle (~once per year). While this may have lined up somewhat with Kotlin's previous ~annual release strategy, this is unwinding now with Kotlin 1.5's upcoming release. As of now, Gradle is on track to miss the entirety of Kotlin 1.4 due to its reluctance to update or allow users to use a new version. This will only continue now that Kotlin's release cycle is accelerated.
I understand that part of this reluctance to allow newer versions is because they may break. However, I think this is easily remedied by simply making it clear what the currently supported version is. Gradle adopts this approach with Java today, where it often supports running on a newer version but may not officially support it until a later release (major or otherwise). Gradle added Java 15 support in the 6.7 minor release, for instance.
The private API usage does offer a good reason for using a fixed version, but again there is precedent for this too as Gradle uses private javac APIs too (namely from the private-ish tools jar) to support cases like incremental compilation.
I would like to propose adopting one of the following strategies (in order of preference, though they're not all mutually exclusive either):

Unbundle Kotlin, remove private API usage or move it behind a flag. "If you want to use these APIs in kotlin-dsl, this must be enabled and comes with these restrictions"
Unbundle Kotlin, use a simple minimum/supported version and denote new un-tested versions as unsupported "your mileage may vary"
Allow/commit to more frequent updates. I.e. allow minor Kotlin version updates in minor Gradle version updates
Unbundle Kotlin, shade the embedded kotlin compiler or its privately used APIs to avoid conflict

I would say my preference order is:
3
2
1
4

This prevents all plugins on the classpath from using different (let alone newer) Kotlin versions though, even if both they and the consuming build are not using kotlin-dsl.

The statement above is not true.
Currently Gradle only exposes (or should only expose) the Kotlin runtime (basically kotlin-stdlib* and kotlin-reflect*, no compiler libs) to the plugin/script classpath. So while plugins that, for one reason or another, are incompatible with the particular version of the Kotlin stdlibs shipped with Gradle are not supported, this is not the same as preventing all plugins on the classpath from using different Kotlin versions.
That being said, I do appreciate how inconvenient and far from ideal the status quo is and agree we should find a better strategy.

Another example of where this causes friction: in most repos, it ends up requiring maintaining brittle infrastructure to support two kotlin versions. If a repo chooses to move ahead, it faces noisy, obscure warnings from kotlinc like this:
w: Runtime JAR files in the classpath should have the same version. These files were found in the classpath:
    /home/runner/.gradle/wrapper/dists/gradle-7.0-all/9m115ut5nwvtxli7nys8pggfr/gradle-7.0/lib/kotlin-stdlib-1.4.31.jar (version 1.4)
    /home/runner/.gradle/wrapper/dists/gradle-7.0-all/9m115ut5nwvtxli7nys8pggfr/gradle-7.0/lib/kotlin-stdlib-common-1.4.31.jar (version 1.4)
    /home/runner/.gradle/wrapper/dists/gradle-7.0-all/9m115ut5nwvtxli7nys8pggfr/gradle-7.0/lib/kotlin-stdlib-jdk7-1.4.31.jar (version 1.4)
    /home/runner/.gradle/wrapper/dists/gradle-7.0-all/9m115ut5nwvtxli7nys8pggfr/gradle-7.0/lib/kotlin-stdlib-jdk8-1.4.31.jar (version 1.4)
    /home/runner/.gradle/wrapper/dists/gradle-7.0-all/9m115ut5nwvtxli7nys8pggfr/gradle-7.0/lib/kotlin-reflect-1.4.31.jar (version 1.4)
    /home/runner/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.4.31/e613be5465ef1e6fd0468707690b7ebf625ea2fe/kotlin-stdlib-jdk8-1.4.31.jar (version 1.4)
    /home/runner/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.4.31/63db9d66c3d20f7b8f66196e7ba86969daae8b8a/kotlin-reflect-1.4.31.jar (version 1.4)
    /home/runner/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.4.31/84ce8e85f6e84270b2b501d44e9f0ba6ff64fa71/kotlin-stdlib-jdk7-1.4.31.jar (version 1.4)
    /home/runner/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.5.0-M2/575e1b43819635280e68bd3d3e8441603e97bbc3/kotlin-stdlib-1.5.0-M2.jar (version 1.5)
    /home/runner/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.5.0-M2/5e6ad08bde4c95d93ca6b366ca5e3a0bb49c70bc/kotlin-stdlib-common-1.5.0-M2.jar (version 1.5)
w: Consider providing an explicit dependency on kotlin-reflect 1.5 to prevent strange errors
w: Some runtime JAR files in the classpath have an incompatible version. Consider removing them from the classpath
e: warnings found and -Werror specified

> Task :agp-handlers:agp-handler-42:kaptGenerateStubsKotlin

FAILURE: Build failed with an exception.

Even worse - it's per-project and it prevents use of -Werror.

@ZacSweers this is the problem on KGP side: https://youtrack.jetbrains.com/issue/KT-41142

The Forward Compatibility section of the kotlin-dsl docs mention this:

The biggest issue is the compatibility between the external kotlin-gradle-plugin version and the kotlin-stdlib version shipped with Gradle. More generally, between any plugin that transitively depends on kotlin-stdlib and its version shipped with Gradle. As long as the combination is compatible everything should work. This will become less of an issue as the language matures.

Kotlin as a language is mature today and guarantees backward compatibility. Is Gradle claiming to add an extra layer of compatibility or is this doc out of date?

For the warn Runtime JAR files in the classpath should have the same version
Seems it's also reported as https://youtrack.jetbrains.com/issue/KT-41142 and already be fixed in org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.10

I thought that as well, but seems like it's a different issue (related to the Kotlin Gradle plugin's dependency).
I upgraded to Kotlin 1.5.10 and this issue still occurs for me.

@Tapchicoma my issue is that Gradle is still using Kotlin 1.4.31 but I'm using Kotlin 1.5.X.
My understanding is that's what this issue is about.

I've investigated further, and found that my specific issue comes from the java-gradle-plugin (specifically that it adds gradleApi() as a dependency).
So it seems that if you are building a Gradle plugin you need to have that project specifically use whatever version of Kotlin is being used by your version of Gradle.
That's not ideal because it means that a plugin that is being developed as part of bigger ecosystem can't share common build logic with the rest of the projects. If that's unavoidable then there should be better documentation/warnings around this.

And the Gradle init template should also reflect that.

Just to add on to my previous comment, not only can the plugin project not share common build logic with the rest of the projects, but none of the other projects can use buildSrc to configure Kotlin since adding the implementation dependency in buildSrc will cause this issue with the plugin project (see https://github.com/detekt/detekt/pull/3835 as a consequence of this).
The only way I can think of to achieve this would be to make the plugin an entirely separate gradle project and include the other project as an included build (although I'm not sure in that case if the incompatible version of Kotlin would be included in the classpath).

As a plugin author, I support @ZacSweers on this ticket. I don't want to be told by Gradle what Kotlin version to use. My order of preference is 1, 3, 4, 2.

@eygraber FWIW, you can compile a Gradle plugin with Kotlin 1.5 without the warning by using dev.gradleplugins:gradle-api:7.1.1 instead of gradleApi() in your dependencies. Something like:
dependencies {
  compileOnly("dev.gradleplugins:gradle-api:7.1.1")
}

// remove gradleApi() automatically added by `java-gradle-plugin`
// This is fragile because it's based on the name of the dependency. Maybe it would be easier to not apply
// `java-gradle-plugin` in the first place
val apiDependencies = project.configurations.getByName("api").dependencies
val gradleApi = apiDependencies.firstOrNull {
  val targetComponentId = (it as? DefaultSelfResolvingDependency)?.targetComponentId?.toString()
  targetComponentId == "Gradle API"
}

if (gradleApi != null) {
  apiDependencies.remove(gradleApi)
}

dev.gradleplugins:gradle-api:7.1.1 artifacts are redistributed by Nokee (see also this issue) and are resolved together with all other dependencies. Meaning if two versions of kotlin-stdlib are in the dependencies, they will be resolved according to Gradle conflict resolution rules and the highest one will be selected (1.5 here). I'm not sure why the warning happens with gradleApi() but it looks like it somehow bypasses the standard conflict resolution rules.

The higher one won't be selected because Gradle always forces the version it uses internally

@ZacSweers it won't change anything at runtime (when executing the build) but I'm pretty sure it will remove the warning when compiling the plugin.
A plugin build is actually pretty similar to any other Kotlin lib build. I found out the only reason it output warning was because java-gradle-plugin was forcing api(gradleApi()). Not 100% what's that doing behind the scene but it seems like it forces another kotlin-stdlib version in the compile classpath.

@melix You posted a great explanation of this issue and why it currently works this way on the Android Study Group Slack. Would you mind pasting it here?

I cannot speak for the Gradle team anymore, but this is the answer I gave to this question in a private group. It's adapted from a conversation so please bare with me with the layout. I will let the Gradle team answer/comment on the rest :)
You cannot change the version of Kotlin that Gradle uses. It's a "runtime": Kotlin build scripts have to be compiled, so Gradle has to come with one version of Kotlin. Your plugins, if written in Kotlin, therefore have to use the same version of Kotlin, because that's the runtime they will use for their own classes: the build scripts will be compiled by Gradle using the embedded version, and the same version is used to load your plugin. This doesn't mean that your plugins can't use a different Kotlin version for their task actions: use the worker API to isolate things properly. And it doesn't mean either that your apps can't use a different Kotlin version: because the compiler is executed in a different daemon, that's not a problem. In other words, the only issue is for your own plugin classes.
And to be clear, when I mean to load your plugin, I mean think about code like tasks.withType<MyTaskWrittenInKotlin>: the type MyTaskWrittenInKotlin is using Kotlin internal types which are also used by the withType extension method: they are effectively sharing the same Kotlin runtime. What happens within the task action is under your control, though. There you can isolate things properly using the worker API.
When you write:
plugins {
    id("my-plugin-written-in-kotlin")
}

the plugins block is compiled... in Kotlin. So Gradle has to come with a runtime with Kotlin. That's the embedded version. This script will resolve the plugins block, but as you are aware there are many others (settings, buildscript block, ...). All in all, it's during that phase that the plugins are resolved and that their classpath is computed. As a complicating factor, plugins are cached by classloader, by set of plugins applied, so different sets of plugins on different subprojects may have different resolved versions.
All in all, it means that even before we know that we need to apply your plugin, Kotlin is already initialized and evaluated. Then, the full build script needs to be compiled. And there, depending on what you have on classpath, you may have plugins written in different versions of Kotlin which require different levels of language. This is no surprise: Gradle for example had to enable experimental flags to support script compilation, or enable certain type of inference, ... Meaning that the horrible truth is that different Kotlin versions are not compatible.
To some extent, this is very similar to this problem: take a plugin written in Java, compiled for Java 16. Try to run a Gradle build with Java 8: it will blow up, because class format is not compatible. There's simply nothing we can do against this, except telling plugin authors: try to use the lowest version of Java possible to maximize compatibility.
Because you can't switch the Java version of the Gradle runtime: it's simply not possible: the user has to run on a higher version in this case. So why I understand the frustration, there are good reasons why it's designed this way, and no, it's not easy to change. Groovy users have lived for years with this limitation tbh (for plugins written in Groovy) but this is also why, in my personal case, am trying to write my plugins in pure Java now, to avoid those problems. And btw, I'm not even talking about the cost of having multiple language runtimes at execution time: I'd bet you would end up complaining that Gradle uses too much resources.
So, the rules are simple:

you want to write your plugin in Kotlin? Fine, live with the restriction: use the embedded version
you want to support executing task actions with any version of Kotlin? use the worker API

We cannot have our cake and eat it too...

Could Gradle compile the script using 1.4 in a first step. And during execution use 1.5 if some plugin pulls it in its classpath? Code compiled with 1.4 compiler should run fine with stdlib 1.5?

I guess in theory it's doable. But, what a nightmare. As I said the compiler options that Gradle has to inject are different for different Kotlin versions, because features are experimental in one release, not in another, etc. The script compilation infrastructure is already quite complex, so if it has to know about different Kotlin versions and the user classpath in addition, that's becoming too complex. That's just an incentive for having weird compilation errors and very hard to diagnose bugs. And I'm not even mentioning performance. Kotlin compilation is slow enough, I'm not sure that spawning even more compilation daemons just for script compilation is a good idea. But, I live that decision to the Gradle team.

Gradle just runs on the specified JDK. Why not allow something similar for the Kotlin?
It will give some freedom for the consumer to use, play, fix report things that are not compatible. He/she can do something proactive.
However, that would open the box of the Pandore - why not do similar for Groovy (I don't know what other runtime dependencies Gradle needs).

What we want is the ability to specify the Kotlin stdlib version in gradle.properties or in a shebang in settings.gradle[.kts].
That's all we need to unblock use of libraries that follow the Kotlin train from within Gradle plugins that are not using isolated workers.
That wouldn't require to upgrade dependencies on the compiler within Gradle since newer stdlibs are a superset of the older ones.

What we want is the ability to specify the Kotlin stdlib version in gradle.properties or in a shebang in settings.gradle[.kts]

It breaks the principle of isolation: a user shouldn't have to know what version of Kotlin to use to build a project. It may be that a transitive dependency uses a higher version and you wouldn't notice. This isn't, IMHO, a viable solution.

We do that today with java versions but Gradle doesn't impose a specific version. Many plugins use guava and users accept that they may need to address transitive version mismatches. Yet, kotlin must abide by different rules because Gradle uses it internally. I think that's pretty unfortunate.
A good example of this is happening right now with Gradle 7.2 - it updates the embedded kotlin version to 1.5.21 but doesn't offer guidance to plugin authors. Should they continue to set their language/API version to 1.4? Can they raise it only if their min Gradle version is 7.2+? Etc etc.
I'd understand if it was a default but the fact that Gradle actively prevents changing the version is problematic to the community at large because it results in fragmented versioning and frankly most plugin authors don't realize this is even going to be a problem until they're burned by it.

What we want is the ability to specify the Kotlin stdlib version in gradle.properties or in a shebang in settings.gradle[.kts]

It breaks the principle of isolation: a user shouldn't have to know what version of Kotlin to use to build a project. It may be that a transitive dependency uses a higher version and you wouldn't notice. This isn't, IMHO, a viable solution.

I'm going to use these words against that statement itself:
Kotlin libraries and their developers shouldn't have to know what Kotlin version to use just to make sure that they can be used in Gradle plugins.
IMHO, this is a tradeoff, and as a user, I'd prefer to give up that "principle of isolation" when opting-in to the latest Kotlin stdlib version in my Gradle project, so I can use recent libraries and some Gradle plugins with the latest features (and fixes?).

IMHO, this is a tradeoff, and as a user, I'd prefer to give up that "principle of isolation" when opting-in to the latest Kotlin stdlib version in my Gradle project, so I can use recent libraries and some Gradle plugins with the latest features (and fixes?).

I heartfully disagree with this: as a user, I want to apply a plugin and I don't care what language it's written in. Forcing me to put a Kotlin version so that the plugin works is counter-intuitive. As a plugin author, you care, but users shouldn't.

Some users (or many?) write mini-plugins in buildSrc, in kotlin. So they still end up having to care.
Also I guess those of us who are also using gradle to build Kotlin projects, suffer from this issue more, because you have to work with different kotlin versions in one project. If I'm not mistaken most of us here who are in favor of this issue are exactly from this category of users.

Yes, there are two distinct use cases here:

writing plugins for general purpose reuse
writing plugins only for use in the current build

In the latter case it might  make sense to be able to tell Gradle what version of Kotlin to include so that you can write your plugins in the same version as the actual source.
Worth noticing, however, that you'd be quite likely to run into issues with 3rd party plugins you use that were written in Kotlin and depend on features that Kotlin has removed in a later version... Mind you, this presumably becomes an issue when Gradle upgrades its Kotlin version. Non-backwards compatibility is a big deal. The Java stdlib gets that more right than most.

depend on features that Kotlin has removed in a later version

This, again, exists with any dependency or major java release.
It's also a frankly nonsensical argument to use kotlin as an example of that as it guarantees backward compatibility across its minor versions and there's no inkling of a Kotlin 2.0 coming any time soon. Let's keep to realistic examples.

We have some of our own plugins written in Kotlin and as we upgrade Gradle, things tend to break because we enforce a policy of no version conflict in the classpath.
We end up doing this in all Kotlin projects:
// this is necessary to fix a conflict in the Kotlin stdlib dependencies:
// -+- org.jetbrains.kotlin:kotlin-stdlib:1.5.10
//  \-- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.8 (requested org.jetbrains.kotlin:kotlin-stdlib:1.3.71)
configurations {
    kotlinCompilerPluginClasspathMain {
        resolutionStrategy.force "org.jetbrains.kotlin:kotlin-stdlib:$kotlinCompilerVersion"
        resolutionStrategy.force "org.jetbrains.kotlin:kotlin-stdlib-common:$kotlinCompilerVersion"
    }
    kotlinCompilerPluginClasspathTest {
        resolutionStrategy.force "org.jetbrains.kotlin:kotlin-stdlib:$kotlinCompilerVersion"
        resolutionStrategy.force "org.jetbrains.kotlin:kotlin-stdlib-common:$kotlinCompilerVersion"
    }
}

It's ugly but fixes the conflicts.

@renatoathaydes Looks like you do this in a Groovy script. settings.gradle, build.gradle, or an init script?
That gets me thinking that an init script could do the job of forcing a recent Kotlin stdlib to ensure compatibility with libraries. It could even be exported as a plugin (Plugin<Gradle>).

@renatoathaydes I believe you're talking about something unrelated to this issue. The build itself will still be forced to use the version Gradle embeds, there's no current way to force this resolution to another version (or else this issue wouldn't exist!). You can write a Gradle plugin in whatever language you want, that doesn't change the fact that it's forced to the runtime of whatever Kotlin version the consuming project's Gradle version embeds.

@LouisCAD build.gradle for every project using Kotlin.
@ZacSweers I believe it's the same issue because when I add a plugin to the buildScript classpath, it introduces a conflict with what I believe is Gradle's built-in Kotlin (which breaks our build as we use a strict strategy that forces us to resolve versions "manually"  - this may be harder than it needs to be, but it's current policy due to nasty issues we've had before).
Once we get a conflict, there's two options:

look at every plugin's transitive dependencies to see which one is causing it, then exclude the problematic transitive dependency.
force Gradle to use a particular version as I show above.

I actually prefer to do the first option, but for some reason, Gradle does not show me which plugins are actually conflicting, which is why I was forced to use the other option.
I understand that, as you say, at runtime Gradle will use whatever Kotlin it wants, but my problem is when trying to compile my project's build script (as the kotlinCompilerPluginClasspathMain configuration resolution is what fails).
Our build is bloody complex though, so I might be missing something.

is happening right now with Gradle 7.2 - it updates the embedded kotlin version to 1.5.21 but doesn't offer guidance to plugin authors

Seconding this. As a plugin author there is little to no guidance on how to properly setup a Gradle plugin written in Kotlin to offer the higher compatibility to our users.
Would be great if the Gradle team could include the:

Version of the embedded Kotlin
suggested api/language language
In this page for every release: https://docs.gradle.org/7.1.1/userguide/compatibility.html

For what it's worth, this seems to work for me to create an artifact compatible with Gradle's embedded Kotlin version, even in a project using a greater version of the Kotlin plugin:
plugins {
    `embedded-kotlin`
}

dependencies {
    implementation(kotlin("stdlib-jdk8", embeddedKotlinVersion))
}

kotlin.target.compilations.all {
    kotlinOptions {
        apiVersion = embeddedKotlinVersion.split('.').take(2).joinToString(".")
        jvmTarget = JavaVersion.VERSION_8.toString()
    }
}

depend on features that Kotlin has removed in a later version

This, again, exists with any dependency or major java release.

Dependency, yes. But the kotlin stdlib is a rather large dependency.
I don't think I've ever experienced an issue with binary compatibility in the java stdlib in Java 1.3 - 11. Not sure whether it's still as stable now.

It's also a frankly nonsensical argument to use kotlin as an example of that as it guarantees backward compatibility across its minor versions and there's no inkling of a Kotlin 2.0 coming any time soon. Let's keep to realistic examples.

https://kotlinlang.org/docs/compatibility-guide-15.html#gradually-change-the-return-type-of-collection-min-and-max-functions-to-non-nullable suggests that with the Kotlin 1.6 stdlib binaries a binary compiled against Kotlin 1.4 that relies on emptyList().min() returning null will start failing with an exception.

that relies on emptyList().min() returning null will start failing with an exception.

It will be binary compatible change, reintroduced API will just have different JVM name

I don't think I've ever experienced an issue with binary compatibility in the java stdlib

Is it somehow different in Kotlin? All stable Kotlin versions are binary compatible (excluding experimental APIs)

that relies on emptyList().min() returning null will start failing with an exception.

It will be a binary compatible change, reintroduced API will just have different JVM name

Cool, if Kotlin are guaranteeing backwards binary compatibility then I withdraw my comment.
Does anyone have a link to an explicit statement of that guarantee? My googling is letting me down.

@Mahoney
https://kotlinlang.org/docs/components-stability.html#stability-of-subcomponents
https://kotlinlang.org/docs/language-committee-guidelines.html

I have updated gradle to  7.2-rc-3 and issue has gone

@z0mb1ek no, it's not. It probably looks "gone" because the embedded runtime was updated to 1.5.21 and that's what you're using in the project currently. This is just a coincidence and will happen again as soon as the next Kotlin version is out and they are mismatched again.

https://github.com/gradle/gradle/issues/16345
Hi,
This topic has gotten a lot of attention and discussion. We wanted to share our understanding of the problem, some solutions/challenges and what can be done right now.
First, let's make sure we're on the same page. When a project "uses Kotlin", this could mean several different things.
A project may be written in Kotlin.
A project may be written in some other language and uses Kotlin DSL build scripts.
A project may be written in some other language and applies external plugins that are written in Kotlin.
A project may be written in some other language and writes plugins in buildSrc or an included build in Kotlin.
From Zac's original post, my understanding is that the trouble stems from (3) and (4). Namely, you are trying to write a Gradle plugin in Kotlin or use a library written in Kotlin that uses a version of Kotlin different from the version embedded in Gradle. On top of that, the Kotlin runtime target for Gradle can be lower than what you would like to use.
For example, Gradle 7.3 embeds Kotlin 1.5.31 with a Kotlin 1.4 target level.  This makes it difficult to write a plugin that wants to use another Kotlin version or a plugin that wants to set a Kotlin 1.5 target level. It also makes it difficult for that plugin to depend on another library that is written in another Kotlin version or targets a different Kotlin target level.
Is that a good summary of the main problem?
Second, what are some of the ways to fix this?
When we say "Gradle embeds Kotlin", it may be unclear how deep that goes.
Kotlin DSL scripts are written in Kotlin and Gradle uses the Kotlin Gradle Plugin (KGP) and the Kotlin compiler to compile them.
KGP is written in Kotlin.
The integration with KGP is itself written in Kotlin.
Some other parts of Gradle are now written in Kotlin (like the configuration cache).
The dependency on Kotlin is deeper than just the compiler. It's possible to hide some of this from plugins and build scripts, but ultimately, a plugin expects to run with some version of Kotlin available and build scripts are written for a particular version of Kotlin.
Gradle could be made to isolate every plugin and only expose the version of Kotlin required for each plugin (if needed). Setting aside how this would impact performance, an implication of this approach would be that no plugins could expose Kotlin types as part of their ABI.  This is complicated.
For now, we do not want to make the version of the embedded Kotlin changeable; however, there are other changes we should make to alleviate some of the pain when developing with Kotlin. From the other comments in this issue, we think there are other related problems that could be fixed without changing the embedded Kotlin version.
We should investigate problems around
Cross-compilation in Kotlin: we should provide samples that show how you can use versions of Kotlin that are different from the embedded version for the scenarios described above.
Multi-version Gradle plugin development in Kotlin: we should provide samples that show how you can develop a plugin that targets multiple versions of Gradle (that have different versions of Kotlin). We need to make it clear which version of Kotlin and target level you need to use.
Composite builds with Kotlin: we should provide samples that show how to use buildSrc/composite builds with Kotlin.
We have good communication with JetBrains, so if there are changes we can push on the KGP side to make this better, we will.

Perfect summary.
Speaking only about plugins written in kotlin or using dependencies written in kotlin...
Let me ask a naive question, why can't I use a newer version of kotlin in my plugins? Kotlin promises to be backwards compatible in minor releases (the exceptions are incubating/experimental features, but that should be fine), so I expect, for example, to be able to use kotlin 1.6 in my Gradle plugins with Gradle 7.2.

@CristianGM

Let me ask a naive question, why can't I use a newer version of kotlin in my plugins? Kotlin promises to be backwards compatible in minor releases (the exceptions are incubating/experimental features, but that should be fine), so I expect, for example, to be able to use kotlin 1.6 in my Gradle plugins with Gradle 7.2.

There are three parts to compatibility. The "target level" I mentioned above, the Kotlin standard library and what version Gradle provides at runtime. We stick with a target level for an entire major release (1.4 for Gradle 7.x), update the version of the standard library in minor releases and expose a single version of Kotlin that's used when executing build scripts. This is the same kind of strategy we use for Groovy, too.
I believe the Kotlin team tries to make it so Kotlin 1.X can target down to 1.(X-2). So you could compile against Kotlin 1.6, target Kotlin 1.4 and run on Gradle, sometimes. If you had a dependency that needed a higher Kotlin target level to work, you may not be able to run on Gradle. What would break would depend on how the embedded Kotlin version would handle the mismatch.
The other part is the standard library itself. Depending on which versions of Gradle you want to support, you need to constrain your plugin to the API of the lowest version. If you compile against Kotlin 1.6, target Kotlin 1.4 and are careful to only use Kotlin 1.4 APIs, I think this could work. But it's probably easier to compile against Kotlin 1.4.  You could use newer APIs, but you would have to be careful to only use the newer methods in versions of Gradle that had them.
The last part is the crux of what OP is asking to change. Gradle embeds a particular version of Kotlin and uses that version for the plugins/build logic. If you built your plugin with Kotlin 1.6 and Gradle had a way to swap that version in, it would break other plugins that were written for a different version of Kotlin or Gradle itself (since part of Gradle is written in Kotlin).

I think what confuses me is:

If you built your plugin with Kotlin 1.6 and Gradle had a way to swap that version in, it would break other plugins that were written for a different version of Kotlin or Gradle itself (since part of Gradle is written in Kotlin).

Why? If your code worked with 1.4/1.5 it should work fine with 1.6
In any case, I should try to understand the issue better, for sure is more complex than I think.

I think what confuses me is:

If you built your plugin with Kotlin 1.6 and Gradle had a way to swap that version in, it would break other plugins that were written for a different version of Kotlin or Gradle itself (since part of Gradle is written in Kotlin).

Why? If your code worked with 1.4/1.5 it should work fine with 1.6
In any case, I should try to understand the issue better, for sure is more complex than I think.

By the time we know that there may be another Kotlin version, we've already started using Kotlin. We would need to isolate the "already running" Kotlin from everything else and this would also need to apply to plugins so that the different versions of Kotlin are not mixed in Gradle's runtime or into other plugins.

If we can't specify the kotlin version used in plugins/dsl, next best thing would be to make sure every new gradle release (every 6 to 9 weeks) is updated to use the latest version of kotlin, that will at least reduce the pain a bit.
Seems Gradle has been doing a good job of keeping up with kotlin releases recently, would love to see kotlin 1.6 bundled with Gradle in the next Gradle version following the 1.6 kotlin release.

Gradle could be made to isolate every plugin and only expose the version of Kotlin required for each plugin (if needed).

@big-guy coud we have something like this:

Plugin authors have an easy way to shadow the Kotlin stdlib
Plugin authors have an easy way to distribute 2 variants of their plugin: one with the "embedded" stdlib and the other packaged as "external"
When running a build, Gradle selects the variant based on the Plugin variant metadata and current Gradle embedded Kotlin version

This way, plugin authors that want to use newer versions could ship 2 versions of their plugin jar and depending on the version of Gradle that's running it, the appropriate variant will be selected. There could even be a small message saying that "plugin $xyz is using a newer version of Kotlin that is not supported yet, expect more memory usage"
This is somewhat related to Multi-version Gradle plugin development except that:

setup will be easier. Plugin authors wouldn't have to think about which version of Gradle uses what version of Kotlin as Gradle would do the selection automagically.
The plugin codebase can be 100% using the newest Kotlin version.

Plugins with a relocated stdlib wouldn't be able to use Kotlin types in their ABI but I think most of these types (collections mainly) are aliased to the equivalent Java types so that could be transparent to relocation? Two Kotlin types that could be problematic are kotlin.Unit and kotlin.Metadata but maybe Jetbrains can make it so that they are binary compatible forever and do not require relocation?
Thoughts?

I think Shadow already works fine for building Gradle plugins: https://imperceptiblethoughts.com/shadow/plugins/#special-handling-of-the-java-gradle-plugin-development-plugin
Last time I tried, I think I had some issues with Kotlin reflection but everything else just worked out of the box.

@ephemient shadow has issues where it renames "kotlin" Strings which is problematic as it's also the name of the kotlin extension so any plugin doing something like extensions.named("kotlin") will most likely fail (see https://github.com/johnrengelman/shadow/issues/232). Maybe there's a way to fine tune this. This is where an "easy" setup would be nice.
Looking ahead and beyond relocation itself, it's also about easy publishing and most importantly, Gradle selecting the appropriate plugin variant at runtime based on its current Kotlin version and plugin metadata.

Well. I didn't try applying or working with the Kotlin plugin from within a custom plugin, so I didn't run into that. Does sound like a serious blocker though.
IMO having an unshadowed version for newer Gradle isn't that useful, since (with this approach) you can't use Kotlin types in public API regardless. Do you have a reason I'm missing?
There's a number of other Kotlin types that can leak into public API if you're not careful, e.g. kotlin.jvm.internal.Function*/DefaultConstructorMarker, but they probably don't come up much in Gradle plugins.

you can't use Kotlin types in public API regardless.

This is the part I would like to have a better understanding of. Looks like a bunch of kotlin types in the APIs could still work (Lists, Sets, ...) with the exceptions that you mentioned (Function*, DefaultConstructorMarker, ...). Maybe that can be enforced with some tooling.
Overall, I'd very much better like:

Write your plugin using the Kotlin version of your choice with some (TBD) limitations on your plugin public API

Compared to:

Not being able to use okio in a plugin because it transitively pulls a Kotlin stdlib version not supported by Gradle

My gut feeling is that the very vast majority of plugins APIs would relocate very well and that the few edges cases could be acceptable or easily workaroundable. I'll try on a test plugin see how that goes.

The community affected by this can easily lower the language/api version, the problem is that we shouldn't need to just because Gradle wants to use Kotlin internally.

I think the problem is how we define 'internally'. My understanding is that there are at least two parts to this:
Compile-time
Gradle needs to compile the user build.gradle.kts files. This is the 100% internal part. The Kotlin compiler version for this can lag one version behind the latest one most of the times. According to the evolution rules for binary formats, Gradle lagging one version behind the latest Kotlin version should mostly work:
Preferably (but we can't guarantee it), the binary format is mostly forwards compatible with the next feature
release, but not later ones (in the cases when new features are not used, e.g. 1.3 can understand most
binaries from 1.4, but not 1.5).

This means that if you want to use T & Any available in preview in 1.6.1-M1 in your plugin for an example, this will mostly fail at compile time because the current 1.5 compiler will not now what to do with this. Note that this only affects the public API of a plugin. If you want to use T & Any as an implementation detail, I would expect this to work. As I said above, I think it's ok to have such limitations in the plugin public API. It's a much lighter restriction than having to write the whole plugin in an old version of Kotlin.
Run-time
Once Gradle has compiled all the build.gradle.kts scripts to some .jar files, it needs to run them. Because these .jar files use Kotlin, they need one version of kotlin-stdlib.jar in the classpath.
This is where the chicken and egg problem kicks in. These .jar files also contain some code that resolves dependencies and might end up needing a more recent version of kotlin-stdlib but in order to run the dependency resolution code, they require kotlin-stdlib. As @big-guy mentioned in this comment:
By the time we know that there may be another Kotlin version, we've already started using Kotlin. We would need
to isolate the "already running" Kotlin from everything else and this would also need to apply to plugins so
that the different versions of Kotlin are not mixed in Gradle's runtime or into other plugins.

I'd say this usage isn't really 100% internal anymore because this Kotlin version is the Kotlin version that plugins will see during configuration so plugins are indeed affected by the Kotlin version used by Gradle.
All in all, it's not one but 2 versions of Kotlin that Gradle uses, the compile-time and run-time. The compile-time is mostly internal. The run-time is the problematic one IIUC. It just happens that they are the same today but this doesn't have to be always the case. Gradle could make an update just after every Kotlin release to bump the run-time version. That would be a path forward but would still put a fixed (albeit more recent) runtime version in the classpath.

By the time we know that there may be another Kotlin version

Wouldn't this be solvable via some configuration? Gradle or system property, for example. It might be a chicken and egg problem at runtime but we still control the hen house too

Hi folks, I am stepping back from this issue. Gradle appears intent on continuing to hamstring the use of Kotlin in plugins to protect their own internal use of it (configuration caching, kotlin-dsl, etc). I am not sure why the above proposals are not actionable, and at this point I am beyond frustrated with the backchannel/subtweet messages from Gradle folks that just amount to a vague "you just don't understand". You're right, I don't understand. I'd like to, but it's frustrating to be swatted away as a nuisance for asking, especially as a Gradle employee asked me to write this issue up in the first place.

@ZacSweers I can understand, in my experience, Gradle employees don't drive, they ride on high horses, and are immune to logical reasoning from the community. Case in point: https://github.com/gradle/gradle/issues/18006.

We would like to remind everyone that the Code of Conduct applies here, so please write comments according to it. Part of the answer from @asarkar is unacceptable and in clear violation of that code. There is also definitely no “intent to hamstring the use of Kotlin” by the Gradle team.
This is a complicated issue. We listed several related problems and potential solutions in this comment. We would like to address those problems, but we don’t think we can do exactly what was asked in the original description at this time.

Wouldn't this be solvable via some configuration? Gradle or system property, for example. It might be a chicken and egg problem at runtime but we still control the hen house too

Is there any chance something like this will ever be implemented?
I run into this problem constantly, and my internal Gradle plugins are a mess of managing which versions of dependencies they can use, and which have to wait for the next version of Gradle to get updated (unless the library has moved on to a newer version of Kotlin, in which case I now have to go find the newest version that would work with it).

Can this be reopened? Or is there any other issue where this is being tracked?
Is there any place (maybe milestones) to check estimated dates or which future Gradle version will have an specific Kotlin version?
Now that Kotlin 1.7.0 and this issue is hard to be resolved, at least knowing in which version of Gradle or date we will be able to use an specific version of Kotlin should be great.

@ephemient that is great, hope it adds future versions of Gradle and not only the released ones.


[Wait for #2796][Graph] add inplace direction setting through layer property

added inplace direction setting for binary tensor operation layer.
Self evaluation:

Build test:   [X]Passed [ ]Failed [ ]Skipped
Run test:     [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Seungbaek Hong<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2797. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.


Support for Apple silicon Mac?

Hi, thank you for your nice work!
I wonder if this codebase is currently suitable for Mac with Apple silicon? For example, following the installation step, platform.machine() return x86_64 in my case, probably because my python is compiled using Rosetta 2. However, I think my mac needs the Ubuntu with arm64?
https://github.com/xlang-ai/OSWorld/blob/e2d6842535114a5b428bd712728a2af84872c16a/desktop_env/envs/__init__.py#L192
Meanwhile, when I downloaded the arm64 version of ubuntu, it seems that there was also an inconsistency between its corresponding vmx naming and the current code, which causes a bug.
https://github.com/xlang-ai/OSWorld/blob/e2d6842535114a5b428bd712728a2af84872c16a/desktop_env/envs/__init__.py#L246
Does the current codebase support apple silicon, or am I configuring something wrong?

Hi, thanks for your question, actually OSWorld supports Apple silicon across different OS platforms.  I think this bug is due to the reason we didn't consider platform.machine() return x86_64 in Apple silicon, we will add a fix on that. As for your second question, it may due to the fact that the .zip file of arm-version of the OSWorld image is formatted in the mismatched name, I will change that and re-upload it. Sorry for that.


Improve Configuring Active Directory User Stores for SCIM 2.0 document

Is your suggestion related to a missing or misleading document? Please describe.
The doc [1] needs to be updated with information in [2].
[1] https://is.docs.wso2.com/en/latest/learn/configuring-active-directory-user-stores-for-scim-2.0-based-inbound-provisioning/
[2] https://kasunir.medium.com/configure-your-active-directory-as-a-secondary-user-store-in-wso2is-5-10-0-b9e677e04001

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Let's not run GH Unit Tests on doc-only PRs

What is the change?
If a PR only touches the doc/ directory, we don't need to run unit tests.
Why is the change being made?
I am just trying to waste-not-want-not here. Let's make some of our PRs faster.
The documentation for the GH Actions feature I'm hoping to use is here.

Checklist

[X] This PR has only one purpose or idea.
[X] Tests have been added/updated to verify that the new/changed code works.

[X] The code style follows good practices.
[X] The commit message(s) follow good practices.

[X] The release notes (location doc/release/0.X.rst) are up-to-date with any important changes.
[X] The documentation is still up-to-date in the doc folder.
[X] No requirements were altered.
[X] The dependencies are still up-to-date in pyproject.toml.

I forgot to say, I tested this PR and it worked fine:

/doc/ only commit didn't run tests: https://github.com/terrapower/armi/commit/3a423f9fcc59801f7f4c3041e77d915ea4b4cdd9
/armi/ "real" changes DID run tests: https://github.com/terrapower/armi/commit/6b3be6cf783be56432edb03aedf42aef4f55cbe1


[OLS ] release catalog changes for 0.1.4

Description
Release OLS 0.1.4 and updated catalog with  0.1.4 images.
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue #
Closes #

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

/lgtm
/approve


Video not responsive in Video Viewer Activity

Video height exceeds background height in iPhone 6/7/8 landscape mode due to which overflow occurs in the y-direction.
Preview:

Yes, you're right.

Fixed in https://github.com/llaske/sugarizer/pull/934


Set auth cookie only if it does not exsist on the Request

#55

https://github.com/nelmio/NelmioSecurityBundle/issues/55

Makes sense, thanks!


Grammar tweaks for initialize request

@brettcannon,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Thanks @brettcannon. Parts of the changes got already fixed. I merged the rest by hand.


dotnet crash on macOS

It seems like the latest Ionide version 3.34.0 causes dotnet to crash unexpectedly on macOS.
Steps to reproduce:

Open .NET Core F# project in VS Code with Ionide
A few seconds later I'll get a popup saying that dotnet crashed
F# Intellisense stops working, etc.

When I revert back to version 3.33.0 then everything works fine again.

Yeah, @baronfel has been looking into it. Current known workaround is using net runtime in FSAC settings

All I've been able to determine so far is that file parsing and checking in the .net core FSAC triggers a stack overflow. I haven't been able to nail down a cause or a minimal repro case yet.

FCS updates didn't solve the problem :(

Updating to latest preview sdks (2.1.600-preview-009426, or 3.0.100-preview-010184) did not fix the problem. I was wondering if the dotnet version used to invoke the FSAC process might reveal something interesting, but I get similar behavior regardless of the sdk version.

There's an issue that looks very similar in the mono repository: https://github.com/mono/mono/issues/11933
In our case at work when I look at the dumps I see tons of lexer stacks, so my guess is that the lexer isn't tail recursive or something in a particular spot.

I’m pretty sure I’ve seen this bug in the past on the different mono versions (especially back in the 4.X days), however I’m really surprised it’s also happening on Core.

@dustinmoris or anyone, i dont have a mac atm (my old air is dead 😄)
can someone help me troubleshoot this? maybe we can screenshare.

https://github.com/Microsoft/visualfsharp/issues/6346 this is the same issue, @enricosada, and according to dsyme it's a mono/coreclr/f# lexer tail recursion problem.

It seems like the latest Ionide version 3.34.0 causes dotnet to crash unexpectedly on macOS.
Steps to reproduce:

Open .NET Core F# project in VS Code with Ionide
A few seconds later I'll get a popup saying that dotnet crashed
F# Intellisense stops working, etc.

When I revert back to version 3.33.0 then everything works fine again.

seems severe, not just a corner case with a really really big file.
Maybe is related, or not. can you add some info @dustinmoris or anyone here with same problem? happy to discuss in chat or continue in FSAC repo.
My questions:

or newer FCS contained in ionide v3.34.0 has a bug, and should be reverted in FSAC to the version shpped in v3.33 until works

that mean we can bisect the error in FCS, if is a regression in the lexer

or v3.34.0 should work as is in previous version of mono/.netcoresdk, if is a regression of the runtime

@enricosada It's certainly a severe issue, because it instantly crashes Ionide and makes it entirely unusable in the latest version. As said before when I revert back it works all fine. I've been working with v3.33 for the last few weeks now, but it's annoying because VSCode obviously shows me all the time the notifications to update to latest Ionide.
If you are free tomorrow then I'd happily screen share with you and do whatever helps!

got a minimal repro in https://github.com/fsharp/FsAutoComplete/issues/356#issuecomment-486266694

Can this still be repro'd? I believe both @baronfel and @MangelMaxime reported it's working fine in Ionide 4.0

@Krzysztof-Cieslak Indeed for me this seems fixed on my project

I’m still experiencing the dotnet crash with dotnet 3 and ionide 4.1.0. It’s been broken for some time (probably around March) and I’ve been running net FSAC since. Happy to help narrow down a repro case if someone can walk me through the process.

I spent a few hours trying to narrow this down - I believe it’s connected to large/complex CEs - in my case Chiron serializers/deserializers and async CEs. Somewhere around 10 do! or let! calls seems to trigger a stack overflow.
Weirdly, it also happens even if the CE in question is commented out, I’m guessing its done on an initial file parse?
Unsure where the best place to raise this issue is - does it belong in dotnet/fsharp or fsharp/FsAutoComplete?

If that’s lexer issue (just as previous problems) then I’m not surprised it’s crashing also on commented out code.
Is there any stack trace or anything in F# output panel?

Nothing very useful:
[18:18:48 INF] [Compiler] Inside Compiler Function 4
Process is terminating due to StackOverflowException.
[Info  - 6:18:49 PM] Connection to server got closed. Server will restart.
[Error - 6:18:49 PM] Request textDocument/codeAction failed.
Error: Connection got disposed.
	at Object.dispose (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:9455:25)
	at Object.dispose (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:6090:35)
	at LanguageClient.handleConnectionClosed (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:8325:42)
	at LanguageClient.handleConnectionClosed (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:29390:15)
	at closeHandler (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:8312:18)
	at CallbackList.invoke (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:17154:39)
	at Emitter.fire (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:17213:36)
	at closeHandler (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:8791:26)
	at CallbackList.invoke (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:17154:39)
	at Emitter.fire (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:17213:36)
	at StreamMessageReader.fireClose (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:18154:27)
	at Socket.<anonymous> (/Users/sam/.vscode/extensions/ionide.ionide-fsharp-4.9.0/fsharp.js:18194:46)
	at Socket.emit (events.js:208:15)
	at Pipe.<anonymous> (net.js:588:12)
[Error - 6:18:49 PM] Request textDocument/codeAction failed.

As with every update - I hope FCS updates solve this issue.

@Krzysztof-Cieslak I can confirm SO on large CEs appears to have been fixed with the latest Ionide update, thanks.


Installation instructions not working: ImportError: serverextension

Hi guys!
I tried to install jupyterlab both from pip and the current master from yesterday following the
instructions in the readme.md.
Basically it failed for me at the last step where
jupyter serverextension enable --py jupyterlab
would return
ImportError: serverextension
Any ideas?
I am running macOSX 10.10.2 and python 2.7
Do I need to be on python 3 or update to the latest macOS ?
Thanks!

It looks like you have a version of jupyter notebook older than 4.2, can you please try pip install -U notebook?

Ah thanks! I had a problem that notebook wasn't updating due to my virtualenv getting confused with $PYTHONPATH. Thanks! jupyterlab looks great!

Awesome, thanks for the feedback!


Sub-project idea. highj-typeable

One thing that gets annoying quickly is having the same or very similar code for deriving Equal, Ord and Hash instances of your data structures. To resolve this I was thinking of a special data structure called Typeable that provides the shape of the data structure your modeling as a GADT.
This Typeable can then be used to auto-magically derive Ord, Equal, Hash and possibly more from a given Covariant functor or an Invariant functor. (automatic Serialization comes to mind too)
This might be nice to have in highj-typeable.
It is then possible to auto-derive the Typeable using Java's reflection if the user wishes. Otherwise the user can define Typeable by hand just incase their data type is not so plain.

For Serialization as an example use-case. Someone could write a pair of say SourceEff and SinkEff containing a collection of effects for reading/writing to/from an unknown data source. Then bundle those two into something like LoadSave, then this LoadSave becomes an instance of an Invariant functor.
Information Typeable can then be used to automatically derive a LoadSave. Which gives the end-user the ability to read/write any Typeable into any language independent data format like XML and JSON.
I've used such a pattern before and it does lower the amount of repetitive code someone would have to write.
One problem comes with multiple versions of the file format. So Typeable would have to have some mechanism to branch by version number (different invmaps on each version), and possibly tag data/sub-data with default values. A LoadSave can store a warning if a field is not present and take on the default value, so the program can just continue with a warning, and the user of the program does the appropriate edit to mend the field, then the file can heal itself on save.

One more use-case for Typeable that comes to mind is the automatic generation of user-interface forms. Mainly just as a default form to use until someone has the time spare to do a hand written UI.
Its just a no-frills user-form with no input validation or anything. For just when you need to prototype something quickly. I'm not saying we should do this as a sub-project, its just another use-case for Typeable.

Sounds interesting. Could be also useful when we want to have a Show type-class


Create event-streaming.md

Signed-off-by: Vasu Singh<EMAIL_ADDRESS>Describe your changes
Localize https://glossary.cncf.io/event-streaming/
Related issue https://github.com/cncf/glossary/issues/1411
Checklist before opening this PR (put x in the checkboxes)

[X] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[X] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

Hello @vasucp1207! Thanks for your contribution!

@edsoncelio, done all the changes take a look now.

/LGTM


FIX: Hierarchy was incorrectly unexpanding nodes that were previously expanded

Given the following structure with a TreeMultiselectField:

If this is saved and then you re-open the tree, you'll see the following bug:

Opening the tree marks the field as dirty, and if you save the object it will remove the relations that are no longer ticked (Grandchild 1 and Grandchild 2 in this case).
This is due to a problem in Hierarchy when it recurses through nodes to find ones that need to be marked for inclusion when generating the tree structure. It correctly marks Child 1 to be expanded, but then when it marks Child 2, it marks Child 1 as not expanded but having children (this also explains the less serious bug with the arrow pointing down when it shouldn't).
XHProf comparison before and after shows it doesn't appear to recurse any further than it did previously.
Changes to other tests aren't because of this bug, but just because I had to add more children/grandchildren to confirm the bug is fixed.

Please fix the tests. :D
The regression is caused by an assumption about intrinsic ordering of items in the hierarchy; If you don't explicitly order them, it's non-deterministic. Postgres is simply pointing out the error in our assumption. ;)
Actual code seems fine.

Adding default_sort on Title should be enough.

Oops, missed your comments sorry @tractorcow, have added this now. Will see what postgres does!

Absolute @madmatt :D

Notice an error when testing permissions tab; Can you please check locally to see if this error is real?

bump

The failing Behat test seems to work fine locally but I'm not running it the same way Travis is. Have just force-pushed a rebase up so will see if that fixes the issue or not

@tractorcow, @dhensby: Yay, it works now! Now I'm not sure if the 3 branch was the right place to merge into, but it should all be up to date, I've rebased on top of 3 today. Let me know if it's wrong :)

looks like a bug fix, so could go into 3.2 if you're so inclined?

I don't know what the usual practice is TBH, happy to be guided. I'm happy for it to just go out with the next 3.x release, whenever that is - for now I'll just have to maintain a fork but it's not a massive bug, and is easy to keep rebasing as needed.

@madmatt I'd like it in 3.2 and we're hoping to do a release pretty soon, so if you PR against 3.2 it could make the next set of stables.
I'm sure you know how to do this but please can you:

Close PR
New branch from 3
Cherry pick this commit (6168ced)
Open a new PR against 3 (mentioning this commit)

@dhensby This is already a PR against the 3 branch actually - did I do it wrong?

@madmatt derp - I meant 3.2

Closed in favour of opening #5823


Updated docs for reuse_port、dso directive changes.

core.md file append below:
Note:
Removed reuse_port directive after the Tengine-2.3.0 version and use the official reuseport of Nginx, detailed reference document.
dso.md file append below:
Note
Removed dso_tool tool and dso directive after the Tengine-2.3.0 version, If before using Tengine dso feature, you could switch to the Nginx official load_module directive, detailed reference document1 、document2.

ACK


Add comments - docstrings format

Little to know in-code comments explaining functions

@Yogiski, below is the Sphinx docstring format. Can you review your comment in this format, please? Let me know how can I help.
https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html
The Sphinx docstring format
In general, a typical Sphinx docstring has the following format:

"""[Summary]

:param [ParamName]: [ParamDescription], defaults to [DefaultParamVal]
:type [ParamName]: [ParamType](, optional)
...
:raises [ErrorType]: [ErrorDescription]
...
:return: [ReturnDescription]
:rtype: [ReturnType]
"""

For some reason, I can not capture docstring contents in ReadTheDocs although it works fine locally (local HTML file). This is what I can see in log file from ReadTheDocs's run:
WARNING: autodoc: failed to import module 'candi' from module 'CanDI'; the following exception was raised:
Traceback (most recent call last):
  File "CanDI/data.py", line 68, in _init_index_tables
    assert os.path.exists(new_path)
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/candi/conda/docs/lib/python3.8/site-packages/sphinx/ext/autodoc/importer.py", line 66, in import_module
    return importlib.import_module(modname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/candi/conda/docs/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/docs/checkouts/readthedocs.org/user_builds/candi/checkouts/docs/CanDI/__init__.py", line 2, in <module>
    data = data.Data() #Global object data instantiated on import required for access by GeneQuery Objects
  File "/home/docs/checkouts/readthedocs.org/user_builds/candi/checkouts/docs/CanDI/data.py", line 29, in __init__
    self._init_index_tables()
  File "/home/docs/checkouts/readthedocs.org/user_builds/candi/checkouts/docs/CanDI/data.py", line 71, in _init_index_tables
    raise(RuntimeError, "You are missing essential index table: {}. exiting...".format(new_path))
TypeError: exceptions must derive from BaseException

I think this line fails

An AssertionError can only occur when an assert statement fails.
Source link

I'm not sure what we can do to deal with this!

@Yogiski this is also well done!


[MacOS] Unable to load assets with cyrillic characters in path

Unable to load assets with "ё" or "й" characters in path on iOS. The same assets are successfully loaded on Android.
Unit-test passed on Windows, but failed on MacOS, All other cyrillc characters can be used without error.
Steps to Reproduce

Create new flutter project
Create text asset 'assets/data/ё.txt' with 'ok' text in it
include asset in pubspec.yml

  assets:
    - assets/data/

Create unit-test test/asset_test.dart

import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  TestWidgetsFlutterBinding.ensureInitialized();
    test('Check ё', () async {
      expect(await rootBundle.loadString('assets/data/ё.txt'), 'ok');
    });
  });
}

Execute flutter test on MacOS

Expected results:
Test passed
Actual results:
Test failed with error: unable to load asset
MacOS: BigSur
XCode: 12.4
Flutter: 2.5.3 stable

Code sample
Sample project to reproduce with app on iOS or with unit-test on MacOS:
assets_ru.zip

Logs
[+11135 ms] [VERBOSE-2:ui_dart_state.cc(209)] Unhandled Exception: Unable to
load asset: assets/data/2. ё.txt
[   +2 ms] #0      PlatformAssetBundle.load
(package:flutter/src/services/asset_bundle.dart:224:7)
[        ] <asynchronous suspension>
[        ] #1      AssetBundle.loadString
(package:flutter/src/services/asset_bundle.dart:68:27)
[        ] <asynchronous suspension>
[        ] #2      MyHomePage.build.<anonymous closure>.<anonymous closure>
(package:assets_ru/main.dart:43:23)
[        ] <asynchronous suspension>

Hi @andrew-tpz, thanks for filing the issue.
Please provide the complete output of flutter doctor -v.
Thank you

[✓] Flutter (Channel stable, 2.5.3, on macOS 11.4 20F71 darwin-x64, locale
    ru-RU)
    • Flutter version 2.5.3 at /Users/Shared/Flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (6 weeks ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/coder/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    • CocoaPods version 1.10.1

[✗] Chrome - develop for the web (Cannot find Chrome executable at
    /Applications/Google Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] Connected device (1 available)

! Doctor found issues in 2 categories.

P.S. This Mac is used for iOS builds only.

Hi @andrew-tpz, Thanks for filing the issue. I am able to reproduce the issue, With the Cyrillic  character the asset PlatformAssetBundle.load fails to load the asset. But this works on Windows looks like this is specific to macOS platform.

Windows logs
PS C:\flutter_projects\counter> flutter test
00:01 +1: All tests passed!
PS C:\flutter_projects\counter>

macOS logs
mahesh@Maheshs-MacBook-Air-M1 triage_issues % flutter test
Running "flutter pub get" in triage_issues...                    1,713ms
00:08 +0 -1: Check ё [E]
  Unable to load asset: assets/data/ё.txt
  package:flutter/src/services/asset_bundle.dart 224:7  PlatformAssetBundle.load

00:08 +0 -1: Some tests failed.

flutter doctor -v (MacOS)
[✓] Flutter (Channel stable, 2.5.3, on macOS 12.0.1 21A559 darwin-arm, locale en-GB)
    • Flutter version 2.5.3 at /Users/mahesh/Documents/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (6 weeks ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 13.1, Build version 13A1030d
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 60.1.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.61.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.28.0

[✓] Connected device (3 available)
    • iPhone 12 mini (mobile) • 1FBDAEA9-D062-4831-AFEC-820D955D6D88 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-5
      (simulator)
    • macOS (desktop)         • macos                                • darwin-arm64   • macOS 12.0.1 21A559 darwin-arm
    • Chrome (web)            • chrome                               • web-javascript • Google Chrome 96.0.4664.55

• No issues found!

[✓] Flutter (Channel master, 2.6.0-12.0.pre.789, on macOS 12.0.1 21A559 darwin-arm, locale en-GB)
    • Flutter version 2.6.0-12.0.pre.789 at /Users/mahesh/Documents/flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6ae721de3d (5 days ago), 2021-11-21 15:22:37 -0500
    • Engine revision ad00d12833
    • Dart version 2.16.0 (build 2.16.0-35.0.dev)
    • DevTools version 2.8.0

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 60.1.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.61.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.28.0

[✓] Connected device (3 available)
    • iPhone 12 mini (mobile) • 1FBDAEA9-D062-4831-AFEC-820D955D6D88 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-5
      (simulator)
    • macOS (desktop)         • macos                                • darwin-arm64   • macOS 12.0.1 21A559 darwin-arm
    • Chrome (web)            • chrome                               • web-javascript • Google Chrome 96.0.4664.55

• No issues found!

flutter doctor -v (Windows)
PS C:\flutter_projects\counter> flutter doctor -v
[√] Flutter (Channel stable, 2.5.3, on Microsoft Windows [Version 10.0.19042.1348], locale en-US)
    • Flutter version 2.5.3 at C:\flutter_sdk\stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (6 weeks ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[√] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at C:\Users\mahesh\AppData\Local\Android\sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.4)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.11.31727.386
    • Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2020.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[√] VS Code (version 1.61.2)
    • VS Code at C:\Users\mahesh\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.28.0

[√] Connected device (4 available)
    • sdk gphone x86 (mobile) • emulator-5554 • android-x86    • Android 11 (API 30) (emulator)
    • Windows (desktop)       • windows       • windows-x64    • Microsoft Windows [Version
      10.0.19042.1348]
    • Chrome (web)            • chrome        • web-javascript • Google Chrome 96.0.4664.45
    • Edge (web)              • edge          • web-javascript • Microsoft Edge 95.0.1020.44

• No issues found!

Hi, @maheshmnj, The problem is specific to iOS and MacOS, I attached demo project to reproduce (in first message).


set CLI version to branch name if not release

Set the anoma --version to a branch name and the head commit hash when not building from a release tagged branch. The tagged release branch can continue to use crate_version!() clap macro

closed by #733


Don't call build.New for index SBOM

We need to call a few methods that happen to hang off of build.Context, but they don't actually depend on the dirfs that build.New generates, which is expensive if we've written a large number of files to disk.
This skips that entirely and uses build.NewOptions to get the data we need to generate the index SBOM.

Before:
apko publish --arch amd64 --keyring-append  --repository-append     8.38s user 1.58s system 200% cpu 4.964 total```

After:
apko publish --arch amd64 --keyring-append  --repository-append     8.28s user 1.55s system 203% cpu 4.832 total


[DO NOT MERGE] Backport some fixes from master to ozone-1.4

What changes were proposed in this pull request?
Backport the following fixes from master to ozone-1.4 for 1.4.1 (Do not include Etag-related PRs, these will be backported in separate PRs):
HDDS-10680. Duplicate delete key blocks sent to SCM (#6513)
HDDS-10692. ozone s3 getsecret prints some internal details (#6531)
HDDS-10630. Add missing parent directories deleted between initiate and complete MPU (#6496)
HDDS-10677. Bump plotly.js to 2.25.2 (#5915)
HDDS-10676. Bump browserify-sign to 4.2.3 (#6509)
HDDS-10674. Bump ip to 1.1.9 (#6243)
HDDS-10673. Bump es5-ext to 0.10.64 (#6284)
HDDS-10672. Bump express to 4.19.2 (#6442)
HDDS-10669. Bump axios to 0.28.0 (#6242)
HDDS-9200. [Snapshot] Added logs and metrics for snapshot purge and set property APIs (#6453)
HDDS-10590. [Snapshot] Synchronized snapshot purge, set snapshot property and SstFilteringService (#6456)
HDDS-9426. Calculate Exclusive size for deep cleaned snapshot's deleted directories (#6099)
HDDS-10477. Make Rocksdb tools native lib compatible with all chipset with the same arch (#6341)
HDDS-10552. Downgrade Surefire to 3.0.0-M4 (#6406)
HDDS-9802. Tool to fix corrupted snapshot chain (#6386)
HDDS-10496. Fetch dependencies for cache using actual build (#6359)
What is the link to the Apache JIRA
How was this patch tested?
https://github.com/xichen01/ozone/actions/runs/8732721413

Thanks @xichen01 for working on the backport.  There is already a PR for EC-related fixes: #6551.  Let's rebase this one when that's merged.

https://issues.apache.org/jira/browse/HDDS-10681
https://issues.apache.org/jira/browse/HDDS-10682
https://issues.apache.org/jira/browse/HDDS-10704
Remove above PRs which have been merged in EC-related fixes: https://github.com/apache/ozone/pull/6551

Thanks @xichen01 for working on the backport. There is already a PR for EC-related fixes: #6551. Let's rebase this one when that's merged.

Done

Thanks @xichen01 for updating the PR, my part LGTM.  (There is some minor difference in HDDS-10552 due to HDDS-10013 not being backported, but I think that's OK.)
Please remember to push from CLI instead of merging in GitHub UI.  Also, please do not cherry-pick each commit to ozone-1.4 from your own branch.  We should preserve the commit hashes from this PR:
git checkout ozone-1.4
git pull
git merge --ff-only ozone-1.4-backport
git push

Let's merge #6557 first, which is already based on top of ozone-1.4 while this PR is not.  Rebase this one only afterwards.

Thank you @xichen01 for the backport.
Could you also help to backport the recent fix HDDS-10717?

Done.

Thanks @xichen01 . My part looks good.

Thanks @xichen01 for completing this backport.


Add Kubernetes events input

Description of Changes
Adds a Kubernetes Events input operator.
Notes:

Uses the Watch API, so no need for polling. It's all handled by the client library.
Uses the fake library distributed with the client library to test the portions I can
We currently require that this is run inside a Kubernetes cluster, just like with the metadata decorator. Unlike the metadata decorator, this operator could be useful outside of a k8s cluster, so it may be worth it to add client configuration in the future.
Because we require running in a cluster, it makes it very difficult to test the Start() method. For now, testing that method is omitted.

Please check that the PR fulfills these requirements

[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)
[x] Add a changelog entry (for non-trivial bug fixes / features)
[x] CI passes

Log Files
Logs / Second
CPU Avg (%)
CPU Avg Δ (%)
Memory Avg (MB)
Memory Avg Δ (MB)

1
1000
3.0173354
-0.775908
25.406385
-1.6981411

1
5000
8.241651
+0.13800526
41.768456
+1.7310104

1
10000
13.258871
-1.1207333
51.766163
-4.4341316

1
50000
57.242527
-0.34573364
188.55186
+10.09375

1
100000
109.20885
-8.0698395
300.05887
+19.68454

10
100
5.293243
+0.48280954
30.280441
+1.6879044

10
500
10.379739
-0.06878853
39.915813
-0.023033142

10
1000
15.672676
-1.5173149
53.712418
+0.29795074

10
5000
61.657585
-7.704094
163.77061
-21.083237

10
10000
119.39308
-4.09536
343.02533
+4.0599365


Refactor reduce complexity

Q
A

Documentation
no

Bugfix
no

BC Break
no

New Feature
no

RFC
no

QA
yes

Description
rise readibility by using early returns
remove condition depth

@codisart As this is improvement without any functionality change I've target master branch here. Thanks for the PR!

Thanks, @codisart!


Made interval serializable and updated equals method accordingly

It would be great if interval was serializable

Codecov Report

Merging #5 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master       #5   +/-   ##
=======================================
  Coverage   71.64%   71.64%
=======================================
  Files           2        2
  Lines          67       67
  Branches        5        5
=======================================
  Hits           48       48
  Misses         15       15
  Partials        4        4

Impacted Files
Coverage Δ

time/src/main/kotlin/com/kizitonwose/time/Time.kt
88% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a552a3c...f67a5f2. Read the comment docs.

For some reason, I missed this PR. Merging right now and thanks for contributing!


feat(form-control): simplify scenario where the validationTarget is not set when the constructor runs

I definitely would like this tested, should related to #39

I believe this is an improvement either way, but would love for either @muratcorlu or @hrmcdonald to let me know if this fixed any weirdness on their ends before merging. If I don't hear anything by tomorrow morning I'll go ahead and merge this.

I believe this is an improvement either way, but would love for either @muratcorlu or @hrmcdonald to let me know if this fixed any weirdness on their ends before merging. If I don't hear anything by tomorrow morning I'll go ahead and merge this.

Sorry for the late response, I'll check things out tonight! If y'all get anything merged and I have a scenario this doesn't cover I can open up a different issue for whatever that might be.

@muratcorlu Perfect. I'll count this as working then. @hrmcdonald if you have any other issues, please feel free to open a separate thread, but it looks like the initial issue is fully resolved now so I'll merge this and close out that issue.
Thanks y'all!


Fix #22332: Adjust tests for hidden scrollbar

On macOS and mobile devices the scrollbar is not part of the document, but rather floats above it. This means that it does not affect body/fixed element padding and scrollbarWidth is equal to 0.
The problem in #22332 was caused by the fact that the test expected padding-right to be increased while opening a modal, but threw an error when it was not (because padding + 0 = padding).
A copy of the original getScrollbarWidth() function has been added, as well as checks to skip the tests that would be meaningless anyway.

There's no point running the test when no changes are made to the DOM. I will move the conditional outside and if that fails too I will remove the opening test altogether.

thank you for your comments.
<EMAIL_ADDRESS>From: Ilias
Date: 2017-04-03 17:20
To: twbs/bootstrap
CC: Subscribed
Subject: Re: [twbs/bootstrap] Fix #22332: Check if the scrollbar affects the document before running the padding tests (#22337)
There's no point running the test when no changes are made to the DOM. I will move the conditional outside and if that fails too I will remove the opening test altogether.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@Johann-S I modified the tests to check if the padding/margin is changed as expected, even if that is zero. Should no longer give errors, but please give it a try.

Thank you @Deilv every tests passed (https://travis-ci.org/twbs/bootstrap/jobs/218028318)


Browser/Node.js packaging story

I've been thinking about the best way to go about packaging automerge-rs such that it can become the One True Automerge Backend, used by default in mainline automerge, and installable with no more fuss than an npm install. Here's what I've learned so far, and what I think the path forward could be.
The majority of the groundwork for this is already laid by the automerge-backend-wasm crate, using wasm-pack. The tricky part is getting the wasm bundle that wasm-pack generates to function seamlessly in the various JS environments that automerge currently supports (i.e. Node.js and web browsers via a bundler such as webpack). The tooling in this area seems to be relatively immature (see e.g. https://github.com/rustwasm/wasm-bindgen/issues/2265), so sadly this is cutting-edge rust+wasm+js technology. I expect this story to improve significantly in the coming years.
wasm-pack is able to generate output for a variety of different targets, most importantly, "bundler", "nodejs" and "web". Notably, none of these output formats work for both Node.js and the browser. See https://github.com/rustwasm/wasm-pack/pull/705 for a long discussion and a variety of workarounds. In particular, I think this workaround is probably the best way forward for automerge-rs: https://github.com/rustwasm/wasm-pack/pull/705#issuecomment-529705607
e.g.

package.json:
{
  "name": "automerge-backend-wasm",
  "files": [
    "pkg-bundler/index_bg.wasm",
    "pkg-bundler/index.js",
    "pkg-bundler/index.d.ts",
    "pkg-node/index_bg.wasm",
    "pkg-node/index.js",
    "pkg-node/index_bg.js",
    "pkg-node/index.d.ts"
  ],
  "main": "pkg-node/index.js",
  "module": "pkg-bundler/index.js",
  "types": "pkg-bundler/index.d.ts",
  "sideEffects": "false"
}

and then building with:

wasm-pack build --target bundler --out-dir pkg-bundler
wasm-pack build --target nodejs --out-dir pkg-node

The downside of this approach is that it builds two distinct wasm bundles. The bundles are nearly identical but differ in a few minor details. This is a fairly minor downside I think, as in a production app, in Node.js install time is not a big concern, and in a web app, the bundler will only pull the web version of the wasm bundle to the client.
With this in place, the following should work:
import wasmBackend from 'automerge-backend-wasm'
import Automerge from 'automerge'

wasmBackend.initCodecFunctions(Automerge)
Automerge.setDefaultBackend(wasmBackend)

// ... use Automerge, but it's fast now

With node.js it works out of the box, but with webpack you need to enable the experimental asyncWebAssembly feature.
Ideally all this would be under the hood in automerge and simply importing stock automerge would get you wasm-backed goodness. Unfortunately, the ecosystem is just not there yet. However, I think we can still support automerge-rs as the primary backend for automerge via asm.js. The setup we'd need to accomplish this is a little arcane in order to avoid needing complicated "tree-shaking" / dead-code-elimination strategies to ship the right code to clients, but I think during the transitional period, the package structure of automerge could look like this:
automerge
└── automerge-backend-asmjs

automerge-wasm
└── automerge-backend-wasm

i.e. two automerge main packages, the "default" one backed by automerge-rs compiled to asmjs, and a separate one backed by the wasm version, for those who want to make the tradeoff of dealing with wasm bundling issues in exchange for performance improvements.
Another possibility could be to arrange the packages like this:
automerge
├── automerge-backend-asmjs
└── automerge-backend-wasm

and have require('automerge') return an asmjs-backed API. To pull in the wasm version, you could require('automerge/wasm').
The API I'd really prefer here is something like the following:
const Automerge = require('automerge')
Automerge.setBackend(require('automerge-backend-wasm'))

However, I don't see a path to having a "pluggable" backend that doesn't do one of the following undesirable things:

Increase boilerplate for all users of automerge (i.e. require "registering" a backend)
Always ship the default backend to clients, even if you never use it

Hence the various package arrangement strategies above.

This seems like a great start towards the goal of having a single backend!
As to your first undesirable trait though, could we create a more intelligent default backend based on the environment and allow for overrides? It seems like a one line check is all that's needed to add these smarts for browser vs node: https://github.com/iliakan/detect-node/blob/master/index.js#L2
This would then shift things from being boilerplate to configurable.
When looking into this, I saw that removing asm.js support has at least been raised: https://internals.rust-lang.org/t/should-we-downgrade-drop-the-asmjs-target/9835
I think we both agree that asm.js is a stopgap while the wasm ecosystem matures (and time works its magic on any lingering IE support requirements), so this probably isn't worth worry about today, just keeping an eye on.

could we create a more intelligent default backend based on the environment

the problem is, you have to be able to convince the bundler not to include the default backend when it's overridden. so it has to be a static compile-time check. if we can find a way to do that then i'm all for it.
good to know that asmjs is on its way out. hopefully it doesn't disappear before wasm support gets good.

This is a great write up, I love it. I think the majority of Automerge clients right now are either using a bundler with tree shaking abilities, or they are applications which are not loading their code over the network anyway (i.e Node or Electron), so I don't think having both the WASM bundles in the package is a problem.
Vis a vis doing all this under the hood of Automerge: I completely agree that this is the ideal situation and we want to figure out howto get there. The main reason we can't do that now - and I think the reason you hint at in your ecosystem comment - is that chrome doesn't allow us to import WASM modules larger than 4kb in the main thread. asm.js does allow us to get around that but I vaguely recall that the asm.js file size was much larger, although I will have to check that. Effectively I think this:
const Automerge = require('automerge')
Automerge.setBackend(require('automerge-backend-wasm'))

has to live in a service worker.
Anyway, I'll follow up with some feedback on the PR.

Sorry to be so late to the conversation here.  I'd like to reopen this conversation as I need to resolve the automerge-wasm packaging story shortly.
I don't agree that we should use asm.js.  In my limited testing of it - it's been larger and slower than the native js implementation and therefore is never really going to be a valid candidate to replace it.  It wont be adopted if its functionally worse and there's no point in creating it if no one will adopt it.
I don't see any option but to embrace the async nature of wasm.  We could make an automerge-backend-wasm package that does an async require and can be passed into setBackend and/or create an automerge-wasm which also has an async require and does exactly that for you.  There is also the fact that Node 14 has top level await which opens up some other options as well.
I'm no javascript packaging guru so happy to hear more ideas.

@orionz I'd be happy to see a solution that doesn't require asm.js, but I think the dichotomy in the original post still stands:

However, I don't see a path to having a "pluggable" backend that doesn't do one of the following undesirable things:

Increase boilerplate for all users of automerge (i.e. require "registering" a backend)
Always ship the default backend to clients, even if you never use it

I just published automerge-wasm npm package which works in both node and web apps.  The automerge-js package featuring the classic API will be coming soon

Has any thought been given to React Native? I think that's one definite downside of the automerge-wasm approach compared to the old JS-only implementation of Automerge because React Native doesn't have well defined WebAssembly support yet. I guess the optimal approach would be to compile the Rust code to a native (non-WASM) library for React Native, with non-WASM bindings to the JS land, but that's non-trivial to implement.
Just posting this as a reminder that web/Node.js are not necessarily the only platforms that the users (including me) want to support.

My current plan for this is to first build an Android and Swift wrapper (we need this anyway) and then to build a React Native bridge which is based on top of those wrappers. No timelines on that yet but that's the general idea.


use random port when gather local udp port

Summary
use random port when gather local udp port
Motivation
listenUDP() loop from minPort to maxPort,  it will loop many times if there are many connection already existed.
Start loop from a random port will  more effective.
Describe alternatives you've considered
A clear and concise description of the alternative solutions you've considered.
Additional context
https://github.com/libnice/libnice/blob/master/agent/agent.c
this libnice use nice_rng_generate_int() to calculate the start port.

portStart := rand.Intn(j-i+1) + i
portCurrent := portStart
a.log.Errorf("start from %d", portCurrent)
for {
	laddr = &net.UDPAddr{IP: laddr.IP, Port: portCurrent}
	c, e := a.net.ListenUDP(network, laddr)
	if e == nil {
		return c, e
	}
	a.log.Debugf("failed to listen %s: %v", laddr.String(), e)
	portCurrent++
	if portCurrent > j {
		portCurrent = i
	}
	if portCurrent == portStart {
		break
	}
}

@yuanchao0310
Would it be useful if we released a new tag of pion/webrtc using the updated ICE?
Thanks again for contributing this improvement :)


[RR2]  Middlewares by transport

At the moment we have middlewares in RR2 which can be collected by interface from middleware-provider plugins. But we do not have a separation between http, https, http2, fcgi middleware. This ticket is RFC to discuss our strategy to handle such cases.

I think this ticket can be closed as I can hardly imagine the situation when RR serves multiple endpoints with different middleware at the moment.

I think this ticket can be closed as I can hardly imagine the situation when RR serves multiple endpoints with different middleware at the moment.

@wolfy-j Yep, agree

@wolfy-j Yep, agree


Akka-Http for Scala plugin

Title: Akka-Http for Scala Plugin Contribution
Link: https://github.com/lopiter/pinpoint/tree/add-akka-http-plugin
Target: Akka-Http for Scala
Supported Version: 2.12:10.1.0-RC1 , 2.11:10.0.10 ( I tested these versions)
Description: AkkaHttp(https://doc.akka.io/docs/akka-http/current/index.html)
ServiceTypes: AKKA_HTTP_SERVER, AKKA_HTTP_SERVER_INTERNAL

Are you going to create pull request?

#1301

Created in #3834


feat: use webmap id as env variable

when we get a staging and production web map id we can update the variables

I wonder if we are going to end up using more than one web map? I'm thinking that the overview web map might be different? Not sure.

Ok, should we use this for now or remove it and wait to see what happens?

I'm good either way. Just wanted to let you know my thoughts about what might happen in the future.

Ok, since this is sort of a no op and has a 50% chance of working in the future, let's just merge it.


Move Terraform code to dedicated directory and add main.tf in it

Signed-off-by: Łukasz Sitkiewicz<EMAIL_ADDRESS>

@lsitkiew  There was some discussion today about this approach. The general consensus was there should not be a main.tf for full system setup.    If will be best to keep all things modular and composable for future use.  In the examples dir it would best to have a few components like:
deploy-performance-server
deploy-capacity-server
deploy-clients
full_cluster_setup (that calls the other modules)
run_io500 (that runs after servers/clients are setup) (this can be bash or ansible)
Please try and move the full main.tf steps into the examples directory and refactor it into full_cluster_setup.

Hey folks, I am really sorry for making you wait here. I second what Keith just said, avoiding having a top level main and, instead, having a folder with modules and another with examples. Here are two examples:
https://github.com/terraform-google-modules/terraform-google-vm
https://github.com/SchedMD/slurm-gcp/tree/master/tf