[Refactor] XContent base classes from xcontent to core library

Refactors the xcontent base classes from the xcontent module to the core library so serializable xcontent contracts can be used across other libraries (e.g., new aggregation library, other community contributed libraries). Since opensearch-core is intended to be the foundation library for all of opensearch, this PR continues the effort to refactor those foundation classes from the :server module to appropriate library and base module plugins in order to reduce the surface area of the :server module.
Note: The surface area of this change is large due to xcontent being used as a foundation across the entire project.
relates #5910

Codecov Report

Merging #5902 (8dc57d3) into main (7bd1291) will increase coverage by 0.00%.
The diff coverage is 32.69%.

@@            Coverage Diff             @@
##               main    #5902    +/-   ##
==========================================
  Coverage     70.83%   70.84%
+ Complexity    58766    58758     -8
==========================================
  Files          4769     4769
  Lines        280686   280790   +104
  Branches      40533    40540     +7
==========================================
+ Hits         198826   198925    +99
+ Misses        65561    65527    -34
- Partials      16299    16338    +39

Impacted Files
Coverage Δ

...ensearch/gradle/precommit/ThirdPartyAuditTask.java
0.00% <0.00%> (ø)

...ch/plugin/noop/action/bulk/RestNoopBulkAction.java
0.00% <ø> (ø)

...java/org/opensearch/client/GetAliasesResponse.java
88.75% <ø> (ø)

...main/java/org/opensearch/client/NodesResponse.java
0.00% <ø> (ø)

...ava/org/opensearch/client/NodesResponseHeader.java
0.00% <ø> (ø)

...ava/org/opensearch/client/RestHighLevelClient.java
41.75% <ø> (ø)

...pensearch/client/cluster/RemoteConnectionInfo.java
73.17% <ø> (ø)

.../opensearch/client/cluster/RemoteInfoResponse.java
100.00% <ø> (ø)

...g/opensearch/client/core/AcknowledgedResponse.java
70.58% <ø> (ø)

.../org/opensearch/client/core/BroadcastResponse.java
97.67% <ø> (+2.32%)
:arrow_up:

... and 665 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Am I correct in assuming that the Java package change (from org.opensearch.common to org.opensearch.core) makes this a breaking change that cannot be backported to the 2.x branch?

@nknize I have difficulties understanding the reason for this change

Refactors the xcontent base classes from the xcontent module to the core library so serializable xcontent contracts can be used across other libraries

What exactly is stopping other libraries to use x-content module? We publish it as an artifact for everyone to use, I think consolidating all related x-content classes (pulling from core & co if needed) would be the way to shape this change.

Since opensearch-core is intended to be the foundation library for all of opensearch, this PR continues the effort to refactor those foundation classes from the :server module to appropriate library and base module plugins in order to reduce the surface area of the :server module.

We have more or less concise set of modules, why do we want to beef everything into opensearch-core vs keeping a set of modules? Also, there is more profound issue that this change introduces (or better to say, amplifies) the packages split brain: the o.o.c.xcontext package is now split between different modules, a showstopper for any future modularity efforts.

....a showstopper for any future modularity efforts.

On the surface it seems that way so let me address each of these questions separately.

What exactly is stopping other libraries to use x-content module?

The build is configured such that library modules may not depend on other library modules except opensearch-core.  This heavily restricts us from using libraries to trim down the bloated :server module (I'll give an example below). It was designed this way so that core retains generics and abstract contracts used across all of opensearch with decoupled concrete implementations in specific libraries, modules, and plugins.

We have more or less concise set of modules, why do we want to beef everything into opensearch-core vs keeping a set of modules?

This change isn't beefing everything into core. It's teasing out the reusable generics and interface contracts to the core library which is used to implement the concrete classes and architecture details across modules and concrete libraries.

Also, there is more profound issue that this change introduces (or better to say, amplifies) the packages split brain: the o.o.c.xcontext package is now split between different modules

It seems that way, but if you look at the change it's sticking w/ the intended design and refactoring the XContent base classes and interfaces to the core library while leaving the concrete implementations of those MediaTypes (e.g., JSON, CBOR, SMILE, YAML) in the x-content library. As it turns out, we might need to refactor those concrete implementations to an x-content module in a follow up PR but I'm deferring that unless it's absolutely necessary.
The driver for this is to further modularize the :server module by teasing out the generics to libraries and implementations to modules. Let me give an example where the cross library dependency limitation currently restricts a big part of this (hence the xcontent change).
Aggregations
Third party libraries, plugins, modules, etc, want to build new aggregations all the time. We provide this ability and it's super powerful. To do so is quite boiler plate: 1. If the CoreValuesSourceType library doesn't already provide the requisite ValueSource, write the needed ValuesSourceType the new aggregation depends on, 2. write the Aggregator, 3. write the Aggregator Factory, 4. write the AggregationBuilder, 5. register the aggregation through your plugin using the AggregationSpec
As you can see in this process there is a large number of abstract classes and interfaces required to support implementing a new concrete aggregation anywhere in the "ecosystem". Organizationally, these base classes are spread across the aggregations and aggregations.support along with the mechanisms needed to implement the aggregation phase(s) in the SearchModule and SearchService. This means for anyone wanting to build a custom aggregation they would take a build.gradle dependency on "org.opensearch:opensearch:${opensearchVersion}" which pulls in a bloated server jar containing much more than the classes and dependencies they need to build their aggregation.
I think most on this project agree this is a modularization anti-pattern, so to remedy we're striving to make the codebase much more modular. So how do we achieve that here with this clumsy aggregation foundation and user facing "API" (which isn't exactly well documented, or implemented, or supported)? At first pass you might say, "hey, how about we stick with the intent of the libraries and modules design and refactor the generics and base classes from the bloated :server module to a new opensearch-aggregations library with the concrete implementation in a new aggs-common module that continue to use the search.aggregations.* namespace? I'd agree with this, in fact I'd go a step further and say this is where I think we should go with, not just aggregations, most of the big opensearch mechanisms (field mappers, field types, doc value types, field data, etc). In the end the :server module should be nothing more than the "motherboard" providing the concrete implementation of all the opensearch base components within the libraries and concrete extendable mechanisms provided by the modules and optional plugins.
But if you embark down that journey you will quickly find the cross library dependency restriction blocks much of this objective and the tight dependency requirement of the :server module with other modules quickly leads to JarHell issues. Take the aggregation decoupling use case, for example. Aggregations, like field mappers, are serializable and therefore depend on XContent in order to implement the ToXContentFragment logic. So how about refactoring all the XContent base classes to the opensearch-x-content library then? That works until you refactor any base interfaces to a library other than opensearch-core, because libraries cannot depend on each other.
So that leaves us with a few options:

remove the cross library dependency restriction
refactor all base classes that would result in a cross library dependency situation into modules instead
Do nothing, leave all base classes and interfaces that would result in a cross library dependency in the server module and leave users to depend on the massive jar
refactor opensearch core base classes and interface contracts that would result in a cross library dependency to the opensearch-core library and the implementation details to other libraries, modules, and plugins

With this PR I chose Option 4 because it not only fits the modularization objective but after the aggregation framework is refactored to a new module (and supporting library) users in the aggregation scenario above would only need to take a dependency on the opensearch-core and a (soon coming) opensearch-aggregations library to implement a new concrete aggregation in their own plugin.
I hope this helps paint a clearer picture?

I hope this helps paint a clearer picture?

Thanks @nknize , yes, it does. I missed the restriction on opensearch-core usage.

refactor opensearch core base classes and interface contracts that would result in a cross library dependency to the opensearch-core library and the implementation details to other libraries, modules, and plugins

Now it make sense, I think we should not open the gate for cross library dependency restriction (I believe you also don't want to do that) but what about something in the middle: introduce opensearch-core-xcontent module and have a set of core modules (not just opensearch-core), the opensearch-core-aggregations could follow fe?
I am afraid that the limitation on just one opensearch-core would lead to it becoming a kitchen sink of everything the plugin / extension / ... may need.

Am I correct in assuming that the Java package change (from org.opensearch.common to org.opensearch.core) makes this a breaking change that cannot be backported to the 2.x branch?

This change intends to target 3.0 only (I just added the labels, sorry about that). However, that wouldn't preclude a backport if we decided to pull it into 2.x since mixed cluster rolling upgrades are only mixed nodes not mixed jars. The side effect, however, is a namespace ripple across all plugins that depend on xcontent base classes. I'd wager that's at least 90% of plugins. The refactor would have to be done before releasing the 2.x version that would include this change.

I think the namespace change would preclude a backport since it would violate our compatibility guarantees even if the change is trivial for plugins. Can this change be done without the Java package name change in a way that makes it backward compatible?

introduce opensearch-core-xcontent module and have a set of core modules

This would work as a intermediate step. However, my hope was something along the following (let me know your thoughts):

refactor CoreValuesSourceType and ValuesSource implementations to opensearch-core
a new :aggregation-framework module that implements the core aggregation framework (services, plumbing, etc).
a new :aggregations-common module that implements the core aggregations (e.g., avg, sum, histogram, extended_stats, basically all the aggs in the aggregations namespace)
a new :opensearch-aggregations library that provides the base classes, interfaces, contracts used by 2. and 3. above

With the above design the two new modules take a dependency on the opensearch-core and opensearch-aggregations libraries to implement the framework and core aggregations provided by opensearch min distribution. Users then only need take a dependency on the opensearch-core and opensearch-aggregations libraries to implement their concrete implementation.
By moving everything into modules users would need to take a dependency on the modules which brings in concrete implementations they may not care about.

I am afraid that the limitation on just one opensearch-core would lead to it becoming a kitchen sink of everything the plugin / extension / ... may need.

I agree this could be a risk, but that's already the case w/ :server module today because there is no clear definition of what goes where. I'm hoping this refactor PR trailblazes those decisions so we can document this in the package-info.java javadocs and enforce in our PR reviews. To track and facilitate ongoing discussion about future refactor PRs I opened the #5910 meta issue.

@andrross

violate our compatibility guarantees

Are you referring to a cosmetic guarantee that a dependency namespace doesn't change w/in minor releases? Remind me if we have this documented somewhere?

Are you referring to a cosmetic guarantee that a dependency namespace doesn't change w/in minor releases? Remind me if we have this documented somewhere?

This is the most extensive semver documentation I'm aware of. I think it comes down to interpretation of "API". If we consider these XContent classes as part of the API used by plugins, then a namespace change is certainly a breaking change to that API.

I would go further, moving classes to different JARs even under the same package is a breaking change: we could not guarantee that some use only subset of JARs and as such, the classes won't be there anymore, requiring to pull in new dependency.

...moving classes to different JARs even under the same package is a breaking change

It is. I think the key here, though, is whether it breaks a "forward facing API" (as defined by the project) that formally complies w/ SemVer. We're allowed to "break" @opensearch.internal and @opensearch.experimental classes because they're not classified as forward facing @opensearch.api. I think that's the "one way door" change that needs to be thought about and maybe github labeled in PR reviews. Longer term I'd like opensearch to have something more enforceable and automated like Apache Spark's Developer Annotations. I added the javadoc tags as a first step for the missingJavadoc task because there wasn't anything else. Longer term I think we can add more automation and better enforcement through gradle checks using a well defined annotation implementation.

With the above design the two new modules only take a dependency on the opensearch-core and opensearch-aggregations libraries to implement the framework and core aggregations provided by opensearch min distribution.

Aha, the bigger picture starts to emerge, I think it is a viable path ahead.

I agree this could be a risk, but that's already the case w/ :server module today because there is no clear definition of what goes where.

That we need to clean up I think.
Thanks @nknize , could I ask you please to formalize your decisions in the issue https://github.com/opensearch-project/OpenSearch/issues/5910 (and create sub-issues if necessary), that could help other to understand the path forward. The only note I would like to have (this is a good time) is to ensure that the core modules (opensearch-core) and the implementation modules (x-content) do not have package  split-brains. I won't matter much right now but would eliminate yet another set of breaking changes in the future.
Thank you.

...could I ask you please to formalize your decisions in the issue https://github.com/opensearch-project/OpenSearch/issues/5910 (and create sub-issues if necessary)

Definitely! I'll copy the details to the meta issue and reference this conversation for posterity.

The only note I would like to have (this is a good time) is to ensure that the core modules (opensearch-core) and the implementation modules (x-content) do not have package split-brains.

I agree. This should be formally written down somewhere (maybe CONTRIBUTING.MD?) and somehow enforced in the PR reviews. That will be difficult and we'll likely end up w/ future PRs introducing a split brain in the meantime. At least until all maintainers (and future maintainers) develop the muscle memory to check for this on PR review.

I'll go ahead and explicitly state the concern I was getting at with my questions around backporting: given that the 3.0 release is not yet scheduled, I don't love the idea of a structural change that impacts so many files like this. We've already had problems with divergence between the main and 2.x branches causing complications in development (/cc @kartg) and this seems likely to exacerbate that problem. I'd prefer to see changes like this deferred until there is at least a timeframe for 3.0, especially if 3.0 may not be released for over another year from now.

I don't love the idea of a structural change that impacts so many files like this. We've already had problems with divergence between the main and 2.x branches causing complications in development.

Let's work through that here because we shouldn't shy away from large progress (especially for so long) because backporting is "complicated". We should figure out inventive ways to reduce the friction to enable incremental progress. Even if that means we figure out better ways to use worktrees or some other guy mechanism.

Fundamentally our branching strategy somewhat favors development on the next major version at the cost of backporting to a divergent branch for the current major version, and I think that's the thing I'm poking at here. I'm definitely not an expert in this area and changing that strategy is a bigger conversation than this specific change. I admit holding off on desirable changes to avoid divergence is really bad as it impedes progress. Though I think the technical solutions here are likely limited because at some point human judgment will be needed to apply a logical change across divergent code bases. Maybe other folks don't see it as a big problem, but I do worry about introducing bugs anytime a developer has to do non-trivial conflict resolutions across branches.

Maybe other folks don't see it as a big problem, but I do worry about introducing bugs anytime a developer has to do non-trivial conflict resolutions across branches.

I share your concern @andrross , the change of such a scope would basically make backporting automation useless (except for trivial cases) - it will always end up either with conflicts or compilation error.
From the other side, we have to move forward, so what option do we have?

Move the bits between modules but no package changes: significantly smaller impact area
It would help now but only delay the package change for later

Or we could try to:

Move the bits to the core package
But keep the old classes around (same package but deprecate for removal), they would be proxies that delegate to the core ones
In 3.0.0 we could remove the old classes

Just some thoughts ....

I really like the direction this is going in, but the timing is giving me some heartburn.  What if we held off on merging this change until we had a date for 3.0 (~6-8 weeks out).  This would minimize the amount of time we have where developers would have to straddle (for want of a better word) this change with the risk of making a mistake.
In a related question, do you have any reading of the tealeaves, Nick, when the next version of Lucene might be coming out?

In a related question, do you have any reading of the tealeaves, Nick, when the next version of Lucene might be coming out?

My gut says not for a while. 9.5 release was just proposed a week ago w/ majority favoring. History is trending toward a longer tail between major Lucene releases. Which could be a good thing for OpenSearch decoupling major releases from Lucene changes. Also gives reasonable time to think through Migrations and BWC such that we could absorb lucene index breaking changes across minor releases.

From the other side, we have to move forward, so what option do we have?

Move the bits between modules but no package changes: significantly smaller impact area
It would help now but only delay the package change for later

@reta This is basically the approach we've taken with the inclusive naming project. There are some very widely used classes (e.g. org.opensearch.action.support.master.AcknowledgedResponse) that for various reasons can't really live side-by-side with a renamed variant, so we've deprecated that package name but are waiting to do the very intrusive s/master/clustermanager refactoring until we have a better idea of when the last minor release of the 2.x branch will be.

I lean this direction as well. I'd be fine refactoring this PR to retain the namespace and defer that to a later PR. This change is mostly around moving the extensibility needle and reducing the surface area of :server while fostering a thoughtful discussion on reduced impact ways of doing that. It seems to be trending that direction so I thank you all for the thoughtful comments.

I'd be fine refactoring this PR to retain the namespace and defer that to a later PR.

If we take this approach, then we're moving some classes from :server to :core. Given that any project currently getting these dependencies from :server will still get them (transitively) from :core after the refactoring, I will argue that this is in fact backwards compatible. @reta Do you agree or am I playing too fast and loose with compatibility concerns?

@reta Do you agree or am I playing too fast and loose with compatibility concerns?

@andrross In this particular case I agree with your conclusions: the :server has to come with :core as the dependency (and :x-content also comes with :core as well), so we should not break anyone (assuming no package changes).

One thing I forgot to mention is that if/when we commit to using the jigsaw modularity framework we have to ensure unique package names within modules. The jigsaw framework does not allow packages to be split between modules. If we define the namespace refactor as a breaking change then I highly suggest we do this before 3.0, so this decision will only be temporary while we tidy up our house by moving classes to appropriate libraries.

opened #6218 which will reduce the surface area change of this PR

Gradle check failure is unrelated and looks like it could be a bug in jackson-dataformat-yaml:2.14.2 or snakeyaml:1.32.  Opened bug report in #6411. Refiring check and will address separately.

This is a breaking change in version 2.10!
Where is the communication of breaking changes? This should be part of the changelog.

This is a breaking change in version 2.10!

@AndreVirtimo you are absolutely right, the change was slipped away but it should have been in the change log and release notes for 2.10, sincere apologies for that.

This is a breaking change in version 2.10!

Can you share your use case? Where / how did you run into this? Did you get a NoClassDefFound? Do you have an implementation that relies on it?
The reason I ask is because the core technically does not (yet) provide a formal plugin API (we're working on this; feel free to track history through #5910) and dependency libraries are (should) already be handled by the min-distribution, bundle, or docker releases. I want to make sure you didn't stumble on a release artifact bug.

@nknize A breaking change info for developers would be important (+ necessary migration steps).
E.g. for a OpenSearch plugin, you must adjust the plugin version for the target OpenSearch version. This means also you must re-compile the plugin. This will break due to this and the other related refactoring. In our plugin, nearly every class (> 200) had to be adjusted. Not only moved classes but also API changes happened in 2.10. (Strings.toString(XContentBuilder) removed etc.)
But also in other code that uses the REST High Level Client we use the XContentBuilder quite often and like it. The RestStatus class and others also moved.
It was an extremely suprising and big change for a minor release (developer point of view). ;-)

@olfuerniss thank you for sharing your experiences. It's always helpful to get feedback from others in how they're using the artifacts. It sounds like you're developing your own plugin? Similar to your comments, the OpenSearch plugins in general deal with these kinds of issues every time-based minor release cycle for reasons outlined in #5910. As I mentioned, the refactoring effort is a move to fix this in the future to support an official Developer API and (since it's not there yet and requires all core maintainers pulling in the same direction) will require some rocky moments like this since the project attempts to minimize major releases.
Just to add a few things to open the aperture here:

A breaking change info for developers would be important (+ necessary migration steps).

There is a >breaking label that is supposed to be used for this purpose. Lot's of low hanging areas for improvement here. For example, labels (beyond just >breaking) could/should be better used and applied by maintainers (this is an area I've been trying hard to pull folks along), the changelog process is manual, non-standard, and clumsy (this was discussed on a separate issue, moved to a new project wide proposal, and months overdue on being revised. I think your experiences and wishes could help nudge that along to fix these communication struggles).

It was an extremely suprising and big change for a minor release (developer point of view)

+1. Until the core API is defined (through JPMS, library exports, and other java mechanisms like sealed classes), plugin developers within and outside the opensearch project org are going to experience changes like this relatively often. It's a side effect of forking a code base that did not establish a formal developer API (for these exact painful reasons your describing). This is why the refactoring is happening to begin with but requires a delicate balance of how much deprecated code surface area and internal logic paths should be retained solely for backwards compatibility purposes vs moving to a well supported secure Generally Available API. On this project we talk about SemVer in three areas. Progress toward Developer API enforcement was only added as recent as two months ago (to assist with the API blast radius from PRs like this). But expectations need to be set. Annotations aren't yet fully applied, and where they are applied many will change over the coming months because of all the maintainers on core only one worked at elastic when the core plugin framework and xcontent classes were first designed and written so the annotation choice is only best effort.


Revert "Add default min-height and top/bottom padding"

Reverts chanzuckerberg/czi-prosemirror#31 because it also affects comments

Not restoring since there were conflicting changes that merged in with https://github.com/chanzuckerberg/czi-prosemirror/commit/153bf84c17b70d555a9c8f991e07406d02edec6b


fix(neotest): support adapter key on adapter for playwright

What is this PR for?
Similar to https://github.com/LazyVim/LazyVim/issues/847 , only this adapter adds a setup function into the 'adapter' key in the plugin table (source).
It is somewhat annoying that neotest does not provide a recommended way for adapters to define their configuration function (at least not explicitly)...
Perhaps we should work around it temporarily, but reach out to either neotest or neotest-playwright authors?
Does this PR fix an existing issue?
I didn't see anyone complaining about playwright adapter in issues or discussions :/
Checklist

[x] I've read the CONTRIBUTING guidelines.

ty


Hot loading not working

When making changes to the Home app\components\Home.tsx component (change text from Home to Home1234), the UI doesn't get updated.
Note, the same thing (hot loading) works in the standard React repo (using plain javascript).

@DonJayamanne Thanks for reporting this. Let me know if my fix solves this problem for you!

Hot reload still not working for .ts files;
[HMR] The following modules couldn't be hot updated: (Full reload needed)
This is usually because the modules which have changed (and their parents) do not know how to hot reload themselves. See http://webpack.github.io/docs/hot-module-replacement-with-webpack.html for more details.
logUpdates @ process-update.js:81
applyCallback @ process-update.js:49
hotApply @ bootstrap cb95e1a…:531
cb @ process-update.js:52
hotUpdateDownloaded @ bootstrap cb95e1a…:311
hotAddUpdateChunk @ bootstrap cb95e1a…:283
webpackHotUpdateCallback @ bootstrap cb95e1a…:4
(anonymous) @ 0.f6e7942….hot-update.js:1
process-update.js:89 [HMR]  - ./app/containers/HomePage.tsx
(anonymous) @ process-update.js:89
logUpdates @ process-update.js:88
applyCallback @ process-update.js:49
hotApply @ bootstrap cb95e1a…:531
cb @ process-update.js:52
hotUpdateDownloaded @ bootstrap cb95e1a…:311
hotAddUpdateChunk @ bootstrap cb95e1a…:283
webpackHotUpdateCallback @ bootstrap cb95e1a…:4
(anonymous) @ 0.f6e7942….hot-update.js:1
process-update.js:89 [HMR]  - ./app/components/Home.tsx

I just replaced ts-loader with react-hot-loader & awesome-typescript-loader and its works like a charm.
BTW thank you for such a great repo @iRath96

Thanks for letting me know 👍 I've added react-hot-loader as well now.


Add init syntax for scripts directory

Add init syntax for scripts directory.
@bmosaicml was running into issues importing the scripts.data_prep.convert_dataset_hf file in the test_training.py test but import scripts was working. This PR should fix these import errors.
_______________________________ ERROR collecting tests/test_training.py ________________________________
ImportError while importing test module '/Users/jeremy.dohmann/LLMWork/llm-foundry/tests/test_training.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_training.py:17: in <module>
    from scripts.data_prep.convert_dataset_hf import main as main_hf  # noqa: E402
E   ModuleNotFoundError: No module named 'scripts.data_prep'

I'd prefer we not do this. Anything that needs to be imported should probably be part of the llmfoundry package properly, and for testing scripts specifically we can just append to the path in the test file so the script main can be imported.

Agreed closed. Strange that Jeremy saw this bug earlier.

I resolved this with Jeremey earlier. It was an issue specific to his python setup.


Avoid - characters in generated union names

Should be a pretty simple one, but when dealing with negative selector values for a union type, the generated name might end up like Arm_minus_-1 which fails compilation. Credit to @moohax for the find.
Looks like negative values were already considered, so I just added Math.Abs to shore things up.
Failing example:
> $server = Get-RpcServer C:\windows\system32\wbiosrvc.dll
> $server[0] | Get-RpcClient

Doh, I clearly had the right idea ;-) Thanks for the patch.


Issue #88 disallow directory listing

Fixes #88 by disallowing directory listings

@meltsufin added extra comments and a test


Text changes completed

Task name: AzureFunctionAppV1
Description: Received label changes for a function app
Documentation changes required: (Y/N)
Added unit tests: (Y/N)
Attached related issue: (Y/N)
Checklist:

[x] Task version was bumped - please check instruction how to do it
[x] Checked that applied changes work as expected

Can you change the label.appName to read "Azure Functions App name"? I would also like to suggest that the help.appName could be updated to something like "Enter or Select the name of an existing Azure Functions App. The Function Apps listed will be based on the selected app type."

@FinVamp1  Your review comments are addressed.


carthage update does not build framework

Output:
*** Cloning meteor-ios
*** Checking out meteor-ios at "0.1.2"
*** xcodebuild output can be found in /var/folders/pj/0_h5pbg1787dcz5j24_rs36m0000gn/T/carthage-xcodebuild.JQWjmo.log
Project "Pods.xcodeproj" has no shared schemes

same issue here

Digging through, I see that Pods.xcodeproj does in fact not have any shared schemes, but I'm confused  as to why carthage is even looking in it. Probably has something to do with it automatically searching all .xcodeproj's for shared schemes

I also cloned the repo, opened the workspace, but I'm unable to build any of the targets :/

Adding an explicit commit to my Cartfile.resolved solved the problem.
Cartfile.resolved:
github "martijnwalraven/meteor-ios" "3b598cc9421df9affbef155ee3ce6dc5ff0572a9"

I think we need to bump the version to reflect the latest commit.

@justinmakaila cool, that worked for me too.. So this might be a Carthage bug?

@justinmakaila after installing it, did you run into this by any chance?
https://github.com/martijnwalraven/meteor-ios/issues/26

I'm not sure, I'm still writing code to test.
It's probably an issue because the dependencies are not specified within a Cartfile.resolved in the repository.
I'll take a look in a bit, see if I can file a PR.

I've released a new version that should at least allow the Meteor framework to build again using Carthage. But I'm afraid that isn't of much use because of #26.
Having to support both CocoaPods and Carthage is a pain, so suggestions for improvements are very welcome. I'm struggling with finding a way to embed dependencies specified as Pods, but maybe that is not the way to go.
Using Cartfile.resolved won't work for dependencies that do not have an Xcode project but only build using CocoaPods (such as https://github.com/mattt/InflectorKit).

@martijnwalraven thank you for looking into this.
I was thinking about forking InflectorKit and turning it into a framework that can be installed by Carthage, but I don't have a lot of time so I ended up switching to CocoaPods.

@thatjuan and @martijnwalraven I have a fork of InflectorKit, PocketSocket, and SimpleKeychain which solves this issue, I just need to get an issue I'm having personally with Carthage worked out.
I've never found CocoaPods to be a desirable solution, as the workflow requires writing more code, where Carthage is as simple as changing some Xcode settings (once!) and using git tag to maintain versions.
That said, both are almost required to be considered a "modern" cocoa framework, which is a bummer when you have dependencies which are not "modern".

@justinmakaila we're on the same page regarding CocoaPods. I switched back only because I really needed to get meteor-ios working for this project.
Thanks for working on getting this up and running with Carthage and let me know if there's anything I can pitch in. I've run into several issues with Carthage as well, but I'm sure it'll mature fairly quickly.

Forks are here:
PocketSocket
InflectorKit
SimpleKeychain
I also have a fork of this repo, which has a Cartfile which points to those dependencies here.
Once my issues with Carthage are resolved, I'll file a PR.

Great work! Ideally, I'd like to find a way to build the Meteor framework target using the Cartfile dependencies. Right now, the target uses the Pods project generated by CocoaPods, which means there could be version conflicts with the versions specified in the Cartfile.
Do you think it would make sense to include the dependent projects in the workspace and link to the framework targets? Maybe using submodules would help here? Or maybe include the Carthage/Build directory? In any case, it would be nice if we could get rid of the checked in Pods directory completely. I just want to make sure people cloning the repository can run the examples without using either CocoaPods or Carthage.

That looks good! I merged your pull request. The test targets weren't working, so I had to make a few changes. Some of these had to do with directories I had renamed in a previous commit.
But I also had to add OCMock.framework to Carthage/Build. OCMock doesn't support Carthage unfortunately, but I did find a fork which at least has an iOS framework target. I added it to Cartfile.private but it still wouldn't build. For now, I've built it manually and copied the .framework file to the build directory.
I also had to add a 'Copy Dependencies' phase to the test targets. I'm not sure why it wouldn't use the embedded frameworks in Meteor.framework. Do you know anything about this?

This should also be closed with #27.
I cleaned up some of the build phases of the targets, the "Copy Dependencies" on the Meteor target was overkill, because those dependencies are linked, and shouldn't be copied into the actual Meteor.framework.
I checked OCMock into the repo and got all the Unit Tests to pass, but I'm running into a lot of failures with the server integration tests. Do those pass for you?

The server integration tests require a running test server (run 'meteor' in 'Tests/Test Server'). There are two tests that fail for me right now, both having to do with logging in. I think that might have to do with the tests expecting a user with a certain password to exist in the test server. I'll look into it.


Document AzureWebJobsStorage__accountName

Document previously undocumented appSetting AzureWebJobsStorage__accountName
The setting is documented elsewhere:
https://learn.microsoft.com/EN-us/azure/azure-functions/functions-identity-based-connections-tutorial#edit-the-azurewebjobsstorage-configuration

@ggailey777
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

email and Teams msg to author

@snaits Drat! I just added this setting before I saw your PR. I'll go and cherry-pick your commits so you can get credit for helping out. Thanks!

The commits in this PR are now reflected in the live article. Thanks for contributing!


Feature suggestion: Record event status to InfluxDB

Expected behavior
As an operator I can configure an instance of this handler to record event statuses in InfluxDB.
Context
Sensu records an intentionally limited history of statuses for each event (entity/check pair). As an operator I would like to record these statuses in a long-term data store so I can plot statuses against other measurements.

This is very useful. What would be the desired workflow for this feature?
Add a new flag? Maybe "-check_result_only", and it would then only process the event.check.status field? The associated handler would not need to worry about the "has_metrics" filter.
My own personal preference is to setup the check_name as the measurement, with "entity", and "namespace" as tags, and "status" and possibly "output" as fields.

Closed by https://github.com/sensu/sensu-influxdb-handler/pull/34.


About the Direction of Input Image

Dear Organizers,
I have a question regarding the direction of the input image. When our algorithm is evaluated on the Grand Challenge platform, we receive a single image, SimpleITK.Image, as input. I am curious if the direction information stored in this SimpleITK.Image is physically accurate. I have observed that all images in the dataset are in the 'RAI' direction. Are there any images in different directions for the Final Test Phase? If such images exist, will the direction information be stored accurately in the SimpleITK.Image. Understanding this aspect is essential for ensuring our algorithm functions correctly.
I look forward to your reply. Thank you!
Best regards,
Haoshen

Dear @Mors20,
Yes, the metadata is physically accurate and will be so in the test set as well. Anyway, all images in the test set will have the same direction.
Best regards,
Federico.


[Feature] Add an option requiring that space is pressed before triggering a combo.

Some users reported interest in new ways to trigger combos (#280). An interesting way would be to require the user to press the spacebar before any combo substitution is performed.

Here are some test builds. To enable the option, check the box Preferences ➜ Combos ➜ Combo triggering ➜ Trigger only after spacebar is pressed.

Installer
Portable edition

I've checked the test build with [Trigger only after spacebar is pressed]. With this option enabled, the space works as a triggerer but not as an actual character. It should be better to add a space automatically next to the word (where the cursor is) otherwise the spacebar should be pressed twice. What's more, it should be better to work also with special characters like dot (.), comma(,) etc.
Example
bff --> best friend
You're my bff.
If I write bff at the end of a sentence like this with the spacebar option activated the combo won' t be triggered.
I recommend to add this feature or make the spacebar option be set individually on every combo.
If you see TEXT REPLACEMENT in IOS, you should understand what i mean.

I decided to remove the space character at the end, because in some case you do not want it, and you always have the option to add a space character at the end of your snippet.
However It makes sense to add a preference setting for that. I've created a new issue for that.
Commas, periods and other punctuation marks can be used in a keyword and as a consequence they cannot be used as a trigger .


Two songs are playing at the same time and "Pause, Stop and Play" button not working properly after I changed app's setting to "Wait until finished"

Expected behavior
I expected that the player would move to my selected next song after finishing the first song without playing two songs at the same time.
Actual behavior
I changed this app's setting to "wait until finished" after then this app is playing two songs at the same time and "Pause" "Stop" "Play" button doesn't work as well.
How to reproduce

Open the app.
Go to "Browse Music Library" and select a few songs to add on your selection dashboard.
Then go to this app's options and click on "Change app settings".
Select "Wait until finished" under the "Loop switching behavior" and save it.
Then get back to your player's dashboard and play a song. [The app will indicate this song in red color]
Tap on another one and this song will indicate as green color.
Notice that your first selected(Red color) song is playing. After that, fast forward this song or let this song to the end.
After finishing your song, the player will move to the green song.
Carefully hear that both songs are playing at the same moment. You can also click on "Pause, Stop, Play" to notice some weird behavior of this app.

Browser/App version: V0.9
Operating system: Android 6.0.1

Recording Of The Bug

Thank you for submitting this. So far I am unable to reproduce the error, despite the very detailed description you gave. As soon as I find time, I will revisit this again and see if I can figure it out. Maybe you can provide what kind of device you were using to use the app? Thank you!

I was used Samsung A5 when these issues occurred.
Hope you will fix these issues and make this app bug free.
Thanks

Oh, of course! But please be aware that this is a project I do in my free time, so I do not get to spend time on it very often. Will look at it as soon as I can though!

update: Just to let you know that I am still working on this - really sorry, but I just don't find enough time for the project to get stuff doe faster. I am in the middle of migrating the whole thing to using oboe for playback and AndroidX for the whole appearance.
This means quite a bit of the app has to be refactored, so it will still take time - but I will address this and the other issues in the process. Everything should be resolved when I finally release 1.0 of this litthe project.

This should be resolved with the release of Loopy 2


feat: notify members when securing the bag 💰

Description ✏
Closes #534

Monitors the #career-secured-the-bag channel using the addSlackMessage job
if messages come from secure the bag channel (using SLACK_SECURE_THE_BAG_CHANNEL_ID), extract the userId and text and pass to our notification function
Our function checks the text for keywords such as ['join, 'offer', 'signed'...] and if true we send a dm using the job('notification.slack.send') with a congratulatory message and link to their member profile Work History

Type of Change 🐞

[x] Feature - A non-breaking change which adds functionality.
[ ] Fix - A non-breaking change which fixes an issue.
[ ] Refactor - A change that neither fixes a bug nor adds a feature.
[ ] Documentation - A change only to in-code or markdown documentation.
[ ] Tests - A change that adds missing unit/integration tests.
[ ] Chore - A change that is likely none of the above.

Checklist ✅

[x] I have done a self-review of my code.
[ ] I have manually tested my code (if applicable).
[ ] I have added/updated any relevant documentation (if applicable).

A question for @ramiAbdou: I wasn't sure if we should store the channelId for #secured-the-bag with Redis or as an environment variable. Using an env variable seems more natural, but I used Redis because it was consistent with the other jobs being called in add-slack-message.ts. What do you think?

@tomas-salgado Yeah I agree that an environment variable definitely seems more natural, the context behind storing the other ones in Redis keys is there were multiple opportunities/auto reply channels and having them be Redis keys would allow us to add/remove channels without a rebuild (similar to feature flags). Dealing with arrays in environment variables is also a tad annoying cause you have to parse manually (from a comma-separated string). In retrospect, these values are not likely going to change often so probably should've just gone the environment variable route.
I don't feel strongly enough to change them, but if you do feel free to put up a follow-up PR!


Added RdBu_r colours from -50 to 50 by 1

Hi,
Here's code with blue&red colours

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[Enhancement] (nereids)implement RecoverCommand in nereids

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
recover command is supported in legacy planner but not in nereids(new doris fe planner). you need implement the following use command in nereids:

[ ] https://github.com/apache/doris/issues/42544
[ ] https://github.com/apache/doris/issues/42545
[ ] https://github.com/apache/doris/issues/42547

Solution
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@LiBinfeng-01  Pls assign all these 3 to me. thanks. also do we need to remove the legacy code for corresponding function or keep it as for now?

@Vallishp Thanks, you need replay to the all 3 issues, then I can assign them to you one by one

@Vallishp thx，these issues have been assigned to you. You don't need to remove the legacy code after implement same features of new one


Ensure full method docstring coverage

I've gotten lazy with some recent PRs and haven't made sure all the functions/methods have docstrings. This should be addressed. A lower-priority task would be ensuring that all docstrings are up to date and reflect the current arguments. I personally use VSCode autoDocstring to do the templating.

It might be useful to explore pydoctest as a way to help with this.

It would be useful for the docs pages to have some more summary content. We should look into whether there are more opportunities to include this in the actual files (e.g. can you have a file-level docstring?) and if not include it in the docs folder.


C++ example about BP209

The example includes device connection, parameter setting, data display, and 2D reconstruction image display.

Good job!


Fixed error caused by PR #1261

Fixes a small error in PR #1261. Now makes a check to ensure records[0] is not undefined before getting its scrollHeight.

          frecords.css({ overflow: 'hidden', top: records.css('top') });
         if (this.show.emptyRecords && !bodyOverflowY) {
             var max      = Math.floor(records.height() / this.recordHeight) - 1;
-                var leftover = records[0].scrollHeight - max * this.recordHeight;
+                if (records[0]) var leftover = records[0].scrollHeight - max * this.recordHeight;
             if (leftover >= this.recordHeight) {
                 leftover -= this.recordHeight;
                 max++;

Woudn't this result in leftover being undefined if records has no items, which means the next lines would crash in that case?

This is better, the test does not crash anymore.
But it calls addEmptyRow with an undefined height, which adds the following html junk:

Good point. It technically works the way this is used, but it would be much safer and cleaner to make sure leftover is defined regardless. I have updated the PR. Thanks!


[NEW-FEATURE] log button action in  DataExchange Edit window

/cib

Branch 466-_NEW-FEATURE_log_button_action_in_DataExchange_Edit_window created!


Update approuter and include links to the endpoints in the webpage

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Fix link to https://www.kitura.dev

Fix link that was not working

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

As this change is trivial and unlikely to break CI build. Would it be ready to merge?

@kikeenrique Yeah, we'll figure out the CI separately.  Thanks for the update.


[tabs]: Dynamically generating tabs produces unusual behaviour

Summary of the bug
Dynamically generating tabs produces the following issues:

Tab panel appears above the tabs
The first tab is not selected
If the second tab is selected and another tab is dynamically removed, the second tab loses the selected state and the first tab is selected
All tabPanels are displayed when tabs are dynamically loaded and a tab is selected

🪜 How to reproduce
See below
📸 Screenshots or code
Code example which can be added to Stackblitz or application to reproduce:
Link to stackblitz: https://stackblitz.com/edit/vitejs-vite-ywsvuc?file=src%2FApp.tsx
🖥 📱 Device

Type: Desktop
Device: Windows
OS version: N/A
Browser version: Chrome 122

🧐 Expected behaviour

The tab panel should appear below the tabs
The first tab should be selected when dynamically adding all tabs
Only selected tab panel should appear

📝 Acceptance Criteria
If relevant, describe in full detail the different interactions and edge cases that the component or patterns needs to fulfil.
Given the tabs has no data
When I add tabs dynamically
Then the first tab should appear selected with the corresponding tab panel appearing underneath
Additional info
Tell us anything else useful to help us fix or understand the problem.

Work has been done on the linked branch to address the issues. Currently fixed the first tab not selecting, and the issue around the second tab losing selection during removal could not be replicated. The other two issues are still outstanding.

Regarding the first issue "Tab panel appears above the tabs" I think this is because IcTabContext doesn't mandate the order in which its child components are rendered (i don't know much about slots)
I stuck a couple of  tags between the Group and Panels and it ended up sandwiching the panels:

So i think the solution to this might be specifying TabContext's child slots, but that would be a breaking change...  will see if I can solve this by just modifying the Dynamic Rendering story

Regards "All tabPanels are displayed when tabs are dynamically loaded and a tab is selected"
it seems that every not-yet-displayed tab gets displayed when you click a new tab. This makes me think that when a tab is dynamically added, some field isn't set right to make it hidden when configureTabs() fires. Looking into it.
I solved "Tab panel appears above the tabs" by adding an empty span inside the tab context. Bit of a botch but it is a solution without a breaking change.


Support rerun test builds

related: https://github.com/adoptium/aqa-tests/issues/3431 and https://github.com/adoptium/aqa-tests/pull/4030
Signed-off-by: lanxia<EMAIL_ADDRESS>

The issue that is reported by GitHub Code Scanning / CodeQL is not introduced by this PR. This PR just uses the API that is in question. More restricted API queries can be implemented for future enhancement.


feat(control_data_collecting_tool): smoothing and bug fix

Description
Update the content of #33. Summary of changes are:

Smoothed the lateral acceleration constraint
bug fix: small modification abaebea

Related links

Tests performed
Tested on an actual vehicle.
see: how to test
Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

see #79 why this PR is closed without merge.


Ball-and-stick morphology not plotting correctly

I have the following swc morphology:
1 1 0.0 -10.0 0.0 20.0 -1
2 1 0.0 0.0 0.0 20.0 1
3 1 0.0 10.0 0.0 20.0 2
4 3 0.0 10.0 0.0 2.0 3
5 3 0.0 100.0 0.0 2.0 4
When I plot this with:
import matplotlib.pyplot as plt
import neurom
import neurom.viewer
neurom.viewer.draw(neurom.load_neuron('ballandstick.swc'))
plt.show()
I get:

It seems the diameter of the soma is not plotted correctly ? (Or is there maybe an option to enable plotting the diameter of the soma ?)

ballandstick.swc.zip

It seems that the default soma plotting draws an 'outline' of the three points, so that's why you get the line.
You can try neurom.viewer.draw(neurom.load_neuron('ballandstick.swc', outline=False))
Which will draw a circle, but it will look wrong since the radius is calculated as the average distance to the center ((10 + (-10))/2, 0, 0), which won't be 10, since one of the coordinates is (0, 0, 0) itself.
How the points of the soma are interpreted has been discussed before, and it looks like there needs to be figured out as per #530.

Regardless of the above, the documentation needs to be cleared up so it's easier to find these properties.

You there is something very wrong with the soma plotting, it seems the radius of the soma is completely ignored ? (I hope this is only the case for plotting, and not the internal representation of the soma ?)
I corrected the position of the soma by moving the 0,0,0 position to the first line:
1 1 0.0 0.0 0.0 10.0 -1
2 1 0.0 -10.0 0.0 10.0 1
3 1 0.0 10.0 0.0 10.0 2
4 3 0.0 10.0 0.0 2.0 3
5 3 0.0 100.0 0.0 2.0 4

(As i mentioned the radius is completely ignored though)
I think the cleanest way out of this would be to add an option to allow the soma to be plotted like any other section, as a set of cylinders.
And for plotting the sphere, I would just calculate the total surface area of the cylinders and put calculate a radius that generates a sphere with the same surface area.

@wvangeit unfortunately you are right Werner. The radius of the soma is completely ignored in principle. The actual radius is computed assuming that the points of the soma belong to the contour. This computation gives a radius that is used for the plotting of the sphere.

@lidakanari But is this only used for plotting, or also in the internal representation in NeuroM (to calculate total surface areas etc)

This is independent of the plotting. It is used in the internal representation of the soma.

After discussion and investigation w/ @wvangeit, it seems like the best course of action is:

Assume that SWC, when multiple points are specified, is a series of connected cylinders, with point locations at each x, y, z, and with radius defined by r
If there is a single point, assume it's a sphere with radius r, at x, y, z
If the point count is 3, see if it follows the pattern in http://neuromorpho.org/SomaFormat.html, and then interpret as 2 stacked cylinders with the point between them as 0, 0, 0
Raise a message if the parent ordering ordering isn't linear (NEURON drops these points, so we want to make people aware of that.

Finally, the area of the soma will be calculated as the area of the cylinders (without the end caps), and radius will be that of the equivalent sphere with that area.

We have made this change back and forth a few times in the past. I am a bit skeptical about this proposal...
Is it compatible with the computations here: "An equivalent radius (rs) is computed as the average distance of each point of the single contour from this center. " from the http://neuromorpho.org/SomaFormat.html ? I've discussed this issue with Giorgio and Rembrandt and they both proposed the computation to be performed as described here as opposed to the cylinders approach. Of course this depends on the file format issues that are not yet resolved as @mgeplf pointed out.
I am afraid that this change might not be compatible with the contour definition in the ascii files from the lab. In most cases the diameters are set to zero or a random value. When the data are converted (into csv or h5) the diameters values that are propagated in the new formats do not correspond to the actual diameters of the soma.

But isn't that just a format problem ? I agree that you can throw away the radii in the ASC format (and the SomaFormat quote also refers to how to get the radius from the ASC file, not from SWC).
But for SWC other tools like Neuron do take the SWC radii into account.

Yes, it is obviously a file format problem. I have a proposal:
Case1: If the soma is a contour I would avoid changing the computation to assume cylinders according to the contour.
Case2: As Mike says (I think it is already like this)
Case3: If the soma is a three point ensure it is of the expected format:
According to the definition from the same link, the swc "correct" format should be:
1 xs ys zs rs -1
2 1 xs (ys-rs) zs rs 1
3 1 xs (ys+rs) zs rs 1
So in order to avoid confusion I would propose to check if:

P1.r == P2.r == P3.r
P2.y - P1.y == - P1.r
P3.y - P1.y == P1.r
If those conditions are met then we can safely say that the swc format is the expected one and therefore we can proceed to treat the soma as "cylinders".

But so just to be sure: Case1 doesn't exist in SWC.
And yes, for the rest, it's basically what Brian is doing: https://github.com/brian-team/brian2/blob/master/brian2/spatialneuron/morphology.py#L1040

It does in practice... However the conversion into this case is obscure and depends on the converter.

I agree w/ @wvangeit, except I would treat the 3 point case as the 'normal' case, unless it follows the 'neuromorpho' style, at which point it would be loaded neuromorpho style.

Sure, about case 3 I think we all agree.
I don't understand the disagreement... If case1 doesn't exist why should we change how we currently treat it?

Currently the SWC reader is treating the points as a contour, but in NEURON, it's treated as any other compartment style set of cylinders.  The idea would be to switch over to that representation, since the standard (http://www.neuronland.org/NLMorphologyConverter/MorphologyFormats/SWC/Spec.html), doesn't explicitly say they should be contours, and thus implicitly suggests they should be interpreted the same as the neurites (ie: cylinders.)

So, to recap the discussion on Friday:

SWC doesn't do contours by default, although some software may output (we should track down these packages, and try and not use them - may need an option for the loader to specify these special cases)
When contours are needed, use the h5 format, now that @lidakanari has access to a script that does the   neurolucida conver to h5 more 'correctly'
In the future, we may need to parse SWC+, but not now.

Does that sound right?

@mgeplf Agreed! I'd like to add that if anyone needs to produce SWC with a soma in contour format it will be better to produce an SWC+ following the conventions for the "contour soma" type. This way we can avoid misunderstandings and interpret the data correctly.


[BUG] 71.0.0 fails with 'importlib_metadata' has no attribute 'EntryPoints' on Python 3.8

setuptools version
71.0.0
Python version
3.8
OS
Ubuntu 20.04
Additional environment information
No response
Description
This started a few hours ago when our pipeline picked up 71.0.0, works fine on 70.3.0.
python3 -m pip install --upgrade --no-cache-dir --no-color --no-input --upgrade pip packaging setuptools python-debian
Requirement already satisfied: pip in /usr/local/lib/python3.8/dist-packages (24.1.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (24.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (71.0.0)

python3 -m pip install --no-cache-dir --no-color --no-input https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz --upgrade
Collecting https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz
  Downloading https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz (549 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 549.6/549.6 kB 690.9 kB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [10 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 14, in <module>
        File "/usr/local/lib/python3.8/dist-packages/setuptools/__init__.py", line 19, in <module>
          from .dist import Distribution
        File "/usr/local/lib/python3.8/dist-packages/setuptools/dist.py", line 30, in <module>
          from . import _entry_points
        File "/usr/local/lib/python3.8/dist-packages/setuptools/_entry_points.py", line 44, in <module>
          def validate(eps: metadata.EntryPoints):
      AttributeError: module 'importlib_metadata' has no attribute 'EntryPoints'
      [end of output]

Expected behavior
No error
How to Reproduce
See Description
Output
See Description

This behavior is by design. Setuptools now will prefer installed packages over its vendored ones, but it can't yet declare them. That means that if you have an installed dependency but it's older than the one setuptools needs, it will not work.
The workaround/fix is to install these dependencies when installing setuptools. You may be tempted to install setuptools[core], but please don't as that's unsupported and will likely break in the future.
Eventually, we plan to have setuptools declare (and thus install) its own dependencies so you don't have to do that, but for Setuptools to do that, we first have to solve https://github.com/pypa/packaging-problems/issues/342 (an ongoing effort).

Thanks, importing those, specifically importlib_metadata fixed the problem.


Add dev dependencies

go.mod is all we require to produce reproducible builds. By default go build will update mod file if there are any missing deps for any OS, architecture.
when we do go get -u this will update the dependency and its transitive dependency as well. Since we did go get -u earlier, this was an indicator for modules to update its transitive dependencies and there was a recent commit to the deps you have mentioned. Hence, the go.mod file changed.
always prefer to use just go get so that we don't implicitly update its transitive dependencies. More can be read here - https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies
go.sum:
go.sum is not a lock file. It contains cryptographic hashes of the dependencies and can contain deps that we no longer use as well. This is just another protection to cross-check the deps hash with the one in your local mod cache. This should be committed as well. We can verify that using go mod verify
This PR also locks the versions of the tools we need but will not be included in the final build since it is behind a build flag

@mikiquantum merging this. Let me know if there are any concerns here :-D


Add sso-start-url information to profile name process input

I've got multiple aws organizations and identity centers, and to make matters worse the same account names in them all.
Would it be possible to add information about sso url to the profile name process? that way i could customize the naming and take sso url into consideration

You can use the components option to customize the profile name format.
There is no dynamic component for instance-level attributes, but you can use a literal to name each instance.
I work with Identity Center instances for OrgA and OrgB.
OrgA and OrgB each have two accounts called Name1 and Name2. I can assume 1 role in each account.
Today I handle it by using a different command for each instance to populate the profiles.
aws-sso-util configure populate \
--sso-start-url "..." \
--sso-region "..." \
--region "..." \
--components "OrgA,account_name,account_id,role_name"

aws-sso-util configure populate \
--sso-start-url "..." \
--sso-region "..." \
--region "..." \
--components "OrgB,account_name,account_id,role_name"

OrgA/OrgB: literal to name the Identity Center instance
account_name, account_id, role_name: dynamic components

That generates a list of profiles like this:
OrgA.Name1.111111111111.Role1
OrgA.Name2.222222222222.Role1
OrgB.Name1.333333333333.Role1
OrgB.Name2.444444444444.Role1

You can distinguish the two "Name1" accounts by the different prefixes "OrgA" and "OrgB".
I like to include the account ID in the profile name for extra clarity.
Such a long name may be awkward to type. To avoid that I run the aws configure list-profiles command, copy the profile I need from the output, and paste it where I need it.

Sorry for the very long delay (and it will be a while longer as I get spun back up on my projects). This is a good idea and I will add it. However, I believe you can accomplish this today by adding the start URL to the command you're giving to --profile-name-process, e.g.
aws-sso-util configure populate --sso-start-url $AWS_SSO_START_URL ... --profile-name-process "my_profile_process $AWS_SSO_START_URL"

Note to self: add a note to the docs that arguments may get added over time (but will never get reduced or rearranged), so process formatters should not rely on a specific number of total arguments, i.e., unpack sys.argv[:7] rather than sys.argv


[BUG] gateway/fs migration docs

Describe the bug
Some misleading steps in the docs needs to be clarified/removed
https://min.io/docs/minio/linux/operations/install-deploy-manage/migrate-fs-gateway.html and https://blog.min.io/minio-gateway-migration/

Export/Import the existing deployment’s configurations with mc admin config export/import only to fs. For gateway all config are in env. vars. MINIO_CACHE_* and MINIO_GATEWAY_SSE env are no longer relevant in the newer cluster - all other env.var can be brought in as is
Export/Import bucket metadata and iam assets applies only to fs mode
Tiering not available in gateway mode nor in FS mode. Tiering is only available in SNSD, SNMD and MNMD

To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Based on initial walkthrough:

gateway is envvar only and most should be 1:1 transfer.
We can just drop the tiering section

Maybe tabs to deal with FS vs Gateway? Or make them two procedures on the same page with shared steps.


Standardize wiki page titles

The project’s official and legal name is “OpenHistoricalMap”, but the wiki keeps calling it “Open Historical Map”, at least in article titles, based on an old logo that’s no longer used on English-language articles. There seems to be a quiet consensus to rename the wiki articles to start with “OpenHistoricalMap” instead.

I plan to move the pages so en masse using my administrator privileges. I’ll also need to fix up the documentation wiki pages (P31) statements in each data item to match the new spelling.
One wrinkle is that iD assumes that tagging article titles are prefixed with “Open Historical Map” as of OpenHistoricalMap/iD#178. The code is rather painful to rewrite to handle multiple spellings, so I’ve pushed OpenHistoricalMap/iD#180 with a small change to expect the new spelling. We’ll need to coordinate the deployment of this change with the actual page move.

This is done.


uncaught exception in Haskell main thread: TypeError: Cannot read property 'd1' of null

Seems similar to https://github.com/ghcjs/ghcjs/issues/477
Description
Clean ./install.sh  on
gnezdo@codeworld:~/codeworld$ uname -a
Linux codeworld 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u3 (2019-06-16) x86_64 GNU/Linux
gnezdo@codeworld:~/codeworld$ cat /etc/issue
Debian GNU/Linux 9 \n \l

at this commit goes for a long time and eventually stops with:
cabal: Entering directory '/tmp/cabal-tmp-11587/distributive-0.6.1'
[1 of 1] Compiling Main             ( /tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup.hs, /tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/Main.js_o )
Linking /tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup.jsexe (Main)
uncaught exception in Haskell main thread: TypeError: Cannot read property 'd1' of null
TypeError: Cannot read property 'd1' of null
    at h$$tk5 (/tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:836547:13)
    at h$runThreadSlice (/tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:6787:11)
    at h$runThreadSliceCatch (/tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:6757:12)
    at h$mainLoop (/tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:6752:9)
    at /tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:7238:13
    at h$handleErrnoC (/tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:4758:9)
    at /tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup:21092:13
    at FSReqWrap.oncomplete (fs.js:135:15)
Linking /tmp/cabal-tmp-11587/distributive-0.6.1/dist/setup/setup.jsexe (Main)
Run the 'configure' command first.
cabal: Leaving directory '/tmp/cabal-tmp-11587/distributive-0.6.1'

Tmux-worth of log in a gist

@luite
In case this is a GHCJS change (which seems likely), then I can tell you the production server is using https://github.com/ghcjs/ghcjs/commit/eeeb0cde48e093e278fc1a4f418b48a2d23aa08c  There were rather a lot of changes, including bumping GHC versions, on June 15 shortly afterward, and that might be where the regression crept in.  If so, then CodeWorld has been uninstallable all summer, and that would be unfortunate. :(

#1186 got this building again, but I'll leave the issue open to cover tracking down the root cause and updating to the latest GHCJS again.


Fix css class conflicts

On mounted apps that use the DfE autocomplete the GovUK all replaces any customization of the GovUK base which is not the desired output.
Importing only what's needed can decrease the level of friction with any apps that use the library.
Also wrapped the CSS under the suggestions class so it also doesn't cause conflicts with anything on another app. The name 'suggestions' will be changed in the future.

I suppose it's still possible that we could import something in govuk/base which affects an override? Could we get away without the dependency on govuk-frontend at all?

I suppose it's still possible that we could import something in govuk/base which affects an override?
Yes, it is still possible but not likely as the govuk/all. I tested locally and I couldn't catch any conflict yet.

Could we get away without the dependency on govuk-frontend at all?
Maybe we could but I don't know yet. Maybe someone can help if we're in hurry? @frankieroberto do you know anything about this?

The autocomplete component doesn’t follow or depend upon govuk-frontend at all (as it predates it and hasn't been updated) so we need to include the govuk-frontend styles somewhere in order to get the fonts to match.
It's all a bit of a mess... 😬  I'll have a think about how we might do this in the least painful way (and I'll ask @edwardhorsford too).


fix: remove default value for background_image in button-icon mixin

if we wish to allow a default value of none it should not use the url()
see: https://stackoverflow.com/questions/1161061/css-background-image-urlnone-leading-to-errors-in-apache-log-file

Discussed with @viktoria-mahmud and we are both happy for this to go in.
cc @discodavey (tagged as it is an adjustment to something you recently introduced, happy to discuss when you get back)


Macos error

ERROR: ConnectBlock(): Incorrect AAL transactions or hashes (hashStateRoot, hashUTXORoot)
2018-07-01 11:27:34 Misbehaving: <IP_ADDRESS>:3888 peer=0 (0 -> 100) BAN THRESHOLD EXCEEDED
2018-07-01 11:27:34 InvalidChainFound: invalid block=0000d5dab5e76310ae640e9bcfa270c2eb23a1e5948bdf01fc7ed1f157110ab7  height=1  log2_work=17.000022  date=2017-09-08 05:43:21
2018-07-01 11:27:34 InvalidChainFound:  current best=000075aef83cf2853580f8ae8ce6f8c3096cfa21d98334d6e3f95e5582ed986c  height=0  log2_work=16.000022  date=2017-09-06 10:50:29
2018-07-01 11:27:34 ERROR: ConnectTip(): ConnectBlock 0000d5dab5e76310ae640e9bcfa270c2eb23a1e5948bdf01fc7ed1f157110ab7 failed
2018-07-01 11:27:34 InvalidChainFound: invalid block=0000d5dab5e76310ae640e9bcfa270c2eb23a1e5948bdf01fc7ed1f157110ab7  height=1  log2_work=17.000022  date=2017-09-08 05:43:21
2018-07-01 11:27:34 InvalidChainFound:  current best=000075aef83cf2853580f8ae8ce6f8c3096cfa21d98334d6e3f95e5582ed986c  height=0  log2_work=16.000022  date=2017-09-06 10:50:29
2018-07-01 11:27:34 ProcessNetBlock: ACCEPTED

I am having the same error on OSX.
Other things that I noticed:

If I run my wallet offline and try to generate e block manually I get: RootNotFound;
If I copy my Qtum folder from a wallet with blocks synced  it works well;
If I copy my Qtum folder from a wallet without blocks yet it also works well;

It seems to be something with the state itself, maybe some issue with the BerkeleyDB.
Debugging validation, the hash is ok until the globalState is updated.
We'll keep looking on this.


Copy paste indent

Type: Feature Request
Copy pasting code breaks the indentation. Every time I have to copy and paste a block, I'm spending precious flow-breaking moments reformatting the pasted code, because VSCode cannot retain the correct indentation. Take a look at how Sublime does it. I can focus on work, not hitting tab and backspace a bunch of times just because the editor cannot do it.
VS Code version: Code 1.91.1 (f1e16e1e6214d7c44d078b1f0607b2388f29d729, 2024-07-09T22:06:49.809Z)
OS version: Windows_NT x64 10.0.19045
Modes:

what language?
have a look the editor.autoIndent setting

Thanks for the reply, RedCMD.
Language HTML/PHP.
I tried all the editor.autoIndent settings. Nothing was better.

can you share a code snippet of a before and after please
and you settings file

Here are a couple of examples:
And I copy lines by dragging up with mouse, which means, it is taking newline from the previous line.
1.
Code initially...
			<button type="button" onclick="document.getElementById('levelup-ikkuna').style.display='none';">Peruuta</button>
			<input type="submit" value="Hyväksy" disabled>
		</div>

Copy this...
<input type="submit" id="hyvaksy_nappi" value="Hyväksy" disabled>
Replace the input line, resulting...
			<button type="button" onclick="document.getElementById('levelup-ikkuna').style.display='none';">Peruuta</button>
			<input type="submit" id="hyvaksy_nappi" value="Hyväksy" disabled>
			</div>

So the closing div is now wrong.

Code initially...
			</form>
		</div>
<?php
get_footer();
?>

Copy this...
	</main><!-- #main -->
</div><!-- #ensisijainen -->

Paste it in the middle, resulting...
			</form>
		</div>
		</main><!-- #main -->
</div><!-- #ensisijainen -->
<?php
get_footer();
?>

So, this time the closing main is in the wrong location. Or if you're expecting one indentation based on previous lines, then the closing div is in the wrong location.
Settings file:
settings.json


Automated Tests for Newsfeed

Note: This issue is light on details because the Newsfeed is a new feature under active development - we'll likely have more specifics as we get closer to launch-ready on the Newsfeed.
Success Criteria

[ ] Automated test that verifies the behavior / rendering of the Newsfeed page

Notes
The exact details here are still a little TBD as this is a new feature under active development - automated tests around the feature should be considered a necessary element to going live with the Newsfeed, but there's room for the devs launching the Newsfeed to define what success looks like here (e.g. does this need to be an E2E test, or is an integration/unit test sufficient? What needs to be tested ).

This doc has a writeup of the manual test plan for follows/notifications that we can adapt automated testing - https://docs.google.com/spreadsheets/d/1sntEHafJLqKJPSu6t4swlHPaXlw9Om4SsTuhVgeY5Sg/edit?gid=1013620796#gid=1013620796


Workaround for SD write when SD card is ejected

When the SD card has been ejected, a write() to the card never returns.
This only happens if CONFIG_SDIOWAIT_WRCOMPLETE is set.
It turns out that in the blocking case waitevents and wkupevent are both 0, hence we get stuck later waiting for a semaphore.
The "fix" is to check waitevents and wkupevent for 0 and return with an error for this case.
The other changes are drive-by changes but did no seem to make a difference.
This seems to fix:
https://github.com/PX4/Firmware/issues/12071
https://github.com/PX4/Firmware/issues/13087
@davids5 it looks like the regression/problem appeared with:
https://github.com/PX4/NuttX/commit/1ace391fcffcceb4f8c3672204383224063c551e

It doesn't seems to fix PX4/Firmware#13087.
Removing the SD card still lead to mavlink to halt (when statfs() is called on outgoing mavlink message)

@BazookaJoe1900 ok, thanks for testing. This means my assumption that it's the exact same issue might have been wrong!


KYLIN-4283: return SUCCEED while IOException happend in GarbageCollec…

…tionStep
see: https://issues.apache.org/jira/browse/KYLIN-4283

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Codecov Report

Merging #977 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #977      +/-   ##
============================================
+ Coverage     25.43%   25.43%   +<.01%
- Complexity     6153     6155       +2
============================================
  Files          1415     1415
  Lines         85878    85877       -1
  Branches      12042    12042
============================================
+ Hits          21840    21845       +5
+ Misses        61907    61904       -3
+ Partials       2131     2128       -3

Impacted Files
Coverage Δ
Complexity Δ

...age/hbase/steps/HDFSPathGarbageCollectionStep.java
0% <ø> (ø)
0 <0> (ø)
:arrow_down:

...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java
69.3% <0%> (-0.31%)
7% <0%> (ø)

...a/org/apache/kylin/dict/Number2BytesConverter.java
82.53% <0%> (+0.79%)
18% <0%> (+1%)
:arrow_up:

...ream/core/storage/columnar/ColumnarStoreCache.java
55.68% <0%> (+3.4%)
8% <0%> (ø)
:arrow_down:

...he/kylin/dict/lookup/cache/RocksDBLookupTable.java
78.37% <0%> (+5.4%)
7% <0%> (+1%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2bea28...65d2202. Read the comment docs.

Pull Request Test Coverage Report for Build 5336

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.003%) to 27.919%

Files with Coverage Reduction
New Missed Lines
%

core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java
5
77.81%

Totals

Change from base Build 5335:
0.003%

Covered Lines:
23976

Relevant Lines:
85877

💛  - Coveralls

I agree; +1


LHN - LHN displays incorrect last action when editing text + attachment

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.4-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4695552
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause - Internal Team
Action Performed:

Open New Expensify app
Navigate to any conversation
Write something in the compose box
Send an attachment to a conversation while there is text in compose box
Edit a text message in a sent message
Note the text of the last action in LHN

Expected Result:
After editing a message with an attachment, LHN should show text + attachment or text and "..." if the text is long
Actual Result:
When sending text + attachment to a conversation, LHN shows text + attachment as last action, after editing the message in LHN only the attachment is shown
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/f59fe463-5d21-4d3b-85fa-e8732f3dd15d
View all open jobs on GitHub
Issue OwnerCurrent Issue Owner: @Pujan92

We think that this bug might be related to #vip-vsp

@JmillsExpensify  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Agree with the project. I also added the external label.

Proposal
Please re-state the problem that we are trying to solve in this issue.
Displays incorrect last action when editing text + attachment
What is the root cause of that problem?
This issue originates from the backend. We sent optimistic data with the updated lastMessageText, but when callingUpdateComment, it has not updated yet.

What changes do you think we should make in order to solve the problem?
Should double check UpdateComment API
What alternative solutions did you explore? (Optional)

Thanks @huult, @JmillsExpensify It seems to be a backend issue as updateComment should return the correct lastMessageText. Can we add an internal engineer here?

@JmillsExpensify We need to assign an internal engineer here.

Issue not reproducible during KI retests. (First week)

Issue not reproducible during KI retests. (Second week)

@JmillsExpensify This seems resolved in the backend, We can close this issue.

Issue not reproducible during KI retests. (Third week)

Thanks, closing!


Kai-tso suisiann.ithuan.tw

CI chhut-hiān
ModuleNotFoundError: Module not found: Error: Can't resolve 'ten1su4' in '/home/travis/build/i3thuan5/tau3_phah4-ji7/src'

In-ūi
npm WARN notsup Unsupported engine for<EMAIL_ADDRESS>wanted: {"node":"^6.2.2"} (current: {"node":"14.13.0","npm":"6.14.8"})

In-ūi https://github.com/i3thuan5/TenSu/blob/master/package.json#L93


Support for byte arrays

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Set proxy_timeout to 10m in nginx.conf

Fixes #655.
This is a teporary solution for long-polling idle connections to
apiserver. It will make Nginx not cut them for the duration of expected
timeout. It will also make Nginx extremely slow in realizing that there
is some issue with connectivity to apiserver as well, so it might not be
perfect permanent solution.

This also fixes the faily "Master | wait for the apiserver to be running" task


Allow creating releases manually

This patch is a quick workaround for the problem that the automatically cut release tags do not trigger the GitHub Actions workflow to create a release on GitHub. This patch allows committers to trigger the workflow manually.
Your pull request should…

[x] have a concise title
[x] close an accompanying issue if one exists
[x] be against the correct branch
[x] include migration scripts and documentation, if appropriate
[x] pass automated tests
[x] have a clean commit history
[x] have proper commit messages (title and body) for all commits

Invoking the „being pragmatic“ rule since I need this for cutting the new release in an hour


Pull request event fix

Fixed the touch event's on mobile devices. Your library works for Desktop emulation.
Updated the PR correctly.
Tested the code on Google Nexus 6p and working perfect with the fix.
Great work keep it up!
Thanks
Nirus

Thx for your help !


allow sending discriminator with Discord username

Add a new flag to the Discord account UseDiscriminator which will append the user's "#xxxx" discriminator to the username while UseUserName is also enabled.

Thanks for the PR!
Did you test this? As it's not specified in config (https://github.com/42wim/matterbridge/blob/master/bridge/config/config.go#L75) it'll always return false.
Could you also add documentation about this setting to the matterbridge.toml.sample https://github.com/42wim/matterbridge/blob/master/matterbridge.toml.sample#L708

Sorry, I've tested it both ways now and it's working

Looks good, thanks!


bicep decompile only works with .json extensions

Bicep version
Bicep CLI version 0.3.1 (d0f5c9b164)
Describe the bug
The decompile command ignores any extensions other than ".json". For example I tried to convert "postgresql.arm". It dropped the extension and attempted to decompile "postgresql.json" which doesn't exist.
We'll be able to work around it as we're just conforming to existing naming convention however it does seem a little unintuitive. Very minor so happy for it be marked as "won't fix" if it's too much hassle.
To Reproduce

Export an arm template and call it postgresql.arm
Run az bicep decompile --files postgresql.arm
Expect it to decompile but instead get "Failed to read file:///E:/repos/bicep/postgresql.json"

Additional context
PS E:\repos\bicep> az bicep decompile --files postgresql.arm
WARNING: Decompilation is a best-effort process, as there is no guaranteed mapping from ARM JSON to Bicep.
You may need to fix warnings and errors in the generated bicep file(s), or decompilation may fail entirely if anIf you would like to report any issues or inaccurate conversions, please see https://github.com/Azure/bicep/issues.
E:\repos\bicep\postgresql.arm: Decompilation failed with fatal error "Failed to read file:///E:/repos/bicep/postgresql.json"

Good catch, we should definitely be able to either relax or give this a better error message.
The only thing we need to block is trying to decompile .bicep -> .bicep.

We should attempt to parse the file as JSON. If that fails, we can return the message.


SFDX docker image tag change

This updates the image tag to the new format as per this stack exchange answer and verified on docker hub
Stack Exchange: https://salesforce.stackexchange.com/questions/354209/manifest-for-salesforce-salesforcedxlatest-not-found-manifest-unknown-manifes

Issue reported here: https://github.com/microsoft/vscode-dev-containers/issues/1045

LGTM!


Importing screenshot with space in the name are not url decoded?

New Issue Checklist

[x] Updated fastlane to the latest version
[ ] I have read the Contribution Guidelines

Issue Description
I was initializing fastlane on an existing project. It downloaded all the meta and screenshots fine as I would expect. However when I try to run the deliver to push it to appstore, I found some the screenshots were missing in the generated Preview.html file.
Then I renamed my screenshots %20 with Space and it seems to work.
###Before

###After

Complete output when running fastlane, including the stack trace and command used
deliver
[21:15:14]: Login to iTunes Connect<EMAIL_ADDRESS>[21:15:18]: Login successful

+--------------------+-------------------------------------+
|                  deliver 1.10.3 Summary                  |
+--------------------+-------------------------------------+
| username           |<EMAIL_ADDRESS>            |
| app_identifier     | com.xxxxx.xxxxx |
| ipa                | CN.ipa                              |
| metadata_path      | ./fastlane/metadata                 |
| screenshots_path   | ./fastlane/screenshots              |
| skip_binary_upload | false                               |
| skip_screenshots   | false                               |
| app_version        | 2.2.2                               |
| skip_metadata      | false                               |
| force              | false                               |
| submit_for_review  | false                               |
| automatic_release  | false                               |
+--------------------+-------------------------------------+

[21:15:19]: Making sure the latest version on iTunes Connect matches '2.2.2' from the ipa file...
[21:15:20]: '2.2.2' is the latest version on iTunes Connect
[21:15:20]: Loading './fastlane/metadata/en-US/description.txt'...
[21:15:20]: Loading './fastlane/metadata/en-US/keywords.txt'...
[21:15:20]: Loading './fastlane/metadata/en-US/release_notes.txt'...
[21:15:20]: Loading './fastlane/metadata/en-US/support_url.txt'...
[21:15:20]: Loading './fastlane/metadata/en-US/marketing_url.txt'...
[21:15:20]: Loading './fastlane/metadata/en-US/name.txt'...
[21:15:20]: Loading './fastlane/metadata/en-US/privacy_url.txt'...
[21:15:20]: Loading './fastlane/metadata/copyright.txt'...
[21:15:20]: Loading './fastlane/metadata/primary_category.txt'...
[21:15:20]: Loading './fastlane/metadata/secondary_category.txt'...
[21:15:20]: Loading './fastlane/metadata/primary_first_sub_category.txt'...
[21:15:20]: Loading './fastlane/metadata/primary_second_sub_category.txt'...
[21:15:20]: Loading './fastlane/metadata/secondary_first_sub_category.txt'...
[21:15:20]: Loading './fastlane/metadata/secondary_second_sub_category.txt'...
[21:15:20]: Verifying the upload via the HTML file can be disabled by either adding
[21:15:20]: `force true` to your Deliverfile or using `deliver --force`
Does the Preview on path './Preview.html' look okay for you? (y/n)
n

[!] Did not upload the metadata, because the HTML file was rejected by the user

Configuration Files
Please copy the complete content of your Fastfile and any other configuration files you use below:
Fastfile:
# INSERT CONTENT HERE

Environment
fastlane version (fastlane -v): fastlane 1.63.1
Do you use bundler, rbenv or rvm? NO

@palaniraja Is this still an issue using latest versions?

Yes. I still see the %20 in latest (fastlane 1.81.0) too.

@palaniraja Thanks for following up. We are going to look this over as a team. Hope that you have a great day! 👍

I have reproduced this problem locally for file names that contain %20

I've also verified that deliver is not responsible adding %20 to the file names.

Hi @palaniraja - we've merged a fix for this in #4196, which will be in the next release! :rocket:

Hi @palaniraja - this shipped as part of deliver 1.11.1! 🚚 ✨

Thanks for the heads up. Sorry I didn't get a chance to check this week. I'll update if I see any issue.
Thank you all for the fix.


Fix gateway-cli error handling and auto-completion

While working on #3880 I've noticed that errors returned from the handler functions in the gateway-cli are not processed properly, resulting in a generic error message ("Error: Bad status returned 500 Internal Server Error"). These should be handled similar to how fedimint-cli does it. Also, the auto-completion for gateway-cli subcommands does not seem to work.

Error-handling is a little more of a server-side thing. That gateway API usually doesn't provide useful error messages.

Yeah, you're right, I thought that the server is actually returning useful errors messages and the gateway-cli is just not passing them through. Still, once we get the server to the return useful errors messages, the gateway should also pass them on and not just return the status code.


cmd/govim: use popup windows instead of balloons for hover information

Raised for discussion at https://groups.google.com/forum/#!topic/vim_dev/-LDoaq3wAl8
It might be possible to do this without changes in Vim (using the balloon callback, responding with nothing, but showing a popup)... but will wait for responses in that thread in any event.
Note as part of this we can also drop the current suggestion on triggering hover information via a mapping:
https://github.com/myitcv/govim/wiki/vimrc-tips#-tip-triggering-hover-information-via-a-key-mapping
and provide a useful default mapping.
We can and should use enhanced hover information.

As part of using enhanced hover information, we should look to render the markdown returned by gopls using an approach similar to https://groups.google.com/forum/#!msg/vim_dev/Armm3XaIleo/uLyrBSpJAgAJ.


Add support for finetune sdk

Discussion in notion: https://www.notion.so/cohereai/Finetune-SDK-3041fd7bf67344dfa9f54a55f02406fe?pvs=4
This PR aims to add support for finetunes to the sdk. Since we have to use the BlobheartAPI for this, the code looks slightly different than for the other endpoints.
How to use this:
co = cohere.Client("YOUR_API_KEY")
csv_data = CsvDataset(train_file="/path/to/your.csv", delimiter=",")
finetune_id = co.finetune.create(
    name="finetune-name",
    dataset=csv_data,
    finetune_type="GENERATIVE"
)

print(co.finetune.get(finetune_id))

Things that need more work:

naming in general. especially around the finetune_dataset.py file
Not sure what to name the InMemoryDataset class. As is this class only supports labeled data. So you can't use it for generative with text. Should we change the constructor to also accept `Union[Iterable[str], Iterable[tuple[str, str]]). Or should we create more classes with different naming?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Raphael Cristal seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

document a little (at least the existence of the client) in readme
add appropriate things in docs/ for autogenerated docs
version 4.3.0, changelog

I'd like to discuss this more next week, not stoked about the SDK hitting api.os.cohere.ai as opposed to api.cohere.ai. This has all sorts of implications, API versioning being one of them

Closing this PR until we iron out the API stuff


Appium hangs when finder is not working.

During the tests I have found the Appium request stalls when a command is not working. For instance, I would like to add the following code in https://github.com/truongsinh/appium-flutter-driver/blob/master/example/python/example.py.
counter_finder = FlutterFinder().by_semantics_label("counter_semantic")
counter_element = FlutterElement(driver, counter_finder)
print(counter_element.text)

However the code stops at the last line. Here is the log of Appium:
2021-03-20 13:56:43:570 - [HTTP]
2021-03-20 13:56:43:571 - [HTTP] --> GET /wd/hub/session/971ef296-5825-44d1-a6d7-97e9f1c397f2/element/eyJmaW5kZXJUeXBlIjogIkJ5U2VtYW50aWNzTGFiZWwiLCAiaXNSZWdFeHAiOiBmYWxzZSwgImxhYmVsIjogImNvdW50ZXJfc2VtYW50aWMifQ==/text
2021-03-20 13:56:43:572 - [HTTP] {}
2021-03-20 13:56:43:572 - [debug] [MJSONWP (971ef296)] Calling AppiumDriver.getText() with args: ["eyJmaW5kZXJUeXBlIjogIkJ5U2VtYW50aWNzTGFiZWwiLCAiaXNSZWdFeHAiOiBmYWxzZSwgImxhYmVsIjogImNvdW50ZXJfc2VtYW50aWMifQ==","971ef296-5825-44d1-a6d7-97e9f1c397f2"]
2021-03-20 13:56:43:572 - [debug] [FlutterDriver] Executing Flutter driver command 'getText'

then nothing happens forever.
So I have two questions here:

Did I do anything wrong with the code. I got the label counter_semantic from https://github.com/truongsinh/appium-flutter-driver/blob/master/example/flutter_app_under_test/lib/main.dart#L43 but let me know if I did something stupid.
Even if something is wrong with the test code, why is the code hanging instead of returning an error?

Any help will be appreciated.

Hi, @hex0cter! Don't know if it's still an issue, but what we did to work around this issue was to call flutter:waitFor and pass in the finder as a parameter, before we initialized the Flutter element itself. Because it raises an exception if the finder is not found in the given time it worked quite well in our case.
Not a python user so can't give you the exact example, but in ruby, we did something similar to this:
defined a method that checks if a finder exists
  def displayed?(finder, wait)
    @driver.driver.execute_script('flutter:waitFor', finder, wait)
    true
  rescue Selenium::WebDriver::Error::UnknownError
    false
  end

Then created a method that returns the element or raises an exception, similar to this:
  def element(finder, wait)
    raise Selenium::WebDriver::Error::NoSuchElementError, 'Element not found' unless displayed?(finder, wait)

<EMAIL_ADDRESS>finder: finder)
  end

Although the example is in ruby, I think the general approach can be used in python as well.