refactor aggregations

add specialized version for full cardinality
refactor aggregation collection

split collection and flushing

add staging collector

 aggregation::tests::bench::bench_aggregation_average_f64                 10,506,069       9,218,078           -1,287,991  -12.26%   x 1.14
 aggregation::tests::bench::bench_aggregation_average_u64                 9,062,582        9,367,880              305,298    3.37%   x 0.97
 aggregation::tests::bench::bench_aggregation_average_u64_and_f64         13,326,599       13,442,273             115,674    0.87%   x 0.99
 aggregation::tests::bench::bench_aggregation_histogram_only              17,879,982       16,766,669          -1,113,313   -6.23%   x 1.07
 aggregation::tests::bench::bench_aggregation_histogram_only_hard_bounds  16,116,178       15,454,050            -662,128   -4.11%   x 1.04
 aggregation::tests::bench::bench_aggregation_histogram_with_avg          37,076,780       35,573,842          -1,502,938   -4.05%   x 1.04
 aggregation::tests::bench::bench_aggregation_range_only                  12,534,568       13,489,130             954,562    7.62%   x 0.93
 aggregation::tests::bench::bench_aggregation_stats_f64                   9,463,641        9,657,604              193,963    2.05%   x 0.98
 aggregation::tests::bench::bench_aggregation_terms_few                   14,573,134       13,908,208            -664,926   -4.56%   x 1.05
 aggregation::tests::bench::bench_aggregation_terms_many2                 35,757,387       30,942,369          -4,815,018  -13.47%   x 1.16
 aggregation::tests::bench::bench_aggregation_terms_many_order_by_term    39,661,657       32,838,242          -6,823,415  -17.20%   x 1.21
 aggregation::tests::bench::bench_aggregation_terms_many_with_sub_agg     102,926,064      100,122,452         -2,803,612   -2.72%   x 1.03

Codecov Report

Merging #1875 (6deb590) into main (71f43ac) will decrease coverage by 0.04%.
The diff coverage is 86.80%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main    #1875      +/-   ##
==========================================
- Coverage   94.68%   94.64%   -0.04%
==========================================
  Files         297      298       +1
  Lines       54493    54621     +128
==========================================
+ Hits        51594    51697     +103
- Misses       2899     2924      +25

Impacted Files
Coverage Δ

src/aggregation/staging_collector.rs
68.00% <68.00%> (ø)

src/aggregation/metric/stats.rs
93.52% <85.00%> (-2.23%)
:arrow_down:

src/aggregation/segment_agg_result.rs
92.47% <92.79%> (+1.18%)
:arrow_up:

src/aggregation/bucket/histogram/histogram.rs
99.55% <100.00%> (-0.01%)
:arrow_down:

src/aggregation/bucket/range.rs
97.24% <100.00%> (-0.03%)
:arrow_down:

src/aggregation/bucket/term_agg.rs
98.17% <100.00%> (-0.01%)
:arrow_down:

src/aggregation/collector.rs
96.66% <100.00%> (-0.04%)
:arrow_down:

src/aggregation/mod.rs
98.21% <100.00%> (ø)

src/query/boolean_query/block_wand.rs
96.85% <0.00%> (-0.21%)
:arrow_down:

... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


[Crash] on dimension change

On my server, my friend and I have the same client mod set, but our configs may differ slightly.
My friend was getting crashes whenever they changed  dimensions, both when dying in the nether and respawning in the overworld, and when using a nether portal.
I have no issues.
Here are logs:
nether-death-latest.log
nether-portal-latest.log
Here's the complete contents of my mods folder (sorry but I've updated some mods since then so it doesn't match the set from the logs exactly)

Fixed in v3.0.0


Support new status 'noIndex'

Kite will soon support a new file status noIndex. It is returned by /clientapi/status for files which live outside of the indexed directories (i.e. usually $HOME).
Needed changes:

Display Kite: ready (unindexed) for file status noIndex
Drop support of the ignored state (and syncing, if still in use)

Done in #195.


Cannot Create Azure AD Application

Hi,
im trying to create a azure add apllication using "cbd azure configure-arm"
but im getting following error:
waiting...
Your application id: d6ac9626-4693-4256-b11b-e79dab052511
waiting...

Any Idea ?
Can you provide a manual way to deploy the app ?
Kind Regards
Thomas

Hi @ThomTel
The cbd tool use a docker container that is build from this github repo. You could follow the 'cli_tools' script related part (This part do the magic part by using the Azure CLI), of course you could use this script without docker, but the Azure cli needs to be installed.
Br,
Tamas


ExternalIP for every chi-service

Hello
Help me please. I can't deploy chi-services with external IP.
How can I do it?
Thanks

yea

example

Can you, please, elaborate what are exact issues you are struggling with?
Provided .yaml is expected to produce one CHI-level LoadBalancer service. However, I do not have OpenStack available and can't check exact OpenStack's reaction.

Hello @sunsingerus
Today I have one service CHI LoabBalancer and 6 CHI ClusterIP (ClusterIP: None) services for every clickhouse shard. How can I get cluster IP-address for every CHI ClusterIP service?

ClusterIP does not have external IP, intenal only, it is its main point. If you need real external IP you have to use LoadBalancer only. More info is available in kube docs
serviceTemplatescan be used to customize what Service object would be associated with cluster/shard/replica, like the following (IMPORTATNT - this is NOT the full spec, check multiple examples available and setup your needed configuration):
spec:
  configuration:
    clusters:
      - name: customized
        templates:
          clusterServiceTemplate: cluster-service-template
        layout:
          shards:
            - name: shard2
              templates:
                shardServiceTemplate: stard-service-template
              replicas:
                - name: replica0
                  templates:
                    replicaServiceTemplate: replica-service-template

  templates:
    serviceTemplates:
      - name: cluster-service-template
        metadata:
          annotations:
            # For more details on Internal Load Balancer check
            # https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
            cloud.google.com/load-balancer-type: "Internal"
            service.beta.kubernetes.io/aws-load-balancer-internal: "true"
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
            service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
            service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
        spec:
          type: LoadBalancer
      - name: stard-service-template
        metadata:
          annotations:
            # For more details on Internal Load Balancer check
            # https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
            cloud.google.com/load-balancer-type: "Internal"
            service.beta.kubernetes.io/aws-load-balancer-internal: "true"
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
            service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
            service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
        spec:
          type: LoadBalancer
      - name: replica-service-template
        metadata:
          annotations:
            # For more details on Internal Load Balancer check
            # https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
            cloud.google.com/load-balancer-type: "Internal"
            service.beta.kubernetes.io/aws-load-balancer-internal: "true"
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
            service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
            service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
        spec:
          type: LoadBalancer


[sw/silicon_creator] Names of OTP entries in the creator and owner partitions

It looks like all entries in the creator partition start with the CREATOR_SW_CFG_ prefix while most items in the owner partition do not start with OWNER_SW_CFG_. I think this is something we should fix, right?

I think the change will make it easier to audit the system by making it easier to map config options to OTP logical partitions.

this sounds good to me, but i think we should only do this if there isn't much impact to design / dv.
@msfschaffner
@cindychip
can probably best comment on that.


Pseudopotential Library collections from QMCPACK group

Hi all,
There is a community website (https://pseudopotentiallibrary.org/) for pseudopotentials/effective core potentials(CEPP,ccECP,eCEPP), which are especially designed for high accuracy correlation electron calculations(like QMC, Coupled Cluster etc.), These PP/ECP and basis sets have Gamess/Nwchem/Molpro formats in the site, can be also download directly from it's github site (https://github.com/QMCPACK/pseudopotentiallibrary/tree/master/recipes).
Could be these PP/ECP & Basis sets tranfered into BSE(just a data transfer,should be very easy), then more people could know and use them.

Should be somewhat easy, although it might take a bit to understand the directory structure and naming.


feat: Update the testnet banner to display when the user is on the Arbitrum testnet as well

Feature Request
Describe the Feature Request
The goal of this issue is to ensure the testnet banner is displayed when users are connected to Arbitrum Sepolia, in addition to Ethereum Sepolia.
Describe Preferred Solution

Update the condition that displays the banner

Definition of Done

[ ] The testnet banner should be displayed when the user is connected to either the Arbitrum testnet or the Ethereum testnet

Related Code
TestnetBanner.tsx

Hey, I'd like to contribute to this project. I have a commit for this issue. But I'm not have access to git push. How could I contribute to the project?

@guilhermebins Nice! It's yours
You can contribute by creating a fork and submitting a pull request. I see that you've already done it 🙂


Fix JSON code indenting

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: vlad-ro sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@vlad-ro : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Thanks, @vlad-ro!
#sign-off


ReplayKit - Use a single token/identity for Broadcast Extension & App

Hi,
We have implemented screen sharing with Twilio and its working fine. But one of my client wants to share screen and keep continue call. That means in the app there is on going call and user starts sharing screen from control centre (from iPhone). Application and extension (in xCode to share screen you need to add extension) both are different things and they can not use same videoTrack as both have different target.
Question is: How to share screen from extension and keep continue call from app (With same user) ?

Hi @kushJadia,

Question is: How to share screen from extension and keep continue call from app (With same user) ?

In this case you would likely need two Participants, with two distinct tokens for the presenter's device.

Extension - Share video from ReplayKit.
App - Share microphone, and playback remote audio.

If sharing the screen outside of your app is a hard requirement, then this is probably the easiest solution.
Alternatively, it might be possible to share frames across process from the extension to the main app and have the app connect to the Room and share all the Tracks. I have not tried this approach personally, so I can not speak to its feasibility.
Best,
Chris

I'll add a +1 for this feature. I've also had the same feature request recently with the same basic requirements.
We'd like to be able to join a video call, then at some point after we join the call share our screen (not just the app window) to the call and continue to hear the other participants on the call.
Unfortunately sharing data between a broadcast upload extension and it's host app is very limited. Too limited to share streaming video for sure.
I'd argue that connecting two separate participants with two distinct tokens would degrade the User Experience since it would cause one actual participant to have appear as two in the call's user list.
I agree the correct way to implement something like this (given the limitations of what the iOS SDK allows) would be to connect in the app for mic and playback of remote audio and from the extension for screen sharing via ReplayKit. Ideally we would have a way to merge the two connections though so they appear as a single participant to the other participants in the call.

Hey @KrisConrad,
I hear you, and I wish that I had a single Participant solution to offer at this time.

We'd like to be able to join a video call, then at some point after we join the call share our screen (not just the app window) to the call and continue to hear the other participants on the call.

Since each process (the extension and the app) needs to have a secure connection with our servers there isn't any way for those to be represented as the same Participant in a Room. Our current usage model requires that identities be unique, so when the second process joins with the same identity as the first, the first process will be disconnected.

Unfortunately sharing data between a broadcast upload extension and it's host app is very limited. Too limited to share streaming video for sure.

I need to research this more before claiming defeat. For video samples, the main data we care about is the IOSurface that backs each CVPixelBuffer. The surfaces are designed to be shared across processes between Apple's frameworks (like mediaserverd) and your app, so if there is any way we could funnel the IDs fast enough...

I'd argue that connecting two separate participants with two distinct tokens would degrade the User Experience since it would cause one actual participant to have appear as two in the call's user list.

As a workaround, you could have some designation that is signaled in the extension Participant's identity, or in the Track names that the app could use to merge the two Participants.
I will gather more feedback from internal stakeholders and get back to you.
Regards,
Chris

@ceaglest @KrisConrad  Thanks for your help.

Hi @kushJadia,

Question is: How to share screen from extension and keep continue call from app (With same user) ?

In this case you would likely need two Participants, with two distinct tokens for the presenter's device.

Extension - Share video from ReplayKit.
App - Share microphone, and playback remote audio.

If sharing the screen outside of your app is a hard requirement, then this is probably the easiest solution.
Alternatively, it might be possible to share frames across process from the extension to the main app and have the app connect to the Room and share all the Tracks. I have not tried this approach personally, so I can not speak to its feasibility.
Best,
Chris

Hi @ceaglest ,
We are trying to implement the ScreenShare in our App. As you mentioned above, Is it possible to share the screen tracks from extension to App.
Are we have any other way to accomplish this ?
Thanks,
Sathish

Hey @KrisConrad,
I hear you, and I wish that I had a single Participant solution to offer at this time.

We'd like to be able to join a video call, then at some point after we join the call share our screen (not just the app window) to the call and continue to hear the other participants on the call.

Since each process (the extension and the app) needs to have a secure connection with our servers there isn't any way for those to be represented as the same Participant in a Room. Our current usage model requires that identities be unique, so when the second process joins with the same identity as the first, the first process will be disconnected.

Unfortunately sharing data between a broadcast upload extension and it's host app is very limited. Too limited to share streaming video for sure.

I need to research this more before claiming defeat. For video samples, the main data we care about is the IOSurface that backs each CVPixelBuffer. The surfaces are designed to be shared across processes between Apple's frameworks (like mediaserverd) and your app, so if there is any way we could funnel the IDs fast enough...

I'd argue that connecting two separate participants with two distinct tokens would degrade the User Experience since it would cause one actual participant to have appear as two in the call's user list.

As a workaround, you could have some designation that is signaled in the extension Participant's identity, or in the Track names that the app could use to merge the two Participants.
I will gather more feedback from internal stakeholders and get back to you.
Regards,
Chris

Hi @ceaglest ,
I have disconnected from the App room and joined the room via extension for screen sharing.
But the problem is,
I can't able to listen the other participants AudioTrack from the extension. It would we really helpful If we get any solution.

Hi @ceaglest ,
We are waiting for more than 3 days. So, can you please checkout.

Hi @sathishvgs,
Sorry that I missed your earlier message.
I don’t believe that it’s possible to play audio from an extension, or at least we were not successful in doing so yet.
If you want to play remote audio it should be possible to join from both the extension (for your screen) and in your app at the same time. These techniques are demonstrated separately in the replaykit example app but you could combine them if needed.
Your app would need to account for two Participants and display them in some way that is appropriate.
Regards,
Chris

Hello ReplayKit developers,
While our current object & auth models don't allow several applications to publish Tracks under a single Participant we have made some improvements to how broadcast extensions work in the 2.10.0 release. Your extension can now connect (as a unique Participant) without subscribing to any Tracks.
If your goal is to have screen sharing in a traditional video conference, then the best solution I can offer at this time is to connect one ReplayKit broadcast extension Participant to share the screen, and another app Participant to publish the camera and microphone, and receive audio/video Tracks.
You can try using the extension with 2.10.0 in our updated ReplayKitExample.
Best,
Chris


Fix/cleanup-code

A PR to clean out all the unused variables, commented-out code sections and unhelpful comments in the codebase.

This is a pretty simple PR so I am going to merge without review. YOLO


GCS Connector is missing in maven

Looks like monix-gcs was never released as there no artifact in maven - https://search.maven.org/search?q=a:monix-gcs*

@YulawOne thanks for reporting that, I noticed yesterday when doing the release... 🤦

@YulawOne released monix-gcs 0.3.1

@YulawOne released monix-gcs on 0.3.3
https://oss.sonatype.org/#nexus-search;quick~monix-gcs

Please, let us know if you have issues any other issues 👍


need advice about how to split the model into two GPU

Say I have two 24GB,
what's the best strategy which splits the checkpoints or running computed tensor into two cards, and which is optimal for utilizing the free VRAM as much as possible ?
checkpoints: DiT, CLIP, T5, AE
running tensors: for each of above
anything else...
Given the optimal target, can you split these 8 or more items into two groups ?

Many people don't use 48(or more)GB cards just because these A/H cards is not versatile not because they can't afford. IMO AI model developers should stand by these common people, not nvidia. Gotta take more care of the GPU usage cases.


Gulp stats slow on call. Execute is fast

What is causing Gulp to need 15 seconds before it starts? Anyone having a fix, or information what causes this?
I'm using Gulp in a Vagrantbox. Calling Gulp from a terminal inside PHPStorm.

Probably a vagrant problem—I used to run my gulpfile outside of my vagrant box for that reason.
Follow steps here: https://stefanwrobel.com/how-to-make-vagrant-performance-not-suck
If that doesn't work, try running Gulp outside of the vagrant box. If you install node and npm using nvm, it's really easy to remove without leaving any apps or files anywhere.

If that doesn't work, try running Gulp outside of the vagrant box. If you install node and npm using nvm, it's really easy to remove without leaving any apps or files anywhere.

@callumacrae I'm bit of a newbie with all of this to be honest. How do I do this?

https://github.com/creationix/nvm#install-script


MultilineBlockLayout with LineLength and without assignment

We have #7284 with some fix.
In the tests there are some_result = lambda do |, and for such code there are no offenses.
But in the documentation from this PR there is blah { | (as "good"), without assignment, and such code still brings offense from MultilineBlockLayout.

Expected behavior
No offenses.
Actual behavior
There is offense from MultilineBlockLayout.
Steps to reproduce the problem
Just a code with multiline parameters, exceeding LineLength, and without assignment, line in the documentation with blah { |.
RuboCop version
0.80.0 (using Parser <IP_ADDRESS>, running on ruby 2.6.5 x86_64-linux)

Sorry, forgot to mention @jonas054 in case if they're interested.

Can you add a concrete example of the whole block where you see this problem?

Can you add a concrete example of the whole block where you see this problem?

As I wrote: the documentation (blah { |) from this PR is not "good", as wrote in this PR, and make offenses.
Example of the same code with resolved other default cops:
# test.rb
# frozen_string_literal: true

blah do |
  _long_list,
  _of_parameters,
  _that_would_not,
  _fit_on_one_line
|
  foo(i)
  bar(i)
end

> rubocop test.rb
The following cops were added to RuboCop, but are not configured. Please set Enabled to either `true` or `false` in your `.rubocop.yml` file:
 - Style/HashEachMethods (0.80)
 - Style/HashTransformKeys (0.80)
 - Style/HashTransformValues (0.80)
For more information: https://docs.rubocop.org/en/latest/versioning/
Inspecting 1 file
C

Offenses:

test.rb:3:9: C: Layout/MultilineBlockLayout: Block argument expression is not on the same line as the block start.
blah do | ...
        ^

1 file inspected, 1 offense detected

> rubocop -V
0.80.1 (using Parser <IP_ADDRESS>, running on ruby 2.7.0 x86_64-linux)

The difference, as I see, in an assignment.

Thanks, @AlexWayfer, having the whole example here helps clear up what the issue is.

I can't reproduce the problem. With the example you gave, @AlexWayfer, I get the same offense report, but only because the argument list does fit on a single line. If I make it longer the problem goes away.

I can't reproduce the problem. With the example you gave, @AlexWayfer, I get the same offense report, but only because the argument list does fit on a single line. If I make it longer the problem goes away.

That's true. I can't reproduce it locally too. I don't remember a project with this issue. So, I'm going to close this and I'll reopen if I'll have more details.
Sorry to bothering you.


Display source code using dark mode

Creating a "dark mode" style for this application is outside of the scope of the project, so we're not going to do it.
However, I propose to use a "dark mode" style to display the source code in the modal windows. In addition to being the most common style for developers, showing it in "dark mode" further emphasizes the difference between the application interface and its code:
Before / After

Looks  better 👍

👍


ios-sdk-pkgs: Init from iOS SDK from XCode

Motivation for this change
This is basically a rewrite of the existing iOS support to take advantage of the new cross compilation infrastructure.
Limitations/issues:

Currently the SDK is used impurely but I want to eventually have the user import the SDK into the store and use it purely.
SDK versions newer than 10.3 are not supported. These will require a newer cctools which has it's .tbd linking support split out into libtapi which seems hard to build.

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

CC @ElvishJerricco @shlevy

Perfect!

Is there an example for using this anywhere?

Ah I should add/change some lib/systems/examples.nix entries.


Update the PDO version and clean up tests

Upgrade the PDO version to 0.2.63 which includes
the canonical shell functions.
Clean up the digital asset and exchange tests so
they are more consistent and use the service groups and service database switches provided by the
canonical PDO shell commands.

thanks, looks good, tested locally.


App apparently requires macOS 10.14 or later?

Describe the bug
Hello. In the README requirements section it says macOS Sierra (10.12) or later is required. I have recently downgraded to High Sierra (10.13) and when I launch the app it says macOS 10.14 is required.
To Reproduce
Launch on macOS High Sierra (10.13).
Expected behavior
App should launch on High Sierra.
Screenshots

Desktop (please complete the following information)

macOS version: 10.13.6 (17G14042)
Safari version: 13.1.2 (136<IP_ADDRESS>.5)
Passafari version: 0.8.1
How was Passafari installed (Github release, built self, ...): GitHub relase

Additional information
It seems this is set here:
https://github.com/adur1990/Pass-for-macOS/blob/e91cb9c791ca661aaed84de48ebddfe7d32fb5f2/passformacos.xcodeproj/project.pbxproj#L395
and/or here:
https://github.com/adur1990/Pass-for-macOS/blob/e91cb9c791ca661aaed84de48ebddfe7d32fb5f2/passformacos.xcodeproj/project.pbxproj#L452

Oh dang, thanks for pointing this out. I will inspect this asap.

Thanks!


Create isiscorbino.txt

Hi, I'd need to add my shcool to JetBrains.
Schoolname is ISIS E. Corbino
It is located in Contursi Terme, Salerno, Italy

@mrobottox First stage of verification passed, official domain is isiscorbino.edu.it as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@mrobottox Thank you for your patience while waiting for a review and processing your request. Pull request merged.


[Impeller] Deprecate --trace-skia flag and instead have a --trace-graphics flag.

By default, Flutter tells Skia to do very little tracing, since it can impact application performance significantly. However, sometimes it is helpful to view more verbose Skia tracing to diagnose where it is spending CPU time during rendering (e.g. in a long Canvas::flush call or during display list dispatching).
We should adopt a similar strategy for Impeller. Rather than creating a --trace-impeller flag, it would be easier for developers to just have a --trace-graphics flag. This would either turn on Skia tracing when Skia is used or extra tracing in Impeller when impeller is used. It could also potentially turn on other per-frame traces in the engine that are typically too expensive to leave on all the time if we found it necessary.
We should also audit impeller-based tracing and see if there are traces that should be kept off by default as we get closer to declarign it ready for GA.
/cc @chinmaygarde @jonahwilliams @gaaclarke @zanderso @christopherfujino

(I'm less concerned with the name(s) of the flag than that we eventually avoid multiple-trace-events-per-frame from Impeller and that we have a good way to turn that on when needed)

IMHO, the current tracing system which effectively ignores trace categories isn't as flexible as it could be. And, we are re-inventing stuff that projects like Perfetto have solved. I'd rather integrate with Perfetto and have as many traces as we want but each within discrete categories. For categories that are disabled, the performance overhead is negligible. We could then have tracing present but disabled in all modes (even release modes which some g3 clients have requested support for).
But, fundamentally, I agree with the ask here in this bug. I also don't particularly care if we alias trace-skia to mean trace-graphics if that makes the transition easier. So far, with Impeller, we haven't really done too much profiling with traces. Instead, resorting to native profilers and frame capture tools.

Right now, it's more expensive to use the Dart VM timeline recorder than using native tracing on Android via Perfetto. It's also likely more expensive than using Signposts, though I haven't measured that (I think @gaaclarke might have). We should try to drive that down for sure, but it will likely take time to get there.

Yea signposts are faster from what I recall.  Dart's timeline recorder eats up a lot of time querying the thread time.  Presumably signposts can be more efficient about synchronizing with the kernel since the thread time never has to reach userland.

Do we need this at all? I don't recall timeline tracing every being all that useful for GPU workloads. We are already judicious with the number of traces we add. I don't think we need ways to toggle traces from different subsystems.

I don't think we need this. If folks want to trace engine workloads they should use platform native sampling profilers. Dart timeline events have too much overhead.


Fully migrate all exercises to ounit2

The rest of the migrating to ounit2
/closes #442
/closes #364

@georgyo, I was just hitting this last night. I manually changed it to oUnit2 in the dune file, however, I'm still receiving an error.

opam list shows I have the following installed:
ounit2                2.2.6        OUnit testing framework

Any ideas?

@curtisault  The biggest different in ounit2 is that the package name is all lowercase. So oUnit becomes ounit2.

@curtisault The biggest different in ounit2 is that the package name is all lowercase. So oUnit becomes ounit2.

I didn't even notice the casing. That did the trick.
Thank you.


investigate how to integrate blocktrades

https://steemit.com/blocktrades/@blocktrades/blocktrades-introduces-crypto-affiliates-program-for-entrepreneurs

@nosus
嵌入代码（包括推荐号）
Can't load Exchange widget.

@nosus
嵌入代码有更新，官方把 show_powered_by_blocktrades这个参数选项给去掉了
Can't load Exchange widget.

UI updated.

verified on dev

verified on prod and close it.


Refactor to fix levelization

High Level Overview of Change

Remove Application & Database dependency in PerfLog. Replace it with a callback passed into the constructor.
Fixes the circular dependency between ripple/nodestore and ripple/basics

Context of Change
#4059 and #4063 introduced some great and necessary changes related to improve and speed up ledger syncing. Unfortunately, they also introduced a new circular dependency in the rippled code. This PR refactors that code to remove the dependency from ripple/basics to ripple/nodestore (and indirectly ripple/app).
There are strictly no functional changes.
There are two commits, since I don't know how important it is that the new callback be available to unit tests. The tests work just fine without it, so the second commit inlines the static function added in the first commit.
Type of Change

[X ] Refactor (non-breaking change that only restructures code)

PerfLog should be able to reach into anything to grab data.

If that's the case, then it doesn't belong in ripple/basics.

Is the callback in the constructor something for the general case going forward? Alternately, how can we make it so PerfLog can reach into any object from which it wants to collect data for reporting?

If you mean, can the callback be changed to add functionality, then the answer is yes. It's written so that it has access to the ApplicationImp, and thus anything underneath it. Thus PerfLog will be able to reach into any object that ApplicationImp knows about to collect data.

Additionally, PerfLog should be able to receive data submitted to it, similar to Beast::Journal, from anywhere.

There's nothing preventing that.

PerfLog should be at something like omniscient levelization so it can see everything, and so it can receive data pushed to it. I realize these are contradictory on their face, but if the thing that feeds data to PerfLog can be reached by all classes, then it should be a matter of header indirection, right?

You could describe what I've done here as "header indirection". I didn't change any functionality, or really even what data is available to be logged. I just changed the way it's organized to remove the circular dependency.

Where does it belong if not ripple/basics?
What needs to be done so that when I want to add things to collect for PerfLog, it doesn't require a callback to be fed to the constructor? That's kludgey.

@ximinez  Would an abstract class header with some interface and with no ripple headers of its own be the way to do what I'm asking?

I'm kinda with @mtrippled.  I'd like a fix for this levelization problem that has more legs.
There are currently three things that PerfLog is rattling around inside to get information from:

NetworkOPs
the NodeStore and
the ShardStore

I have the sense that this could grow in the future.  It would be nice to put something in place to make that growth easy.
Suppose PerfLog provided a registration mechanism that allowed folks who want to be called to provide a callback.  This way NetworkOPs, NodeStore, and ShardStore could all register their callbacks with PerfLog.  That would keep PerfLog from needing to know the intimate details of everything that needs to put stuff in the PerfLog.
Would a mechanism like that meet all the needs @mtrippled?

@scottschurr I prefer to just reach in--is that not viable? Registering and callbacks to simply execute a function seem complex to me.

As it is, PerfLog only needs Json and to know the job queue types from the rest of rippled--so that as jobs are submitted it knows how to track their durations. That seems like a pretty clean interface that should help to avoid levelization problems.

@mtrippled, yeah, just reaching in is much easier to code.  But it also means that PerfLog becomes dependent (must know about) everything that it reaches into.
As it stands PerfLog is dependent on NetworkOPs and NodeStore and ShardStore.
If we use registration, then NetworkOPs, NodeStore, and ShardStore each, individually, become dependent on PerfLog.  PerfLog has no dependency on any of them.  Adding another source of information in the future is as simple as adding registration in the object that wants to provide the information.
I'm not insisting that registration is the only way to solve the problem.  But it's the one that seems the cleanest to me at the moment.  I'm open to other options.  Frequently @ximinez will look at a suggestion like this and come up with a better idea.

@ximinez Would an abstract class header with some interface and with no ripple headers of its own be the way to do what I'm asking?

This might work.... I'll try it and report back.

> If we use registration, then NetworkOPs, NodeStore, and ShardStore each, individually, become dependent on PerfLog. PerfLog has no dependency on any of them. Adding another source of information in the future is as simple as adding registration in the object that wants to provide the information.
If I'm understanding correctly, @mtrippled 's objection to my changes is effectively that the code to build the report is in two places: PerfLogImp::report() and the Application callback. He wants it in one place: PerftLogImp::report(). Registration would not give him that, and in fact would make it "worse" because now you've got four places with the code. Is that more or less correct?

If I'm understanding correctly, @mtrippled 's objection to my changes is effectively that the code to build the report is in two places: PerfLogImp::report() and the Application callback. He wants it in one place: PerftLogImp::report(). Registration would not give him that, and in fact would make it "worse" because now you've got four places with the code. Is that more or less correct?

I'm not convinced that's a legitimate objection.  NetworkOPs, NodeStore, and ShardStore already have functions dedicated to being used by PerfLog.  NodeStore and ShardStore implement getCountsJson() specifically to provide support for PerfLog.  NetworkOPs provides stateAccounting() for the same reason.
But, again, registration is not the only way to solve this problem.  I'm good with other choices as well.

@ximinez it's mainly that I don't want to jump through flaming hoops to measure things. Instead, create a function in the class that contains the data I want (if it doesn't already exist), then call the function when PerfLog runs the report.

I thought it was that way already--with the exception of job types and how to make Json objects, PerfLog doesn't know anything about rippled, so I assumed cyclical dependencies were pretty much impossible.

Or is levelization different than cyclic dependencies? Is there a way to make PerfLog level 0 or something? Make it like a global?

PerfLog doesn't know anything about rippled, so I assumed cyclical dependencies were pretty much impossible.

That's almost, but not quite, true.  What we have to address is this little knot of badness:
https://github.com/ripple/rippled/blob/develop/src/ripple/basics/impl/PerfLogImp.cpp#L301-L306
Those few lines are digging around in the Application to dig further into the NodeStore, the ShardStore, and NetworkOPs.  That's the section of code we need to fix.

@mtrippled @scottschurr I took all your feedback to heart, and had a good discussion offline with Mark. I think the new result is much better, and as an added bonus, it resolves an old circular dependency between ripple/basics and ripple/rpc!


Changes to HG_RETURN_VALUE breaks API

This commit https://github.com/mercury-hpc/mercury/commit/80869e0e67a39b772e4477d37445e30b4ff89eef breaks our Mochi stack. Thallium uses HG_RETURN_VALUES and expect X to take 1 argument. Would it be possible to introduce an HG_RETURN_VALUES2 if Mercury now needs it to take 2 arguments, but leave HG_RETURN_VALUES? Otherwise this will force all the versions of thallium up to this point to become incompatible with future versions of Mercury, and all future versions of thallium to become incompatible with older versions of Mercury.

Oh great, I just see your issue now and I went ahead today to create the release 🫤 I was not expecting that someone was using that directly... well I will need to see what I can do. Just curious, what are you using the HG_RETURN_VALUES macro for ?

I guess I'll just retag and remake a release with that fix on Monday...

I'm using it here: https://github.com/mochi-hpc/mochi-thallium/blob/main/include/thallium/margo_exception.hpp#L47 to generate the name of the error code when throwing exceptions.

Yeah I should have.

Fixed but as a heads-up I'll have to address this properly in a 3.0 release so you should start using that routine instead :)

Will do, thanks :-)


Add react dev tools to electron app

Depends on https://github.com/SocialGouv/archifiltre/pull/344

Download Artifacts :

win-ia32
win-x64
linux-x64


Error when editing keyboard shortcuts

VSCode Version: Code - Insiders 1.15.0-insider (9912a545c9728ce5e6b3cdfcc9442ee47955aa42, 2017-07-21T05:07:17.763Z)
OS Version: Darwin x64 16.6.0
Extensions:

Extension
Author (truncated)
Version

EditorConfig
Edi
0.9.4

vscode-docker
Pet
0.0.16

html-css-class-completion
Zig
1.8.0

vscode-intelephense-client
bme
0.7.2

ruby
gro
0.0.6

prettify-json
moh
0.0.3

vscode-duplicate
mrm
1.1.0

php-docblocker
nei
1.2.0

editorconfiggenerator
nep
0.2.1

vetur
oct
0.9.2

Ruby
reb
0.12.1

Steps to Reproduce:

Preferences > Keyboard Shortcuts
Edit the Open Next Editor shortcut and set it to ctrl+tab
Get this error: Error 'Error: Unable to write keybindings. Please open Keybindings file to correct errors/warnings in the file and try again.' while editing keybinding. Please open

Reproduces without extensions: Yes

@greggilbert Can you please open your keybindings.json file and check if there are any errors in that file.. you can open that file from the keybindings editor or through the global command Open Keyboard Shortcuts File

@sandy081 I tried clearing out my keybindings and I was able to save. The issue is that there was a trailing comma in what I had there:
// Place your key bindings in this file to overwrite the defaults
[
    { "key": "shift+cmd+left",      "command": "workbench.action.previousEditor"},
    { "key": "shift+ctrl+tab",      "command": "workbench.action.previousEditor"},
    { "key": "cmd+1",                "command": "workbench.action.openEditorAtIndex1" },
    { "key": "cmd+2",                "command": "workbench.action.openEditorAtIndex2" },
    { "key": "cmd+3",                "command": "workbench.action.openEditorAtIndex3" },
    { "key": "cmd+4",                "command": "workbench.action.openEditorAtIndex4" },
    { "key": "cmd+5",                "command": "workbench.action.openEditorAtIndex5" },
    { "key": "cmd+6",                "command": "workbench.action.openEditorAtIndex6" },
    { "key": "cmd+7",                "command": "workbench.action.openEditorAtIndex7" },
    { "key": "cmd+8",                "command": "workbench.action.openEditorAtIndex8" },
    { "key": "cmd+9",                "command": "workbench.action.openEditorAtIndex9" },
]

The keybindings file worked just fine with the final comma there, so I guess it just doesn't work when trying to add a keybinding to that sort of list?

@greggilbert Yes, we bail out if there are any errors in keybindings file while writing into it from keybindings editor. It's same in settings. But we ignore trailing comma which is not the case here.
Will make it resilient for trailing comma case.


Determine a fix for the zoom page transition performance

The zoom page transition is the default page transition for Android, and used heavily by google customers and customer: money. Unfortunately, the raster performance of this transitions leaves a lot to be desired. Currently, the transition uses a FadeTransition and ScaleTransition. The changing scale factor essentially disables raster caching of child pictures, and apparently causes increases in native memory compared to the baseline transition: https://github.com/flutter/flutter/issues/100972
To solve this, we originally investigated switching from a ScaleTransition using a transform layer, to a ScaleTransition using an ImageFilterLayer with a ImageFilter.matrix. In theory, this should be faster than a transform layer for this case, since the pages aren't animated at all while the transition occurs and this allows a single raster cache entry to be used for the entire page (which can also absorb the opacity from the FadeTransition). While switching did improve memory consumption, it regressed worst frame raster performance by about 1.5x on our gallery benchmark (https://flutter-flutter-perf.skia.org/e/?begin=1650989645&end=1651012744&keys=X809a5f88fb68c1b259242471f65ee081&num_commits=50&request_type=1&xbaroffset=28565) . This is likely due to either the performance of Skia's ImageFilter, or perhaps specific to shader compilation for the first use of this filter.
Interestingly enough though, we accidentally rolled a version of flutter into google3 that used a ImageFilterLayer  🙃 and it significantly improved the performance of a number of customer money benchmarks. A "worst raster frame" benchmark improved from 104.7 ms to 41.75 ms.  A follow-up roll reverted the change and the benchmark regressed again. b/232993351
Its quite difficult to identify performance improvements that are as good as the above change, so I attempted a fix in the engine/raster cache to special case a scale transform matrix filter: https://github.com/flutter/engine/pull/33562 , but that stalled out since it conflicted with another refactor PR.
@xster @zanderso can you help determine how we should prioritize this?

(As to why things regressed on the gallery and improved on customer: money, I suspect that is due to the gallery being fairly trivial while money's application is not)

I think the problem was entirely due to ImageFilter.Matrix not resulting in a simple matrix blit as we had originally thought. Moving away from using ImageFilterLayer to accomplish this  and instead using a layer that is aware that it is just trying to apply a transform and doing it directly in the cache Draw method should provide a lot of improvement. Has that avenue been investigated here?

No, you're right - its not that changing the scale invaldiates it - its that the child animation, intentional or accidental invalidates it.

https://flutter-flutter-perf.skia.org/e/?queries=sub_result%3D90th_percentile_frame_rasterizer_time_millis%26sub_result%3D99th_percentile_frame_rasterizer_time_millis%26sub_result%3Daverage_frame_rasterizer_time_millis%26sub_result%3Dworst_frame_rasterizer_time_millis%26test%3Dold_gallery__transition_perf

About a 50% reduction in 99%, 90%, and average frame raster time. Worst frame time might have gone down a bit too, though its harder to tell since it is much less stable

A reported performance issue with the old implementation for reference: https://github.com/flutter/flutter/issues/108246


NIFI-6762: Submit new List Processor - ListMultipleS3Buckets

Description of PR
Nifi 6762: Submit new List Processor - ListMultipleS3Buckets
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
https://issues.apache.org/jira/browse/NIFI-6762

[ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Yes

[ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
Yes, just rebased.

[ ] Is your initial contribution a single, squashed commit?
Two commits in this PR Contribution.

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
YES
[ ] Have you written or updated unit tests to verify your changes?
YES
[ ] Have you verified that the full build is successful on both JDK 8 and JDK 11?
YES
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0? YES
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
NOT Applicable
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
NOT Applicable
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
NOT Applicable

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?
NOT Applicable

I'd not worry about v1 listing at all, as it only adds more testing, and even third party stores all seem to do v2. V2 scales better.
Bear in mind multipart upload timestamps are set to when an upload starts, not when it finishes; if you are polling for changes that can catch you out


Does not work in IE

All functions like add, dark theme, remove background tabs and close icons doesn't show up

Can confirm.
Don't think it should be available on internet explorer. It's deprecated browser.

Can confirm.
Don't think it should be available on internet explorer. It's deprecated browser.

We won’t be supporting IE.


[RELEASE-1.11] Add defaulting for all GRPC probe types

Changes

Backport https://github.com/knative/serving/pull/14773

/assign @skonto

/lgtm
/approve

/unhold

/retest

  Warning  Failed          4m48s (x5 over 6m6s)  kubelet            Error: ImagePullBackOff
  Normal   Pulling         4m37s (x4 over 6m7s)  kubelet            Pulling image "image-registry.openshift-image-registry.svc:5000/openshift-marketplace/serverless-knative-operator"
  Warning  Failed          4m37s (x4 over 6m7s)  kubelet            Failed to pull image "image-registry.openshift-image-registry.svc:5000/openshift-marketplace/serverless-knative-operator": rpc error: code = Unknown desc = reading manifest latest in image-registry.openshift-image-registry.svc:5000/openshift-marketplace/serverless-knative-operator: manifest unknown: manifest unknown
  Normal   BackOff         69s (x20 over 6m6s)   kubelet            Back-off pulling image "image-registry.openshift-image-registry.svc:5000/openshift-marketplace/serverless-knative-operator"

/test 411-test-e2e-aws-ocp-411

/retest-required


targeting bitcoin puzzle 67

I want to particular target bitcoin puzzle 67.
1BY8GQbnueYofwSuFAT3USAhGjPrkxDdW9
Start
0000000000000000000000000000000000000000000000040000000000000000
End
000000000000000000000000000000000000000000000007ffffffffffffffff
Can someone help in creating BAT file for cubitcrack?

Hi,
For the range for 67 bit add this:
--keyspace 0x40000000000000000:0x80000000000000000 1BY8GQbnueYofwSuFAT3USAhGjPrkxDdW9

THANKYOU MAN!
last question
will the brute force attack will be random or sequential, if sequential than how can we make attack random. Since I believe random attack has more chance for success.

BitCrack is sequential like most GPU tools, if you want to add randomization, I recommend you implement a later that sends the job to bitCrack.

You can use the stride option. Or there is a modified version with random option ->  https://github.com/djmuratb/BitCrack2

thanx man!
in modified version how should I create the bat file if I want to target puzzle 67 randomly?

BitCrack.exe -b 64 -t 256 -p 1024 --rstride 5 --keyspace<PHONE_NUMBER>0000000:7ffffffffffffffff 1BY8GQbnueYofwSuFAT3USAhGjPrkxDdW9
pause

If we want to target this address
1FeexV6bAHb8ybZjqQMjJrcCrHGW9sb6uF
In random mode
How should we create the bat file?

Use the -r argument.
Anyone has idea if Bitcrack supports vanity search? I have lost the
original brichard link of the full bitcrack functions...
Thanks.
On Wed, Nov 20, 2024 at 6:27 PM learnercrypto @.***>
wrote:

If we want to target this address
1FeexV6bAHb8ybZjqQMjJrcCrHGW9sb6uF
In random mode
How should we create the bat file?
—
Reply to this email directly, view it on GitHub
https://github.com/brichard19/BitCrack/issues/426#issuecomment-2488010080,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHWRFWLEJEDERJ45BKNHUCD2BRIYZAVCNFSM6AAAAABRO6SK5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBYGAYTAMBYGA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Fixed Typo in Instructions to advertise custom routes

coma -> comma

@garrett-wood : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@garrett-wood Thanks for this fix.


Mouse gets away from the object it is dragging

In balloons-and-static-electricity the drag handler is provided by SimpleDragHandler, and the pointer can get away from the object it is dragging (if you drag it against the wall for example).  Scenery should provide support for a drag handler that stores the relative location of the object and uses it for dragging.  I am also having this problem with the objects and sliders in Forces and Motion Basics.  This is very closely related to #3
This could be handled in balloons but I think it would be superior and more maintainable to provide support directly in scenery.  In my opinion, this behavior should be fixed before our July milestone.

Recommend handling in new drag listener, see https://github.com/phetsims/scenery/issues/131.
Sound acceptable?

We have good support for this in MovableDragHandler, I wouldn't mind if we also had good support for it in the next generation scenery input listeners and we can track it in #131.


ci: bump update_rust_version to 2.1.5

Bumping update-rust-version CI action script to 2.1.5 because of Cargo.lock issue.

Closing this issue, because the package is deprecated.


Some tests are sensitive to line endings

The following test fails when the contract code (Simple.sol) have CRLF line endings, but succeed with LF line endings. This is likely because a hash of the Solidity source code is stored in the swarm data (which is naturally sensitive to things like line endings). This should be fixed by trimming the swarm data, which we are doing. It seems either we are not properly trimming the swarm data or this hash information is otherwise included in the bytecode which is being deployed.
Kernel without entry procedure > .executeProcedure(bytes24 key, bytes payload) > should execute > Simple Procedure > X() should fail

Any idea how I could re-create this? I'm not sure how to trigger CRLF formatting.

I've been trying to do that, and it seems to have rectified itself. It was a problem we had before, but it seems to have rectified itself. Perhaps it was another issue that was fixed be me resaving the file. All the tests are passing for me now, which was my primary concern, so I'll close it for now unless it crops up again.


can we run objc in macOS 12.1

can we run objc in macOS 12.1 and xcode: 13.2.1
?

I failed, too
MAC: 12.1
Xcode: 13.1

how to fix this?

https://github.com/luokeluo/DebugObjc4


small remark: jupyter.org raises a "gives no information about its identity" warning on firefox

Jupyter.org is behind cloudflare's ssl. I wonder if there's something different we should be doing. @rgbkrk?

Since we're on the free plan, I'm not sure how much else we can change there. We could run our own but then we have more infra. to manage for what's a static page served underneath by GitHub pages.

Having looked into it a bit, I think the warning is actually about images coming from http rather than https, because the warning only shows up on a couple of pages. I think we can fix this.

Ok, awesome.


Community: gather token usage info in BedrockChat during generation

This PR allows to calculate token usage for prompts and completion directly in the generation method of BedrockChat. The token usage details are then returned together with the generations, so that other downstream tasks can access them easily.
This new feature is totally optional, enabled by passing the param compute_token_usage.
This allows to define a callback for tokens tracking and cost calculation, similarly to what happens with OpenAI (see OpenAICallbackHandler. I plan on adding a BedrockCallbackHandler later.
Right now keeping track of tokens in the callback is already possible, but it requires passing the llm, as done here: https://how.wtf/how-to-count-amazon-bedrock-anthropic-tokens-with-langchain.html. However, I find the approach of this PR cleaner.
Thanks for your reviews. FYI @baskaryan, @hwchase17

Hey! For Bedrock you may want to use the tokens returned in the headers "x-amzn-bedrock-*", like

x-amzn-bedrock-output-token-count
x-amzn-bedrock-input-token-count
instead of using anthropic only tokenizer 🤔?

Hey! For Bedrock you may want to use the tokens returned in the headers "x-amzn-bedrock-*", like

x-amzn-bedrock-output-token-count
x-amzn-bedrock-input-token-count
instead of using anthropic only tokenizer 🤔?

Thanks for this comment! I didn't know about those headers. It definitely makes sense and I'll rework my implementation accordingly

The PR is updated! Now token counters are read from the headers in case of simple generation: The resulting LLMOutput will have the fields model_id and usage, containing the token counters.
NB In case of streaming, usage will be an empty dict as extracting the tokens is more complicated. Indeed, the headers are not available, and one would have to extract the token counters from the body, according to the model provider structure. Will do it at a later time :)

cc @3coins

Excellent work!
I'm curious about the differences between the use of "usage" in the response body versus in the headers. According to the Bedrock API documentation, the "usage" is included in the response body as follows:
{
    "id": string,
    "model": string,
    "type": "message",
    "role": "assistant",
    "content": [
        {
            "type": "text",
            "text": string
        }
    ],
    "stop_reason": string,
    "stop_sequence": string,
    "usage": {
        "input_tokens": integer,
        "output_tokens": integer
    }

}

Do you guys have any insights on this? @dmenini @esoler-sage @pratik60
Thanks!

I think usage in headers is for sync calls, and async call' last chunk contains usage in body 🤔


Mistaken commit label

Commit 3bb3f25 is titled "Reworked for GCC 12.2.0/x86_64" but should have been titled "Reworked for GCC 12.2.0/aarch64".

Added a comment on 3bb3f25.


Fix timeouts for subject-domicile-country TAD data export

Context
The data API is failing because there are no recent runs of this report. The query always times out. This PR adds a couple of optimisations to stop the timeouts.
Changes proposed in this pull request
Query optimisations TADSubjectDomicileNationalityExport

replace the NOT EXISTS with an anti-join. This seems to have been
the biggest performance issue.
replace array && comparisons with IN for the domicile CASE
clause.

Guidance to review

Is the anti-join equivalent to the NOT EXISTS? We only want to include applications that are either Apply 1 or the most recent Apply 2 application.

Link to Trello card
https://trello.com/c/eabMmT8l/4564-fix-data-api-issues-raised-by-tad
Things to check

[x] If the code removes any existing feature flags, a data migration has also been added to delete the entry from the database
[x] This code does not rely on migrations in the same Pull Request
[x] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[x] API release notes have been updated if necessary
[x] Required environment variables have been updated added to the Azure KeyVault

Reopening to trigger new build


Loss function

Thanks for this great work. I am currently trying to train fast-depth with my own dataset. I have noticed there is not training scripts. So I would like to ask, which depth losses are used in training?
It would be very nice, If anyone can give me a suggestion about which losses should I pick.

I was reading the paper and just wondering the same, came here and didn't find it either

I was reading the paper and just wondering the same, came here and didn't find it either

Hi, I will begin to try some loss function design begin next week. Once I have some useful results, I will report it here.

I read the paper and found that the author in the experimental part mentioned to follow the training method of "Sparse-to-dense: depth prediction from sparse depth samples and a single image"and L1 loss was used in that paper. I tried to train the network with L1 loss, and the result was very bad.

I used the "depth loss" in this paper and seems that the training is starting to converge.
.
I also suspect that I have a shitty dataset, and that is why I am getting so many noise (although you can see the depth more or less in some sense in the high level, there is a lot of noise,  piwelwise)

I used the "depth loss" in this paper and seems that the training is starting to converge.
.
I also suspect that I have a shitty dataset, and that is why I am getting so many noise (although you can see the depth more or less in some sense in the high level, there is a lot of noise, piwelwise)

which dataset are you using?


automatically adjust colormap_range in geoplot

This is a really nice and useful package, thanks for developing it!
When I plot a map from a geopandas df using plot_bokeh, as in
plot_html=df_merge.plot_bokeh(
figsize=(600, 600),
dropdown=['colA','colB'],
)
and select two columns in the dropdown menu, colA and colB, the colormap_range is automatically chosen to be the max range of both columns. If the ranges of both columns are very different, this only looks good for the column with the largest range, and not for the other. Is it possible to have the colormap_range automatically adjust to show the range of the selected column?

Hi,
nice to hear that you like Pandas Bokeh. I already tried to implement your enhancement proposal, however I couldn't directly find the javascript api for the colormapper. When I have time I will try to have a look at this, because I also would like this behaviour, or at least the option to see that.
Cheers,
Patrik

Hi,
Thanks a lot for looking into this. I'll keep an eye out for updates here.
Best regards,
Edo

Any luck on getting this one sorted out? It would be a great feature addition


provisioning: Use AWS CLI to automate provisioning

The previous steps for standing up a new host were somewhat manual.
This further scripts the process, by using the AWS CLI to start the
instance, and pass it a "user data" script to provision itself upon
boot.  This results in a hands-off provisioning process which
completes in 5min.
Additional settings are required for ~/.zulip-install-server.conf.
It is not suited for all roles, as it assumes one instance type and
security group value.  Additionally, not all of the post-provision
process is currently automated -- Nagios SSH key verification, for
instance, is still a manual step.  There are also additional steps for
database or frontend servers.  Regardless, this is a move toward
automated provisioning.

Testing Plan:
GIFs or Screenshots:

This is great, merged, thanks @alexmv!  Sorry for the delayed review.


Update reference table in docs

Notes for Reviewers
This PR fixes #10503
Signed commits

[x] Yes, I signed my commits.

🚀 Preview for commit 71d092fa158a4865f0a9e277bb432833d042278f at: https://65e899c176bd4a18ce8c149e--meshery-docs-preview.netlify.app

🚀 Preview for commit bd8071a614d5eb344534959c52c0f6cfcbb6ddbd at: https://65e89d23bd34661d63535177--meshery-docs-preview.netlify.app

@Althaf66 @RipulHandoo @vishalvivekm @hexxdump @aboobakersiddiqr63 @MUzairS15  @meshery/mesheryctl-contributors @meshery/mesheryctl-maintainers please be aware of this manual step - https://docs.meshery.io/project/contributing/contributing-cli#adding-newremoving-existing-commands-in-the-reference-index

Thank you, @alphaX86. Will you update the mesheryctl GitHub issue templates to include a checklist that has a link to this task? https://docs.meshery.io/project/contributing/contributing-cli#adding-newremoving-existing-commands-in-the-reference-index

Sure Lee, will update it. I'm thinking whether to automate the index itself to avoid this manual issue. Either via a script or with a help of library.


How come relayMessage doesn't have its own documentation?

I'm confused about how relayMessage has a reply, and how do users reply to bots without a prefix?

Ask in the discord server or github discussions. Issues arent the right place.

Ask in the discord server or github discussions. Issues arent the right place.

Where's that?

Ask in the discord server or github discussions. Issues arent the right place.

Where's that?

You can join discord here


【WIP】fix: Fix publish action version

What this PR does / why we need it:
Add your description
Which issue(s) this PR is related to (optional, link to 3rd issue(s)):
Fixes #
Reference to #

Special notes for your reviewer:
/cc @your-reviewer

Release note:

NONE

/assign @gaocegege @FogDong

/lgtm


Test concurrency

Codecov Report

Merging #7 (88cca54) into main (5ffdf30) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #7   +/-   ##
=======================================
  Coverage   50.52%   50.52%
=======================================
  Files          20       20
  Lines        2571     2571
  Branches     1478     1478
=======================================
  Hits         1299     1299
  Misses       1041     1041
  Partials      231      231

Flag
Coverage Δ

packchk-cov
50.52% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


gl_NumWorkGroups in HLSL

I'm trying to compile a shader that uses gl_NumWorkGroups, but it seems like HLSL doesn't have an equivalent semantic. Would there be another way to support this builtin? Since gl_NumWorkGroups should be known when the compute shader is dispatched, the calling program could probably store the value in a cbuffer and the shader could read it out, though that would require the calling program and the shader to agree on the format and binding location of the cbuffer.
Would this be feasible to implement in SPIRV-Cross?

yup, will need to emit some kind of special cbuffer. Shouldn't be too hard.


Update v3.13.0

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Closes #2
Closes #3
Closes #4
Closes #5

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

Hi @synapticarbors, this PR updates the recipe to the newest version and adds me as a maintainer, hope that's alright.

@BastianZim -- Thanks for taking care of this and apologies for neglecting this feedstock.

No worries, my pleasure!


Add Chroma-Key or "Green Screen" Keying-Out of a Color Range

The idea is to be able to use green screen footage in a game. The user would select a range of colors to be considered transparent. I understand this is probably doable via shaders. I am suggesting this a feature in Godot, since it is a media industry standard workflow.
I found an example of someones implementation of this in Godot, but I think it should be a built-in feature.

Since this is achievable with shaders, it might be better to simply implement a shader preset library of commonly used shaders (such as chroma key).

@GiveMeAnthony yes basically I am asking for a UI menu or something very similar that will apply this "Chroma-Key" shader. Like I said, its a very common industry standard thing that gets done in and outside of gaming. This should be built in. For myself, I did this with coding a little shader.

We are moving proposals to official Godot proposals repository, so closing this. The OP is a 👻, but if someone is interested in this feature, make a new proposal over there.

That is a very interesting idea I think! However, I think it will take a lot of effort to make something so good in the game, such as these videos https://greenscreenstock.com/, that they can then be edited.


Error: array subscript has type 'char' [-Weerror-char-subscripts] Cardputer.cpp:89:19

   for (auto& i : __keyboard->keysState().values) {         new_state[i] = 1;         if(i == 's') {             scale = !scale;         }     }
The error is with the  new_state[i] = 1;  specifically the i
This is stopping the compilation and I don't know why its throwing this error.

I'm getting this too! Anyway you can post the .bin?

The issue was fixed this morning, I'll try recompiling later and if it stops throwing the error I'll close the issue.

This issue was fixed, but I am still running into issues


Enhancement: Return clearer CLI responses for non-existent Service or User

When a service or user does not exist, the related remove command returns Service 'NX_Service' removed. or User 'NX_User' removed.   It is not clear that no action was taken.
For the 'show' command, it just returns None for both the service and user commands when it does not exist.  This could be more clear.

PR merged


[ll] Metal core port

Based on the code by @JohnColanduoni . The problem with original code was that it tried to re-use the command buffers already used for submission. Say thanks to @jrmuizel for investigating this ;)
The example either gets stuck or not releasing the resources properly, but it renders 🎆 . More work to follow, hopefully.

Let's see if CI is happy
bors r+

bors r+

oh, travis, can't wait for you too long
windows/linux builds are passing, good to go


NUTCH-2920 -- add an OpenSearchIndexWriter

…iter to OpenSearch
Thanks for your contribution to Apache Nutch! Your help is appreciated!
Before opening the pull request, please verify that

there is an open issue on the Nutch issue tracker which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
the issue ID (NUTCH-2920)

is referenced in the title of the pull request
and placed in front of your commit messages surrounded by square brackets ([NUTCH-2920] Issue or pull request title)

commits are squashed into a single one (or few commits for larger changes)
Java source code follows Nutch Eclipse Code Formatting rules
Nutch is successfully built and unit tests pass by running ant clean runtime test
there should be no conflicts when merging the pull request branch into the recent master branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled master branch.
if new dependencies are added,

are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
are LICENSE-binary and NOTICE-binary updated accordingly?

We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Nutch in general, please sign up for the Nutch mailing list. Thanks!

The fiddly part (for me) was setting up the rest client to deal with a trust store.
I followed https://opensearch.org/blog/connecting-java-high-level-rest-client-with-opensearch-over-https/
I had to update the keystore command in that blog post like so: keytool -importcert -file root_ca.der -keystore my-store.jks -storepass mystorepass -alias opensearch
If there's a better way of handling this, let me know.

I'm less than entirely thrilled with using stored strings for credentials, but that's where we were with Elasticsearch.  Again, if there's a better way, please let me know.

Needs more work on tls vs basic auth etc. Converting to draft.  Will update on Monday.

K, I think this is ready for review.  I'm happy for any and all input!

ok with the *-1x design?

Yes. As said, I haven't tested it with a running OpenSearch instance. If you can confirm that docs/fields appear in the index as expected, that's fine. Otherwise, I can try to run an OpenSearch instance (would be my first time), index a bulk of docs using various index filter plugins (add to the property plugin.includes index-(basic|more|anchor|geoip)). This would be a more realistic test scenario.

I've run this with both the docker "getting started" OpenSearch example  (e.g. docker run -d -p <IP_ADDRESS>:9200:9200 -p <IP_ADDRESS>:9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:1.3.8 with username=admin and pw=admin) and with the example for running OpenSearch locally with the trust store in the post above
The more tests the merrier!
We're using testcontainers over on Tika to test integration with OpenSearch and Solr.  Resource and time intensive, but really, really valuable to have those.  I can try to draft some testcontainers tests in a separate PR.

Sorry, what I intended by my question was: are you all ok with version numbers in the module name.  The current code is deprecated for 3x so I think we'll need to have a 3x at some point.  Or, do we want to target, say 2.x as 'indexer-opensearch' and hope it supports 1.x, etc...

Well, don't really know. Depends on how long we want to maintain it - upgrading and testing (manually, as of now). And also, how long users may want to keep it running.
For the Solr and Elastic indexers we always supported only one version (a recent one, but rarely the latest).

using testcontainers over on Tika to test integration with OpenSearch and Solr

Same for StormCrawler. Might be worth to have a look at the OpenSearch (2.5) module over there.

Merged. After reading about the OpenSearch releases, I agree to include the version number in the plugin's name. It would also allow us to start working on a 2x plugin now. But maybe we keep naming the plugin supporting the latest version without using a specific version number?


Maintenance: Parser inferred result always set event to array type

Summary
When using the Middy.js middleware for parser, customers can specify envelopes and schemas for the middleware to use to parse the event. To ease usage for customers, we introduced types that are supposed to infer the event type based on the arguments passed to the middleware.
Currently when using envelopes, the type of the event is always assumed to be an array when it should not be the case.
For example, this is correct:
import middy from '@middy/core';
import { parser } from '@aws-lambda-powertools/parser/middleware';
import { SqsEnvelope } from '@aws-lambda-powertools/parser/envelopes';
import { z } from 'zod';

export const handler = middy(async () => {
  return {
    statusCode: 200,
    body: JSON.stringify('Hello, World!'),
  };
}).use(parser({ schema: z.any(), envelope: SqsEnvelope }));

but this is not:
import middy from '@middy/core';
import { parser } from '@aws-lambda-powertools/parser/middleware';
import { EventBridgeEnvelope } from '@aws-lambda-powertools/parser/envelopes';
import { z } from 'zod';

export const handler = middy(async () => {
  return {
    statusCode: 200,
    body: JSON.stringify('Hello, World!'),
  };
}).use(parser({ schema: z.any(), envelope: EventBridgeEnvelope }));

In the second case, EventBridge's detail-type field is an object and not an array, but the types here think it's always an array.
Why is this needed?
So that the types when using the Middy.js middleware are correct.
Which area does this relate to?
Parser
Solution
No response
Acknowledgment

[x] This request meets Powertools for AWS Lambda (TypeScript) Tenets
[ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers
Please react with 👍 and your use case to help us understand customer demand.

This is strange. While WebStorm shows me the correct (non array) inferred type for both EventBridgeEnvelope and SqsEnvelope, VSCode shows an array of the custom schema also for both envelopes. I have also tested Zed, it has the same behaviour as VSCode (but faster 😄 ).


[UNDEDRTOW-1896] - fix parsing of trailing :: in IPv6

Issue: https://issues.redhat.com/browse/UNDERTOW-1896
Upstream PR: https://github.com/undertow-io/undertow/pull/1188

Bulk closing of 2.2.x PRs


Any plans to support Apple M1?

M1 is already supported in the latest release(1.0.2): https://cocoapods.org/pods/LookinServer


Hardware timers causes crash

I'm quite sure I have misconfigured the timer example I was trying, but it seems if you try to use one, the video output just stops.
I was mostly curious if you have run into this.
// code in vga.c

bool repeating_timer_callback(struct repeating_timer *t) {
    return true;
}

static void vgaLoop() {

    struct repeating_timer timer;
    add_repeating_timer_ms(500, repeating_timer_callback, NULL, &timer);

    while (1) {
        uint32_t message = multicore_fifo_pop_blocking();
        if (message & 0x01) {
            vgaParams.scanlineFn(message & 0xfff, rgbDataBufferOdd);
        } else {
            vgaParams.scanlineFn(message & 0xfff, rgbDataBufferEven);
        }
    }
}

I suspect the default alarm pool hardware timer IRQ handling is occurring on core 0. This is the same core which is handling the VGA DMA Interrupts (dmaIrqHandler()).
Note: add_repeating_timer_ms() is a wrapper for alarm_pool_add_repeating_timer_us() with alarm_pool_get_default() as the first argument.
You might need to manually create an alarm pool on core 1.
See

https://www.raspberrypi.com/documentation//pico-sdk/group__repeating__timer.html
https://www.raspberrypi.com/documentation/pico-sdk/group__alarm.html

Note: In the actual PICO-56 VGA code ( https://github.com/visrealm/pico-56/blob/main/src/devices/tms9918/vga/vga.c ) you can add a callback for the end of frame.
If you've still having issues using the hardware timer, you could use this to accumulate time in 16.67ms increments. There is also an end of scanline callback which I use to update the PWM audio. So, the PICO-56 audio frequency is tied to the VGA HSYNC frequency (~31KHz).

Ah, I didn't think about that.  I know very little about the timers on the Pico but I will study that some more.
But now that you mention it, I like the idea of the "end of frame" better anyway.  I needed a timer to flash a cursor but it makes more sense to just put a callback at the end of frame.
Thanks again for your help!  I will try not to pester you with things like this.  :-)

I don't mind. No problem at all.

Thanks again.  I've been taking your underlying source for the VGA and I've added sprite support, PETSCII graphics, text scrolling, etc.
I'm easily pushing 20 sprites per scanline.  Each sprite is 16xYY pixels tall (height definable) and 4 colors like NES (out of multiple palettes) all while on a 256 display.  This $4 Pico (RP2040) is incredible.


use getconf DARWIN_USER_TEMP_DIR rather than $TMPDIR

$TMPDIR isn't always available. Addresses #3258.
I don't think this should cause any issues with previous uses of $TMPDIR since it's the same value as DARWIN_USER_TEMP_DIR when set (in my limited testing).
$ echo $TMPDIR
/var/folders/np/5y4sgt8x3r13sk7x0n31z_3m0000gn/T/
$ getconf DARWIN_USER_TEMP_DIR
/var/folders/np/5y4sgt8x3r13sk7x0n31z_3m0000gn/T/

/cc @bdash @mrackwitz

:+1:

:+1: But shouldn't we just initialize TMPDIR with getconf DARWIN_USER_TEMP_DIR, if it is not set?

Not sure what that would net us

It'd respect a custom value for TMPDIR if the user has set one for some reason, as well as ensuring that any subprocesses that care about TMPDIR have a valid value. I don't know if any of that is super important though.

What @bdash wrote, DRY and better readability / self-documentation.


release: 1.1.1

:robot: I have created a release beep boop
1.1.1 (2022-10-09)
Performance Improvements

perfect use go 1.19 (97d4b8e)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/starudream/wake-on-lan/releases/tag/v1.1.1 :sunflower:


[Epic] Complete your user profile (first time user)

Milestone: Authorize
The shared data filing platform will be used by filers of small business lending (SBL) and mortgage lending (HMDA) data to centrally manage their account (user profile, financial institution details, login through Login,gov, etc.).
Epic: Complete your user profile (first time user)
Overview

The user will arrive at this page on their first visit after creating their Login.gov account and/or logging in with Login.gov
This screen will act as a gate to insure the user is fully configured prior to entering the rest of the system.
Once the user has a user profile and is authorized they will not see this page again.
For an authorized user any changes to their user profile will be handled on the "Request changes to your user profile" page.

Steps:

The user logs in through Login.gov and lands on this page
The user will be asked to enter their First name and Last name
Their email address will auto-populate based on the email address they used to sign in with Login.gov.
They will be able to associate with one or more financial institutions either by selecting from a pre-populated list of financial institutions/LEIs OR by filling out and submitting the Salesforce form.
Only the selections from the pre-populated list of financial institutions/LEIs that match their email domain will allow a user immediate access to the system.
Submissions via the Salesforce form will be handled by the support staff (SBL help).

Tasks

[x] Pull together the relevant user stories for this epic
[X] Make a list of the content requirements for this epic
[X] Make a list of the technical requirements for this epic
[X] Post current wireframes and mock-ups
[x] Review wireframes and mock-ups with FEWD, BEWD, Data, SBL Help, Content team members
[ ] Refine user stories to prepare for development phase
[x] Identify tasks for user testing (to validate findability of information, clarity of content and layout, users able to successfully complete important tasks)

User stories
Prioritized for MVP:

[x] #17
[ ] #18
[x] #19
[ ] #23
[x] #27
[ ] #62

Backlog

[ ] #22

Technical requirements

User management API

Reads data from and writes data back to Keycloak

Institutions API

Retrieve institution data by LEI and email domain

User testing
Task ideation
Scenario 1: The user sees all of the financial institutions they are authorized to file for

What do users expect the system to do to verify that they should be associated with an institution?
How much do users trust our system to correctly associate individuals with institutions?
Do users understand how to get additional help?

Scenario 2: The user does not see all of the financial institutions they are authorized to file for

What does the user expect to do so that they can be associated with their institution(s)?
Do users understand each piece of information we are asking them to provide?
What do users expect to happen after submitting this information?

Scenario 3: User logs in to Login.gov using a personal email address

Does the user understand what went wrong?
Does the user know what to do next?

Current designs (MVP)
Updated: 12/28/2023
Scenarios for financial institution association/email domain/email address
Scenario 1: The user sees all of the financial institutions they are authorized to file for

User logs in through Login.gov with their financial institution’s email address.
Front end calls the back end to check for personal email address. This passes.
The front end calls the back end to see if there are any matches between the user’s email domain and a financial institution email domain in our financial institution database.
The user's email domain matches the email domain of a financial institution in our financial institution database
The back end returns the list of matches and the front end presents those matches to the user.
The user sees a list of one or more matches and selects the financial institution(s) that they are authorized to file for by clicking the checkbox.
If the user does not see the financial institution that they are authorized to file for in the list they click on the "Contact support staff" link.
The user clicks submit to submit their first name, last name, and financial institution.
On the back end Keycloak gets updated with the user’s first name, and last name, and the user is added to the Keycloak group (the institution)
The user advances to the shared landing page (authenticated).

Scenario 1

Scenario 2: The user does not see all of the financial institutions they are authorized to file for

User logs in with their financial institution email address.
Front end calls the back end to check for personal email address. This passes.
The front end calls the back end to see if there are any matches between the user’s email domain and a financial institution email domain in our financial institution database.
On the back end the user’s email domain does not match the email domain of a financial institution in our database. The backend does not return any matches.
The front end displays a Salesforce form.
The user fills out the Salesforce form and submits to the support staff.
The support staff receives the form output via email.
[See specific help process - by the end of this process the user's email domain has been associated with the financial institution and their user profile is complete.
The user receives an email from the help team which includes a link to the shared landing unauthenticated page.
The user clicks the log in link, logs in with Login.gov, and advances directly to the shared landing (authenticated) page.

Salesforce form
Salesforce notification

Scenario 3: User logs in to Login.gov using a personal email address

User logs in to Login.gov using their personal email address
On the back end the user’s email address is checked against the deny list and matches one of the denied email domains.
User sees a notification message indicating that they must log in again using their financial institutions email address.
User returns to Login.gov and logs in with their financial institution email address.

User logs in with personal email address

Scenario 4: User submits form without filling out all the required fields

User submits incomplete form

@angelcardoz User testing section has been updated and the relevant checklist item has been marked complete.


Update discv5 to 2.1.1

Motivation
We want the fix from 2.1.1

:tada: This PR is included in v1.6.0 :tada:


Updating vsphere/README.md

Recommending  user to download tagged release instead of git clone https://github.com/kubernetes/kubernetes-anywhere repository
Informing user about not to install weave-net add on.
cc: @tusharnt @BaluDontu @abrarshivani

LGTM


"Invalid 'reference' directive syntax" Typescript2 @types references

VSCode Version: 1.3.1 (latest)
OS Version: Win7

Install typescript2 @types
Reference them

I'm trying to create the references for @types I've installed (TS2beta-latest), but vscode seems to have a problem with the way these are setup?
When looking at one that's downloaded (let's say browserify), it has the same problem.

Do you have a tsconfig.json file? Iff so can you share it here?

Closing since no additional information has been provided in the last 2 weeks.

Apologies @dbaeumer!
I forgot to close this one out! The issue was VSCode wasn't aware of TS2.0 and the tsdk had to be changed.  (For anyone else who runs into this issue).


Add industry "Production" variables

This PR is based on a variable and tag subset of #90 with some additional updates. It adds an updated version of the Production|* category of industry related variables.
Changelist:

the use of the Accounting Type tag has been dropped here. Commodity production expressed in monetary units first requires discussion and agreement with @IAMconsortium/common-definitions-macro-economy
The concept of commodity origin used for metal production has been adjusted for plastics
Several tag descriptions have been extended and improved
Tier classification system used for emissions has been introduced for commodity production

Thanks for the feedback, @danielhuppmann.
I understood the tag feature as a way to group variable elements that have some commonalities and to abstract them to some extent. In the case of primary and secondary production, the general definition from ISO applies to all metals. So I thought
- Non-Ferrous Metals Commodity:
    - Aluminum:
        description: aluminum, commonly produced by electrolysis of alumina or re-melting of scrap
    - Copper:
        description: copper, a soft and ductile metal with high thermal and electrical conductivity
    - Lithium:
        description: lithium, an alkali metal commonly produced through electrolysis of LiCl, mainly used for electrochemical energy storage

# Origin of metal according to ISO IWA 19:2017(E)
- Metal Origin:
    - Primary:
        description: extracted from minerals and free of reclaimed metal scrap
    - Secondary:
        description: that does not directly originate from a primary mineral but from a recycling process or from the processing of waste streams from primary production

- Production|Non-Ferrous Metals|{Non-Ferrous Metals Commodity}|{Metal Origin}:
    description: Production of {Non-Ferrous Metals Commodity} {Metal Origin}
    unit: Mt
    tier: 2

makes sense instead of writing out all 6 variables in production.yaml.
In the case of iron and steel sector, I agree that it does currently rather complicate things since we have steel as the only commodity with primary and secondary production. I did it only to be consistent with the other sectors.
So maybe this is overengineered and your suggested option to include "Primary" and "Secondary" directly in the commodity tag list is more practical in the end.
Yes, pig iron and sponge iron are subcategories of iron to my understanding. In NAVIGATE and ENGAGE they were not explicitely defined as subcategories. But I guess we could update to Production|Iron and Steel|Iron|Pig Iron and Production|Iron and Steel|Iron|Sponge Iron?

Right, there is no definitive answer on the best complexity of tag lists, but this implementation seems over-engineered to me. And you can also set the tier in the tag-list to distinguish between more and less important variables.
About pig iron and sponge iron: if these two are intended to sum up to Production|Iron and Steel|Iron, then I'd introduce the subcategory.
In any case, it would be very useful to add the variable-name from NAVIGATE so that we can implement automated translation and teams have an easier time updating the reporting scripts.

Alright, I have pushed a commit that reverts back to the simpler tag-list scheme. I will keep a backup branch with the more complex tag-lists on my fork.
I have put the tier field into the tags as you suggested, but it seems that nomenclature throws an error, when a tier value is an integer:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[21], line 1
----> 1 dsd = DataStructureDefinition(path="C:/Users\Florian Maczek\PycharmProjects\common-definitions\definitions", dimensions=["variable"])

File ~\anaconda3\lib\site-packages\nomenclature\definition.py:68, in DataStructureDefinition.__init__(self, path, dimensions)
     65 for dim in self.dimensions:
     66     codelist_cls = SPECIAL_CODELIST.get(dim, CodeList)
     67     self.__setattr__(
---> 68         dim, codelist_cls.from_directory(dim, path / dim, self.config)
     69     )
     71 if empty := [d for d in self.dimensions if not getattr(self, d)]:
     72     raise ValueError(f"Empty codelist: {', '.join(empty)}")

File ~\anaconda3\lib\site-packages\nomenclature\codelist.py:199, in CodeList.from_directory(cls, name, path, config, file_glob_pattern)
    173 @classmethod
    174 def from_directory(
    175     cls,
   (...)
    179     file_glob_pattern: str = "**/*",
    180 ):
    181     """Initialize a CodeList from a directory with codelist files
    182
    183     Parameters
   (...)
    197
    198     """
--> 199     code_list = cls._parse_codelist_dir(path, file_glob_pattern)
    200     config = config or NomenclatureConfig()
    201     for repo in getattr(
    202         config.definitions, name.lower(), CodeListConfig()
    203     ).repositories:

File ~\anaconda3\lib\site-packages\nomenclature\codelist.py:288, in CodeList._parse_codelist_dir(cls, path, file_glob_pattern, repository)
    285             code.repository = repository
    286         code_list.append(code)
--> 288 code_list = cls._parse_and_replace_tags(code_list, path, file_glob_pattern)
    289 return code_list

File ~\anaconda3\lib\site-packages\nomenclature\codelist.py:169, in CodeList._parse_and_replace_tags(cls, code_list, path, file_glob_pattern)
    167 # replace tags by the items of the tag-dictionary
    168 for tag_name, tags in tag_dict.items():
--> 169     codes_with_tags = cls.replace_tags(codes_with_tags, tag_name, tags)
    171 return codes_without_tags + codes_with_tags

File ~\anaconda3\lib\site-packages\nomenclature\codelist.py:117, in CodeList.replace_tags(cls, code_list, tag_name, tags)
    115 for code in code_list:
    116     if "{" + tag_name + "}" in code.name:
--> 117         _code_list.extend((code.replace_tag(tag_name, tag) for tag in tags))
    118     else:
    119         _code_list.append(code)

File ~\anaconda3\lib\site-packages\nomenclature\codelist.py:117, in <genexpr>(.0)
    115 for code in code_list:
    116     if "{" + tag_name + "}" in code.name:
--> 117         _code_list.extend((code.replace_tag(tag_name, tag) for tag in tags))
    118     else:
    119         _code_list.append(code)

File ~\anaconda3\lib\site-packages\nomenclature\code.py:121, in Code.replace_tag(self, tag, target)
    118 if isinstance(value, str) and "{" + tag + "}" in value:
    119     # if the the target has the corresponding attribute replace
    120     if attr in target.flattened_dict:
--> 121         mapping[attr] = value.replace(
    122             "{" + tag + "}", getattr(target, attr)
    123         )
    124     # otherwise insert the name
    125     else:
    126         mapping[attr] = value.replace("{" + tag + "}", target.name)

TypeError: replace() argument 2 must be str, not int

As far as I can see there are no other tag-lists with tier fields so far, so maybe this has not been considered in nomenclature development yet. As a quick hack, I have put the tier values in quotation marks for now. e.g:
- Non-Metallic Minerals Commodity:
    - Cement:
        description: a binder substance used for construction
        tier: "1"

I will add navigate/engage fields with the next commit. For some industry sectors, NAVIGATE and ENGAGE templates only contain Production|{Sector Name}|Volume, which I find too general since it cannot be inferred, which commodities of the sector are actually included in the aggregate. Therefore I think we should not include them in this PR here. That means there will not be a navigate/engage mapping for several variables.

I have added the engage/navigate fields now. All equivalent NAVIGATE and ENGAGE variables should be mapped now. However since this PR adds some new commodities, the tag feature also creates navigate and engage values for all tags, while some of them are actually not existent in the legacy templates.
e.g. Production|Non-Metallic Minerals|Glass gets the navigate variable Production|Non-Metallic Minerals|Glass|Volume, which has not been in the NAVIGATE template because Glass is a newly added commodity.
Maybe this could be fixed by implementing a feature in nomenclature, that allows to set the navigate/engage value to a keyword that indicates that this tag does not have legacy equivalents and tells nomenclature not to create a navigate/engage entry for the variable. But I am not familiar enough with nomenclature to judge whether that would be a sensible feature.

@macflo8, I took the liberty to commit my review-suggestions directly to your branch so that I can go ahead and merge your PR.


Release v0.16.1

v0.16.1 Dec. 1, 2020
Enhancements

Pin woodwork version to v0.0.6 to avoid breaking changes #1484

Fixes

Updated Woodwork to >=0.0.5 in core-requirements.txt #1473
Removed copy_dataframe parameter for Woodwork, updated Woodwork to >=0.0.6 in core-requirements.txt #1478
Updated detect_problem_type to use pandas.api.is_numeric_dtype #1476

Changes

Changed make clean to delete coverage reports as a convenience for developers #1464

Documentation Changes
Testing Changes

Update dependency update checker to use everything from core and optional dependencies #1480

Talked with @freddyaboulton @angela97lin about the windows failure. We're seeing it on other PRs too. Given that the windows tests have been passing on main, including on the last merge, I think we should continue the release. I suspect the test failure is due to a network hiccup.


NPM and browserify compatibility

add package.json and module.exports to allow for browserify compatibility.

The script uses the "use strict" feature. This patch breaks usage without NPM. You have to pass global variables as parameters.

I suggest that the script use a UMD wrapper instead, which should come as a standard option for frontend packages nowadays. This little amendment to the standard wrapper should get you the global context:
;(function (root, factory_) {
  function factory($) { return factory_($, this); } /* pass `window` here */

  if (typeof define === 'function' && define.amd) {
    define(['jquery'], factory);
  } else if (typeof exports === 'object') {
    module.exports = factory(require('jquery'));
  } else {
    root.wysiwyg = factory(root.jQuery);
  }

}(this, function ($, window) {
  "use strict";

  $.fn.wysiwyg = ...;
  window.File ...;

}));

added a 'dist' directory


Changed selection direction definition - resolves microsoft/TypeScript#32449

Regression from #340
Addresses microsoft/TypeScript#32449

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: BuckAMayzing sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


[REGRESSION]: NuGetAuthenticate failing when updating existing credentials

New issue checklist

[ ] I searched for existing GitHub issues
[ ] I read pipeline troubleshooting guide
[ ] I checked how to collect logs

Task name
NuGetAuthenticate
Breaking task version
1.244.1
Last working task version
1.238.1
Regression Description
We have pipelines that authenticate to NuGet repos multiple times which now fail with a validation error: "##[error]Error: An existing service connection already exists for the endpoint". This seems to have been added in #20299 by @embetten . We have multiple layers of templates that may authenticate to our repo using different credentials to fetch different things which seems like a very valid thing to do to me. Sorting out all those to restructure them and only authenticate once would be very tricky for us.
Environment type (Please select at least one enviroment where you face this issue)

[ ] Self-Hosted
[X] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Windows 11
Relevant log output
##[error]Error: An existing service connection already exists for the endpoint ...

Full task logs with system.debug enabled
No response
Repro steps
Authenticate to a nuget repo twice in the same pipeline.

This scenario should be fixed with #20344 (really updated here where we allow existing service connections to be overwritten in the environment variable.
It is also more strictly ensuring only one entry can be added per feed URLs to avoid confusion around multiple entries.
let me know if this covers all scenarios that were breaking.


Fix module-linking handling of instance subtypes

When we alias the nth export of an instance, due to subtyping the nth
export may not actually be what we want. Instead we need to look at our
local type definition's nth export's name, and lookup that name off the
export.

This is built on https://github.com/bytecodealliance/wasmtime/pull/2461 so the last commit is all that's relevant here.