[BugFix] Fix replay TransactionState log error (#27383)

backport #27383

https://github.com/Mergifyio backport branch-3.1

[FE PR Coverage Check]
:disappointed: fail : 2 / 12 (16.67%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/transaction/LakeTableTxnLogApplier.java
0
6
00.00%
[84, 85, 86, 87, 88, 89]

:large_blue_circle:
com/starrocks/transaction/OlapTableTxnLogApplier.java
2
6
33.33%
[162, 163, 164, 165]

https://github.com/Mergifyio backport branch-2.5


[RIP-46] Enhanced metrics for timing and transactional messages

Which Issue(s) This PR Fixes

Some enhanced metrics of RIP-46, link to #5350
Brief Description
The following metrics related to timing messages have been added:

rocketmq_timing_end_messages_total, COUNTER, records the number of timing messages that have ended, aggregated by topic.
rocketmq_delay_message_latency, HISTOGRAM, records the distribution of timing message delay durations when sent, aggregated by topic.
rocketmq_timer_message_snapshot, GUAGE, records the distribution of timing messages in the timing wheel (right now). Not aggregated by topic.

The following metrics related to transactional messages have been added:

rocketmq_half_messages, GAUGE, records the number of half messages in transactional scenarios, aggregated by topic.
rocketmq_commit_messages_total, COUNTER, records the number of messages committed in transactions, aggregated by topic.
rocketmq_rollback_messages_total, COUNTER, records the number of messages rolled back in transactions, aggregated by topic.
rocketmq_transaction_finish_latency, HISTOGRAM, records the distribution of delay times for committing transactional half messages, aggregated by topic.

In addition, to calculate metrics related to transactional messages, a new data structure called TransactionMetrics has been added to record the number of transactional half messages under each topic.
How Did You Test This Change?

I will provide a test report later.

Codecov Report

Merging #7500 (ca74b6b) into develop (d7e5c4d) will increase coverage by 0.47%.
Report is 32 commits behind head on develop.
The diff coverage is 35.80%.

@@              Coverage Diff              @@
##             develop    #7500      +/-   ##
=============================================
+ Coverage      42.53%   43.01%   +0.47%
- Complexity      9441     9698     +257
=============================================
  Files           1150     1160      +10
  Lines          82746    84219    +1473
  Branches       10778    10936     +158
=============================================
+ Hits           35196    36226    +1030
- Misses         43139    43499     +360
- Partials        4411     4494      +83

Files
Coverage Δ

...apache/rocketmq/broker/BrokerPathConfigHelper.java
81.25% <100.00%> (+1.25%)
:arrow_up:

...rocketmq/broker/metrics/BrokerMetricsConstant.java
0.00% <ø> (ø)

...etmq/broker/processor/EndTransactionProcessor.java
56.54% <100.00%> (+2.91%)
:arrow_up:

...etmq/remoting/metrics/RemotingMetricsConstant.java
0.00% <ø> (ø)

...tmq/store/metrics/DefaultStoreMetricsConstant.java
0.00% <ø> (ø)

...apache/rocketmq/store/timer/TimerMessageStore.java
53.97% <100.00%> (+0.40%)
:arrow_up:

...ueue/DefaultTransactionalMessageCheckListener.java
81.81% <0.00%> (-2.56%)
:arrow_down:

...saction/queue/TransactionalMessageServiceImpl.java
51.75% <85.71%> (+0.35%)
:arrow_up:

...er/transaction/TransactionMetricsFlushService.java
88.23% <88.23%> (ø)

.../java/org/apache/rocketmq/common/BrokerConfig.java
27.19% <25.00%> (-0.02%)
:arrow_down:

... and 5 more

... and 118 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

LGTM～


Set up project structure

This includes TypeScript, Webpack (for building the frame), ng-packagr (for building the library), Karma (for testing on Chrome), ESLint, Prettier, and GitHub Actions (for CI).
Also, switch to using the term "frame" instead of "binary"; the term "binary" to mean a compiled JavaScript file is a Bazelism not widely understood elsewhere.

(I don't currently have permission to merge PRs even with approval.)

(I don't currently have permission to merge PRs even with approval.)

Made you a maintainer, so you should have permission now.


Wireplumber shows on bar but does not update its volume level until connect and disconnect bluetooth headset.

Hello, and thank you for bringing us Waybar.
Wireplumber module shows on waybar at first instance but the module does not update volume state until I connect and disconnect for example a bluetooth headset. After that, module shows volume as it should be from the start.
Thanks in advance.

Can confirm same issue here, anything we can provide to help resolve this? Or is there to a way to configure the module?

What version of waybar are you running?
The latest release (0.9.17) was on Jan 11 and does not have this fix https://github.com/Alexays/Waybar/commit/4e8ccf36b54cacf5281726d23ea14312a133f977
Can you try compiling from source to see if that fixes your issue?

Yes, that is the first thing I did, I did also today just to see if something changed but no. Neither 0.9.17 nor latest sources fix the issue.
Same issues as this one popped out:
https://github.com/Alexays/Waybar/issues/2086

Hovering the mouse on top of the icon shows "Loopback PCM"

some useful logs:
[2023-04-15 17:55:12.396] [warning] [wireplumber]: (onMixerChanged) - Object with id 68 not found
[2023-04-15 17:55:12.397] [warning] [wireplumber]: (onMixerChanged) - Object with id 68 not found
[2023-04-15 17:55:12.472] [warning] [wireplumber]: (onMixerChanged) - Object with id 68 not found

Hey @smoak! I just compiled from sources and now is working as it should be! Many thanks! I will keep this issue opened until the next release is out to keep testing it.

-This is still happening when changing volume with wireplumber as it seems everytime I press the volume +/- key it prints each time 132 in theese lines which is related to the volume key I guess:
(Each time I press it prints one line)
sway.log
`[2024-01-05 17:29:03.311] [warning] [wireplumber]: (onMixerChanged) - Object with id 132 not found
[2024-01-05 17:29:03.316] [warning] [wireplumber]: (onMixerChanged) - Object with id 132 not found
[2024-01-05 17:29:03.323] [warning] [wireplumber]: (onMixerChanged) - Object with id 132 not found
[2024-01-05 17:29:04.679] [warning] [wireplumber]: (onMixerChanged) - Object with id 132 not found
[2024-01-05 17:29:04.682] [warning] [wireplumber]: (onMixerChanged) - Object with id 132 not found
`
-When I press the mute key nothing is printed.
-When I press the key to mute the mic, then 51 is printed:
(Each time I press it prints one line)
[2024-01-05 17:32:09.321] [warning] [wireplumber]: (onMixerChanged) - Object with id 51 not found [2024-01-05 17:32:09.914] [warning] [wireplumber]: (onMixerChanged) - Object with id 51 not found [2024-01-05 17:32:10.142] [warning] [wireplumber]: (onMixerChanged) - Object with id 51 not found
So this is still around.
Many thanks again for improving WayBar and for taking a look into this.

Can confirm, I have the same problem with the volume level not updating when connecting Bluetooth headphones.

I just tried building master (using arch AUR, commit 21906f0) and I can't reproduce the bug there, seems to be working perfectly.
@izmyname @mrpingo does it work for you?

Not on top of my priority to test this, for now, I'm sorry.

I just tried building master (using arch AUR, commit 21906f0) and I can't reproduce the bug there, seems to be working perfectly.
@izmyname @mrpingo does it work for you?

yup

Also for me! Thanks!


Mac support

Hi, this is an incredible project. I was just wondering if there's a seperate sequence of steps or something like that needed for this to run on a Mac?

Simple Instructions for cli have been added. Make sure Python and pip are installed properly on your system

can i close the issue?


gemspec should accept a glob option in its argument hash

This allows a glob option for gemspec to pass through.
An example case for when this is useful would be a lazy static test suite where running bundle exec rake test for a second time is a no-op and changing some source files will only cause its dependent test to be rerun on the next bundle exec test. That can be done by mapping test outputs to textfiles such as something_test.rb.txt (mainly for the timestamp ) and a) keeping file dependencies of tests explicitly in rake/make/whatever b) plugging in a build system like tup or fabricate.py, which handles such dependencies implicitly by tracking file reads that take place during each build (build = test run). This can be a huge time-saver for repos with large test-suites that take a long time, especially if the tests only require the part of the app they really need.
Therein is problem with current bundler implementation without this patch. If in a repo that has a Gemfile with a gemspec in it, running bundle exec to invoke a test will cause  a call to Dir['{,,**}.gemspec'] (DEFAULT_GLOB), which will recursively scan the whole repo for more gemspec files. Since bundle exec would be used to invoke each test, this would make, in the eyes of build system that track file reads, each test dependent on every other file (it does for tup, anyway). Allowing the glob option to pass through and giving  it a value of  '.gemspec' solves this problem, and it's only a small change to the codebase for that can be used to save a lot of wasted CPU time.

Thanks for the contribution! Unfortunately there is a failing spec (I think because it's now passing other options from opts that it wasn't before.)
Also, would you be able to update the man page for Gemfile with the new option? Thanks!

Thanks for willingness to accept it.
Interesting, it complained of a "development_group"  not being listed as an allowed option, but my git diff shows "development_group" isn't even listed as such in the main master branch.
I added it (+ran the options through sort) an it's green now. The last commit only documents the new option in the man page so it should be green too.

I don't think that's the right way to fix this. valid_keys is about what options are valid to pass to the gem method, and development_group is only valid for the gemspec method. The issue is that with your change, all of the options to gemspec are now passed through to gem. It would be better to pass through glob explicitly.

Done

All green. Kind of fragmented, but there's git diff origin/master and git merge --squash.

I think we need to add tests for this new option.
Also it would make it easier for us, you can squash the commits on the PR.
thanks @pjump

Squashed here (gemspec_options2 branch):
https://github.com/pjump/bundler/tree/gemspec_options2

As for the test, this patch doesn't really do much -- the glob option already exists underneath. This only allows it to be accessed via the gemspec method.
Building upon path_spec.rb I would just test that the option gets passed through with something like:
describe "bundle install with explicit globs" do
  it "supports gemspec syntax with an explicit glob" do
    build_lib "foo", "1.0", :path => lib_path("foo") do |s|
      s.add_dependency "rack", "1.0"
    end

    expect(Dir).to receive(:[]).with('*.gemspec')

    install_gemfile <<-G
      source "file://#{gem_repo1}"
      gemspec :path => "#{lib_path("foo")}", :glob => '*.gemspec'
    G
  end
end

This would, however, require rspec-mocks. I'm not really that familiar with bundler's codebase to do it without it.

@pjump the test looks good.
bundler uses rspec-mocks already.

While you won't be able to use expect(Dir).to receive(:[]).with('*.gemspec'), you can create a gemspec that you don't want to be found, and check to make sure that that gem is not found. Maybe something like build_lib "bad", "1.0", :path => lib_path("foo/bad").

Most of the existing tests in Bundler are end-to-end integration tests, and I think we'd want at least two of those to verify that the feature works as intended and isn't broken by future changes:

a test that installing a Gemfile using a path glob that matches successfully can use the gem correctly
a test that installing a Gemfile fails with a path gem that only has a gemspec that is not matched by the glob... in other words, a test that the glob is actually respected. The test should fail if run against the current master.


Add animated navigation bar for light and dark mode

I would like to request the addition of a new feature: an animated navigation bar that supports both light and dark mode. The design for this feature can be found in this Figma file:
For light mode -
https://www.figma.com/file/Lhmy3Wlvdo1fQeYgmHp6Ry/animated-mobile-navigation-give-away?node-id=0%3A1&t=8UWmPxb6kYDBpQ6I-1
For Dark Mode -
https://www.figma.com/file/5qmXvL7Dm3v2pHi12FWnNe/Flutter-Ui?node-id=119%3A1260&t=hsgBoTbJQNNhFaut-1
The navigation bar should smoothly transition between light and dark mode based on the user's system settings. It should also include support for icons, labels, and customizable colors.
This feature would be a great addition to the repository and would greatly enhance the user experience. Please let me know if you need any additional details or requirements for this feature.
screenshots

Hey, I would like to work on this issue. Can you please assign this issue to me.

Sure ,I am assigning this issue to you @Uttkarsh-raj  . Best of luck .

@Uttkarsh-raj send me a new pr


spack-stack does not build on MacOS Sonoma with Command Line Tools 15.x

Describe the bug
First of all, I think we are okay with MacOS versions earlier that Sonoma (such as Ventura with Command Line Tools 14.x) so this issue is limited to Sonoma.  For example the spack-stack CI tests building on MacOS Ventura with Command Line Tools 14.x - apple-clang 14.0.3).
I've tried two configurations so far:

Sonoma 14.6.1, Command Line Tools 15.1
In this case musl fails to build, and I'm seeing make errors like this:

 >> 193    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 194    make[1]: *** [lib/autotest/autotest.m4f] Error 1
     195    make[1]: *** Waiting for unfinished jobs....
  >> 196    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 197    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 198    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 199    make[1]: *** [lib/autoconf/autoconf.m4f] Error 1
  >> 200    make[1]: *** [tests/wrapper.in] Error 1
  >> 201    make[1]: *** [lib/m4sugar/m4sugar.m4f] Error 1
  >> 202    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 203    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 204    make[1]: *** [lib/m4sugar/m4sh.m4f] Error 1
  >> 205    make: *** [all] Error 2

Sonoma 14.6.1, Command Line Tools 15.3
In this case autoconf fails to build with the following cmake  error:

  >> 193    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 194    make[1]: *** [lib/autotest/autotest.m4f] Error 1
     195    make[1]: *** Waiting for unfinished jobs....
  >> 196    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 197    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 198    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 199    make[1]: *** [lib/autoconf/autoconf.m4f] Error 1
  >> 200    make[1]: *** [tests/wrapper.in] Error 1
  >> 201    make[1]: *** [lib/m4sugar/m4sugar.m4f] Error 1
  >> 202    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 203    autom4te: error: need GNU m4 1.4 or later: /usr/bin/m4
  >> 204    make[1]: *** [lib/m4sugar/m4sh.m4f] Error 1
  >> 205    make: *** [all] Error 2

To Reproduce
Steps to reproduce the behavior:
On a Somoma MacOS, follow instructions to build spack-stack using the latest develop branches from JCSDA/spack-stack and JCSDA/spack
Expected behavior
A clear and concise description of what you expected to happen.
The spack-stack build completes successfully
System:
What system(s) are you running the code on?
Somona 14.6.1, Command Line tools 15.1
Somona 14.6.1, Command Line tools 15.3
Additional context
Add any other context about the problem here.

Just discovered that my installation of Command Line Tools 15.3 was corrupt, which was impacting m4. I'm going to start over with a fresh slate.

Regarding musl. You need to use an external libiconv as provider for iconv. I explained this previously, but the old libc is no longer allowed as provider for iconv, glibc has a conflict with apple-clang/darwin, and neither musl nor libiconv build on macOS with apple-clang. External libiconv from homebrew fixes this.
About m4, I have clang version 15.0.0 on sonoma 14.4 and that works (builds m4). If your external m4 approach works, that's great, otherwise I'd stay away from 15.3 and use 15.0 or 15.1 with external libiconv.

Regarding musl. You need to use an external libiconv as provider for iconv. I explained this previously, but the old libc is no longer allowed as provider for iconv, glibc has a conflict with apple-clang/darwin, and neither musl nor libiconv build on macOS with apple-clang. External libiconv from homebrew fixes this.
About m4, I have clang version 15.0.0 on sonoma 14.4 and that works (builds m4). If your external m4 approach works, that's great, otherwise I'd stay away from 15.3 and use 15.0 or 15.1 with external libiconv.

I have tried using the homebrew libiconv (buildable: false)  but I couldn't get spack-stack to stop trying to build musl. I don't think I was on the exact Sonoma 14.4 version (might have been 14.4.1 or .2) so perhaps that was the issue. Is there a way to back up Sonoma?

Did you set the provider for iconv to libiconv?
I don't know how to back up Sonoma and I wouldn't go there, probably way more complicated than convincing spack to use libiconv.

I'm pretty sure I had the provider for iconv set to libiconv, but I can double check that when I'm rebuilding.
It looks like my m4 (/usr/bin/m4) is corrupt - it won't run when I try mv --version.  I'm going to try the homebrew m4 first.

For the musl issue, I have this in my spack.yaml
packages:
    all:
      prefer: ['%apple-clang']
      compiler:
      -<EMAIL_ADDRESS>      providers:
        mpi<EMAIL_ADDRESS>        iconv: [libiconv]
    ewok-env:
      variants: +mysql

But concretize still shows musl as a package to be built. musl is listed as a dependency of gettext and libxm2. I suppose I could homebrew install these and use them as external packages.  I'm only seeing this issue with Command Line Tools 15.1. Command line tools 15.3 ended up without musl (even without specifying libiconv as the provider of iconv).
At this point 15.3 is offering less resistance, after using homebrew mv, and I'll keep pursuing 15.3 for now. The install is still running.

Weird. I wonder if double colons would help

I agree that this is weird. There has been a lot of weirdness with the Mac lately.
What do the double colons do?

My build on 15.3 got quite far (294 out of 318 packages). I'm now seeing this with eckit:
In file included from /Users/steveherbener/spack-stack/cache/build_stage/spack-stage-eckit-1.       24.5-zc5kjgjtnonxnsrienad3dpnbdqpavr4/spack-src/src/eckit/system/ResourceUsage.cc:15:
205029 /Users/steveherbener/spack-stack/cache/build_stage/spack-stage-eckit-1.24.5-zc5kjgjtnonxnsrienad3dpnbdqpavr4/spack-src/src/eckit/system/ResourceUsage.h:30:5: error: unknown type name 'size_t'
205030     size_t maxResidentSetSize() const;
205031     ^
205032 /Users/steveherbener/spack-stack/cache/build_stage/spack-stage-eckit-1.24.5-zc5kjgjtnonxnsrienad3dpnbdqpavr4/spack-src/src/eckit/system/ResourceUsage.h:32:5: error: unknown type name 'size_t'
205033     size_t numberOfSwaps() const;
205034     ^
205035 /Users/steveherbener/spack-stack/cache/build_stage/spack-stage-eckit-1.24.5-zc5kjgjtnonxnsrienad3dpnbdqpavr4/spack-src/src/eckit/system/ResourceUsage.h:43:5: error: unknown type name 'size_t'
205036     size_t factor_;
205037     ^

Looks like size_t is not defined in the global namespace. Not sure why this changed.

This error was fixed in<EMAIL_ADDRESS>(and newer) releases: https://github.com/ecmwf/eckit/pull/93
What is the appropriate way to deal with this for the 1.8.0 release?

Move to eckit 1.25.0 (or newer) for all platforms - ie, update the eckit require spec in common/packages.py
Have macos only instructions say to change the eckit version in common/packages.py to 1.25.0

I suggest updating to<EMAIL_ADDRESS>unless there is a good reason not to do so


[Perf] Enable profiling in perf pipelines

Would be useful to start collecting a history of profiling data that can be used to pinpoint causes for any performance regressions. The PerfStress framework has the ability to profile with cProfile. We should start using this functionality within the currently active performance pipelines.
Related issue for profiling improvements: https://github.com/Azure/azure-sdk-for-python/issues/27380

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Data Factory:0.30407637,Storage:0.11652765,Azure.Core:0.030107887'

Support was added in https://github.com/Azure/azure-sdk-tools/pull/5533/files


Crash at -40, 124

Possible video crash

desktop_crash_-40,124_Player.log

I was trying to reproduce this in mac, I've been walking around the scene and it doesn't crash but I've realised that the scene that appears in the screenshot is no longer there, is there a different place where this was happening? Otherwise I think we have no way of reproducing it anymore

the only relevant errors that I see in that player log are some errors regarding the broken 'settings controller' prefab, those problems should be fixed at
https://github.com/decentraland/explorer-desktop/pull/300
https://github.com/decentraland/unity-renderer/pull/2316
Hopefully those fixes will solve this crash.
Since we cannot reproduce it anymore because that scene is not there anymore, we'll be closing the issue.


iterator limit doesn't match malloc in protocol list code

as @stevenh said in comments of #13,
In this code:
	sorted_types = (server_type **)malloc(sizeof(server_type*) * n_server_types);
	type = &types[0];
	for (i = 0; type->id != Q_UNKNOWN_TYPE; type++, i++) // ← HERE
	{
		sorted_types[i] = type;
	}

The iterator limit doesn't match malloc.
This issue is there since the initial revision.

This code must be refactored to be usable by some other parts (for example the json protocol listing).

I will probably not have time to code the 8 next days but I can merge existing code and re-push, so if you have a ready-to-use replacement I can copy-past it here and in qstat.c and add the commit to the PR.

@stevenh is this issue can be fixed? It's a 13 years old qstat issue and it blocks 5 pull requests to be merged.
The bug is still there in the last revision: https://github.com/multiplay/qstat/blob/2a95a5d04c8bed54b1a035607989dab101698ce4/qstat.c#L2655


ci(gha): remove job outputs

As raised by @rajatjindal in discord:

The java-dev and javascript-dev actions are failing on GitHub actions with error:
Error: Fail to evaluate job outputs
Error: The template is not valid. System.InvalidOperationException: Maximum object size exceeded

An example failure: https://github.com/dagger/dagger/actions/runs/11887650112/job/33120760976?pr=8973

The gha module was generating outputs for some reason - these outputs contained all of stdout and stderr, which could sometimes exceed the total allowed object size in github actions:
Error: The template is not valid. System.InvalidOperationException: Maximum object size exceeded

However, it didn't seem like we were ever using these outputs - we still need to capture them for the process of producing the summary report, but we don't actually seem to need the job outputs - so I've just gone and removed them, which should hopefully fix the issue.
I don't think we need them @shykes? I suspect they were leftover from an earlier refactor, but hopefully you can confirm?

Merging this now to unblock some broken CI - if it's discovered later that we actually needed these outputs (for some reason that escapes me), then we can evaluate options from there.


(fix): Fix Go snippets for optional primitive aliases

Simple bug fix for supporting aliases to optional primitive values. With this, the generated snippet will include the top-level pointer helpers (e.g. acme.String(...)).
Note that this includes a new test case within the examples definition, so all of the seed snapshots are updated for every generator.

[enum]:                              Found 7 errors and 0 warnings. Run fern check --warnings to print out the warnings.

Looks like fern check is unhappy with the enum test definition. We can fix this in a follow-up PR targeted at that fix (especially given that it will update fixtures across every generator).


cawbird: 1.0.2 -> <IP_ADDRESS>

Motivation for this change
simple version bump of the cawbird client. Has been done as part of some further backports for release-19.09, see the according PR for that.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build cawbird

@GrahamcOfBorg build cawbird

arch failure related to ghostscript


Fix +sign used on counters and parsing error

This pull request fixes two issues:

sign could not be used with c type metrics

If a metric has an error while parsing, the rest of the buffer was not consumed. Until another event have been received.

@luca3m Could you please add an integration test for both of these changes? Thanks!

@armon done

Thanks!


Improve error message when clientViaApplicationDefaultCredentials fails to log in

Fixes #572

I'll admit, I'm not sure how to test that. Any suggestion?

@kevmoo Gentle bump :)
Nothing urgent don't worry


Show how to toggle passthrough mode on supported headsets using underlying SDKs

Following on from the multiple SDK tutorial, show how can toggle the passthrough mode on any SDK that supports it.

https://www.youtube.com/watch?v=DXHsg-9ThkM


Timeout? - took to long to login

If you leave the application sit for a while on the login page and then try to login, you get a message saying took to long to login.
Can we remove that and just make the form work?

Nope.  In order for the OIDC handshake to work, the code passed to the login screen has to be kept around for subsequent phases of the handshake, and that code has to have a timeout.
We could increase the timeout if it seems to short now.

We could increase the timeout if it seems to short now.

What is it set to now? I do think it should be increased.

@hkeeler - can you point me to this setting so I can take a look?

@hkeeler - I think #146 is good for this, yeah?


Add the missing canvas.save and canvas.restore

This should fix https://github.com/flutter/flutter/issues/18708 and https://github.com/flutter/flutter/issues/18713

CC @ebeem

CC @andreidiaconu : please let me know if this fixes your issue https://github.com/flutter/flutter/issues/18713

Oops. That was Dart. Don't mind me :D


The full name attribute is missing in the VC detailed view

Describe the bug
The full name attribute is missing in the VC detailed view
To Reproduce

Open app and head to home page
click on add ID and download VC
Go back to home
Open the downloaded VC card

Expected behavior
All the atribute should be listed properly
Current behavior
The full name attribute is missing in the VC detailed view
Screenshots

Smartphone (please complete the following information):
Device: android
MOSIP Version: 121
MOSIP server: qa-121.mosip.net
APK link : https://github.com/mosip/inji/suites/8476299188/artifacts/376688868

@santhoshsunder @Anushree09-N why is this a critical bug? Please elaborate

verified this issue in qa5 env, the issue is not observed any more, we are able to see full name attribute in the detailed view, adding the screen show bellow, hense closing the jira
Screenshot :

Smartphone :
Device: PIXEL 6
OS: Android 13
MOSIP Version: <IP_ADDRESS>
MOSIP server: qa5.mosip.net
APP link; https://mosip.atlassian.net/browse/DSD-1816


Would you like me to add azure as a feature to datafusion-tui?

Im working on datafusion-tui which is a tool to facilitate interactive data analysis from various sources.  If you would find value in it I can add support for this before your 0.1 release by pinning to git.  If not, i'll just wait until that release.

Just recently came across datafusion-tui, but it looks like a very useful tool that could make my life a lot easier in many cases :). So I'd be happy to give it a try and for this an integration would be very useful.

The tests are passing which are more or less equivalent to what is done in the S3 object store. There are however some updates / improvements i want to make. Not sure what your time-frames are, but I am hoping to get to fixing them so, and itegrate the store with our solution - so maybe it could be one of the last items to tackle before a release :)


[WebJobs][EventGrid] Support Cloud Events in EventGrid WebJobs Extensions

There are a few moving pieces here:

[ ] For the trigger binding, the handshake between the service and the listener when creating a subscription is different. Instead of sending a "SubscriptionValidationEvent" and expecting a certain response object, the service instead follows a different handshake where an OPTIONS request is sent to the underlying webhook (see: https://github.com/cloudevents/spec/blob/1085aeee96d0efd625dd1654a625064fde4be6ac/http-webhook.md#41-validation-request)
[x] The trigger will need to handle deserializing the event payload into the Cloud Event model type.
[x] Output binding will need to accept CloudEvent model types as well.

/cc @pakrym .  I should be able to handle this next milestone (for the November release)

Spoke with @JoshLove-msft who's been maintaining the EventGrid trigger.  We've made some progress here but we might have not done the first bullet so it may not be possible right now to set a function as a target for a subscription of cloud events.  He's going to take a look - but that's likely the only gap left.

Beyond the subscription validation handling, you currently cannot create a CloudEventSchema event subscription for an Azure Function.

@JoshLove-msft

Beyond the subscription validation handling, you currently cannot create a CloudEventSchema event subscription for an Azure Function.

Does that mean it will be available some time?

@JoshLove-msft

Beyond the subscription validation handling, you currently cannot create a CloudEventSchema event subscription for an Azure Function.

Does that mean it will be available some time?

That is the goal, but we don't have a definite timeline yet.

For the trigger binding, the handshake between the service and the listener when creating a subscription is different. Instead of sending a "SubscriptionValidationEvent" and expecting a certain response object, the service instead follows a different handshake where an OPTIONS request is sent to the underlying webhook (see: https://github.com/cloudevents/spec/blob/1085aeee96d0efd625dd1654a625064fde4be6ac/http-webhook.md#41-validation-request)

This turned out to not be the case. There is special handling in the service for the handshake when the subscription is using Azure Functions. In this case it uses the same HttpPostChallenge mechanism as for EventGridSchema since users wouldn't need to interact with this flow in general. However, there were issues in the format of how the CloudEvent HttpPostChallenge is delivered that required fixes.


Room ID generation

generateRoomId method inside home.js

The room Id generated from this method will be inefficient and less secure, as it is generated from set of lowerCase & upperCase Alphabets.

solution

We can use uuid npm package for generating unique Id's, created from cryptographically-strong random values.

Screenshots

Hi @Vaibhav-afk, First of all, thank you so much for taking out the time to create this issue.
Generally, we thought to make room-Id more user-friendly and in a presentational format. So we came up with a pattern
xxxx-yyyy-zzzz. Also, the possible numbers of rooms are 52^12 which is huge.
So I feel, currently, there is not a need for UUID generated room-Ids
What are your thoughts on this @Vaibhav-afk ?

Yeah!  you are absolutely right but in case of scaling, it would help.


Make the Helm chart available in Helm Hub

Motivation
It would make sense to make the chart release so it can be available in the Helm Hub. This allows the project to be used without needing the GitHub project or having to copy the current deployment Helm files in this repo.
Feature
Chart available in Helm Hub.

Hi @arminc
I agree the helm chart could (should) be outside that repo, even if it's convenient for me for maintenance.
Maybe you don't know, but we decided to extract falc helm chart from hub lately, the up to date version is now here : https://github.com/falcosecurity/charts
To keep a coherance, I prefer to use this repo for falcosidekick too.

/assignee @Issif

Just to be clear I didn't ment to add the chart to https://github.com/helm/charts/tree/master/stable. It is fine to keep the chart in this repo but it should be released, like the Falco chart so it can be used directly with helm. But it stil makes sense to add the Helm chart repo to http://hub.helm.sh so it is easier to be found

Since https://github.com/falcosecurity/charts has been included in the Helm Hub ( see https://github.com/helm/hub/pull/409 ), once this chart is moved to https://github.com/falcosecurity/charts then it will be automatically listed on the hub.

@nibalizer is working on a new output, Discord, when it will be merged, I will release a new version of falcosidekick and according helm chart will be in the global repository

@arminc enjoy : https://hub.helm.sh/charts/falcosecurity/falcosidekick


how to set the length of java cpg results ?

hello.
when I export cpg graph of java, the results was omitted.
for example:
"1312" [label = <(<operator>.assignment,this.headerParsePos = HeaderParsePosition.HEADE...)527> ]

why it use "..." instead of the full code?
how to set parameter to export the full text?
please

I have the same problem as you.


[node.js] Enable RC extra services test [APPSEC-11927]

Description
Implement and enable RC extra services test
Motivation

Workflow

⚠⚠ Create your PR as draft
Follow the style guidelines of this project (See how to easily lint the code)
Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
Mark it as ready for review

Once your PR is reviewed, you can merge it! :heart:
Reviewer checklist

[ ] Check what scenarios are modified. If needed, add the relevant label (run-parametric-scenario, run-profiling-scenario...). If this PR modifies any system-tests internal, then add the run-all-scenarios label (more info).
[ ] CI is green

[ ] If not, failing jobs are not related to this change (and you are 100% sure about this statement)

[ ] if any of build-some-image label is present

is the image labl have been updated ?
just before merging, locally build and push the image to hub.docker.com

[ ] if a scenario is added (or removed), add (or remove) it in system-test-dasboard nightly

It seems they are passing now :tada:


Add Ruby 3.3.4

This pull request adds Ruby 3.3.4 to ruby-build.

Ruby 3.3.4 Released
https://www.ruby-lang.org/en/news/2024/07/09/ruby-3-3-4-released/

This commit works in my Ubuntu 24.04 environment as follows.
$ more /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04 LTS"
$ ruby -v
ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux]
$

Thanks, I tagged https://github.com/rbenv/ruby-build/releases/tag/v20240709

Naive question, should this have been bumped to include Openssl 3.1.6 due to security patches?
https://www.openssl.org/news/cl31.txt

@chadlwilson Thanks for your pointing. It should be using OpenSSL 3.0.x, not 3.1 series.

@chadlwilson Thanks for your pointing. It should be using OpenSSL 3.0.x, not 3.1 series.

Oh, OK, even better thanks - including fixing it for 3.3.3! :-)


2GB of memory isn't enough

I tried this plugin on a Drupal project and the generation of the index seems to fail, when PHP tried to allocate more than two GB of memory.
Didn't make much sense to begin with cause I've set the PHP memory limit to -1 in the CLI ini.
So I did a grep -R 'ini_set(' from the ~/.composer/vendor/mkusher folder and realised that the memory is fixed to two GB in padawan/bin/padawan.
Why is that, and would there be a way to increase it, without hacking the PHP component?

hm, I can't say that this limit was set "on purpose", looks like it better to remove it from bin

@TLyngeJ have a look https://github.com/padawan-php/padawan.php/pull/97

Yeah, I believe that should cut it.
memory_limit also set in the padawan-server, but I suppose that safe enough to assume that it wont go near that limit.

@TLyngeJ well, I've merged fix to master, so it should work well now


Breaks at simple parse

 TypeError: this.source.charCodeAt is not a function

var parsed = esprima.parse(fs.readFileSync(require.resolve("angular-ui-mask/src/mask")));line no-sync

Looks like you're not passing a string as the input.

Use readFileSync properly so that it gives a string and not a Buffer. Example (on Node.js REPL):
> fs.readFileSync('hello.js')
<Buffer 63 6f 6e 73 6f 6c 65 2e 6c 6f 67 28 22 48 65 6c 6c 6f 2c 20 77 6f 72 6c 64 21 22 29 3b 0a>

> fs.readFileSync('hello.js', 'utf-8')
'console.log("Hello, world!");\n'

Strange... before 3.0.0 works, installed version 2.7.3. and it works with Buffer

@Delagen That was because of the decision to accept only string as the input (PR #1353).

I think it must be written in Changelog as BREAKING CHANGE

@Delagen Agreed. Extensive documentation is in my radar (track #1477).

You could make the argument that passing a non-string to esprima before was undefined behaviour. I believe I'm correct in saying that it at least wasn't documented to accept anything other than a string.

You could also check your input and tell people what the problem is:

TypeError: Expected a string, got a String Buffer.


[bug] Validator metadata isn't captured when using parse methods

Describe the bug
When calling guard.parse, there is no field for passing in metadata. As a result, metadata isn't piped down to the Validator.validate methods.
To Reproduce
Steps to reproduce the behavior:

Create a dummy validator, and in validate, print out the metadata
Use parse() instead of call

Expected behavior
A clear and concise description of what you expected to happen.
Library version:
Version (e.g. 0.2.0a3)
Additional context
Testing harness to test all methods of using guard (different initialization methods, sync/async methods, call/parse methods).

Addressed in #259


Wall switch doesn’t work

If I have the ratgdo installed my wall switch doesn’t work. The light is on and it sometimes flickers but does t work. It’s a 2.0 wall switch. The ratgdo works fine. All the sensors statuses etc. there’s no errors in the logs. It’s just the wall switch that stops working. Is it a wiring issue? Anyone have suggestions?

Sorry for the delayed response.  Yes this is likely a wiring issue where the 12v line for the wall controller is being pulled down and it's not getting power.


Bump hibernate-core

for real this time

@jjestrel How can I help land this?

@swankjesse fixed merge conflicts. will merge tomorrow morning, get this out and give a PSA


Update environment libs

The set-up is outdated and many programs are not compatible with each other.
This set-up is proven to work with the change of configure_via_anaconda.md in https://github.com/udacity/CarND-Term1-Starter-Kit/pull/117

Hi @mishaboyko
All classroom code is implemented with TensorFlow 1.3.0. We have updated the environment.yml file with the correct dependency versions.


How to use this plugin in ionic

Hi....
i developing ios app using ionic framework and i want to use this plugin for picking file on ios.
So please tell me how to use .

I have the same problem, the usage appears to be very simple,  but for me the problem is the variable PickerFile declaration. I found that must be declared at begining of file as
declare var FilePicker: any;
But once running on device and after the device.isReady() I'm getting an exception
Cannot finde variable : FIlePicker
No full example is disposeable on internet.

Are you using Ionic 1 or Ionic >=2?
For Ionic 1 it should work if you use the plugin after deviceready event
For Ionic 2 the plugin might need typescript declarations, I'm not really familiar with Ionic 2

Thank you for you response,
I'm using ionic 3 (same as 2)
I'll search informnation about typescript declarations . If I find some I'll write here the response.
Thanks again !

I made a wrapper for this component compatible with ionic 2+. You must to decompress in the node_modules/@ionic-native folder
image-picker.zip

This sounds Great ! I'll going to try it now !
I'll report the results.
Thank you a lot !
BEst regards,
Roni

any solution guys, for ionic 2 and 3?

Simply add (window). before the call:
(<any>window).FilePicker.pickFile(successCallback,null);

There is now ionic support in @ionic-native/file-picker:
https://www.npmjs.com/package/@ionic-native/file-picker
https://ionicframework.com/docs/native/ios-file-picker/

As said @markv with this plugin works!
https://ionicframework.com/docs/native/ios-file-picker/


implement tutorial gtag

Fires tutorial start and end events. Also sets document.title which is also useful for gtag events
tutorial_start: {
  category: tutorials,
  label: tutorial folder name,
  value: step tutorial page was loaded,
}

tutorial_end {
  category: tutorials,
  label: tutorial folder name,
}

tutorial_start event
fired when the page and manifest area loaded, and includes the step the page was loaded on.
tutorial_end event
fired when the next button is pressed and the page index matches the length of steps in the laoded manifest -1. Signaling progression
faults

start is fired whenever the page loads so that may include reloads

can likely be deduplicated by session ID cloud-side
hopefully can fire automated event if session has start index of 0 and also fires end event

if initial page loaded is the final page, then next button cannot be pressed unless they go back and press right

doesn't signal progression so this is fine, but may have a mismatched start to end

WOW TIL @AndrewJakubowicz! Thanks so much for that link!
Moved the gtag logic to its own file but still requires a local private method to track state.
Added licenses.
Added tutorial_progress event
Possible confusions for progress is that if they go back, it won't record back progress but it will record if they go back and then right. WDYT? Should It also record back?

Question: should I be recording progress when they hit the back button? Currently only broadcasting forward

I continue to happily LGTM this PR! No strong opinion on back button. Wouldn't it technically be inferable from multiple duplicate progress steps provided the user doesn't hit the end?

Sorry for delay


use imeta as a fallback in output file

In some cases, the trained ML classifier is not able to produce a classification for a profile. In these cases, we should back on the iMeta value for that profile. This should be reflected in the classification quality flag, using 0 for iMeta and 1 for ML classifier.

This has now been demomstrated in the ML tree cross-validation notebook and should be applied to all other notebooks, including plots of what portion of profiles require the imeta fallback option.

This has now been demonstrated in notebooks and implemented in the experiment framework.


feat: expose list/get options for full/basic on schema methods

This corrects a review issue in the unreleased PR: https://github.com/googleapis/nodejs-pubsub/pull/1289

@pradn @hongalex Can y'all verify that this meets the desired functionality? Hoping to get the API finished nailing down before releasing it.

Fixing that flaky test is a separate task...


Added MANIFEST.in

Coverage remained the same at 90.728% when pulling 707f86ba67cc24b07fcb427504cd6fdbfc7cbde0 on feature/refactor into 9fc217a1753f589956d21e93e471ba631490cee9 on develop.


feat: Snippets compatibility with the Monaco Runner

This PR fixes the compatibility between Snippets and Monaco Runner.
```ts {monaco-run}
import hello from './external'

hello() // Hello from snippets/external.ts
```

Thanks for your contribution!
I've just updated the demo slides and used a more robust way to resolve the file path.


Date format with object date

Hello,
i have a problem with the date, when I hydrate my table with a dateObject it dont give me the correct format
i receive it: Wed Jun 06 2018 16:48:58 GMT+0200 (CEST)
when i ask for: { data: 'birthday', type: 'date', dateFormat: 'YYYY-MM-DD', correctFormat: true}
exemple here : https://stackblitz.com/edit/angular-5ykm8c-hcrizl?file=src%2Fapp%2Fheroes%2Fheroes.component.ts
what can i do ? i have to do it with a string or something else ?

 dateFormat: 'YYYY-MM-DD', is used to format string type data 10/10/2010 or 10-10-2010 will become 2010-10-10 but we do not fully support Date() method

Hi @brynnlow,
you can subscribe handsontable/handsontable#4291 to be up to date with the information related to this issue.


New cog icon and dropdown action fix

💬 Description
A couple of little changes here. We've added a new cog icon, and we now stop propogation on the dropdown action. Bonza!
✅ Checklist

[ ] Tests written
[ ] Browser tested
[ ] Added to documentation
[ ] Added to storybook

 Structure Editor Tabs Fixes


[Disscusion] STM32H723 Support

Is it possible to support this chips? STM32H733 is almost the same chip.
STM32H7-23-33.zip

❯ svd2rust -i STM32H723.svd
[INFO  svd2rust] Parsing device from SVD file
[ERROR svd2rust] Error parsing SVD XML file

    Caused by:
        0: In device `STM32H723`
        1: In peripheral `ADC3`
        2: In register `ADC_CFGR`
        3: Parsing field `EXTSEL0` at 5250:15
        4: `EnumeratedValue error: Value 2 out of range [0 - 1]


Experimental removal of Rubygems workarounds (main branch)

Due to unstable version resolution and lack of Ruby version constraints on older gemspecs in the Rubygems database, we had to manually declare versions to be used.
This has seemingly been fixed.
Related:
https://github.com/rubygems/rubygems/issues/3463
https://github.com/rubygems/rubygems.org/pull/2474
https://github.com/rubygems/rubygems/pull/2662
Let's see if it works now, and if the issue with IPv6 fallback is still the case.

Ouhao. Super happy if it is working.

Ran the build several times, it's green apart from a mountable_engine? failure that appears randomly in some builds jobs, but disappears on the second attempt.
3-9-maintenance will probably be even more interesting.
I'll handle the other repos as well after those two.

Have we seen the mountable_engine? failure on main yet? It must be related to #2372

All green, except for mountable_engine? failures. Restarted those jobs - now all green.


Update coc-metals vim plugin

Motivation for this change
The existing coc-metals plugin is quite outdated (v0.8.3). It now lives in a different repository. Also, the nodePackages.coc-metals was updated to the latest v0.9.0 but this plugin wasn't, leaving it in an inconsistent state.
As defined in https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/development/node-packages/node-packages.nix
coc-metals = nodeEnv.buildNodePackage {
    name = "coc-metals";
    packageName = "coc-metals";
    version = "0.9.0";
    src = fetchurl {
      url = "https://registry.npmjs.org/coc-metals/-/coc-metals-0.9.0.tgz";
      sha512 = "rQPCK+x+/Aij/YqjGTIS5c5WA3dPba2RwubGouXSRkl8+F1I4NyAXiXtHAx2V4W1LYvmjd990i6Dtzr3JLx+5A==";
    };

Things done
The existing update.py doesn't work for me, as reported in https://github.com/NixOS/nixpkgs/pull/91656#issuecomment-657206041 and https://github.com/NixOS/nixpkgs/issues/91339#issuecomment-657206742 so I manually updated generated.nix with the necessary information and updated the name of the plugin in vim-plugin-names (hope that's enough?).

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ x ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Continuing the discussion from here: https://github.com/NixOS/nixpkgs/pull/91656#issuecomment-657286681
I did the following on this branch: https://github.com/colemickens/nixpkgs/commits/vim-coc-metal

manually remove coc-metal on the old owner (from both generated.nix and vim-plugin-names)
./update.py --add scalameta/coc-metals

Hey @colemickens , thanks a lot for following up on this!
I googled my problem and every answer out there seems to suggest it's a firewall issue on my side but I'm not really sure since I have my firewall completely disabled (networking.firewall.enable = false;).
I made a screencast: https://asciinema.org/a/RCfANXgfT0Pif6lf1J6dOz76a (any hints very much appreciated).

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/getting-no-route-to-host-with-disabled-firewall-on-nixos/8127/1


Allow to send locations from other sources to AAT subscribers

In order to support A/B testing and testing features on a fraction of the whole user base of an app, we could provide a way to send locations to subscribers from other sources and make them look as if they've come from an AAT publisher.
Now, if some users will try and adopt AAT in their app they probably won't try our feature on all the users. Instead, this will be done gradually, which leads to the issue that there might be a subscriber using AAT but the rider not using it. Therefore, the subscriber either will not show anything (which is bad UX) or it would have to detect that the other side is not using AAT and fallback to another solution. This is not ideal as the subscriber should not know those details and switching between different solutions might be difficult to implement and adds complexity to the project.
Therefore, we could provide some kind of a script or a small program that could take in locations, convert them to AAT's location updates and send them via a channel to the subscriber. The program should also join the presence and do other things that a normal publisher would do in order to keep the UX of the subscriber as close to the real publisher as possible.

I wonder if this might be a reason for us to consider the create of an AAT publishing SDK (or SDKs 😉) for backend use, where the user brings their own stream of locations... My gut says Node.js, but Golang and Python would also be contenders.


error assuming role Error [ConfigError]: Missing region in config

I'm having an issue using this plugin in the gitlab CI that corresponds to this stack trace:
AssumeRoleCredentialPlugin found value for readIamRole CDKAdminAccess. checking if we can obtain credentials
error assuming role Error [ConfigError]: Missing region in config
    at Request.optInRegionalEndpoint (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/services/sts.js:75:30)
    at Request.callListeners (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at Request.runTo (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/request.js:408:15)
    at /builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/request.js:797:12
    at new Promise (<anonymous>)
    at Request.promise (/builds/foo/node_modules/cdk-assume-role-credential-plugin/node_modules/aws-sdk/lib/request.js:783:12) {
  code: 'ConfigError',
  time: 2021-06-29T00:38:56.510Z
}

The error appears to correspond with the following function in https://github.com/aws-samples/cdk-assume-role-credential-plugin/blob/main/lib/assume-role-source.ts:
   /**
   * Try to assume the specified role and return the credentials or undefined
   */
  private async tryAssumeRole(roleArn: string, accountId: string): Promise<AWS.STS.Credentials | undefined> {

    const region = this.config && this.config.settings && this.config.settings.get(["context"]).region;

    region && AWS.config.update({ region });

    const sts = new AWS.STS({
      credentials: await this.defaultCredentials(),  ...(region && { region }),
    });

    let response: AWS.STS.Credentials | undefined;
    try {
      const resp = await sts.assumeRole({
        RoleArn: roleArn,
        RoleSessionName: `${accountId}-session`
      }).promise();
      response = resp.Credentials;
    } catch (e) {
      logging.debug('error assuming role %s', e)
      return undefined
    }
    return response
  }

This always works locally for me using yarn cdk diff and the appropriate AWS_ACCESS_KEY_ID + secret combo. I was hoping that this response would solve the issue but no luck. Any idea what's going on? Why does this plugin, when running in a CI environment rather than on my local machine, always fail to find a region?

I found this helpful to solving the core issue: https://github.com/aws/aws-sdk-js/issues/2929
I think the error was coming from the aws-sdk-js package here:
optInRegionalEndpoint: function optInRegionalEndpoint(req) {
    var service = req.service;
    var config = service.config;
    config.stsRegionalEndpoints = resolveRegionalEndpointsFlag(service._originalConfig, {
      env: ENV_REGIONAL_ENDPOINT_ENABLED,
      sharedConfig: CONFIG_REGIONAL_ENDPOINT_ENABLED,
      clientConfig: 'stsRegionalEndpoints'
    });
    if (
      config.stsRegionalEndpoints === 'regional' &&
      service.isGlobalEndpoint
    ) {
      //client will throw if region is not supplied; request will be signed with specified region
      if (!config.region) {
        throw AWS.util.error(new Error(),
          {code: 'ConfigError', message: 'Missing region in config'});
      }
      var insertPoint = config.endpoint.indexOf('.amazonaws.com');
      var regionalEndpoint = config.endpoint.substring(0, insertPoint) +
        '.' + config.region + config.endpoint.substring(insertPoint);
      req.httpRequest.updateEndpoint(regionalEndpoint);
      req.httpRequest.region = config.region;
    }
  }

Investigating that repo's issues list, I came across the above. It also linked out to https://github.com/aws/aws-sdk-go/search?q=AWS_DEFAULT_REGION&unscoped_q=AWS_DEFAULT_REGION which adds a comment about AWS_DEFAULT_REGION behavior.
The fix for me was to set AWS_REGION in the CI pipeline. (I would likewise guess that setting AWS_SDK_LOAD_CONFIG=true would work)


Improvements for portability, security

Since autoenv can be use in many shells and by itself used many external commands, so it's better to stick with POSIX standard command options, syntax.
This PR makes many changes to make sure autoenv will work in shells which are tested and should be work in any other POSIX compliant shell, system.

cc @kennethreitz

cc @morbeo

@Gnouc want to help maintain this project?

@kennethreitz sure 👍


Signature help not closing on Blur.

Reproduce :
go to https://microsoft.github.io/monaco-editor/playground.html
trigger signature help in one of the editors
focus the other editor
The signature pop up will be kept open.
I tried the workaround mentioned here but after setting editor position to lineNumber 1 and coumn 1. I am seeing flicking effect while Signature popup closes(goes from current postion to new position).  also tried updating the transition: none but that also didn't help. Is there any cleaner way to close parameterHintWidget ?
I also tried :
(this.editor.getContribution('editor.controller.parameterHints') as any).closeParameterHints(); but this also didn't work.

calling this.editor.trigger('keyboard', 'closeParameterHints', null); resolved the issue.


failed generations show up as flagged by safety checker

Hello,
First I wanted to thank you for the simplicity of the setup of this stable diffusion CLI.
Just tried it with a Nvidia GTX 1660 super with 6GB of RAM, but any prompt will output a red-ish image:
$ imagine "a photo of a dog"

edit: it may somehow be related to: https://github.com/Sygil-Dev/sygil-webui/issues/34
Let me know if there is any details I could provide to help you :)
Thanks !

The red color implies it's the NSFW filter. But the fact that it's doing that for every image implies something is probably wrong with the generation. The NSFW filter tends to just flag any blurry image. You could maybe try some different samplers and see if that helps.

Possibly related: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/4558

Can you try this again with 5.0.0 which was just released? If it's still broken can you run it with --show-work and see what's going on?

Still doesn't work :/ the --show-work is interesting though !
Wanted to wrap the images into a summary tag in order to make it collapsible, but the images wouldn't render, so I'm sorry for this.

Thanks.  Yeah something is broken for sure.  Unfortunately without your graphics card it will be hard for me to debug.
This issue also seems similar. It wasn't obvious to me from a quick skim how they fixed it.
https://github.com/invoke-ai/InvokeAI/issues/526
Maybe you'll have better luck with that version of stable diffusion?

I'm inclined to just say imaginairy doesn't support this but if someone want's to figure out the problem I'd be happy to merge a fix.

I'm having similar issues when I try to make masks of my face.
Can we have a hint on how to disable nsfw so we can see what the output really is?
I tried changing some files in ~/.local/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/ but the result is always the same. ( not a python dev, so probably I do something wrong )

Only when you do masks? what version of imaginairy? what OS? what graphics card?

Yeah, on some photos it's not cropping the face correctly, so I did --steps 1 and I started playing with the mask prompt.
But on some photos I get the nsfw ( although there's absolutely nothing nsfw on the photo )
Linux / Arch
GTX 1070 8GB

Yeah I imagine that's kind of annoying.  The filter basically always triggers if you don't have enough steps.
This library doesn't use the diffusers stable diffusion. You'd need to edit things in imaginairy.
Might be a stretch but I'd just directly call get_img_mask in python and save the mask to disk to iterate quickly.
https://github.com/brycedrennan/imaginAIry/blob/master/imaginairy/enhancers/clip_masking.py#L34

no idea how to do that :D
I'll look up some python tutorials to see how to call it.
thank you!

the same on the mac M1 processor.

@djpavlo this thread is about an issue that happens on CUDA GPUs.  Please open a new issue with a detailed description of what's happening, what python version, and a copy/paste of the terminal output you're getting.

I had the same issue far too often, and solved it by hacking SafetyFilter to return False on both functions, and then initialized a new SafetyFilter rather than calling the safety model. You are right that the images in question are blurry, but it helped me figure out what was happening.
While making that change I saw the environment variable you have for relaxing the filter at least. I'd suggest providing a --nsfw=off|relaxed|strict param; strict can be default, but anybody who wants to can do what I did, so it's more user friendly to provide that as a param.

@reubenfirmin Apologies, you did nothing wrong, but I do censor some topics.    On my todo is to add a blurry-image detector so we don't have these confusing outcomes with the safety filter.

As of 7.4.3 the safety check will not flag all-black images.


POJO exemple fasterxml

I try to test ektorp with a little project such as : http://ektorp.org/reference_documentation.html#d100e102
But, I want to use POJO object by using com.fasterxml.jackson instead of org.codehaus.jackson
I update  from
Ektorp : 1.2.1 / Jackson : 2.0.0
to a later verion :
Ektorp : 1.4.1  / Jackson : 2.2.3
I use CouchDB 1.6.1
The problem is that I'm not able to save my POJO object without an id, and I was not able to do a update (rev is needed !).
{
"error" : "bad_request",
"reason" : "Invalid rev format"
}
It works fine before ( with org.codehaus.jackson).
What must I changed in this simple exemple ?
    {
    	package cncouchdb;
    	import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
    	import com.fasterxml.jackson.annotation.JsonProperty;

    	@JsonIgnoreProperties({"id", "revision"})
    	public class Sofa {
    		@JsonProperty("_id")
    		private String id;

  	  	@JsonProperty("_rev")
    		private String revision;

    		private String color;

    		public void setId(String s) {
    			id = s;
		}

		public String getId() {
			return id;
		}

		public String getRevision() {
			return revision;
		}

		public void setColor(String s) {
			color = s;
		}

		public String getColor() {
			return color;
		}
    }

My code of the main method for create is:

HttpClient httpClient = new StdHttpClient.Builder()
		        .url(DATABASE_URL)
		        .build();

		CouchDbInstance dbInstance = new StdCouchDbInstance(httpClient);

		CouchDbConnector db = new StdCouchDbConnector(DATABASE_ID, dbInstance);

		db.createDatabaseIfNotExists();

		Sofa sofa = new Sofa();
		sofa.setColor("red");
		sofa.setId(SOFA_ID_TEST);
		db.create(sofa);

For Update
HttpClient httpClient = new StdHttpClient.Builder()
		        .url(MainCreate.DATABASE_URL)
		        .build();

		CouchDbInstance dbInstance = new StdCouchDbInstance(httpClient);

		CouchDbConnector db = new StdCouchDbConnector(MainCreate.DATABASE_ID, dbInstance);

		db.createDatabaseIfNotExists();

		Sofa sofa = db.find(Sofa.class, MainCreate.SOFA_ID_TEST);
		sofa.setColor("blue");
		db.update(sofa);

My Maven pom :
<properties>
  	<jackson.version>2.2.3</jackson.version>
  </properties>

  <dependencies>
    <dependency>
		<groupId>com.fasterxml.jackson.core</groupId>
		<artifactId>jackson-core</artifactId>
		<version>${jackson.version}</version>
	</dependency>

 	<dependency>
		<groupId>com.fasterxml.jackson.core</groupId>
		<artifactId>jackson-databind</artifactId>
		<version>${jackson.version}</version>
	</dependency>

	<dependency>
    	<groupId>org.ektorp</groupId>
    	<artifactId>org.ektorp</artifactId>
    	<version>1.4.2</version>
	</dependency>

  </dependencies>

The problem might be that Ektorp fails to set the id and/or revision of
your object.
Write a test that asserts that the class org.ektorp.util.Documents is able
to read/write id and rev on your class.
Here's a little more about the Documents class:
http://ektorp.org/reference_documentation.html#d100e495
//H
On Thu, Feb 26, 2015 at 9:52 PM, BRAUD Laurent<EMAIL_ADDRESS>wrote:

I try to test ektorp with a little project such as :
http://ektorp.org/reference_documentation.html#d100e102
But, I want to use POJO object by using com.fasterxml.jackson instead of
org.codehaus.jackson
I update from
Ektorp : 1.2.1 / Jackson : 2.0.0
to a later verion :
Ektorp : 1.4.1 / Jackson : 2.2.3
I use CouchDB 1.6.1
The problem is that I'm not able to save my POJO object without an id, and
I was not able to do a update (rev is needed !).
{
"error" : "bad_request",
"reason" : "Invalid rev format"
}
It works fine before ( com.fasterxml.jackson instead of
org.codehaus.jackson).
What must I changed in this simple exemple ?
{
    package cncouchdb;
    import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
    import com.fasterxml.jackson.annotation.JsonProperty;

    @JsonIgnoreProperties({"id", "revision"})
    public class Sofa {
        @JsonProperty("_id")
        private String id;

    @JsonProperty("_rev")
        private String revision;

        private String color;

        public void setId(String s) {
            id = s;
    }

    public String getId() {
        return id;
    }

    public String getRevision() {
        return revision;
    }

    public void setColor(String s) {
        color = s;
    }

    public String getColor() {
        return color;
    }
}

My code of the main method for create is:
HttpClient httpClient = new StdHttpClient.Builder()
.url(DATABASE_URL)
.build();
    CouchDbInstance dbInstance = new StdCouchDbInstance(httpClient);

    CouchDbConnector db = new StdCouchDbConnector(DATABASE_ID, dbInstance);

    db.createDatabaseIfNotExists();

    Sofa sofa = new Sofa();
    sofa.setColor("red");
    sofa.setId(SOFA_ID_TEST);
    db.create(sofa);

For Update
HttpClient httpClient = new StdHttpClient.Builder()
.url(MainCreate.DATABASE_URL)
.build();
    CouchDbInstance dbInstance = new StdCouchDbInstance(httpClient);

    CouchDbConnector db = new StdCouchDbConnector(MainCreate.DATABASE_ID, dbInstance);

    db.createDatabaseIfNotExists();

    Sofa sofa = db.find(Sofa.class, MainCreate.SOFA_ID_TEST);
    sofa.setColor("blue");
    db.update(sofa);

My Maven pom :

2.2.3

com.fasterxml.jackson.core
jackson-core
${jackson.version}

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>${jackson.version}</version>
</dependency>

<dependency>
    <groupId>org.ektorp</groupId>
    <artifactId>org.ektorp</artifactId>
    <version>1.4.2</version>
</dependency>

—
Reply to this email directly or view it on GitHub
https://github.com/helun/Ektorp/issues/230.

By debug with jar source, I found that previously, the #serializeToJson method in org.ektorp.impl.StdCouchDbConnector#create doesn't return null value, but the new do.
I find this change in Jackson : http://wiki.fasterxml.com/JacksonAnnotationSerializeNulls
In my 1st exemple, it works by adding : @JsonSerialize, but it 's deprecated.
In Ektorp, It looks like to have been changed correctly in class org.ektorp.impl.StdObjectMapperFactory

I started getting this error on the upgrade from 1.3.0 to 1.4.x. I tracked it down to the code in #233


Minor typo fix and readme update on inference phase

during inference there is a small typo in synthesize.py
  File "synthesize.py", line 209
    return NotImplementedError
    ^
SyntaxError: 'return' outside function

Hi @hnhnarek , thanks for your correction!


Improve experimental groups visualisation for complex setups

Is your feature request related to a problem? Please describe.
When having many groups the visualisation was not easy to see, I had one row (over 2 screens) with many imaginary combinations (3 columns, 2x3x3 factors, e.g. 2 treatments, 3 timepoints, 3 genotypes/environments). But maybe 2x3x3 is too much anyway for that view.
Describe the solution you'd like
Maybe would be nice to have in several rows (to scroll down instead of to the right

Issue was reported by @d4straub

Already covered by issue #744


Add Insert method

Add Insert method to Bitvector struct.
Method signature should look like following:
func (v *Bitvector) Insert(...bool) ([]int, error) OR
func (v *Bitvector) Insert(...bool) (int, int, error)
A more detailed description of both signatures can be found in #2. In every case, the signature should be the same as it is later used in the Push method.

This was done in #9.


length of CDR1 region is not divisible by three

Hi, I see these two warnings:
WARNING: Length 26 of CDR1 region in 'IGHV2-S4508*01' is not divisible by three; region info for this gene will not be available
WARNING: Length 23 of CDR1 region in 'IGHV2-S9949*01' is not divisible by three; region info for this gene will not be available

Can you please advise me what I can do about them? As I am not giving IgDiscover a gapped V-gene file, I assume it must be working out the CDR region for itself. Have I missed something? I can provide an .ndm and a .aux file for IgBLAST if there is a way to get IgDisvoer to use them.
thanks!
William

Thanks, this is very helpful. In this case the short sequences were mine, not IgDiscover's.

Ah great!


Add search tracking to GA

We added site search through Algolia, but we are not tracking search terms through GA. It would be helpful to add that in, since it might give us a good insight into what people are searching for.

FYI, you have access to analytics data in your Algolia dashboard. It displays things like most common queries, most common queries without results, number of words typed, and all that including or excluding typos.
You only had 1 day of analytics retention, I just bumped it to 30 days.


chore(business-unit): add typename fields to graphql transformers

This adds the typename fields to the transformers of Division and Company. Thus enabling us to use it in our graphql based tests.

Feel free to release if this blocks you. A release a day, keeps stale pipelines away.


Fixed bug when calling create() multiple times

Moved mkdir outside of uniqid check, allowing for create() to be called multiple times without error.

Great! Thanks!

Released to Packagist as v1.2.5


Expanded power and reset scenarios

Expanded the draft to include additional scenarios related to persistent power to the xPUs, and implications of dual or more xPUs in a host.
Signed-off-by: Ted Streete<EMAIL_ADDRESS>

@tedstreete great contrib, please rebase and fix lint error


tvOS not logging on real device

The fallback was never being registered. Once registered it seems to be configured and allow the user to enable remote logging etc. There may be assumptions elsewhere as the logs still don’t seem to persist. This likely requires more investigation.
Use the caches directory for appleTV hardware as this is the only available directory on tvOS. It is more temporary, and often is cleared on backgrounding, but at least it’s available.

Thanks for taking this on, @mmmcheese.
The in-memory store is not really designed be used. It's there just to make the complier happy and avoid force unwraps. Maybe now is a good time to make it work properly because, here is what I found out about tvOS from quick research:

The first thing to understand here is that tvOS was designed as an always connected system, and thus apps generally don’t store user data on the Apple TV.
Source: https://developer.apple.com/forums/thread/89008

Moreover, your app can only access 500 KB of persistent storage that is local to the device (using the NSUserDefaults class). Outside of this limited local storage, all other data must be purgeable by the operating system when space is low.
Source: https://developer.apple.com/library/archive/documentation/General/Conceptual/AppleTV_PG/index.html#//apple_ref/doc/uid/TP40015241

I don't think the linked guide states anything specific about the library folder. But here what it does say: "Your app can download the data it needs into its cache directory.". So I suggest trying the caches directory. I think it should work well enough. It's OK if occasionally the logs get purged. Pulse does it itself occasionally already anyway.

Thanks for taking this on, @mmmcheese.
The in-memory store is not really designed be used. It's there just to make the complier happy and avoid force unwraps. Maybe now is a good time to make it work properly because, here is what I found out about tvOS from quick research:

The first thing to understand here is that tvOS was designed as an always connected system, and thus apps generally don’t store user data on the Apple TV.
Source: https://developer.apple.com/forums/thread/89008

Moreover, your app can only access 500 KB of persistent storage that is local to the device (using the NSUserDefaults class). Outside of this limited local storage, all other data must be purgeable by the operating system when space is low.
Source: https://developer.apple.com/library/archive/documentation/General/Conceptual/AppleTV_PG/index.html#//apple_ref/doc/uid/TP40015241

I don't think the linked guide states anything specific about the library folder. But here what it does say: "Your app can download the data it needs into its cache directory.". So I suggest trying the caches directory. I think it should work well enough. It's OK if occasionally the logs get purged. Pulse does it itself occasionally already anyway.

The main fix is using the cache directory on tvOS. I happened to leave in the changes to the fallback, but since they don't work they could also be ignored for now.

Thanks, merging it. I'll switch the default back to the library directory if you don't mind. I think it's the right one for these sort of files (reference: https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html).


Support the removal of images

Work is being done upstream here. Therefore, this feature, although needed, is blocked by it. We will work on it once that issue is fixed. It's supposed to get out with distribution 2.1

Soft deletes has been implemented in the registry client with this commit e2de2ecb1337e58d75b4f03420f9847a4c899e80. However, we still need to investigate a bit further how it works under the hood before implementing it on the UI.

For reference, the starting point for the "Distribution supporting deletes" discussion seems to be here: https://github.com/docker/distribution/blob/master/ROADMAP.md#deletes

Not sure if this is the correct place to ask, is there any update on this feature?  In our company, we are evaluating Portus and it seems that this missing feature is really important for maintaining the registry.

@hhcordero sadly, this is still not implemented in docker distribution (the registry) yet. That being said, I'd say that there is work being done in this regard (e.g. pruning). So, hopefully in the near future we will be able to somehow tackle this.

It looks like Portus could probably take advantage of the soft-deletes implemented in https://github.com/docker/distribution/issues/461 now, and let the registry deal with the cleanup under the hood (whenever they get around to it). But note that the reference argument must be a digest and not a tag (See https://github.com/docker/distribution/issues/549). I've asked for clarification but haven't received any.

@JonathonReinhart note that soft deletes are already implemented in Portus itself (take a look at the bin/client.rb file).

and let the registry deal with the cleanup under the hood (whenever they get around to it)

That's the deal for me. We could implement deletes, but that would be lying until there is no pruning or GC support registry's side. Thus, I prefer to wait for that.

Valid point. The one benefit would be removal of the tags, minimizing visual "clutter".

@JonathonReinhart Yeah, but then you'll run into the problem of "I've deleted all of these tags! Why has my sever run out of disk space?". IMO, the principle of least surprise requires us to not implement a feature that actively lies to users, at least until distribution get around to auto-GCing orphaned tag references.

These points all sound perfectly valid to me. Maybe all of these issue
cross-references will get someone's attention over at Docker :-)

One can only hope. I could prod some of the maintainers. :smile_cat:

I'm in complete agreement w.r.t. "principle of least surprise" - if the backing store is a cluttered mess, it should look like the cluttered mess it is.  One needs to know where the disk space has gone.
FYI the reason I found Portus was because I was searching for a (stable) Docker Registry that could handle deletions (v1 registry isn't stable, v2 isn't complete).  Lack of support for deletion is a killer issue for me - contrary to popular belief, disk space is not infinite.  My CI builds that create docker images can churn through terabytes a week.
If Portus provided a docker repository API and the ability to delete images, I would have adopted it.
@cyphar If you can prod docker folks to raise their awareness of this issue, please do so...

@pjdarton  I have the exact same issue.
My current workaround for now using this script as base : https://github.com/burnettk/delete-docker-registry-image and wrap it with some specific stuff to remove my tag/image in a crontab
Hope it helps.
Regarding this issue : +1 for waiting the feature to be fully working on docker side.

I could prod some of the maintainers. :smile_cat:

,

@cyphar If you can prod docker folks to raise their awareness of this issue, please do so...

Not to speak on their behalf, but the Docker maintainers are very, very aware of this issue, and I'm sure they are getting tired of the continuous stream of +1's being thrown at every issue that talks discusses image deletion. See this comment as an example: https://github.com/docker/docker-registry/issues/45#issuecomment-132072847
I would definitely suggest having a look at the deletes section of the roadmap before attempting to engage with them about this issue, otherwise you'll quickly be dismissed.
That being said, I find it quite frustrating that a project as large and corporate-sponsored as Docker hasn't figured out a way to delete things from the registry yet. I mean, would the EXT4 developers ever release a version of their filesystem that didn't let you delete files from it?! So by all means, if SUSE personnel would be able to express that Portus remains incomplete because of Docker shortcomings, please do so.

I'm aware that it's a non-trivial issue, and have some sympathy for the developers who are at the sharp end of this criticism, but I share your frustration that it got released in this incomplete state.
It's advertised as an implementation of the Docker v2 registry, but it's missing mandatory elements of that API.  That was quite a surprise...
If Portus is just bundling the official Docker v2 registry implementation then Portus cannot claim to be a Docker v2 registry until the deletion part of the API is implemented (either by Portus or within the official Docker v2 registry code).

If Portus is just bundling the official Docker v2 registry implementation then Portus cannot claim to be a Docker v2 registry until the deletion part of the API is implemented (either by Portus or within the official Docker v2 registry code).

We do not claim to be a Docker v2 registry. Portus is an authorization service for a Docker Registry implementing v2 (a.k.a. docker distribution). The description of Portus is:

Authorization service and frontend for Docker registry (v2)

More specifically, in the first image of this document, Portus fills the "Authorization Service" box, nothing more, nothing less. Therefore, Portus can only support deletion when docker distribution does it ;)

I would definitely suggest having a look at the deletes section of the roadmap before attempting to engage with them about this issue, otherwise you'll quickly be dismissed.

I agree with @JonathonReinhart on this.

GC support has just been merged in docker distribution and it will be there for 2.4 (see PR). I will investigate on it, and we will hopefully get rid of this issue.

As a temporary work around this worked for us.
https://github.com/burnettk/delete-docker-registry-image

After doing some work on top of distribution 2.4, I've decided that this is in fact no longer blocked. I've already sent a couple of PRs in preparation of the "big" feature :)

Since it's such a big feature (as you pointed out by yourself ;-) wouldn't it be worth to issue a interim 2.0.4 release containing especially this feature? Waiting until autumn is quite a long time...

@holgerreif I'd say no for two reasons:

We've already done some previous work before working on this, so separating the commits related to this feature into its own branch/tag would be troublesome (possibly adding regressions with the porting :sweat_smile:)
We reserve patch-level releases for bug fixes only. This would be against our release procedure.

So, I can only see two options: have it on master until 2.1 is released (the usual procedure); or having a separate branch with the current master plus this feature. In this second option master would still have this feature, so the only gain would be to have a branch with the least amount of other features in it plus this one. This is maybe the best option. What do you think ?

Then I'll stick with master.
@JonathonReinhart @rennhak @pjdarton @Seb-Solon does anybody of you have opinions?

Since it's such a big feature... wouldn't it be worth to issue a interim 2.0.4 release containing especially this feature?

@holgerreif You've contradicted your self here, by suggesting a "big feature" to into a "patch" release.  Have a look at Semantic Versioning, which describes what types of things go into what type of release.
Personally, I'm on 2.0, but plan to move to master to get some other recent fixes. Of course, I wish this project were moving faster and 2.1 were already out, but the only way to hurry open-source is with your time or money, and I don't have a lot of either at the moment :blush:

@holgerreif I've been doing some work around this feature this week and the previous one. I expect to get the feature finished for either this week or the next one. So hopefully you'll be able to test this feature soon enough :)

It's now possible to remove images & tags from the Portus UI. In the following weeks I'll add support for removing namespaces, teams and users, and I'll try to polish any corner case that might exist. Documentation for this is still to be written, but a summary would be:

You have to enable deletes on the Portus config (which should only be activated if you have distribution 2.4, or if you don't care about disk space :sweat_smile:)
Users with the required privileges will be able to remove images/tags.
Administrators are still required to run GC manually once in a while on the registry. The reason for this is that GC is not available from the registry's API, so there's no way from Portus to perform this.
You don't have to enable maintenance mode in the registry to use GC, but it's discouraged because it may end up with inconsistencies due to race conditions (we are working on providing a workaround for this).

@mssola could you at least tell me what configuration to add so i can use this feature?

@EugenMayer certainly. For clarification for other users as well, in my previous comment, when I say:

You have to enable deletes on the Portus config (which should only be activated if you have distribution 2.4, or if you don't care about disk space :sweat_smile:)

I mean that we introduced the delete configurable value. It only has the enabled key, which is set to false by default. On master, you can see it here.

@mssola if found it already, thank you. My issue was, that i was using https://github.com/sshipway/Portus which is rather a fork then just a build for https://hub.docker.com/r/sshipway/portus/ ..
i went forward and made a clean build repo, which uses SUSE/portus without modifications .. publishing them to https://hub.docker.com/r/eugenmayer/portus/ while building is done using https://github.com/EugenMayer/portus-build
If that is out of any interest for you - rip it off :)
I already tried that feature and it works great. If you wonder what i am doing, i am packing portus for rancher.io so it is installable as a "app" ( catalog ) https://github.com/EugenMayer/kontextwork-catalog/blob/master/templates/registry-slim/1/docker-compose.yml

@mssola if found it already, thank you. My issue was, that i was using https://github.com/sshipway/Portus which is rather a fork then just a build for https://hub.docker.com/r/sshipway/portus/ .. and it was very outdated and did not include this feature at all

i went forward and made a clean build repo, which uses SUSE/portus without modifications .. publishing them to https://hub.docker.com/r/eugenmayer/portus/ while building is done using https://github.com/EugenMayer/portus-build

One of the efforts that we are doing for the 2.1 release is to come up with an official production-ready Docker image. For this, we've prepared this image, which has three tags:

head: Users that want to be on the bleeding edge.
2.0: Users that want to be on the current stable release of 2.0.x.
2.0-git: Users that want to be on the current stable release of 2.0.x + some patches that are to be released.

We are still working on the documentation of this image, but an overview of it is already available on Docker hub. I'd recommend giving this image a try.

great! i could just create a wrapper-image to offer a different entry-point ( due to the configuration over ENV ), but in general, i would be very glad to use your official image!

Just had a look at the image and created some feedback https://github.com/SUSE/Portus/issues/912 for now, that image would be not be practical to build on for this purpose. Maybe we can work on this


ValueError: Number of samples must be non-negative, in function_base.py

Here is the traceback when I use command line:
Traceback (most recent call last):
  File "/Users/kylefoley/Documents/pcode/venv4/bin/page-dewarp", line 8, in <module>
    sys.exit(main())
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/__main__.py", line 22, in main
    processed_img = WarpedImage(imgfile)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/image.py", line 58, in __init__
    if not kind:
NameError: name 'kind' is not defined
(venv4) Admins-MacBook-Pro-4:downloads kylefoley$ page-dewarp 0252.jpg
got it
Traceback (most recent call last):
  File "/Users/kylefoley/Documents/pcode/venv4/bin/page-dewarp", line 8, in <module>
    sys.exit(main())
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/__main__.py", line 22, in main
    processed_img = WarpedImage(imgfile)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/image.py", line 90, in __init__
    self.threshold(page_dims, params)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/image.py", line 98, in threshold
    remap = RemappedImage(self.stem, self.cv2_img, self.small, page_dims, params)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/dewarp.py", line 41, in __init__
    page_x_range = np.linspace(0, page_dims[0], width_small)
  File "<__array_function__ internals>", line 5, in linspace
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/numpy/core/function_base.py", line 130, in linspace
    raise ValueError("Number of samples, %s, must be non-negative." % num)
ValueError: Number of samples, -62447745571, must be non-negative.

However, I'm using the module within another module which I created and had to make some adjustments.  The result is basically the same except that the number of samples is different.  I have dewarped 244 images so it's not like this error happens a lot.  Here is the trackback for when I used the module within another module:
Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/pydevd.py", line 1483, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/kylefoley/Documents/pcode/other/ocr.py", line 1779, in <module>
    ins.begin(args[2],args[3:])
  File "/Users/kylefoley/Documents/pcode/other/ocr.py", line 180, in begin
    self.main_loop()
  File "/Users/kylefoley/Documents/pcode/other/ocr.py", line 276, in main_loop
    self.dewarp()
  File "/Users/kylefoley/Documents/pcode/other/ocr.py", line 324, in dewarp
    ins = dewarp.WarpedImage(self.pt, self.im1,self.dest4)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/image.py", line 89, in __init__
    self.threshold(page_dims, params)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/image.py", line 94, in threshold
    remap = RemappedImage(self.dest, self.cv2_img, self.small, page_dims, params)
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/page_dewarp/dewarp.py", line 41, in __init__
    page_x_range = np.linspace(0, page_dims[0], width_small)
  File "<__array_function__ internals>", line 5, in linspace
  File "/Users/kylefoley/Documents/pcode/venv4/lib/python3.8/site-packages/numpy/core/function_base.py", line 130, in linspace
    raise ValueError("Number of samples, %s, must be non-negative." % num)
ValueError: Number of samples, -82235987185, must be non-negative.

That error is from NumPy and is saying you can't make a np.linspace with a negative number
>>> np.linspace(0,1,-1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<__array_function__ internals>", line 5, in linspace
  File "/home/louis/miniconda3/envs/zucker_dewarp/lib/python3.10/site-packages/numpy/core/function_base.py", line 122, in linspace
    raise ValueError("Number of samples, %s, must be non-negative." % num)
ValueError: Number of samples, -1, must be non-negative.

The full output when I run the program on your image is
Loaded 0252.jpg at size='2151x3265' --> resized='430x653'
  got 20 spans with 126 points.
  initial objective is 0.005445525413610505
  optimizing 154 parameters...
  optimization took 16.96 sec.
  final objective is 0.0008200064104791682
  got page dims -203076762.8361408 x 35.684414436800765
  output will be -331529607040x58256
Traceback (most recent call last):
  File "/home/louis/miniconda3/envs/zucker_dewarp/bin/page-dewarp", line 8, in <module>
    sys.exit(main())
  File "/home/louis/miniconda3/envs/zucker_dewarp/lib/python3.10/site-packages/page_dewarp/__main__.py", line 22, in main
    processed_img = WarpedImage(imgfile)
  File "/home/louis/miniconda3/envs/zucker_dewarp/lib/python3.10/site-packages/page_dewarp/image.py", line 80, in __init__
    self.threshold(page_dims, params)
  File "/home/louis/miniconda3/envs/zucker_dewarp/lib/python3.10/site-packages/page_dewarp/image.py", line 84, in threshold
    remap = RemappedImage(self.stem, self.cv2_img, self.small, page_dims, params)
  File "/home/louis/miniconda3/envs/zucker_dewarp/lib/python3.10/site-packages/page_dewarp/dewarp.py", line 41, in __init__
    page_x_range = np.linspace(0, page_dims[0], width_small)
  File "<__array_function__ internals>", line 5, in linspace
  File "/home/louis/miniconda3/envs/zucker_dewarp/lib/python3.10/site-packages/numpy/core/function_base.py", line 122, in linspace
    raise ValueError("Number of samples, %s, must be non-negative." % num)
ValueError: Number of samples, -20720600440, must be non-negative.

The negative number clearly slips into the page dimension (-203076762.8361408). A length is a scalar value and cannot be negative.
Specifically, it slips into the function get_page_dims in the image.py module:
def get_page_dims(corners, rough_dims, params):
    dst_br = corners[2].flatten()
    dims = np.array(rough_dims)

    def objective(dims):
        proj_br = project_xy(dims, params)
        return np.sum((dst_br - proj_br.flatten()) ** 2)

    res = minimize(objective, dims, method="Powell")
    dims = res.x
    print("  got page dims", dims[0], "x", dims[1])
    return dims

Stepping through this you can see the value computed for dims is way off from the initial estimate rough_dims:
> /home/louis/dev/page-dewarp/src/page_dewarp/image.py(34)get_page_dims()
-> res = minimize(objective, dims, method="Powell")
(Pdb) p objective
<function get_page_dims.<locals>.objective at 0x7f094dd1d480>
(Pdb) p dims
array([1.12698885, 1.93404471])
(Pdb) n
> /home/louis/dev/page-dewarp/src/page_dewarp/image.py(35)get_page_dims()
-> dims = res.x
(Pdb) p dims
array([1.12698885, 1.93404471])
(Pdb) n
> /home/louis/dev/page-dewarp/src/page_dewarp/image.py(36)get_page_dims()
-> print("  got page dims", dims[0], "x", dims[1])
(Pdb) p dims
array([-2.03076763e+08,  3.56844144e+01])

Without looking into this further my immediate thought would be that this flaw could be superficially smoothed over (if not quite fixed) by falling back to the rough_dims (which isn't meant to be used for that, it's meant to be the starting point for the minimisation). I expect the root cause is that the parameters are off somehow, hence the params value being passed to the optimisation are spoiling it.
Recall that the params are set in the __init__ method of the WarpedImage class, immediately followed by the get_page_dims call where the negative length shows up.
With that 'hotfix', a result is produced. I'll ship it as I don't have time to look into this further right now, and reviewing this code I'd need to improve the quality in various ways to make it more debuggable (this library is first and foremost a Python 3 rewrite of Matt Zucker's unmaintained Python 2 implementation).

Patched in d22869e, will package in next release


WebResponseExceptionTranslator alternative for customizing error response.

how to customize error response ??
in old spring security oauth2 project, there is a WebResponseExceptionTranslator interface to do it

like this
private void applyAuthorizationServerConfigurer(HttpSecurity http) throws Exception {
    OAuth2AuthorizationServerConfigurer<HttpSecurity> authorizationServerConfigurer =
        new OAuth2AuthorizationServerConfigurer<>();
    http.apply(authorizationServerConfigurer)
        .providerSettings(ProviderSettings.builder().tokenEndpoint("/oauth/token").build())
        .tokenEndpoint((tokenEndpoint) -> tokenEndpoint
            .accessTokenRequestConverter(
              new DelegatingAuthenticationConverter(Arrays.asList(
                  new OAuth2AuthorizationCodeAuthenticationConverter(),
                  new OAuth2RefreshTokenAuthenticationConverter(),
                  new OAuth2ClientCredentialsAuthenticationConverter()
              )
            ))
            // use custom handler
            .errorResponseHandler(new SimpleUrlAuthenticationFailureHandler())
        );

    AuthenticationManagerBuilder builder = http.getSharedObject(AuthenticationManagerBuilder.class);
    builder
        // dao provider
        .userDetailsService(userDetailsService).passwordEncoder(passwordEncoder);
  }

@learnting The implementation of SimpleUrlAuthicFailureHandler takes into context configurations, such as the defaultFailureUrl property, if you implement it by yourself, it will invalidate some external methods，a simple error response interface should be provided to customizing the error return response.

@endink  errorResponseHandler() also accepts an interface AuthenticationFailureHandler
Compared with WebResponseExceptionTranslator, I think AuthenticationFailureHandler  can meet your requirements
see OAuth2TokenEndpointFilter::doFilterInternal

@endink Thanks for your inquiry but questions are better suited to Stack Overflow. We prefer to use GitHub issues only for bugs and enhancements.
As @learnting mentioned, the errorResponseHandler(AuthenticationFailureHandler errorResponseHandler) is the configuration point for customizing the error response.