[Compute] Direct sharing support

Description

Dupe of #18398 as it is reverted in #18449.
In this PR, to avoid the bug in codegen, we do not user code gen tool and do some changes according to service team request.
Testing Guide

History Notes

[Compute ] az sig share: Manage gallery sharing profile.
[Compute] az sig list-shared:  List shared galleries by subscription id or tenant id.
[Compute] az sig show-shared:  Get a shared gallery.
[Compute] az sig image-definition list-shared:  List shared galleries by subscription id or tenant id.
[Compute] az sig image-definition show-shared:  Get a shared gallery image.
[Compute] az sig image-version list-shared:  List shared galleries by subscription id or tenant id.
[Compute] az sig image-version show-shared:   Get a shared gallery image version.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

Compute


Nadeko only greet when first login

Hi,
I activated .greet and .bye message on my bot. However, It's only greeting people on their first log into the server, and sending the bye message when I kick them.
Is there a way to make her greet people at each login, and each leaving?
Thanks.

If you mean telling when people log in and such, you can use
.userpresence
The bot will then tell their statuses.

^

I was more thinking about sending a custom message when people log in.

You cant


Adding in updates for Mac for upcoming events

Updates to the Mac installation
Tested and updated upgrade material
Added in some basic Mac exercises (acknowledge inspiration per Steve's ok)

Signed-off-by: Mary Anthony<EMAIL_ADDRESS>

cool graphics :)

thanks :-)

You know, I understand your concern, but unless we show them the correct output to expect, readers won't be able to verify they are running correct.  There are several techniques in this document that should help avoid this issue.

Readers are given a procedure to follow for setup rather than a narrative.

The instructions tell them to set their environment using this this:
$ $(boot2docker init)
Which is our standard method in house. Only then do I tell them they can use export.  I never actually  illustrate setting that value via export on a command line.

yup, much goodlyness :)
LGTM, with fixes - @fredlf @jamtur01

LGTM with a few copy edits, etc. Good revision. Let's work on always finding the simplest, most pithy to communicate and continue being mindful that we are serving a somewhat diverse group of users in terms of experience.

The first 2 paragraphs still conflate the boot2docker management tool and the Boot2Docker vm:
Because the Docker daemon uses Linux-specific kernel features, you can't run
Docker natively in OS X. Instead, you must install the Boot2Docker application.
The application includes a VirtualBox Virtual Machine (VM), Docker itself, and the
Boot2Docker management tool.

The Boot2Docker management tool is a lightweight Linux virtual machine made
specifically to run the Docker daemon on Mac OS X. The VirtualBox VM runs
completely from RAM, is a small ~24MB download, and boots in approximately 5s.

I would simplify it a little like:
Because the Docker daemon uses Linux-specific kernel features, you can't run
Docker natively in OS X. Instead, you must install Boot2Docker.
Boot2Docker includes Oracle VM VirtualBox, Docker itself, and the
Boot2Docker management tool.

The Boot2Docker management tool creates a lightweight Linux virtual machine (VM) made
specifically to run the Docker daemon on Mac OS X. The VirtualBox VM runs
a read only micro Linux distribution, with an added persistence disk for the Docker images
and containers. After the first initialization bootup sequence, the Boot2docker VM should
be running and ready to use in 5-10 seconds.

I would like to consider discussion on this edit concluded. It's a significant improvement over what was here before, it's largely accurate and has been well vetted. If we see further issues, we can change things down the road.

LGTM as an interim step. +1 to merge.


ols: 0-unstable-2024-04-15 -> 0-unstable-2024-04-28

Description of changes
update ols to a new nightly version, also adding a passthru.updateScript for automatic updates
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 308480 run on aarch64-linux 1

Result of nixpkgs-review pr 308480 run on x86_64-linux 1

1 package built:

ols


Fix testMatch to find test files correctly

The problem occured in #62 and seems to come from this part: ?(*.).
Applying the changes in this PR finds the files correctly and runs tests for .ts and .tsx files inside root and subfolders within /test/.
Can't check if this still works on other OS than Windows though.
PS C:\dev\hidash> npm test

><EMAIL_ADDRESS>test C:\dev\hidash
> tsdx test

 PASS  test/blah.test.ts
 PASS  test/fix/make.test.tsx

Test Suites: 2 passed, 2 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        3.362s
Ran all test suites.

@allcontributors please add @bastibuck for code, bug


Restarts

This adds support for restarting processes to the supervisor along with a few other bugs that might have caused improper shutdowns.
Configuring restarts
// restart up to 10 times with a 10 second delay
svisor.AddProcess("keeper", stolonCmd("stolon-keeper"), supervisor.WithRestart(10, 10*time.Second))

// restart forever with a 3 second delay
svisor.AddProcess("sentinel", stolonCmd("stolon-sentinel"), supervisor.WithRestart(0, 3*time.Second))

supervisor.Run returns an error now which will be nil if all processes exited cleanly or a processError if a process exhausted it's restart limit. The supervisor will exit with a code 1 if a process crashed too many times.
	err = svisor.Run()
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

I also noticed the AddProcess function was hardcoding a /bin/sh -c gosu stolon %s prefix on commands. I removed that so I could test locally and moved the prefix formatting to the local stolonCmd function in start/main.go
I tested this with docker compose locally and it appears to be working. In order to test failures I made a new binary and scripts in this branch: https://github.com/fly-apps/postgres-ha/blob/test-restarts/cmd/restarter/main.go.

Closing, this was merged in a separate PR.


A proposition: rethinking a referenced value from the Polling::startPolling

Hey sir! @cheeghi
Thank you for the efforts on this project. I've used a couple of methods from it for my tasks today and they work very well!
I want to slightly change a $processingUpdate workaround for the Polling class, to make a startPolling method reference-free. It will help to make, for example, a custom delay between message processing. At the current state, this method is strictly depends on that referenced value. I want to place this logic in the more abstract code unit.
What do you think? Can I provide a pull request for this?
Thanks!

That will work for my case, thank you!


Update native-modules-android.md

Those two methods are missing in the documentation and an exception is sent if they are not in the code.

Signed CLA.

When omitting addListener, this is the exception:
`new NativeEventEmitter()` was called with a non-null argument without the required `addListener` method.

When omitting removeListeners, this is the exception:
`new NativeEventEmitter()` was called with a non-null argument without the required `removeListeners` method.

I cannot share a link to the native module which implemented this when I was confronted with the problem as it is proprietary source code. However I can put together a file that implements it quickly, if needed.

Thanks @simonbrunou for improving our docs! This was definitely incomplete without the examples that you added. Can you please address my comment so we can merge it? Thanks!

Thanks @simonbrunou for improving our docs! This was definitely incomplete without the examples that you added. Can you please address my comment so we can merge it? Thanks!

I think it's done, however I'm not familiar with GitHub's interface so correct me if I did something wrong.

I assume the tag should have been on @simonbrunou @Simek

Ugh, yup! Sorry about that! I should not trust the GitHub mention autocomplete. 😅

Haha 😅

@simonbengtsson I will mark this conversation as off-topic now, to reduce the noise in the issue. Thank you so much for the comment and your forbearance! 🙂

I rebased on main and added the changes to the documentation.

It should be good now.

I think we can still improve the comment. I don't think the current one is useful to tell people what this is for, only that's required. We could say something like what we say on iOS.
So instead of:
 // Keep: Required for RN built in Event Emitter Calls.

We could use something like:
@ReactMethod
public void addListener(String eventName) {
  // Set up any upstream listeners or background tasks as necessary
}

@ReactMethod
public void removeListeners(Integer count) {
  // Remove upstream listeners, stop unnecessary background tasks
}

@simonbrunou could you please make these changes? Sorry for the back and forth.

Thanks for improving our website, @simonbrunou!


Refactor common util functions for /test

Changes
Prior to this commit, the uitl functions for e2e tests are distributed over multiple test files in /test. And we had to tag specific test files for unrelevant test suites that needs such util functions. For example, init_test has the 'conformance' build tag on it, which is not necessarily the case as conformance test suite does not need the inti_test but needs the setup there.
This commit cleans up the util functions separated across different test files and put them into /test/util so that we can tag the appropriate test files for build only.
/kind cleanup

Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[ ] Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
[ ] Has Tests included if any functionality added or changed
[ ] Follows the commit message standard
[ ] Meets the Tekton contributor standards (including functionality, content, code)
[ ] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

/ok-to-test

/lgtm


feat: heterogenize TensorProduct.congr and friends

This:

generalizes a handful of results to non-commutative algebras (simply by moving them to an earlier section)
adds heterogenous versions of map, congr, mapBilinear, homTensorHom, leftComm, tensorTensorTensorComm
generalizes lcurry, uncurry, and assoc to take an extra algebra, such that the As in the two different places can separately be replaced with R

[x] depends on: https://github.com/leanprover-community/mathlib/pull/19143
[x] depends on: #6094
[x] depends on: #6187

Extracted from https://github.com/pygae/lean-ga/pull/31. There is more I want to extract from that patch (most notably the heterobasic BilinForm.tensorDistrib, but the rest will come in a future PR.

I get the impression that this work is a lot more impressive than it lets on: I tried mildly generalising some of the results and immediately got into all sorts of problems.

I'll try to reflect this in an update to the PR description :)

All the proofs are very succinct and to be quite frank I think I might struggle with some of them on paper; if everything is R-linear and tensor products are all over R then everything is easy, but once you start throwing around more rings then things seem to become a lot more delicate.

Frankly tensorTensorTensorComm was awful; keeping track of 4 modules tensored over 3 rings was awful, and I think I would have had a much easier time if I could have color-coded the goal view somehow to keep track of them.
Well done! I left a host of comments, some of which don't need acting on and can just be resolved after you've read them if you agree that nothing needs doing. Thanks a lot!


Problems while "make"

gcc -g -O2 -D_BSD_SOURCE -DHAVE_SOCKADDR_SA_LEN -DLIBNET_BSD_BYTE_SWAP -DDSNIFF_LIBDIR=\"/usr/local/share/dsniff/\" -I.  -I./missing -I/usr/local/opt/libpcap/include -I/usr/local/opt/libnet/include -I/usr/local/opt/libnids/include -I/usr/local/opt/openssl/include -I/usr/local/include -c ./ssh.c gcc -g -O2 -D_BSD_SOURCE -DHAVE_SOCKADDR_SA_LEN -DLIBNET_BSD_BYTE_SWAP -DDSNIFF_LIBDIR=\"/usr/local/share/dsniff/\" -I.  -I./missing -I/usr/local/opt/libpcap/include -I/usr/local/opt/libnet/include -I/usr/local/opt/libnids/include -I/usr/local/opt/openssl/include -I/usr/local/include -c ./sshcrypto.c gcc -lglib-2.0 -lgthread-2.0 -o sshmitm sshmitm.o buf.o hex.o record.o ssh.o sshcrypto.o -lresolv  -L. -lmissing -L/usr/local/opt/libnet/lib -lnet -L/usr/local/lib -ldb -L/usr/local/opt/openssl/lib -lssl -lcrypto Undefined symbols for architecture x86_64:   "_ERR_load_crypto_strings", referenced from:       _SSH_init in ssh.o   "_SSL_library_init", referenced from:       _SSH_init in ssh.o   "_SSL_load_error_strings", referenced from:       _SSH_init in ssh.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [sshmitm] Error 1

'
Undefined symbols for architecture x86_64:
"_ERR_load_crypto_strings", referenced from:
_SSH_init in ssh.o
"_SSL_library_init", referenced from:
_SSH_init in ssh.o
"_SSL_load_error_strings", referenced from:
_SSH_init in ssh.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [sshmitm] Error 1
'

Undefined symbols for architecture x86_64:   "_ERR_load_crypto_strings", referenced from:       _SSH_init in ssh.o   "_SSL_library_init", referenced from:       _SSH_init in ssh.o   "_SSL_load_error_strings", referenced from:       _SSH_init in ssh.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [sshmitm] Error 1


Make minor changes

Codecov Report

Merging #264 (462ecb7) into master (bf688a0) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #264   +/-   ##
=========================================
  Coverage     73.71%   73.71%
  Complexity      905      905
=========================================
  Files           105      105
  Lines          2907     2907
  Branches        395      395
=========================================
  Hits           2143     2143
  Misses          654      654
  Partials        110      110

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf688a0...462ecb7. Read the comment docs.


Unable to publish with AssumeRolePolicyDocument in Globals

Based on https://github.com/awslabs/serverless-application-model/pull/1311, it looks like i should be able to put AssumeRolePolicyDocument in Globals, but when trying to run sam publish, I'm getting:
Publish Failed
Error: Invalid Serverless Application Specification document. Number of errors found: 1. Errors: 'Globals' section is invalid. 'AssumeRolePolicyDocument' is not a supported property of 'Function'. Must be one of the following values - ['Handler', 'Runtime', 'CodeUri', 'DeadLetterQueue', 'Description', 'MemorySize', 'Timeout', 'VpcConfig', 'Environment', 'Tags', 'Tracing', 'KmsKeyArn', 'AutoPublishAlias', 'Layers', 'DeploymentPreference', 'PermissionsBoundary', 'ReservedConcurrentExecutions', 'ProvisionedConcurrencyConfig']
Please follow the instructions in https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-publishing-applications.html

I'm using the aws-sam-cli 0.40.0 installed via pip
sam validate says the template is fine

Should be fixed in aws-sam-cli 0.41.0.  Will re-open if still an issue.


How is BSConv being utilized in MobileNet V2 and V3?

Great paper!
Just one small question,
It seems that you have not alter the structure of mobilenet v2 and v3, because it sort of already got BSConv built in?
Dose this imply that the accuracy gain (especially in CIFAR) is purely from the proposed orthonormal regularization loss?

Thank you!
Yes, we decided for keeping the entire structure of MobileNet V2 and V3 as they can be reinterpreted as a BSConv-S model as shown in Sect. 4.2 in the paper. As you have already observed correctly, for those models, the resulting performance gain entirely results from the regularization of the subspace transform.


Added us/ga/k12/forsyth.txt (Forsyth County School District)

Please add Forsyth County School district (https://www.forsyth.k12.ga.us/)
it has 39 schools – 21 elementary, 10 middle, 6 high, 1 virtual 6-12 school, and 1 college and career high school

@eyforia Pull request merged. Thank you!


Document PR process

Technical details:

branch protection is enabled,
require branches to be up-to-date before merging,
require travis + bundlerbot + hound status checks to pass before merging (so I don't accidentally click the Merge button out of habit)
enable "Restrict who can push to this branch", and add bundlerbot to the list

So the process is:

Someone makes a PR
When a maintainer is happy with it and Travis + Hound pass, said maintainer does @bundlerbot r+
Bundlerbot automatically merges the PR + master on a separate branch, makes sure tests still pass on Travis, and pushes it to master if they do.

I have no idea what or how to document this, but it seems like it should be documented somewhere. 🤷

My guess is in the CONTRIBUTING document - if there is such a thing.

 That does seem like a good place for it.

Fixed by #246.


Remove travis if CircleCI works well for us

We'll still need to remove the travis hook from this project. @jackgr - I don't have access to the settings page. I would expect to see it with admin rights but it's not available for me.  Could you or @vaikas-google take care of removing that?

@jackgr I see the issue with the admin rights for the deployment-manager-admins. There is not repository associated with the team. https://github.com/orgs/kubernetes/teams/deployment-manager-admins

Ah. Thanks for catching that. Will fix ASAP.
On Friday, March 4, 2016, Michelle Noorali<EMAIL_ADDRESS>wrote:

@jackgr https://github.com/jackgr I see the issue with the admin rights
for the deployment-manager-admins. There is not repository associated with
the team.
https://github.com/orgs/kubernetes/teams/deployment-manager-admins
—
Reply to this email directly or view it on GitHub
https://github.com/kubernetes/deployment-manager/issues/334#issuecomment-192535748
.

--
Jack Greenfield | Google

:+1:

Done.

Looks like on the TravisCI side the job is still enabled (which I think will try recreating hooks?). @jackgr - could you flip that slider in TravisCI to the "off" position for deployment-manager?

@sgoings Done.

@michelleN thanks for catching that the admins team wasn't configured.
I got to wondering how that could have happened... so, get this...
I knew that if you create a team through Teams tab in the kubernetes organization page, you have to set the repositories it applies to and the rights it has on them.
So, instead of going that route, I went to the Collaborators & teams tab in the Settings page for the kubernetes/deployment-manager repo, hoping it would auto populate some parameters for me, and clicked the button to Create a new team.
Sure enough, it prompted me for the rights it should have, and I chose Admin It then created a team with the automatically generated name deployment-manager-admins. You'd think it would also have added kubernetes/deployment-manager to the repositories and would have set the rights to Admin, but no, it didn't.
Apparently, it only uses the parameters to generate the name.


Terrible worst case performance, almost halting

I tested on a dataset with 1442910 words.
#[test]
fn test_worse_case() -> Result<()> {
    let case = "bring more land under cultivation";
    let conf = crate::config::get_config();
    let db_path = PathBuf::from(conf.data_path.clone());
    let db = offdict::<Strprox>::open_db(db_path)?;
    println!("testing");
    db.search(case, 3, false)?;
    Ok(())
}

I ran it with an input this long and it wouldnt finish.
https://github.com/planetoryd/offdict/tree/strprox
I made unrelated changes to your code. https://github.com/planetoryd/strprox/commits/master/

Also you'd be interested in https://github.com/Validark/DynSDT

I fixed it with https://github.com/planetoryd/strprox/commit/3a0321bf5ad9d8120d3aeda03310e1da0ed79971
not graceful tho

Hi, thanks for letting me know. I'm a little busy right now but I'll check it out later.

loading FST index
loading index from "./data/strprox"
# inconveicen
function=query duration=17.8661ms
fst: []
1/11, active matching set, 1, threshold, 1
2/11, active matching set, 2, threshold, 1
3/11, active matching set, 3, threshold, 1
4/11, active matching set, 4, threshold, 1
5/11, active matching set, 5, threshold, 1
6/11, active matching set, 6, threshold, 1
7/11, active matching set, 7, threshold, 1
8/11, active matching set, 8, threshold, 1
9/11, active matching set, 8, threshold, 1
10/11, active matching set, 305, threshold, 2
11/11, active matching set, 3403, threshold, 2
function=query duration=26.666064ms
meta: ["inconvenience"]
# inconvenize
function=query duration=16.976933ms
fst: ["inconvenient", "inconvenience", "inconveniency", "inconveniently"]
1/11, active matching set, 1, threshold, 1
2/11, active matching set, 2, threshold, 1
3/11, active matching set, 3, threshold, 1
4/11, active matching set, 4, threshold, 1
5/11, active matching set, 5, threshold, 1
6/11, active matching set, 6, threshold, 1
7/11, active matching set, 7, threshold, 1
8/11, active matching set, 8, threshold, 1
9/11, active matching set, 9, threshold, 1
10/11, active matching set, 10, threshold, 1
11/11, active matching set, 10, threshold, 1
function=query duration=885.675µs
meta: ["inconvenience"]
# bring more land under cultivation
function=query duration=23.654728ms
fst: ["bring more land under cultivation"]
1/33, active matching set, 1, threshold, 1
2/33, active matching set, 2, threshold, 1
3/33, active matching set, 3, threshold, 1
4/33, active matching set, 4, threshold, 1
5/33, active matching set, 5, threshold, 1
6/33, active matching set, 6, threshold, 1
7/33, active matching set, 7, threshold, 1
8/33, active matching set, 8, threshold, 1
9/33, active matching set, 9, threshold, 1
10/33, active matching set, 10, threshold, 1
11/33, active matching set, 11, threshold, 1
12/33, active matching set, 12, threshold, 1
13/33, active matching set, 13, threshold, 1
14/33, active matching set, 14, threshold, 1
15/33, active matching set, 15, threshold, 1
16/33, active matching set, 16, threshold, 1
17/33, active matching set, 17, threshold, 1
18/33, active matching set, 18, threshold, 1
19/33, active matching set, 19, threshold, 1
20/33, active matching set, 20, threshold, 1
21/33, active matching set, 21, threshold, 1
22/33, active matching set, 22, threshold, 1
23/33, active matching set, 23, threshold, 1
24/33, active matching set, 24, threshold, 1
25/33, active matching set, 25, threshold, 1
26/33, active matching set, 26, threshold, 1
27/33, active matching set, 27, threshold, 1
28/33, active matching set, 28, threshold, 1
29/33, active matching set, 29, threshold, 1
30/33, active matching set, 30, threshold, 1
31/33, active matching set, 31, threshold, 1
32/33, active matching set, 32, threshold, 1
33/33, active matching set, 33, threshold, 1
function=query duration=1.446931ms
meta: ["bring more land under cultivation"]
# bring more land under cultivatzon
function=query duration=19.093236ms
fst: ["bring more land under cultivation"]
1/33, active matching set, 1, threshold, 1
2/33, active matching set, 2, threshold, 1
3/33, active matching set, 3, threshold, 1
4/33, active matching set, 4, threshold, 1
5/33, active matching set, 5, threshold, 1
6/33, active matching set, 6, threshold, 1
7/33, active matching set, 7, threshold, 1
8/33, active matching set, 8, threshold, 1
9/33, active matching set, 9, threshold, 1
10/33, active matching set, 10, threshold, 1
11/33, active matching set, 11, threshold, 1
12/33, active matching set, 12, threshold, 1
13/33, active matching set, 13, threshold, 1
14/33, active matching set, 14, threshold, 1
15/33, active matching set, 15, threshold, 1
16/33, active matching set, 16, threshold, 1
17/33, active matching set, 17, threshold, 1
18/33, active matching set, 18, threshold, 1
19/33, active matching set, 19, threshold, 1
20/33, active matching set, 20, threshold, 1
21/33, active matching set, 21, threshold, 1
22/33, active matching set, 22, threshold, 1
23/33, active matching set, 23, threshold, 1
24/33, active matching set, 24, threshold, 1
25/33, active matching set, 25, threshold, 1
26/33, active matching set, 26, threshold, 1
27/33, active matching set, 27, threshold, 1
28/33, active matching set, 28, threshold, 1
29/33, active matching set, 29, threshold, 1
30/33, active matching set, 30, threshold, 1
31/33, active matching set, 31, threshold, 1
32/33, active matching set, 31, threshold, 1
33/33, active matching set, 167, threshold, 1
function=query duration=2.608483ms
meta: ["bring more land under cultivation"]
# inconvenize +4
function=query duration=16.86728ms
fst: ["inconvenient", "inconvenience", "inconveniency", "inconveniently"]
1/11, active matching set, 1, threshold, 1
2/11, active matching set, 2, threshold, 1
3/11, active matching set, 3, threshold, 1
4/11, active matching set, 4, threshold, 1
5/11, active matching set, 5, threshold, 1
6/11, active matching set, 6, threshold, 1
7/11, active matching set, 7, threshold, 1
8/11, active matching set, 8, threshold, 1
9/11, active matching set, 9, threshold, 1
10/11, active matching set, 10, threshold, 1
11/11, active matching set, 10, threshold, 1
function=query duration=707.855µs
meta: ["inconvenience", "inconveniency", "inconvenient", "inconveniently"]
# inconceivability
function=query duration=16.957656ms
fst: ["inconceivability"]
1/16, active matching set, 1, threshold, 1
2/16, active matching set, 2, threshold, 1
3/16, active matching set, 3, threshold, 1
4/16, active matching set, 4, threshold, 1
5/16, active matching set, 5, threshold, 1
6/16, active matching set, 6, threshold, 1
7/16, active matching set, 7, threshold, 1
8/16, active matching set, 8, threshold, 1
9/16, active matching set, 9, threshold, 1
10/16, active matching set, 10, threshold, 1
11/16, active matching set, 11, threshold, 1
12/16, active matching set, 12, threshold, 1
13/16, active matching set, 13, threshold, 1
14/16, active matching set, 14, threshold, 1
15/16, active matching set, 15, threshold, 1
16/16, active matching set, 16, threshold, 1
function=query duration=461.45µs
meta: ["inconceivability"]
# inconceivabilizty
function=query duration=17.329149ms
fst: ["inconceivability"]
1/17, active matching set, 1, threshold, 1
2/17, active matching set, 2, threshold, 1
3/17, active matching set, 3, threshold, 1
4/17, active matching set, 4, threshold, 1
5/17, active matching set, 5, threshold, 1
6/17, active matching set, 6, threshold, 1
7/17, active matching set, 7, threshold, 1
8/17, active matching set, 8, threshold, 1
9/17, active matching set, 9, threshold, 1
10/17, active matching set, 10, threshold, 1
11/17, active matching set, 11, threshold, 1
12/17, active matching set, 12, threshold, 1
13/17, active matching set, 13, threshold, 1
14/17, active matching set, 14, threshold, 1
15/17, active matching set, 15, threshold, 1
16/17, active matching set, 15, threshold, 1
17/17, active matching set, 180, threshold, 1
function=query duration=1.61581ms
meta: ["inconceivability"]
# inconceivabilizty +2
function=query duration=20.451395ms
fst: ["inconceivability"]
1/17, active matching set, 1, threshold, 1
2/17, active matching set, 2, threshold, 1
3/17, active matching set, 3, threshold, 1
4/17, active matching set, 4, threshold, 1
5/17, active matching set, 5, threshold, 1
6/17, active matching set, 6, threshold, 1
7/17, active matching set, 7, threshold, 1
8/17, active matching set, 8, threshold, 1
9/17, active matching set, 9, threshold, 1
10/17, active matching set, 10, threshold, 1
11/17, active matching set, 11, threshold, 1
12/17, active matching set, 12, threshold, 1
13/17, active matching set, 13, threshold, 1
14/17, active matching set, 176, threshold, 1
15/17, active matching set, 314, threshold, 2
16/17, active matching set, 3703, threshold, 3
17/17, active matching set, 21139, threshold, 3
function=query duration=214.096506ms
meta: ["inconceivability", "conceivability"]
#

Here is a comparison of strprox with an algorithm that I was previously using for my dictionary app.

Benched by https://github.com/planetoryd/offdict/blob/strprox/offdictd/examples/bench.rs
That algorithm, despite being like a hack, handles my use case well in terms of speed and correction.
And
 12M Dec 30 01:47 fst
204M Dec 29 23:29 strprox

@planetoryd
I'm checking to see if there's something wrong with my implementation compared to the paper. However, the paper does not have experimental results for queries with more than 10 characters, and it suggests segmenting queries into words in combination with techniques from other papers.
In the meantime, I'd like to merge your commits upstream if you don't mind.

Is it possible to implement this algorithm with https://crates.io/crates/fst . IDK

Looks interesting. I'm looking into it.

Is this a feature, or an inherent problem of this algorithm ? I stop the search when the threshold gets too big, and that's result, which is embarrassing.
Perhaps I am not using it correctly. And I should probably introduce other algorithms to nullify its downsides, like pruning radix trie for exact matches before fuzzy is needed

with more lenient caps.

With that kind of performance it's hard to say what the point is. I can get threshold filtered results with FST with levenshtein-distance of one (which is effectively the same as top-k)

The algorithm will always return the requested number of strings if there are enough strings in the dataset, whereas using a threshold of one may return less than that. In this case, it seems that it had to increase the threshold to get more strings, and if you used the threshold of 0, it would only return 1 string rather than 4.
A simple way to implement a top-k algorithm using FST is to try getting strings using Levenshtein automata beginning with an edit distance threshold of 0, then incrementally increasing the threshold by 1 if k strings weren't found. However, reconstructing Levenshtein automata and using the intersection operation incurs the overhead of reiterating through states. I haven't measured the performance of that approach and how it compares to my current implementation. I think reiteration can be avoided by using a Levensthein automaton with a threshold equal to the query length, and buffering states whose edit distances are higher than the current dynamic threshold to check later if less than k strings were found, but this may require a different implementation than the one currently in fst.
I have been working on an algorithm using FST but without Levenshtein automata.

oh thats true. I figured out but forgot about that.
do you have an idea on the impl of cache. some kind of LRU should suffice, but its a lil bit different because of the structure.

For my current implementation of META, the matching set, threshold, and the associated query prefix can be cached to resume searching through autocomplete_step when the user continues typing if the requested number of strings is the same. This can be done for each prefix of a query, and if the user deletes part of the query and continues typing, the cache for the unchanged prefix of the query can be accessed, and the cache for the prefixes of the previous query longer than that can be deleted or updated. The cache can be implemented using a vector where each index corresponds to a prefix length and each element has a cache entry/entries.

I would like it return if there are results that are close and cheap, and not like the top-k manner which is expensive in practice. I'm not sure how to do it well.
  while query_prefix_len <= query_chars.len() {
            debug_println!(
                "{}/{}, active matching set, {}, threshold, {}. results {}/{}",
                query_prefix_len,
                query_chars.len(),
                active_matching_set.matchings.len(),
                threshold,
                result.len(),
                requested
            );
            result.clear();
            let mut set_delta = Default::default();
            let proposed_threshold = self.autocomplete_step(
                &mut active_matching_set,
                &query_chars[..query_prefix_len],
                query_prefix_len,
                threshold,
                requested,
                &mut result,
                &mut set_delta,
            );
            if set_delta.matchings.len() > MAX_SET_DELTA || proposed_threshold > MAX_DISTANCE {
                if requested == 1 {
                    return vec![];
                }
                query_prefix_len = 1;
                requested = 1;
                active_matching_set.matchings.clear();
                continue;
            }
            threshold = proposed_threshold;
            active_matching_set.matchings.extend(set_delta.matchings);
            query_prefix_len += 1;
        }

There is usually one result within distance of one, and in worse cases the remaining of top-K requires searching through distance of 3, 4.

I made some changes in the worst-case branch. It exposes a threshold_topk method that requires the top-k results to have a PED within a certain threshold, which also lets it return less than k results.
I also added an algorithm using FST. It basically scans it but attempts to do early-termination for PED calculations based on the threshold, and shares computations between characters that do not match anything in the query.
The FST algo has better performance than my implementation of META when the top-k includes high PEDs (it can handle your long query test in less than 1 second).
FST, long query

Current META implementation, long query

However, the META implementation is faster for reasonable thresholds using threshold_topk or when the top-k's PEDs are low (such as in the bounded peds test), and when the PEDs are really high then it's faster to use the plain unindexed_autocomplete. Take a look at the bench_noise test, which uses randomly-generated data from make_noise.

I wonder if it's possible to add other metrics like prioritizing more frequently used words.

Linux should be a much better answer

I also found https://www.sciencedirect.com/science/article/pii/S111001682030260X which claims to be an improvement over META

In this paper, we propose an efficient algorithm, named Depth-Based Fuzzy Auto-Completion (DFA), that avoids this redundant work and minimizes the computations to find the appropriate fuzzy auto-completion of the queries. The main idea of DFA is to store the Matching Nodes into a min heap (sorted based on the prefix edit distance with the query string) and retrieve them one by one until the target k is reached.

I had attempted to use that algorithm before META, but it didn't make sense to me. The specific example I found was that when evaluating the string "orange" against a query "oange," when there's a matching node <"o", 1, 0, 1, 0> for the first "o"-s, in CHECK-DESCENDANTS the condition for descendant nodes is that the depth is less than 1 + 0 + 1 = 2, but the depth of the node with "a" in "orange" is 3, so it doesn't consider "orange".

Sorry, my notes aren't detailed. If you prefer, for specific questions you can reach out to me<EMAIL_ADDRESS>or send me contact info for a communications platform like Discord.
The current META implementation does not use Compact Tree (Section 5), which leads to redundant node accesses. There may be a significant difference in performance if it had Compact Tree.
first_deducing collects the minimum DEDs of a node in a map to get the real ED (Lemma 1) to add new matchings.
active_matching_set was a misnomer on my part because the b-matching set isn't necessarily an active matching set. Due to the subset relation between a b-matching set and the next, matchings are never pruned away from it and the PED of a matching may increase for the next query prefix while the max top-k PED may not.

I'd clarify that in practical use it's about returning results as soon as they are produced, which is unlike a hard-coded threshold. So It should be a streaming iterator that gradually returns more distant strings. The ideal interface is to be an iterator, rather than top-k or threshold.

I'm thinking about doing this using an intersection between a nondeterministic Levenshtein automaton (to avoid the construction cost of a deterministic one) and an FST dataset, where states that have higher PEDs than the current target are buffered and checked later once there are no more candidates for the target PED.

I mean the invariants of this algorithm.

I don't know what makes sure that it is visiting enough nodes and why what it doesn't visit certainly doesn't lead to results.
Have you heard of Lean4, or Dafny ? It'd be nice to have some machine checked proofs.
I've been following some Rust formal verification tools

Prusti
Verus

Both are not even usable yet. Prusti needs one PR of std lib merged, and Verus is pre alpha or like that.

I don't have experience with proof assistants yet, although Lean does look interesting.
A b-matching set contains all matchings whose edit distance between a query prefix and the stored prefix is within b.
$\cal{P}(q_i, b_i) = \set{ m | m.i \leq i \land \text{ED}(q[1..m.i], m.n) \leq b_i }$
b is equal to the max PED of the top-k results for the current query prefix by definition.
$|R_i| = k$
$\forall s \in R_i, \forall x \in \cal{T}, x \notin R_i \implies \text{PED}(q_i, s) \leq \text{PED}(q_i, x)$
$\exists s \in R_i, \text{PED}(q_i, s) = b_i$
$\forall s \in R_i, \text{PED}(q_i, s) \leq b_i$
The max PED for an empty query is 0 because all strings have an empty prefix that matches perfectly.
$\forall s \in \cal{T}, \text{PED}(q_0, s) = \text{PED}("", "") = 0$
$\cal{P}(q_0, 0) = {<0,\text{root},0>}$
Adding a character to the query changes the prefix edit distance from the strings whose $\text{PED} \leq b$ by 0 if the next character after the best prefix matches and 1 otherwise.
$b_i \in [b_{i-1}, b_{i-1} + 1]$ (Lemma 4)
Therefore the next b-matching set is a superset of the previous.
$\forall m \in \cal{P}(q_{i-1}, b_{i-1}), m.i \leq i-1 < i \land m.ed \leq b_i \leq b_{i-1} + 1$
$\therefore \forall m \in \cal{P}(q_{i-1}, b_{i-1}), m \in \cal{P}(q_i, b_i)$.
 The edit distance must be greater than or equal to the previous by the definition of Levenshtein edit distance. All the previous matchings used in Lemma 3 for $\cal{P}(q_i, b_i-1)$ are in $\cal{P}(q_{i-1}, b_i-1)$ because $\forall m \in \cal{M}(q_{i-1}, s[1, j-1]), m.ed \leq b_i - 1 \implies m \in \cal{P}(q_{i-1}, b_i-1)$, so they are used to find new matchings (1.2). This results in the entirety of $\cal{P}(q_i, b_i-1)$ when the descendant traversal checks all nodes that match the new character using the inverted index and can have an edit distance within $b_i - 1$ based on the depth difference.
For the computation of $\cal{P}(q_i, b_i) - \cal{P}(q_i, b_i - 1)$ (2), a similar process is used, except not all the previous matchings may be available in $\cal{P}(q_i, b_i - 1)$. When the previous matching isn't in $\cal{P}(q_i, b_i - 1)$, the previous matching also has an edit distance of $b_i$ because edit distance does not decrease, so descendant traversal simply repeats for new matchings, and this way the entirety of $\cal{P}(q_i, b_i) - \cal{P}(q_i, b_i - 1)$ is found.

Hi, sorry for the delay. I finally implemented non-deterministic Levenshtein automata! They can be used for the streaming behavior you wanted, however they are still somewhat slow and memory-expensive.

Constructing deterministic Levenshtein automata for large edit distances isn't feasible because its space complexity is exponential in edit distance.
I think a possible solution for your use case is to segment all English strings into words, map each word to a character, and create an index using strings formed by the mapped characters. Then transform each word in the query to the nearest word in the dataset, map those words to characters the same way, and use the new string formed by those characters with error-tolerant autocomplete.
We could also consider heuristics or existing methods/alternative metrics like n-gram/Jaccard similarity.

I've been working on an alternative implementation of META. I stack FirstDeducing and SecondDeducing in a different way.

It's still in the works

there are 1442910 words in the index

Nice. I'm still busy with other stuff but I'll take a closer look later.

There is some minor error here. I'll be working on it

Some parameters need tuning

It's mostly usable as my daily tool. Prolly has some bugs but now I have to work on something else.


[UI] Initial implement the SQL Lab page

Why are the changes needed?

New SQL Lab Page
New core editor functionality

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[x] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Was this patch authored or co-authored using generative AI tooling?

No.

Wow, that's really impressive!

good job. Thanks.

👍

Merged to master for 1.9


Fix Extract to Variable crash when extracting entire file

Motivation

Test plan

See included automated tests.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_R7kiK1Xz6YPF0D
→ https://go/builds/bui_R7ki7FRymJry1J
→ https://go/builds/bui_R7kiwSjd5QjSLP


Multiple Connection

@fcwu
I have the following scenario where there is a need for simultaneous connections to the same session/screens
a. Student: Who will have access to modify the screen/session - has keyboard and mouse access
b. Teacher: Who will have "view-only" access to the screen/session to view/observe (but not modify) what the Student is doing.
How should that be setup?

Try the latest version. The point is this line
https://github.com/fcwu/docker-ubuntu-vnc-desktop/blob/master/image/etc/supervisor/conf.d/supervisord.conf#L27

Close due to no response


upgrading from previous rules

so, I'm trying now... it isn't working as a drop in replacement.
I'm hitting a few issues:
the version that previously worked won't download:
graal_bindist_repository(
    name = "graal",
    java_version = "11",
    version = "22.1.0",
)

changing to:
graalvm_repository(
    name = "graalvm",
    java_version = "11",
    version = "22.1.0",
)

gives me:
INFO: Repository graalvm instantiated at:
  /Users/oboykin/code/rankresearchv2/WORKSPACE:217:19: in <toplevel>
  /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl:684:36: in graalvm_repository
Repository rule _graalvm_bindist_repository defined at:
  /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl:547:46: in <toplevel>
ERROR: An error occurred during the fetch of repository 'graalvm':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl", line 278, column 17, in _graal_bindist_repository_impl
                fail("Unable to locate GraalVM distribution '%s' at version '%s' for platform '%s'" % (
Error in fail: Unable to locate GraalVM distribution 'oracle' at version '22.1.0' for platform 'macos-aarch64'
ERROR: /Users/oboykin/code/rankresearchv2/WORKSPACE:217:19: fetching _graalvm_bindist_repository rule //external:graalvm: Traceback (most recent call last):
        File "/private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl", line 278, column 17, in _graal_bindist_repository_impl
                fail("Unable to locate GraalVM distribution '%s' at version '%s' for platform '%s'" % (
Error in fail: Unable to locate GraalVM distribution 'oracle' at version '22.1.0' for platform 'macos-aarch64'
ERROR: /Users/oboykin/code/rankresearchv2/build_tools/bazel_rules/jar_scanner/BUILD.bazel:33:13: //build_tools/bazel_rules/jar_scanner:jarscanner-native depends on @graalvm//:native-image in repository @graalvm which failed to fetch. no such package '@graalvm//': Unable to locate GraalVM distribution 'oracle' at version '22.1.0' for platform 'macos-aarch64'

it's a bit hard for me to follow through which platforms are defined. Adding distribution = "ce" gives:
ERROR: /Users/oboykin/code/rankresearchv2/build_tools/bazel_rules/jar_scanner/BUILD.bazel:33:13: //build_tools/bazel_rules/jar_scanner:jarscanner-native depends on @graalvm//:native-image in repository @graalvm which failed to fetch. no such package '@graalvm//': Unable to locate GraalVM distribution 'ce' at version '22.1.0' for platform 'macos-aarch64'

this worked previously for us. We need to build on macos arm and linux.
The default distribution is set to "oracle" which is a change (which could open users to penalties from oracle btw. I think this is a very bad default): https://github.com/sgammon/rules_graalvm/blob/aedd413d30acaa4d8677c63c1772ff8f26df8c53/internal/graalvm_bindist.bzl#L574
It seems like adding distribution = None would trigger this branch:
https://github.com/sgammon/rules_graalvm/blob/aedd413d30acaa4d8677c63c1772ff8f26df8c53/internal/graalvm_bindist.bzl#L193
but that fails differently:
INFO: Repository graalvm instantiated at:
  /Users/oboykin/code/rankresearchv2/WORKSPACE:217:19: in <toplevel>
  /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl:684:36: in graalvm_repository
Repository rule _graalvm_bindist_repository defined at:
  /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl:547:46: in <toplevel>
ERROR: An error occurred during the fetch of repository 'graalvm':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl", line 255, column 31, in _graal_bindist_repository_impl
                dist_name = dist_names[ctx.attr.distribution]
Error: key "" not found in dictionary
ERROR: /Users/oboykin/code/rankresearchv2/WORKSPACE:217:19: fetching _graalvm_bindist_repository rule //external:graalvm: Traceback (most recent call last):
        File "/private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl", line 255, column 31, in _graal_bindist_repository_impl
                dist_name = dist_names[ctx.attr.distribution]
Error: key "" not found in dictionary

so I think the distribution = None may be set to the empty string since the distribution is a string? idk. It's somehow missing the branch.
I'll poke a bit more, but it looks like a lot of the code branches may not have test coverage, or at least cases we need don't.

okay...
I see the confusion now...
both version and java_version are the same. I thought one was the graalvm version and one was the java version. I don't see why we need both. And we should alert of the major version of version isn't the same as java_version.
That was very unclear to me.

with:
graalvm_repository(
    name = "graalvm",
    #components = [],
    distribution = "ce", #"ce",  # `oracle`, `ce`, or `community`
    java_version = "17",  # `17`, `20`, or `21`, as supported by the version provided
    version = "17.0.8",  # earlier version format like `22.x` also supported
)

I can download, but then I hit a new error:
INFO: Repository graalvm instantiated at:
  /Users/oboykin/code/rankresearchv2/WORKSPACE:221:19: in <toplevel>
  /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl:686:36: in graalvm_repository
Repository rule _graalvm_bindist_repository defined at:
  /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl:549:46: in <toplevel>
ERROR: An error occurred during the fetch of repository 'graalvm':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl", line 332, column 33, in _graal_bindist_repository_impl
                ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/graalvm/temp12222422846218164260/graalvm-community-jdk-17.0.8_macos-aarch64_bin.tar.gz to /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/graalvm/temp12222422846218164260: Prefix "graalvm-community-openjdk-17.0.8+9.1/Contents/Home" was given, but not found in the archive. Here are possible prefixes for this archive: "graalvm-community-openjdk-17.0.8+7.1".
ERROR: /Users/oboykin/code/rankresearchv2/WORKSPACE:221:19: fetching _graalvm_bindist_repository rule //external:graalvm: Traceback (most recent call last):
        File "/private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/rules_graalvm/internal/graalvm_bindist.bzl", line 332, column 33, in _graal_bindist_repository_impl
                ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/graalvm/temp12222422846218164260/graalvm-community-jdk-17.0.8_macos-aarch64_bin.tar.gz to /private/var/tmp/_bazel_oboykin/63808c00052f93fa156f2d8173529afd/external/graalvm/temp12222422846218164260: Prefix "graalvm-community-openjdk-17.0.8+9.1/Contents/Home" was given, but not found in the archive. Here are possible prefixes for this archive: "graalvm-community-openjdk-17.0.8+7.1".

making the following change I could bypass the previous issue:
--- a/internal/graalvm_bindist_map.bzl
+++ b/internal/graalvm_bindist_map.bzl
@@ -64,7 +64,7 @@ _AlignedVersions = {
 _VmReleaseVersions = {
     "20.0.2": "20.0.2+9.1",
     "20.0.1": "20.0.1+9.1",
-    "17.0.8": "17.0.8+9.1",
+    "17.0.8": "17.0.8+7.1",
     "17.0.7": "17.0.7+9.1",
 }

fixes I needed to get working here:
https://github.com/sgammon/rules_graalvm/pull/58/files

@johnynek thank you for this very thorough review. the default at oracle was not intended. I'm preparing a set of fixes based on this and your other reported issue.

btw, great point on alerting re/alignment on version and java_version
the reasoning there: GraalVM releases are not "aligned" to OpenJDK releases until the latest version. starting at the latest release, 20.x conveniently refers, always, to Java 20, and so on. obviously, this is an issue with past versions like 22.1.0 which would normally seem newer (but are in fact older).
the inner version is an interesting issue - the change for +    "17.0.8": "17.0.8+7.1",. we can probably fix that by first checking a map of VM release versions specifically for the older bindist. i don't think adding to the old bindist is an issue since the new bindist is auto-generated and so we shouldn't have an ongoing maintenance burden there.

(i also must know the major java version in order to set up a java toolchain -- so perhaps we could drop that functionality for the legacy rule, which would let me drop any requirement for a java_version attribute. let me know your thoughts?)

The oracle toolchain is also broken: the mapping downloads the standard oracle jdk instead of the Graal one, and fails because the file name is completely different: https://github.com/sgammon/rules_graalvm/blob/aedd413d30acaa4d8677c63c1772ff8f26df8c53/internal/graalvm_bindist_map.bzl#L1529-L1537
ERROR: An error occurred during the fetch of repository 'rules_graalvm~override~graalvm~graalvm':
   Traceback (most recent call last):
	File "/home/isaac/.cache/bazel/_bazel_isaac/995a612fe7c069966079f6b33dd349d1/external/rules_graalvm~override/internal/graalvm_bindist.bzl", line 330, column 33, in _graal_bindist_repository_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /home/isaac/.cache/bazel/_bazel_isaac/995a612fe7c069966079f6b33dd349d1/external/rules_graalvm~override~graalvm~graalvm/temp10856015587010176410/jdk-17.0.8_linux-x64_bin.tar.gz to /home/isaac/.cache/bazel/_bazel_isaac/995a612fe7c069966079f6b33dd349d1/external/rules_graalvm~override~graalvm~graalvm/temp10856015587010176410: Prefix "graalvm-jdk-17.0.8+9.1/" was given, but not found in the archive. Here are possible prefixes for this archive: "jdk-17.0.8".
ERROR: <builtin>: fetching _graalvm_bindist_repository rule //:rules_graalvm~override~graalvm~graalvm: Traceback (most recent call last):
	File "/home/isaac/.cache/bazel/_bazel_isaac/995a612fe7c069966079f6b33dd349d1/external/rules_graalvm~override/internal/graalvm_bindist.bzl", line 330, column 33, in _graal_bindist_repository_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /home/isaac/.cache/bazel/_bazel_isaac/995a612fe7c069966079f6b33dd349d1/external/rules_graalvm~override~graalvm~graalvm/temp10856015587010176410/jdk-17.0.8_linux-x64_bin.tar.gz to /home/isaac/.cache/bazel/_bazel_isaac/995a612fe7c069966079f6b33dd349d1/external/rules_graalvm~override~graalvm~graalvm/temp10856015587010176410: Prefix "graalvm-jdk-17.0.8+9.1/" was given, but not found in the archive. Here are possible prefixes for this archive: "jdk-17.0.8".

@nuudlman hm, you're right. i don't know how that was missed. i'm sorry about that. let me fix.

@johnynek / @nuudlman this next release, 0.9.2, should fix these issues. allow me to summarize what's been reported so far below, along with fixes for each

the version that previously worked won't download [.] changing to [...] gives me [...]

I've made a few changes here:

Alias for graal_bindist_repository (no need to change these callsites)
Alias for graal_binary (no need to change these callsites)
Downloads for previous GraalVM versions are fixed
Integration test added for this exact case (it's called legacy-rules, in example/integration_tests)

This test will now be required for future PRs

Adding distribution = "ce" gives [...]

The previous behavior was that passing distribution = None would actually trigger the legacy code, and this would have worked fine had I not defaulted to Oracle GVM. I guess I read their blog post about licensing and thought Oracle GVM was now the same as CE w.r.t. licensing, which is not a safe assumption. That new license is [here], if curious. After reading the license itself it's not as "free" as the title might suggest, so thank you very much for catching this.
Explicitly having to pass distribution = None is awkward and was not the intended outcome. The default has been restored to None, which allows the legacy-style invocation (no value for distribution attr). If a default is needed, it now defaults to CE.

but that fails differently [...]

The previous versions should work now. I think we should keep this map because it allows eliding support for those older versioning schemes in the mapping generator (which is targeting the new release cycle). The sha mismatch seems like a bad global find/replace, so I've fixed that, of course, too.

both version and java_version are the same. I thought one was the graalvm version and one was the java version. I don't see why we need both. And we should alert of the major version of version isn't the same as java_version.

This is driven by two requirements, at least one of which we can safely drop:

Setting up Java toolchains. I need to know the major Java version so that I can pass it to Bazel, which uses it for the java_runtime_version property and related toolchain selection. If there is no desire to set up a Java toolchain, I can drop this logic, and document the difference. Thoughts? Is it appealing to use a GraalVM Java toolchain on previous rule versions? It would possibly save from another JDK download (depending on what's in use), but it's up to you.

GraalVM versions which cover multiple JVMs. After the recent version alignment, 20.x.x conveniently always means "Java 20." Before this release, including the one listed here, each GraalVM engine version was also subordinated by a Java version. In these cases I think we need the Java major version in order to properly calculate the download URL, inner strip_prefix, and so on. If I'm mistaken here, please point it out to me; these multiple version schemes are hard to coordinate in my head 😅

I can download, but then I hit a new error [...] making the following change I could bypass

It looks like GVM CE and Oracle GVM have different inner strip_prefix values, which I hadn't noticed. The prefixes are off by one character. I don't think these prefixes should change, I just think it means they may not be consistent across CE/Oracle releases. As such, I've added a second map to the bindist which covers this mapping.
The fix applied should clear up archive strip_prefix issues for both Oracle and CE.

fixes I needed to get working here

Thank you for filing -- merged -- and, I invited you @johnynek to collab on this repo. Feel free to merge an urgent fix if you need to at any time.
@nuudlman

The oracle toolchain is also broken: the mapping downloads the standard oracle jdk

It looks like I had a copy/paste error at some point, and didn't notice because I was testing component dependency downloads on CE. Very embarrassed about this! It is now fixed, and thank you for your report as well.

I will merge the PR attached after it passes tests, then issue a new release. Thank you both for testing!

@johnynek apologies, i missed this one

And we should alert of the major version of version isn't the same as java_version.

Yes, I agree, to the extent this is possible. I will file an issue to cover the confusion around these properties. I can work to make java_version optional where possible and, at the very least, document why both are needed for some releases of GVM.

@johnynek apologies, i missed this one

And we should alert of the major version of version isn't the same as java_version.

Yes, I agree, to the extent this is possible. I will file an issue to cover the confusion around these properties. I can work to make java_version optional where possible and, at the very least, document why both are needed for some releases of GVM.

Thanks for your work on this.
I'll try to use main at work on Monday.

@johnynek sounds good. thank you likewise for reporting -- i deeply appreciate it.
have a great weekend


doc: add to third party plugins list elasticsearch logger

Hey there!
Do you mind if I add my logger, which parses Gatling's logs and sends them to Elasticsearch, to third-party plugins?
https://github.com/Amerousful/gatling-elasticsearch-logs

Hey,
Sorry, but we can't add this one.
First, the log file format is an internal to the HTML reports module. It has changed several times in the past and is likely to change again in the future (we're considering a binary format).
Then, new reporting features fall in the scope of Gatling Enterprise, so we can't advertise a competing solution.
Thanks for your understanding.

I see. It's a pity, but ok)
thx anyway


Fail to connect after blue screen waiting from Windows 11 official client

xrdp version
<IP_ADDRESS>
Detailed xrdp version, build options
xrdp <IP_ADDRESS>
A Remote Desktop Protocol Server.
Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
See https://github.com/neutrinolabs/xrdp for more information.
Configure options:
Compiled with OpenSSL 3.0.8 7 Feb 2023
Operating system & version
Ubuntu 23.04
Installation method
git clone & make install
Which backend do you use?
xorgxrdp
What desktop environment do you use?
GNOME
Environment xrdp running on
physical machine with RTX 3090
What's your client?
Microsoft official client
Area(s) with issue?
No response
Steps to reproduce

Start Microsoft 'Remote Desktop Connection' on Win11 Pro 23H2
Input the computer's ip address (fetched using 'ip address' on ubuntu), click 'Connect'
Encounter a warning window and click 'Yes'

✔ Expected Behavior
get connected to remote desktop
❌ Actual Behavior
Client behavior
After waiting in blue screen for ~3 minutes, I got this error message

xrdp log
[20240313-21:50:12] [INFO ] address [<IP_ADDRESS>] port [3389] mode 1
[20240313-21:50:12] [INFO ] listening to port 3389 on <IP_ADDRESS>
[20240313-21:50:12] [INFO ] xrdp_listen_pp done
[20240313-21:50:14] [INFO ] starting xrdp with pid 5853
[20240313-21:50:14] [INFO ] address [<IP_ADDRESS>] port [3389] mode 1
[20240313-21:50:14] [INFO ] listening to port 3389 on <IP_ADDRESS>
[20240313-21:50:14] [INFO ] xrdp_listen_pp done
[20240313-21:52:50] [INFO ] Socket 12: AF_INET connection received from <IP_ADDRESS> port 32313
[20240313-21:52:50] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[20240313-21:52:50] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[20240313-21:52:50] [INFO ] Security protocol: configured [SSL|RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[20240313-21:52:50] [ERROR] SSL_read: I/O error
[20240313-21:52:50] [ERROR] libxrdp_force_read: header read error
[20240313-21:52:50] [ERROR] Processing [ITU-T T.125] Connect-Initial failed
[20240313-21:52:50] [ERROR] [MCS Connection Sequence] receive connection request failed
[20240313-21:52:50] [ERROR] xrdp_sec_incoming: xrdp_mcs_incoming failed
[20240313-21:52:50] [ERROR] xrdp_rdp_incoming: xrdp_sec_incoming failed
[20240313-21:52:50] [ERROR] xrdp_process_main_loop: libxrdp_process_incoming failed
[20240313-21:52:50] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[20240313-21:52:50] [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed
[20240313-21:52:54] [INFO ] Socket 12: AF_INET connection received from <IP_ADDRESS> port 32315
[20240313-21:52:54] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[20240313-21:52:54] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[20240313-21:52:54] [INFO ] Security protocol: configured [SSL|RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[20240313-21:52:54] [INFO ] Connected client computer name: LAPTOP-469AJF4T
[20240313-21:52:54] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc006 is unknown (ignored)
[20240313-21:52:54] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc00a is unknown (ignored)
[20240313-21:52:54] [INFO ] xrdp_load_keyboard_layout: Keyboard information sent by the RDP client, keyboard_type:[0x04], keyboard_subtype:[0x00], keylayout:[0x00000804]
[20240313-21:52:54] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
[20240313-21:52:54] [INFO ] TLS connection established from <IP_ADDRESS> port 32315: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
[20240313-21:52:54] [INFO ] xrdp_caps_process_pointer: client supports new(color) cursor
[20240313-21:52:54] [INFO ] xrdp_process_offscreen_bmpcache: support level 1 cache size 10485760 MB cache entries 100
[20240313-21:52:54] [INFO ] xrdp_caps_process_codecs: nscodec, codec id 1, properties len 3
[20240313-21:52:54] [WARN ] xrdp_caps_process_codecs: unknown codec id 5
[20240313-21:52:54] [INFO ] xrdp_caps_process_codecs: RemoteFX, codec id 3, properties len 49
[20240313-21:52:54] [WARN ] Cannot find keymap file /etc/xrdp/km-00000804.ini
[20240313-21:52:54] [WARN ] Cannot find keymap file /etc/xrdp/km-00000804.ini
[20240313-21:52:54] [INFO ] Loading keymap file /etc/xrdp/km-00000409.ini
[20240313-21:52:54] [WARN ] local keymap file for 0x00000804 found and doesn't match built in keymap, using local keymap file
[20240313-21:52:59] [INFO ] connecting to sesman on <IP_ADDRESS>:3350
[20240313-21:52:59] [INFO ] xrdp_wm_log_msg: sesman connect ok
[20240313-21:52:59] [INFO ] sesman connect ok
[20240313-21:52:59] [INFO ] sending login info to session manager. Please wait...
[20240313-21:52:59] [INFO ] xrdp_wm_log_msg: login successful for user fs on display 10
[20240313-21:52:59] [INFO ] login successful for user fs on display 10
[20240313-21:52:59] [INFO ] loaded module 'libxup.so' ok, interface size 10296, version 4
[20240313-21:52:59] [INFO ] started connecting
[20240313-21:52:59] [INFO ] lib_mod_connect: connecting via UNIX socket
[20240313-21:56:29] [INFO ] connection problem, giving up
[20240313-21:56:29] [INFO ] some problem
[20240313-21:56:29] [ERROR] xrdp_wm_log_msg: Error connecting to user session
[20240313-21:56:29] [INFO ] Error connecting to user session

Anything else?
No response

As mentioned in issue form,  could you add the log?

Links? References? Anything that will give us more context about the issue you are encountering! We recommend attaching xrdp.log, xrdp-sesman.log, xrdp/xorg.conf or screenshots to clarify context.

Sure. xrdp.log is shown above, here is other logs
xrdp-sesman.log
[20240313-21:50:12] [INFO ] sesman_main_loop: sesman asked to terminate
[20240313-21:50:12] [INFO ] starting xrdp-sesman with pid 5851
[20240313-21:52:59] [INFO ] Socket 12: AF_INET connection received from <IP_ADDRESS> port 49740
[20240313-21:52:59] [INFO ] Terminal Server Users group is disabled, allowing authentication
[20240313-21:52:59] [INFO ] ++ created session (access granted): username fs, ip <IP_ADDRESS>:32315 - socket: 12
[20240313-21:52:59] [INFO ] starting Xorg session...
[20240313-21:52:59] [INFO ] Starting session: session_pid 6232, display :10.0, width 2560, height 1600, bpp 24, client ip <IP_ADDRESS>:32315 - socket: 12, user name fs
[20240313-21:52:59] [INFO ] [session start] (display 10): calling auth_start_session from pid 6232
[20240313-21:52:59] [ERROR] sesman_data_in: scp_process_msg failed
[20240313-21:52:59] [ERROR] sesman_main_loop: trans_check_wait_objs failed, removing trans
[20240313-21:52:59] [INFO ] Starting X server on display 10: Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log
[20240313-21:53:09] [WARN ] Timed out waiting for X server on display 10 to startup
[20240313-21:53:09] [INFO ] Session started successfully for user fs on display 10
[20240313-21:53:09] [INFO ] Starting the xrdp channel server for display 10
[20240313-21:53:09] [INFO ] Session in progress on display 10, waiting until the window manager (pid 6233) exits to end the session
[20240313-21:53:09] [WARN ] Timed out waiting for X server on display 10 to startup
[20240313-21:53:09] [ERROR] There is no X server active on display 10
[20240313-21:53:09] [ERROR] A fatal error has occurred attempting to start the window manager on display 10, aborting connection
[20240313-21:53:09] [WARN ] Window manager (pid 6233, display 10) exited quickly (0 secs). This could indicate a window manager config problem
[20240313-21:53:09] [INFO ] Calling auth_stop_session and auth_end from pid 6232
[20240313-21:53:09] [INFO ] Terminating X server (pid 6234) on display 10
[20240313-21:53:09] [INFO ] Terminating the xrdp channel server (pid 6247) on display 10
[20240313-21:53:09] [INFO ] X server on display 10 (pid 6234) returned exit code 1 and signal number 0
[20240313-21:53:09] [INFO ] xrdp channel server for display 10 (pid 6247) exit code 0 and signal number 0
[20240313-21:53:09] [INFO ] cleanup_sockets:
[20240313-21:53:09] [INFO ] Process 6232 has exited
[20240313-21:53:09] [INFO ] ++ terminated session:  username fs, display :10.0, session_pid 6232, ip <IP_ADDRESS>:32315 - socket: 12

xrdp/xorg.conf
I find xrdp/xorg.conf at this path /etc/X11/xrdp/xorg.conf, and here is its content:
Section "ServerLayout"
    Identifier "X11 Server"
    Screen "Screen (xrdpdev)"
    InputDevice "xrdpMouse" "CorePointer"
    InputDevice "xrdpKeyboard" "CoreKeyboard"
EndSection

Section "ServerFlags"
    # This line prevents "ServerLayout" sections in xorg.conf.d files
    # overriding the "X11 Server" layout (xrdp #1784)
    Option "DefaultServerLayout" "X11 Server"
    Option "DontVTSwitch" "on"
    Option "AutoAddDevices" "off"
EndSection

Section "Module"
    Load "dbe"
    Load "ddc"
    Load "extmod"
    Load "glx"
    Load "int10"
    Load "record"
    Load "vbe"
    Load "glamoregl"
    Load "xorgxrdp"
    Load "fb"
EndSection

Section "InputDevice"
    Identifier "xrdpKeyboard"
    Driver "xrdpkeyb"
EndSection

Section "InputDevice"
    Identifier "xrdpMouse"
    Driver "xrdpmouse"
EndSection

Section "Monitor"
    Identifier "Monitor"
    Option "DPMS"
    HorizSync 30-80
    VertRefresh 60-75
    ModeLine "1920x1080"<PHONE_NUMBER> 1968 2000 2080 1080 1083 1088 1111 +hsync -vsync
    ModeLine "1280x720" 74.25 1280 1720 1760 1980 720 725 730 750 +HSync +VSync
    Modeline "1368x768" 72.25 1368 1416 1448 1528 768 771 781 790 +hsync -vsync
    Modeline "1600x900" 119.00 1600 1696 1864 2128 900 901 904 932 -hsync +vsync
EndSection

Section "Device"
    Identifier "Video Card (xrdpdev)"
    Driver "xrdpdev"
    Option "DRMDevice" "/dev/dri/renderD128"
    Option "DRI3" "1"
EndSection

Section "Screen"
    Identifier "Screen (xrdpdev)"
    Device "Video Card (xrdpdev)"
    Monitor "Monitor"
    DefaultDepth 24
    SubSection "Display"
        Depth 24
        Modes "640x480" "800x600" "1024x768" "1280x720" "1280x1024" "1600x900" "1920x1080"
    EndSubSection
EndSection

I follow this issue comment and replace param=Xorg  with param=/usr/lib/xorg/Xorg so that the X server start successfully.
Then I follow this issue comment to stay logged out at my physical machine, and I successfully connect to it from my Win11 MS Official Remote Desktop Connector.
Although there is a bit latency, the desktop looks odd and can't change configuration in settings, it still works amazing!


Add "Deploying an 11ty Site to Azure Static Web Apps" tutorial link

Preview
https://deploy-preview-1060--11ty.netlify.app/docs/tutorials/

Thank you!


Error handling in FC2LiveStream.get_meta

I got an exception in my instance of autofc2 with following stacktrace:
2023-03-18 16:24:24 [fc2 [redacted]] Fetching stream info
Task exception was never retrieved
future: <Task finished name='Task-2733449' coro=<AutoFC2.handle_channel() done, defined at [redacted]/lib/python3.11/site-packages/fc2_live_dl/autofc2.py:90>
 exception=KeyError('data')>
Traceback (most recent call last):
  File "[redacted]/lib/python3.11/site-packages/fc2_live_dl/autofc2.py", line 93, in handle_channel
    await fc2.download(channel_id)
  File "[redacted]/lib/python3.11/site-packages/fc2_live_dl/FC2LiveDL.py", line 97, in download
    await live.wait_for_online(self.params["wait_poll_interval"])
  File "[redacted]/lib/python3.11/site-packages/fc2_live_dl/fc2.py", line 209, in wait_for_online
    while not await self.is_online():
              ^^^^^^^^^^^^^^^^^^^^^^
  File "[redacted]/lib/python3.11/site-packages/fc2_live_dl/fc2.py", line 215, in is_online
    meta = await self.get_meta(refetch=refetch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[redacted]/lib/python3.11/site-packages/fc2_live_dl/fc2.py", line 277, in get_meta
    data["data"]["channel_data"]["title"]
    ~~~~^^^^^^^^
KeyError: 'data'
2023-03-30 06:50:59 [fc2 [redacted]] Fetching stream info

Since there is "Fetching stream info" line after stacktrace, I assume handle_channel task got finished after unhandled exception and was restarted by AutoFC2._main, so whole thing would probably continue working fine after that. But as get_meta makes network request, parses and processes response, it would make sense to handle cases where something of that goes wrong.
The response that caused exception must have been a valid json, since it was successfully decoded, so it might be some kind of 50x error, making it related to #10. I tried to trigger different network-related errors by blocking internet access and pointing api url to different places, and result is generally the same: unhandled exception in get_meta causing restart of handle_channel task.
From my understanding, it would make sense to check for non-200 status, catch KeyError (valid json without expected fields), aiohttp.client_exceptions.ContentTypeError (not a valid json),  asyncio.TimeoutError (in case of network timeout) or any other exception and raise custom exception with appropriate error text, which could then be caught in FC2LiveDL.download .
FC2LiveStream.wait_for_online seems like a good place to handle exponential backoff from #10, but which of possible errors should cause retry and which should be raised immediately? Also, what should be lower and upper limit on delay before retrying? Since there is wait_poll_interval in wait_for_online context, it would make sense to start with it, but should upper bound involve hardcoded value of 5-10 minutes, be multiple of wait_poll_interval, or both?

Hi, thanks for the detailed report. Sorry it took a while for me to respond, somehow this issue got buried.
Yeah I think we can add some retry and exponential backoff at FC2LiveStream.wait_for_online. For simplicity's sake maybe we can just catch all exceptions apart from KeyboardInterrupt. The exact number for the backoff doesn't matter too much but I usually go from 1 second, doubling until some hardcoded limit. I think 5 minutes is a good number.
Let me know if you'd like to make a PR for this.

Ended up doing it in wait_for_online, since it's indeed simpler. Hopefully the way warning about exception is printed will still provide enough information.
Additionally get_meta now calls raise_for_status to get non-200 status code error reported as such.
Perhaps #10 can also be closed now?


CAMEL-20281 - Camel-AWS Components: Make it possible to use AwsSessio…

…nCredentials to support temporary credentials - AWS Secrets Manager
Description

Target

[x] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[x] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[x] I checked that each commit in the pull request has a meaningful subject line and body.

[x] I have run mvn clean install -DskipTests locally and I have committed all auto-generated changes

/component-test aws


how can I get JSON logs to be recognized by the fluentd addon?

Please run microk8s.inspect and attach the generated tarball to this issue.

Will do if necessary, but my system is very vanilla.
I did a microk8s.reset; microk8s.enable dns fluentd, then ran one app that logs lines in JSON to stdout. I can see the lines in Kibana after adding the data source, but the entire json log line is showing up as the value of the log:  property, and I can't access its structured internal info.
I thought this might be a problem with the es or fluentd config for a while, but I now think that some microk8s component responsible for taking container log output and writing it to /var/log is breaking the json by prepending the non-json data, but I can't find the component, or how to configure it so that it doesn't do that.
The fluentd config map has
# CRI Log Example:
# 2016-02-17T00:04:05.931087621Z stdout F [info:2016-02-16T16:04:05.930-08:00] Some log text here

After that is the multi_format config that looks like it should handle that prefixed format or JSON, but I can't see a way to make it understand lines that are "JSON after a prefix", and I'm starting to think that isn't what it is supposed to do, it shouldn't be getting broken JSON in the first place.
What I see in /var/log/{pod,containers}/ matches the "CRI" log format described in the configmap:

2019-11-14T13:54:44.26232227-08:00 stdout F {"level":30,"time":157376848426 ... more JSON

kubectl log MYPOD is outputing valid json lines like:

{"level":30,"time":157376848426 ... more JSON

Whatever is reading from the kube container's log stream and writing to disk needs to be informed that my logs are JSON, or at least some lines are, there is some noise not under my control like npm start blah at the beginning that is not JSON.
I thought it might be "journald" doing this, but I haven't found how/if microk8s is running it, much less how to configure it.
I also found a "cri" plugin in microk8s.ctr plugins ls, but haven't found a way to configure it either.
I'd appreciate any suggestions!
Thanks.

I think you will have to play with the fluentd configuration, by record transformer.  What is happening as i understand it, fluentd is actually converting the raw log to json.
        <pattern>
          format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
          time_format %Y-%m-%dT%H:%M:%S.%N%:z
        </pattern>

Since you are outputting json, it becomes a nested json document.  The field log now contains a raw json, which is your output.
I'm not super familiar with fluentd, but maybe you can try this https://stackoverflow.com/questions/56049210/fluentd-nested-json-parsing

Thanks for the pointers. I can look at that, and will, but I'd still like if someone could explain to me what component is taking my container's log output line, and prepending the unstructured text. The pipeline is  this, AFAICT:
container writes:
   JSON to stdout
kubelet (or containerd?):
   collects and buffers the JSON, `kubectl log ` can display it
SOMETHING?:
  reads log from kube,
  transforms `JSON` to `TIME stdout F JSON`
  writes the transfomed line to /var/log/pods/...
fluentd:
   reads /var/log/pods/...
   pushes to elastic search

I'd like to know the entity named "SOMETHING" above, and where its configuration can be found.
Note that fluentd's current microk8s config would already parse the logs perfectly if they were what the container had written, verbatim (JSON), because it came setup to parse whole JSON lines.

AFAICT its the containerd writing it to /var/log/pod.

https://github.com/containerd/cri/pull/421/files#r152399214 makes me think, that, too, and I haven't found any configuration switches for it. So, it looks like fluentd will have to handle this.

@sam-github when you have figured out how to configure fluentd to handle nested json, please let me know. 😁. It will surely come in handy.
Thanks!

@sam-github do you already have a solution for this problem? Also the Kubernetes logs like pod name, namespace etc seems to be logged as json instead of beeing parsed.

No solution, its still broken, I believe that it is caused by an incomplete implementation of the contained/cri log format, the fluented config lost JSON support at that time. See some of the linked issues above. Not something I work with directly ATM, though, so I won't be looking into  it anymore.

Things getting a bit more complicated as current fluentd plugin is broken and outdated. See https://github.com/ubuntu/microk8s/issues/701

We recently updated fluentd addon.  Perhaps it will be in 1.19 channel or latest/edge.

@lehmamic Any luck on this ?

The version should work. We don’t use the addon anymore, we installed a proper efk stack


Adding photos and captions

Find an easy way to add photos and captions quickly to the webpage

I think the current one is okay


Prefix in Bicep starter module should have a different default value

As the default value of Prefix in Bicep starter module (described in [User Guide] Starter Module Bicep Complete) is problematic, deploying starter modules with Bicep failed.
Expected Behavior

Current Behavior

Possible Solution

Prefix of Bicep starter modules default to something like alz (which I guess aligns with the whole ecosystem around ALZ)

https://github.com/Azure/ALZ-PowerShell-Module/blob/main/docs/wiki/examples/powershell-inputs/inputs-azure-devops-bicep.yaml
https://github.com/Azure/ALZ-PowerShell-Module/blob/main/docs/wiki/examples/powershell-inputs/inputs-github-bicep.yaml
https://github.com/Azure/ALZ-PowerShell-Module/blob/main/docs/wiki/examples/powershell-inputs/inputs-local-bicep.yaml

Reflect that in doc

Steps to Reproduce
Try to deploy a starter module with Bicep just by following the step 1 to 3, using the default setting (i.e. landing-zone for Prefix) showed in the doc and examples.
Context (Environment)

Operating System and version as reported by $PSVersionTable.OS: Microsoft Windows 10.0.22631
PowerShell versions as reported by $PSVersionTable.PSEdition: Core

Detailed Description

This parameter in Bicep starter module will eventually be passed into parTopLevelManagementGroupPrefix in managementGroupsScopeEscape.bicep, which requires its length to be up to 10. However, its current default value is landing-zone and doesn't comply with the limit, resulting in an error when deploying the starter module.
Here's an example in case of using GitHub Action, showing the action stuck in the step What If: Management Groups Deployment keep outputting the following error:
/actions-runner/_work/alz-mgmt/alz-mgmt/infra-as-code/bicep/CRML/customerUsageAttribution/cuaIdManagementGroup.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/actions-runner/_work/alz-mgmt/alz-mgmt/infra-as-code/bicep/bicepconfig.json).

Getting the latest status of all resources...

InvalidTemplate - Long running operation failed with status 'Failed'. Additional Info:'Deployment template validation failed: 'The provided value for the template parameter 'parTopLevelManagementGroupPrefix' is not valid. Length of the value should be less than or equal to '10'. Please see https://aka.ms/arm-syntax-parameters for usage details.'.'
Deployment failed with exception, this is likely an intermittent failure so entering retry loop...
Retrying deployment with attempt number 15 after 160 seconds...

Ran into the same error, will open PR to fix this now. Thanks @juyamagu!


 So this is not related to AllTheTrims, or the logs you provided are not the correct logs.

This crash was caused by flywheel (used by create). Please verify that you are not crashing when AllTheTrims is removed from an identical modlist. I will admit that it is a little odd that both AE2 and flywheel are crashing in these cases.

Wait flywheel isn't even out for 1.20, you're using some 1.19 mods here. I'm gonna lock this, create a new issue when you are certain that the crash is caused exclusively by AllTheTrims


fix: deprecate NPM deployment - fix readme

:tada: This PR is included in version 1.1.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[WIP] Try see if we're running test twice

I think we run each test twice - once to see
if notebook starts, and once to run ./verify. This
seems fair enough, but am trying to get travis to tell
me if this is indeed the case.

The first run tries to build the image and the second tries to build+verify. This makes it easier to debug where the mistake is when things are broken.
The second build should be "free" as we should be reusing the built image.

The first run tries to build the image and the second tries to build+verify. This makes it easier to debug where the mistake is when things are broken.
The second build should be "free" as we should be reusing the built image.

https://travis-ci.org/jupyter/repo2docker/jobs/469294997#L733-L743 confirms that we run things twice and that the verify round is more or less free.


faced "No such file or directory " when query hudi By Kyuubi 1.7

Whey query the hudi tables used to get below error messages even after refresh tables:
No such file or directory: s3a://xxxx/3cb781bd-4efc-4277-9b35-6d436f39ddc1-0_24-373672-1708766_20230727133636.parquet
A clear and concise description of what you expected to happen.
Environment Description

Hudi version :0.13.1

Spark version :3.3.2

Hive version :3.2

Hadoop version :3.2.2

Kyuubi version :1.7.0

Storage (HDFS/S3/GCS..) : s3

Running on Docker? (yes/no) :No

I am using Kyuubi integrated with Hudi 0.13.1  to query hudi tables which is updated by another  spark streaming job (spark3.0 ) with hudi 0.9.0 But used to face below error messages even after refresh tables:
No such file or directory: s3a://xxxx/3cb781bd-4efc-4277-9b35-6d436f39ddc1-0_24-373672-1708766_20230727133636.parquet
Is it because query table with  Hudi 0.13.1  but the table is upserted by hudi 0.9.0?

@njalan I am not sure if this particular error is the cause for this but you should try upgrading your table as hudi 0.9 and Hudi 0.13.1 uses different table_version which can cause lot of issues.


Show "last message read marker" in scrollbar

That line was red and showed in the scrollbar in chaterino1 but in chaterino2 it is grey, doesnt show up in the scrollbar and hard to see if the chat gets scrolled up a lot.
fourtf edit (ty @hemirt):

[x] the line should reset when switching tabs
[ ] the line should be shown in the scrollbar
[x] customizable color

Also, in chatterino1 the line is invisible when it resets and only displayed when the window loses focus, but in chatterino 2 it is visible all the time and changing window doesnt reset it, you have to put chatterino2 in the background to make the line reset.

left is chatterino1 right is chatterino2.
chatterino1:
you change tab and the red line goes to the bottom. (so much to the bottom that it is hidden)

chatterino2:
you change tab and the line doesnt reset, even if new people come and type you still have the grey line where it was before. It only gets back to the bottom if the entire chatterino2, not the tab, loses focus.

I think this sums it up

[ ] the line should reset when switching tabs
[ ] the line should be shown in the scrollbar
[ ] customizable color

This issue is out of scope of the Chatterino project.

This issue is out of scope of the Chatterino project.

It was a thing in Chatterino1, so is it hard to "reimplelent" in ch2? I'd reopen and add backlog instead.


DO NOT MERGE - Feature/heatmap update to echart

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

This requires changes to catch up to 2.1.


fix(floatingMenu): wrap usage in if statement

Similar to #167

👍

Bluemix toolchain: Delivery Pipeline deployed carbon-components-react to Bluemix Design System, including commit 0fa7dff96531de6ad25660e1d165f97d7ff3583c


Ruby on Rails capstone project - Budget app

Ruby on Rails capstone project - Budget app
Description
A mobile web application where you can manage your budget. You have a list of transactions associated with a category, so that you can see how much money you have spent and on what. It allows:

Sign up and Sign for authentication
Introduces new transactions associated with a category.
It shows the money spent on each category.

@DammyShittu Okay thanks. I will try to fix those issues.

@DammyShittu  Thanks for your professional code review. Thanks for the tips. I have done what you requested of me. I would appreciate a positive feedback because I have already repeated a block. Thanks in anticipation

@CollinsTatang Thanks bro. I appreciate you guys. Thanks for the good work you guys are doing. I am proud of you guys. Many thanks


Fix activity indicator with delayed animations on some shape

Trying to add a new activity indicator named rupee

and comparing the animation gif in doc and the current one in demo app, I see an issue :
some animation which use beginTime doesn't working
I see for working animation that the animation group is created only one time, not for each additional layer. So I edit the bugged ones.

          1 Error

:no_entry_sign:
Any changes to library code need a summary in the Changelog.

          1 Warning

:warning:
Consider adding / updating the demo app.

SwiftLint found issues
Warnings

File
Line
Reason

ActivityIndicatorAnimationRupee.swift
52
Lines should not have trailing whitespace.

ActivityIndicatorAnimationRupee.swift
54
Lines should not have trailing whitespace.

ActivityIndicatorAnimationRupee.swift
33
Function parameters should be aligned vertically if they're in multiple lines in a method call.

ActivityIndicatorAnimationRupee.swift
34
Function parameters should be aligned vertically if they're in multiple lines in a method call.

ActivityIndicatorAnimationRupee.swift
35
Function parameters should be aligned vertically if they're in multiple lines in a method call.

ActivityIndicatorAnimationRupee.swift
36
Function parameters should be aligned vertically if they're in multiple lines in a method call.

ActivityIndicatorAnimationRupee.swift
31
where clauses are preferred over a single if inside a for.

  Generated by :no_entry_sign: Danger

Demo app is updated automatically using enum ;)

@phimage But I can't find the animation rupee in the demo app playground 🤔

@JakeLin did you build framework?
Then build and launch app?
I do not know why and since when but I see no target or compile dependencies between projects

@phimage thanks, you are absolutely right, I can see it now. I removed the dependence by accident when I upgraded the project to support Xcode 9.3. Can you add it back to this PR? Or I can do a comment in master later on?

@JakeLin  I merge so you can edit main branch (or make new PR)


formatting fixes in gh-pages

again, please leave open for a bit; i'll poke around today and send a ping later on.

Ooops, did merging automatically close this? If so... apologies!

No worries, I'll open another PR later.
On Jun 2, 2016 11:49 AM, "Gregory M. Kurtzer"<EMAIL_ADDRESS>wrote:

Ooops, did merging automatically close this? If so... apologies!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/gmkurtzer/singularity/pull/44#issuecomment-223386196,
or mute the thread
https://github.com/notifications/unsubscribe/AFMEPqDj2vSwpMUsgysMYKyWqOQcO9cCks5qHyXHgaJpZM4Isy7i
.

Thanks!


refactor: Master => develop

Merge latest release back into develop

Builds ready [175f6c1]builds: chrome, firefox, opera
bundle viz: background, ui, portal-inpage, portal-contentscript, ui-libs, bg-libs, phishing-detect
dep viz: background
all artifacts


add ExAC non TCGA  to annotations

ExAC now provides a site VCF that excludes TCGA samples. There is also a non-psych sites VCF.
We could probably merge all 3 in the provenance and add, e.g. AC_ntcga=...;AC_npsyc=...;
so that we don't add 2 extra huge files to the downloads.

Yes please! I've been using this in my VEP annotations. I was thinking I could just use Gemini's ExAC, then I wondered "wait, what version are they using?" I suggest the nonTCGA ExAC numbers become the default for Gemini.


fix: user theme should replace default theme

Yes. It's should. But it's not necessary. If you don't use it, it won't cause any problem. Let me test this.

No description provided.

Actually many tests failed. And I don't want to fix this actually. However if you can fix it, I will accept this pr happily.

Codecov Report

Merging #81 (96bfa40) into main (8576871) will decrease coverage by 1.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #81      +/-   ##
==========================================
- Coverage   91.49%   90.44%   -1.05%
==========================================
  Files          44       44
  Lines        1986     1999      +13
  Branches      624      631       +7
==========================================
- Hits         1817     1808       -9
- Misses         86      100      +14
- Partials       83       91       +8

Impacted Files
Coverage Δ

src/lib/index.ts
87.58% <100.00%> (-0.10%)
:arrow_down:

src/utils/algorithm/combineConfig.ts
84.21% <0.00%> (-15.79%)
:arrow_down:

src/plugin/index.ts
83.33% <0.00%> (-8.34%)
:arrow_down:

src/lib/utilities/dynamic.ts
90.51% <0.00%> (-3.64%)
:arrow_down:

src/utils/algorithm/compileStyleSheet.ts
80.26% <0.00%> (-1.56%)
:arrow_down:

src/lib/utilities/handler.ts
76.66% <0.00%> (-1.12%)
:arrow_down:

src/utils/tools.ts
96.70% <0.00%> (-0.04%)
:arrow_down:

src/lib/utilities/preflight.ts
100.00% <0.00%> (ø)

src/plugin/typography/index.ts
88.88% <0.00%> (ø)

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8576871...96bfa40. Read the comment docs.

Thank you very much for fixing this problem and please keep going.


Update links.py

Here is Little contribution for fixing SSL errors

[x] My submission is formatted according to the guidelines in the contributing guide
[x] My addition is ordered alphabetically
[x] My submission has a useful description
[x] The description does not have more than 100 characters
[x] The description does not end with punctuation
[x] Each table column is padded with one space on either side
[x] I have searched the repository for any relevant issues or pull requests
[x] Any category I am creating has the minimum requirement of 3 items
[x] All changes have been squashed into a single commit

It will also remove 404 errors for urllib .


Merge 'manager' into 'main'

This PR proposes the following features for merge into main:

API: /m endpoints for managing media
API: /v endpoints for managing virtual machines
Removal of the old vrfb code to make space for the new RFB code
Virtual machine creation and management

Additionally, I would suggest the following branches for deletion:

api: Its code is contained within this PR
cleanup: Its code is contained within this PR
fixes: The import System fix has been cherry-picked into this PR

I agree to both of them, I will perform those changes and merge the pull request

I will replace the copyright headers in a separate branch, this should be a separate action IMO


Error when I change my index

I did several rules with a particular index using Elastalert jertel docker and it works well.
I have created a new rule with another index of Elasticsearch and encounterd an issue never seen before where I'm stucked
File "/usr/local/lib/python3.11/site-packages/elastalert/elastalert.py", line 1235, in handle_rule_execution     num_matches = self.run_rule(rule, endtime, rule.get('initial_starttime'))                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/lib/python3.11/site-packages/elastalert/elastalert.py", line 889, in run_rule     if self.is_silenced(rule['name'] + "._silence") or self.is_silenced(silence_cache_key):                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/lib/python3.11/site-packages/elastalert/elastalert.py", line 1728, in is_silenced     until_ts = res['hits']['hits'][0]['_source']['until']                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ KeyError: 'until'
I don't understand at all what it means and how to solve it, any clue ?

ElastAlert is no longer maintained. Please use ElastAlert2

Well I know, if not I wouldn't been able to make it work for my other rules lol.
There are lots of problem with the index Elastalert use : elastalert_status, elastaler_error, ect..
They make lots of errors and this errors has zero info on why and where they come from.
Took me days to know that these indexes where the problems


Set Dev Version Fix

I broke the SetTestPipeline where we target a specific package with my merge of pyproject.toml.

Thank you for fixing this, Scott.

Thanks for the report!


Release a new version

I tried to write:
g = lettermod(arrowsfromg, g, 100, 200)

but I get an error:
juil. 05 20:38:13 bestos systemd[1]: Started Keyd remapping daemon.
juil. 05 20:38:13 bestos keyd[1371937]: WARNING: failed to set effective group to "keyd" (make sure the group exists)
juil. 05 20:38:14 bestos keyd[1371937]: CONFIG: parsing /etc/keyd/test.conf
juil. 05 20:38:14 bestos keyd[1371937]: keyd: src/config.c:450: parse_fn: Assertion `*nargs < MAX_DESCRIPTOR_ARGS' failed.
juil. 05 20:38:14 bestos systemd[1]: keyd.service: Main process exited, code=dumped, status=6/ABRT
juil. 05 20:38:14 bestos systemd[1]: keyd.service: Failed with result 'core-dump'.
juil. 05 20:38:14 bestos systemd[1]: keyd.service: Scheduled restart job, restart counter is at 5.
juil. 05 20:38:14 bestos systemd[1]: keyd.service: Start request repeated too quickly.
juil. 05 20:38:14 bestos systemd[1]: keyd.service: Failed with result 'core-dump'.
juil. 05 20:38:14 bestos systemd[1]: Failed to start Keyd remapping daemon.

using v2.4.3. What am I missing?

Ah, seems like it is fixed in master, but it would be cool to have a release so that my package manager can update it.

Sorry, a release is long overdue. I have just published one.

Thanks!


No dropout?

I don't see the dropout implemented in your script.
If it's there, where is it? Especially during training.

No dropout is widely used to train modern LMs (such as T5, GPT series, and Llama) and we adopt this setting.