Deadlock by doing create/start/getContainer via varlink connection

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
I'm working on #3387 and discovered a deadlock problem when doing a series of CreateContainer/StartContainer/GetContainer multiple times via systemd varlink connection.
GetContainer will then block until i restart  the io.podman systemd service. Also podman on the cli is blocked until i restart the service.
Steps to reproduce the issue:

Compile this test program

package main

import (
	"fmt"

	"github.com/pascomnet/podman-varlink-test/iopodman"
	"github.com/varlink/go/varlink"
)

func main() {

	varlinkConnection, err := varlink.NewConnection("unix://run/podman/io.podman")
	if err != nil {
		fmt.Printf("failt to start task, could not connect to podman: %v\n", err)
		return
	}
	defer varlinkConnection.Close()

	args := []string{"redis"}

	createOpts := iopodman.Create{
		Args: args,
	}

	containerID, err := iopodman.CreateContainer().Call(varlinkConnection, createOpts)
	if err != nil {
		fmt.Printf("failt to start task, could not create container: %v\n", err)
		return
	}
	fmt.Printf("Created %s\n", containerID)

	_, err = iopodman.StartContainer().Call(varlinkConnection, containerID)
	if err != nil {
		fmt.Printf("failt to start task, could not start container: %v\n", err)
		return
	}
	fmt.Printf("Started %s\n", containerID)

	info, err := iopodman.GetContainer().Call(varlinkConnection, containerID)
	if err != nil {
		fmt.Printf("failt to get container: %v\n", err)
		return
	}
	fmt.Printf("ok. got info %v\n", info)
}

Run it:

Created<PHONE_NUMBER>be04564db2c6c53e5ba1092b5d3bab073dcf78de7927490c4fc813
Started<PHONE_NUMBER>be04564db2c6c53e5ba1092b5d3bab073dcf78de7927490c4fc813
ok. got info {9662752041be04564db2c6c53e5ba1092b5d3bab073dcf78de7927490c4fc813 docker.io/library/redis:latest .......

Run it repeatedly  until it locks up

Created b11cf12254306e7463b534c81b0ba4144d3e1a15b761353670f95446c3e19c8d
Started b11cf12254306e7463b534c81b0ba4144d3e1a15b761353670f95446c3e19c8d

Additional information you deem important (e.g. issue happens only occasionally):
The time of repeations varies, sometimes it locks up on 2nd try, sometimes it takes a while.
Output of podman version:
Version:            1.4.2
RemoteAPI Version:  1
Go Version:         go1.12.5
OS/Arch:            linux/amd64

Output of podman info --debug:
debug:
  compiler: gc
  git commit: ""
  go version: go1.12.5
  podman version: 1.4.2
host:
  BuildahVersion: 1.9.0
  Conmon:
    package: podman-1.4.2-1.fc30.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 0.2.0, commit: d7234dc01ae2ef08c42e3591e876723ad1c914c9'
  Distribution:
    distribution: fedora
    version: "30"
  MemFree:<PHONE_NUMBER>
  MemTotal:<PHONE_NUMBER>
  OCIRuntime:
    package: runc-1.0.0-93.dev.gitb9b6cc6.fc30.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8+dev
      commit: e3b4c1108f7d1bf0d09ab612ea09927d9b59b4e3
      spec: 1.0.1-dev
  SwapFree:<PHONE_NUMBER>
  SwapTotal:<PHONE_NUMBER>
  arch: amd64
  cpus: 2
  hostname: run-fedora
  kernel: 4.4.0-116-generic
  os: linux
  rootless: false
  uptime: 109h 43m 20s (Approximately 4.54 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 5
  GraphDriverName: overlay
  GraphOptions:
  - overlay.mountopt=nodev
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 2
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

Additional environment details (AWS, VirtualBox, physical, etc.):
I am running this stuff in a nested Fedora 30 LXC container on a ubuntu 18.04 host (Linux 4.4.0-116-generic)
Also i tried other varlink calls instead of GetContainer():

InspectContainer()  : it locks up very rarly
Ps() : it never locks up

Also i tried sleeping before GetContainer() but it did not change the behavior.

Some more info:
I did a fresh fedora 30 setup, installed latest podman, enabled varlink systemd socket and ran the test binary.
It is reproducible there. (podman 1.4.4, linux  5.0.9-301.fc30.x86_64 )

So this is somewhere between StartContainer and GetContainer. Both of those do need to grab the container lock, but both of them give it up quickly once their work is done.
I wonder if we're not seeing some sort of issue with Varlink trying to order requests here, leading to a sort of priority inversion?

@towe75 i am back from PTO and ready to look at this .. this is very similar to another reported issue I was debugging today but you are doing it in the context of a sample program ... i'd like to use it.  it looks like when you did this, you had a special environment (imports).  any chance you peek at this quickly and modify your example to be built from from github.com/containers/libpod/towe75 or some such?

@baude thank you for checking. Find my example attached to this post: podman-issue3572.zip
It contains:
├── go.mod
├── go.sum
├── iopodman
│   └── iopodman.go
└── main.go
Just unpack it anywhere an run go build. It's module enable, so no need to place it into GOPATH.
I used go1.12.7 on fedora 30 to test it.

@towe75 do you hang out on #podman on freenode?

@towe75 can you confirm that if you run podman varlink outside of systemd, you see no deadlock?

@baude  sorry could not catch you in freenode, was quite busy.
Anyhow, i tried this:
systemctl disable io.podman.socket
/usr/bin/podman varlink --timeout 50000 unix:/run/podman/io.podman

in another shell i ran my test binary in a loop. I was able to produce 30 containers without a deadlock whereas with systemd it usually stops working after 2-3 iterations. I'll do a longer test later.
So looks like systemd socket has some effect.

@baude is working on this one still, I believe. We've tracked the deadlock into c/storage.

So, I finally managed to debug this and find the culprit.

Now in the podman codebase there are several places also handling LISTEN_FDS:
pkg/rootless/rootless_linux.c:
https://github.com/containers/libpod/blob/031437bf5b19b5121224cdec301dd9945f38e610/pkg/rootless/rootless_linux.c#L529-L536
https://github.com/containers/libpod/blob/031437bf5b19b5121224cdec301dd9945f38e610/pkg/rootless/rootless_linux.c#L721-L735
libpod/oci_internal_linux.c:
https://github.com/containers/libpod/blob/031437bf5b19b5121224cdec301dd9945f38e610/libpod/oci_internal_linux.go#L250-L254
In our case the last one causes a close(3)
6386  dup2(3, 19)                       = 19
6386  fcntl(19, F_SETFD, FD_CLOEXEC)    = 0
6386  fcntl(0, F_SETFD, 0)              = 0
6386  fcntl(1, F_SETFD, 0)              = 0
6386  fcntl(2, F_SETFD, 0)              = 0
6386  fcntl(3, F_SETFD, 0)              = 0
6386  dup2(18, 4)                       = 4
6386  dup2(19, 5)                       = 5
6386  execve("/usr/libexec/podman/conmon",  ...

3 dupped to 19
CLOEXEC on 19
3 is set to 0
19 dupped to 5
execve closes 19 because CLOEXEC

Which makes no sense anyway, because LISTEN_FDS begins at 3 and 3 now is 5 and nothing adjusted LISTEN_FDNAMES ???
Anyway commenting out this code block and this issue is gone:
https://github.com/containers/libpod/blob/031437bf5b19b5121224cdec301dd9945f38e610/libpod/oci_internal_linux.go#L250-L254
I don't know what the use case of this code block is and how it would work without adjusting LISTEN_FDNAMES.
Also one has to review the correct handling of the code in pkg/rootless/rootless_linux.c.
You really have to take care of the fact, that some code used one of the LISTEN_FDS and the fd has been closed and the number is reused. One marker could be:
if ((fcntl(f, F_GETFD) & FD_CLOEXEC) == 0)
So, if the CLOEXEC flag is set, don't close the fd on execve. But that is only guesswork also.
@baude @mheon @rhatdan

I'll tag in @giuseppe here - I'm pretty sure he wrote this for SD_NOTIFY

I am not really familiar with this part, but some things I've observed:

if I do rm -fa from the CLI, the test program unblocks
if I open a new varlink connection before each command (i.e. reopen the connection before Start and before GetContainer), then the hang doesn't happen

I don't know what the use case of this code block is and how it would work without adjusting LISTEN_FDNAMES.

I also don't understand that code block.  Why do we need to propagate the LISTEN_FDs down to conmon?  I think it should be safe to remove it.  @mheon @baude do you know why it was added?

Hmmm.
My original assumption was that was your code - it's definitely not mine... I think it might well end up being @rhatdan

commit a1a79c08b72793cf2f75490d8ffc844c3d16bd4a
Author: Peter Hunt<EMAIL_ADDRESS>Date:   Mon Jul 1 13:55:03 2019 -0400

commit 989f5e30627eb90a66fa06f87c40d161476395ed
Author: Daniel J Walsh<EMAIL_ADDRESS>Date:   Sun Feb 4 07:29:47 2018 -0500

I don't think it makes sense to use LISTEN_FDS and NOTIFY_SOCKET, if podman is in varlink mode. This should really be skipped here.

I agree these should only be handled for not 'remote' podman.


Carry #45: Add etcd TLS client code and update README.md.

Carry #45
Adds etcd tls client auth

/cc @mavenugo @vieux

LGTM

Was reading the diff again and missed that the check was outside of the Set Options section. So I will make the change to refactor this part in another commit.

LGTM

Thanks for carrying this one.  This week's been crazy.  I appreciate it.

@vieux PTAL

@abronan will review before EOD.

@mavenugo :+1:

@abronan though the changes are small, I have to spend some time to understand the etcd TLS setup. Will do some investigation and try it out. pls dont hold up for me though :)
this is an important PR and we have to get this done for consul and zk as well.

@mavenugo Ok let's merge it and do the change incrementally for the other backends, this PR offers the way to setup TLS with certs filepath so this is helpful in the case of libnetwork to include this in your config ;)

LGTM


⚠ Turnos 24x7 has degraded performance

In 01e6eb1, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 1006 ms

Resolved: Turnos 24x7 performance has improved in 551c743.


Version 0.7 and 0.8 of solidity

For mocking the link token, having version 0.7 and 0.8 would be wonderful

@PatrickAlphaC do you have a recommended workaround of doing local tests on 0.8.x with contracts that use ChainLink without these for working mocks ?

Bump


Curious about syntax

Just wondering.
counterDecorator() vs @counter.

Of course there is also the ES2015 tagged template syntax.

I was hoping it would be compatible with the @decorator syntax, but I haven't actually tried it!
I use the plain old functions.


Ability to load custom fixers

Hi, currently I have many custom fixers, would be nice if there was a way to load them via the config file

@lukeraymonddowning is working on it.

Hi,
I'm curious if this was sorted, and perhaps a documented way to apply our own custom fixers?

There was a PR for it.. https://github.com/laravel/pint/pull/26 but it got dropped due to some complications... I have switched to ECS (https://github.com/symplify/easy-coding-standard) until pint supports custom rules...

Hi! Is there any update on this? Thank in advance.


Publish Latest Version to Packagist

Could the latest version (3.5.0 at time of writing) please be published to Packagist so it can be installed through composer? At time of writing the latest version on Packagist is 3.4.3.

Apologies, it's done


Handle JournalEntryExerciseMarker

Can handle exercise markers (JournalEntryExerciseMarker) in x23 models. For now, does not add them to any BG effect.
Also added tests for false unit types.

I believe it is failing because it is installing an older version of mmhistorytools which does not have the Unit object updated ... could be wrong, as I'm not 100% certain how Travis CI works.

For now, I'm going to submit a PR into your branch that removes the unit dependency by just copying the enumerations.


Do we need a mysql data base because it keeps in a loading loop

Yes

Install MySQL database and when prompted, select developer mode, then perform a hard reinstall of Helix and ixhl2rp


api/tes: ListTask tag filters

Based on #453
https://github.com/buchanae/funnel/compare/list-filtering...adamstruck:tag-filter?expand=1

so apparently dynamo doesn't allow the following:
tes.Task{
    Tags: map[string]string{"foo": ""},
}

In the above case the Tags field is dropped during serialization.

Ok this PR should be ready to go.

How many of the databases will work with tasks created before this PR?
Also, does Dynamo drop tags with an empty value still?

I was wrong about Dynamo which is why I deleted the comment. All of the databases except datastore should work with tasks created before this PR.


Crash with Peculiars.

When the Peculiars mod is present, going into creative inventory and mousing over a salt item crashes the game.

Please provide a crash log.

crash-2023-05-29_18.21.06-client.txt

I cannot reproduce it.
From the log, error does not seem to be related to Salt at all. Peculiars mixin cannot find the Atmospheric class for some reason. It is especially strange because you don't even have atmospheric installed.


 Marking the build as failed.

@tingyuz @bnevis-i The security test on master are failed to run.  Do you have any idea?

Initial triage suggests that kong-migrations is failing to start.

Apparently the upstream postgres release was changed 4 days ago from a WARNING that POSTGRES_PASSWORD is unspecified to an ERROR.
https://hub.docker.com/layers/postgres/library/postgres/9.6/images/sha256-92cba31ae0ab4480e227d03b68cfc9406b6c69d0b731f0b20e12de00c3d4c311?context=explore
(Old prior to 9.6.17)
****************************************************
WARNING: No password has been set for the database.
         This will allow anyone with access to the
         Postgres port to access your database. In
         Docker's default configuration, this is
         effectively any other container on the same
         system.

         Use "-e POSTGRES_PASSWORD=password" to set
         it in "docker run".
****************************************************

(New: 9.6.17 or later)
Error: Database is uninitialized and superuser password is not specified.
       You must specify POSTGRES_PASSWORD for the superuser. Use
       "-e POSTGRES_PASSWORD=password" to set it in "docker run".

       You may also use POSTGRES_HOST_AUTH_METHOD=trust to allow all connections
       without a password. This is *not* recommended. See PostgreSQL
       documentation about "trust":
       https://www.postgresql.org/docs/current/auth-trust.html

Fix submitted.
cc: @mkbhanda @tingyuz

Colin submitted an unmerged fix for this days ago
https://github.com/edgexfoundry/developer-scripts/pull/219
Pulling my PR and referring to Colin's instead.

is this related to https://github.com/edgexfoundry/go-mod-secrets/issues/49 ?

@cloudxxx8  No.  go-mod-secrets 49 causes services to fail to get the mongodb password.  This one just causes kong to fail to start because the Postgres database doesn't come up.

I believe this issue can be closed now.

@bnevis-i since the blackbox test is still blocked now, I will close this issue once we can verify it on Jenkins

@cloudxxx8 Is this still an issue?

yes, let's close this issue.
thanks for the reminder.


Allow width/height overwrite and option to add separator

Allow to overwrite height/width of the digits.
Option to add time separator text

@azhararmar added in 6f1089090d7f4cac8beb7b2399988183f8f398b8
please update to 2.1.0


Iterate over transposed orbs in iterchunks as well

@rbcasperson I reworked this function to be more focused...also, notice that I created a dedicated test file for just this one function. You should start doing that too!
Please review this.

I see that you took the running of iterchunks on the transposed orbs from needsShuffle and just put it into iterchunks. Because when would you only iterate over the horizontal chunks?
I see the separate test file too. evaluate() could certainly use one too.


Upgrade k8s.io/* to v0.27.3, sigs.k8s.io/controller-runtime to v0.15.0

How to categorize this PR?

/area open-source dev-productivity
/kind enhancement
What this PR does / why we need it:
Upgrade dependencies

k8s.io/* to v0.27.3
sigs.k8s.io/controller-runtime to v0.15.0

Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
In draft until integration test is fixed.
Release note:

The following dependencies are updated:
- `k8s.io/*` : `v0.26.3` -> `v0.27.3`
- `sigs.k8s.io/controller-runtime`: `v0.14.5` -> `v0.15.0`

/test all

/assign

/test pull-gardener-e2e-kind

/cc @shafeeqes @acumino

/assign

/assign

You can also vendor v0.27.4 directly and update envtest version here, https://github.com/gardener/gardener/blob/74020278eabec42dd107577462844b43b1b857ff/hack/prepare-envtest.sh#L21

/test all

/unassign
due to vacation

/test all

/test pull-gardener-e2e-kind-migration-ha-single-zone
/test pull-gardener-e2e-kind-ha-single-zone

/retest

/retest

/test pull-gardener-e2e-kind-migration-ha-single-zone

/retest

/retest

Since<EMAIL_ADDRESS>is released, should we directly update to that and ctrl-runtime v0.16?

Since<EMAIL_ADDRESS>is released, should we directly update to that and ctrl-runtime v0.16?

Sure

Since<EMAIL_ADDRESS>is released, should we directly update to that and ctrl-runtime v0.16?

Isn't this PR too big already, there might be more breaking changes with the new dependencies. I would vote for a new PR as it would also help during revendoring in extensions.

/retest

As discussed in https://github.com/gardener/gardener/pull/8217#discussion_r1257819054 for worker controller,
Can you also do the same for https://github.com/gardener/gardener/blob/bc94ce9bd342f3c18f206018edeca7278fcb4cfc/pkg/provider-local/controller/controlplane/add.go#L51-L54, Let's return an error for NewActuator. This was missed in #8217.

Did you check whether we can keep calling the generate-{internal-}groups.sh scripts from the k8s.io/code-generator repository and don't need to copy them?

We cannot simply pass the internalversion to CLIENTSET_NAME_VERSIONED variable for the generate-{internal-}groups.sh. There were more changes done to the script which needs more insight. I prefer to do it as a part of separate PR.


Dropped item stays in drop position

I have a board with panels that have list-group-items generated with ng-repeat for each panel. Dragging and dropping from one panel to the other works fine but if I drag item from panel and drop it in the same panel the item drops and stays in the position it is dropped(not align with other items in panel) and starts the onDrop event.
Would you happen to know what causes this and how to fix it?

I am not sure, if I understand correctly your issue, but you can try my patch "revert item to the origin place if it is outside of dropped objects". You can find it in pull requests.

Thanks for the help but this doesn't help my situation. My project is like a kanban board and if I drop an item from one panel to another it aligns with other items in that panel but if I drag item from a panel and drop it in the same panel it have been dragged the item stays where it is dropped and doesn't align with other items in that panel.


python311Packages.garth: 0.4.39 -> 0.4.41

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

The diff is here https://github.com/matin/garth/compare/0.4.39...0.4.41

Result of nixpkgs-review pr 265324 at de43277f run on x86_64-linux 1

6 packages built successfully:

python310Packages.garminconnect
python310Packages.garth
python310Packages.withings-sync
python311Packages.garminconnect
python311Packages.garth
python311Packages.withings-sync


Upgrade walinux agent to 2.2.36

Microsoft and Pivotal field folks have discovered that some of the early
2.2.3x builds are imcompatible with azure stack, and requested we
upgrade to 2.2.35 or higher. Stevenson has validated
that the agent fails running on stack only on Xenial 170+, when we
upgraded the version of walinux agent.
Signed-off-by: David Stevenson<EMAIL_ADDRESS>

We'd also love this in Xenial 250, and future releases


Scala REPL fails with IncompatibleClassChangeError when sbt-launch.jar is in %SCALA_HOME%/lib

I installed the latest typesafe stack and then downloaded the sbt-launch.jar into the %SCALA_HOME%/lib.  REPL fails to run
Exception in thread "main" java.lang.IncompatibleClassChangeError: Expecting non-static method scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/m
utable/ArrayOps;
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:33)
    at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:89)
    at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

When that jar is deleted REPL runs fine.
How do I fix this?

"Note: do not put sbt-launch.jar in your $SCALA_HOME/lib directory, your project's lib directory, or anywhere it will be put on a classpath."
https://github.com/harrah/xsbt/wiki/Setup

Thanks a lot Jason.
On Sun, Oct 9, 2011 at 3:00 PM, Jason Zaugg <
<EMAIL_ADDRESS>
"Note: do not put sbt-launch.jar in your $SCALA_HOME/lib directory, your
project's lib directory, or anywhere it will be put on a classpath."
https://github.com/harrah/xsbt/wiki/Setup
--
Reply to this email directly or view it on GitHub:
https://github.com/harrah/xsbt/issues/218#issuecomment-2338080

--
Regards,
Dinesh V


failed to create artifacts directory

Bug Description
Preflight fails to run because it's looking for artifacts dir to be present.
Version and Command Invocation
preflight version 0.0.0 <commit: d6389c597717aabbce381ab6a9c2047717d781eb>
Steps to Reproduce:
(How can we reproduce this?)

./preflight check operator

Expected Result
Able to execute and display output
Actual Result
Error: open artifacts/results.json: no such file or directory
The checks that will be executed are the following:

ScorecardBasicSpecCheck
ScorecardOlmSuiteCheck
OperatorPackageNameIsUnique
DeployableByOLM
ValidateOperatorBundle

Usage:
preflight check operator  [flags]
Flags:
-h, --help   help for operator
time="2021-09-07T15:37:07Z" level=fatal msg="open artifacts/results.json: no such file or directory"

/assign rocrisp


Move check cluster configuration to the begining

Description
Changes proposed in this pull request:

because we do not have srtages in the upgrade kyma process, the check step must be the first one - to prevend unecessary running other steps after the reconciler is called

Related issue(s)

/unhold


Color edit without tooltip fix

Description

The QColorSwatch class had a typo in the assignment of the tooltip if no explicit tooltip was provided.

I fixed the typo by removing the second underscore
self.setToolTip(tooltip or trans.__('click to set color'))
to
self.setToolTip(tooltip or trans._('click to set color'))

Type of change

[X] Bug-fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

References
None

How has this been tested?

[X] I wrote two tests creating the respective classes with default values and explicitly provided values
[X] all tests pass with my change

Final checklist:

[X] My PR is the minimum possible work for the desired functionality
[X] I have added tests that prove my fix is effective or that my feature works

@Czaki I actually really did not like the inner for loop. I changed it to use pytest.mark.parametrize and added all created widgets to the qtbot.

@Czaki I think that makes a lot of sense.
I modified your suggestion to check explicitly for the expected value depending on the input for tooltip and color.

@Czaki Thanks for the hint, I did not use the or statement this way, yet. I will go with your suggestion :)

@Czaki I am still not sure how merging works :) do I need to write someone and ask for the merge and that you reviewed it or do I just need to wait?


organisational issues

With #2, the internal documentation works and it is time to check if that's also the case for pandas and xarray. There are also a few open issues:

I'd like to move the main repository out of my own personal "organization", but where should this live?
~naming: accessors seems to be available in PyPI so we could go with that, but sphinx-ext-accessors or something similar would also be possible.~ it is now named sphinx-autosummary-accessors
~location of the templates: right now, they are in the main templates directory, so they are used with :template: accessor_method.rst, but pandas and xarray seem to use autosummary/accessor_method.rst instead. We probably should copy that~ done
license: I chose MIT, but pandas is under the BSD 3-clause. I don't really care about the license, but I think there has to have been at least some discussion about that. Also, since this is mostly code from pandas it would probably be good to ask them.

cc @benbovy, @shoyer, @TomAugspurger

I think it could fit in xarray-contrib.

I like the sphinx- prefix in the name, maybe sphinx-autosummary-accessors or sphinx-autodoc-accessors would be even more explicit?

I think it could fit in xarray-contrib.

I've been thinking about that, but if we want to have pandas also use it this might be a bit strange (not sure if that's really an issue, though)

maybe sphinx-autosummary-accessors or sphinx-autodoc-accessors would be even more explicit

I like those, although I'd rule out the autodoc name since we don't use it directly

okay, with this we resolved all of my initial questions except the licensing. And since we were able to close / resolve #7, this means that we can a release this.
I'll raise this on the pandas issue tracker (and also invite them to use this :grin:).

closed by #13, I guess?


0.20 Not in Maven

The changelog shows 0.20 as the current version with a release date, but 0.20 doesn't seem to be available via maven.

+1

+1

So sorry! I just published 0.20.


Allow partial pushdown of filters beneath multiple joins

Summary of the changes / Why this improves CrateDB
Resolves https://github.com/crate/crate/issues/14899
The following stmt can now have the filter pushed down:
select * from (
    select t2.x as t2x
    from t1
    left join t2 on t1.x = t2.x
    left join t3 on t2.x = t3.x
) as v
where t2x > 5;

Rename[t2x] AS v
  └ Eval[x AS t2x]
    └ NestedLoopJoin[LEFT | (x = x)]
      ├ HashJoin[(x = x)]
      │  ├ Collect[doc.t1 | [x] | true]
      │  └ Collect[doc.t2 | [x] | (x AS t2x > 5)] <-- filter pushed down
      └ Collect[doc.t3 | [x] | true]

Note that if 1) the joins were not nested, or 2) the joins were not aliased,
or 3) the filter had queries for all relations of the joins (i.e. where t1x=1 and t2x=1 and t3x=1) the pushdown would have worked without this PR.
Checklist

[x] Added an entry in the latest docs/appendices/release-notes/<x.y.0>.rst for user facing changes
[x] Updated documentation & sql_features table for user facing changes
[x] Touched code is covered by tests
[x] CLA is signed
[x] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in the latest docs/appendices/release-notes/<x.y.0>.rst
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

if (matchesRhs.isEmpty() == false && matchesLhs.isEmpty()) {
    rightQuery = query;
    it.remove();
} else if (matchesRhs.isEmpty() && matchesLhs.isEmpty() == false) {
    leftQuery = query;
    it.remove();
}

the original filters could result in more than 1 left and right queries, I will double check this.

Unrelated test failed:
io.crate.types.StringTypeTest.test_doc_values_write_and_read_roundtrip_inclusive_doc_mapper_parse

retest this please

The current approach seems to be tailored to the very specific nested-join use-case. There are more cases which need to be considered:

Push filter to the left-hand-side of a left-outer-join:

Filter[(t1.a > 1)]
  └ Join[LEFT | (a = b)]
      ├ Collect[doc.t1 | [a] | true]
      └ Collect[doc.t2 | [b] | true]

becomes:
Filter[(t1.a > 1)]
  └ Join[LEFT | (a = b)]
      └ Filter[(a > 1)]
	    └ Collect[doc.t1 | [a] | true]
      └ Collect[doc.t2 | [b] | true]

Push filter to the left-hand-side of a left-outer-join nested:

Filter[(t1.a > 1)]
  └ Join[LEFT | (t2.b = t3.c)]
     ├ Join[INNER | (t1.a = t2.b)]
     │  ├ Collect[doc.t1 | [a] | true]
     │  └ Collect[doc.t2 | [b] | true]
     └ Collect[doc.t3 | [c] | true]

becomes:
Join[LEFT | (t2.b = t3.c)]
  ├ Filter[(a > 1)]
  │  └ Join[INNER | (t1.a = t2.b)]
  │    ├ Collect[doc.t1 | [a] | true]
  │    └ Collect[doc.t2 | [b] | true]
  └ Collect[doc.t3 | [c] | true]

Push filter to the right-hand-side of a right-outer-join:

Filter[(t2.b > 1)]
  └ Join[RIGHT | (a = b)]
      ├ Collect[doc.t1 | [a] | true]
      └ Collect[doc.t2 | [b] | true]

becomes:
  Join[RIGHT | (a = b)]
      ├ Collect[doc.t1 | [a] | true]
      └ Filter[(b > 1)]
	     └ Collect[doc.t2 | [b] | true])]

Push filter to the right-hand-side of a right-outer-join nested:

Filter[(t2.b > 1)]
  └ Join[RIGHT | (t2.b = t3.c)]
     ├ Collect[doc.t3 | [c] | true]
     └ Join[INNER | (t1.a = t2.b)]
        ├ Collect[doc.t1 | [a] | true]
        └ Collect[doc.t2 | [b] | true]

becomes:
 Join[RIGHT | (t2.b = t3.c)]
   ├ Collect[doc.t3 | [c] | true]
   └ Filter[(a > 1)]
	  └ Join[INNER | (t1.a = t2.b)]
         ├ Collect[doc.t1 | [a] | true]
         └ Collect[doc.t2 | [b] | true]

The use-case you described is a special case where an outer-join can be converted to an inner-join when the filter can eliminate the null values from the outer-side:
Filter[(t2.b = 1)]
 └Join[LEFT | (t1.a = t2.b)]
    ├ Collect[doc.t1 | [a] | true]
    └ Collect[doc.t2 | [b] | true]

becomes:
Join[INNER | (t1.a = t2.b)]
  ├ Collect[doc.t1 | [a] | true]
  └ Filter[(t2.b = 1)]
     └ Collect[doc.t2 | [b] | true]

and in the nested case:
Filter[(t2.b = 1)]
  └ Join[LEFT | (t2.b = t3.c)]
     ├ Join[LEFT | (t1.a = t2.b)]
     │  ├ Collect[doc.t1 | [a] | true]
     │  └ Collect[doc.t2 | [b] | true]
     └ Collect[doc.t3 | [c] | true]

becomes:
Join[LEFT | (t2.b = t3.c)]
  ├ Join[INNER | (t1.a = t2.b)]
  │  ├ Collect[doc.t1 | [a] | true]
  │  └ Filter[(t2.b = 1)]
  │      └ Collect[doc.t2 | [b] | true]
  └ Collect[doc.t3 | [c] | true]

I think it may be easier to handle the use cases 1 to 4 in MoveFilterBeneathJoin, which probably makes the special case also easier. Consider the following plan:
Filter[(t1.b = 1)]
  └ Join[LEFT | (t2.b = t3.c)]
     ├ Join[LEFT | (t1.a = t2.b)]
     │  ├ Collect[doc.t1 | [a] | true]
     │  └ Collect[doc.t2 | [b] | true]
     └ Collect[doc.t3 | [c] | true]

First push the filter to the left-hand-lhs of the top join (t2.b = t3.c)] which is use-case 1
 Join[LEFT | (t2.b = t3.c)]
     ├ Filter[(t2.b = 1)]
     |     └Join[LEFT | (t1.a = t2.b)]
     │          ├ Collect[doc.t1 | [a] | true]
     │          └ Collect[doc.t2 | [b] | true]
     └ Collect[doc.t3 | [c] | true]

now we have the simple case as subtree:
Filter[(t2.b = 1)]
 └Join[LEFT | (t1.a = t2.b)]
    ├ Collect[doc.t1 | [a] | true]
    └ Collect[doc.t2 | [b] | true]

which can be already handled by RewriteFilterOnOuterJoinToInnerJoin.

Thank you for providing detailed review @mkleen .
On current master, the use-cases 1~4 you described are already working:
cr> create table t (a int, b int);
CREATE OK, 1 row affected  (1.722 sec)
cr> create table t2 (a2 int, b2 int);
CREATE OK, 1 row affected  (1.736 sec)
cr>  create table t3 (a3 int, b3 int);
CREATE OK, 1 row affected  (1.649 sec)

cr> explain select * from t left join t2 on a = a2 where a > 1;
+------------------------------------------------------+
| QUERY PLAN                                           |
+------------------------------------------------------+
| NestedLoopJoin[LEFT | (a = a2)] (rows=unknown)       |
|   ├ Collect[doc.t | [a, b] | (a > 1)] (rows=unknown) |
|   └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown) |
+------------------------------------------------------+
EXPLAIN 1 row in set (0.308 sec)

cr> explain select * from t left join t2 on a = a2 left join t3 on a2 = a3 where a > 1;
+---------------------------------------------------------+
| QUERY PLAN                                              |
+---------------------------------------------------------+
| NestedLoopJoin[LEFT | (a2 = a3)] (rows=unknown)         |
|   ├ NestedLoopJoin[LEFT | (a = a2)] (rows=unknown)      |
|   │  ├ Collect[doc.t | [a, b] | (a > 1)] (rows=unknown) |
|   │  └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown) |
|   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)    |
+---------------------------------------------------------+
EXPLAIN 1 row in set (0.009 sec)

cr> explain select * from t right join t2 on a = a2 where a2 > 1;
+----------------------------------------------------------+
| QUERY PLAN                                               |
+----------------------------------------------------------+
| NestedLoopJoin[RIGHT | (a = a2)] (rows=unknown)          |
|   ├ Collect[doc.t | [a, b] | true] (rows=unknown)        |
|   └ Collect[doc.t2 | [a2, b2] | (a2 > 1)] (rows=unknown) |
+----------------------------------------------------------+
EXPLAIN 1 row in set (0.003 sec)

cr> explain select * from t right join t2 on a = a2 right join t3 on a2 = a3 where a3 > 1;
+----------------------------------------------------------+
| QUERY PLAN                                               |
+----------------------------------------------------------+
| NestedLoopJoin[RIGHT | (a2 = a3)] (rows=unknown)         |
|   ├ NestedLoopJoin[RIGHT | (a = a2)] (rows=unknown)      |
|   │  ├ Collect[doc.t | [a, b] | true] (rows=unknown)     |
|   │  └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown)  |
|   └ Collect[doc.t3 | [a3, b3] | (a3 > 1)] (rows=unknown) |
+----------------------------------------------------------+
EXPLAIN 1 row in set (0.003 sec)

which I believe are handled by RewriteFilterOnOuterJoinToInnerJoin and got myself confused and expanded the new use-case to RewriteFilterOnOuterJoinToInnerJoin instead of MoveFilterBeneathJoin. (Sorry I should have doubled checked with you before diverging your initial suggestion)
At this point, I think we should make clear separation of RewriteFilterOnOuterJoinToInnerJoin and MoveFilterBeneathJoin as a prerequisite. WDYT?

You can see the exact order of rule execution using explain verbose e.g.:
explain verbose select * from t left join t2 on a = a2 left join t3 on a2 = a3 where a > 1;

+------------------------------------------------------+-----------------------------------------------------------+
| STEP                                                 | QUERY PLAN                                                |
+------------------------------------------------------+-----------------------------------------------------------+
| Initial logical plan                                 | Join[LEFT | (a2 = a3)] (rows=unknown)                     |
|                                                      |   ├ Filter[(a > 1)] (rows=0)                              |
|                                                      |   │  └ Join[LEFT | (a = a2)] (rows=unknown)               |
|                                                      |   │    ├ Collect[doc.t | [a, b] | true] (rows=unknown)    |
|                                                      |   │    └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown) |
|                                                      |   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)      |
| optimizer_rewrite_join_plan                          | NestedLoopJoin[LEFT | (a2 = a3)] (rows=unknown)           |
|                                                      |   ├ Filter[(a > 1)] (rows=0)                              |
|                                                      |   │  └ Join[LEFT | (a = a2)] (rows=unknown)               |
|                                                      |   │    ├ Collect[doc.t | [a, b] | true] (rows=unknown)    |
|                                                      |   │    └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown) |
|                                                      |   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)      |
| optimizer_rewrite_filter_on_outer_join_to_inner_join | NestedLoopJoin[LEFT | (a2 = a3)] (rows=unknown)           |
|                                                      |   ├ Join[LEFT | (a = a2)] (rows=unknown)                  |
|                                                      |   │  ├ Filter[(a > 1)] (rows=0)                           |
|                                                      |   │  │  └ Collect[doc.t | [a, b] | true] (rows=unknown)   |
|                                                      |   │  └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown)   |
|                                                      |   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)      |
| optimizer_rewrite_join_plan                          | NestedLoopJoin[LEFT | (a2 = a3)] (rows=unknown)           |
|                                                      |   ├ NestedLoopJoin[LEFT | (a = a2)] (rows=unknown)        |
|                                                      |   │  ├ Filter[(a > 1)] (rows=0)                           |
|                                                      |   │  │  └ Collect[doc.t | [a, b] | true] (rows=unknown)   |
|                                                      |   │  └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown)   |
|                                                      |   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)      |
| optimizer_merge_filter_and_collect                   | NestedLoopJoin[LEFT | (a2 = a3)] (rows=unknown)           |
|                                                      |   ├ NestedLoopJoin[LEFT | (a = a2)] (rows=unknown)        |
|                                                      |   │  ├ Collect[doc.t | [a, b] | (a > 1)] (rows=unknown)   |
|                                                      |   │  └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown)   |
|                                                      |   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)      |
| Final logical plan                                   | NestedLoopJoin[LEFT | (a2 = a3)] (rows=unknown)           |
|                                                      |   ├ NestedLoopJoin[LEFT | (a = a2)] (rows=unknown)        |
|                                                      |   │  ├ Collect[doc.t | [a, b] | (a > 1)] (rows=unknown)   |
|                                                      |   │  └ Collect[doc.t2 | [a2, b2] | true] (rows=unknown)   |
|                                                      |   └ Collect[doc.t3 | [a3, b3] | true] (rows=unknown)      |
+------------------------------------------------------+-----------------------------------------------------------+

You are right, optimizer_rewrite_filter_on_outer_join_to_inner_join does indeed push the filter (a > 1) into the lhs of the left-join (a = a2). I like the idea of separating RewriteFilterOnOuterJoinToInnerJoin and MoveFilterBeneathJoin and moving over the logic into MoveFilterBeneathJoin.
The 4 use cases should work when any of the described plans appear as tree or subtree within the full logical plan. They can occur as an intermediate step while the optimizer is running. Therefore I would take the 4 use cases add them to MoveFilterBeneathJoinTest, create them directly as a logical plan and make sure every single transformation works correctly.

Hi @mkleen , the PR is not completely done(should be mostly done) yet but it would be great if you can have a look.
As discussed I have refactored RewriteFilterOnOuterJoinToInnerJoin and MoveFilterBeneathJoin into RewriteOuterJoinToInnerJoin and MoveFilterBeneathJoin. They should not be too much different from the first time you reviewed, but added a lot more tests than we discussed since I see it lacking.

I am a little bit confused now, it seems that the rules behavior changed a lot. What I expected was the following:

Extend MoveFilterBeneathJoin to also handle LEFT/RIGHT outer joins to push filter to the preserved side
Modify RewriteFilterOnOuterJoinToInnerJoin that it does not do filter push down and leave that to MoveFilterBeneathJoin

It looks like we have now MoveFilterBeneathJoin which pushes filter to the left and to right of a join and RewriteOuterJoinToInnerJoin  which matches on Join with Filter instead of Filter with Join.

I am not sure if it's a good idea to mix the two different use-cases together in MoveFilterBeneathJoin. I would move the filter push down to the preserved side of the join (e.g. push filter to the left side of left-outer-join) out of RewriteFilterOnOuterJoinToInnerJoin put keep the logic to push filters to the non-preserved side together with outer-to-inner-join logic.

Thank you for reviewing @mkleen. I am guessing preserved side means that there would be no rewrites from outer to inner joins? So what you would like to see is that RewriteOuterJoinToInnerJoin pushes down filters AND rewrites outer to inner join and MoveFilterBeneathJoin pushes down filter but no rewrites?

Thank you for reviewing @mkleen. I am guessing preserved side means that there would be no rewrites from outer to inner joins? So what you would like to see is that RewriteOuterJoinToInnerJoin pushes down filters AND rewrites outer to inner join and MoveFilterBeneathJoin pushes down filter but no rewrites?

A join side is preserved if the join returns all or a subset of the rows from the side. If a side is not preserved, it can provide extra null rows.
I would not mix the use cases. There are basically two cases:

Push filters down to the preserve side for inner, left/right and cross-join
Change a left/right join with a filter on top to an inner join when the filter can be pushed to the non-preserved side and avoid null values

With the current state, we need now both rules for the second case.

@jeeminso Looks good so far 👍

@jeeminso Looks good so far 👍

Thank you @mkleen , I have refined this further and added more tests.

One more detail. Does the special use-case where we first need a pushdown and then a rewrite also work now ?
Filter[(t1.b = 1)]
  └ Join[LEFT | (t2.b = t3.c)]
     ├ Join[LEFT | (t1.a = t2.b)]
     │  ├ Collect[doc.t1 | [a] | true]
     │  └ Collect[doc.t2 | [b] | true]
     └ Collect[doc.t3 | [c] | true]

to:
Join[LEFT | (t2.b = t3.c)]
  ├ Join[INNER | (t1.a = t2.b)]
  │  ├ Collect[doc.t1 | [a] | true]
  │  └ Filter[(t2.b = 1)]
  │      └ Collect[doc.t2 | [b] | true]
  └ Collect[doc.t3 | [c] | true]

One more detail. Does the special use-case where we first need a pushdown and then a rewrite also work now ?
Filter[(t1.b = 1)]
  └ Join[LEFT | (t2.b = t3.c)]
     ├ Join[LEFT | (t1.a = t2.b)]
     │  ├ Collect[doc.t1 | [a] | true]
     │  └ Collect[doc.t2 | [b] | true]
     └ Collect[doc.t3 | [c] | true]

to:
Join[LEFT | (t2.b = t3.c)]
  ├ Join[INNER | (t1.a = t2.b)]
  │  ├ Collect[doc.t1 | [a] | true]
  │  └ Filter[(t2.b = 1)]
  │      └ Collect[doc.t2 | [b] | true]
  └ Collect[doc.t3 | [c] | true]

Yes the case is handled:
cr> explain verbose select * from t1 left join t2 on t1.x = t2.x left join t3 on t2.x = t3.x where t2.x = 1;
+------------------------------------------------------+-------------------------------------------------------+
| STEP                                                 | QUERY PLAN                                            |
+------------------------------------------------------+-------------------------------------------------------+
| Initial logical plan                                 | Join[LEFT | (x = x)] (rows=unknown)                   |
|                                                      |   ├ Filter[(x = 1)] (rows=0)                          |
|                                                      |   │  └ Join[LEFT | (x = x)] (rows=unknown)            |
|                                                      |   │    ├ Collect[doc.t1 | [x] | true] (rows=unknown)  |
|                                                      |   │    └ Collect[doc.t2 | [x] | true] (rows=unknown)  |
|                                                      |   └ Collect[doc.t3 | [x] | true] (rows=unknown)       |
| optimizer_rewrite_join_plan                          | NestedLoopJoin[LEFT | (x = x)] (rows=unknown)         |
|                                                      |   ├ Filter[(x = 1)] (rows=0)                          |
|                                                      |   │  └ Join[LEFT | (x = x)] (rows=unknown)            |
|                                                      |   │    ├ Collect[doc.t1 | [x] | true] (rows=unknown)  |
|                                                      |   │    └ Collect[doc.t2 | [x] | true] (rows=unknown)  |
|                                                      |   └ Collect[doc.t3 | [x] | true] (rows=unknown)       |
| optimizer_rewrite_filter_on_outer_join_to_inner_join | NestedLoopJoin[LEFT | (x = x)] (rows=unknown)         |
|                                                      |   ├ Join[INNER | (x = x)] (rows=unknown)              |
|                                                      |   │  ├ Collect[doc.t1 | [x] | true] (rows=unknown)    |
|                                                      |   │  └ Filter[(x = 1)] (rows=0)                       |
|                                                      |   │    └ Collect[doc.t2 | [x] | true] (rows=unknown)  |
|                                                      |   └ Collect[doc.t3 | [x] | true] (rows=unknown)       |
| optimizer_rewrite_join_plan                          | NestedLoopJoin[LEFT | (x = x)] (rows=unknown)         |
|                                                      |   ├ HashJoin[(x = x)] (rows=unknown)                  |
|                                                      |   │  ├ Collect[doc.t1 | [x] | true] (rows=unknown)    |
|                                                      |   │  └ Filter[(x = 1)] (rows=0)                       |
|                                                      |   │    └ Collect[doc.t2 | [x] | true] (rows=unknown)  |
|                                                      |   └ Collect[doc.t3 | [x] | true] (rows=unknown)       |

the optimization starts from the filter already push down by JoinPlanBuilder.

👍


Request: Even more helpful error messages

If I do something like:
message =
  1

format: String -> String
format str =
  "Hello" ++ str

main =
  text (format message)

I get an error:
The 1st argument to function `format` has an unexpected type.

11|   text (format message)
As I infer the type of values flowing through your program, I see a conflict
between these two types:

    String

    number

But from the error it is not clear enough what I am doing wrong. I cannot see if I am expecting a string and giving a number or the other way around.
This is pretty easy to figure out in this trivial example, but in a complex app figuring out where the wrong type is coming from can be quite difficult.
Would it be possible to provide more information in the error? e.g.
The 1st argument to function `format` has an unexpected type.

11|   text (format message)
As I infer the type of values flowing through your program, I see a conflict
between these two types:

    format expects: String

    given: number

Just adding an indication of what is expected and what was given will help a lot.

Version 0.16 of the compiler, which will be released in the next few days, will have this feature.


Allow the user to change the draw settings (paint, width, cap, pattern, etc.)

Width option is working

Paint option is working

Line cap option is working

Pattern option is working


how do i setup the half way modal.

I guess this might be related to you question :)

thansk for the response. this is exactly what i needed.
On Mon, Jun 19, 2017 at 2:49 AM, Matteo Mazzarolo<EMAIL_ADDRESS>wrote:

Closed #35
https://github.com/react-native-community/react-native-modal/issues/35.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/react-native-community/react-native-modal/issues/35#event-1128520967,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGglp4Xfh2-UjsUILeMDf3b9UYKKwO0jks5sFhnrgaJpZM4N9E0r
.


Investigate Scourge related system error

Hey, so this happened when Scourge was dead no throne but still in game hope that helps you ^^


Cursor meta

Coverage remained the same at 100.0% when pulling 7fa4c3144ad9659e44de3fc5d62522341a962611 on cursor-meta into 27dfead68eb0861cafd4fc60190cb61f3902bd0d on main.


update the grafana helm chart app version to 7.0.5

Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

What this PR does / why we need it:
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[ ] Title of the PR starts with chart name (e.g. [stable/mychartname])

See also: https://github.com/helm/charts/pull/23130

@bykvaadm
#23130 merged. This PR could be closed.


Anchor Text for msmarco-document and msmarco-document-v2

Dataset Information:
We have extracted anchor text pointing to documents in MS MARCO (version 1 and version 2) from several Common Crawl snapshots that can be used as additional retrieval features or for the training of models (e.g., in a distant supervision style like DeepCT).
Links to Resources:

Code and links to the data
Paper

Dataset ID(s) & supported entities:

Dataset ID: msmarco-document/anchor-text and msmarco-document-v2/anchor-text

Checklist
Mark each task once completed. All should be checked prior to merging a new dataset.

[x] Dataset definition (in ir_datasets/datasets/[topid].py)
[x] Tests (in tests/integration/[topid].py)
[ ] Metadata generated (using ir_datasets generate_metadata command, should appear in ir_datasets/etc/metadata.json)
[x] Documentation (in ir_datasets/etc/[topid].yaml)

[ ] Documentation generated in https://github.com/seanmacavaney/ir-datasets.com/

[x] Downloadable content (in ir_datasets/etc/downloads.json)

[x] Download verification action (in .github/workflows/verify_downloads.yml). Only one needed per topid.
[x] Any small public files from NIST (or other potentially troublesome files) mirrored in https://github.com/seanmacavaney/irds-mirror/. Mirrored status properly reflected in downloads.json.

Additional comments/concerns/ideas/etc.
I would be happy to help integrate the anchor texts into ir_datasets. I am not sure what a good Dataset ID would be, it can make sense to integrate it as a subset into the existing msmarco-document and msmarco-document-v2 Ids but it might also make sense to have it as independent Ids.

It looks like I have the basic integration working. In the current version, the anchor texts pointing to a document are concatenated to produce the text of a GenericDoc. I think it might also be helpful to build a second representation where a document has a list of anchors (i.e., not concatenating them) because this might be helpful for training models like DeepCT.

Thanks @seanmacavaney for the feedback!
I have changed the implementation accordingly so that the anchor-text-documents now provide the doc_id, text, and anchors.
The main parts are done, but two things are still missing:

I have not generated the Metadata because it looks like this needs to download all other datasets as well
I was able to generate the Documentation, but I had to change a small thing in the associated script since otherwise the script failed for some Optional datatypes from other datasets and I would not like to push my changes in the script since they are only a workaround. But I checked that the generated documentation for the anchor-text looks like expected:

Would it be ok when we merge the current state and you can  help with the generation of the metadata and documentation?

Awesome, thanks 🤘! This looks great. I opened a PR for this, and I'll take care of the metadata and documentation.

Nice, thanks! Please let me know when I can help further (I already saw that the automated checks failed, but this seems to be caused by the missing metadata).

@mam10eks -- can you accept the PR here with the metadata when you get a chance? https://github.com/mam10eks/ir_datasets/pull/1

Excellent, thanks again @mam10eks!


tmt should install debuginfo packages in recommends / require via dnf debuginfo-install

This came up in:
https://src.fedoraproject.org/rpms/gdb/pull-request/9
Maybe we should make sure tmt is smart enough to know it should install debuginfo packages specially?

Hm, seems like a nice feature.

So this means that the install prepare plugin would filter all *.debuginfo packages and install them using dnf debuginfo-install? If that's it I believe the implementation should be easy.

Do we support yum? (rhel-7 and older) If so debuginfo-install is a way to go and it is provided by either yum-utils or dnf-utils

If debuginfo-install is the way which would work for both old and new releases, the implemetation should not be difficult, I guess. Shall we consider this for 1.5?

The solution should cover all currently supported Fedoras, CentOS and RHEL7+.


Please expose marked

I'd love to see easyMDE.marked exposed as a property. This would allow me to add custom markdown rendering. For example, I would like to add a renderer for images. My MD files use relative references, and I'd like to convert them to absolute references so they display even if the editor is on a different page.
Alternatively, I'd suggest a new option "imageBaseUrl" or the like that you could feed to a marked img renderer that in turn uses the option to convert relative image URLs to absolute URLs. Something similar would be great to have for links as well.
Doing a pull request is currently difficult, because I am using the browser version.

I found a way around (during the sanitizing step). Will create pull requests when adding stuff.


refactor: remove unnecessary config logic

As far as I know, both of the config.ExternalAddress and config.ExternalDNS can get the value from the  cmd arguments. I don't know why need to take the value from the daemonset's template again.
whether the config.Mode is job or server, the logic should be the same.

/assign @oilbeater

Thanks! @xieyanker


Add option to generate Sttp code to Postman

Postman allows to generate code from given http call. Currently it does not support any Scala library, it would be nice to generate Scala+Sttp code. Contribution would need to be made in https://github.com/postmanlabs/postman-code-generators .

Hey @mmatloka I can pick this issue up.
Is there already an issue on the postman-code-generators repo related to this change or should we create one?

Hey @mmatloka, I'll create a ticket on the postman-code-generators repository for this change


Migrate to Azure Web App from Heroku

Target url:
https://tweety-core.azurewebsites.net

Stuck because this
https://github.com/Azure/app-service-announcements-discussions/issues/129
The alternatives can use docker container, but need good CI/CD

Temporary use container ci/cd


[AutoPR sql/resource-manager] Add algorithm types to threat detection disabled alerts description

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/3876

(message created by the CI based on PR content)

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/2671


Add test

Please add phpstan config to this repo
for example https://github.com/szepeviktor/phpstan-wordpress/blob/master/phpstan.neon

Added in v0.1.2


Make high level user documentation

We should make a high level overview of the tool and what it can do, this could either be a readme or github wiki or maybe even rust documentation

We should also provide documentation for sd-lang

The doc directory also needs to published now that github changes are done


Numbers in label name doesn't work.

@hello
out 48
ret
@main
psh %hello1
jmp %hello
@hello1

This example should output '0', but it doesn't.

It prints zero, but due to bug in your code there appears a stackunderflow. Otherwise, I understand the issue.


Deployment of MulticallV3 to Flare's Coston2 network

Please deploy MulticallV3 on the 2nd test network of Flare.
I've sent you 1 C2FLR to pay for deploy costs, you can set 25 gwei as gas price and no priority fee.

Chain
Explorer
Funds sent
Transaction

Coston2
https://coston2-explorer.flare.network/
1 C2FLR
Link to explorer

Thanks @alexdupre, can you provide an RPC URL and chain ID for Coston2? I searched https://chainlist.org/ but don't see it there

RPC URL: https://coston2-api.flare.network/ext/C/rpc
ChainId: 114

Done, thanks! https://coston2-explorer.flare.network/address/0xcA11bde05977b3631167028862bE2a173976CA11/contracts


✨ Append original image's tags as new label

As discussed on Slack, this PR will add image’s original tags as an additional label.

I think I might have found one additional issue, please do not merge yet.

I think I might have found one additional issue, please do not merge yet.

Dokku’s ps:restart triggers docker-image-labeler again, causing it to overwrite the already added original-tags label; so I changed this PR to only append the original-tags label if it is not already present.

Dokku’s ps:restart triggers docker-image-labeler again, causing it to overwrite the already added original-tags label; so I changed this PR to only append the original-tags label if it is not already present.

What was happening when it was "overwriting"? Can you show the label before/after with the previous version?

What was happening when it was "overwriting"? Can you show the label before/after with the previous version?

Before last commit:
$ docker image tag haugene/transmission-openvpn:3.3 dokku/transmission:3.3
$ dokku tags:deploy transmission 3.3
$ docker container inspect transmission.web.1

[
  {
    "…": "… omitted… ",
    "Config": {
      "…": "… omitted… ",
      "Labels": {
        "autoheal": "true",
        "com.dokku.app-name": "transmission",
        "com.dokku.container-type": "deploy",
        "com.dokku.docker-image-labeler/original-tags": "[\"dokku/transmission:3.3\",\"haugene/transmission-openvpn:3.3\"]",
        "com.dokku.dyno": "web.1",
        "com.dokku.image-stage": "release",
        "com.dokku.process-type": "web",
        "dokku": "",
        "org.label-schema.schema-version": "1.0",
        "org.label-schema.vendor": "dokku",
        "org.opencontainers.image.revision": "de2598df732811bfb4ac25395c5f27d6fa3b0d87",
        "org.opencontainers.image.source": "https://github.com/haugene/docker-transmission-openvpn"
      }
    }
  }
]

$ dokku ps:restart transmission
$ docker container inspect transmission.web.1

[
  {
    "…": "… omitted… ",
    "Config": {
      "…": "… omitted… ",
      "Labels": {
        "autoheal": "true",
        "com.dokku.app-name": "transmission",
        "com.dokku.container-type": "deploy",
        "com.dokku.docker-image-labeler/original-tags": "[\"dokku/transmission:3.3\",\"dokku/transmission:latest\"]",
        "com.dokku.dyno": "web.1",
        "com.dokku.image-stage": "release",
        "com.dokku.process-type": "web",
        "dokku": "",
        "org.label-schema.schema-version": "1.0",
        "org.label-schema.vendor": "dokku",
        "org.opencontainers.image.revision": "de2598df732811bfb4ac25395c5f27d6fa3b0d87",
        "org.opencontainers.image.source": "https://github.com/haugene/docker-transmission-openvpn"
      }
    }
  }
]

$ docker image ls | grep transmission
haugene/transmission-openvpn                        3.3                          ff46391746b7   7 weeks ago     122MB
dokku/transmission                                  latest<PHONE_NUMBER>fb   41 years ago    122MB
dokku/transmission                                  3.3                          57cafa84f7cc   41 years ago    122MB
# Notice the different checksums for both images in the dokku/ namespace

After last commit:
$ docker image tag haugene/transmission-openvpn:3.3 dokku/transmission:3.3
$ dokku tags:deploy transmission 3.3
$ dokku ps:restart transmission
$ docker container inspect transmission.web.1

  {
    "…": "… omitted… ",
    "Config": {
      "…": "… omitted… ",
      "Labels": {
        "autoheal": "true",
        "com.dokku.app-name": "transmission",
        "com.dokku.container-type": "deploy",
        "com.dokku.docker-image-labeler/original-tags": "[\"dokku/transmission:3.3\",\"haugene/transmission-openvpn:3.3\"]",
        "com.dokku.dyno": "web.1",
        "com.dokku.image-stage": "release",
        "com.dokku.process-type": "web",
        "dokku": "",
        "org.label-schema.schema-version": "1.0",
        "org.label-schema.vendor": "dokku",
        "org.opencontainers.image.revision": "de2598df732811bfb4ac25395c5f27d6fa3b0d87",
        "org.opencontainers.image.source": "https://github.com/haugene/docker-transmission-openvpn"
      }
    }
  }
]

$ docker image ls | grep transmission
haugene/transmission-openvpn                        3.3                          ff46391746b7   7 weeks ago     122MB
dokku/transmission                                  3.3                          57cafa84f7cc   41 years ago    122MB
dokku/transmission                                  latest                       57cafa84f7cc   41 years ago    122MB
# Checksum now remains the same

Basically what I think was happening is that Dokku ran docker-image-labeler during ps:restart on the dokku/transmission:latest image, which at that point was already once re-labelled by docker-image-labeler, thus did not contain in the RepoTags field what would be truly the original tags – they’d already be labelled in the com.dokku.docker-image-labeler/original-tags field, only to be promptly overwritten with invalid values.

Before last commit:
$ docker image tag haugene/transmission-openvpn:3.3 dokku/transmission:3.3
$ dokku tags:deploy transmission 3.3
$ docker container inspect transmission.web.1

[
  {
    "…": "… omitted… ",
    "Config": {
      "…": "… omitted… ",
      "Labels": {
        "autoheal": "true",
        "com.dokku.app-name": "transmission",
        "com.dokku.container-type": "deploy",
        "com.dokku.docker-image-labeler/original-tags": "[\"dokku/transmission:3.3\",\"haugene/transmission-openvpn:3.3\"]",
        "com.dokku.dyno": "web.1",
        "com.dokku.image-stage": "release",
        "com.dokku.process-type": "web",
        "dokku": "",
        "org.label-schema.schema-version": "1.0",
        "org.label-schema.vendor": "dokku",
        "org.opencontainers.image.revision": "de2598df732811bfb4ac25395c5f27d6fa3b0d87",
        "org.opencontainers.image.source": "https://github.com/haugene/docker-transmission-openvpn"
      }
    }
  }
]

$ dokku ps:restart transmission
$ docker container inspect transmission.web.1

[
  {
    "…": "… omitted… ",
    "Config": {
      "…": "… omitted… ",
      "Labels": {
        "autoheal": "true",
        "com.dokku.app-name": "transmission",
        "com.dokku.container-type": "deploy",
        "com.dokku.docker-image-labeler/original-tags": "[\"dokku/transmission:3.3\",\"dokku/transmission:latest\"]",
        "com.dokku.dyno": "web.1",
        "com.dokku.image-stage": "release",
        "com.dokku.process-type": "web",
        "dokku": "",
        "org.label-schema.schema-version": "1.0",
        "org.label-schema.vendor": "dokku",
        "org.opencontainers.image.revision": "de2598df732811bfb4ac25395c5f27d6fa3b0d87",
        "org.opencontainers.image.source": "https://github.com/haugene/docker-transmission-openvpn"
      }
    }
  }
]

$ docker image ls | grep transmission
haugene/transmission-openvpn                        3.3                          ff46391746b7   7 weeks ago     122MB
dokku/transmission                                  latest<PHONE_NUMBER>fb   41 years ago    122MB
dokku/transmission                                  3.3                          57cafa84f7cc   41 years ago    122MB
# Notice the different checksums for both images in the dokku/ namespace

After last commit:
$ docker image tag haugene/transmission-openvpn:3.3 dokku/transmission:3.3
$ dokku tags:deploy transmission 3.3
$ dokku ps:restart transmission
$ docker container inspect transmission.web.1

  {
    "…": "… omitted… ",
    "Config": {
      "…": "… omitted… ",
      "Labels": {
        "autoheal": "true",
        "com.dokku.app-name": "transmission",
        "com.dokku.container-type": "deploy",
        "com.dokku.docker-image-labeler/original-tags": "[\"dokku/transmission:3.3\",\"haugene/transmission-openvpn:3.3\"]",
        "com.dokku.dyno": "web.1",
        "com.dokku.image-stage": "release",
        "com.dokku.process-type": "web",
        "dokku": "",
        "org.label-schema.schema-version": "1.0",
        "org.label-schema.vendor": "dokku",
        "org.opencontainers.image.revision": "de2598df732811bfb4ac25395c5f27d6fa3b0d87",
        "org.opencontainers.image.source": "https://github.com/haugene/docker-transmission-openvpn"
      }
    }
  }
]

$ docker image ls | grep transmission
haugene/transmission-openvpn                        3.3                          ff46391746b7   7 weeks ago     122MB
dokku/transmission                                  3.3                          57cafa84f7cc   41 years ago    122MB
dokku/transmission                                  latest                       57cafa84f7cc   41 years ago    122MB
# Checksum now remains the same

Basically what I think was happening is that Dokku ran docker-image-labeler during ps:restart on the dokku/transmission:latest image, which at that point was already once re-labelled by docker-image-labeler, thus did not contain in the RepoTags field what would be truly the original tags – they’d already be labelled in the com.dokku.docker-image-labeler/original-tags field, only to be promptly overwritten with invalid values.

Should we extend the repo-tags list instead? As in, decode it if it exists and append any new repo tags to that list?

Should we extend the repo-tags list instead? As in, decode it if it exists and append any new repo tags to that list?

Hmm… I wonder if that’s possible…
I have no idea how Docker works internally but I was somewhat under the impression that the RepoTags list must be dynamically computed based on what sort of tags exist on the system.

Hmm… I wonder if that’s possible…
I have no idea how Docker works internally but I was somewhat under the impression that the RepoTags list must be dynamically computed based on what sort of tags exist on the system.

I wonder if Dokku is like.. internally removing that tag on the image, even though we aren't using that tag?

I wonder if Dokku is like.. internally removing that tag on the image, even though we aren't using that tag?

Merging so I can write tests and any refactors for next release.


solve the problem of duplicate comments and blank lines

solve the problem of duplicate comments and blank lines
reduce the data growth of the item table
in the same case, the size of ItemChangeSets can be reduced
when revoke configuration, if there is only one comment or blank line, it will not be deleted.

What's the purpose of this PR
XXXXX
Which issue(s) this PR fixes:
Fixes #5000
Brief changelog
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Read the Contributing Guide before making this pull request.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit tests to verify the code.
[ ] Run mvn clean test to make sure this pull request doesn't break anything.
[ ] Update the CHANGES log.

Summary by CodeRabbit

New Features

Enhanced handling of comment and blank items for improved processing efficiency.
Streamlined logic for managing deleted items, ensuring only valid entries are processed.

Bug Fixes

Adjusted test assertions to reflect changes in item resolution logic, improving accuracy in handling item states.

Refactor

Simplified code structure for better maintainability and clarity in various methods related to item management.

// TODO
code：com.ctrip.framework.apollo.portal.service.ItemService#revokeItem
Remaining issues：
When revoking a configuration, the number of configuration lines will be reset, which will cause confusion between the number of comments and configuration lines. This cannot be solved at present.

### Add a question about revoke configuration

The initial configuration is as follows

The second step only deletes the fourth line, and the comment on the third line is still retained.
But when I revoke the configuration, I find that the comment on the third line is also deleted.

The reasons why this happens are as follows：
L218：oldKeyMapItem contains a comment or a blank line
L241：The comment or blank line in oldKeyMapItem will be deleted.

So  I removed this comment or blank line in #5232
https://github.com/apolloconfig/apollo/blob/31e6486791de3ad60f059f34957b343839aa5ebb/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ItemService.java#L218C1-L219C1
https://github.com/apolloconfig/apollo/blob/31e6486791de3ad60f059f34957b343839aa5ebb/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ItemService.java#L241

// TODO code：com.ctrip.framework.apollo.portal.service.ItemService#revokeItem Remaining issues： When revoking a configuration, the number of configuration lines will be reset, which will cause confusion between the number of comments and configuration lines. This cannot be solved at present.

I mentioned this problem in my reply above, but I don't have a good solution to solve it completely.
I submitted some small optimizations for this problem. With a small number of changes, the line num can be kept correct as much as possible.
But this code cannot solve the situation where a lot of comments and blank lines are added before undoing.
It also cannot solve the situation where the number of lines of comments changes a lot.
So this is not a complete solution
The code is as follows：
https://github.com/youngzil/apollo/blob/859bbdbf00934e6f308c8685bee616c106b42ebe/apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/service/ItemService.java#L233C6-L238C8


update task progress at least once per second

What purpose does this serve?

When processing long recipes - esp in the soul binder - the progress seems to be stuck.
For nearly all other machines/recipes (except things like dark steel ingots) the ~6% progress updates happens faster than 1 second.
The long delay between updates is noticeable now with the progress tooltips.


Cannot delete pods completely with clear containers

Description

I tested the cri-o with clear containers runtime and found the pod cannot be deleted completely.
Steps to reproduce the issue:

Using kubernetes to create the pod
[xxx]# ./cluster/kubectl.sh get pod
NAME       READY     STATUS    RESTARTS   AGE
pod-test   1/1       Running   0          11m

we can also see from crictl:
[xxx]# crictl pods
PODSANDBOX ID       CREATED             STATE               NAME                NAMESPACE           ATTEMPT
17cd228356cf7       11 minutes ago      SANDBOX_READY       pod-test            default             0
[xxx]# crictl ps -a
CONTAINER ID        IMAGE                                     CREATED             STATE               NAME                ATTEMPT
9c25758ceb0a3      xxx  9 minutes ago       CONTAINER_RUNNING   pod-t               0

Deleting the pod from kubernetes:
[xxx]# ./cluster/kubectl.sh delete pod pod-test
pod "pod-test" deleted
[xxx]# ./cluster/kubectl.sh get pod
No resources found.

The pod is deleted from kubernetes successfully, while in fact, the pod is in "NotReady" state.
[xxx]# crictl pods
PODSANDBOX ID       CREATED             STATE               NAME                NAMESPACE           ATTEMPT
17cd228356cf7       14 minutes ago      SANDBOX_NOTREADY    pod-test            default             0
[xxx]# crictl ps -a
CONTAINER ID        IMAGE               CREATED             STATE               NAME                ATTEMPT
We delete the pod using "criclt", and failed.
[xxx]# crictl rmp 17c
FATA[0005] removing the pod sandbox failed: rpc error: code = Unknown desc = failed to delete container k8s_POD_pod-test_default_08acddcb-ff60-11e7-b62e-f0921cee3497_0 in pod sandbox 17cd228356cf7b872d316a5b2e0f0f4dccc0981646b80bcd919ff7380c186014: /usr/bin/cc-runtime delete --force 17cd228356cf7b872d316a5b2e0f0f4dccc0981646b80bcd919ff7380c186014 failed: failed to connect to proxy after 5s: dial unix /run/virtcontainers/pods/17cd228356cf7b872d316a5b2e0f0f4dccc0981646b80bcd919ff7380c186014/proxy.sock: connect: connection refused
(exit status 1)

if we just use "crictl" to stop the pod, it's successful.
[xxx]# crictl pods
PODSANDBOX ID       CREATED             STATE               NAME                NAMESPACE           ATTEMPT
08182b3a31ad9       2 minutes ago       SANDBOX_READY       pod-test            default             0
[xxx]# crictl ps -a
CONTAINER ID        IMAGE                                     CREATED             STATE               NAME                ATTEMPT
bb65bbe26e757       xx   9 seconds ago       CONTAINER_RUNNING   pod-t               0
[xxx]# crictl stop bb65
bb65
[xxx]# crictl rm bb65
bb65
[xxx]# crictl stopp 0818
Stopped sandbox 0818
[xxx]# crictl rmp 0818
Removed sandbox 0818
[xxx]# crictl pods
PODSANDBOX ID       CREATED             STATE               NAME                NAMESPACE           ATTEMPT

I check the log from "crio" daemon, there is not so much differences. While i check the runtime log, i found the differences:
(1) When using "crictl", the last step is to destroy vm, while the cri-o does not have these cmd:
Jan 22 16:23:19  cc-proxy[17435]: time="2018-01-22T16:23:19.245629005+08:00" level=info msg="hyper(cmd=\"destroypod\", data=\"\")" client=13 name=cc-proxy pid=17435 source=proxy
.....
Jan 22 16:23:19  cc-runtime[18637]: time="2018-01-22T16:23:19+08:00" level=info msg="Stopping VM"
......
time="2018-01-22T16:23:19.382775035+08:00" level=info msg="UnregisterVM()" client=14 name=cc-proxy pid=17435 source=proxy
(2) I found there is "stopping and stopped shims", while in cri-o, i just not found "stopped shims"
Jan 22 16:23:19  cc-runtime[18637]: time="2018-01-22T16:23:19+08:00" level=info msg="Stopping shim" shim-pid=17473 source=virtcontainers subsystem=shim
Jan 22 16:23:19 d cc-runtime[18637]: time="2018-01-22T16:23:19+08:00" level=info msg="Stopped shims" pod-id=5de6e8195caf73c1e9691a1d1503e1b40f211b873694aaa538a30eb1fb103662 shim-count=1 source=virtcontainers subsystem=pod
I'm not sure if the issue should be cri-o or clear containers. just firstly put it here.
Describe the results you received:
Describe the results you expected:
Additional information you deem important (e.g. issue happens only occasionally):
Output of crio --version:
[xxx]# crictl --version
crictl version 1.0.0-alpha.0
the kubernetes is "release-1.9"
(paste your output here)

Additional environment details (AWS, VirtualBox, physical, etc.):

@ddongchen @sameo Is this still an issue?

@ddongchen @sameo is this still an issue?

Closing since this is a year old.

@rhatdan Sorry to bring it back up - just ran into this issue myself:
root@k8s-master-01:/etc/kubernetes# crictl pods
POD ID              CREATED             STATE               NAME                                       NAMESPACE           ATTEMPT
7494df16eb57b       41 minutes ago      Ready               coredns-86c58d9df4-phlqc                   kube-system         0
443c57e280ff2       41 minutes ago      Ready               calico-kube-controllers-644fcf8fbf-k6b9g   kube-system         0
02935a1675edd       41 minutes ago      Ready               coredns-86c58d9df4-t6lbw                   kube-system         0
root@k8s-master-01:/etc/kubernetes# crictl ps -a
CONTAINER ID        IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID
root@k8s-master-01:/etc/kubernetes# crictl rmp 02935a1675edd
FATA[0000] removing the pod sandbox "02935a1675edd" failed: rpc error: code = Unknown desc = sandbox container "02935a1675edd398ee6332e6006853b40a148b41d5c79fb143b74cb0be0a946b" is not fully stopped

I just restarted the host and the state changed from Ready to NotReady and I was able to delete the pod. Prior to that, it was just dangling and I wasn't able to delete the pod via any subset of commands.


Snapshot for JSON file prints unusable output

Package version
2.0.5
Describe the bug
Bug description:

When using Japa snapshots with JavaScript Objects it compares passed data with snapshot correctly, but result, which is printed is unusable and always show all the lines.
Also when printed, AssertionError.actual compared to AssertionError.expected differ, so AssertionError.actual seems to print object on which have been JSON.stringify() called twice.
For clarity I am showing usage bellow.

Case 1 - file snapshot:
Note - using Math.random() for book.rating, so it is always different, so the snapshot fails (after the first one is produced).
  test('json snapshot', ({ assert }) => {
    const book = {
      id: '1',
      title: 'some-book',
      rating: Math.random(),
    }

    assert.snapshot(book).match()
  })

Produced output:
...
  Assertion Error: 'Snapshot file > json snapshot 1' snapshot comparison failed

- Expected  - 5
+ Received  + 5

- {
-   "id": "1",
-   "rating": 0.1091535569921307,
-   "title": "some-book",
- }
+ "{
+   \"id\": \"1\",
+   \"rating\": 0.6270380896300971,
+   \"title\": \"some-book\",
+ }"

   at Object.match src/integrations/assert.ts:16
...

Case 2 - inline snapshot:
  test('inline snapshot', ({ assert }) => {
    const book = {
      id: '1',
      title: 'some-book',
      rating: Math.random(),
    }
    assert.snapshot(JSON.stringify(book)).matchInline(JSON.stringify(book))
  }).pin()

Produced output:
...
  Assertion Error: 'inline snapshot' snapshot comparison failed

- Expected  - 1
+ Received  + 1

- {"id":"1","title":"some-book","rating":0.6508982624640216}
+ "{\"id\":\"1\",\"title\":\"some-book\",\"rating\":0.6508982624640216}"

   at Object.matchInline src/integrations/assert.ts:32
...

Expected behaviour:
For the Case 1, I would like, that it correctly show Number of changed lines as well as correct encoding (without double stringifying object - same for Case 2), e.g. how Jest does it (https://jestjs.io/docs/snapshot-testing#updating-snapshots):

Reproduction repo
No response

Hey, any updates on this? All needed to fix this is to remove double serialization in src/snapshot_file.ts as follows:
  /**
   * Returns the data needed for a future assertion
   */
  getSnapshotTestData(test: Test, value: any) {
    const snapshotName = this.getSnapshotName(test)

    if (!this.#cachedContent) {
      throw new Error(`Snapshot file ${this.#snapshotPath} not found`)
    }

    this.incrementTestCounter(test)

    const expected = prepareExpected(this.#cachedContent[snapshotName])
    const received = serializeSnapshotValue(value, this.#options.prettyFormatOptions)

    return {
      snapshotName,
-     expected: prepareExpected(this.#cachedContent[snapshotName]),
+    expected,
-     received: serializeSnapshotValue(received, this.#options.prettyFormatOptions),
+    received,
      pass: expected === received,
    }
  }

Can provide PR if needed 😄

@Julien-R44

@thetutlage

Sorry for the super late reply, especially as the fix was indeed super simple :/
New release is coming soon


Gauges for stats not available via events

First of all, a thousand thank yous for writing this. Was hoping for someone to replace the aerogear metrics lib with something elegant, and this is it!
Curious if you'd consider adding gauges for stats like number of users, number of sessions (per client), etc.? I can do a PoC if it's something you'd be interested in.

Thank you.
Regarding your request for number of users/sessions - can you explain how to measure them?
Number of user sessions this would be keycloak_event_user_total{type="LOGIN"} - keycloak_event_user_total{type="LOGOUT"}. Or do i miss something?

I made an example here where you can see the changes: https://github.com/kokuwaio/keycloak-event-metrics/compare/main...xgp:keycloak-event-metrics:xgp/gauges?expand=1
They are not metrics you can derive from events, but from the session. For the example, I put them on a timer, but you could also use Gauges that have a function that looks at the stats when observed.

Ok, got it. Let me think about this. Right now i would change something:

add env flag to enable this behavior (like KC_METRICS_EVENT_REPLACE_IDS), maybe KC_METRICS_DATA_COUNT?
interval should be a flag too (default 60s), maybe KC_METRICS_DATA_COUNT_INTERVAL?
avoid transitive dependency com.google.common
create testcases in integration test (must have for me)

I'll pick up your branch and will work on this next week.
Thx for the request!

Great. Thanks for the feedback. I updated the branch to include KC_METRICS_DATA_COUNT and KC_METRICS_DATA_COUNT_INTERVAL. I also extracted out the configuration into a separate class to make it easier to add new params in the future. https://github.com/kokuwaio/keycloak-event-metrics/compare/main...xgp:keycloak-event-metrics:xgp/gauges?expand=1

avoid transitive dependency com.google.common

The google guava library is a Keycloak dependency (https://github.com/keycloak/keycloak/blob/main/pom.xml#L670), so this would not be pulling in an additional transitive dependency. I use it in all of my extensions. As long as it's marked provided in the pom, it doesn't produce problems.

create testcases in integration test (must have for me)

Great rule! I will take a pass at updating and creating new tests this weekend.

Just an update that I spent quite a bit of time wrestling with the tests, but I wasn't able to get them working or understand your testing setup. My apologies, but there's just too much for me in there that I'm not familiar with. Apologies for taking your time.

Please check release 1.0.0

@sschnabe Very cool. I'm glad you were able to incorporate some of those metrics.
In case you want to have the Gauges calculated when they are called, rather than using a timer task, I found that you can create a Gauge that is lazy evaluated. I made an example here: https://github.com/xgp/keycloak-event-metrics/blob/xgp/gauges/src/main/java/io/kokuwa/keycloak/metrics/RealmGauges.java#L33


Add support with 4D0024 encoding

Some wowza cam does not support 4D0024 encoding. I have a quick fixed for replacing from 42c01f to 4D0024. Thanks

Hi !
First of all, thank you for your contribution, it's really appreciated!
As far I can see, you are creating profileIds options that have  the only purpose of finding/replacing things inside the sdp description. As the code is really generic and  have nothing related with  profileId (except the configuration inside examples/play.html) I would not call it that way.
In the code there is already a method to update the sdp dynamically, it's used on the publish method: https://github.com/koala-interactive/wowza-webrtc-player/blob/master/src/webrtc-wowza-player.ts#L166-L168
It could be a better way to extend this callback and call it the play method too.
For now it's used like this:
player.setConfigurations({
  sdpHandler(sdpData, originalHandler) {
    return originalHandler(sdpData);
  },
});

We can add a third parameter type: 'play' | 'publish' and call it with the proper type at the right place.
Then users will be able to customize the sdp directly:
player.setConfigurations({
  sdpHandler(sdpData, originalHandler, type) {
    if (type === 'play' && sdpData.sdp) {
      // Your own logic
      sdpData.sdp = sdpData.sdp.replace(/4D0029/g, '42C01F');
    }

    return originalHandler(sdp);
  },
});

What do you think ?

I think this is a good idea. I will submit a pull request again for this approach.


feat: add Polaris button component

https://github.com/user-attachments/assets/78a9aae9-3df8-4fae-acf9-4b715433a04e
This is the Burron


Changing date in range selector is resetting time to 12

I have a range plugin date picker and when I am changing date, it is not retaining user selected time. Instead it is resetting it to 12.
Steps to reproduce:

Select from date in range picket and change time
Select a diff. date

Expected: Time selection should retain
Actual: Time selection getting reset to 12
Similar issue was raised here https://github.com/flatpickr/flatpickr/issues/1393
but looks like it is not working for react-flatpickr. Here is the codesandbox link
https://codesandbox.io/s/elegant-tharp-bfm1ee
Version Used: 3.9.0 (I tried upgrading to latest version as well but no help)
Platform: Window/Chrome
Mode: Range

Please provide a codesandbox link that demonstrates the behavior you're talking about, thanks!

@jacobmischka  Any updates on this?


Draft: Support for results without proper CRLF

This PR is currently meant as a base for discussion
Some mail servers seem to not return proper line breaks on multiline responses.
Example for UIDL (each row represents one returned data of the WebSocket):
+OK
\r\n
1 <id>
\r\n
2 <id>
\r\n
.

Currently we're expecting the CRLF right at the end of a line which is why this response isn't correctly read and can lead to locks (by not terminating) or wrong data.
This PR provides a naive solution by alwys appending the CRLF to the end if not present.

Now, although I understand the need to have workarounds in place if this is a very prevalent phenomenon, have these issues been reported to the mail servers? It'd be nice to have a tracker number if so.
Injecting content seems a step above checking for a bad delimiter, even though I know both are to fix things.

I reported this behaviour to the provider but they're not willing to fix it since seemingly all (whatever that means) other mail clients have no trouble with it. And also this report isn't publicly available to share.
And regarding injecting content: I agree with you, that this is kind of a risky solution. Currently it's what I went for, because I need this short term. Another solution would be to check in all places, where a delimiter is expected, and only slice the content if it's present.
I'll try to implement this in the coming days. Currently I'm quite busy and I'm not sure when I can get on it, though.

If you want to get the tests to work, you need to add a file, pop.config.json, pointing to your own testing mail server (see the pop.config-sample.json file).


CPU usage 100% in Linux (ubuntu 20.04)

CPU usage 100% in Linux (ubuntu 20.04)
The project XamlControlsGallery under Ubuntu 20.04 and using Avalonia 0.10.0-rc1 has a CPU usage of 100%
To Reproduce
Steps to reproduce the behavior:

Download the XamlControlsGallery source code
Update reference to Avalonia 0.10.0-rc1
Run

Expected behavior
The app shouldn't use 100% of CPU
Screenshots

Desktop:

OS: Linux Ubuntu 20.04
Version 0.10.0-rc1

Does your system have a gpu or is it relying on software rendering?

I'm using a GeForce GTX 760M (Xorg)

Driver version
NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0

I experienced same problem using 0.10.0-preview5 and nvidia gtx 1050ti but using latest 0.10.0-rc1 problem solved.

Hi, I cleaned the build, changed to net5.0 but the cpu is always 100%
This seems really related to Linux because the same project run on Windows on a Celeron J1900 with Intel integrated graphics uses 2/5% of  CPU.
XamlControlsGallery-master.zip

Thanks, It's ok its related to known bug ( https://github.com/AvaloniaUI/Avalonia/issues/2012 ) if you comment out this cpu goes ok
      <!-- <ProgressBar IsIndeterminate="True" Margin="0 40" /> -->

btw for my project where I not used progress bar I had that problem of cpu when using the previous 0.10.0-preview5 while 0.10.0-rc1 solve, so seems there are some progress fixating the issue of high cpu usage on x11 excluding in fact the progressbar when indeterminate mode.

You're right, many thanks!
I had this problem with a different project. In my project I don't have any indeterminate progress bar but I was able to fix the issue.
I have a spinner with an animation set to INFINITE. The spinner is shown or hidden...well when it's hidden it seems that the animation runs and uses 100% of the CPU.
I solved the issue by stopping and starting the animation when i show/hide the spinner.
So it seems that on Linux animations run even on hidden elements, while on Windows not?

CPU usage is probably caused by triggering the render timer more frequently. Everything else should be the same for all platforms.

Ok, many thanks for the help!


Sprint 2 - Team 5 Feature ticket: Detailed Storyline & instructions

Description
Finish the rest chapters to complete the storyline, update the basic information of each npc. Add the in-game instructions page and the background of storyline and instructions.
SubFeature/component (optional)
Stroyline: chapters, background, information of npcs, clues about traitors
Instructions: in-game content and background
Examples (optional)

Instructions pop-up:  A detailed page where player can view the instructions for the game(keyboard control)

Dependencies

Information about npcs (#13 ).
Clues about traitor (#9  ).

Milestones

[x] Finish the basic info of each npc (Sep. 9)
[ ] Finish the chapters of stroyline(Sep. 11)
[ ] The background design for stroyline and instructions (Sep. 12)
[ ] In-game implementation (Sep. 14)

Completion Deadline: Sep. 14
Extras (optional)

[ ] Enhance in-game design

Documentation

Main description of feature
User documentation
...

Members

JOJO (@ShokuJO) (Guanhao Qiao)
Qing Zhang (@Versprechen080) (Qing Zhang)
Yuqiao Tong（@CodingDavid2）（yuqiaotong）
Lulu Zhang (@lily441) (Lulu Zhang)

my surname is zhong


pull request for e2e test at 12/29/2016 01:45:11

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Dynamic saml discovery service

All Submissions:

[ x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x ] Have you added an explanation of what problem you are trying to solve with this PR?
[ x] Have you added information on what your changes do and why you chose this as your solution?
[x ] Have you written new tests for your changes?
[x ] Does your submission pass tests?
[x ] This project follows PEP8 style guide.

I read the changes, It seems an important PR. What's the difference between the previous code, can you comment the feature that come with this PR?

The SAMLBackend start_auth() and disco_query() methods have been
modified so that disco_query() can check the context for the base URL
to use for the discovery service. This allows request microservices
the opportunity to set the base URL for the discovery service to use.

In a request microservice one can do something like
context.decorate(SAMLBackend.KEY_SAML_DISCOVERY_SERVICE_URL, "https://my.org/disco")

Tomorrow I'll do tests, It seems to be also related to  https://github.com/IdentityPython/SATOSA/pull/220


M11215008

Checks

[x] Your YAML file has required fields as listed in Specs
[x] All checks have passed yaml-lint passed
[x] You agree to contribute your work under MIT license

Waiting your individual PRs, closing this BIG PR for now.


Bug with multiple Casts (getDevices)

I'm getting a list with multiple instances of the same Chromecast when calling getDevices. The reason for this seems to be that the singleDevice variable is the same reference for all casts - move it inside the loop and it should be fine!
RCT_REMAP_METHOD(getDevices,
                 resolver:(RCTPromiseResolveBlock)resolve
                 reject:(RCTPromiseRejectBlock)reject)
{
  NSMutableArray *devicesList = [[NSMutableArray alloc] init];
  NSMutableDictionary *singleDevice= [[NSMutableDictionary alloc] init]; //Should be inside array :)
  for (NSString *key in [self.currentDevices allKeys]) {
    GCKDevice *device = self.currentDevices[key];
    singleDevice[@"id"] = key;
    singleDevice[@"name"] = device.friendlyName;
    [devicesList addObject:singleDevice];
  }
  resolve(devicesList);
}

+1

fixed! thank @codetony25 !