rke.cattle.io/external-ip is not updated when EC2 public IP is changed

What kind of request is this (question/bug/enhancement/feature request):
This is a bug
Steps to reproduce (least amount of steps as possible):
In RKE cluster in AWS stop and then start EC2 instance that is a cluster worker node with a public IP.
Result:
AWS assigns new public IP to the instance. Rancher shows correct new external IP in nodes view, but annotation rke.cattle.io/external-ip is not updated and contains old IP. This causes an issue to external DNS service which updates DNS to old IPs and it is not possible to connect to cluster.
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):
2.3.6
Installation option (single install/HA):
Single install

Cluster information

RKE Cluster hosted in AWS

Kubernetes version

Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T17:01:15Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

Docker version

Client:
 Version:      1.10.3
 API version:  1.24
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      linux/amd64

Duplicate of https://github.com/rancher/rancher/issues/14102 although 👍 for fixing

FYI:
Also see this in v2.4.4
I restarted machine with rancher server which did not help. But when I manually restarted the EC2 server the cluster in operational again in UI. Note that the IP in my case was changed on a server that has etcd, control plane + worker and this is the only etcd atm in the cluster.
One issue that still remains that the IP in the list of nodes is still recorder as the old one.

Same problem here, this is a big issue.
Did someone found a way to force this annotation to be changed ? Even manually ?


Make sure that containers run in the timezone of the docker engine

Currently the docker container are all running in the CET timezone. Support that they run in the timezone of the docker host (docker engine).
This will assure that if an epoc is converted into a timestamp the right timezone is used.

The volume map does not work for windows, which I did not think about when doing it.
The other option for changing the timezone of the container is by using the TZ environment variable: -e TZ="Europe/Zurich" .
The issue has been reopened to support this flag as well and document it. The version with the volume mapping we will keep (undocumented) until we know that TZ works fine. If that is the case, we can then remove the solution with the volume mappings.

Instead of mapping the /etc/timezone file from the docker host a file from the stack should be used. This way it is compatible to Windows. The file can be generated by the modern-data-platform docker container.


perf(Google Chrome): get open tabs faster

Description
I add console.time and console.timeEnd pairs to check the time cost of checkAppInstalled() and getOpenTabs().
The result shows that checkAppInstalled() takes 160+ms on my machine, that's longer than getOpenTabs which takes only about 110ms
Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Just fixed grammar in change log. Could you take another look?


Id can also be "computer"

I just received the other value "computer" for the Id, not only Local, Network or OneDrive.

Détails du document
⚠ Ne pas modifier cette section. C’est obligatoire pour docs.microsoft.com ➟ Liaison des problèmes GitHub.

ID: b9c825f6-87e4-caf4-2876-b0838b511e44
Version Independent ID: 41fb3f15-cd3a-611b-1af2-f2e2e6e97419
Content: StorageProvider.Id Property (Windows.Storage) - Windows UWP applications
Content Source: https://cpubwin.visualstudio.com/DefaultCollection/windows-uwp/_git/winrt-api-build?path=%2fwinrt-api-build%2fwindows.storage.storageprovider.yml&version=GBlive&_a=contents
Product: uwp
Technology: winrt-reference
Git Login: knicholasa
Microsoft Alias: nichola

Thanks for bringing this to my attention, I will reach out to the team that owns this area and update the docs.

Turns out there are some bit fields that are not part of this API's set but which might show up since they represent Win32 file type attributes and are not filtered out. The team is aware of this issue and is tracking it internally. Thanks for raising awareness. Closing the issue now.


Install fail on fedora 31

Hey guys, I tried installing on a Fedora 31 unsuccessfully... Hope this helps and maybe you can give me some clues as to what I need to do to get to my instaling mininet:
Expected/Desired Behavior:
flawless install of mininet
Actual Behavior:
SyntaxError: invalid syntax
make: *** [Makefile:52: install] Error 1
Detailed Steps to Reproduce the Behavior:
install.sh -fnpv
Additional Information:
Traceback (most recent call last):
File "setup.py", line 11, in
from mininet.net import VERSION
File "/home/learnux/mininet/mininet/net.py", line 99, in
from mininet.cli import CLI
File "/home/learnux/mininet/mininet/cli.py", line 181
except Exception, e:
^
SyntaxError: invalid syntax
make: *** [Makefile:52: install] Error 1
System was up to date before install attempt.
Feel free to contact me if more information is necessary.
Thanx.

Hi learnuxme!
What version of mininet? Do you use the latest version of this repo?
Could you upload the entire output of install.sh?
Including the output on stderr:
./install.sh -fnpv  > log 2>&1
Best regards

That said, it looks like an issue with python2 vs python3.
except Exception, e is only allowed in python2, not python3.

Hi learnuxme!
What version of mininet? Do you use the latest version of this repo?
Could you upload the entire output of install.sh?
Including the output on stderr:
./install.sh -fnpv > log 2>&1
Best regards

Hi Jufil, to give you a quick reply: I am trying to install what seems to me like the latest stable versionof mininet, version 2.2.2, and have the latest version of python available on Fedora 31, which is 3.7.
Also, being just a noob apprentice sysadmin I did not achive the command you suggested I did(./install.sh -fnpv > log 2>&1), sorry.

I see, no worries!
Did you install mininet from the Package repo? Then you do not need to install it by install.sh any more.
However, that version is relatively old and does not work with python3 (thus the error).
You can check if its installed by "mn -version".
Then make sure, that your python interpreter has version 2.7: python -V should print somthing like 2.7.16
Depending on your use case, this might be sufficient. Then, you can only import it by the python2 interpreter.
Personally, I would recommend you to uninstall the version from the package repo (dnf remove mininet) and install it from this git Repo so you have the latest version.
If you want to install it for python3, specify it like so: PYTHON=python3 ./install.sh -fnv
I think, you do not require the -p flag for now, so you can try executing install.sh without it.
I don't really use fedora, so I'm not perfectly sure how the package manager handles the different python versions.
Best regards

Hi JuFil,
I actually installed version 2.2.2 from this git repo, and since install failed and I seem to have Python 3.8.2, I will retry including the python argument you suggested...

... Failed.
pyflake and pep8 missing. Installed them and now openflow is blocking the install:
"fatal: destination path 'openflow' already exists and is not an empty directory."
Although I deleted the entire folder...
Coming to think of it, could it be useful to write an entire process on how to instll mininet for Fedora? Which packages need to be installed as prerecquisite, etc?
Shoud I contact the dev team?
...Or am I just the biggest dumbest noob ever?

And thank you so much for your help, JuFill.
I am not giving up, I will succeed and install a fully functionnal mininet on my Fedora, now running version 32.

Hi learnuxme,
that install.sh has the exact purpose to install all prerecquisites and do everything needed to install mininet.
1st: Version 2.2.2 does not work with python3. If you want to install it with 2.2.2, you'll need to install it with python2. So make sure that your Python executable has version 2, hence python --version prints  "Python 2.7.X". I do not know how Fedora handles different python interpreters, maybe you need to use python2, then you need to execute PYTHON=python2 ./install.sh..
2nd: Is there a specific reason why you need version 2.2.2? I assume it would be more convenient to use the current version, hence the current version of the master branch.
3rd: That says that the build directory was not cleaned. In the current version, the build dir is defined by:
MININET_DIR="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )/../.." && pwd -P )"

# Set up build directory, which by default is the working directory
#  unless the working directory is a subdirectory of mininet,
#  in which case we use the directory containing mininet
BUILD_DIR="$(pwd -P)"
case $BUILD_DIR in
  $MININET_DIR/*) BUILD_DIR=$MININET_DIR;; # currect directory is a subdirectory
  *) BUILD_DIR=$BUILD_DIR;;
esac

IMHO, the install script is not perfect. I might issue a PR to improve the install.sh to eliminate such inconveniences.
4th: Try to perform the phases one-by-one. Hence, run ./install.sh -f, then ./install.sh -n, and ./install.sh -v
I hope I could help
Best regards

Hi JuFill,
Do not hope, you can rest assured you are being of GREAT help, and I cannot thank you enough for this time you spend on me.
Following you advice, I removed the openflow and mininet directories from the 2.2.2 install script, then attempted an install from 2.3.0d6 (my mistake, seeing this "d6" suffix tricked into thinking it was some beta version). I also followed your advice and started intalling each package separately:
./install.sh -f: worked like a charm!
./install.sh -v:  no success with the error message:
No match for argument: openvswitch-controller
BUT I found a workaround: installed the appropriate openvswitch-devel package.
./install.sh -n: unsuccessful... Failing in installing dependencies:
No match for argument: pyflakes
Package pylint-2.4.4-3.fc32.noarch is already installed.
No match for argument: python-pep8
Package python3-pexpect-4.7.0-5.fc32.noarch is already installed.
although, I made sure to have the latest 2.2.0 pyflakes packages.
Any clues?
Thanx to your assistance, I am getting there. Will look into it and try and solve this on my own, but don't hesitate to get back to me if any tips on solving this issue.
Thank you so much again.
Regards

Hi @learnuxme,
I created the pull request #1003 to make the installation work on Fedora 32. I did some quick tests, and it seems to work. I would appreciate if you could take a look and provide some feedback.
Regards,
Adriano.

Hi @learnuxme,
I created the pull request #1003 to make the installation work on Fedora 32. I did some quick tests, and it seems to work. I would appreciate if you could take a look and provide some feedback.
Regards,
Adriano.


add feed auth

enable NuGet installer cross-plat (TS) task to work with VSTS feeds

Hi @IvyMH, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

You also have to update the task version in task.json

@IvyMH, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, MSBOT;


Remove Ian from admin list

resolution_fixed   |    by<EMAIL_ADDRESS>
Since Ian's no longer with us and mails to his CSIRO account are bouncing I'll be removing him from the admin list
The original message was received at Wed, 16 Jul 2014 15:54:10 +1000
from accessdev.nci.org.au [<IP_ADDRESS>]

   ----- The following addresses had permanent fatal errors -----
<EMAIL_ADDRESS>    (reason: 550 #5.1.0 Address rejected.)
    (expanded from<EMAIL_ADDRESS>
   ----- Transcript of session follows -----
... while talking to act-mx.csiro.au.:
>>> RCPT<EMAIL_ADDRESS><<< 550 #5.1.0 Address rejected.
550 5.1.1<EMAIL_ADDRESS>User unknown

Issue migrated from trac:141 at 2024-01-31 17:38:08 +1100

Scott Wales<EMAIL_ADDRESS>changed status from new to closed

Scott Wales<EMAIL_ADDRESS>committed [0]

In [changeset:"0f0395a07aba245b8e6e72ea6699e85428bd5182/puppet"]:
#CommitTicketReference repository="puppet" revision="0f0395a07aba245b8e6e72ea6699e85428bd5182"
Remove Ian from admin lists

Thanks for all your help, sad to see you go

Fixes #141

Scott Wales<EMAIL_ADDRESS>changed resolution from `` to fixed


Stop exporting *TestCase.php, Traits and Interfaces, and ci-integration.sh

I know this is scheduled for 3.0. Although I don't understand why (tests are not part of the public API), so here's a PR anyway. Feel free to close it or keep it around for 3.0. Whatever you do with it is up to you.

Some *TestCase classes are required because the src directory provides base test classes (for third party rules authors) that depend on those. This is to be fixed in the next major version but since this doesn't seem to be handled yet, maybe you can retarget your pull request onto branch 3.0?
Regarding ci-integration.sh, IIRC it is required for tests in CI.

afaik ci will checkout the repo entirely, no? ci-integration.sh is not necessary in distributions thus can stay here.

@kubawerlos oops yeah, i didn't stick around long enough to see the builds failed. Will check in follow up PRs.


Creating a custom gateway proxy with VS selector not configures the proxy correctly: no listening ports

Gloo Edge Version
1.14.x (latest stable)
Kubernetes Version
1.24.x
Describe the bug
When defining several gatewayProxies using the Helm Chart, it's required to define the virtualService selector for properly routing traffic.
This works fine for proxies with just one port, (HTTP or HTTPS) But when configuring both HTTP and HTTPS ports, the proxy deployed is not able to open any port.
Helm config:
gatewayProxies:
  default: # Proxy name for public access (Internet facing)
    envoyLogLevel: debug
    disabled: false # overwrite the "default" value in the merge step
    kind:
      deployment:
        replicas: 1
    service:
      httpPort: 80
      httpsFirst: false
      httpsPort: 443
      type: LoadBalancer
    gatewaySettings:
      customHttpsGateway: # using the default HTTPS Gateway
        virtualServiceSelector:
          gateway-type: default # label set on the VirtualService
      customHttpGateway: # using the default HTTPS Gateway
        virtualServiceSelector:
          gateway-type: default # label set on the VirtualService

Generated GW Resources:
apiVersion: gateway.solo.io/v1
    kind: Gateway
    metadata:
      name: default
      namespace: default
      labels:
        app: gloo
    spec:
      bindAddress: "::"
      bindPort: 8080
      httpGateway:
        virtualServiceSelector:
          gateway-type: default
      useProxyProto: false
      ssl: false
      proxyNames:
      - default
    ---
    apiVersion: gateway.solo.io/v1
    kind: Gateway
    metadata:
      name: default-ssl
      namespace: default
      labels:
        app: gloo
    spec:
      bindAddress: "::"
      bindPort: 8443
      httpGateway:
        virtualServiceSelector:
          gateway-type: default
      useProxyProto: false
      ssl: true
      proxyNames:
      - default

Proxy listening ports:
/ $ netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:8081            <IP_ADDRESS>:*               LISTEN      7/envoy
tcp        0      0 <IP_ADDRESS>:19000         <IP_ADDRESS>:*               LISTEN      7/envoy

Steps to reproduce the bug

configure Helm Chart for deploying some gatewayProxies
Configure at least one of them with customHttpsGateway and customHttpGateway (just one works fine)
Deploy solution
No Ports will be open by the proxy

Expected Behavior
Both Ports will be listening together
Additional Context
No response

SAME ISSUE !! Did you fix this ? bro


Beam worker closing gRPC connection with many workers and large shuffle sizes

When I run a job with many workers (100 or more) and large shuffle sizes (millions of records and/or several GB), my workers fail unexpectedly with

python -m apache_beam.runners.worker.sdk_worker_main
E0308 12:59:18.067442934     724 chttp2_transport.cc:1103]
  Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings"
Traceback
(most recent call last):
 File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main

   return _run_code(code, main_globals, None,
 File "/usr/local/lib/python3.8/runpy.py", line 87,
in _run_code
   exec(code, run_globals)
 File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
line 264, in <module>
   main(sys.argv)
 File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
line 155, in main
   sdk_harness.run()
 File "/usr/local/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 234, in run
   for work_request in self._control_stub.Control(get_responses()):
 File "/usr/local/lib/python3.8/site-packages/grpc/_channel.py",
line 426, in __next__
   return self._next()
 File "/usr/local/lib/python3.8/site-packages/grpc/_channel.py",
line 826, in _next
   raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous
of RPC that terminated with:
       status = StatusCode.UNAVAILABLE
       details = "Socket closed"

       debug_error_string = "{"created":"@1646744358.118371750","description":"Error received from
peer ipv6:[::1]:34305","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Socket

closed","grpc_status":14}"
>

This is probably related to or even the same as BEAM-12448 or BEAM-6258, but since one of them is already marked as fixed in a previous version and both reports have large tails of unreadable auto-generated comments, I decided to create a new issue.
There is not much more information I can give you, since this is all the error output I get. It's really hard to debug and with the large number of workers I don't even know if the worker reporting the error is actually the one experiencing it.
Imported from Jira BEAM-14070. Original Jira may contain additional context.
Reported by: phoerious.

Bump. This is happening to me constantly. If I have too many workers or shuffle a lot of data, it's pretty much impossible to get a job past the first stages as they keep failing with
2022/09/19 14:35:00 Received signal: terminated
2022/09/19 14:35:00 Python (worker 2-1) exited: signal: terminated```

either with or without a stacktrace.

Recent versions of beam have grpc.keepalive_timeout_ms already set to 300000ms, so that can't be the only thing. The issue is dependent on overall processing speed, however. So if a node is a bit slow or if the shuffle just takes a while to process, the job crashes.

Folks, this is a deal breaker. It happens ALL THE TIME. It is literally impossible to run any large job on Beam with Apache Flink and I have no idea how to fix it. I would submit a pull request, but I need a pointer where to start looking.

We met the same problem in PyFlink (Based on beam portability framework). It took me some time to locate and reproduce the problem. The root cause is that the BDP ping period is locally-decided. We can reduce the KEEP_ALIVE_TIME time on the server side or increase the grpc.keepalive_time_ms on the client side. Here is the PR of the modification I made in pyflink
https://github.com/apache/flink/pull/21363

Also running into this with the Spark RDD Runner on the Python SDK Harness:
22/11/30 03:35:04 INFO Executor: Finished task 505.0 in stage 8.0 (TID 1145). 18572 bytes result sent to driver
22/11/30 03:35:55 INFO Executor: Finished task 434.0 in stage 8.0 (TID 1074). 18572 bytes result sent to driver
E1130 03:36:33.423173032    2378 chttp2_transport.cc:1016]             ipv4:<IP_ADDRESS>:38291: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 20000ms
22/11/30 03:36:33 ERROR py:641: Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "/databricks/python3/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py", line 634, in _read_inputs
    for elements in elements_iterator:
  File "/databricks/python3/lib/python3.7/site-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/databricks/python3/lib/python3.7/site-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Socket closed"
        debug_error_string = "UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:38291 {grpc_message:"Socket closed", grpc_status:14, created_time:"2022-11-30T03:36:33.423534531+00:00"}"
> Traceback (most recent call last):
  File "/databricks/python3/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py", line 634, in _read_inputs
    for elements in elements_iterator:
  File "/databricks/python3/lib/python3.7/site-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/databricks/python3/lib/python3.7/site-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Socket closed"
        debug_error_string = "UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:38291 {grpc_message:"Socket closed", grpc_status:14, created_time:"2022-11-30T03:36:33.423534531+00:00"}"

I do have a shuffle/GroupBy, but I don't understand why that would cause this. As I understand it the SDK Harness is only used to execute DoFns, and the GroupBy/shuffle is done on the Spark side, which shouldn't affect the SDK Harness.

The latest version of Beam is much more stable in this regard, but it still happens at times, particularly if a node is a bit oversubscribed.

Any progress? Beam is so brittle, it's almost unusable. There should never be a situation where any part of Beam fails a job because some magical number of pings was reached.

No luck. During shuffle, it still fails with
E1220 18:51:26.523480166     219 chttp2_transport.cc:1031]   ipv6:%5B::1%5D:45461: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 20000ms

I have played around with all sorts of values in channel_factory.py without success so far. I'm trying grpc.keepalive_time_ms with a value of 60000 now. Let's see. I suppose ServerFactory should also set permitKeepAliveWithoutCalls(true).

60000 seems to be working. Can we make this change upstream, please?

According to this documentation, the server may send GOAWAY with ENHANCE_YOUR_CALM to the client if "the client's GRPC_ARG_KEEPALIVE_TIME_MS setting is lower than the server's GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS." Therefore, it seems that we should consider decreasing the value of GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS to 19 seconds, instead of modifying the KEEP_ALIVE_TIME_SEC on the server side. What do you think @HuangXingBo ? Alternatively, we could increase the value of grpc.keepalive_time_ms in the channel_factory.py to a value higher than GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS, which is set to 300 000 ms by default. In my case, I chose to set it to 300 001 ms, and I have not encountered the error again.

Whatever you do, please leave enough wiggle room. Very often these timeouts occur because individual nodes respond slower than normal because of dying hardware, network congestion etc. These can introduce delays of several seconds or even minutes.


NIFI-1843 Default log level for LogAttribute.

added a logger entry for LogAttribute to specify INFO as log level
replaced some tab characters to whitespace (existing checkstyle error)

@ijokarumawak
Thanks for addressing this.  Verified build and contrib locally and will merge.


Add end-to-end tests: add CI importing documents to the Meilisearch server

To prevent having this kind of issue in the future, we need to add a CI that checks the Meilisearch server can receive documents and well process them.
A basic version of the CI script should

Run the Meilisearch server without any argument
Add a simple document, ex: [{"id": 1, "name": "toto l'asticot"}]
Check the document was well processed.

The script should be run in a new job in this file.
We also need to add this job to our bors.toml file.

The #2254 PR fixes this issue in another way: All the tests are now running with the default CLI parameters instead of the Default implementation. Doing so exposes the values that Clap would have given you if a user had run Meilisearch without any arguments, except the DB and dumps path.


[flutter_tools] throw exception instead of string from startApp

Description
Update to throw exception and add test case. We were catching on Exception, so the string was slipping through - but this is still a fatal error so we need to exit.

Related: https://github.com/flutter/flutter/issues/52272


If all windows are closed preferences does not open

under OSX 10.10.3 atom build 0.199.0 if you close all windows then hit atom->preferences the preferences windows does not come up. You have to do it twice.
If using keyboard shortcuts you must hit command+, two times in order to actually get the presences window.
Expected behavior: hit it once and the preferences window comes up. If no windows are currently open then it should not only open a window, but then automatically open preferences.

Thanks for the report! This is a duplicate of https://github.com/atom/settings-view/issues/503 – please subscribe there for updates.


[CQLReplicator on Glue] Set replication starting timestamp

Is it possible to have the same functionality as the ECS version about ENABLE_REPLICATION_POINT and STARTING_REPLICATION_TIMESTAMP?
https://github.com/aws-samples/cql-replicator/blob/main/ecs/README.MD?plain=1#L121-L122
Basically, replicating data after a specific point of time, not all the data.

yeah, it's possible.

1/ Add a flag --start-replication-from [epoch-time] in cqlreplicator and CQLReplicator.scala. Works only in combination with --writetime-column [regular-column].
2/ Replicates rows if writetime(regular-column)>epoch-time
3/ Add a filter statement in keysDiscoveryProcess():  val primaryKeysDf = sparkSession.read.option("inferSchema", "true").table(source).persist(StorageLevel.DISK_ONLY)
4/ Add checks:  regular-column is supplied,  epoch-time is greater 0


check Napa identifier compliance for (old) "Stegen" study gold:Gs0114663

The project.Makefile in the main branch already has some code that illustrates making filtered downloads from specified collections from a specified MongoDB server. It will need to be enhanced to use more complex, multi-collection filtering (for which there aren't any API endpoints yet)
# user is responsible for providing pre-tested, properly escaped, filtered https://api-napa.microbiomedata.org/docs#/metadata requests
# todo: explicitly running this through the python interpreter emits less logging
assets/filtered-api-requests/filtered-request-result.yaml:
	$(RUN) build-datafile-from-api-requests \
		--output-file $@ \
		--api-url "https://api-napa.microbiomedata.org/nmdcschema/study_set?filter=%7B%22id%22%3A%22nmdc%3Asty-11-aygzgv51%22%7D&max_page_size=999999" \
		--api-url "https://api-napa.microbiomedata.org/nmdcschema/biosample_set?filter=%7B%22part_of%22%3A%22nmdc%3Asty-11-aygzgv51%22%7D&max_page_size=999999" \
		--api-url "https://api-napa.microbiomedata.org/nmdcschema/omics_processing_set?filter=%7B%22part_of%22%3A%22nmdc%3Asty-11-aygzgv51%22%7D&max_page_size=999999"

use shcema v7.8.0

new Study id: nmdc:sty-11-aygzgv51
why isn';t thi working?
https://api-napa.microbiomedata.org/studies?filter=gold_study_identifiers%3A%27gold%3AGs0114663%27&per_page=25

# user is responsible for providing pre-tested, properly escaped, filtered https://api-napa.microbiomedata.org/docs#/metadata requests
# todo: explicitly running this through the python interpreter emits less logging
assets/filtered-api-requests/filtered-request-result.yaml:
	$(RUN) build-datafile-from-api-requests \
		--output-file $@ \
		--api-url "https://api-napa.microbiomedata.org/nmdcschema/study_set?filter=%7B%22id%22%3A%22nmdc%3Asty-11-aygzgv51%22%7D&max_page_size=999999" \
		--api-url "https://api-napa.microbiomedata.org/nmdcschema/biosample_set?filter=%7B%22part_of%22%3A%22nmdc%3Asty-11-aygzgv51%22%7D&max_page_size=999999" \
		--api-url "https://api-napa.microbiomedata.org/nmdcschema/omics_processing_set?filter=%7B%22part_of%22%3A%22nmdc%3Asty-11-aygzgv51%22%7D&max_page_size=999999"

https://api-napa.microbiomedata.org/studies?filter=id%3Agold%3AGs0110138&per_page=25

todo:

add more pattern slot_usages

#1212
preferable for LinkML to check identifier patterns without this workaround
may need to apply some of these rules to an older schema or filter out expected errors

@mbthornton-lbl write a tool to fetch records that are related to some entity though multiple collections (looping solution)

Related Issue:
#1609

@brynnz22 wrote a guide for using the find and metadata endpoints in the nmdc -runtime API:
https://github.com/microbiomedata/NMDC_documentation/blob/main/docs/howto_guides/api_gui.md

work happened on this last sprint, this sprint @mbthornton-lbl  will expand to check records with legacy ids. Everything validates with linkml-validate for the records that have updated identifiers.

@mbthornton-lbl did you want to move this ticket over to the next sprint or are you working off of a different ticket to include records that haven't been re-ided yet.

@aclum I will move this to the next sprint - I have been able to extract the Stegen study, including following was_informed_by gold seq project identifiers / alternative identifiers.  The extracted results show the Old ientries still in place. I need to check if the delete-records tools was run or not

@mbthornton-lbl doesn't appear to be active in the last 2 weeks.  Can you remove from the sprint if you're not working on it.

Update Results:

ran extract-study on nmdc:sty-11-aygzgv51
ran linkml-validate on the resulting yaml data

No validation errors related to metagenomic data
Errors for metabolomic and NOM
4 Types of Errors
[ERROR] [./local/nmdc:sty-11-aygzgv51.yaml/0] Additional properties are not allowed ('alternative_identifier' was unexpected) in /data_object_set/1127

[ERROR] [./local/nmdc:sty-11-aygzgv51.yaml/0] 'id' is a required property in /data_object_set/1195
[ERROR] [./local/nmdc:sty-11-aygzgv51.yaml/0] 'name' is a required property in /data_object_set/1195
[ERROR] [./local/nmdc:sty-11-aygzgv51.yaml/0] 'description' is a required property in /data_object_set/1195

[ERROR] [./local/nmdc:sty-11-aygzgv51.yaml/0] ['nmdc:omprc-11-s1vt0h44'] is not of type 'string' in /metabolomics_analysis_activity_set/7/was_informed_by

[ERROR] [./local/nmdc:sty-11-aygzgv51.yaml/0] ['nmdc:omprc-11-dz40jm83'] is not of type 'string' in /nom_analysis_activity_set/23/was_informed_by

Unable to convert to RDF with lonkml-convert:
ValueError: File "nmdc:sty-11-aygzgv51.yaml", line 12852, col 5:  Unknown argument: alternative_identifier = [['emsl:output_499451']]

we went over these errors in our 1x1, I've slacked Yuri about the alternative_identifiers slot missing the 's' and that was_informed_by should be a string, not an array. The id, name, and description errors are b/c the data objects don't exist. Yuri has been provided a list of how to remap these since this is not an issue in prod.

linkml-validate results: No issues found
`linkml-convert to RDF datastore complete

Reviewed with @turbomam - complete


Used default value if empty string in request

Some problems with default value
For example, we have route like this
@router.get("/all")
async def get_all(
    itemId: Optional[int] = Query(None),
    page: int = Query(1),
    size: int = Query(10),
):
    return {"itemId": itemId, "page": page, "size": size}

and I try to send request with empty value GET /all?itemId. I hope, I'll get default value from Query(None), but in starlette into request empty value always is empty string. If I send this request GET /all?itemId, I'll get validation error, because we compare string with int.
I decided the best way equate empty string to empty value.

I think what you’re trying to do is /all?itemId=.

I think I want to upvote this issue and add more cases. Sometimes the user/frontend sends us the backend API with these parameters:

itemId=
itemId=null (because frontend mostly javascript)
itemId=undefined (because frontend mostly javascript)

It would be nice if FastAPI filtered those parameters to be a default parameter or None

Line 618 if value in (None, "", "null", "undefined"):

"", "null" and "undefined" are valid strings and shouldn't be cast to None or raise an error when used with a required parameter.

Some problems with default value
For example, we have route like this
@router.get("/all")
async def get_all(
    itemId: Optional[int] = Query(None),
    page: int = Query(1),
    size: int = Query(10),
):
    return {"itemId": itemId, "page": page, "size": size}

and I try to send request with empty value GET /all?itemId. I hope, I'll get default value from Query(None), but in starlette into request empty value always is empty string. If I send this request GET /all?itemId, I'll get validation error, because we compare string with int.
I decided the best way equate empty string to empty value.

You might want to just filter empty values out in your frontend.

Hey there! Thanks for the interest! :cake:
The problem here is that query parameters are actually more of a concept than a type. In the end, they are part of the URL, which is just a string, it doesn't have any notion of types, so all query parameters are naturally strings, and if they don't have a value, they are by default just strings. For example, it's not really possible to explicitly send a query parameter with a value of None. So, a query parameter that is provided but doesn't have a value, like ?itemId= would indeed be a string (an empty string). And then a query parameter without the equal sign, as in ?itemId would be something a bit weird, as those parameters are expected to be key-value pairs (https://developer.mozilla.org/en-US/docs/Learn/Common_questions/What_is_a_URL#parameters).
Before deciding that parameters that don't have an equal sign should be treated as None, it would probably be better to find a reference in a spec (e.g. an IETF RFC, a W3C spec) specifying that.
If that was the case, then it would probably make sense to fix this in Starlette itself.

Now, if that was the case, this implementation would have a problem, that a URL with ?itemId="", which is actually passing the required value (it's just that it's an empty string, but the "required" value is given) would be taken as invalid for "not passing a required value".
It's possible to do the check with None because if the parameter is ever in the URL, the value will never be None, it would only be None when the parameter is not sent in the URL.

Given that, I'll close this PR now. But if you think this still should be done and you find some references to some specs that define that, please create a new PR (and some tests please).
Thanks for the effort! :coffee:

Thanks for not only a comment, but the whole knowledge. Keep a good work, Sir


[SUPPORT][metadata] Slow performance when inserting to a lot of partitions and metadata enabled

Tips before filing an issue

Have you gone through our FAQs?

Join the mailing list to engage in conversations and get faster support at<EMAIL_ADDRESS>

If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced
Recently we enabled metadata for one big table. This table includes ~700k partitions in the test environment and the usual insert affects ~600. Also it's important to say it's even more pressing if we have enabled the timeline server.
The issue we have is visible in the slow getting small files stage - e.g. for 600 insert-affected partitions(600 tasks in this stage) it takes ~2-3 minutes, and for 12k tasks(partitions affected) - ~40 minutes.
important details:

Metadata is enabled
Timeline server is enabled
Metadata table hfile file size - about 40MB
Number of log files: "hoodie.metadata.compact.max.delta.commits" = "1"
Only file listing is enabled in metadata
I dug into this a lot and enabled detailed logs and this is what I think maybe the issue:
In logs "Metadata read for %s keys took [baseFileRead, logMerge] %s ms" - half of the values are single digit numbers, half are > 1000, e.g. Metadata read for 1 keys took [baseFileRead, logMerge] [0, 12456] ms.
 Do you aware of any issue similar to this(in 0.12.1 and 0.12.2)? As I see in the code it seeks up to a partition path, may it happen that readers are somehow not being reused, so these 40mb files are seeked again and again thousands of times?
As a remediate, we turned off the embed server, so as I understand same thing will be done on executors, but it will be less problematic since parallelism is much bigger. Also, the downside is markers logic listing.

To Reproduce
Steps to reproduce the behavior:
Probably generating a large set of partitions(up to 30-40 MB hfile size) and running insert to thousands of partitions may reproduce it.
Expected behavior
getting small files should not be so long. Without metadata + embed server it takes ~40sec to check 12k partitions, while with metadata it's 40+ minutes.
Environment Description

Hudi version : 0.12.1-0.12.2

Spark version : 3.3.0-3.3.1

Hive version :

Hadoop version : 3.3.3

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) : yes

Additional context
I can try to make a reproduction if you don't know about anything like this.
Stacktrace
Add the stacktrace of the error.

I tried to change the settings to
        "hoodie.hfile.max.file.size" = "45829120"
        "hoodie.hfile.block.size" = "548576"
        "hoodie.hfile.compression.algorithm" = "snappy"

it looked to be fast before the first compaction happened, I mean when I had a couple of.log files it was fast - ~1200 partitions lookup in ~60MB file lookup was taking seconds, but after the first compaction, it started to be even slower.

As I understood, it works fast until the first HBase record appears, I checked and it's even about the same size, but while it's a log - it cached in some internal spillable map, but once it becomes HFile - it's all their code and it looks either in your code/their code it becomes slow.

@alexeykudinkin I saw you were touching logblocksreader, can you check if the HFile integration you have has the same performance?

So I debugged it to a point where I'm 99% confident that the issue is a suboptimal lookup in hbase file. As I checked in the minor environment it's about 400mb uncompressed(stored as 40mb compressed) - and it takes seconds to find partition files in it.

Confirmed to be the same in 0.13.1...

@VitoMakarevich Closing this issue as it is expected only. Feel free to reopen if you think otherwise.


Save in progress submissions between new submission workflow steps

[x] Explicitly save the incomplete Submission object between workflow steps

Both forward and back directions in the workflow?
submission.submitted must be false until the user clicks the final Submit button in the workflow review step. The workflow review step sets this to true

[ ] Change submissions view to filter out in progress submissions?
[x] Submissions table must correctly display in-progress submissions

Allow user to edit or remove them

[x] Test
[ ] Test

1d

Modifications required to get the Submissions/Submission Details pages to work nicely are a bit more complicated than initially thought. We can take this opportunity to allow users to resume or destroy "draft" submissions, allowing them to start a submission, then finish it at another time or another computer. A few things need to happen to support this:

[ ] Extend the submission-handler to delete records
[ ] Add new 'Action' in the last column of the Submissions table
[ ] Add this action to Submission Details page
[ ] Submissions table and Submission Details page need to be updated to accommodate incomplete or missing data
[ ] Extend submission status logic to handle these Draft submissions
[ ] Add an explicit ‘Abort’ feature to the submission workflow that will remove the “draft” submission

[ ] Confirmation box on Abort?

If we want to actually delete the Fedora object corresponding to a "draft" submission, we have to think about other objects which reference a draft submission or are created along with a draft submission.

Need to modify the way Publication objects are saved during the workflow. It is currently saved during the final review step. However, in order to initialize draft submissions in the workflow, we need to be able to link and load a publication object. The Publication object should be saved after the publication information has been entered - during the first step (workflow-basics). This will involve making several changes.

Note for Abort button:
The nicer looking place to put this button in the submission workflow requires adding the button to all submission workflow steps. The 'abort' action will need to be bubbled from controller/submissions/new up to each workflow component.
Example:

controllers/submissions/new defines the base action abort
controllers/submissions/new/basics

Inject a reference to the submissions/new controller: Ember.inject.controller('submissions.new')
Define abort action that explicitly sends abort to the injected controller

templates/submissions/new/basics add abort action to the instantiation of workflow-basics component
components/workflow-basics add new cancel action

When this action is invoked, call sendAction('abort') to push the action back down to the basics controller

templates/components/workflow-basics finally add the new button with the reference {{action 'cancel'}}

Will close this and open tickets with remaining issues that will be addressed later


Scrollbar cannot work well in Mac

Hi,
Scrollbar cannot work well in Mac when I use it in my react project.
I find the scrollbarWidth is always 0, when I debug the code.
This is your code in getScrollbarWidth.js
var scrollbarWidth = false;

function getScrollbarWidth() {
  ...
  scrollbarWidth = div.offsetWidth - div.clientWidth;
  ...
}

The values of offsetWidth and clientWidth are both 0.
Then whenever I use it, the value of scrollbarWidth is 0.
It cannot work.
When this code is running, all values of offsetWidth, clientWidth and getBoundingClientReact are 0, but getComputedStyle is OK, so maybe you can use it.
My Code is:
<Framework>

         <div className={`page-body ${module}`}>

                 <Scrollbars
                   autoHide={true}
                   autoHeight={false}>

                          <RouterSwitch>
                               <For each="menu" of={menus}>
                                  <Route
                                     {...menu}
                                     key={menu.name}
                                     exact={true}
                                     component={menu.component}
                                  />
                              </For>
                              <Redirect to="/components/icon"/>
                            </RouterSwitch>

                  </Scrollbars>

</Framework>

The error occurred when using this component for the page first load or refresh. Both Mac and iPad happen.  My browser is chrome or safari, react is 16.6.1

Sorry, it cause by my code import less from 'less';


Random shutdown crash with OSS CI build

https://dev.azure.com/vscode/a4cdce18-a05c-4bb8-9476-5d07e63bfd76/_build/results?buildId=49353
Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x7fed53aac000
Process uptime: 47 seconds

Thread 0 (crashed)
 0  Electron Framework!extensions::ExtensionRegistry::RemoveObserver(extensions::ExtensionRegistryObserver*) [observer_list_internal.h : 30 + 0x0]
    rax = 0x00007fed53aac000   rdx = 0x0000000000000000
    rcx = 0x00000001108e3b90   rbx = 0x00007fed524aa778
    rsi = 0x00007fed55ee7810   rdi = 0x00007fed524aa6a0
    rbp = 0x00007ffee6783260   rsp = 0x00007ffee6783258
     r8 = 0x000000000000a662    r9 = 0x000000000000a67a
    r10 = 0x00000000fc000003   r11 = 0xfffffffffff94f70
    r12 = 0x00007fed524aa818   r13 = 0x00007fed52775620
    r14 = 0x00007fed55ee7800   r15 = 0x00007fed52775620
    rip = 0x000000010b87f8c0
    Found by: given as instruction pointer in context
 1  Electron Framework!extensions::ProcessManager::Shutdown() [process_manager.cc : 289 + 0x5]
    rbp = 0x00007ffee67832c0   rsp = 0x00007ffee6783270
    rip = 0x000000010b8a10d1
    Found by: previous frame's frame pointer
 2  Electron Framework!DependencyManager::DestroyContextServices(void*) [dependency_manager.cc : 127 + 0x9]
    rbp = 0x00007ffee6783310   rsp = 0x00007ffee67832d0
    rip = 0x000000010ce8560c
    Found by: previous frame's frame pointer
 3  Electron Framework!electron::ElectronBrowserContext::~ElectronBrowserContext() [electron_browser_context.cc : 158 + 0xb]
    rbp = 0x00007ffee6783360   rsp = 0x00007ffee6783320
    rip = 0x0000000109582b3a
    Found by: previous frame's frame pointer
 4  Electron Framework!electron::ElectronBrowserContext::~ElectronBrowserContext() [electron_browser_context.cc : 154 + 0x5]
    rbp = 0x00007ffee6783380   rsp = 0x00007ffee6783370
    rip = 0x0000000109582dbe
    Found by: previous frame's frame pointer
 5  Electron Framework!electron::api::Session::~Session() [ref_counted_delete_on_sequence.h : 70 + 0xa]
    rbp = 0x00007ffee67833e0   rsp = 0x00007ffee6783390
    rip = 0x000000010950ed43
    Found by: previous frame's frame pointer
 6  Electron Framework!electron::api::Session::~Session() [electron_api_session.cc : 294 + 0x5]
    rbp = 0x00007ffee6783400   rsp = 0x00007ffee67833f0
    rip = 0x000000010950f09e
    Found by: previous frame's frame pointer
 7  Electron Framework!electron::ElectronBrowserMainParts::PostMainMessageLoopRun() [callback.h : 98 + 0x3]
    rbp = 0x00007ffee6783450   rsp = 0x00007ffee6783410
    rip = 0x0000000109585325
    Found by: previous frame's frame pointer
 8  Electron Framework!content::BrowserMainLoop::ShutdownThreadsAndCleanUp() [browser_main_loop.cc : 1095 + 0x6]
    rbp = 0x00007ffee67834e0   rsp = 0x00007ffee6783460
    rip = 0x000000010b0b98e1
    Found by: previous frame's frame pointer
 9  Electron Framework!content::BrowserMainRunnerImpl::Shutdown() [browser_main_runner_impl.cc : 178 + 0x5]
    rbp = 0x00007ffee6783520   rsp = 0x00007ffee67834f0
    rip = 0x000000010b0bb184
    Found by: previous frame's frame pointer
10  Electron Framework!content::BrowserMain(content::MainFunctionParams const&) [browser_main.cc : 49 + 0x6]
    rbp = 0x00007ffee6783570   rsp = 0x00007ffee6783530
    rip = 0x000000010b0b6f04
    Found by: previous frame's frame pointer
11  Electron Framework!content::ContentMainRunnerImpl::RunServiceManager(content::MainFunctionParams&, bool) [content_main_runner_impl.cc : 530 + 0x5]
    rbp = 0x00007ffee67835f0   rsp = 0x00007ffee6783580
    rip = 0x000000010af60266
    Found by: previous frame's frame pointer
12  Electron Framework!content::ContentMainRunnerImpl::Run(bool) [content_main_runner_impl.cc : 879 + 0x10]
    rbp = 0x00007ffee6783660   rsp = 0x00007ffee6783600
    rip = 0x000000010af5febb
    Found by: previous frame's frame pointer
13  Electron Framework!service_manager::Main(service_manager::MainParams const&) [main.cc : 454 + 0x9]
    rbp = 0x00007ffee67839b0   rsp = 0x00007ffee6783670
    rip = 0x000000010d4e3fe8
    Found by: previous frame's frame pointer
14  Electron Framework!content::ContentMain(content::ContentMainParams const&) [content_main.cc : 19 + 0x8]
    rbp = 0x00007ffee6783a40   rsp = 0x00007ffee67839c0
    rip = 0x000000010a1e0684
    Found by: previous frame's frame pointer
15  Electron Framework!ElectronMain [electron_library_main.mm : 23 + 0x5]
    rbp = 0x00007ffee6783ad0   rsp = 0x00007ffee6783a50
    rip = 0x00000001094b0594
    Found by: previous frame's frame pointer
16  Electron!main [electron_main.cc : 274 + 0xa]
    rbp = 0x00007ffee6783b80   rsp = 0x00007ffee6783ae0
    rip = 0x000000010947b110
    Found by: previous frame's frame pointer
17  libdyld.dylib + 0x1acc9
    rbp = 0x00007ffee6783b98   rsp = 0x00007ffee6783b90
    rip = 0x00007fff6b337cc9
    Found by: previous frame's frame pointer
18  libdyld.dylib + 0x1acc9
    rbp = 0x00007ffee6783b98   rsp = 0x00007ffee6783b98
    rip = 0x00007fff6b337cc9
    Found by: stack scanning

We don't build with extensions in MS build, so this crash shouldn't bother in the product builds. But something to fix for the OSS build.


Add missing package/PDF generation

I presume you guys typically, for consistency sake, just do another normal numbered release for this, despite no Java changes? ->> Went with straight 4.0.3 bump
Related:

https://github.com/CleanSpeak/cleanspeak-app/pull/24
https://github.com/CleanSpeak/cleanspeak-load-tests/pull/5

https://github.com/CleanSpeak/cleanspeak-issues/issues/82


ClickHouse connector - sink

Pull request
Description
This pull request adds a working version of the ClickHouse sink connector.
This is still a draft, but reviews are welcome :).
Checklist

[x] The RFC, if required, has been submitted and approved
[x] Any user-facing impact of the changes is reflected in docs.tremor.rs
[x] The code is tested
[x] Use of unsafe code is reasoned about in a comment
[ ] Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
[x] The performance impact of the change is measured (see below)

Performance

After discussion with @mfelsche, we decided that this branch will be merged as is
I'll add a changelog entry at lunch time, so that the checklist is completed.


Feat/exact line merging

This branch contains a first version of a quadtree implementation for efficient line merging.
The algorithm works on most pdf's, but for some there is an error that I do not yet understand. (Basically, a point can't be added to the tree structure).
I will continue working on this after I returned.

@redur I allowed myself to try this out (installing quads manually, since it's not yet in the Conda environment file).
Initially, I ran into "max recursion depth exceeded" errors as well. They seemed to pop up rather randomly, not always on the same files. My maximum recursion depth was set to 3000, which should normally be enough for our purposes.
I'm not quite sure yet what caused this excessive recursion, but I think it might have had to do with the fact that merge_parallel_lines_quadtree got called from scratch after every individual merge. I've now replaced this with an approach that uses a queue of lines (to which the merged lines are added, so that they are also considered). Now I don't run into any recursion depth errors anymore.
I've also added some code to deal with the case when None is returned when merging two lines.
The pipeline now runs successfully, and only takes a little longer than before (ca. 10 minutes compared to ca. 8 minutes).
However, I think that there is still some bug somewhere, as a visual inspection of the lines in the PNG artifacts shows, that there are still quite a lot of lines that appear to fulfill the merge conditions, but are nevertheless drawn separately.

@redur I allowed myself to try this out (installing quads manually, since it's not yet in the Conda environment file).
Initially, I ran into "max recursion depth exceeded" errors as well. They seemed to pop up rather randomly, not always on the same files. My maximum recursion depth was set to 3000, which should normally be enough for our purposes.
I'm not quite sure yet what caused this excessive recursion, but I think it might have had to do with the fact that merge_parallel_lines_quadtree got called from scratch after every individual merge. I've now replaced this with an approach that uses a queue of lines (to which the merged lines are added, so that they are also considered). Now I don't run into any recursion depth errors anymore. I've also added some code to deal with the case when None is returned when merging two lines.
The pipeline now runs successfully, and only takes a little longer than before (ca. 10 minutes compared to ca. 8 minutes).
However, I think that there is still some bug somewhere, as a visual inspection of the lines in the PNG artifacts shows, that there are still quite a lot of lines that appear to fulfill the merge conditions, but are nevertheless drawn separately.

Thank you for working on this. I will have a look too.

Line merging using quad trees is now improved.

Recursion depth had to be put to 10'000. The problem is building of the quad tree. I am not 100% sure why the tree gets so deep.
Visual inspection now shows no obvious cases where the merge condition is met, but lines are not merged.

Furthermore, shall we remove "all other" merging functionalities to keep the code lean?

The quadtree implementation of the line merging now works as intended.
The scores on the Zurich dataset slightly decreases. I checked all cases and with the exception of one, the "new" result is not worse than the "old". The change in the scores are often in cases where lot's of mistakes happen and now the splits are slightly different. The majority of the decrease from 88.2% to 87.9% is one single case where one split is not performed any more.

Update: I pinged the maintainer of quads if he would support a contribution to remove points from a quad tree.


Snippet to dump Mac App Store apps with mas

I was missing info on how to dump apps installed via Mac App Store. If you have mas-cli, the following works to append these to a Brewfile:
mas list | awk '{ print "mas '"'"'" $2 "'"'"', id: " $1 }' >> Brewfile

Does brew bundle dump not already do this for you?

I'll check again later, but I don't think it did.


created eslint file

initialized new eslint file and set the rules to airbnb
ran tests and confirmed there were no errors.

@dhuang612 Nice work dude, this is awesome! Any chance we can please add this as a step in the travis config please?

@joshghent sure thing.
Completed and squashed extra commit.
Let me know if there is anything else that needs to be done!

noticed that this application runs with yarn, switched the travis config to yarn lint

Awesome stuff @dhuang612! Thank you so much! 🎉
Hope you enjoyed contributing to open source! There are many more issues on this repo and others that you'd be more than welcome to take your pick of!


Unloading/reloading dynamic plugins

What problem does this solve or what need does it fill?
Currently bevy supports the loading of plugins at runtime. By adding support for unloading/reloading will allow gamedevs to make changes to a module and test it without restarting the game process and resetting state. Shortening the feedback loop to test gameplay changes makes dev faster.
What solution would you like?
Add unload and reload methods to App via DynamicExt trait. I am happy to explore and flesh this out further but would like to get some feedback on the idea first. Is this even possible? How much work would it be?
What alternative(s) have you considered?
None
Additional context
None

Musl doesn't support dlclose at all and makes it a nop. Other libc's (for example macOS's libc) only allow it when TLS is not used and silently ignore dlclose otherwise. In any case it is pretty much impossible to do it safely as bevy can't enumerate al possible places a pointer into the plugin dylib is left over to santize.

Musl doesn't support dlclose at all and makes it a nop. Other libc's (for example macOS's libc) only allow it when TLS is not used and silently ignore dlclose otherwise.

I don't know much about dynamic loading at the OS level so sorry if the following comments/questions are stupid.
Isn't the dynamic lib loaded into memory of the game process. To "close" or unload it we can simply drop all the libloading::Library and libloading::Symbol instances?

In any case it is pretty much impossible to do it safely as bevy can't enumerate al possible places a pointer into the plugin dylib is left over to santize.

Correct me if I am wrong, it seems like the issue is that the way that bevy is designed, pointers to the dynamic lib ie libloading::Symbol are spread everywhere and therefore difficult to drop when trying to unload or reload a library. Hot reloading is possible to do but would require a lot of changes to bevy?

To "close" or unload it we can simply drop all the libloading::Library and libloading::Symbol instances?

No, dropping libloading::Library uses dlclose, which can't be depended upon as I said.

Hot reloading is possible to do but would require a lot of changes to bevy?

Not, just bevy, but to rust the language too. The changes to rust would cause wide spread breakage. Even something as simple as starting a thread or registering a tracing logger layer will keep pointers to the dylib. Unity can handle unloading because it is programmed in C# which uses a GC. To make a GC work, the .net runtime knows about every pointer in the whole program. Furthermore Unity uses a feature of .net to create compartments. And finally I think the .net runtime will keep dylibs in memory if there are still pointers to it pending, while still allowing further method calls to be replaced. The closest thing we can do would be to use wasm rather than native dylibs.
tl;dr: The equivalent of dlclose can only be made safe in managed languages. Rust is not a managed language. Wasm is a possiblity for reloading dynamic plugins.

To "close" or unload it we can simply drop all the libloading::Library and libloading::Symbol instances?

No, dropping libloading::Library uses dlclose, which can't be depended upon as I said.

Hot reloading is possible to do but would require a lot of changes to bevy?

Not, just bevy, but to rust the language too. The changes to rust would cause wide spread breakage. Even something as simple as starting a thread or registering a tracing logger layer will keep pointers to the dylib. Unity can handle unloading because it is programmed in C# which uses a GC. To make a GC work, the .net runtime knows about every pointer in the whole program. Furthermore Unity uses a feature of .net to create compartments. And finally I think the .net runtime will keep dylibs in memory if there are still pointers to it pending, while still allowing further method calls to be replaced. The closest thing we can do would be to use wasm rather than native dylibs.
tl;dr: The equivalent of dlclose can only be made safe in managed languages. Rust is not a managed language. Wasm is a possiblity for reloading dynamic plugins.

thanks for that explanation, really helpful : )

This current implementation of dynamic plugins are being deprecated and will be removed in Bevy 0.15, which is why this is being closed.
I do think this is a useful feature that should be considered if we ever create a new system, though.

Would webassembly be able to offer such a new system (previously discussed in thoughts on WebAssembly modding. Extism is an example of a universal plugin framework, yet I dunno if it'd support the needs of Bevy.

Would webassembly be able to offer such a new system

Perhaps, though for now I think it's best as a 3rd party plugin.


fix(table): resolve double fetch issue in useInfiniteScroll hook

Summary
Resolved the issue where the useInfiniteScroll hook was fetching data twice on initial render. This fix ensures that data is fetched only once during initial render and additional data is fetched upon scrolling.
Issue

Closes #3251.

Changes

Updated useInfiniteScroll hook to prevent double fetching.
Verified the fix by testing the component and ensuring only one initial fetch.

Closes #
📝 Description
I use isLoadingRef to prevent concurrent load calls, implements a debounced loadMore function with useCallback, and simplifies the IntersectionObserver logic.
The hook now handles both loader-based and scroll-based detection more efficiently, using debounce for scroll events. I also improved cleanup for the IntersectionObserver and better TypeScript typing. These changes collectively make the hook more robust, reducing issues like multiple simultaneous loads and excessive re-renders, while providing a smoother infinite scrolling experience
⛳ Current behavior (updates)
Table useInfiniteScroll hook fetches twice
🚀 New behavior

Updated useInfiniteScroll hook to prevent double fetching.

💣 Is this a breaking change (Yes/No):
No
📝 Additional Information

Summary by CodeRabbit

New Features

Improved infinite scroll performance with a new loadMore callback mechanism.
Added a debounce feature to prevent multiple data load requests in a short time frame.

I noticed that there's a test failure related to the NextUI.Input component inside an Autocomplete component. However, my changes were only to the useInfiniteScroll hook and didn't touch these components. This seems to be a pre-existing issue in the test suite. Could someone from the maintenance team please advise on how to proceed?


wrong snipet for adding bit script

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
the script tag is not self-closing while in the guide to add the blazer-UI, it is self-closing.

Expected Behavior
it should be like this:

Steps To Reproduce
No response
Exceptions (if any)
No response
.NET Version
No response
Anything else?
No response

The issue is accepted and planned. Resolving it will start ASAP.

Resolving this issue has started. It will be announced when this issue is resolved.

This issue has been resolved and it will be available in the next pre-release version. It will be announced when the pre-release becomes ready.

The pre-release has been published. Please give it a try and give feedback


Feature Request: Custom Shortcut Commands for Frequent Prompts

Description
I'm a regular user of Copilot Chat, and I find myself frequently using certain prompts. To enhance efficiency and user experience, I propose custom shortcut commands, similar to the existing ones like /tests and /fix. This feature would allow users to create their own shortcuts, like /xxx, which are tied to specific prompts they use often.
Proposed Feature

Custom Shortcut Commands: Users can set custom shortcuts for their frequently used prompts.
Settings Format: The setting for this feature could follow a format similar to: custom_prompt::{"shortcut":"custom_prompt_text"}. For example, if a user wants to create a shortcut for a review prompt, they could set it as custom_prompt::{"review":"Please review the following code"}.
Ease of Use: This would significantly streamline the workflow for users who rely on specific prompts regularly.
Customization: It allows for a more personalized and efficient use of the Copilot Chat interface.

I believe this feature would be a valuable addition to Copilot Chat, making it even more user-friendly and efficient for regular tasks. Thank you for considering this request.

What I typically do is /, type a few characters, hit tab. It can be very quick this way

It could be helpful for certain code commenting style. For instance I would like to have a possibility to design custom prompt with some static data eq:
########
Author: Doe, John
Date: 2024-01-01
Description:
########
Is there any chance I could do that in the near future?


Create initial database in background

To prevent slowdowns when loading the app, run the database migrations via dispatch_async instead of directly in the app delegate startup

write a  demo ?


add configurable prompt symbol

Thanks for the superfast prompt! It works faster than https://starship.rs 👍
One thing that was missing for me is the configurable prompt symbol, because my font is actually missing the ❯ symbol.

Looks good. Thank you!


update to image-spec rc7

updated to spec v1.0.0-rc7.
Signed-off-by: zhouhao<EMAIL_ADDRESS>

ping @opencontainers/image-tools-maintainers

LGTM.

LGTM


Fix #353 #355 #366

Address issue #353, PRs #355, #366
Regenerate IR stubs using the fixed constexpr generator

Also addresses #367 (dup of #353)

:+1: Looks good to me


Impro 1783 part2

Addresses #GitHubissuenum
Description:
Updated plugin for field texture along with CLI and associated acceptance tests.
Testing:

[X] Ran tests and they passed OK
[X] Added new tests for the new feature(s)

CLA

[ ] If a new developer, signed up to CLA

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

@Kat-90 you need to update your acceptance test data repository and regenerate the checksum file again.  It's missing a lot of recently added data entries.

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

Complexity increasing per file
==============================
- improver_tests/test_field_texture.py  1
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/test_field_texture.py  1
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

May be worth a rebase against latest master, dropping the commits that have already gone in?

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- improver_tests/acceptance/test_field_texture.py  1
- improver/cli/field_texture.py  1

See the complete overview on Codacy


Missing method implementation for Enum.each_value

Describe the bug
This is not a bug but a "nice to have". Given an enum type like MyEnum = Types::String.enum('foo', 'bar'), when I want to iterate over the values, I'm doing it with MyEnum.values.each, but it would be nice if the Enum type had the each_value method similar to a Hash object.
To Reproduce
Run the following lines in an IRB session with the dry-types gem installed:
require 'dry-types'

MyEnum = Types::String.enum('foo', 'bar')
MyEnum.each_value { |value| puts value } # Throws a NoMethodError

Expected behavior
It would be nice if Dry::Types::Enum mimics the interface of a Hash object and implements the each_value method.
My environment

Affects my production application: YES
Ruby version: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [arm64-darwin21]
OS: macOS 13.4.1

@edgarjs I have made a PR that implements this method here #471 :+1:


Feature/PYPE-763 get path returns none

Current get_representation_path use 3 ways how to get path, with path_from_representation, path_from_config and path_from_data.
Issue is that they do not stop if path was formatted but not found in path_from_representation and path_from_config. Since that time method should end because there is not much possibility that other ways are working (There is something wrong if yes).

This is the same problem that's already being solved by upstream #424
Let's fix that code instead, as it is waiting for some minor tweaks only. Let's not diverge in small things.


Initial jupyter notebook segmentation looks different from GUI; then self-corrects on second run

Hello--first, have to say cellpose 2.0 is amazing, and the ability to run the "model zoo" to find an optimal solution to a specific problem is a huge improvement compared to prior segmentation solutions.
Through the GUI, I identified model 'CPx' as working for my segmentation problem out-of-the-box. However, when I went run the same model in a Python / jupyter lab environment (for batch run purposes), I am seeing a very different output. I am not sure how to reconcile the differences or where the issue is.
The jupyter lab snippet is here:
# RUN CELLPOSE
from cellpose import models, io

master_diam=90

# DEFINE CELLPOSE MODEL
model = models.CellposeModel(model_type='CPx',diam_mean=master_diam)
imgs = [io.imread(filename) for filename in files]
masks, flows, styles = model.eval(imgs, diameter=master_diam, channels=channels)

diams=[master_diam]
io.masks_flows_to_seg(imgs, masks, flows, diams, files, channels)
io.save_to_png(imgs, masks, flows, files)

cp segmentation masks

expected (correct) output from GUI run segmentation

EDIT 1. OK now it's working. No idea what I changed.
EDIT 2. OK figured it out, this seems to happen only for the first run in jupyter lab. Running the exact same code a second time produces the correct output. Have seen this both with 'CPx' and a pretrained model; initial output is totally different, then somehow self corrects for the second run.

this is indeed strange, can you please turn on the logger in jupyter and send the output?
turn on logger with logger = io.logger_setup()

closing due to inactivity, please let us know if you have any feedback


Presentation vs. viewer pane

Thank you for sharing this terrific template with many examples.
Can I trouble you with a question please?
When I have a document and format: revealjs, render shows the output in the Rstudio presentation pane, which is very convenient.
However, when I switch to format: metropolis-revealjs, the document renders in the viewer pane.
Do you know if it would be possible to keep the presentation pane instead?
I tried mocking around with this, but so far, no luck... hence my reaching out to you dear Shafayet.
thank you again.
thomas

As far as I have seen, If you render a quarto document, It shows rendered output directly in the browser. I am actually confused by the line "When I have a document and format: revealjs, render shows the output in the Rstudio presentation pane"!
Can you add more details regarding this?

most definitely - please kindly take a look at my post here. I don't think this is an issue on your end, but more on the Rstudio side.
Since I have your attention, can I ask you a newbie question en passant please? Your template works great, especially with level 2 slides, but when i change to level 3 slides (slide-level: 3 in the yml section), I do not feel like I am getting the same nice layout anymore, in particular slide heading fonts are not differentiated and even turn into small text... Does it mean i need to go mess with the .csss file or is there another easier solution? sorry for bothering you with this pedestrian question. much appreciated your help.

Would You kindly open another issue regarding the problem you have stated with a minimal reproducible example and a screenshot of the problem? That would really help me!

allow me not to open another issue - it's really the same. However, do allow me to do everything you ask for, here.

file -> new file -> quarto presentation

click the render button, the file compiles fine, of course, and shows up in the presentation pane of the output window:

now change the format from revealjs to metropolis-revealjs, hit render, and now the slides show up in the Viewer pane of the output window.

NOTE: I skip the part where I have to install your package, as it's not relevant. Your pacakge works just fine. That's not the issue here.
Hopefully this clears it up - let me know if not.


Docs for autodiff and custom loss

@FedyuninV proofreading the tutorial here as well, have a look, please. Minor wording only.


[libra-dev] Add function to create different transaction scripts

Motivation
Adding function in libra-dev to create different transaction scripts. Refactored signed transaction function to take in bytes of script object rather than individual parameters.
Test Plan
cargo x test -p libra-dev

looks great, for completeness pls add an test function for libra_ScriptP2P_from() as well.

@bors-libra r=thefallentree

:pushpin: Commit add411d has been approved by thefallentree

@bors-libra r=thefallentree

:bulb: This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: #4199

:pushpin: Commit add411d has been approved by thefallentree

@bors-libra retry

@bors-libra r=thefallentree

:pushpin: Commit add411d has been approved by thefallentree

:bulb: This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: #4204

:bulb: This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: #4204

:pushpin: Commit add411d has been approved by thefallentree

:hourglass: Testing commit add411db0d09c8b5734f3dacca89e43762d31c68 with merge 72a12ed7e1965c4d65d9b691739ff5e0135dfe9a...

:broken_heart: Test failed - checks-circle_commit_workflow

@bors-libra retry

@bors-libra r=thefallentree

:bulb: This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: #4212

:pushpin: Commit add411d has been approved by thefallentree

:hourglass: Testing commit add411db0d09c8b5734f3dacca89e43762d31c68 with merge 161cbf6ab8f27c89e9f03d1e0082e0ea750ba1f8...

:sunny: Test successful - checks-actions_land_blocking_test, checks-circle_commit_workflow
Approved by: thefallentree
Pushing 161cbf6ab8f27c89e9f03d1e0082e0ea750ba1f8 to master...


GDScript: Remove unused GDScriptParser::DataType::method_info field

This reduces the GDScriptParser::DataType size by 112 bytes, from 240 to 128.

While not used at the moment, this would be the best way to implement argument hints on Callable.call.
I swear I saw an issue for this some time ago, can't find it right now though.
If only my PC would screenshot everything I do, so that I could search for it with advanced AI technology /s

So here is a draft which actually introduces a usage of the field: #96375


Remove .NET 7 docs

Remove the API docs for .NET MAUI 7, which is now out of support.

This LGTM. The build report is initially concerning with its talk of deleting files without redirection (e.g. File docs/xml/Microsoft.Maui.Controls/Editor.xml with URL /dotnet/api/microsoft.maui.controls.editor.cursorposition was deleted without redirection. To avoid broken links, add a redirection.). But when I viewed the content on the preview site, it was all fine. On closer of examination of the build report the deletions are fine because the files haven't been deleted at all - sub-portions of the files have been deleted because of type hierarchies having changed (e.g. Editor.CursorPosition has changed to InputView.CursorPosition).

Note: Deprecate the .NET 7 moniker in OPS after merging this PR.


rbac: remove the deprecated RBAC mixer adapter

The Mixer RBAC adapter is deprecated about one year ago for 1.1 (https://github.com/istio/istio/pull/7961), remove it completely for 1.3.
Fixes https://github.com/istio/istio/issues/6377
Please provide a description for what this PR is for.
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[X] Policies and Telemetry
[X] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: yangminzhu
To complete the pull request process, please assign mandarjog
You can assign the PR to them by writing /assign @mandarjog in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

mixer/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yangminzhu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-unit-tests-master
1452ad8ffde69830552c60ad6038b5b455455965
link
/test istio-unit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@yangminzhu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-unit-tests-master
1452ad8ffde69830552c60ad6038b5b455455965
link
/test istio-unit-tests-master

integ-mixer-k8s-presubmit-tests-master
1452ad8ffde69830552c60ad6038b5b455455965
link
/test integ-mixer-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-unit-tests-master
/test integ-mixer-k8s-presubmit-tests-master

@mandarjog @douglas-reid @liminw @geeknoid The PR is ready for review, PTAL, thank you!

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: liminw, yangminzhu
To complete the pull request process, please assign mandarjog
You can assign the PR to them by writing /assign @mandarjog in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

mixer/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@geeknoid Would you mind to review this PR? Thank you.

/approve
/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: geeknoid, liminw, yangminzhu
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS [geeknoid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yangminzhu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-racetest-master
1452ad8ffde69830552c60ad6038b5b455455965
link
/test istio-racetest-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-racetest-master

/test istio-racetest-master

@yangminzhu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-racetest-master
1452ad8ffde69830552c60ad6038b5b455455965
link
/test istio-racetest-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-racetest-master


Request MTU

Hi!
First of all, thank you for this lib, very helpful! I only have one question.
Can I increase somehow the MTU size?
Bests,
Balazs

Hi @Hulcsa !
Thanks for the kind words!
This has not been included because we didn't need it and I wasn't certain how to expose it properly.
However, this is definitely something we want to add.
The doc here states that the MTU changed callback can happen after a connection and after a change request. Consequently, I'm not sure how we should expose it.
Could you tell me more about your use case and more importantly, the order of the operations (you connect, then what do you, when you want to change MTU, if you can change it multiple times, if you need to know current MTU first…)?

Well, I only need to set it once before any communication. If i could find out what is the current MTU that would be nice too.

Ok, so in next version, this will be added.
The MTU value will be cached so you can it at any time

I won't be able to put this in the next version as I wrote previously, because my tests didn't exhibit the documented behavior regarding the onMtuChanged advertised to be called after a connection change.
I may provide an experimental API after Kotlin 1.3.10 migration though.


Update feed.njk

👷 Deploy Preview for startling-cocada-e45985 processing.

Name
Link

🔨 Latest commit
d6f9ad8ea3c5d63a8e0f9a64e863990bd4ffd82d

🔍 Latest deploy log
https://app.netlify.com/sites/startling-cocada-e45985/deploys/643311980f8ba600088be693


Sneaking on an airship does not let you walk.

Tested in jenkins 460.
Enable physics for a ship and sneak.
You will notice that you cannot move.
ples fix

Try clicking shift

10 million iq
if i want to build on an airship for example i would want to sneak and walk at the same time as to not fall off

fixed


Use lockfile to prefill resolver index

Summary
Use the lockfile to prefill the InMemoryIndex used by the resolver. This enables us to resolve completely from the lockfile without making any network requests/builds if the requirements are unchanged. It also means that if new requirements are added we can still avoid most I/O during resolution, partially addressing https://github.com/astral-sh/uv/issues/3925.
The main limitation of this PR is that resolution from the lockfile can fail if new versions are requested that are not present in the lockfile, in which case we have to perform a fresh resolution. Fixing this would likely require lazy version/metadata requests by VersionMap (this is different from the lazy parsing we do, the list of versions in a VersionMap is currently immutable).
Resolves https://github.com/astral-sh/uv/issues/3892.
Test Plan
Added a deterministic! macro that ensures that a resolve from the lockfile and a clean resolve result in the same lockfile output for all our current tests.

One thing I'm wondering if any of the metadata stored in the lockfile is potentially mutable, meaning resolution would fail in the presence of a lockfile.

One thing I'm wondering if any of the metadata stored in the lockfile is potentially mutable, meaning resolution would fail in the presence of a lockfile. e.g. we probably can't rely on the locked metadata of path dependencies, and maybe also direct URLs?

Yeah, this is the right instinct. We need to check if they're up-to-date... Take a look at RequirementSatisfaction which performs these checks when (e.g.) determining whether we can use a value from the cache.
In general, we assume that HTTP direct URL requirements are immutable and thus require --upgrade or --refresh or similar. For file-based direct URL requirements, the rules are a little more nuanced.

On the airflow benchmark, which is a very large lockfile, this speeds up uv lock  by 2x:
$ hyperfine "../uv/target/profiling/baseline lock" "../uv/target/profiling/uv lock"
Benchmark 1: ../uv/target/profiling/baseline lock
  Time (mean ± σ):     183.6 ms ±   4.6 ms    [User: 199.8 ms, System: 124.6 ms]
  Range (min … max):   176.1 ms … 190.4 ms    15 runs

Benchmark 2: ../uv/target/profiling/uv lock
  Time (mean ± σ):      93.6 ms ±   2.0 ms    [User: 78.9 ms, System: 16.2 ms]
  Range (min … max):    90.5 ms …  99.1 ms    31 runs

Summary
  ../uv/target/profiling/uv lock ran
    1.96 ± 0.06 times faster than ../uv/target/profiling/baseline lock

From a user perspective, 100ms feels relatively instant while I can feel some lag at 200ms. There is probably still room for some improvement here.
The big win here is that with a cold cache, uv lock can still run in 100ms due to the presence of a lockfile, where it would previously take over 30 seconds.

The fact that you can resolve so quickly with a cold cache is crucial.

What if we intentionally run the resolver in Offline mode for this?

One remaining issue is that we don't check for yanked releases (see https://github.com/astral-sh/uv/issues/3892#issuecomment-2212528949). I'm not sure how we should tackle this, does PyPI have a fast-path to check if a specific version was yanked? It will likely still result in significant overhead.

Missing extras/development groups are detected and require a fresh resolution. This unfortunately means that projects with extras that don't have optional dependencies, such as requests[security], always require a fresh resolve. The solution to this is either to add extra information to the lockfile or eventually add per-dependency fallbacks in the resolver to fetch up-to-date metadata. See the passing lock_new_extras test.

Can you explain this piece in a little more detail? What are "projects with extras that don't have optional dependencies"? Does security not exist as a valid extra for requests?

This is looking good.

Can you explain this piece in a little more detail? What are "projects with extras that don't have optional dependencies"? Does security not exist as a valid extra for requests?

Ah I think I was confused about this. security is an extra provided by requests, but it doesn't enable any optional dependencies so it doesn't show up in the lockfile. Turns out it is deprecated so I don't think this is a problem. I was thinking that some packages might provide extras that enable features dynamically without introducing optional dependencies (like cargo features) but I don't think that's expected (or even possible?)?

Ohhh ok, I see. Yes, this makes sense.

I was thinking that some packages might provide extras that enable features dynamically without introducing optional dependencies (like cargo features) but I don't think that's expected (or even possible)?

Yeah I believe this is not possible. AIUI, extras in Python disappear after dependencies are resolved. They aren't themselves directly visible from the package that defines them, unlike Cargo features, which can be used in arbitrary ways.


"You modified ... twice in a single render" deprecation warnings

Running on Ember 1.13, I get the following deprecation warnings about modifying a property twice in a single render:

graphHeight due to xAxis
graphWidth, due to yAxis
[d] for unknown reasons

I tried changing some of the components' init hooks to didInitAttrs, but that breaks because their parent components expect them to exist during initialization.
See also emberjs/ember.js#11156

Brace yourself, this will be redundant but it's worth laying out exactly what's happening.
The nf-graph renders, yields to it's block. In that block, we have our nf-(xy)-axis, when that axis is initialized it sets itself on the nf-graph component. The nf-graph component has a computed property for graphHeight and graphWidth. One of the dependentKeys is the height of the nf-(xy)-axis. The timing of this, is such that, when the nf-(xy)-axis is initialized, the nf-graph is mid-render. The graphHeight recomputes, and the height={{graphHeight}} in the nf-graph template changes, raising the deprecation warning.

Yeah, this is a biproduct of having the scales contained at the nf-graph level, and not defined by the nf-(xy)-axis components, which is probably where they should be.
I'm open to any solution that can be provided. We're still stuck on Ember 1.11 - 1.12 on our apps on my team at Netflix because of other deprecation issues and breaking changes, so I haven't seen this since I tried (and failed) to upgrade to 1.13.


refactor SGD, part 3: factor out more sparse/dense common code

Still more SGD refactoring. -55 lines of code.

Not mergeable :-(

Rebased and force-pushed a new version + a commit that removes another five lines.

Wait a moment, just saw your comment about y; will push one more commit.

Alright, ready!


Is it working on IOS?

Hello. Is it support IOS build?

Yes epic online services supports IOS but I have not configured it as I don't have a way to test it.

Thank you

Should we use Native platrofm SDKs ? Why we just can't use C SDK on all platforms (including mobile) ?

We don't have access to the source code for EOS for IOS. Epic Online services only gives us the compiled native library to be used on the platform.

I will try to get an example for using EOSG plugin with IOS, it will need modifying the Godot xcode project a bit.

Thanks


Fix RejectExecutionHandler of Blocking Single Threaded executor

Fixes #16783.

Description
As discussed in issue https://github.com/apache/druid/issues/16783 ,  there are times when intermediateTempExecutor thread get stuck in WAITING state and the executor is not able to terminate. This causes the main task to get stuck in PUBLISHING state forever, until we kill it or pendingCompletionTimeout interrupts the thread.
The underlying cause of this issue is that this intermediateTempExecutor is implemented using newBlockingSingleThreaded, which created a thread pool executor with maxCoreSize: 1 and a SynchronousQueue.
The rejectedExecutionHandler of this ThreadExecutor should not add tasks when it's in shutDown state because, a SynchronousQueue is used and if a task is put() in it in ShutDown state, it will forever stuck in WAITING state as executor will not take() the task to schedule it. In a SynchronousQueue there should always be a take() WAITING for every put() operation and vice-versa.
In the issue, I have added thread dumps too that confirm tasks being stuck during rejectedExecution() when task is added in queue during a shutDown state. The Unit test confirms the thread getting stuck.
In StreamAppenderator, it happens when  there is a race condition here
        Futures.allAsList(publishWaitList).cancel(true);
        Futures.allAsList(handOffWaitList).cancel(true);
        if (appenderator != null) {
          appenderator.closeNow();
        }

We don't wait for appenderator.drop() to be interrupted in handOffWaitList and appenderator.closeNow() shuts down the exeuctors. Thus a new task is sometimes added in intermediateTempExecutor through appenderator.drop() while executor is already shutDown.

Fixed the bug ...
https://github.com/apache/druid/issues/16783
Renamed the class ...
Added a forbidden-apis entry ...

Release note

Key changed/added classes in this PR

MyFoo
OurBar
TheirBaz

This PR has:

[ ] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

@kfaraz . Thanks for the review. I've addressed all your comments

Hey @kfaraz, 1 compaction-integration test and 1 perfect-rollup-parallel-batch-index integration test failing with same error.
Are the changes related? Looks like a flaky test to me. Can we re-run these two checks?
org.apache.druid.java.util.common.ISE: Error while making request to indexer [404 Not Found No task reports were found for this task. The task may not exist, or it may not have completed yet.]
	at org.apache.druid.tests.coordinator.duty.ITAutoCompactionTest.testAutoCompactionDutyCanUpdateCompactionConfig(ITAutoCompactionTest.java:716)

org.apache.druid.java.util.common.ISE: Error while making request to indexer [404 Not Found No task reports were found for this task. The task may not exist, or it may not have completed yet.]
	at org.apache.druid.tests.indexer.ITPerfectRollupParallelIndexTest.testIndexData(ITPerfectRollupParallelIndexTest.java:127)

@kfaraz Thanks! Can we merge this now?


strike_expandableSection component

Salesforce added a lightning:accordion and lightning:accordionSection in Winter 18. However, these do now allow for a single section to be expanded/collapsed. It would be great if we also had an implementation of the expandable section.

at the moment we don't have this on our radar. however, we will be adding a fiddler for the accordion soon. closing for now with the possibility that we'll implement such a component in the future.

No worries. Our team is considering developing this component for our internal use. If we choose to do so do you think this aligns with what Strike is trying to offer? (If so I would open a PR).

@dsharrison first off thanks for using Strike and your support. Since Strike came out of our need for components in our client work that is always our test for adding a new component. As of now we have not come across any use cases that need a more sophisticated accordion than for our clients than what Salesforce's base accordion component offers (not to say this may not happen in the future). We have not done PRs on entire components to-date because we are trying to be very conscious of the time required to maintain the existing set. The components we have written all follow similar patterns which helps when maintaining them. Our main fear around doing a PR on a component that we have not written is that it would just be different enough from the rest that it would make maintaining the whole set more complicated. We would be happy to consider your PR but you may try and pattern is after other components we have written to increase the likely hood of us being able to pull it in. Hope that helps.


Fix drop (v) operator

So as to keep, instead of remove, the first n sorted values

Thanks 😄

Could the pypi package be updated with this fix?

Released as version 1.1.1.