Add field-selector option for kubectl top pod

What would you like to be added:
Adding option --field-selector to kubectl top pod, which is similar to kubectl get pod
Why is this needed:
I need to get utilization statistic for pods in a specific node. This can be helped by using --field-selector=spec.nodeName=<...>

/priority backlog

Adding this is at the thin line between feature request and compatibility with other kubectl commands. So I'm going treat this as a the latter. When working on it make sure to use ResourceBuilder like other command. This will ensure all the mechanics used in other command will be available for you for free.

@soltysh I would like to work on this if it is available.
/assign @AnishShah

@soltysh I would like to work on this if it is available.

Sure go ahead.

/assign

/remove-lifecycle rotten

/remove-lifecycle stale


Publish give MSB3030 error when wwwroot/dist assets are deleted and recreated as part of MSBuild Exec task

I have custom webpack configuration in my project, which generate javascript bundles in wwwroot/dist folder when running project under dev mode. Here is how it looks:

While these dev assets are still in wwwroot/dist directory and we do dotnet publish or publish from visual studio IDE. There is a custom task as configured in angular 2 spa template csproj file.
 <Target Name="RunWebpack" AfterTargets="ComputeFilesToPublish">
    <!-- As part of publishing, ensure the JS resources are freshly built in production mode -->
    <!--<Exec Command="npm install" />-->
    <Exec Command="npm run build:prod" />

    <!-- Include the newly-built files in the publish output -->
    <ItemGroup>
      <DistFiles Include="wwwroot\dist\**\**;" />
      <ResolvedFileToPublish Include="@(DistFiles->'%(FullPath)')" Exclude="@(ResolvedFileToPublish)">
        <RelativePath>%(DistFiles.Identity)</RelativePath>
        <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
      </ResolvedFileToPublish>
    </ItemGroup>
  </Target>

npm run build:prod is an npm script which cleans wwwroot/dist directory and then creates minified assets, which look like this:

However, as soon as npm run build:prod finishes, I get following error:

It looks like the files are read by MSBuild and there is a race conditions between MSBuild and its Exec task in a sense that MSBuild doesn't wait for the Exec task to finish first.
This project is upgraded from project.json based architecture and we never had this issues as there was pre-publish task which finished first and give dotnet publish chance to pickup latest wwwroot/dist files.
Not sure if https://github.com/aspnet/JavaScriptServices/issues/735 is linked to this as well.

I suspect that, rather than being a race condition, it might be something more like the WebSDK publish task being set up in such a way that it collects references to the files to be published first, then runs your <Exec> task, and then tries to copy some files that no longer exist. But I don't know for sure - you'd have to read through how the WebSDK MSBuild code works internally. It's possible that your current custom setup just doesn't work with the dynamic ResolvedFileToPublish replacement in some way.
It's likely that this issue could be reproduced on a simple ASP.NET project without using JavaScriptServices templates, since the only part relevant to your report is the small snippet of MSBuild config you posted. If you think there's a bug in the WebSDK, could you please report it to its issue list at https://github.com/aspnet/websdk/issues?
I'll close this issue here because it's certainly not something controlled in the JavaScriptServices repo code. Hope that's OK!

@SteveSandersonMS Thanks.
This has fixed it:
https://github.com/aspnet/websdk/issues/162#issuecomment-288850434
Here is my updated csproj.
https://github.com/asadsahi/AspNetCoreSpa/blob/master/AspNetCoreSpa.csproj


(GH-10) Add GitHub actions to exercise unit tests

Fixes #10
In commit 80100e7c unit tests were added.  This commit adds a GitHub Actions
file which will run the tests on merge to prod and any Pull Requests against
the prod branch.

Example successful run - https://github.com/glennsarti/localhost-proxy/actions/runs/297227644

poke

Been enough time.  Closing.


feat(minErr):  set max depth for angular error JSON stringify

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
feature
What is the current behavior? (You can also link to an open issue here)
https://github.com/angular/angular.js/issues/15402
What is the new behavior (if this is a feature change)?
Does this PR introduce a breaking change?
Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

Other information:
closes #15402

Wouldn't it be simpler and more performant just to provide a specialized replacer function to the JSON.stringify method, which tracked depth? Or is that not possible?

I couldn't think of a way to track depth from jsonReplacer (in a reasonable way) :(

Yeah, fair enough.
This project https://github.com/Canop/JSON.prune ended up having to rewrite JSON.stringify completely!

@gkalpak
I have added some unit tests and we can configure the max depth by passing it in the placeholder
var exampleMinErr = minErr('example');
throw exampleMinErr('one', 'This {0,maxDepth=2}', foo);

Could please give this PR another look ?

Thanks for your feedback 😄
I have done all the changes that you had requested.
Please tell me if there is something else missing.

@gkalpak
I prefer your suggestion about using errorHandlingConfig is a getter and setter,
I have changed the implementation to
function errorHandlingConfig(config) {
if (!isObject(config)) {
return {
objectMaxDepth: objectMaxDepthInErrorMessage
};
} else {
if (isDefined(config.objectMaxDepth)) {
objectMaxDepthInErrorMessage = isValidObjectMaxDepth(config.objectMaxDepth) ? config.objectMaxDepth : NaN;
}
}
}
so it will work as a setter when config (object type ) is supplied.
please tell me if there anything missing.

@petebacondarwin
@gkalpak
I have change it, it is now identical to the docs of 'angular.bootstrap' config in the identation
please tell me if it needs to change ?

@gkalpak
I did all the changes that you had requested.

Changed

var objectMaxDepthInErrorMessage = 5;

To be
var minErrConfig = {
  objectMaxDepth: 5
};

Removed maxDepth from copy docs

Regarding my earlier comment, i turns out that we don't need to change ng-closure-runner/minErr, because the change in this PR are not applicable there.


refactor: remove unused context attribute from conn

The existing context attribute of conn was never used.

Thanks


Map/feature/interaction props

This PR changes the way interactions (draw and select) are added to the EOxMap.
Interactions are now part of the EOxLayer-Definition, just like the source or the style. This represents most real-world applications, as interactions usually are linked to a specific layer, especially the implemented draw and select (including hover)-interactions. The interactions are generated as part of the layer generation, thus guaranteeing that the layer already exists.
This also means that the addDraw and addSelect-Methods became obsolete, and so is the hassle of having to wait for the initial render until interactions can be set. Therefore the play-functions in the storybook are not needed anymore.
I did not succeed in making the layers-attribute interactive and forcing re-renders of the component with the new arguments, but minor documentation improvements as well as a fix for the ab-compare have been added as part of this PR.

In general I like that addSelect and removeSelect do not longer have to be handled by the parent application. Some comments are detailed below.
Current Usage
Currently we're doing something like this for both select and hover interactions:
      const eoxmap = this.getEoxMap();
      if (Object.hasOwn(eoxmap.selectInteractions, "selectInteraction"))
        eoxmap.removeSelect("selectInteraction");
      document.querySelector("eox-map").addSelect(this.selectedLayer.id, {
        id: "selectInteraction",
        condition: "singleclick",
        idProperty: "id",
        style: {
          "stroke-color": "white",
          "stroke-width": 3,
        },
      });

Configuring Interactions
The new verbose form of configuring interactions contains details that should be hidden from the parent application. In particular the type and source of the interaction layer are too much detail:
https://github.com/EOX-A/EOxElements/blob/9a746dc90330db7220f148c821cb040351e3f790/elements/map/map.stories.js#L264-L283
I see that already a shorthand configuration has been added to mitigate this:
https://github.com/EOX-A/EOxElements/blob/9a746dc90330db7220f148c821cb040351e3f790/elements/map/src/select.ts#L86-L97
Removing Interactions
When switching pages and setting new layers with interactions to the map, we must remove the previously added interaction beforehand:
      if (Object.hasOwn(eoxmap.selectInteractions, "selectInteraction"))
        eoxmap.removeSelect("selectInteraction");

I see that removeInteraction is untouched by this PR. I think that it would be consistent if the attached interaction was also removed automatically when a layer is removed from the map.

I see that removeInteraction is untouched by this PR. I think that it would be consistent if the attached interaction was also removed automatically when a layer is removed from the map.

This is now tracked in https://github.com/EOX-A/EOxElements/issues/471.


ERROR: BLOB longer than INT_MAX bytes

Describe the bug
I am trying to download Ghost in the Shell, but even before the first file starts downloading I get an error:
anime: ./ghost-in-the-shell/ghost-in-the-shell_1.mp4
INFO:anime_downloader.downloader.base_downloader:./ghost-in-the-shell/ghost-in-the-shell_1.mp4

ERROR: BLOB longer than INT_MAX bytes

To reproduce
anime -ll DEBUG dl 'Ghost' --choice 2

2020-05-28 21:19:24 desktop anime_downloader.session[1196] DEBUG uncached request
DEBUG:anime_downloader.session:uncached request
send: b'GET /anime/ghostintheshell/%5Bbonkai77%5D.Ghost.in.the.Shell.(1995).%5BBD.1080p.Dual.Audio.x265.HEVC.10bit%5D.mp4 HTTP/1.1\r\nHost: edge-32.cdn.bunny.sh\r\nuser-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) Gecko/20100101Firefox/56.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nreferer: https://twist.moe/anime/ghostintheshell/[bonkai77].Ghost.in.the.Shell.(1995).[BD.1080p.Dual.Audio.x265.HEVC.10bit].mp4\r\nCookie: __cfduid=d732937cae412e05951d1302b2d8863b41590711526\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: nginx
header: Date: Fri, 29 May 2020 00:18:47 GMT
header: Content-Type: video/mp4
header: Content-Length:<PHONE_NUMBER>
header: Connection: keep-alive
header: Last-Modified: Thu, 09 Aug 2018 22:51:29 GMT
header: ETag: "5b6cc571-835409c0"
header: X-Proxy-Cache: HIT
header: X-Proxy-Serve: EU-CDN
header: Accept-Ranges: bytes
2020-05-28 21:19:25 desktop anime_downloader.session[1196] DEBUG uncached request
DEBUG:anime_downloader.session:uncached request

Traceback (most recent call last):
  File "/home/tzulberti/envs/downloader/bin/anime", line 8, in <module>
    sys.exit(main())
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/anime_downloader/cli.py", line 53, in main
    cli()
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/anime_downloader/commands/dl.py", line 139, in command
    range_size=chunk_size)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/anime_downloader/sites/anime.py", line 385, in download
    downloader.download()
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/anime_downloader/downloader/base_downloader.py", line 55, in download
    self.check_if_exists()
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/anime_downloader/downloader/base_downloader.py", line 35, in check_if_exists
    self.source.stream_url, headers=headers, stream=True)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/core.py", line 136, in request
    **kwargs
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/core.py", line 109, in send
    return send_request_and_cache_response()
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/core.py", line 97, in send_request_and_cache_response
    response = super(CachedSession, self).send(request, **kwargs)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests/sessions.py", line 665, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests/sessions.py", line 665, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests/sessions.py", line 245, in resolve_redirects
    **adapter_kwargs
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/core.py", line 109, in send
    return send_request_and_cache_response()
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/core.py", line 99, in send_request_and_cache_response
    self.cache.save_response(cache_key, response)
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/backends/base.py", line 46, in save_response
    self.responses[key] = self.reduce_response(response), datetime.utcnow()
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/backends/storage/dbdict.py", line 166, in __setitem__
    sqlite.Binary(pickle.dumps(item)))
  File "/home/tzulberti/envs/downloader/lib/python3.6/site-packages/requests_cache/backends/storage/dbdict.py", line 130, in __setitem__
    self.table_name, (key, item))
OverflowError: BLOB longer than INT_MAX bytes

pd: This is an amazing app

I'm guessing this happens when the internal downloader encounters a file larger than the 32 bit integer limit. aria2 most likely handles this, are you using it?

Looks like this is a problem with requests cache.

We should have a flag to disable it.

@tzulberti does this error still occur for you?

I was unable to replicate it


[2017.2] Updates 2017.2 branch to actually 2017.2

Checked out this pull request with 2017.2 0b4. The current version in the 2017.2 branch will not work with 2017.20b2 or 2017.20b4. This pull request seems to fix all of those issues. While I have not pushed anything to a WMR device as of yet, everything works smoothly in the editor.


[Explore]: Can we avoid downtime when repushing apps

Background
In CF for VMs when a new version of the app is pushed, there is a short downtime period, even if the app has been scaled to more then one instances in advance.
However, in Korifi we could leverage the K8S pod disruption budget, and on a repush, instead of recreating the statefulset, we could just update its image. Then K8S would take care to gradually reschedule the statefulset pods so that no downtime occurs.
Acceptance Criteria
Proof of concept (and a proposal if controversial) whether updating the statefulset on repush makes sense and does that work.
Timebox
No timebox
Dev Notes
No response

Analysis of current behaviour
From the bottom up...
Statefulset-runner
AppWorkload Controller
This reconciles AppWorkloads with Statefulsets. A Statefulset is created for each AppWorkload. The Statefulset name is constructed from the AppWorkload GUID and Version. The AppWorkload is set to own the Statefulset.
Korifi Controllers
CFProcess Controller
This reconciles CFProcesses with AppWorkloads. It creates AppWorkloads owned by the CFProcesses. The name is constructed from a hash including the CFApp's app-rev, stored in an annotation on the CFApp. Thus we have a new AppWorkload each time the CFApp's app-rev changes.
The reconciler also deletes all AppWorkloads which do not match the current app-rev. And if app's desired state is not STARTED or the CFProcess number of desired instances is zero, all AppWorkloads for the process are deleted, including the one for the current app-rev.
CFApp Webhook
When a CFApp's DesiredState is changed from STARTED to STOPPED, its app-rev is incremented.
What happens during cf push
On cf push, a new CFPackage is created and source uploaded against it. Then a new CFBuild is created associated with the CFPackage. This triggers a BuildWorkload creation, and eventually a droplet reference bubbles up and is stored on the CFBuild.
Then the CLI sets that droplet reference as the current droplet on the app and restarts the app. This is where the problems begin.
CFProcess Controller
The restart command first sets the app to STOPPED and then STARTED. These changes might be so quick that a reconciler never sees the STOPPED state, but the CFApp hook will always bump the app-rev.
If the process reconciler triggers while the app is in the STOPPED state, it will delete the AppWorkloads associated with the process. Then when it triggers with the app in the STARTED state, it will create a new AppWorkload with a different name from previously as the app-rev has changed.
If the process reconciler triggered after the transition of the app back to started, it will create a new AppWorkload with a new name (as above), and will then delete the old AppWorkload.
In both cases, the deletion of the AppWorkload causes the running Statefulset to be garbage collected and all instances of the running app go down.
Statefulset AppWorkload Controller
This will see a new AppWorkload and create a new Statefulset for it.
The ideal behaviour
The Statefulset controller takes care to set a pod disruption budget on the stateful sets it produces so that when image references change, or pods need to be moved between nodes, a certain number or proportion of pods are left running during the update. If a cf push just resulted in a new image reference being set in its spec, the app would update gracefully. Note that changing the image ref is sufficient for the Statefulset to restart all pods, so the extra restart command from the CF CLI is unnecessary.
Restarting
Since it is unnecessary to restart an app in korifi when its droplet reference is updated, the CF CLI code be modified to omit this step when it knows it is targeting a korifi deployment. Then the app would never enter the STOPPED state, causing AppWorkloads and Statefulsets to be deleted. Also the app-rev would not change, so the AppWorkload with the previous app-rev would not be deleted, and so the Statefulset would persist, and would update gracefully.
We cannot totally ignore restarts in korifi, as they are required for applying updates to env vars, for example. But note that there is a way to gracefully restart pods in a statefulset: kubectl rollout restart statefulset <statefulset-name>. If there was a way to tell the statefulset controller that a restart was required, then it could apply this sort of strategy for the restart. A possible way, which goes against the normal declarative kubernetes idiom (but then how does one implement such a command on kubernetes without extending the API?), is to use a property on the AppWorkload to state a restart is required. The controller should perform the restart and clear the field.

We discovered that the kubectl rollout restart just puts an annotation on the statefulset.spec.template.metadata.annotations map. This causes the statefulset controller to restart the pods gracefully.
So we spiked an attempt at implementing graceful restart in korifi. We did following:

POST /v3/app/APP_GUID/actions/restart now just sets an annotation of korifi.cloudfoundry.org/restartedAt: [now] on the CFApp, and sets DesiredState to STARTED.
The CFProcess reconciler copies this annotation to the AppWorkload
The CFProcess reconciler now just uses the CFProcess name for the AppWorkload name (it does not use app-rev), so it long-lived.
The statefulset-runner's AppWorkload controller now sets the korifi.cloudfoundry.org/restartedAt annotation from the AppWorkload onto the statefulset at the spec.template.metadata.annotations level.

This now results in a graceful restart without downtime (when pod count > 1).
We noticed that cf restart actual does a stop followed by a start, as does cf push and cf restage, so we spiked changing the CLI code too. This change fixes all 3 commands to restart gracefully!

It was decided that instead of changing cf cli behaviour we should impleent the /v3/deployments endpoint and support cf restart --strategy=rolling. This has been done in #2525


Einsum optimization setting guidelines

Issue type
Performance
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
binary
TensorFlow version
2.16
Custom code
No
OS platform and distribution
Linux Ubuntu 20
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Existing tensorflow supports einops, which enable seamless tensor reduction ops. Parameter to this functionality (kwarg) is the optimize parameter, i.e.
optimize: Optimization strategy to use to find contraction path using opt_einsum. Must be 'greedy', 'optimal', 'branch-2', 'branch-all' or 'auto'. (optional, default: 'greedy').

Is it documented somewhere what the use cases that benefit from these options are in practice?  Further, are effects of these expected to this library perhaps? https://optimized-einsum.readthedocs.io/en/stable/
Simple benchmarking doesn't really reveal any (significant?) differences. Thanks!
Standalone code to reproduce the issue
A simple benchmark of form:

import tensorflow as tf
import numpy as np
import time

# Define three large matrices A, B, and C of size 1000x1000
A = tf.random.uniform((1000, 1000), dtype=tf.float32)
B = tf.random.uniform((1000, 1000), dtype=tf.float32)
C = tf.random.uniform((1000, 1000), dtype=tf.float32)
D = tf.random.uniform((1000, 1000), dtype=tf.float32)
E = tf.random.uniform((1000, 1000), dtype=tf.float32)

greedy = []
non_greedy = []
for _ in range(1000):
    # Perform matrix multiplication using tf.einsum with optimize='greedy'
    start_time_greedy = time.time()
    result_greedy = tf.einsum('ij,jk,kl,lm,mn->in', A, B, C, D, E, optimize='greedy')
    end_time_greedy = time.time()

    # Perform matrix multiplication using tf.einsum with optimize='optimal'
    start_time_branch_all = time.time()
    result_branch_all = tf.einsum('ij,jk,kl,lm,mn->in', A, B, C, D, E, optimize='optimal')
    end_time_branch_all = time.time()

    # Print the execution times
    greedy.append(end_time_greedy - start_time_greedy)
    non_greedy.append(end_time_branch_all - start_time_branch_all)

print(f"Greedy time: {np.mean(greedy)}")
print(f"Optimal time: {np.mean(non_greedy)}")

Relevant log output
Greedy time: 0.067701251745224
Optimal time: 0.06806995582580566

@SkBlaz,
I tried to execute the mentioned code and observed that the Greedy and optimal time taken is almost similar. Kindly find the gist of it here and let me know if the understanding the right in this case.
Greedy time: 0.002096245050430298
Optimal time: 0.0019992468357086183

Thank you!

@tilakrayal thnaks! Yes, you replicated the case. I'm asking, basically, when do these params actually yield something different in practice? I.e. how does one know when to use what (if at all possible to know his upfront)?


jemalloc issue in ripgrep on aarch64-linux

From https://github.com/microsoft/vscode/issues/179121#issuecomment-1502624811
@mycraftmw

2023-04-11 11:01:28.153 [debug] RipgrepSearchEH#search /home/tongyu/.vscode-server-insiders/bin/7f03ec6f7a48791d8027761b22773b454fc52478/node_modules/@vscode/ripgrep/bin/rg --hidden --ignore-case -g '!/.git' -g '!/.svn' -g '!/.hg' -g '!/CVS' -g '!/.DS_Store' -g '!/Thumbs.db' -g '!/node_modules' -g '!/bower_components' -g '!**/*.code-search' --no-ignore-parent --follow --crlf --fixed-strings --no-config --no-ignore-global --json -- 'hello' '.'

cwd: /home/tongyu/bitswiss
2023-04-11 11:01:28.161 [debug] RipgrepSearchEH#search : Unsupported system page size
: Unsupported system page size
memory allocation of 5 bytes failed

2023-04-11 11:01:28.220 [debug] RipgrepSearchEH#search No data from stdout
2023-04-11 11:01:28.220 [debug] RipgrepSearchEH#search No result from parser
2023-04-11 11:01:28.220 [debug] RipgrepSearchEH#search

Probably a regression from making aarch64-linux ripgrep build statically compiled to fix https://github.com/microsoft/vscode-ripgrep/issues/34. We might need to dynamically compile ripgrep instead of using musl, and find a container to compile it in.
related https://github.com/BurntSushi/ripgrep/issues/2180
// cc @roblourens

Looks like jemalloc just needs to be configured with --with-lg-page to support 16K page size.
This can be achieved by setting the environment variable:
JEMALLOC_SYS_WITH_LG_PAGE=14 (16K page size)
Or to support 64K page size as well:
JEMALLOC_SYS_WITH_LG_PAGE=16 (64K page size)
In either case, it'd still work for 4K/16K(/64K) page sizes.

@andreamah Do we have any good news on this? I found the similar thing happened before on other application like https://github.com/sharkdp/fd/issues/1085, probably we can use the same approach to solve this one?

Any update on this issue ?

Temporary Workaround:
To address the issue until it is fixed, follow these steps:
Install ripgrep and obtain its path. For Rocky Linux or any RHEL-based Linux distribution, run the following command with root privileges:
sudo dnf install -y ripgrep
For Ubuntu, use the following command:
sudo apt install ripgrep
Alternatively, you can install ripgrep using Cargo:
cargo install ripgrep
Verify the installation and retrieve the path by running:
which rg
Replace the broken rg with a symlink by executing the following script. Please note that you should remove the echo commands to commit the changes:
(RIPGREP=/usr/bin/rg; find ~/.vscode-server/bin -name rg | while read file _ ; do echo rm -f $file ; echo ln -sf $RIPGREP $file; done)
This workaround should help resolve the issue temporarily.

This should be fixed in tomorrow's VS Code Insiders. Please try it out and let me know if it works for you!

I tried

Version: 1.79.0-insder (user setup)
Commit: 6d3febd
Date: 2023-05-19T20:39:14 (about 3 hours ago)
OS: Windows_NT x65 10.0.22621`

but it seems not fixed or not included the fix. execute from the remote server(aarch64)
$ find .vscode-server-insiders -name rg -exec {} \;
and got

: Unsupported system page size
: Unsupported system page size
memory allocation of 5 bytes failed
find: ‘.vscode-server-insiders/bin/6d3fedb7c2cf965915c785dc07701eb978947506/node_modules/@vscode/ripgrep/bin/rg’ terminated by signal 6

(after resetting the remote environment by deleting .vscode-server-insiders folder on the remote, and vscode reloaded. - 'Developer: Reload Window')

Are you running 16K page size or 64K?

I got

$ getconf PAGE_SIZE
65536

on the aarch64 remote (Rocky Linux 8, aws r7g instance/Graviton3 processors).
Meanwhile, executing the same command above on x86-64 remote, I got the normal rg help print out as like

$ find .vscode-server-insiders/bin -name rg -exec {} ;`
error: The following required arguments were not provided:

USAGE:
rg [OPTIONS] PATTERN [PATH ...]
rg [OPTIONS] -e PATTERN ... [PATH ...]
rg [OPTIONS] -f PATTERNFILE ... [PATH ...]
rg [OPTIONS] --files [PATH ...]
rg [OPTIONS] --type-list
command | rg [OPTIONS] PATTERN
rg [OPTIONS] --help
rg [OPTIONS] --version
For more information try --help

and correcting the hand typed typo above of vscode insider version info

Version: 1.79.0-insider (user setup)
Commit: 6d3fedb7c2cf965915c785dc07701eb978947506
Date: 2023-05-19T20:39:14.598Z
Electron: 22.5.2
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621

Is this fixed in the latest release version of VScode?
I just check on my arm64 container but it seems not working.
VScode Version: 1.79.0 (Universal)

Samw with @RuinanSun
Switched to newest insider build. 1.80.0. Problem not solved.
Went to server, killed .vscode server and deleted $HOME/.vscode-server-insiders. Tried again to bootstrap. Problem is still NOT resolved. Search is still limited to open files.

Here is some useful link detailing what I think is the problem. Not sure if it is resolved by now. But if info from the discussion is correct, I think you will have to either compile on the target system, ship different precompiled versions of binary and select the correct one https://github.com/jemalloc/jemalloc/issues/467 .

We do cross-compile for different systems for ripgrep https://github.com/microsoft/ripgrep-prebuilt. I'll have to take another look.

So, my interpretation is, that the page-size is hard-coded into jemalloc. So, when the system page size at runtime differs from the one it is compiled for, it will fail.
So, the one shipped right now fails out of the box:
mlanger@localhost:~/.vscode-server-insiders/bin/db6bcdf309cc1565c502a54b73c33a63bdabfbd8/node_modules/@vscode/ripgrep/bin$ ./rg
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
memory allocation of 5 bytes failed
Aborted (core dumped)

From your linked repo, I understand that you now focus on 16k page size. That would also not help me (if my suspicion is right), as my system's page size is: 64k
$ getconf PAGESIZE
65536

@bashimao can you check if the following ripgrep build runs fine on your aarch64 setup ? The build is generated from https://github.com/microsoft/ripgrep-prebuilt/pull/30 to use 64k for jemalloc.

Thanks @bashimao. ripgrep built from source works for me!

@RuinanSun @gene203 @bashimao can any of you verify this:

can you check if the following ripgrep build runs fine on your aarch64 setup ? The build is generated from microsoft/ripgrep-prebuilt#30 to use 64k for jemalloc.

I untar-ed the file and the binary 'rg' is working good with remote search. (I sym-linked it with vscode 1.79)

$ tar zxvf ripgrep--aarch64-unknown-linux-musl.tar.gz
rg
$ ls -l rg
-rwxr-xr-x 1 sysadmin sysadmin 4111744 Jun 12 21:34 rg
$ find ~/.vscode-server/bin -name rg -exec ln -sfn ~/rg {} ;

This bug has been fixed in the latest release of VS Code Insiders!
@andreamah, you can help us out by commenting /verified if things are now working as expected.
If things still don't seem right, please ensure you're on version 45b31e9a872da8cc3cc8fff41c1eeb2063be04b7 of Insiders (today's or later - you can use Help: About in the command palette to check), and leave a comment letting us know what isn't working as expected.
Happy Coding!

I can confirm it is fixed with the vscode insider downloaded a minute ago of

Version: 1.80.0-insider (user setup)
Commit: da7f05a70eb760e4ce1fe230ae158ae214433341
Date: 2023-06-27T15:13:35.848Z
Electron: 22.3.14
ElectronBuildId: 21893604
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621

(assume the commit hash changed since)

❯ find ~/.vscode-server-insiders -name rg -exec {} -V \;
ripgrep 13.0.0

thanks so much for checking! @gene203


fixed image locations in summit labs

Yes, I did. The images doesn't load.

I know that images aren't showed in the GitHub repo but the documentation is made with the objective to be made using asciidoctor in order to build the final HTML file and not having them available directly in the repo.

Another thing, I don't think that we have to adapt the documentation to the new Strimzi version. This is the demo of the RH Summit 2018 where we used Strimzi 0.3.0 during the lab. We don't want to update this demo every time a new Strimzi version will come.

I agree with you with the demo update strategy, but the documentation (broken images, etc) I think it should be always working independent of the version.
Just my 2cents.

I see your point but Strimzi is a fast evolving project and breaking changes happen quite easily. For this reason the summit 2018 demo should work out of box without changes. We'll hve something different in the future as you suggest with doc not related to release.


Update pubspec.yaml

The intl dependency version has been updated.
Note: If there is a better way for me to contribute to this pull request, please let me know.
Thanks!

Hi @StevenSnary, PR is merged and pushed to pub.dev with version 1.0.2.
Sorry for the delay.

No problem - Thanks!
On Tue, Dec 27, 2022 at 6:01 AM Christoph Rothermel <
@.***> wrote:

Hi @StevenSnary https://github.com/StevenSnary, PR is merged and pushed
to pub.dev with version 1.0.2.
Sorry for the delay.
—
Reply to this email directly, view it on GitHub
https://github.com/Rodiii/flutter_cupertino_date_textbox/pull/1#issuecomment-1365817824,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACPESNNKTUDTTQYIYOYB2B3WPLECBANCNFSM6AAAAAAS765AOU
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Steve Snary

This message and any attachment are privileged, confidential and contain
private information.  The information contained herein is not to be
forwarded or shared with others in any way without the express consent of
the sender.  If you are not the addressee named above any disclosure,
reproduction, copying, distribution, or other dissemination or use of this
communication is prohibited. If you have received this transmission in
error, please notify the sender immediately and destroy this e-mail.


How much is a download limit for YouTube

Hello
How much is a download limit for YouTube?
Per minute or hour

Unknown.


Removed forcing disaggregation parameters from list of global param file options

[x] closes #830
[ ] tests passed
[ ] new tests added
[ ] science test figures
[ ] ran uncrustify prior to final commit
[x] ReleaseNotes entry

Thanks @dgergel


State transitions in application wired to screens

The purpose of this issue is to integrate the application state transitions with screens.

Closed by #5


Add Object2X maps

Final jar size is ~1MB
Again, lots of stuff in the pom.xml because it otherwise simply gets filtered out with the shade filters.

These individual submodules will require "xbase" and "objectbase" as the pom.xml files imply.
ie. object2intmap requires intbase and objectbase
object2charmap requires charbase and objectbase


rottentomatoes_list not produceing entries

tasks:
  Rottentomatoes_DVDs:
    rottentomatoes_list:
      api_key: '{{ secrets.rottentomatoes.api }}'
      dvds:
        - new_releases
        - top_rentals
        - current_releases
    accept_all: yes

this task doesn't produce anything.
digging around in the code i believe the problem is line 57 of rottentomatoes_list.py
            if type(l_names) is not list:
                continue

this always evaluates to true and all the l_names items hit the continue statement breaking out of the parent for loop.
i tried :
            if type(l_names) is not list:
                log.warning('type(%s) is not list, its %s, continuing' % (l_names, type(l_names)))
                continue

and got this :
WARNING  rottentomatoes_list Rottentomatoes_DVDs type([u'new_releases']) is not list, its <type 'list'>, continuing
I gather this bit of code is trying to skip the api_key. For now I'm replacing line 57 with:
if l_type == 'api_key'
hacky and defiantly not defensive programing, but it works for me.

Using type that way is way wrong. Should use isinstance or issubclass

there's almost nothing that have to with rotten tomatoes plugin that isn't in need of refactor.


Added the Correct Slack link in contributing.md

Signed-off-by: Akash Gupta<EMAIL_ADDRESS>Description
Added the correct link to slack chat room in contributing.md
This PR fixes #1483
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Welldone @akashgp09 🥳, what do you think about using slack.layer5.io instead? 🤔

Thank you for the contribution @akashgp09!

Welldone @akashgp09 🥳, what do you think about using slack.layer5.io instead? 🤔

I did it already 😊.


Feature Request: AR Mode

Hello!
This tool looks awesome! When I'm playing around with it, I see that it supports the WebXR "VR" features, but it doesn't have support for AR. Would it be possible to enable this? Maybe by providing a default "pass through" image of the user's surroundings? (As I say this, I'm thinking that there would probably need to be spatial anchors, "plane detectors", etc. which would make this significantly more complicated, especially to be as customizable as the rest of the browser extension appears to be.)

Would be nice to not have "AR NOT FOUND"

The regular WebXR emulator already has an AR mode.

On my list! Not sure when I will get to it though, late this year, early next year?

@felixtrz Would love AR mode since using the other WebXR extension is horrible! Let me know if I can help

@felixtrz Would love AR mode since using the other WebXR extension is horrible! Let me know if I can help

Thank you for the offer! I am currently working on it as my top priority. I can definitely use your feedback once the first version of this comes out, and I will definitely welcome PRs that build on top of that!

We have released AR support in https://github.com/meta-quest/immersive-web-emulator/releases/tag/v1.1.0 , with features like plane-detection and anchor support that are not available in Mozilla's emulator. More details here: https://developer.oculus.com/blog/immersive-web-emulator-1-1/


Maximum number formed from the combinations of two digit numbers given as input by the user

💥 Proposal
Maximum number formed from the combinations of two digit numbers given as input by the user in C++
Have you read the Contributing Guidelines on Pull Requests?
Yes

/assign


WIP | Device Orientation Manipulation

This PR adds device orientation change support to the device API (see #131).
Last thing to do is add some description to the docs, at least from my perspective.
@talkol @rotemmiz Can you give me feedback for the implementation? Is there anything I could do better next time? The docs will follow tomorrow or the day after 👍

[x] remove extension to MethodInvocation.m
[x] use earl grey function directly in setOrientation
[x] move JS implementation to Simulator.js
[x] write unit tests
[x] add docs

@rotemmiz Can you give me final feedback, this PR is done from my side (at least as far as I can tell 👍 )


Investigate re-rendering bug

Navigate to a pet's home screen
Modify the tag line using the "set tagline" button
Note that the tagline next to the Set Tagline to Super Dog button has changed
Set the tagline to super using the Set Tagline to Super Dog button
Note that the the text next to the "Set Tagline To Super Dog" has not re-rendered
In the console, we see that the compare the before and after results, but determine that they haven't changed. Odd!

This is probably due to the fact that we're holding this value in state. Probably not worth explicitly fixing, as I'll probably massively redesign the demo projects at some point


shorthand/base#_getAttrsForRequest  should normalize relationships too

The method only normalized the attributes and leaves out the relationships.
If I have a payload like the following
{
  "data": {
    "attributes": {
      "does-mirage": true,
      "name": "sam"
    },
    "relationships": {
      "company": {
        "data": {
          "id": "1",
          "type": "companies"
        }
      },
      "github-account": {
        "data": {
          "id": "1",
          "type": "github-accounts"
        }
      }
    },
    "type": "github-account"
  }
}

The output should be something like
{name: "Sam", doesMirage: true, companyId: 1, githubAccountId: 1}

I'm working on a PR, will submit shortly :)

Nice... good catch. Would the document ever contain has-many relationships? I.e. relationships that primary document model does not own the foreign keys to? Is there any way to check for this? Possibly by introspecting the model?

@samselikoff I believe is totally possible, it can be checked on the type of .data https://github.com/samselikoff/ember-cli-mirage/pull/562/files#diff-26f9dc309eb3c49fa6ea23f36ffa4d6fR85  --- if it is an object then it is a belongsTo if is an array then a hasMany.
I'd say let's handle this scenario for now, and I'll explore how has many can come in..  I'm migrating our tests suite to mirage so probably this will pop :)

btw, is there a way to mix collections?  say I have a vehicles end-points... I would like to return something like schema.car.all() + schema.motorcycle.all()

To answer https://github.com/samselikoff/ember-cli-mirage/issues/561#issuecomment-182849101
The following seems to do the trick
this.get('/api/vehicles', function(schema, request) {
  let collection = schema.car.all();

  collection.push(...schema.motorcycle.all());

  // added a JSONAPI serializer for vehicles, since it is JSONAPI
  // it will look at object.type to create the response (not the collection modelName)
  //
  collection.modelName = 'vehicle';

  return collection;
});

Interesting... so would this be a polymorphic relationship?

Interesting... so would this be a polymorphic relationship?

Correct, that's what we are trying to get, I'll let you know if we run into any issues with it.


fix: add new columns in schema.sql

Fix #468

Any updates on this one ? All tests are failing

As far as I remember @bonustrack asked us to wait, but I'm not sure exactly. Do you think we can proceed? @bonustrack

I've changed the new types on the proposals table to be:

If you can update it, then all good for me. Later I'll also add some types on the votes table

Updated


osu!Stats crashes on undefined users

Stacktrace :
C:\Users\Krystal\Documents\owo-whats-this\out\commands\awwwsooo\osu.js:34
                    if (user.user_id == undefined) {
                            ^

TypeError: Cannot read property 'user_id' of undefined
    at IncomingMessage.<anonymous> (C:\Users\Krystal\Documents\owo-whats-this\node_modules\request\request.js:1001:12)
    at IncomingMessage.g (events.js:291:16)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)```

Fixed in #21

Issue still persists on public bot. Reopening

Re-Closing, Issue might have been fixed now


Remove allocation in simple redis command checks

closes https://github.com/shotover/shotover-proxy/issues/294
All the cases that match on multiple commands are left alone as I have no idea if it would help there and it would make it a lot harder to read.
I removed the non-CLUSTER command fast path from RedisClusterPortsRewrite as:

inspecting the code I can see that the logic is allocationless even without the fast path
The redis_cluster_ports_rewrite benchmark does not demonstrate any difference in performance. Both sit at 430ns on my machine.
But maybe theres an argument to be made that we should keep the fast path because it helps protect against accidentally regressing on performance in the regular path?

I'm for removing the fast-path because it's not actually doing anything performance wise at the moment. If we do regress on the regular path we can address that in the best way at the time (maybe reintroducing the fast-path or a different way).


One fix for "com.orientechnologies.orient.core.exception.OSerializationException: Type String cannot be serialized because is not part of registered entities. To fix this error register this class"

I'm frequently running into the error "com.orientechnologies.orient.core.exception.OSerializationException: Type String cannot be serialized because is not part of registered entities. To fix this error register this class" which is caused by a OObjectLazyMap being created when the map only contains java types. I have found that the cause for the errors occurring to me is that an OObjectLazyMap is created when the fields have a lower version number than the document version number. It's likely that the application is badly coded, I'm however porting it from a working JPA implementation.
The fix is by adding a !OReflectionHelper.isJavaType(genericMultiValueType) check which solves the problem (my problem at least).
Had a few issues with my lack of experience of creating pull requests (I'm old, this new fancy git stuff :-) and the code formatter (which by the way reformats recently cloned code) so there's a few formatting issues. The unit test is ugly, but I really haven't time to create a really nicely documented one, it do what it should, proves the error and my change).

nice pull request, checking it

look good and test passes, merging.


Intercept HTTP requests which are made by dart:io HttpClient

This API might be a good starting point.
Sometimes Flutter makes HTTP requests itself.  For example, for the network image widget. This would allow to track them.
A few gotchas: The requests from the SDK itself are maybe tracked, too. Also if it's used with http/dio integrations, there may be duplicated trackings because http and dio use dart:ios HttpClient

https://github.com/ueman/sentry-dart-tools/tree/main/sentry_plus/lib/src/http

Since this is pretty interesting for monitoring network image widgets, you can trivially work around this limitation by using an image provider backed by http or dio.


Make binary wheels for all platforms

Making binary wheels and uploading them to PyPi, would allow to pip install tslearn without needing a compiler or Cython.
Usually this requires quite a bit of work, see e.g. https://github.com/MacPython/scikit-learn-wheels/. However there is a shortcut with https://github.com/regro/conda-press that might allow generating wheels from conda-forge builds. I have not used it yet personally, but it could be worth a try.

OK, we should definitely give it a try.
conda-press option seems very attractive.
Then I guess the release process should be something like:

release source archive on PyPI
wait for conda-forge to generate builds
use conda-press to turn them into wheels locally
release wheels on PyPI

Yes, we could try it for the current version. But we need to check that generated wheel can be installed correctly.

And I guess we should check that on various OS, and if possible on instances that do not have C++ build tools installed, right?

OK, I am starting to play with that idea, locally.
With the following code, I can generate all OSX and Windows wheels for tslearn (as soon as the corresponding artifact is available on conda-forge, of course):
TARGET_TSLEARN_VERSION=0.2.1
for OSNAME in osx-64 win-64;
do
    for PYTHONVER in py27 py36 py37;
    do
        conda press --subdir ${OSNAME} --skip-python --fatten tslearn=${TARGET_TSLEARN_VERSION}=${PYTHONVER}*;
    done;
done

I guess this can be a good starting point. I still have issues at the moment when it comes to building linux wheels, which is a pity. I will try to dig that later.
Also, I would say that building fat wheels (ie. that includes the dependencies, if I understood correctly) is a bit of a pity, no? Are there use cases where this would be safer than lighter wheels without dependencies included?
Finally, maybe we should ask for help for the community of users (eg. via Twitter) once we have uploaded the wheels on PyPI so that people can test them, no? But if we do so, maybe the test will be biased (tslearn users are likely to have installed tslearn successfully in the past :P)
@rth: I'd be happy to read your feedback on those points :)

A bit more insight into the failure case for Linux. For all python versions, it seems to fail at some point when running the following command (through a subprocess call, but anyway):
install_name_tool -add_rpath @loader_path/../lib /var/folders/q_/n_cbhlk54t7667mq9jl141g80000gr/T/tslearn-0.2.1-py27hc1659b7_060mkj2zw/lib/python2.7/site-packages/tslearn/cycc.so

And if I run the command in terminal, the error message is (once again for all 3 python versions):
error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: input file: /var/folders/q_/n_cbhlk54t7667mq9jl141g80000gr/T/tslearn-0.2.1-py27hc1659b7_060mkj2zw/lib/python2.7/site-packages/tslearn/cycc.so is not a Mach-O file

Not sure what this means / how to deal with that, but maybe I could post an issue on the conda-press github.

Thanks for looking into this @rtavenar !

Are there use cases where this would be safer than lighter wheels without dependencies included?

As far as I understood fat wheels with --skip-python  will only include non-python dependencies (e.g. dependencies on C or C++ libraries).

A bit more insight into the failure case for Linux.
maybe I could post an issue on the conda-press github.

Definitely worth reporting. For me Linux and Windows builds fine, while the MacOS fails with,

rewriting RPATH for lib/python2.7/site-packages/tslearn/cycc.so
Traceback (most recent call last):
  File "/home/rth/.miniconda3/envs/conda-press-env/bin/conda-press", line 4, in <module>
    main()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/main.xsh", line 63, in main
    only_pypi=ns.only_pypi
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/condatools.xsh", line 866, in artifact_ref_dependency_tree_to_wheels
    only_pypi=only_pypi
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/condatools.xsh", line 799, in package_to_wheel
    only_pypi=only_pypi
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/condatools.xsh", line 766, in artifact_to_wheel
    wheel.rewrite_rpaths()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/wheel.xsh", line 582, in rewrite_rpaths
    current_rpath = $(patchelf --print-rpath @(fspath)).strip()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 22440, in subproc_captured_stdout
    return run_subproc(cmds, captured="stdout")
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 22424, in run_subproc
    command.end()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 15671, in end
    self._end(tee_output=tee_output)
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 15690, in _end
    self._raise_subproc_error()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 15816, in _raise_subproc_error
    raise subprocess.CalledProcessError(rtn, spec.args, output=self.output)
subprocess.CalledProcessError: Command '['patchelf', '--print-rpath', '/tmp/tslearn-0.2.1-py27heacc8b8_07l8p5764/lib/python2.7/site-packages/tslearn/cycc.so']' returned non-zero exit status 1.

will open an issue about it.

Finally, maybe we should ask the community of tslearn users for help (eg. via Twitter) once we have uploaded the wheels on PyPI so that people can test them, no? But if we do so, maybe the test will be biased (tslearn users are likely to have installed tslearn successfully in the past :P)

Well, it's also possible to just test locally, upload wheels PyPi, announce the release see if there any complaints.
I have checked the python 3.7 wheels, overall it seems to work fine.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure.

        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.

        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________________ test_shapelets ________________________________________________________________

    def test_shapelets():
        n, sz, d = 15, 10, 2
        rng = np.random.RandomState(0)
        time_series = rng.randn(n, sz, d)
        y = rng.randint(2, size=n)
        clf = ShapeletModel(n_shapelets_per_size={2: 5},
                            max_iter=1,
                            verbose=0,
                            optimizer="sgd",
                            random_state=0)
        clf.fit(time_series, y)
        np.testing.assert_allclose(clf.shapelets_[0],
                                   np.array([[0.56373, 0.494684],
                                             [1.235707, 1.119235]]),
                                   atol=1e-2)
        np.testing.assert_allclose(clf.predict(time_series),
                                   np.array([0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0,
                                             1, 0]))

        from sklearn.model_selection import cross_validate
        cross_validate(clf, time_series, y, cv=2)

        model = ShapeletModel(n_shapelets_per_size={3: 2, 4: 1},
                              max_iter = 1)
        model.fit(time_series, y)
        for shp, shp_bis in zip(model.shapelets_,
                                model.shapelets_as_time_series_):
            np.testing.assert_allclose(shp,
>                                      to_time_series(shp_bis, remove_nans=True))
E           AssertionError:
E           Not equal to tolerance rtol=1e-07, atol=0
E
E           (shapes (4, 2), (0, 2) mismatch)
E            x: array([[-0.073456, -0.050393],
E                  [-0.031255, -0.020855],
E                  [ 0.055282,  0.036814],
E                  [ 0.027849, -0.002043]])
E            y: array([], shape=(0, 2), dtype=float64)

test_shapelets.py:38: AssertionError

those in scikit-learn are probably irrelevent, but the shape mismatch in test_shapelets is concerning. Will try to check other python versions later today or tomorrow.

Are there use cases where this would be safer than lighter wheels without dependencies included?

As far as I understood fat wheels with --skip-python will only include non-python dependencies (e.g. dependencies on C or C++ libraries).

More specificaly, for instance on Linux the size of the wheel is mostly due to the /lib/ folder containing,
ls lib
libasan.so        libatomic.so.1      libgomp.so        libitm.so.1      liblsan.so.0.0.0      libtsan.so        libubsan.so.1
libasan.so.5      libatomic.so.1.2.0  libgomp.so.1      libitm.so.1.0.0  libquadmath.so        libtsan.so.0      libubsan.so.1.0.0
libasan.so.5.0.0  libgcc_s.so         libgomp.so.1.0.0  liblsan.so       libquadmath.so.0      libtsan.so.0.0.0
libatomic.so      libgcc_s.so.1       libitm.so         liblsan.so.0     libquadmath.so.0.0.0  libubsan.so

the Python dependencies are not included in the wheel.

error: /install_name_tool: [...] cycc.so is not a Mach-O file

Following discussion in https://github.com/rtavenar/tslearn/pull/154 it looks like one actually need to build it on the target platform, which would explain why it fails on Linux for you and on MacOS for me. Why Windows succeeds is still unclear.

Thanks for the PR! I just pushed a source dist to PyPI. If everything goes well, conda-forge should trigger soon, and we could try to use the artifacts then.
One thing I have in mind: at least for testing, we could use TravisCI (maybe pointing to a dedicated repo) to test the following steps:

conda press
wheel install
wheel tests

on all platforms/OS combinations.
But then, I am unsure how we could upload the wheels to PyPI without publishing my PyPI credentials. Is there a way?

OK, so I have tried to set up something there.
The idea is to try what I wrote in the previous post, but I have some issues:

on OSX, it fails at installing python...
on Linux, the wheel seems to build, and install goes fine, but when I try to run pytest --pyargs tslearn, no tests are found.

If you have any idea why, @rth , I'd be happy to read it :)

@rtavenar Sorry for the delay, will try to have a look at that and the paper later today.

Update: this is probably something I should have a look at to automate the builds, but I have no time at the moment.

Maybe that blog post would be something to try (use the conda smithy for wheel building on each platform) ?

Another option would be to use GitHub Actions such as this one: https://github.com/actions/starter-workflows/blob/master/ci/python-package.yml
It seems rather doable to trigger builds on new releases, but I am not sure if building on multiple platforms in parallel is OK. I'll try to look at it soon.

Maybe that blog post would be something to try (use the conda smithy for wheel building on each platform) ?

Conda-smithy could be worth a try but it doesn't build real manylinux wheels that would work on most linux platforms and is still somewhat experimental.

Another option would be to use GitHub Actions such as this one

FWIW in scikit-learn we have migrated to use Azure Pipelines for all CI and building wheels (e.g. https://github.com/MacPython/scikit-learn-wheels/pull/23)

There is also https://github.com/joerick/cibuildwheel which I have not tried.

To give an example of smaller project that builds wheels on all platform in Azure pipelines in CI, there is for instance: https://github.com/giotto-ai/giotto-tda/blob/master/azure-pipelines.yml
I think adding Windows wheels would already be a significant improvement. Most linux users should have a working compiler by default, so compiling there is a bit less problematic (although I agree it would be good to add it as well).

Thanks for the feedback.
Both options have pros and cons. cibuildwheel is appealing since it would not require to change the CI system, but in their examples (eg here), they only have Python3.8 for Windows, not sure why...
If we decided to switch to Azure pipelines, would you be available to give a hand on this?

If we decided to switch to Azure pipelines, would you be available to give a hand on this?

Yes, I have some availability to help for that. If you enable it on the repo, I can make a first PR.

OK, I connected an Azure account with this repo: https://dev.azure.com/romaintavenard/tslearn/_build
It created a default YAML configuration file which fails for the moment (expected).
Let me know if I did something wrong.

Looks like we have good news on this matter:
https://github.com/rtavenar/tslearn/actions/runs/85697155
Works for all 3 platforms, and Python 2.7, 3.5--3.8, I just ignored PyPy on Windows since it lead to an error.

Great that it's working! cibuildwheel does indeed look quite user friendly.

Hell yeah!
    - name: Build wheel
      env:
        CIBW_SKIP: "pp*-win*"
        CIBW_BEFORE_BUILD: "pip install numpy cython"
      run: |
        python -m cibuildwheel --output-dir wheelhouse


build: bump toolchain to nightly-2024-03-12

Signed-off-by: Richard Chien<EMAIL_ADDRESS>hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
There have been quite some changes, so I want to merge now to reduce burden.
Also since we've code freezed for 1.8, so it's a good timing now.
blocker https://github.com/rust-lang/rust/issues/122618 (workarounded)

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[ ] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Blocked by https://github.com/risingwavelabs/jsonbb/issues/3

All tests pass now. Only fmt/clippy issues.

https://github.com/rust-lang/rust/issues/111137
This error raises after merge main.
error: could not compile `curve25519-dalek` (lib) due to 5 previous errors

error[E0658]: use of unstable library feature 'stdarch_x86_avx512'
  --> /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/curve25519-dalek-4.1.1/src/backend/vector/ifma/field.rs:26:5
   |
26 |     _mm256_madd52lo_epu64(z.into(), x.into(), y.into()).into()
   |     ^^^^^^^^^^^^^^^^^^^^^
   |
   = note: see issue #111137 <https://github.com/rust-lang/rust/issues/111137> for more information
   = help: add `#![feature(stdarch_x86_avx512)]` to the crate attributes to enable
   = note: this compiler was built on 2024-04-10; consider upgrading it if it is out of date

error[E0658]: use of unstable library feature 'stdarch_x86_avx512'
  --> /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/curve25519-dalek-4.1.1/src/backend/vector/ifma/field.rs:25:9
   |
25 |     use core::arch::x86_64::_mm256_madd52lo_epu64;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: see issue #111137 <https://github.com/rust-lang/rust/issues/111137> for more information
   = help: add `#![feature(stdarch_x86_avx512)]` to the crate attributes to enable
   = note: this compiler was built on 2024-04-10; consider upgrading it if it is out of date

error[E0658]: use of unstable library feature 'stdarch_x86_avx512'
  --> /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/curve25519-dalek-4.1.1/src/backend/vector/ifma/field.rs:34:5
   |
34 |     _mm256_madd52hi_epu64(z.into(), x.into(), y.into()).into()
   |     ^^^^^^^^^^^^^^^^^^^^^
   |
   = note: see issue #111137 <https://github.com/rust-lang/rust/issues/111137> for more information
   = help: add `#![feature(stdarch_x86_avx512)]` to the crate attributes to enable
   = note: this compiler was built on 2024-04-10; consider upgrading it if it is out of date

error[E0658]: use of unstable library feature 'stdarch_x86_avx512'
  --> /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/curve25519-dalek-4.1.1/src/backend/vector/ifma/field.rs:33:9
   |
33 |     use core::arch::x86_64::_mm256_madd52hi_epu64;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: see issue #111137 <https://github.com/rust-lang/rust/issues/111137> for more information
   = help: add `#![feature(stdarch_x86_avx512)]` to the crate attributes to enable
   = note: this compiler was built on 2024-04-10; consider upgrading it if it is out of date

error[E0635]: unknown feature `stdsimd`
  --> /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/curve25519-dalek-4.1.1/src/lib.rs:13:70
   |
13 | #![cfg_attr(all(curve25519_dalek_backend = "simd", nightly), feature(stdsimd))]
   |                                                                      ^^^^^^^

Maybe the change is lost 🤔

@stdrc @wenym1 Please take a look at these commits. Thanks!

https://github.com/madsim-rs/madsim/issues/205

madsim-rs/madsim#205

May have been fixed in the latest version (by https://github.com/madsim-rs/madsim/pull/202). I have updated madsim. Let's see if it works.

:eyes:
running 1 test
seed = 5
Configuration {
    config_path: Temp(
        "/tmp/.tmpKfpUSz",
    ),
    frontend_nodes: 2,
    compute_nodes: 0,
    meta_nodes: 3,
    compactor_nodes: 2,
    compute_node_cores: 2,
    etcd_timeout_rate: 0.0,
    etcd_data_path: None,
    per_session_queries: [],
}
test batch::test_serving_cluster_availability ... FAILED

failures:

failures:
    batch::test_serving_cluster_availability

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 110 filtered out; finished in 6.92s

--- STDERR:              risingwave_simulation::integration_tests batch::test_serving_cluster_availability ---
thread '<unnamed>' panicked at src/tests/simulation/tests/integration_tests/batch/mod.rs:138:31:
called `Result::unwrap_err()` on an `Ok` value: "0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n21\n22\n23\n24\n25\n26\n27\n28\n29\n30\n31\n32\n33\n34\n35\n36\n37\n38\n39\n40\n41\n42\n43\n44\n45\n46\n47\n48\n49\n50\n51\n52\n53\n54\n55\n56\n57\n58\n59\n60\n61\n62\n63\n64\n65\n66\n67\n68\n69\n70\n71\n72\n73\n74\n75\n76\n77\n78\n79\n80\n81\n82\n83\n84\n85\n86\n87\n88\n89\n90\n91\n92\n93\n94\n95\n96\n97\n98\n99\n100\n101\n102\n103\n104\n105\n106\n107\n108\n109\n110\n111\n112\n113\n114\n115\n116\n117\n118\n119\n120\n121\n122\n123\n124\n125\n126\n127\n128\n129\n130\n131\n132\n133\n134\n135\n136\n137\n138\n139\n140\n141\n142\n143\n144\n145\n146\n147\n148\n149\n150\n151\n152\n153\n154\n155\n156\n157\n158\n159\n160\n161\n162\n163\n164\n165\n166\n167\n168\n169\n170\n171\n172\n173\n174\n175\n176\n177\n178\n179\n180\n181\n182\n183\n184\n185\n186\n187\n188\n189\n190\n191\n192\n193\n194\n195\n196\n197\n198\n199\n200\n201\n202\n203\n204\n205\n206\n207\n208\n209\n210\n211\n212\n213\n214\n215\n216\n217\n218\n219\n220\n221\n222\n223\n224\n225\n226\n227\n228\n229\n230\n231\n232\n233\n234\n235\n236\n237\n238\n239\n240\n241\n242\n243\n244\n245\n246\n247\n248\n249\n250\n251\n252\n253\n254\n255\n256\n257\n258\n259\n260\n261\n262\n263\n264\n265\n266\n267\n268\n269\n270\n271\n272\n273\n274\n275\n276\n277\n278\n279\n280\n281\n282\n283\n284\n285\n286\n287\n288\n289\n290\n291\n292\n293\n294\n295\n296\n297\n298\n299\n300\n301\n302\n303\n304\n305\n306\n307\n308\n309\n310\n311\n312\n313\n314\n315\n316\n317\n318\n319\n320\n321\n322\n323\n324\n325\n326\n327\n328\n329\n330\n331\n332\n333\n334\n335\n336\n337\n338\n339\n340\n341\n342\n343\n344\n345\n346\n347\n348\n349\n350\n351\n352\n353\n354\n355\n356\n357\n358\n359\n360\n361\n362\n363\n364\n365\n366\n367\n368\n369\n370\n371\n372\n373\n374\n375\n376\n377\n378\n379\n380\n381\n382\n383\n384\n385\n386\n387\n388\n389\n390\n391\n392\n393\n394\n395\n396\n397\n398\n399\n400\n401\n402\n403\n404\n405\n406\n407\n408\n409\n410\n411\n412\n413\n414\n415\n416\n417\n418\n419\n420\n421\n422\n423\n424\n425\n426\n427\n428\n429\n430\n431\n432\n433\n434\n435\n436\n437\n438\n439\n440\n441\n442\n443\n444\n445\n446\n447\n448\n449\n450\n451\n452\n453\n454\n455\n456\n457\n458\n459\n460\n461\n462\n463\n464\n465\n466\n467\n468\n469\n470\n471\n472\n473\n474\n475\n476\n477\n478\n479\n480\n481\n482\n483\n484\n485\n486\n487\n488\n489\n490\n491\n492\n493\n494\n495\n496\n497\n498\n499\n500\n501\n502\n503\n504\n505\n506\n507\n508\n509\n510\n511\n512\n513\n514\n515\n516\n517\n518\n519\n520\n521\n522\n523\n524\n525\n526\n527\n528\n529\n530\n531\n532\n533\n534\n535\n536\n537\n538\n539\n540\n541\n542\n543\n544\n545\n546\n547\n548\n549\n550\n551\n552\n553\n554\n555\n556\n557\n558\n559\n560\n561\n562\n563\n564\n565\n566\n567\n568\n569\n570\n571\n572\n573\n574\n575\n576\n577\n578\n579\n580\n581\n582\n583\n584\n585\n586\n587\n588\n589\n590\n591\n592\n593\n594\n595\n596\n597\n598\n599\n600\n601\n602\n603\n604\n605\n606\n607\n608\n609\n610\n611\n612\n613\n614\n615\n616\n617\n618\n619\n620\n621\n622\n623\n624\n625\n626\n627\n628\n629\n630\n631\n632\n633\n634\n635\n636\n637\n638\n639\n640\n641\n642\n643\n644\n645\n646\n647\n648\n649\n650\n651\n652\n653\n654\n655\n656\n657\n658\n659\n660\n661\n662\n663\n664\n665\n666\n667\n668\n669\n670\n671\n672\n673\n674\n675\n676\n677\n678\n679\n680\n681\n682\n683\n684\n685\n686\n687\n688\n689\n690\n691\n692\n693\n694\n695\n696\n697\n698\n699\n700\n701\n702\n703\n704\n705\n706\n707\n708\n709\n710\n711\n712\n713\n714\n715\n716\n717\n718\n719\n720\n721\n722\n723\n724\n725\n726\n727\n728\n729\n730\n731\n732\n733\n734\n735\n736\n737\n738\n739\n740\n741\n742\n743\n744\n745\n746\n747\n748\n749\n750\n751\n752\n753\n754\n755\n756\n757\n758\n759\n760\n761\n762\n763\n764\n765\n766\n767\n768\n769\n770\n771\n772\n773\n774\n775\n776\n777\n778\n779\n780\n781\n782\n783\n784\n785\n786\n787\n788\n789\n790\n791\n792\n793\n794\n795\n796\n797\n798\n799\n800\n801\n802\n803\n804\n805\n806\n807\n808\n809\n810\n811\n812\n813\n814\n815\n816\n817\n818\n819\n820\n821\n822\n823\n824\n825\n826\n827\n828\n829\n830\n831\n832\n833\n834\n835\n836\n837\n838\n839\n840\n841\n842\n843\n844\n845\n846\n847\n848\n849\n850\n851\n852\n853\n854\n855\n856\n857\n858\n859\n860\n861\n862\n863\n864\n865\n866\n867\n868\n869\n870\n871\n872\n873\n874\n875\n876\n877\n878\n879\n880\n881\n882\n883\n884\n885\n886\n887\n888\n889\n890\n891\n892\n893\n894\n895\n896\n897\n898\n899\n900\n901\n902\n903\n904\n905\n906\n907\n908\n909\n910\n911\n912\n913\n914\n915\n916\n917\n918\n919\n920\n921\n922\n923\n924\n925\n926\n927\n928\n929\n930\n931\n932\n933\n934\n935\n936\n937\n938\n939\n940\n941\n942\n943\n944\n945\n946\n947\n948\n949\n950\n951\n952\n953\n954\n955\n956\n957\n958\n959\n960\n961\n962\n963\n964\n965\n966\n967\n968\n969\n970\n971\n972\n973\n974\n975\n976\n977\n978\n979\n980\n981\n982\n983\n984\n985\n986\n987\n988\n989\n990\n991\n992\n993\n994\n995\n996\n997\n998\n999"
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/core/src/result.rs:1654:5
   3: core::result::Result<T,E>::unwrap_err
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/core/src/result.rs:1171:22
   4: integration_tests::batch::test_serving_cluster_availability::{{closure}}::{{closure}}
             at ./tests/integration_tests/batch/mod.rs:138:5
   5: madsim::sim::task::Executor::block_on::{{closure}}::{{closure}}
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:230:32
   6: async_task::raw::RawTask<F,T,S,M>::run
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/raw.rs:563:17
   7: async_task::runnable::Runnable<M>::run
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/runnable.rs:782:18
   8: madsim::sim::task::Executor::run_all_ready::{{closure}}
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:271:59
   9: std::panicking::try::do_call
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/std/src/panicking.rs:552:40
  10: std::panicking::try
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/std/src/panicking.rs:516:19
  11: std::panic::catch_unwind
             at /rustc/a07f3eb43acc5df851e15176c7081a900a30a4d7/library/std/src/panic.rs:149:14
  12: madsim::sim::task::Executor::run_all_ready
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:271:17
  13: madsim::sim::task::Executor::block_on
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:238:13
  14: madsim::sim::runtime::Runtime::block_on
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/runtime/mod.rs:126:9
  15: madsim::sim::runtime::builder::Builder::run::{{closure}}::{{closure}}::{{closure}}
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/runtime/builder.rs:128:35
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
context: node=0 "main", task=0 (spawned at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/runtime/mod.rs:126:19)
note: run with `MADSIM_TEST_SEED=5` environment variable to reproduce this error

🤡

🤡

cc @bakjos Could you help investigate this error on the latest nightly toolchain? I have no idea where the illegal instruction is. Thanks!

@wangrunji0408 I guess the issue is due to some changes done in the nightly-2024-02-05+ although the compilation fixes were done in https://github.com/dalek-cryptography/curve25519-dalek/pull/619,  there are still some issues with the unstable simd feature, I think this can be fixed by forcing a different backend using
RUSTFLAGS='--cfg curve25519_dalek_backend="serial"'

@xxchan, @wangrunji0408 Do you want me to push this change, to see if it fixes it?

@bakjos Thanks for the investigation. I think you cannot push directly to this branch. If you are willing to help, you can create another PR against this PR’s branch and trigger CI there.

@xxchan It turns out the issue is because something made between nightly 2024-03-12 and nightly 2024-03-13, the nightly 2024-03-12 compiles the code successfully but the 2024-03-13 fails  I haven't been able to figure it out what's making it fail. This is the full change list between both versions https://github.com/rust-lang/rust/compare/4a0cc881dcc4d800f10672747f61a94377ff6662...a165f1f65015b1bd4afd2ec50700aaacf2e0c485

@bakjos Thanks a lot for your investigation. BTW just FYI there's a tool for finding rustc's regression https://github.com/rust-lang/cargo-bisect-rustc/

@xxchan btw this error doesn't happen on osx, runs without issues using the 2024-04-26 nightly

Looking forward for the toolchain bump. I met another stale toolchain issue similar to https://github.com/zowens/crc32c/issues/58 related to arm when bumping opendal version to 0.46. In other words, without this PR, we cannot upgrade opendal version in storage.

In other words, without this PR, we cannot upgrade opendal version in storage.

This is a little unfortunate, but the situation is not that bad...
For bumping opendal, I don't think the newly added<EMAIL_ADDRESS>is needed by us: https://github.com/apache/opendal/pull/4533. So a quick workaround is: we can use a patched opendal with this feature removed, or simply use<EMAIL_ADDRESS>For bumping toolchain, there are several ways to proceed:

Try to reproduce the core dump, and report to either deno or rust.
Let's trust rust, and YOLO: just disable deno UDF for now, and move on.

I don't have the energy to push forward this now. I appreciate it if anyone can take over the work.

Try longevity https://buildkite.com/risingwave-test/longevity-test/builds/1394

@xxchan pushed this one fixing all the clippy warnings https://github.com/risingwavelabs/risingwave/pull/16754

Oops. It seems we do this chore work duplicately. 🤪

:) No worries, I will close mine, seems like it's finally close to being done

--- STDERR:              risingwave_simulation::integration_tests batch::test_serving_cluster_availability ---
thread '<unnamed>' panicked at src/tests/simulation/tests/integration_tests/batch/mod.rs:138:31:
called `Result::unwrap_err()` on an `Ok` value: "0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n21\n22\n23\n24\n25\n26\n27\n28\n29\n30\n31\n32\n33\n34\n35\n36\n37\n38\n39\n40\n41\n42\n43\n44\n45\n46\n47\n48\n49\n50\n51\n52\n53\n54\n55\n56\n57\n58\n59\n60\n61\n62\n63\n64\n65\n66\n67\n68\n69\n70\n71\n72\n73\n74\n75\n76\n77\n78\n79\n80\n81\n82\n83\n84\n85\n86\n87\n88\n89\n90\n91\n92\n93\n94\n95\n96\n97\n98\n99\n100\n101\n102\n103\n104\n105\n106\n107\n108\n109\n110\n111\n112\n113\n114\n115\n116\n117\n118\n119\n120\n121\n122\n123\n124\n125\n126\n127\n128\n129\n130\n131\n132\n133\n134\n135\n136\n137\n138\n139\n140\n141\n142\n143\n144\n145\n146\n147\n148\n149\n150\n151\n152\n153\n154\n155\n156\n157\n158\n159\n160\n161\n162\n163\n164\n165\n166\n167\n168\n169\n170\n171\n172\n173\n174\n175\n176\n177\n178\n179\n180\n181\n182\n183\n184\n185\n186\n187\n188\n189\n190\n191\n192\n193\n194\n195\n196\n197\n198\n199\n200\n201\n202\n203\n204\n205\n206\n207\n208\n209\n210\n211\n212\n213\n214\n215\n216\n217\n218\n219\n220\n221\n222\n223\n224\n225\n226\n227\n228\n229\n230\n231\n232\n233\n234\n235\n236\n237\n238\n239\n240\n241\n242\n243\n244\n245\n246\n247\n248\n249\n250\n251\n252\n253\n254\n255\n256\n257\n258\n259\n260\n261\n262\n263\n264\n265\n266\n267\n268\n269\n270\n271\n272\n273\n274\n275\n276\n277\n278\n279\n280\n281\n282\n283\n284\n285\n286\n287\n288\n289\n290\n291\n292\n293\n294\n295\n296\n297\n298\n299\n300\n301\n302\n303\n304\n305\n306\n307\n308\n309\n310\n311\n312\n313\n314\n315\n316\n317\n318\n319\n320\n321\n322\n323\n324\n325\n326\n327\n328\n329\n330\n331\n332\n333\n334\n335\n336\n337\n338\n339\n340\n341\n342\n343\n344\n345\n346\n347\n348\n349\n350\n351\n352\n353\n354\n355\n356\n357\n358\n359\n360\n361\n362\n363\n364\n365\n366\n367\n368\n369\n370\n371\n372\n373\n374\n375\n376\n377\n378\n379\n380\n381\n382\n383\n384\n385\n386\n387\n388\n389\n390\n391\n392\n393\n394\n395\n396\n397\n398\n399\n400\n401\n402\n403\n404\n405\n406\n407\n408\n409\n410\n411\n412\n413\n414\n415\n416\n417\n418\n419\n420\n421\n422\n423\n424\n425\n426\n427\n428\n429\n430\n431\n432\n433\n434\n435\n436\n437\n438\n439\n440\n441\n442\n443\n444\n445\n446\n447\n448\n449\n450\n451\n452\n453\n454\n455\n456\n457\n458\n459\n460\n461\n462\n463\n464\n465\n466\n467\n468\n469\n470\n471\n472\n473\n474\n475\n476\n477\n478\n479\n480\n481\n482\n483\n484\n485\n486\n487\n488\n489\n490\n491\n492\n493\n494\n495\n496\n497\n498\n499\n500\n501\n502\n503\n504\n505\n506\n507\n508\n509\n510\n511\n512\n513\n514\n515\n516\n517\n518\n519\n520\n521\n522\n523\n524\n525\n526\n527\n528\n529\n530\n531\n532\n533\n534\n535\n536\n537\n538\n539\n540\n541\n542\n543\n544\n545\n546\n547\n548\n549\n550\n551\n552\n553\n554\n555\n556\n557\n558\n559\n560\n561\n562\n563\n564\n565\n566\n567\n568\n569\n570\n571\n572\n573\n574\n575\n576\n577\n578\n579\n580\n581\n582\n583\n584\n585\n586\n587\n588\n589\n590\n591\n592\n593\n594\n595\n596\n597\n598\n599\n600\n601\n602\n603\n604\n605\n606\n607\n608\n609\n610\n611\n612\n613\n614\n615\n616\n617\n618\n619\n620\n621\n622\n623\n624\n625\n626\n627\n628\n629\n630\n631\n632\n633\n634\n635\n636\n637\n638\n639\n640\n641\n642\n643\n644\n645\n646\n647\n648\n649\n650\n651\n652\n653\n654\n655\n656\n657\n658\n659\n660\n661\n662\n663\n664\n665\n666\n667\n668\n669\n670\n671\n672\n673\n674\n675\n676\n677\n678\n679\n680\n681\n682\n683\n684\n685\n686\n687\n688\n689\n690\n691\n692\n693\n694\n695\n696\n697\n698\n699\n700\n701\n702\n703\n704\n705\n706\n707\n708\n709\n710\n711\n712\n713\n714\n715\n716\n717\n718\n719\n720\n721\n722\n723\n724\n725\n726\n727\n728\n729\n730\n731\n732\n733\n734\n735\n736\n737\n738\n739\n740\n741\n742\n743\n744\n745\n746\n747\n748\n749\n750\n751\n752\n753\n754\n755\n756\n757\n758\n759\n760\n761\n762\n763\n764\n765\n766\n767\n768\n769\n770\n771\n772\n773\n774\n775\n776\n777\n778\n779\n780\n781\n782\n783\n784\n785\n786\n787\n788\n789\n790\n791\n792\n793\n794\n795\n796\n797\n798\n799\n800\n801\n802\n803\n804\n805\n806\n807\n808\n809\n810\n811\n812\n813\n814\n815\n816\n817\n818\n819\n820\n821\n822\n823\n824\n825\n826\n827\n828\n829\n830\n831\n832\n833\n834\n835\n836\n837\n838\n839\n840\n841\n842\n843\n844\n845\n846\n847\n848\n849\n850\n851\n852\n853\n854\n855\n856\n857\n858\n859\n860\n861\n862\n863\n864\n865\n866\n867\n868\n869\n870\n871\n872\n873\n874\n875\n876\n877\n878\n879\n880\n881\n882\n883\n884\n885\n886\n887\n888\n889\n890\n891\n892\n893\n894\n895\n896\n897\n898\n899\n900\n901\n902\n903\n904\n905\n906\n907\n908\n909\n910\n911\n912\n913\n914\n915\n916\n917\n918\n919\n920\n921\n922\n923\n924\n925\n926\n927\n928\n929\n930\n931\n932\n933\n934\n935\n936\n937\n938\n939\n940\n941\n942\n943\n944\n945\n946\n947\n948\n949\n950\n951\n952\n953\n954\n955\n956\n957\n958\n959\n960\n961\n962\n963\n964\n965\n966\n967\n968\n969\n970\n971\n972\n973\n974\n975\n976\n977\n978\n979\n980\n981\n982\n983\n984\n985\n986\n987\n988\n989\n990\n991\n992\n993\n994\n995\n996\n997\n998\n999"
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/result.rs:1654:5
   3: core::result::Result<T,E>::unwrap_err
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/core/src/result.rs:1171:22
   4: integration_tests::batch::test_serving_cluster_availability::{{closure}}::{{closure}}
             at ./tests/integration_tests/batch/mod.rs:138:5
   5: madsim::sim::task::Executor::block_on::{{closure}}::{{closure}}
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:230:32
   6: async_task::raw::RawTask<F,T,S,M>::run
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/raw.rs:563:17
   7: async_task::runnable::Runnable<M>::run
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/runnable.rs:782:18
   8: madsim::sim::task::Executor::run_all_ready::{{closure}}
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:271:59
   9: std::panicking::try::do_call
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panicking.rs:552:40
  10: std::panicking::try
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panicking.rs:516:19
  11: std::panic::catch_unwind
             at /rustc/4a0cc881dcc4d800f10672747f61a94377ff6662/library/std/src/panic.rs:146:14
  12: madsim::sim::task::Executor::run_all_ready
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:271:17
  13: madsim::sim::task::Executor::block_on
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/task/mod.rs:238:13
  14: madsim::sim::runtime::Runtime::block_on
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/runtime/mod.rs:126:9
  15: madsim::sim::runtime::builder::Builder::run::{{closure}}::{{closure}}::{{closure}}
             at /risingwave/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.27/src/sim/runtime/builder.rs:128:35

The failure can be triggered by madsim bump. https://github.com/risingwavelabs/risingwave/pull/16760

Try longevity buildkite.com/risingwave-test/longevity-test/builds/1394

This looks good. I think we can merge after madsim issue is resolved.

Seems good to go?

Let's go!


OTIS Crash Data?

On the Fatal Crashes dataset it says this:

For those conducting analysis, this dataset by PPD and OTIS’ crash data should not be compared, or should be used together cautiously.

However that link to OTIS crash data no longer works. Is that dataset no longer available? The only other dataset I see that covers all crashes (not just fatal crashes) is this one: https://opendataphilly.org/datasets/vehicular-crashes/. However that appears to end in 2020.
Which dataset is the most comprehensive and up to date crash data set?

Hi @kettle11 - I submitted a PR to update that link to the OTIS data entry - you are right that it was meant to connect to the /vehicular-crashes dataset. OTIS also recently published the latest data, so I submitted a pull request for that as well. In the interim, you can find those download links on the metadata for this dataset here: https://metadata.phila.gov/#home/datasetdetails/5543865420583086178c4eba/representationdetails/64f22d9087c9db002570f98a/

@Kistine Thank you for the prompt and helpful response! It's much appreciated.

Fix is live, thanks for reporting the issue! Thanks Kistine for fixing it so fast!


Support primary keys that aren't named id

I understand that when giving a order_by: column other than the default :id, extra care is required to ensure that unique results are accounted for because we cannot ensure that the user specified column is unique, and hence adding :id is required. However, I am thinking that the 'spirit' of this behavior is really not about :id specifically but about if we are sorting by the table's primary key or not. I have a few cases where unfortunately the table's primary key is not called :id, and in fact also does not even have a column named :id, and hence the 'more complex' sorting case is triggered and fails because there is no :id column.
I may or may not take a swing at this change if it seems worthwhile, but at a glance it seems that in Paginator something like order_by ||= :id could be replaced with order_by ||= relation.primary_key.to_sym, and then a few more spots also would have to be updated where we are relying on the behavior that the primary key is :id.

Yeah, I have the same issue.

Hello @Aesthetikx. This is an issue that popped up a couple of times already, and not an uncommon use case either. I tried to introduce support for custom primary keys before but hit a roadblock when I had to change the tests, which are unfortunately not very maintainable. Feel free to open a new PR and to take a look at my attempt from last year here but keep in mind that it may require some major refactoring of the tests 🙁
I remember that in the end I discovered it wasn't worth it or needed for my specific use case so I ended up dropping the change but I think it would be a very valuable addition to this gem.

@Aesthetikx maybe you already found, but there is a "fork" that has what we want: https://github.com/healthie/activerecord_cursor_paginate


Merge @groupBy and @http

Currently groupBy operator is used for batching and only supports Http. Even the implementation currently is very http specific. We can merge them.
Before:
type Post {
  id: Int!
  userId: Int!
  title: String!
  body: String!
  user: User @http(path: "/users", query: [{key: "id", value: "{{value.userId}}"}]) @groupBy(key: "id", path: ["id"])
}

After:
type Post {
  id: Int!
  userId: Int!
  title: String!
  body: String!
  user: User
    @http(
      path: "/users", query: [{key: "id", value: "{{value.userId}}"}],
      batch: {
        groupByPath: ["id"],
        selectKey: "id"
      }
    )
}

Hi @tusharmath, I'd like to work on this. So that I can understand more tailcall's codebase.
Do you mind assigning me, please ?

@tobihans Fantastic! Let us know on the discord channel, if you get stuck or need any kind of help understanding some bits.

I will, thank you !


Remove version: bornova.numara version 3.18.5

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/bornova/numara-calculator/releases/download/v3.18.5/Numara-3.18.5-x64.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 910)


Refactor repo checking

makes sense. 👍
One question / idea: would it make sense to include it directly in the shellutils? maybe that's overkill, but I imagine that most of the time it'll be better/safer to call it from there (even if not strictly required).

That's a good point. Benefits of being on its own: can be called without sourcing shellutils. Drawbacks: needs more boilerplate, and pollutes libexec.


Fix lxd module

Motivation for this change
LXD systemd service wouldn't start due to (recent?) Nixpkgs Go changes.
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

Thanks!


BUG: Fix datetime conversion for timin/tmax=None cases in Annotations.crop

What does this implement/fix?
In Annotations.crop function,
onset values must be passed as seconds instead of dates (first argument).
Another change allows passing float and None at the same time.
Currently, if tmin or tmax is None, the value is obtained as datetime and it can not be compared with float.
By calculating absolute values before checking tmin > tmax, float can be used with None.
Additional information
In most cases, this bug does not affect
because original annotations data has tmin=0, and too large tmax just allow all period.

@rcmdnk can I ask you to add a bug fix entry in latest.inc file? so the bug fix gets documented in the what's new page / release notes? 🙏

@agramfort
done!

@agramfort
Added test_crop_with_none in mne/tests/test_annotations.py
There are two checks:

crop(tmin=None, tmax=None), which should remain as is.

It fails if orig_time != 0 at the previous version.

crop(tmin=<float>, tmax=None) which was reflected at the previous version.

thanks a lot @rcmdnk


chore(doc): doc for custom subcommands look up.

What does this PR try to resolve?
as the https://github.com/rust-lang/cargo/issues/13194 metions, the lookup rules for custom subcommands are only reflected in comments inside the code, and it is time to inform users of this behavior through documentation.
How should we test and review this PR?
Additional information

Thank you :)
@bors r+

:pushpin: Commit d71b2bc5dfd1e3b840e705053be5a0b3e0967827 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit d71b2bc5dfd1e3b840e705053be5a0b3e0967827 with merge 1f7059724151cc013fbc0b8b5875bee5a90614a1...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 1f7059724151cc013fbc0b8b5875bee5a90614a1 to master...


dom/TagName.MAIN missing?

I think the tag name MAIN is missing in the enums.

Great that you look at the source code and make comments.
But it would be better if you'd figure out what commit actually introduced HTML5 tags and from there on you could go figure out why it's missing.
You could then open a PR and fix this stating why this was missing and why you need it and also add all other tag names that are missing while you're at it.
For example the HTML5 tag names where added to this project nearly five years ago in 03451dec299109612bf33376809ef4edca2a6883 and they were taken from the HTML5 draft specification that is linked at tagname.js#L22. In that draft the main tag was not proposed yet therefore it was not added back then.
You're welcome to open a PR and add tag names according to the current specification.
I would be very astonished if one of the guys who maintain this project as part of their job would look into it in that much detail based on a single sentence you drop in an issue. So please try to do some research on your own.

@b-strauss @binarykitchen and everyone else sorry for my toxic behavior. @pauldraper thanks for taking care of this.


Check if peer is banned and when refreshing masterfile

Fixes: #1423
Don't include the node in masterlist when it is banned
Put the banning check in the refresh loop - which happens every 30 seconds.
Therefore there is upto a 30 second delay where a banned node would still be returned.
[And demonstration of NSubstitute in one test for @dangershony to show syntax difference to Moq]

I have tested this on.... <IP_ADDRESS>   @StratisIain @dangershony

That server is currently running this PR branch.


Initial and maximum TCPConnection read buffer sizes

In TCPConnection, the initial read buffer size is 64 bytes, and it grows when a full buffer is read. It would be good to make the initial size configurable as a constructor parameter, as well as being able to specify a maximum read buffer size.

This should also apply to UDPSocket - it should have the same read buffer options as TCPConnection.

@jemc there is already "size" parameter.

@Nyarum thanks for your help in closing this.


XML/XPath library clarification

Currently we depend on all of the following for XML handling:

simple-xml-2.6.jar
jdom-1.0.jar
jaxen-1.1-beta-10.jar
dom4j-1.6.1.jar

Can someone clarify for me the need for all 4? Would narrowing to one XPath expression parser and one xml parser be beneficial? If this isn't possible, why?

Hi Cannon,
Yes, one parser would absolutely be beneficial. Sakai is a kernel + tools project so tool developers have the freedom to choose their own dependencies. If you were interested in consolidating these parsers, the contribution would definitely be appreciated, reviewed, and merged if solid.

I was looking at a potential consolidation job. I have some time this weekend so I'll see what can be done.

After some initial investigation, I'd like to create a few utils within the kernel to handle XML operations. Seeing as though hibernate requires dom4j, I think dropping jaxen in favor of dom4j for XPath support would be the best option. Any qualms there?


Export multiple tables to a single compressed file

See #1658 for context details

@Kartones @javisantana @rochoa @xavijam AFAIK the current APIs don't support exporting multiple files at once, do they? this is required for the new export-datasets dialog (see linked issue)

Currently is the SQL Api the one that exports a table, so the issues here are:

Can SQL API multiple files? And more importantly, should it have batching support and do multiple exports? IMO no, but is not my call.
Should we (me) implement a new REST api endpoint that you send to multiple table names and it exports the tables, and then zips the file and then returns to the user? I see it a better option, but this raises other serious concerns.

Current frontend servers have small space. Current import servers are for imports. So either we put this data at import servers or need to add "export servers".
This will also mean new code, new jobs to fetch, compress and cleanup data.
This also means forget about instant download for multi-file downloads (anything that is not a single table download=. UI should be purely async, more like "we'll send you an email when the export is ready", because this will take more even with two small tables, having to spawn jobs, etc..
How much time do we keep the zipped file available for download? 24 hours? 48?
And probably more issues, but let's first attack "big problems".

ccing @lbosque as this also means changes to systems.
As I told @viddo, this is not trivial, selecting N items for deletion means N calls to a DELETE endpoint, exporting N items in a single file is way harder to do, possible but not quick to implement.

for the moment let's forget this.
The way to implement this would be to have a batch thing in the SQL API

Ok, so we remove the export option from the modal (we keep the text but not the link if there are multiple tables, and we remove this option also from the submenu when selecting multiple datasets)
Hey @viddo close when done :P

There are no links to export datasets, and the dialog already looks like that, so nothing more to do then, closing issue