Add ability to launch DS UI in a web browser

For #10655
For #10656
Notes:

The requirement is to launch UI in a browser
This is primarily for testing purposes,
I.e. we'll need to debug this for tests, meaning we'll need to copy URL a lot, hence I added an additional message to display & copy the URL (based on an environment variable).
I.e. none of this goes into production, necessary packages used in dev will get excluded from vsix.

First step for UI tests in ipywidgets

export class WebPanel implements IWebPanel {

Would it be better to just have another implementation of IWebPanel that starts the server instead of opening a VS webview?
Then the webPanelProvider would just serviceContainer.get and it would just work based on the container?

Refers to: src/client/common/application/webPanels/webPanel.ts:22 in 0c4d7fb. [](commit_id = 0c4d7fb59da07f4b93b43ed57108ddedb04642cb, deletion_comment = False)

export class WebPanel implements IWebPanel {

Essentially you're shortcutting the vs code stuff here if there's a webBrowserPanel

In reply to: 600905824 [](ancestors = 600905824)

Refers to: src/client/common/application/webPanels/webPanel.ts:22 in 0c4d7fb. [](commit_id = 0c4d7fb59da07f4b93b43ed57108ddedb04642cb, deletion_comment = False)

Codecov Report

Merging #10657 into ds/ipyWidgets will increase coverage by 0.08%.
The diff coverage is 10.44%.

@@                Coverage Diff                @@
##           ds/ipyWidgets   #10657      +/-   ##
=================================================
+ Coverage          60.45%   60.53%   +0.08%
=================================================
  Files                583      581       -2
  Lines              31655    31572      -83
  Branches            4504     4495       -9
=================================================
- Hits               19136    19112      -24
+ Misses             11545    11488      -57
+ Partials             974      972       -2

Impacted Files
Coverage Δ

src/client/datascience/webViewHost.ts
9.33% <0%> (ø)
:arrow_up:

...nt/common/application/webPanels/webBrowserPanel.ts
10.9% <10.9%> (ø)

...rc/client/common/application/webPanels/webPanel.ts
11.76% <11.11%> (-0.08%)
:arrow_down:

...atascience/interactive-window/interactiveWindow.ts
15.48% <0%> (-1.61%)
:arrow_down:

...lient/datascience/jupyter/liveshare/serverCache.ts
65.27% <0%> (-1.39%)
:arrow_down:

...ient/datascience/interactive-ipynb/nativeEditor.ts
8.69% <0%> (-0.66%)
:arrow_down:

...science/jupyter/liveshare/guestJupyterExecution.ts
18% <0%> (-0.37%)
:arrow_down:

...ractive-window/interactiveWindowCommandListener.ts
57.85% <0%> (-0.25%)
:arrow_down:

...active-common/intellisense/intellisenseProvider.ts
36.92% <0%> (-0.17%)
:arrow_down:

...lient/datascience/jupyter/jupyterSessionManager.ts
7.96% <0%> (-0.15%)
:arrow_down:

... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d95970f...0488b66. Read the comment docs.

@rchiodo  Please re-review.
Its not complete, need to fix issues with IoC Container. Then need to write tests.
Have a basic test where we check if all 3 cells are visible and we run a cell.
Once I've fixed IOC and other hacks, i'll complete this, and add tests for ipywidgest separately (at that point its just writing tests)


chore(deps): update locks

v14.12.2 has been released and contains a fix for the issue. Feel free to reopen the issue or to submit a new one if you meet any problems.


proto run npm/pnpm/yarn fails with program not found if bin not in path

What version?
0.23.5
Which command?
No response
What happened?
> proto run pnpm
Error:   × program not found

My path includes ~/.proto/shims but NOT ~/.proto/bin per here.
node_depman_plugin version 0.5.3
node_plugin version 0.5.3
Trace logs?
$ proto run pnpm
[DEBUG 2023-11-30 11:23:44] proto  Running proto v0.23.5  args=["C:\\Users\\dan\\.proto\\shims\\proto.exe", "run", "pnpm", "--log", "trace"]
[TRACE 23:44.785] starbase::app  Running startup phase with 1 systems
[TRACE 23:44.785] starbase::app  Running analyze phase with 1 systems
[TRACE 23:44.786] starbase::app  Running execute phase with 2 systems
[DEBUG 23:44.790] proto_core::user_config:run  Loading config.toml  file="C:\\Users\\dan\\.proto\\config.toml"
[TRACE 23:44.791] starbase_utils::fs:run  Reading file  file="C:\\Users\\dan\\.proto\\config.toml"
[DEBUG 23:44.791] proto_core::tool_loader:run  Traversing upwards to find a configured plugin  tool="pnpm"
[DEBUG 23:44.792] proto_core::tools_config:run  Loading .prototools  file="C:\\Users\\dan\\.prototools"
[TRACE 23:44.792] starbase_utils::fs:run  Reading file  file="C:\\Users\\dan\\.prototools"
[TRACE 23:44.802] warpgate::loader:run  Creating plugin loader  cache_dir="C:\\Users\\dan\\.proto\\plugins"
[TRACE 23:44.802] warpgate::loader:run  Loading plugin pnpm  plugin="pnpm"
[TRACE 23:44.803] warpgate::loader:run  Plugin already downloaded and cached  plugin="pnpm" path="C:\\Users\\dan\\.proto\\plugins\\pnpm-50fe3b6c66bfa9dd0663411862e9dd8fb7e301f8387f8dcd03a890094c977902.wasm"
[DEBUG 23:44.803] proto_core::tool_loader:run  Loading WASM plugin  source="C:\\Users\\dan\\.proto\\plugins\\pnpm-50fe3b6c66bfa9dd0663411862e9dd8fb7e301f8387f8dcd03a890094c977902.wasm"
[TRACE 23:44.804] proto_core::tool_loader:run  Storing tool identifier  id="pnpm"
[TRACE 23:44.804] proto_core::tool_loader:run  Storing user configuration  config={"auto_clean":true,"auto_install":true,"node_intercept_globals":true}
[TRACE 23:44.805] proto_core::tool_loader:run  Storing proto environment  env={"arch":"x64","os":"windows","home_dir":{"path":"/userhome/","virtual_prefix":"/userhome","real_prefix":"C:\\Users\\dan"},"proto_dir":{"path":"/proto/","virtual_prefix":"/proto","real_prefix":"C:\\Users\\dan\\.proto"}}
[DEBUG 23:44.805] proto_core::tool:run  Creating tool pnpm and instantiating plugin
[DEBUG 23:44.806] proto_core::tool_manifest:run:load  Loading manifest.json  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\manifest.json"
[TRACE 23:44.806] starbase_utils::fs:run:load  Opening file  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\manifest.json"
[TRACE 23:44.807] starbase_utils::fs_lock:run:load  Locking file  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\manifest.json"
[TRACE 23:44.808] starbase_utils::fs_lock:run:load  Unlocking file  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\manifest.json"
[DEBUG 23:45.017] proto_core::tool:run  Created tool pnpm and its WASM runtime
[TRACE 23:45.018] warpgate::plugin:run  Calling plugin function register_tool  plugin="pnpm" input={"id":"pnpm"}
[TRACE 23:45.019] warpgate::plugin:run  Called plugin function register_tool  plugin="pnpm" output={"inventory":{},"name":"pnpm","plugin_version":"0.5.3","type":"DependencyManager"}
[TRACE 23:45.019] proto_core::version_detector:run  Attempting to find local version from config files  tool="pnpm"
[DEBUG 23:45.020] proto_core::user_config:run  Loading config.toml  file="C:\\Users\\dan\\.proto\\config.toml"
[TRACE 23:45.020] starbase_utils::fs:run  Reading file  file="C:\\Users\\dan\\.proto\\config.toml"
[TRACE 23:45.021] proto_core::version_detector:run  Checking directory  tool="pnpm" dir="C:\\Users\\dan"
[DEBUG 23:45.021] proto_core::tools_config:run  Loading .prototools  file="C:\\Users\\dan\\.prototools"
[TRACE 23:45.022] starbase_utils::fs:run  Reading file  file="C:\\Users\\dan\\.prototools"
[DEBUG 23:45.023] proto_core::version_detector:run  Detected version from .prototools file  tool="pnpm" version="latest"
 file="C:\\Users\\dan\\.prototools"
[DEBUG 23:45.024] proto_core::user_config:run  Loading config.toml  file="C:\\Users\\dan\\.proto\\config.toml"
[TRACE 23:45.025] starbase_utils::fs:run  Reading file  file="C:\\Users\\dan\\.proto\\config.toml"
[DEBUG 23:45.025] proto_core::tool:run  Resolving a semantic version or alias  tool="pnpm" initial_version="latest"
[DEBUG 23:45.026] proto_core::tool:run  Loading available versions  tool="pnpm"
[TRACE 23:45.026] starbase_utils::fs:run  Reading file metadata  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\remote-versions.json"
[DEBUG 23:45.027] proto_core::tool:run  Loading from local cache  tool="pnpm" cache="C:\\Users\\dan\\.proto\\tools\\pnpm\\remote-versions.json"
[TRACE 23:45.027] starbase_utils::fs:run  Reading file  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\remote-versions.json"
[TRACE 23:45.028] starbase_utils::json:run  Parsing JSON  file="C:\\Users\\dan\\.proto\\tools\\pnpm\\remote-versions.json"
[TRACE 23:45.029] warpgate::plugin:run  Calling plugin function resolve_version  plugin="pnpm" input={"initial":"latest"}
[TRACE 23:45.030] warpgate::plugin:run  Called plugin function resolve_version  plugin="pnpm" output={}
[DEBUG 23:45.030] proto_core::tool:run  Resolved to 8.11.0  tool="pnpm" version="8.11.0"
[DEBUG 23:45.031] proto_core::tool:run  Checking if tool is installed  tool="pnpm" install_dir="C:\\Users\\dan\\.proto\\tools\\pnpm\\8.11.0"
[DEBUG 23:45.032] proto_core::tool:run  Tool has already been installed, locating binaries and shims  tool="pnpm" install_dir="C:\\Users\\dan\\.proto\\tools\\pnpm\\8.11.0"
[DEBUG 23:45.032] proto_core::tool:run  Locating executable for tool  tool="pnpm"
[TRACE 23:45.033] warpgate::plugin:run  Calling plugin function locate_executables  plugin="pnpm" input={"context":{"tool_dir":{"path":"/proto/tools/pnpm/8.11.0","virtual_prefix":"/proto","real_prefix":"C:\\Users\\dan\\.proto"},"version":"8.11.0"}}
[TRACE 23:45.034] warpgate::plugin:run  Called plugin function locate_executables  plugin="pnpm" output={"globals_lookup_dirs":["$PROTO_HOME/tools/node/globals/bin"],"primary":{"exe_path":"bin/pnpm.cjs","no_bin":true,"parent_exe_name":"node"},"secondary":{"pnpx":{"no_bin":true,"shim_before_args":"dlx"}}}
[DEBUG 23:45.034] proto_core::tool:run  Found an executable  tool="pnpm" exe_path="C:\\Users\\dan\\.proto\\tools\\pnpm\\8.11.0\\bin/pnpm.cjs"
[TRACE 23:45.035] warpgate::plugin:run  Calling plugin function pre_run  plugin="pnpm" input={"context":{"tool_dir":{"path":"/proto/tools/pnpm/8.11.0","virtual_prefix":"/proto","real_prefix":"C:\\Users\\dan\\.proto"},"version":"8.11.0"},"passthrough_args":[]}
[TRACE 23:45.036] warpgate::plugin:run  Called plugin function pre_run  plugin="pnpm" output=
[DEBUG 23:45.036] proto::commands::run:run  Running pnpm  bin="node.exe" args=["C:\\Users\\dan\\.proto\\tools\\pnpm\\8.11.0\\bin/pnpm.cjs"]
[TRACE 23:45.042] starbase::app  Running shutdown phase with 1 systems
Error:   × program not found

Operating system?
Windows
Architecture?
x64

@rotu Ah damn. This is a troublesome one.
In the last PR (https://github.com/moonrepo/proto/pull/308), I made the change that tools that require a parent (mainly npm/pnpm/yarn) will be executed with the parent .exe. But since you removed the ~/.proto/bin folder, that .exe is no longer available.
This change was made because otherwise, we have nested shim executions, which cause problems with CTRL+C and signals on Windows. For example, this is a standard flow.

shim: npm install -> Runs ~/.proto/shims/npm.cmd, which in turn runs proto run npm -- install.

not shim: proto run npm -- install -> Requires node, so runs node ~/.proto/tools/npm/.../cli.js install under the hood.

shim: node ~/.proto/tools/npm/.../cli.js install -> Runs ~/.proto/shims/node.cmd ..., which causes a bunch of problems.
not shim: proto run node -- ~/.proto/tools/npm/.../cli.js install.

As you can see, it kind of just keeps going all the way down. The intermediate shims are problematic, so I updated it to skip the shims.
I'm open to feedback on this. Windows shims seem to be causing a bunch of problems and I'm not 100% sure what the ideal path forward is now. Some ideas:

Add settings for controlling which shims are created: .cmd, .ps1, etc.
Add settings for toggling shims/bin creation of or off.
Investigate a new shim pattern for Windows.

Testing here, this may work: https://github.com/moonrepo/proto/pull/313

I thought the "terminate batch job?" was the main shim issue. Admittedly I don't understand the cmd shim hack at all right now:

The problem was 2 things: 1) hitting ctrl+c either wouldn't work, or you had to spam it constantly for the process to exit. for the most part, you had to open a new terminal window. 2) when it did exit with ctrl+c, the terminal history is lost for some reason. Also required a new terminal window.

That does sound crummy.
I found an explanation of the hack: https://github.com/microsoft/terminal/issues/217
It creates a fatal error (goto an undefined label) and runs the command in the shell process right before it burns down! I'm not at all surprised it splatters the history!
I recommend taking another look at this alternate solution. It does seem to play nice with npx (I think it was an argument handling thing) and I think it will nest better. Instead of running the executable in the process as it's going down, you're directly intercepting the error code before it gets back to the shell and then exiting gracefully:
setlocal enabledelayedexpansion
proto.exe run node -- %* & set RC=!errorlevel! & call;
EXIT /b %RC%

Had another idea. Maybe we resolve it to proto run node instead of node.exe. This assumes that the tool IDs match up though, which is probably ok for the node case.

Yeah! I think that's a very reasonable solution.

The .ps1 files actually work great. The problem is the .cmd files.

But the .cmd files are necessary since processes look for node which resolves to node.cmd. The only thing that uses the .ps1 files is a PowerShell session, where a function or Cmdlet provides a better user experience anyway.

And the other problem is that .ps1 is not in PATHEXT, so node wouldn't run node.ps1, only node.cmd.
We could modify PATHEXT on install, but that's a can of worms.

You need to ensure .ps1 is in PATHEXT (so the user can omit the extension), ensure that the default shell handler is to execute the script file with PowerShell and not edit it with a text editor (maybe not a great idea?), and make sure the script gets executed in the current PowerShell process or a subshell, not a new window.

Just had a thought, thanks to this comment:

An .exe shim doesn't necessarily need to be a new executable for each file. It could just be a symlink to proto.exe which acts based on argv[0] to delegate to the underlying tool binary.

Maybe, just maybe, we can provide 2 executables: proto(.exe) and proto-shim(.exe). This could use the argv[0] hack to "determine the correct tool" to run, and simply call proto under the hood. We could probably then eliminate bash/ps1/cmd shims entirely for both OS's.

Just had a thought, thanks to this comment:

An .exe shim doesn't necessarily need to be a new executable for each file. It could just be a symlink to proto.exe which acts based on argv[0] to delegate to the underlying tool binary.

Maybe, just maybe, we can provide 2 executables: proto(.exe) and proto-shim(.exe). This could use the argv[0] hack to "determine the correct tool" to run, and simply call proto under the hood. We could probably then eliminate bash/ps1/cmd shims entirely for both OS's.

Now we're on the same page! I said one executable, but I don't have a strong feeling about one or two. This might be able to fully replace the current shims, but there's some references to platform-specific behavior, so I'm not sure where it wouldn't work (or if we even care about those platforms):

If the executable was invoked through a symbolic link, some platforms will return the path of the symbolic link and other platforms will return the path of the symbolic link’s target.

and here

The first element is traditionally the path of the executable, but it can be set to arbitrary text, and might not even exist. This means this property should not be relied upon for security purposes.

Yeah I'll do some testing and see what's feasible. I was thinking a secondary shim just so it's super lightweight.
As for this issue, can you try 0.23.6 and see if that works?

Testing some stuff here. https://github.com/moonrepo/proto/pull/315
If there's certain flows you think I should test (besides signals), let me know.

Good news, both unix/windows resolves the arg name correctly. Also got ctrl+c working super easily. https://github.com/moonrepo/proto/blob/0.24-shim/crates/cli-shim/README.md
I'll test piping/redirects etc too, but this seems like a good win. The file size is also 400kb.

Yeah I'll do some testing and see what's feasible. I was thinking a secondary shim just so it's super lightweight.
As for this issue, can you try 0.23.6 and see if that works?

It works! Only two minor issues:

Since I moved proto.exe to shims, proto upgrade merely installs the new proto instance to bin which is not in my path. Recommend instead symlinking proto.exe into my path:

New-Item -Path $env:PROTO_HOME\shims\proto.exe -ItemType SymbolicLink -Value $env:PROTO_HOME\bin\proto.exe

After uninstalling node and npm, proto run npm did not reinstall node. This is probably okay.

Was going to land this change in the next moon version, but I can probably land it sooner: https://github.com/moonrepo/moon/pull/1208/files#diff-8804ae5ea9955fa400fa6ff23b4e911bb4fc5451e09244b0beec7aac21286222R25

Was going to land this change in the next moon version

I don't have a strong feeling about this but I'm excited that moon is making progress!
Admittedly, I still don't understand moon and I don't use it in any meaningful way! I work on a npm-based monorepo project under Mercurial source control. I can put proto through its paces, but not (yet) moon.

It was mainly about adding the PROTO_INSTALL_DIR env var. This way, you can install it to a custom location.
I'm going to close this. Let's keep the shim convo in that new PR.


Add header - group task

Add header , font ,text and images.

Add header.
Add font.
Add text.
Add images.

You have done a great job. Please make sure linters are passing. In your case, Stylelint is failing

Also, our project is about movies not restaurant. Please Stick to topic. If you clone the repository and checkout to "display-items-homepage" branch and open in live server, you will be able to see the homepage I already created


[zh] Sync1.25 /tasks/job/pod-failure-policy.md

Added new zh text to this topic:
content/zh-cn/docs/tasks/job/pod-failure-policy.md

Preview: https://deploy-preview-36602--kubernetes-io-main-staging.netlify.app/zh-cn/docs/tasks/job/pod-failure-policy/

/approve

/lgtm


Template file path cleanup

Adds documentation, tests, and error handling to #56

Coverage remained the same at 100.0% when pulling b9079bdb8295af024a6417fb455fc028561e2ddc on template-file-cleanup into c1f781d553e40ae10421941665fea7eb109bf369 on develop.

Coverage remained the same at 100.0% when pulling b9079bdb8295af024a6417fb455fc028561e2ddc on template-file-cleanup into c1f781d553e40ae10421941665fea7eb109bf369 on develop.

Coverage remained the same at 100.0% when pulling b8159c66d4ef825207c73315bab3b1ea53103638 on template-file-cleanup into c1f781d553e40ae10421941665fea7eb109bf369 on develop.

Coverage remained the same at 100.0% when pulling b8159c66d4ef825207c73315bab3b1ea53103638 on template-file-cleanup into c1f781d553e40ae10421941665fea7eb109bf369 on develop.

Coverage remained the same at 100.0% when pulling b8159c66d4ef825207c73315bab3b1ea53103638 on template-file-cleanup into c1f781d553e40ae10421941665fea7eb109bf369 on develop.


Evaluate replacement for React-Konva

React-Konva causes issues with the debug build (it makes the browsertab with grafana crash and freeze completely) and doesn´t add that much value overall.  We should consider using other options like directly using the canvas API (which maybe would also allow for later integration of 3d visualisation) or utilise other libraris.
If we decide to stick with Konva we should do further investigations what exactly causes the dev build to crash the browsertab and see if we can up with solutions.

Possible Replacements for Konva
react-zoom-pan-pinch
Example: https://codesandbox.io/s/qmn69?file=/src/index.js
Under-the-hood: normal html elements utilizes css transform to zoom and pinch
Pro

easy to use
supports "normal" dom elements - no canvas required

this means dom events are available
grafana components can be used as well

Con

positioning and sizing of elements could turn out difficult
scales not great with ~200+ elements

Three.js / react-three-fiber
Example: https://codesandbox.io/s/react-three-fiber-react-spring-svg-parallax-55tzr?file=/src/index.js:211-228
Under-the-hood: html5 canvas
Pro

good performance
2d - 3d possible

Con

not exactly "easy to use" / steep learning curve
2d is possible but focus of the library is on 3d rendering

using HTML5 canvas direclty (without library)
Example<EMAIL_ADDRESS>Pro

free control about rendering

allows for later implementation of 3d rendering

scales very well even with ~10k elements

e.g. through zooming unvisible elements could be filtered befor rendering to boost performance

Con

higher implementation effort (manual implementation of:)

panning and zooming
click, hover and other events regarding components

instead of using jsx Elements everything needs to be "drawn" (a function call on the canvas context) => could result in unclear code

vx/ visx (React + D3.js implementation)
Example: https://airbnb.io/visx/zoom-i
Under-the-hood: html svg
Pro

reasonable performant (with svg ~2000 elements)
benefits of DOM objects when working with svg (events)
rather ease to use
lightweight library / modular import possible

Con

spread documentation (part d3 part vx)

Summary
Based on the collected pros and cons visx meets our needs the most.
But befor jumping to the implementation we will first rethink our internal data structure and only than revisit this.

Since we decided in our last meeting to continue development with vx/ visx. Im closing this issue. Integration will follow with the next big PR.


Generate checksums from file

Without a additional tool you can´t fastly verfiy files with a checksum. You can only use the cmd:
eg. CMD:
certutil -hashfile  SHA256
Possible algorithm:
MD2 MD4 MD5 SHA1 SHA256 SHA384 SHA512
It would be great to have these settings in a new tab "Checksum" by right clicking on a file, where all checksums listed (which can be calculated by certutil) .
There should be a field where you can copy in an existing checksum and compare it with the calculated ones. If it matches, the matching hash could be marked.

I'm using http://implbits.com/products/hashtab/.

My version is at https://github.com/malxau/yori/tree/master/hash .  Note Windows includes support for hashing natively so all the real complexity is enumerating files, not calculating hashes.

Check also this https://www.binaryfortress.com/HashTools/

Wasn't this one of the options in  Send to X? If not, that seems like a very good place for it to live.

Powershell has this, I want this as well.  awesome idea!

Feels like a good addition for #641 as a larger enhancement ... powermenu

An additional idea, add this to the Property window

it would be great to add this to the file property tab too.

An interesting option, which I currently use, is Hashcheck. It adds an item in the context menu and a tab on the Property window. It is also capable of calculating hashes for folders, recursively. Supports all the suggested algorithms and is BSD 3-Clause licensed.

I was about to open this issue, but want to point out that it will duplicate this functionality for anyone with 7-zip installed so it'd be nice to disable the feature or have PT replace basic 7-zip functionality.

this can be easily added to your context menu via reg hacks.. checkout :
How to Add Hash to Context Menu of Files in Windows 8 and Windows 10

You can create a utility from it, when you want

Sure wish these hash values could be displayed in the Edge Browser's download file list.  You could immediately verify the app you have just downloaded matches the website that you got it from, without skipping out to file explorer.
But I guess PowerTools do not bind with browsers.  Still - thought I would mention it anyway!

Suddenly I thought of this feature and planned to open a new one. Glad to see someone already proposed this even though it is years ago.

Just to add here, as most solutions here are freeware or unmaintained. The one that is open source and actively maintained right now is @namazso's OpenHashTab.

This feature is definitively needed for every power users ! It's a shame it has never built-in directly in Microsoft Windows systems.
Hopefully it will be implemented in standard in PowerToys one day and take the best from gurnec/HashCheck, HashTab and namazso/OpenHashTab which all have great functionalities.

Generating the checksum is a good idea, but what is the purpose ?
The checksum must be useful for something after it was created.
Many years ago, i worked on a tools generating hash for files.
There were two purposes for the hashes :

check file integrity
quickly find potential duplicated files (same size + same hash)

MD5 and SHA-1 hashes may be too trivial for file integrity (thinking as seen alongside downloads).

To make sure an image or executable hasn't been tampered with is one example. A more contemporary situation might be giving a program access through a software firewall, this would be an easy way for it to detect  a new binary.
I'm not sure what the point would be, probably different for everyone. Just seems like a  common part of a power user's toolkit.

It will be great combine this with File Locksmith. These are not related options, but it make sense to me combine this type options in a ui for Advance File Options/Properties.

Have a look at Thio Joe's video: https://www.youtube.com/watch?v=aNDw1QMV-lM

The Files app has this built in:

I would also look forward to have this feature added. It really makes identifying duplicate files much easier. If i can help, tell me, i have a lot of Dev Experience. Can make a PR if needed

A nice concept from @astropingo can be found here https://github.com/microsoft/PowerToys/issues/30317

I suddenly realised that this issue only meant to generate hash but not verify with said hash.

1+ for this feature, I'm currently using https://github.com/idrassi/HashCheck.
Please also add fast and modern checksum algorithms like:

BLAKE3: https://github.com/BLAKE3-team/BLAKE3
XXH3: https://github.com/Cyan4973/xxHash

And no reason to implement deprecated / "not considered collision secure" algorithms, like CRC, MD2, MD4, MD5 and SHA-1. But include:

SHA-2, as Microsoft has stopped supporting SHA-1 for various things, like Windows Updates: https://aka.ms/sha1deprecation
SHA-3, as Microsoft has started using it for some stuff too. Supported in .NET 8: https://learn.microsoft.com/en-us/dotnet/standard/security/cross-platform-cryptography#sha-3

I'm also using HashCheck. That's why I support the inclusion of hash generation/checking tool in PowerToys as ways of checking for file legitimacy or integrity.

Would love this too; I've installed the registry edit from https://www.tenforums.com/tutorials/78681-add-file-hash-context-menu-windows-8-10-a.html which runs Get-FileHash on files. But it doesn't support recursively generating a hash for an entire folder like 7zip does, and I certainly miss that. Fwiw, I do see some suggestions for how to make that happen in https://stackoverflow.com/questions/64467425/md5-checksum-hashing-with-powershell-for-a-whole-directory


How to use the Adroit environment?

Hi! Thank you for presenting this very interesting benchmark. I would like to try test some DRL algorithms on this environment, however, I'm not sure exactly how to use it in my code? (I would like to use it in a similar way as DMControl or OpenAI gym)
It would be really helpful if you could point me to a tutorial or documentation page on how to use this benchmark! Thank you so much for the help! Really appreciate it!

Please see here -- https://github.com/vikashplus/mj_envs

Thanks a lot for your help!


lacking documentation: how to pass a spring to animated component

https://www.react-spring.io/docs/hooks/basics says following:

If you want to animate React components, styled-components, or elements on other platforms, then do this:

// React components
const AnimatedDonut = animated(Donut)

Let's assume that I have a following spring definition:
const props = useSpring({opacity: 1, from: {opacity: 0}})

Could you please tell me how to pass the defined spring to the animated component?

Hey @jutunen, it's on the docs:

Finally: distribute animated props among the view
The return value is an object containing animated props.
return <animated.div style={props}>i will fade</animated.div>

Can't see any props passing on this line:
// React components
const AnimatedDonut = animated(Donut)

That's because that's only recreating your component to be ready to be animated, you still need to render it.
const props = useSpring({opacity: 1, from: {opacity: 0}})
return (
    <AnimatedDonut style{props}/>
)

That's because that's only recreating your component to be ready to be animated, you still need to render it.
const props = useSpring({opacity: 1, from: {opacity: 0}})
return (
    <AnimatedDonut style={props}/>
)

Now this is documentation!
Hopefully this info gets to place where it belongs to, instead of being buried here among other issues.

That's because that's only recreating your component to be ready to be animated, you still need to render it.
const props = useSpring({opacity: 1, from: {opacity: 0}})
return (
    <AnimatedDonut style={props}/>
)

Now this is documentation!
Hopefully this info gets to place where it belongs to, instead of being buried here among other issues.

You can submit a PR, it's open source ; )


Make requests not use &mut reference

Needing a mutable reference means that only one request can be made at a time (?). Should that be fixed?

This would be a good improvement indeed. The original implementation of restson did not have async support so this was not really an issue then (all the requests were blocking). But now with async this is quite a big limitation.
In the current implementation, capturing the response headers would not work without mut. Though, that would have to be refactored anyway to work correctly with multiple concurrent requests.

There is now a version that does not need mut client in the const-client branch: https://github.com/spietika/restson-rust/tree/const-client
@Tuetuopay had already implemented this feature in his fork, and the relevant commits were cherry-picked from there with minor changes (e.g. handling for lib-simd-json added).
Test case making concurrent requests: https://github.com/spietika/restson-rust/blob/const-client/tests/get_async.rs#L78
This version is not yet published to crates.io since it still needs some minor additional work. E.g. updating the readme.

Oohh thanks for reminding me I've forgotten to upstream this branch for ages :smile:
FWIW I've been using this branch in production, well, since making it. So I can vouch for it working (ignoring the fact  wrote it).

Good to hear that the crate has been useful in your project! The implementation is solid and the tests are green, so I am happy to take these changes in. I will make some updates to the readme and then push a new version to crates.io.

Published in v1.2.0


Rda auth

RDA removed the need for a login and changed their download URLs.

Haven't actually taken a test run yet. Grabbing a bite to eat and will try my best to get that done and release this evening.

@encassano
I've tested:

[x] Daily 1979-01-02
[ ] Daily 2010-12-31 - 2011-01-01
[ ] Daily 2012-01-01
[ ] Daily 2023-08-01
[ ] Monthly 1979-02
[ ] Monthly 2010-12
[ ] Monthly 2011-01
[ ] Monthly 2023-07

Woops, two commits were missed. #43


Nuget

Anyone know who builds the Nuget packages? We are a bit behind I think.

Not me, until now I didn't even realise it was published to NuGet.
If you take a look here it seems to have been published by user bogatykh.

ahh... I didnt even to think to look there, thanks!


Modified documentation

Changes include:

Updating the landing page
Removing Search and Index links
Updating the What Is using text from "Inference optimization with MIGraphX"
Creating an install and build section
Changing the title of "contributing to MIGraphX"
Changing the intro of "contributing" to smooth it out
Updated the index file
Updated the toc

I added some comments but if you are going to spend time modifying the documentation it would be good to make a clear distinction between the developer and user use of the product. Separate out the installing and building. A developer is interested in building the code, a user is interested in installing the code. Mixing them together creates confusion.

This was a first pass at exactly that. This is why "build" and "install" are separate sections. I'll make it clear at the top of the build sections that this is for developing for and contributing to MIGraphX rather than for using the MIGraphX API.

There is only one way to build the code since all three use rbuild. Having two other ways is pointless.

Which steps from which doc would you prefer me to keep?

Good starting point for updating the documentation. I agree with the changes Chris proposed. Main change would be simplifying the installation section to just use rbuild. Hopefully we find some time later to add in a section to teach how to use MIGraphX if you just want to run models. Approving, but please make the changes Chris mentioned.

I really do want to add that section too. Hopefully soon.

I've removed all the build sections except build with rbuild, removed the prerequisites, and moved What is to the front of the page. I also added a link to the github repo at the top.


linalg_ext.topk needs to be more general than the current definition.

Based on review of PR #9162 it seems to me that the same linalg_ext.topk is being used in two separate contexts. To me that is an indication that linalg_ext.topk is very specific and needs to be generalized.
Background
Current definition of linalg_ext.topk is
%result:2 = iree_linalg_ext.topk dimension(...) ins(%input_values : ....) outs(%output_values, %output_indices : ...) {
  ....
} -> (tensor<...>, tensor<...>)

In this incarnation, the %result#0 contains the top K values from %input_values (the definition of top provided by the region of the op). %result#1 provides the position of these top K values in %input_values along the dimension specified.
In another incarnation, the operation is
%result:2 = iree_linalg_ext.topk dimension(...) ins(%input_values, %input_indices : ....) outs(%output_values, %output_indices : ...) {
  ....
} -> (tensor<...>, tensor<...>)

%result#0  is same as above. %input_indices is the same size as %input_values. Here %result#1 contains values from %input_indices  (instead of position in %input_values).
The second incarnation is needed for when the dimension (which is a reduction dimension) is distributed across threads, and tracking the original index during the merge step. The specification of this operation though is a bit strange (IMO). It seems to suggest that the op-definition is being massaged to fit the eventual need, but the computation itself does not seem to represent what one would expect from a topk.
Suggestion
It seems to me some generalization is needed. I think it is better to have a linalg_ext.maxk operation, both the above ops would be instances of that op. The op would be something like
%result:n = linalg_ext.maxk dimension(...)
    ins(%input0, %input1, %input2, ...., %inputm : ....)
    outs(%output0, %output1,... %outputn : ...) {
^bb0(%arg0, %arg1, %arg2...., %argm, %argmp1, %argmp2, ... %argmpn) :
  ...
  linalg_ext.yield %t0, %t1, ... %tn : ....
}

For the case of topk proper (the first instance above) you could use linalg_ext.index to get the position along the reduction dimension (or some other op, like linalg_ext.reduction_dim that is a placeholder for the position along the reduction dim, but linalg_ext.index seems good enough for this case).
Both instances above can be represented using this op, they just have different bodies.
FYI @KoolJBlack @ThomasRaoux  (also @hanhanW ). Thanks Kojo and Thomas for the initial exploration on topk op, it was extremely helpful.

Big +1 on generalizing the op! Thanks for the exploration. Now I understand better what's the difference between Linalg ops and linalg_ext.top_k ops.
The main differences are:

There are memref loads before scf ops.
The scf.for ops yield values in linalg_ext.top_k ops.

We're not able to represent such ops in Linalg. Because Linalg aims to have perfect nested loops.
func.func @_topk_1d_dim0_max_dispatch_0() {
  %c0 = arith.constant 0 : index
  %c1 = arith.constant 1 : index
  %c10 = arith.constant 10 : index
  %c3 = arith.constant 3 : index
  %c64 = arith.constant 64 : index
  %0 = hal.interface.binding.subspan set(0) binding(0) type(storage_buffer) offset(%c0) alignment(64) : memref<10xf32>
  memref.assume_alignment %0, 64 : memref<10xf32>
  %1 = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) offset(%c0) alignment(64) : memref<10xi32>
  memref.assume_alignment %1, 64 : memref<10xi32>
  %2 = hal.interface.binding.subspan set(0) binding(2) type(storage_buffer) offset(%c0) alignment(64) : memref<3xf32>
  memref.assume_alignment %2, 64 : memref<3xf32>
  %3 = hal.interface.binding.subspan set(0) binding(3) type(storage_buffer) offset(%c64) alignment(64) : memref<3xi32>
  memref.assume_alignment %3, 64 : memref<3xi32>
  scf.for %arg0 = %c0 to %c10 step %c1 {
    %4 = memref.load %0[%arg0] : memref<10xf32>
    %5 = memref.load %1[%arg0] : memref<10xi32> // This will be `%arg0` if input_indices is not specified.
    %6:2 = scf.for %arg1 = %c0 to %c3 step %c1 iter_args(%arg2 = %4, %arg3 = %5) -> (f32, i32) {
      %7 = memref.load %2[%arg1] : memref<3xf32>
      %8 = memref.load %3[%arg1] : memref<3xi32>
      %9 = arith.cmpf ogt, %arg2, %7 : f32
      %10 = arith.cmpf ogt, %7, %arg2 : f32
      %11 = arith.cmpi eq, %9, %10 : i1
      %12 = arith.cmpi slt, %arg3, %8 : i32
      %13 = arith.andi %11, %12 : i1
      %14 = arith.ori %9, %13 : i1
      %15 = arith.select %9, %arg2, %7 : f32
      %16 = arith.select %14, %arg3, %8 : i32
      memref.store %15, %2[%arg1] : memref<3xf32>
      memref.store %16, %3[%arg1] : memref<3xi32>
      %17 = arith.select %9, %7, %arg2 : f32
      %18 = arith.select %14, %8, %arg3 : i32
      scf.yield %17, %18 : f32, i32
    }
  }
  return
}

The needs are:

Initial values for scf.for op. In this case, they are input_values and input_indices.
The body of computation.

I borrowed the idea from Mahesh and extend it a bit more. To generalize the op, we can ask linalg_ext.yield to yield #output + #input values. The leading #output values will be stored into outs, and the rest of values will be yield by scf.yield.
%result:n = linalg_ext.maxk dimension(...)
    ins(%input0, %input1, %input2, ...., %inputm : ....)
    outs(%output0, %output1,... %outputn : ...) {
^bb0(%arg0, %arg1, %arg2...., %argm, %argmp1, %argmp2, ... %argmpn) :
  ...
  linalg_ext.yield %t0, %t1, ... %tn, %y0, %y1, ..., %ym : ....
}

The approach does not satisfy the needs for marking indices optional. But it generalizes it to other cases. We'll be able to use it in ArgMin/Max lowering as well. (In this context, we no longer need stack allocation for ArgMin/Max.)
I don't have a good solution for indices at this moment. Maybe having a new block could address the issue. But it makes the op a bit more complicated. E.g.,
%result:2 = linalg_ext.maxk dimension(...)
    ins(%input0: ....)
    outs(%output0, %output1 : ...) {
^bb0(%in):
  %idx = linalg_ext.index ...
  linalg_ext.yield %in, %idx ...  // these will be passed to scf.for iters.
^bb1(%arg0, %arg1, %argmp0, %argmp1) :
  ...
  linalg_ext.yield %t0, %t1, %y0, %y1: .... // The first two will be stored into outs, and the rest will be used in the next iteration.
}

Thinking a little bit more about this,  there is an advantage to making linalg_ext.maxk an interface and linalg_ext.topk an op that implements this interface (maxk doesnt have to be an interface to start with but could just be the base op in Tablegen for something with low overhead). The reason for this is that linalg_ext.topk when tiled + distributed will need a different operation as to represent the merge. That operation can be a linalg_ext.maxk. Knowing the op is a linalg_ext.topk (as opposed to recognizing the operation is a linalg_ext.topk by looking at the op region) is much easier to manage.
To some extent this analogy holds : linalg_ext.topk is to linalg_ext.maxk like linalg.matmul is to linalg.generic. In other words ,if linalg_ext.maxk were an interface similar to LinalgOp, linalg_ext.topk would be a named op.
So lets stay the course with linalg_ext.topk for now to learn more about how this all shakes out.


[Proposal][Task Plugin] Add Aliyun EMR Serverless Task Plugin

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Purpose

Aliyun EMR Serverless Spark is a cloud-native, fully-managed serverless product designed specifically for large-scale data processing and analysis. It provides enterprises with a one-stop data platform service, including task development, debugging, scheduling, and operations maintenance, which greatly simplifies the entire data processing lifecycle. By using EMR Serverless Spark, enterprises can focus more on data processing and analysis, thus improving work efficiency.
This proposal aims to provide the DolphinScheduler community users with an out-of-box plugin which they could use to conveniently interact with Aliyun EMR Serverless Spark.

Environment for Manual Testing

Aliyun EMR Serverless Spark is currently in public beta and free to use until the end of 25th, June, 2024. I have added detailed documents with examples in the Issue & PR. Reviewers / Users interested in this plugin could test it free of charge.

Maintenance

I will fix any known bugs / security vulnerabilities directly related to Aliyun Serverless Spark task plugin.
I will submit a test report for Aliyun Serverless Spark task plugin every time before a new DolphinScheduler release to make sure the plugin functions well.
The community could retire this plugin if test report not received before release for three consecutive times.

Detailed Examples
Submit Jar tasks

Parameters
Example Values / Operations

region id
cn-hangzhou

access key id

access key secret

resource queue id
root_queue

code type
JAR

job name
ds-emr-spark-jar

entry point
oss://datadev-oss-hdfs-test/spark-resource/examples/jars/spark-examples_2.12-3.3.1.jar

entry point arguments
100

spark submit parameters
--class org.apache.spark.examples.SparkPi --conf spark.executor.cores=4 --conf spark.executor.memory=20g --conf spark.driver.cores=4 --conf spark.driver.memory=8g --conf spark.executor.instances=1

engine release version
esr-2.1-native (Spark 3.3.1, Scala 2.12, Native Runtime)

is production
Please open the switch

Submit SQL tasks

Parameters
Example Values / Operations

region id
cn-hangzhou

access key id

access key secret

resource queue id
root_queue

code type
SQL

job name
ds-emr-spark-sql-1

entry point
Any non-empty string

entry point arguments
-e#show tables;show tables;

spark submit parameters
--class org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver --conf spark.executor.cores=4 --conf spark.executor.memory=20g --conf spark.driver.cores=4 --conf spark.driver.memory=8g --conf spark.executor.instances=1

engine release version
esr-2.1-native (Spark 3.3.1, Scala 2.12, Native Runtime)

is production
Please open the switch

Submit SQL tasks located in OSS

Parameters
Example Values / Operations

region id
cn-hangzhou

access key id

access key secret

resource queue id
root_queue

code type
SQL

job name
ds-emr-spark-sql-2

entry point
Any non-empty string

entry point arguments
-f#oss://datadev-oss-hdfs-test/spark-resource/examples/sql/show_db.sql

spark submit parameters
--class org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver --conf spark.executor.cores=4 --conf spark.executor.memory=20g --conf spark.driver.cores=4 --conf spark.driver.memory=8g --conf spark.executor.instances=1"

engine release version
esr-2.1-native (Spark 3.3.1, Scala 2.12, Native Runtime)

is production
Please open the switch

Submit PySpark Tasks

Parameters
Example Values / Operations

region id
cn-hangzhou

access key id

access key secret

resource queue id
root_queue

code type
PYTHON

job name
ds-emr-spark-python

entry point
oss://datadev-oss-hdfs-test/spark-resource/examples/src/main/python/pi.py

entry point arguments
100

spark submit parameters
--conf spark.executor.cores=4 --conf spark.executor.memory=20g --conf spark.driver.cores=4 --conf spark.driver.memory=8g --conf spark.executor.instances=1

engine release version
esr-2.1-native (Spark 3.3.1, Scala 2.12, Native Runtime)

is production
Please open the switch

Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Closed by #16126


Specifying initial state for Stateful Recurrent layers

Hi, is it possible to specify the initial state for a stateful recurrent layer? For example, I have the following code, will this work?
conv_lstm = ConvLSTM2D(128, (3,3), stateful=True)
x = Input(batch_shape=(1,1,32, 32, 3))
if need_initial_flag == True:
    h_tm1 = Input(batch_shape=(1,32, 32, 3))
    y = conv_lstm(x, initial_state=[h_tm1, h_tm1])
else:
    y = conv_lstm(x)

The idea is to have the need_initial_flag turned on in the beginning of a sequence and then turned off.
I am not sure if the logic is correct for Keras implementation. Thanks! @farizrahman4u

To be more clear, I want to define my model to have two inputs. The first input is the image data. The second input is some data else which is used to generate the initial states (Note they are not initial states). I want to define a model that has some recurrent layer (e.g. ConvLSTM2D). The layer is stateful because the sequence is long. And I want the model to set the initial states when the second input is some actual data while NOT to set the initial states when the second input is None or something similar.
How can I achieve the above goal? Is there any examples? Thanks!

@fchollet  any idea? Thanks!


Snapshotting infinite acceptation of screenshots

Hello!
Set up the fastlane and it works good, and today I finally made all the UITests for snapshotting, but unfortunately I faced with infinite loop problem of question-answering. Please take a look and let me know what could I do to fix it.
fastlane v. 2.81.0
Locale is set to en_US.UTF-8
Thank you in advance!

Does it accept an uppercase Y?
Are you actually typing a simple y or is this some other character that just looks the same? (don't know about your keyboard etc)

@janpio Yep, I typed all the variants, cases, letters, keyboard settings, restarted MacBook, terminal, with or without other programs running, etc. Nothing helps :/

This is weird, I cannot even fiind that You must enter a valid # error you're getting...

The same problem just happened for match passphrase

@Incorporalis Are you using the standard macOS terminal with bash? Or are you using something different?

@joshdholtz currently i'm using fresh MacOS with no any terminals. Tried it with iTerm before - the same error. I just don't know where to look.

@Incorporalis Are you able to run fastlane env and paste that output here?


feat: include ci files or not based on user option

superseded by https://github.com/HotSprings-GmbH/data-science-project-template/pull/19


USAGE output

Hey there,
Love the package :+1:
However, I was wondering if there was an easy way to modify the "USAGE" output. I would prefer to remove the '[global options] command [command options] [arguments...]' output.
Could you point me in the right direction?
Thanks,
Abe

Hi @abrahamrhoffman, you can override the output by setting the help template (https://github.com/codegangsta/cli/blob/master/help.go#L14) in your code. E.g.
...
cli.AppHelpTemplate = `NAME:
   {{.Name}} - {{.Usage}}
USAGE:
   {{.HelpName}} <your custom usage>
...

You can see the App struct for the values available (https://github.com/codegangsta/cli/blob/master/app.go#L13).
Let me know if this helps!

@jszwedko Thanks! That worked great! :dancers:


FFT bars wriggling around too quickly even to a sinewave

Hi,
I tried out the projekt and I recognized, that the FFT Bars in every viewtype are wriggling around (in level) very quickly, even if i have a constant sinus tone as input signal. Does anyone recognized this issue and knows the reason? I also tried out different fft sizes...
Thanks, Rob

Hi, you might want to limit the fps. Back to the day I made this, almost every phone is 60hz. Nowadays there are 90hz, or even 120hz, which is too much I think.


tile38: 1.33.4 -> 1.34.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/tidwall/tile38/releases.
meta.description for tile38 is: Real-time Geospatial and Geofencing
meta.homepage for tile38 is: https://tile38.com/
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.34.0 with grep in /nix/store/52ribfvpkp2pqac93rd3vsv2hjhan51l-tile38-1.34.0
found 1.34.0 in filename of file in /nix/store/52ribfvpkp2pqac93rd3vsv2hjhan51l-tile38-1.34.0

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

tile38

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/52ribfvpkp2pqac93rd3vsv2hjhan51l-tile38-1.34.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A tile38 https://github.com/r-ryantm/nixpkgs/archive/3f21cf55338d17a3c22d6f3f563fdc9e0b00a84a.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/3f21cf55338d17a3c22d6f3f563fdc9e0b00a84a#tile38

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/52ribfvpkp2pqac93rd3vsv2hjhan51l-tile38-1.34.0
ls -la /nix/store/52ribfvpkp2pqac93rd3vsv2hjhan51l-tile38-1.34.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review

x86_64-linux

:white_check_mark: 1 package built:

tile38

Maintainer pings
cc @autra @imincik @l0b0 @nh2 @nialov @sikmir @willcohen for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 363806

x86_64-darwin

:white_check_mark: 1 package built:

tile38


New parameter for sqlQuery()

A select query fetches only up to a 1000 records.
Can this be modified to fetch r number of records?
For e.g.:
results  = client.sqlQuery(queryStr,r=100000)

Hi. Currently you have to ensure to chunk your response per 1000 entries.
You should provide some PRIMARY KEY, like ID INTEGER AUTO_INCREMENT
then query it like this
SELECT id, something FROM XXX WHERE ID > @lastId LIMIT 1000

Then make assigment lastId = result[-1][0]


Add Share

Name: Share
Description: Share is a Web3 social writing platform where authors can publish books/webtoon/articles as NFT and readers can interact with their favorite authors.
Thumbnail URL: https://d2x76rid835axn.cloudfront.net/image/Share-01.png
Category: Social Writing
Project URL: theshr.xyz

@BellangerSebastien a size of 512px x 512px is recommended for thumbnail.

Heyy,
the new image : https://shr-dev.s3.eu-west-3.amazonaws.com/image/thumblensverse.png
Thanks :)

@BellangerSebastien added under Other category for now


Cannot find vicuna-v1-7b lib in preferred path "dist/vicuna-v1-7b/float16/vicuna-v1-7b_metal_float16.dylib" or other candidate paths%

Cannot find vicuna-v1-7b lib in preferred path "dist/vicuna-v1-7b/float16/vicuna-v1-7b_metal_float16.dylib" or other candidate paths%

This issue should be fixed by latest HEAD. Please try again and feel free to open a new one if it doesn't work


About the trained model for baseline methods

Hi, @paul007pl ,
Thanks for releasing the code for the benchmark and baseline models. Do you also provide the trained models (checkpoints) for baseline methods (e.g. [1] PCN;   [2] ECG;   [3] VRCNet)? It'll be useful for having consistent comparison results.
Thanks~

Thanks for your suggestion.
I have the pretrained models, and I may provide those results later.
Currently, a "blank" benchmark can somehow encourage every participator to try and understand different methods.
Another concern is that everyone would like to report their best results, which may not be easy to achieve.
Rather than reporting very good results at the begining, I think it is open for participators to test and evaluate each method by themselves.
Because the "Train Dataset" and the "Test Dataset" are the same with the released MVP dataset (2048 points), at current stage, you can check our CVPR2021 paper to compare different methods (see Table 4) https://arxiv.org/abs/2104.10154.
In short, the results on the "Test Dataset" are consistent (Table 4), which can be used in your report and research paper;
and inconsistent results on the "Extra-Test Dataset" can be okay at current stage.
Good luck and have fun~

Thanks @paul007pl , I'll train it first.


Anchor points added (Shift+Left Click) are placed at distant coordinates

Creating new points is a pain because they are always spawned off in the distance. Some sort of raycast from camera or ability to lock coordinate axes for a new point's spawn location could be useful. Additionally it would be nice to add a point locking system where the user can lock the position of individual anchor points (and/or that anchor point's control points).

@daneobyrd Hello, I'm facing the exact same problem. When the PathCreator is near the Center (0, 0, 0) then everything is fine. But when its somewhere else (500, 500, 500) the points are placed at distants coordinates. Do you every found a solution?
Thanks!


Add New Reco API Version

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month. 8/25
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month. 8/24
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[x] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

Regarding the API readiness validation failure, note that this is an RPaaS-based RP, so the new version can't be available until we publish it

Here is the PR merged to private repo with the same version:
https://github.com/Azure/azure-rest-api-specs-pr/pull/6651


Bug with copyDirSyncRecursive and nested directories

Let's say I do this command:
copyDirSyncRecursive("./templates/assets", "out/assets");
Then, I do this:
copyDirSyncRecursive("./resources", "out")
I would expect that the call like this just creates a new resources directory under out. And it does--but it also erases the out/assets directory made by the first one. The workaround is to do something like:
var srcDir = "./resources";
copyDirSyncRecursive(srcDir, "out" + "/" + path.basename(srcDir))

About to hop on a plane to Portugal but will look into this when I touch down. ;)
Sent from my iPhone
On Mar 1, 2012, at 15:22, Garen<EMAIL_ADDRESS>wrote:

Let's say I do this command:
copyDirSyncRecursive("./templates/assets", "out/assets");
Then, I do this:
copyDirSyncRecursive("./resources", "out")
I would expect that the call like this just creates a new resources directory under out. And it does--but it also erases the out/assets directory made by the first one. The workaround is to do something like:
var srcDir = "./resources";
copyDirSyncRecursive(srcDir, "out" + "/" + path.basename(srcDir))

Reply to this email directly or view it on GitHub:
https://github.com/ryanmcgrath/wrench-js/issues/22


How close is Mosby 3 to be production ready?

Hi everyone!
So, my question is pretty basic: I'm about to begin the development of a new app and I'm trying to decide if to use Mosby 2 or 3. Is Mosby 3 almost there or still not recommended for production usage? I'm gonna be using MVP, so MVI is not needed and out of question.

Hi there,
MVI is ready.
MVP (especially MVP-ViewState) has been refactored a little bit (separate branch MVP here on github). The refactoring mainly completed, but needs some little clean up. Also LCE components will be moved in a separate module. I would say 90% of that is already done and I have plans to finish that over the weekend. My plan is to make a final Mosby 3.0 release on Monday / Tuesday next week.
I think you should go with latest 3.0-alpha version or wait until next week for the final 3.0 release ... There might only be a little change of ViewState (Generics) from 3.0-alpha to the final 3.0

@sockeqwe Awesome! I'll go for the 3.0 then. Thanks!


Error on copyTabAsMarkdownLink

Hi, thank you for your work on this extension. I get the following error when copying a tab as link:

This seems related to #54, which was closed without resolution.

Does this happen for everything, or for specific pages (if so, could you please provide an example)?
Also, which browser is this?


[QUESTION] Live IBC bridges

Hello Team, the Bridge section of Readme.md has only 3 bridges. Are there any more bridges that needs to be included in the list, especially the ones which are currently live? Thanks in advance.

Those are Rust bridges, not just IBC (nomic is a sidechain).
I think wormhole would deserve to be there, as it does integrate a cosmos chain (terra) and other's look at it.
Also, Peggy or the various Ethereum bridges out there.
I don't have links to those teams, but second your point.

Hey @ethanfrey , appreciate your response for my query!


Footer Section

The copy text below Get Notified is not the same as in figma
Hint text color on signup form is different

Updated  please check.

updated but not using the same text as in figma yet

Updated please check


Add Accessor Parameter for Async Attribute

Suggestion is to add accessor parameter for async (and maybe other generator attributes, too) so that original methods can be private etc.

Right now the async method copies the accessor of the sync method. In which case would you need a different access type?

I am implementing an async method from the interface but I don't want to use the boilerplate Task.Run.  Because of this I have to annotate a public method but then I see IDE's warning for more strict access type.
This is also the same for RequiredArgsConstructor, IDE showing the fields "can be made readonly"

I am having a hard time understanding the exact scenario. Can you share a code snippet example?
As for the IDE warning for the strict access type, it might be a good idea to turn that warning off if you are not interested in it.
The [RequiredArgsConstructor] will only consider readonly fields so it is actually necessary to make them readonly.

Here I'm using Async to implement the interface's InitializeAsync method.
I like seeing the default IDE suggestions as I believe they help code quality. For RequiredArgsConstructor and initialized readonly fields I guess I will be creating my own constructor for the warning.

Now I see what you mean. I could add a parameter: [RequiredArgsAttribute(IncludeInitializedMembers = false)]. That would solve your problem, right?
Can you tell me what you were expecting from the [Async] attribute?

To be able to add accessor parameter to the attribute, like [Async(Public)] or  [Async(Protected)]

Could you show me a code snippet in which that would be useful?

Like in this example: sync Initialize method will not exposed on the child classes. But with the accessor parameter public InitializeAsync method of the interface will be implemented

sync Initialize method will not exposed on the child classes

Since your class is sealed, I don't see how you could have child classes the method could be exposed to.
The [Async] attribute copies the sync method's accessor. So in this case, InitializeAsync() would also be public. If you make Initialize() private, InitializeAsync() would also be private.

I think you are taking my examples too literally. I made it sealed to suppress disposable pattern warning.
I care about IDE warnings, I might oversee stuff and it helps maintaining same code code style across developers.
Using lombok on Java doesn't cause warnings like this and there are always always solutions like giving the setter annotation accessor parameter. If using some features cause warnings and clutters the code I would rather not use those features

I'm sorry, however I am going to need a concrete reproduction scenario which I can take literally in order to narrow down the issue and figure out a solution.

Closing due to inactivity.


[10.x] Add possibility to append directly

Description:
Improvement: Direct Attribute Appending in Eloquent Model Methods using the Append attribute for methods
Description:
This pull request introduces a new feature that allows attributes to be appended directly within Eloquent model methods, eliminating the need to manually add them to the appends array. This improvement enhances the flexibility and readability of Eloquent models by enabling developers to define appended attributes directly within the method, simplifying the process of exposing custom attributes in API responses.
Usage Examples:

Appending Full Name:
Before:
class User extends Model {
    protected $appends = ['roles'];

    public function roles() : Attribute
    {
        ....
    }
}

After:
class User extends Model {
    #[Append]
    public function roles(): Attribute
     {
        return Attribute::make(......)
    }
}

With this enhancement, the price attribute is automatically appended without the need to modify the $appends array. This results in more concise and organized code.

Dynamic Status Attribute:
Before:
class Order extends Model {
    protected $appends = ['price'];

    public function price(): Attribute
    {
        return Attribute::make(......)
    }
}

After:
class Order extends Model {
    #[Append]
    public function price() : Attribute
    {
        return Attribute::make(......);
    }
}

Benefits:

Simplified Code: Developers can directly define appended attributes within their respective methods, reducing the need for extra array declarations and improving code clarity.
Enhanced Readability: Attributes are logically associated with their defining methods, making it easier for other developers to understand the purpose of the appended attributes.
Reduced Error-Prone Code: Eliminates the possibility of forgetting to add attributes to the appends array, reducing potential bugs.

Notes: This feature maintains backward compatibility by still allowing developers to use the traditional method of adding attributes to the $appends array and doesn't add any breaking change

Will consider this for Laravel 11 in the future. Thanks.


feat(catalog): Support defining metadata on relations.

Hey, I just made a Pull Request!
This PR adds support for metadata on relations. It adds the new metadata field to the stitched entity's relation data and to the relation database. I've also added a way to define this for consumesApi and providesApi,
spec:
    providesApi/consumesApi:
        - targetRef: 'my-api'
           metadata:
               version: '1'

I was interested in overlap between this and #11027, this may or may not be a sane approach for that, and I can remove these updates to Component as needed.
Context: I initially implemented an environmentOverrides key on Component in https://github.com/backstage/backstage/pull/21560. From feedback (https://github.com/backstage/backstage/pull/21560#issuecomment-1826773724, https://github.com/backstage/backstage/issues/16389#issuecomment-1819928947 and Catalog SIG), metadata on relations came up. The idea would be to support overrides on the annotation per environment.
TODO:

[ ] Update documentation for Component.

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Ooh this is a tricky one, might have been a good use case for a #22096 (possibly asking @backstage/catalog-maintainers for input in an RFC first but not strictly required). The design for the input structures and dedupe semantics might take a fair bit of consideration to get right.

@freben I think it makes sense to go back and make this into a BEP. There's still some open design questions on my side, and that seems like a good venue for hashing those out.

@sennyeya  could we close this PR for now in favour of the BEP?

Sure, Ill update and reopen once/if the BEP goes through


Project doesn't build on Arduino 1.8

Hello
Downloaded the source code and the first issue I found is that there are 2 filenames with the same name, difference is on capitalized letters and windows doesn't accept it.
After that opening the project will give errors like:
'USISR' was not declared in this scope

Don’t know how that happened. I have deleted the duplicate file. The error that you have is usually caused by having the wrong target selected.

Hi @killercode, I'm having the same issue as you (USISR etc.), did you find a solution yet?

Is this an issue with the Attiny85 or the Nano version? My advice would be to see if you can build the nano version which uses fairly standard Arduino libraries.

Hi @jeffmer, you are correct, the nano version builds fine after reinstall/cleaning. Thanks!

Yes, I solved it:
1- Installed the library attinycore
2- removed the duplicate file
3- renamed the folder to match the .ino file name
Also I added an ifndef on the MAX30102.h file
I will post a fix for this now

Hello, here is a fix for the problems I found
Pull Request: #5
Hope it helps


float with exponent and underscores

The example for toml 0.4 doesn't work, because of this line:
key2 = 1e1_000

I investigated a litte bit and the issue is because the result is infinity and there is a check against that:
 number
            .parse()
            .map_err(|_e| self.error(start, ErrorKind::DateInvalid))
            .and_then(|n: f64| {
                println!("{:?}", n);
                if n.is_finite() {
                    Ok(n)
                } else {
                    Err(self.error(start, ErrorKind::NumberInvalid))
                }
            })

I'll close this, since this seems to be are weird corner case from the example


Equivalent of ansible parted module

Ansible has parted module to help manage partitions on disk, I haven't seen anything like that for pyinfra. It would make setting up new nodes a lot easier and more error-proof.
https://docs.ansible.com/ansible/latest/collections/community/general/parted_module.html

As I only needed to resize one part and create another with rest of free space I decided to code it using shell commands, if someone needs to do similar thing - code below.
# Requires parted, installed below.
# TODO: remove hardcoded /dev/vda
def setup_disk():
    # parted format
    # id:start:end:size:fs:label:flags, a total of 7 objects
    # 1:1049kB:2097kB:1049kB:::bios_grub;
    # 2:2097kB:258MB:256MB:fat16:EFI System Partition:boot, esp;
    # 3:258MB:770MB:512MB:ext4::;
    # 4:770MB:10.7GB:9967MB:ext4::;
    parted = server.shell("parted -m /dev/vda print list | awk '$1+0'").stdout
    parted = parted.split("\n")
    partitions = []
    for line in parted:
        line = line.split(":")
        if len(line) == 7:
            partitions.append({
                "partition": f"/dev/vda{line[0]}",
                "start": line[1],
                "end": line[2],
                "size": line[3]
            })
    has_vda5 = False
    for part in partitions:
        # Resize vda4 to end at 50GB
        if part["partition"] == "/dev/vda4" and part["end"] != "50.0GB":
            # https://stackoverflow.com/questions/52509644/how-do-i-get-over-parted-confirmation-request-in-a-script
            print("Resizing /dev/vda4 to end at 50GB")
            server.shell("echo Yes | parted ---pretend-input-tty -m /dev/vda resizepart 4 50GB")
            server.shell("resize2fs /dev/vda4")
        if part["partition"] == "/dev/vda5":
            has_vda5 = True
    if not has_vda5:
        print("vda5 not found, creating and setting up fstab")
        server.shell("parted -m /dev/vda mkpart longhorn 50GB 100%")
        server.shell("mkfs.ext4 /dev/vda5")
        files.directory("/var/lib/longhorn", present=True)
        files.line("/etc/fstab", line="/dev/vda5 /var/lib/longhorn ext4 defaults 0 0")
        server.shell("mount -a")


Update constants - CODATA

The current version uses CODATA 2014.
Update to CODATA 2018.

Note to assignee/self: The CODATA 2022 bulletin is accepted and due to be published here.


22Q1 planning and work log

Reorder items using drag-and-drop.
🟢 Actively being worked on:

[ ] ...

🟡 Top / next to do:

See https://github.com/open-telemetry/opentelemetry.io/issues/989#issuecomment-999577091
[ ] ...

🔴 Blocked/awaiting:

[ ] ...

🟠 Other to do:

[ ] ...

☑ Done (selected).
For a complete list, see closed 22Q1 PRs

...

References:

#769

Oops, I'd forgotten about this placeholder issue. Closing in favor of #1035.


bug: SAR opcode panics if the shift is greater than the type size

Bug Report
The result should be 0 if the shift is greater than the type size, so we should use wrapping_shr

Do you talk about the exec_sar() function implementing the SAR opcode ?
There are several tests which validate that 0 (if value is positive) or -1 (if value is negative) is returned if more than 256 bits have to be shifted and I don't see any panic when executing them 🤔
But I probably missed something 😅

if more than 256 bits have to be shifted and I don't see any panic when executing them 🤔

I think the problem is if shift==256. But even then, SAR should use wrapping instead of panicking shift, because the EVM doesn't have this panicking semantic


Drop sample size from Dataset.coordinates

Currently, we store sample sizes in both the Dataset.metadata and the Dataset.coordinates attributes. We should drop the n column in Dataset.coordinates and make sure that CBMA algorithms grab that info from the metadata instead.
NOTE: Originally part of #179, but that PR sprawled and will probably be closed.

Please forgive me if we've discussed this before, but have we considered (at least in the architecture if not the current implementation) the possibility of voxel-wise-varying sample size?
It's now an edge case, but with larger and larger N (for an individual, non-meta analysis), you have the problem of ever eroding analysis mask since all voxels are required.  We are now building tools that allow the number of subjects contributing to an analysis vary by voxel (within limits, of course).

@nicholst Apologies for closing the issue with #232.
I don't know if we've discussed it, but I don't remember talking about it.
Is this something that would apply to CBMA methods, or just IBMA ones? For CBMAs, I'm guessing that the adjustment would be more of a guess, while for IBMAs, that information could actually be accessible.
I should note that, while sample size is not stored in the Dataset's coordinates dataframe (it's in the metadata one instead), at the point where the CBMA estimator operates on the data, sample size (and any other relevant metadata) will be in the dataframe. So any adjustment done internally within the estimator will still have access to that info.

Is this something that would apply to CBMA methods, or just IBMA ones? For CBMAs, I'm guessing that the adjustment would be more of a guess, while for IBMAs, that information could actually be accessible.

Oh, sorry!  This is only for IBMA.

Gotcha. I think it would be good to open an enhancement issue for this once it's ready then.

I've opened #274 to continue discussion of handling missing data, and will change the title and summary for this issue to match the bug of unknown cause.


Interpolate fft cuda

Codecov Report

Merging #300 (9c2ece2) into evaluate_fft_cuda (d28ecce) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@                Coverage Diff                 @@
##           evaluate_fft_cuda     #300   +/-   ##
==================================================
  Coverage              96.43%   96.44%
==================================================
  Files                     92       92
  Lines                  11535    11547   +12
==================================================
+ Hits                   11124    11136   +12
  Misses                   411      411

Impacted Files
Coverage Δ

fft/src/polynomial.rs
94.50% <100.00%> (+0.25%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Fix typescript Store interface definition

A dispached action does not actually have to be of type Action. For example a thunk function does not have a type property.

This is done by extending Dispatch type with custom signatures, see tests.


Indexing exclude models where column is false

Hi,
Great plugin!
Right now I am using in my search
Organizer::search($request->keywords)
		    ->where('approved', 'true')
		    ->get();

This works great, but I was wondering if there was a way to not even index the Organizer models until approved = true? Sorry if this has already been done I tried looking through the other questions but couldn't find anything.

Hi @chrisgrim, sorry for the late reply.
Yes, it's possible, have a look here https://laravel.com/docs/6.x/scout#conditionally-searchable-model-instances

Amazing @babenkoivan  Thanks so much!!!


Add European biogeographic regions to region definitions

This PR adds the list of biographic European regions to the region-definitions such that they can be used with timeseries data in the IAMC format.
Are there other attributes that should be added here? Or do we need to add a more elaborate description of the source?
fyi @iiasa/natura-connect-admin

This is just copying the table from Wikipedia (unless you point me to a different source).
The list just enables us to use these regions in the IAMC-data-format of the Natura-Connect-Explorer. We will add the maps for spatial visualization later.

This is just copying the table from Wikipedia (unless you point me to a different source).
The list just enables us to use these regions in the IAMC-data-format of the Natura-Connect-Explorer. We will add the maps for spatial visualization later.

AFAIK there is not a ready to use existing data source (although wikipedia is largely correct but there might be smaller areas in some countries). Just to be sure one needs to intersect the layer with the country shapefiles in a GIS. I have just done that and the resulting csv file can be found here https://gist.github.com/Martin-Jung/1ec401fa555630113bf3884a9c66f33f
Do you have a script to create these yaml files or is that easier done manually?

Do you have a script to create these yaml files or is that easier done manually?

I guess you can simply do a groupby by bio-region and then copy-paste the country-name column.
But keep in mind that the country lists in this yaml file are only informative anyway, they will not be used directly for visualization. So no need to put too much love into them...


Implement string interpolation (requires lexer modes)

Implement string interpolation in the form of "variable = ${variable}", which requires support for lexer modes. The system used to change the lexer for DSLs is similar, but it's controlled at the parser level rather than the lexer (which may be a better option anyways, since only the parser knows when interpolation ends).

Closed by 0d3ad75. The implementation approach for lexer modes can be re-examined when more DSL work is done.


Taking the whole speech sequence as input without chunking

Hi Micro, many thanks for the great work and sharing the code! It's super useful for the work I am doing. So for the speaker identification task, each training sample has the length of 3200 since fs=16000; cw_len=200; wlen=int(fs*cw_len/1000.00)=3200. And for testing, voting over even smaller chunks are performed.
I have a speech classification task that, I think, it's the best to take the whole speech sequence as input without chunking. Currently, is it possible to use variable length input sequence for sincnet at all? If not, then I would pad each batch with zeros to the max length (in each batch). Would sincnet be affected with padded batches?
Oh btw, my speech sequences vary in length by a lot actually. What would be your suggestion Micro? Many thanks again!

Hi,
thank you very much for the interest in my work!
Given the variable-length input sequence, do you want in output a fixed
length vector or a variable length sequence? Which classification task are
you considering?
Best,
Mirco
On Tue, 30 Apr 2019 at 07:03, bluemonk482<EMAIL_ADDRESS>wrote:

Hi Micro, many thanks for the great work and sharing the code! It's super
useful for the work I am doing. So for the speaker identification task,
each training sample has the length of 3200 since fs=16000; cw_len=200;
wlen=int(fs*cw_len/1000.00)=3200. And for testing, voting over even
smaller chunks are performed.
I have a speech classification task that, I think, it's the best to take
the whole speech sequence as input without chunking. Currently, is it
possible to use variable length input sequence for sincnet at all? If not,
then I would pad each batch with zeros to the max length (in each batch).
Would sincnet be affected with padded batches?
Oh btw, my speech sequences vary in length by a lot actually. What would
be your suggestion Micro? Many thanks again!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/39, or mute the thread
https://github.com/notifications/unsubscribe-auth/AEA2ZVRMFPUCVN4XQMTZHWDPTARPPANCNFSM4HJKSUZQ
.

@mravanelli Speech emotion classification, 5-6 categorical emotion classes, which I think it's best not to chunk right.
Actually whether or not the output has a fixed or variable length, does not matter much, since I have a layer after sincnet that outputs fixed length vector representation over the whole input sequence.

Ok,
chunking is normally a good idea. However, if you would like to avoid
chunking you can just give in input to the convolutional neural network a
sequence of samples and you have in output another sequence whose length
and dimensionality depends on the hyperparameters of the convolutional
layers (filter length, number of filters,...). The time context used by
each prediction depends on the receptive field of the convolutional neural
network (more depth, more context).  If you would like an implementation
more similar to the latter case, you might want to take a look here:
https://github.com/santi-pdp/pase
Best,
Mirco
On Tue, 30 Apr 2019 at 11:31, bluemonk482<EMAIL_ADDRESS>wrote:

@mravanelli https://github.com/mravanelli Speech emotion
classification, 5-6 categorical emotion classes, which I think it's best
not to chunk right.
Actually whether or not the output has a fixed or variable length, does
not matter much, since I have a layer after sincnet that outputs fixed
length vector representation over the whole input sequence.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/39#issuecomment-487999635,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEA2ZVV3C3H3SSFZXUHJYZLPTBQ5LANCNFSM4HJKSUZQ
.

Thanks @mravanelli !
Can I please ask why is chunking normally a good idea? Reading your code, you extract one cw_len long chunk from each original input sample. Therefore you are not creating more training data in this case, but kind of treating each small chunk as the representative of the original speech sample. It is kinda counter-intuitive (for someone like me who has little experience in speech processing at least).

Actually my bad. You're actually creating more training samples in this case. I still don't understand why this would be a good idea in general.

Actually, the difference between chunking/non-chunking is not that big.
Even if you do not explicitly do chunking, you implicitly do it using the
convolution operation and their limited receptive field.
In our case, we have in input frames of 200 ms and we have three
convolutional layers followed by a couple of fully connected layers. The
advantage is that with a relatively shallow neural network (5 layers only)
we can embed a pretty large context. If you would like to embed the same
context without chunking and using convolutional layers only you need many
more convolutional layers (because the receptive field increases with the
depth of the network). Said that both solutions can work equally well.
Mirco
On Tue, 30 Apr 2019 at 12:04, bluemonk482<EMAIL_ADDRESS>wrote:

Actually my bad. You're actually creating more training samples in this
case. I still don't understand why this would be a good idea in general.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/39#issuecomment-488012617,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEA2ZVQR3LZ7AUSTJF275TLPTBUZ7ANCNFSM4HJKSUZQ
.

Hi Mirco,
Thanks for these suggestions.  I want to run speaker ID on utterance level just for sake of comparing results. What changes in config file and code is need to run utterance-level speaker ID on TIMIT. Perhaps, global pooling before final layer or there is a better work around.

Actually, utterance-level speaker ID is what we are doing, because we
average the predictions on all the chunks.
Mirco
On Sat, 4 May 2019 at 00:11, hdubey<EMAIL_ADDRESS>wrote:

Hi Mirco,
Thanks for these suggestions. I want to run speaker ID on utterance level
just for sake of comparing results. What changes in config file and code is
need to run utterance-level speaker ID on TIMIT. Perhaps, global pooling
before final layer or there is a better work around.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/39#issuecomment-489291897,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEA2ZVTEMKE4PJUUT3LYF3LPTUEHFANCNFSM4HJKSUZQ
.

In my case, I want a max polling for computer vision papers, how to do that?

@mravanelli If I am not mistaken the network trains on frame-level (10ms) right? I want to do global max-pooling just before the softmax layer and train on utterance-level Loss function.

Hi!
the version implemented at the link below is closer to what you would like to do.
https://github.com/santi-pdp/pase


Remove tab whitespace that was added during dot notation merge.

Hey Jan,
Here's a fix for the tabs that were added to the project as part of the dot notation merge. Each tab was painstakingly retired and replaced with 2 spaces over a matter of days.
Thanks for the 'stache.
Brandon

Merci! :)


(FACT-2724) Confine blocks behave differently with Facter 4, causing spec tests to suddenly fail

problem
virtual fact was returning nil on physical macOS hosts. The test mocked virtual fact by using
allow(Facter.fact(:virtual)).to receive(:value).and_return('physical')

and because Facter.fact(:virtual) returned nil on physical macOS hosts, the mock was basically
allow(nil).to receive(:value).and_return('physical')

fix
on macOS physiscal host, facter will return virtual => physical

CLA signed by all contributors.


Allow recent omnibus-software ruby version convention.

This allows master to build at chef/omnibus-software#556

@jaym @btm @schisamo @chef/engineering-services

Looks reasonable to me. Right now CI doesn't check that the msi-installed windows push client actually executes, so it might be worth making sure of that before merging.


iOS support

Hi there! This project is pretty cool. I'm glad to see the rust bindgen objective-c features.
Depending on effort, I'm up for picking up this task or a subtask. Mind writing some expectations? If you're interested in running some actual unit tests in an iOS simulator, cargo-dinghy bundles up unit tests, runs them with xcrun and lldb to get the exit status. Thoughts?

Hi! First of all, thank you a lot! I wrote the first version of this project based on your uikit-sys build.rs!
Recently I added apple-bindgen subcrate, the bindgen-running crate. Unlike apple-sys, it is including iOS support - without bindgen rules. Well, many of them will fail. To be honest, I only tested CoreFoundation for iOS.
The primary missing connection between apple-sys and apple-bindgen is TARGET to sdk matching. Because I found apple-sdk already made great job, I sent a patch for this feature.
Because it was only 3 days ago, I would wait for a few more days. Depending on whether they accept it or not, I'd like use the new version of apple-sdk or adding the patch back to apple-sys to add iOS(and every other platforms) support.
Second part is configure.py to add framework names as features. The script need to take arguments to handle platforms. And blocklist for bindgen-imcompatible frameworks.
The last part is Bindgen rule. What we need for that is trying to build the generated output of iOS and add correct rules for the files. I remember you already find rules for it in uikit-sys build.rs
I am really grateful that you left comment to say you want to take any part of the project. It is the best feedback ever for my projects' first week!
dinghy looks really good. Using it looks really good idea. Thank you for introducing a nice project 😄

Oh, you were the author of Objective-C support of bindgen! No wonder how I found uikit-sys from bindgen document. 😲

Oh, you were the author of Objective-C support of bindgen! No wonder how I found uikit-sys from bindgen document. 😲

I'm not quite the original author. I came across the mild/broken support for it and just kinda incrementally added small stuff that made sense and a few PRs later it felt like structured rust calling into objective-c. This project and uikit-sys are some of the only projects using it.
I've also been watching objc2 with hopes of having better memory management.
There's still plenty of things I wish I had more time to add/fix. Wrote some of it down in https://github.com/madsmtm/objc2/issues/85#issuecomment-988252089. As mentioned there, https://github.com/rust-lang/rust-bindgen/pull/1784 is a feature that's pretty important but haven't gotten around to fixing or showing it's worth the performance loss.

apple-sdk 0.2.0 is published


Remote azure storage

I create an Azure Storage class Adapter for remote storage.
I'm using a third party library for the actual Storage system.
I documented the options.
The library for azure storage has a small glitch in it, making listbackups command impossible. I fix it and submitted a PR to it, I think it should be merge soon.
Cheers !

Coverage decreased (-1.6%) to 70.829% when pulling 4533b524e5b0186896b93c65120e897274c0a79e on dorianamouroux:remote_azure_storage into 94b7848d2f986ecf6bc3e758e2cd3bfdb95bb6d8 on django-dbbackup:master.

Hello @dorianamouroux,
Thanks a lot for this contribution.
We are developing DBbackup 3 and in this version we'll drop support for storages. As Django already have a great storage API and its third apps (like azure_storage), we want to focus only backup and restore mechanisms.
Your contribution could be interesting as a snippets on documentation because it demonstrates how to use a third part storage with DBBackup.
What do you think about ?

I think it's too bad that we drop support for storage.
For projects that use DBBackup, it's really convenient to have storage options.
We can use different configurations for backup files and media files without having to hack anything (and have a cleaner codebase).
I understand the idea behind it, maybe split off the backup and storage part would be a good solution.
Though you can use my code as snippets for documentation if you don't keep it for storage system 👍

@dorianamouroux
I like your idea of make a separate repo for storage but my problem is the time used for support this.
@benjaoming @mjs7231 What do you think about ?

I can give some of my time for it, I like the project.

@dorianamouroux : If you have some time and some $ on Azure, could I suggest you to contribute to django-storages Azure backend ?
I would like to be sure it work's with DBBackup.

django-storages is really outdated and looks barely maintained anymore. The Azure Backend they offer is missing all required features we expect for a storage backend, it does only work for collectstatic (and --clear doesn't work). Also, you cannot pass parameters to the constructor thus you cannot give different parameters for backups and normal mode. They doesn't seems to be very responsive to contribution neither.
I used this storage backend https://github.com/Rediker-Software/django-azure-storage there is no documentation, but I managed to make it works for backups (with a PR). There is still work to do on this package because I cannot list the different backups. I might write my own Azure Backend.

Hello @dorianamouroux
What about this storage ?

Django DBBackup doesn't contains object storage anymore.
This issue is no more relevant.


Archive file not created

With the following configuration, the archive file is not created (I do not have any error message). I tested for this month (february/march) so it may have something to do with the 29 february (?). I'm using NLog 4.2.3 with .Net 4.6
<target name="File" xsi:type="File" fileName="${basedir}/${shortdate}.log"
      keepFileOpen="false" encoding="iso-8859-2"
      layout="${longdate}|${pad:padding=5:inner=${level:uppercase=true}}|${message} ${exception:format=tostring}"
      archiveFileName="${basedir}/archive.{#}.zip" archiveNumbering="Date"
      archiveDateFormat="MMMMyyyy" archiveEvery="Month"
      maxArchiveFiles="24" enableArchiveFileCompression="true" />

Edit: this could be related to https://github.com/NLog/NLog/issues/1271

I'm pretty sure this is https://github.com/NLog/NLog/issues/1271
which will be fixed in https://github.com/NLog/NLog/pull/1274
@bhaeussermann can you confirm this? Any idea when this bug has been introduced?

@Bidou44 can you test this with NLog 4.3-RC2?

This is probably fixed in 4.3. If not, just let us know


Feature request: Possible addition of Show/Hide toggle?

Hello,
Thank you very much for this extension, amazing work!
I just wanted to ask if you are planning on adding a Show/Hide toggle for Model, LoRa, Positive Prompt, Size in the future?

Best Regards!

It has been implemented.


Required permissions are missing

Account Administrator is listed as the prerequisite, but there is no such role as Account Administrator. Please clarify the required permissions.
A user account in both the source and target directory is listed as the prerequisite, but again it is not clear what level of permissions it requires.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 9afab524-67c2-2e76-fd7a-46238cb9d249
Version Independent ID: ae2da577-6026-f322-1c09-dbcc6068c01a
Content: Transfer an Azure subscription to a different Azure AD directory
Content Source: articles/role-based-access-control/transfer-subscription.md
Service: role-based-access-control
GitHub Login: @rolyon
Microsoft Alias: rolyon

@rpilot
You are correct that there is no role called "Account Administrator" in Azure. The correct role required for transferring a subscription is "Owner" role1. The user account that you use to do the transfer must have Owner access on both the source and target subscriptions.
Regarding the user account in both the source and target directory, it should have the necessary permissions to manage the subscriptions. Specifically, it should have the Owner role on both subscriptions.
I hope this clarifies your doubts. Let me know if you have any other questions.

@ManoharLakkoju-MSFT Thanks for the clarification!


exFAT EASYROMS partition still impossible to mount on the CHI

Describe the Issue (If applicable)
It's still impossible to mount the EASYROMS in windows, it's not recognized as a readable partition by the disk manager.
Using the exFAT fix on ArkOS menu doesn't work (it doesn't detect any issue).
Note that it's properly recognized before the resizing, but that operation seems to corrupt the partition table.
I tested several SD cards of various sizes, mostly 32GB and more.
How can the issue be reproduced? (If applicable)
Just flashing the image clean to a SD card, boot it on the CHI for the first initialization, then plug it back to windows.
What device are you using?
GameForce CHI
What version of ArkOS are you on?
v2.0 (09/24/20230)
Anything else you'd like to include that may help to review this issue or feature request?

Yea I need to update the image for the chi to resolve it but I no longer have a Chi so I don't know when I will be able to resolve that for the Chi.  The partition is not corrupted, just not set in manner that some computers can easily see it.  For now, what you can do is when you burn the image on to the sd card, before inserting the sd card inti the Chi, replace the expandtoexfat.sh file in the boot partition with this one https://github.com/christianhaitian/rk3326_core_builds/raw/rk3326/shell-scripts/expandtoexfat.sh then insert the sd card into the unit.
If you already have a current setup you'd prefer to not have to recreate, If you're comfortable with ssh, you can remote into the unit and run this command printf "t\n3\n7\nw\n" | sudo fdisk /dev/mmcblk0 and that should take care of the issue as well.  It should be non destructive but I can't guarantee that due to the nature of this hobby.

Thanks a lot for the quick reply, I will try it out!

The SSH command worked perfectly, it's now back to normal exFAT. Thanks for the help!

Awesome.  Enjoy