Change Liquibase#close to throw LiquibaseException

There is no need to force callers to handle Exception.
The implementation of the close method can currently throw only a DatabaseException. But since the library has a parent exception class - LiquibaseException - it is used instead to not force callers to change their catch clauses if additional logic is added in the future.
┆Issue is synchronized with this Jira Story by Unito

Thanks for the PR! We will review this and get back to you. While this seems like a simple change, we have to be careful of unintended side-effects.

➤ Steve Donie commented:
We suggest that this should be code reviewed, and then it should be merged with the other Low Risk PRs that are scheduled for Sprint 7.2, to be tested by existing smoke tests.

Does this change in returned exceptions to a lower type break users who have existing code compiled against the version with the wider exception type?

It should not. LiquibaseException is-a Exception, so existing catch clauses or throws declarations will still work and compile successfully.

Did a test merge of this into a branch DAT-4459 that we are using for Low Risk merges into master, which will be version 4.0.0. That branch will be merged at the start of Sprint 7.2, which will be Thursday April 2.

Merged it into master for the 4.0 release. It may not make it into 4.0 beta1, but it will be part of the 4.0 final release for sure.

Merged it into master for the 4.0 release. It may not make it into 4.0 beta1, but it will be part of the 4.0 final release for sure.

Hello, @nvoxland, it seems this change was not part of the 4.0.0 release.

@andreyakostov I just checked the Liquibase.java source code, and it looks like this PR is in the 4.0.0 release. Please respond and let us know if you are experiencing something that makes you believe otherwise.

Hello, @kevin-at-datical, I am referencing the liquibase-core library in my project from Maven Central. My IDE downloads the library source and there I see the old signature of close(), which throws Exception.

Yeah, sorry. We have a difference in how the buildserver setup can work in master vs. 4.0 so can't do an easy merge like we're working on and it has to be cherry picked. I must have missed getting that done after merging the PR.
We'll have to get this into 4.0.1 instead.

Hello @nvoxland, just a reminder that this change is still not released.

Thanks for the reminder, @andreyakostov. I cherry picked it into 4.2.x for a 4.2.1 release we are planning in the next few days.
We're working on fixing up the issues that are making it too easy for things like this to get missed.


FileParser need to be exported

There is no way to reach FileParser through require("node-file-parser").
One has to require("./node_modules/node-file-parser/src/interfaces/FileParser") to reach it.

Agreed.

Three years later, but I'll plan it in for this month :-)


update to Legolas v0.5

Breakingness is debatable, re- https://github.com/beacon-biosignals/LegolasFlux.jl/issues/20
I think these are all the changes needed? Not 100% sure though. Tests pass at least.
closes #20

Maybe we should test that an old table can still be loaded. Could save out the DigitsModel from the example on main, check it in as a test, and then make sure we can load it here?

Maybe we should test that an old table can still be loaded. Could save out the DigitsModel from the example on main, check it in as a test, and then make sure we can load it here?

was goign to suggest exactly this.
I think it's also good to use the digitsmodel as a test for extensibility

I think it's also good to use the digitsmodel as a test for extensibility

OIC this is included in teh tests, very good

Should that go in some Legolas guidance, or here?

Hm, good question.  I think there are some specific points that are particularly relevant for upgrading schemas intended to be used like the Model extensions here, so I think it's worth calling them out in as concise a way as possible, since at the moment it's a bit extract those actionable things from teh legolas documentation/tour/various PR discussions.  I do think a general legolas upgrade checklist would be good to include in legolas but not that's probably a slightly larger thing

I agree the current set of

teh legolas documentation/tour/various PR discussions

is not conducive to quickly learning how to upgrade, but I think we should PR docs there specifically, since none of the points you raised seem LegolasFlux specific. I'll merge this, then make that PR (which will use some of this PR as an example), then make another PR here linking out to those docs.


Change DLC to NNPA for consistency in accelerator name

This patch replaces  DLC by NNPA for consistency in accelerator name.

Jenkins Linux s390x Build #4332 [push] Change DLC to NNPA for c... started at 22:14

Jenkins Linux ppc64le Build #3448 [push] Change DLC to NNPA for c... started at 22:16

Jenkins Linux amd64 Build #4320 [push] Change DLC to NNPA for c... started at 21:14

Jenkins Linux s390x Build #4332 [push] Change DLC to NNPA for c... passed after 54 min

Jenkins Linux ppc64le Build #3448 [push] Change DLC to NNPA for c... passed after 57 min

Jenkins Linux amd64 Build #4320 [push] Change DLC to NNPA for c... passed after 1 hr 4 min


Podman dnsname not working for default network

Describe the bug
The podman dnsname plugin is not working for the default network, as the defaultNetwork.dnsname.enable option implies.
I am able to work around this by creating a custom network manually, but this is undesirable since there doesn't seem to be an easy way to declaritively create a podman network.
Steps To Reproduce

Configure podman how it's shown in https://nixos.wiki/wiki/Podman
Run nixos-rebuild switch
Check if container dns is working on the default network  as shown in https://github.com/containers/dnsname/blob/main/README_PODMAN.md (The --network foobar argument is removed in the example below).

sudo podman run -dt --name web quay.io/libpod/alpine_nginx:latest
5139d65d22135e9ecab511559d863754550894a32285befd94dab231017048c2

sudo podman run -it --name client quay.io/libpod/alpine_nginx:latest curl http://web.dns.podman/
podman rulez

Expected behavior
I expected the defaultNetwork.dnsname.enable option to enable dns on the default network.
To work around the issue, I created another network by running podman network create foobar and added the --network=foobar option to extraOptions.
Screenshots
NA
Additional context
I am running this in a proxmox lxc container (https://nixos.wiki/wiki/Proxmox_Virtual_Environment#LXC), which might may be causing the problem, though it works fine when I create a custom network. I poked around for a while, but didn't find anything conclusive. I'm not really familiar with nixos or podman.
I want to configure an influxdb and a telegraf container. Here's what I have for my current config with my workaround.
  virtualisation = {
    podman = {
      enable = true;
      defaultNetwork.dnsname.enable = true;
      dockerCompat = true;
      dockerSocket.enable = true;
      extraPackages = [ pkgs.zfs ];
    };
  };
  users.groups.podman.gid=995;
  virtualisation.oci-containers.containers = {
    influxdb2 = {
      image = "influxdb:2.5";
      volumes = [
        "/mnt/user/appdata/influxdb2:/var/lib/influxdb2"
      ];
      ports = ["8086:8086"];
      extraOptions = [
        "--network=foobar"
      ];
    };

    telegraf = {
      image = "telegraf:1.24";
      user = "telegraf:${builtins.toString(config.users.groups.podman.gid)}";
      volumes = [
        "/:/hostfs:ro"
        "/var/run/podman/podman.sock:/var/run/docker.sock:ro"
      ];
      environment = {
        HOST_ETC = "/hostfs/etc";
        HOST_PROC = "/hostfs/proc";
        HOST_SYS = "/hostfs/sys";
        HOST_VAR = "/hostfs/var";
        HOST_RUN = "/hostfs/run";
        HOST_MOUNT_PREFIX = "/hostfs";
        INFLUX_TOKEN="<redacted>";
      };
      cmd = [
        "--config"
        "http://influxdb2.dns.podman:8086/api/v2/telegrafs/0a4528fca40bd000"
      ];
      extraOptions = [
        "--hostname=nixos"
        "--network=foobar"
      ];
    };
  };

Notify maintainers
@mikroskeem
Metadata
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
this path will be fetched (0.01 MiB download, 0.04 MiB unpacked):
  /nix/store/48mnkga4kh84xyiqwzx8v7iv090i7z66-stdenv-linux
copying path '/nix/store/48mnkga4kh84xyiqwzx8v7iv090i7z66-stdenv-linux' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.64-1-pve, NixOS, 22.11 (Raccoon), 22.11pre425156.872fceeed60`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Unable to reproduce with my system with default network.
Tried with:
$ sudo podman run --rm -ti --name web1 python:3.10 python -m http.server 8080
$ sudo podman run --rm -ti --name webclient python:3.10 curl -v http://web1:8080

 - system: `"x86_64-linux"`
 - host os: `Linux 5.19.16, NixOS, 22.11 (Raccoon), 22.11.20221024.1e684b3`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(root): `""`
 - channels(mark): `""`
 - nixpkgs: `/nix/store/083m43hjhry94cvfmqdv7kjpvsl3zzvi-source`

I will try with newer nixpkgs and/or virtualisation.oci-containers soon.

Thanks for trying to reproduce the problem.
I found a workaround for getting dnsname working on the default network.
I noticed that, for the foobar network I created, /root/.config/cni/net.d/foobar.conflist gets created. Then I copied /etc/cni/net.d/87-podman-bridge.conflist into that folder, then dns started working on the default network.
cp /etc/cni/net.d/87-podman-bridge.conflist /root/.config/cni/net.d/
podman system reset
reboot


BugFix: Updating default node_version to v4.3 since …

Did this because v4.2 seems to be no longer hosted.
This does not fail gracefully, failing to install Node.js and stopping the playbook.
Bumping the version returns to expected functionality
Fixes #24

:+1:

So 4.2 was removed yesterday it looks like in favor of 4.3. Thanks for bumping this @ryanlelek, easy enough to bump locally too.
https://nodejs.org/en/blog/release/v4.3.0/

Thanks for the fix!
I need to hassle @rvagg to automate pushing this to Ansible galaxy at some point :smile:


Wrapping/unwrapping object with map inside it alters order of the map's elements

I have to wrap an instance of MyObject. The MyObject class has a TreeMap<Integer, String> myMap member - the order of the elements matters. Once I unwrap the object, the map is there, but the items inside it map are all messed up, and that's not good, as I need to calculate the hash of the map, and therefore I get two different hashes before wrapping it and after unwrapping it. It also restores the map as a HashMap object in the unwrapped object, instead of a TreeMap.

If you define your Map type as a TreeMap instead, Parceler will marshal to that type.

Alright, so this is happening because I defined my map using the interface Map and not the implementation class TreeMap. I do care about the order in this case, but I must say, it is a rather odd scenario too.

You could also use SortedMap instead of Map and Parceler will use a TreeMap for the default implementation.

Excellent! Thanks for the info.


Fix beautify package and add some CI features

beautify package don't truncates the log message
Add libraryInfo.go file, to support the signature display at the beautify package with constants
Add script to check that the variable LibraryModuleInfo is in sync with the latest git tag

Codecov Report

Merging #10 (d3061ac) into main (c175900) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main       #10   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           11        11
  Lines          328       328
=========================================
  Hits           328       328

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c175900...d3061ac. Read the comment docs.


API for opening files in registered custom editors

When an extension registers a custom editor, vscode automatically opens in the appropriate editor based on viewtype. However, other extensions currently have no way of doing this. It would be great if there were an API to determine the registered viewtype associated with a filename, so that it could be passed to the vscode.openWith command. Or another alternative would be to provide a command or API to open a file in the registered custom editor automatically (something like workspace.openTextDocument(), but which honors registered custom editors).
The use case for this is to open a file programmatically in the appropriate custom editor, which comes from a different extension.

You can use the vscode.openWith command with the id of the custom editor registered by other extensions.
No plans to expose a public api for getting all ids for a file since you usually interested in a specific editor type, not a list of all valid editors (which would only be meaningful if you know what each of the corresponding editors do)

How about a public API to open a file with whatever custom editor happens to be registered (the same behavior as when a user opens a file by double-clicking on it)? My example is the Test Explorer extension, which opens Ply .flow files in a text editor only because it has no way of allowing it to open in the Ply custom editor.


Feature: central coordination

It would be really nice to dump in a job description like "for repo noahgibbs/ruby, with SHA abc123, run all the normal benchmarks X times and send back a speed diff versus head-of-master, as well as all crashes/problems." But especially with x86 plus ARM, you want multiple workers running various jobs and coordinating. Our current "cron runs twice a day" architecture isn't going to stretch to cover that use case -- especially because it should allow multiple simultaneous job requests.
We'll need some kind of central coordinator. Maybe Jenkins? That would certainly be adequate to all our future needs, and it's not too hard to administer. There may be something simpler and adequate, not sure. But Jenkins beats rolling our own, so we should not roll our own.
Our most unusual requirement is that a worker must be 100% certain not to run anything else while benchmarking. And even that's not very unusual, just not necessarily the default. For the same reason, the coordinator should not double as a benchmarking worker, though it could be a reporting/miscellaneous worker.
Only the benchmarking workers need to be the expensive AWS metal instances. Everything else can be Google Cloud.
Not sure when I'll get to this. Not this month, at a minimum.

This is now underway. The result will soon be at DNS name ci.yjit.org, though not accessible to the public. The public face will remain speed.yjit.org and yjit.org.


Feature: MVP implementation of mvp

In this PR:

Base of new dashboard. Shows host information such as hostname, last connected, online status, ....
Realtime WebSocket support
New API routes/core logic to support getting health/hosts
Lots of customization through environment varialbes
More backend tests
Added some front-end tests
Some unrealted refactoring
Closes #12

@padukkas this is a huge PR. Don't feel the need to review everything. Just go over whatever you want and ask any questions.

Codecov Report

Merging #18 (eea3b5c) into main (58a1b82) will increase coverage by 0.38%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
+ Coverage   99.29%   99.67%   +0.38%
==========================================
  Files           5        7       +2
  Lines         141      311     +170
==========================================
+ Hits          140      310     +170
  Partials        1        1

Impacted Files
Coverage Δ

internal/logger/logger.go
100.00% <100.00%> (ø)

internal/service/health.go
100.00% <100.00%> (ø)

internal/service/host.go
100.00% <100.00%> (ø)

internal/utils/time.go
100.00% <100.00%> (ø)

internal/utils/variable.go
97.43% <100.00%> (+0.46%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 58a1b82...eea3b5c. Read the comment docs.


Create table with concatenated results from annotation

e.g. results/annotation/bacmet/bactmet_combined_results.tsv  which is just the concatenation (without duplicate headers) of all the individual bacmet results.txt with a sample label column added as needed.

Also fix, the repeated headers in the current DIAMOND concatenated results


bezier subdivisions algorithms

tesselateBezier Recursion termination when (d2 + d3)(d2 + d3) < engine->tessTol * (dxdx + dy*dy). I know d2 and d3 both are area, using cross product. Do you mean the condition is the edges are too short or the angle is too small?

The algorithm is adapted from here: http://antigrain.com/research/adaptive_bezier/

I understand tesselateBezier()  algorithms now. Another question is QuadTo() why not use midpoint.
    float vals[] = { WAC_COMMANDS_BEZIERTO,
        0.5f*(x0 + cx), 0.5f*(y0 + cy),
        0.5f*(cx + x), 0.5f*(cy + y),
        x, y };

I don't know what you imply by using midpoint. Is there another algorithm that converts quadric bezier to cubic by using a midpoint?

you are right the algorithm: http://fontforge.github.io/bezier.html


Update Developer's Guide for Add Feature

Fixes #95
Updated UML diagram attached in file.

Maybe can standardize the font sizes used in the UML diagram, but otherwise looks good.


How can I make it so pressing enter once creates a new paragraph?

Right now if I type these 2 lines
this is a paragraph
this is a different paragraph
it returns

this is a paragraph
this is a different paragraph

How can I make it so it outputs:

this is a paragraph

this is a different paragraph

Without having to press the enter button twice?

No idea why this got closed, but you have two options:

https://github.com/erusev/parsedown/wiki/Tutorial:-Get-Started (4th paragraph)
https://github.com/erusev/parsedown/blob/master/Parsedown.php#L58 (rewrite that as protected $breaksEnabled = true;


update to selector + respective test

Coverage remained the same at 69.035% when pulling 66f2c8b5d4f50e04bccd05eb0571926af7cc3e8a on 370-init-price into 6f737e178fc8bbe739966a98f7350acff1f1215a on master.


Suppress GCC warnings in prevector.h

This commit suppresses GCC's -Wmaybe-uninitialized for a couple of methods
in prevector.h. The code has not been modified since 2015, and is covered by
unit tests in prevector_tests.cpp.

Are there any difficulties to fix the actual warning instead? Even well test covered code could flake because of uninitialized variables.

The code is not any different in bitcoin. So I guess the warning is supressed in bitcoin differently, yet the code is the same. It does not look beneficial to me to change prevector from what bitcoin is having, except it was done also on bitcoin.

https://github.com/bitcoin/bitcoin/pull/14028 is what Bitcoin does. Their code landed in 0.18, probably, so we could just close this PR and cherry-pick their commit.

Yeah I suggest cherry-pick their commit. It's interesting that they went exactly the same path: suppress warning (https://github.com/bitcoin/bitcoin/pull/13902), and then fix them instead (https://github.com/bitcoin/bitcoin/pull/14028)

I cherry-picked their commit. After Travis finishes, it will need to be regular-merged into master.

utACK 142eec4621486d919700a2a6ecfd0ed8c38a40aa
Please make sure to update PR description as well as commit message. This should go as merge-commit, right?


Add support for filename patterns

Please add support for filename patterns. I'd like to copy file to [name][hash].[ext]. Plugin should add the file to the webpack stats so that it would be possible to generate filename mapping file.

Any news?

Filename patterns are now supported in v4.0.0


Modal images dimension

Currently, the modal images are limited with the doc images max-height = 500px.

https://cyberbotics.com/doc/guide/samples-environments

When the image is big, If the window is too small, the image is cut, it is too big the image is limited to a height of 500px.
Setting explicitly the max dimension as a percentage of the parent modal window solves these 2 issues:

https://cyberbotics.com/doc/guide/samples-environments?version=enhancement-modal-window-image-size

👍


Audio desync when converting

Hi! Thanks for maintaining this fork.
I see this was already brought up by another user but got closed without follow-up.
Issue: tried converting a video recording with audio enabled but audio was desynced.
Re-encoding the source footage with another ffmpeg-using program, then converting the re-encoded file with Webmconverter results in properly synced audio (but at the cost of some video quality loss due to the first re-encode).
This occurred with footage recorded by Nvidia Geforce Experience. I have tested with other videos, which seem to get converted with audio properly synced. So it could be related to the codecs the Nvidia utility uses? I have included mediainfo codecs.

I can't reply the issue  with a clip recorded by Nvidia Geforce Experience, the result is ok in my testing. Provide a clip in a free hosting service.

Here's a webm that has the audio desync from a Shadowplay recording;
https://drive.google.com/file/d/1aRJr3tpUS1aiJ4fxNno5Lhmp5F-Jq8PD/view
And here's if I try a mp4 conversion (same thing happens);
https://drive.google.com/file/d/1eudpYsFNo4zCkAruQZ9MAWy5o827ko1Q/view
And here's the original Shadowplay clip if you want it (I set it to 5 minutes by default so it's a big file);
https://drive.google.com/file/d/1vlPZiLTcrxn9l0gi0RzcBNMrzKhkF_EL/view

Here's a webm conversion that has the audio synced properly;
https://drive.google.com/file/d/1loZ3WUi5Vx6Oo4jNSzTBurgsuH4oTaxl/view
And here's the original Shadowplay clip (I changed it to 15 seconds so I'm not uploading another huge file);
https://drive.google.com/file/d/1lTIrLLhiQIUxwnaMOD5283sEFE38WYpO/view

The audio desync happens maybe... 30-45% of the time. It only happens with certain .mp4 files, but when it does it happens every time you try to make a clip off them.

log file for the first audio desynced webm:
Arguments for pass 1: -f nut -i pipe:0   -an -c:v libvpx-vp9 -pix_fmt yuv420p -threads 8 -slices 4 -metadata title="Street Fighter V 2022.08.13 - <IP_ADDRESS>.DVR" -lag-in-frames 16 -auto-alt-ref 1 -ac 2 -c:a libopus -b:v 14392k -qcomp 0 -fs 15728640 -b:a 64k -tile-columns 1 -row-mt 1 -pass 1 -passlogfile "C:\Users\oryx\AppData\Local\Temp\063ea330-23f9-4ecd-95eb-f34028349501" -f webm -y "NUL"
Arguments for pass 2: -f nut -i pipe:0   -c:v libvpx-vp9 -pix_fmt yuv420p -threads 8 -slices 4 -metadata title="Street Fighter V 2022.08.13 - <IP_ADDRESS>.DVR" -lag-in-frames 16 -auto-alt-ref 1 -ac 2 -c:a libopus -b:v 14392k -qcomp 0 -fs 15728640 -b:a 64k -tile-columns 1 -row-mt 1 -pass 2 -passlogfile "C:\Users\oryx\AppData\Local\Temp\063ea330-23f9-4ecd-95eb-f34028349501" -f webm -y "C:\Users\oryx\Desktop\Street Fighter V 2022.08.13 - <IP_ADDRESS>.DVR.webm"
--- CREATING AVISYNTH PROXY ---
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, nut, from 'pipe:0':
  Metadata:
    encoder         : Lavf56.40.101
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 59.94 fps, 59.90 tbr, 3833.71 tbn
  Stream #0:1: Audio: pcm_f32le (PFD  / 0x20444650), 48000 Hz, stereo, flt, 3072 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> vp9 (libvpx-vp9))
[libvpx-vp9 @ 000001b423603d80] v1.12.0-37-g7b65e4698
Output #0, webm, to 'NUL':
  Metadata:
    title           : Street Fighter V 2022.08.13 - <IP_ADDRESS>.DVR
    encoder         : Lavf59.26.100
  Stream #0:0: Video: vp9, yuv420p(progressive), 1280x720, q=2-31, 14392 kb/s, 59.90 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.36.100 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/14392000 buffer size: 0 vbv_delay: N/A
frame=  470 fps=155 q=0.0 size=       1kB time=00:00:00.00 bitrate=N/A speed=   0x

frame=  511 fps=156 q=0.0 Lsize=       1kB time=00:00:00.00 bitrate=N/A speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded
--- FFMPEG HAS EXITED ---
--- ENTERING PASS 2 ---
--- CREATING AVISYNTH PROXY ---
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, nut, from 'pipe:0':
  Metadata:
    encoder         : Lavf56.40.101
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 59.94 fps, 59.90 tbr, 3833.71 tbn
  Stream #0:1: Audio: pcm_f32le (PFD  / 0x20444650), 48000 Hz, stereo, flt, 3072 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> vp9 (libvpx-vp9))
  Stream #0:1 -> #0:1 (pcm_f32le (native) -> opus (libopus))
[libvpx-vp9 @ 00000280af4aeb00] v1.12.0-37-g7b65e4698
Output #0, webm, to 'C:\Users\oryx\Desktop\Street Fighter V 2022.08.13 - <IP_ADDRESS>.DVR.webm':
  Metadata:
    title           : Street Fighter V 2022.08.13 - <IP_ADDRESS>.DVR
    encoder         : Lavf59.26.100
  Stream #0:0: Video: vp9, yuv420p(progressive), 1280x720, q=2-31, 14392 kb/s, 59.90 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.36.100 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/14392000 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: opus, 48000 Hz, stereo, flt, 64 kb/s
    Metadata:
      encoder         : Lavc59.36.100 libopus
frame=  510 fps=5.5 q=10.0 size=   11264kB time=00:00:08.51 bitrate=10843.1kbits/s speed=0.0911x

frame=  511 fps=5.4 q=17.0 Lsize=   15043kB time=00:00:08.55 bitrate=14413.0kbits/s speed=0.0896x
video:14980kB audio:55kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.049479%
--- FFMPEG HAS EXITED ---

Video converted succesfully!

--- Final file size is 14.7 MB ---

log file for the second clip that works fine;
Arguments for pass 1: -f nut -i pipe:0   -an -c:v libvpx-vp9 -pix_fmt yuv420p -threads 8 -slices 4 -metadata title="GGXXACPR_Win 2022.08.16 - <IP_ADDRESS>.DVR" -lag-in-frames 16 -auto-alt-ref 1 -ac 2 -c:a libopus -b:v 15998k -qcomp 0 -fs 15728640 -b:a 64k -tile-columns 1 -row-mt 1 -pass 1 -passlogfile "C:\Users\oryx\AppData\Local\Temp\b37c76ec-260b-4122-914c-e16beaba3bdf" -f webm -y "NUL"
Arguments for pass 2: -f nut -i pipe:0   -c:v libvpx-vp9 -pix_fmt yuv420p -threads 8 -slices 4 -metadata title="GGXXACPR_Win 2022.08.16 - <IP_ADDRESS>.DVR" -lag-in-frames 16 -auto-alt-ref 1 -ac 2 -c:a libopus -b:v 15998k -qcomp 0 -fs 15728640 -b:a 64k -tile-columns 1 -row-mt 1 -pass 2 -passlogfile "C:\Users\oryx\AppData\Local\Temp\b37c76ec-260b-4122-914c-e16beaba3bdf" -f webm -y "C:\Users\oryx\Desktop\GGXXACPR_Win 2022.webm"
--- CREATING AVISYNTH PROXY ---
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, nut, from 'pipe:0':
  Metadata:
    encoder         : Lavf56.40.101
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 60 fps, 60 tbr, 61440 tbn
  Stream #0:1: Audio: pcm_f32le (PFD  / 0x20444650), 48000 Hz, stereo, flt, 3072 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> vp9 (libvpx-vp9))
[libvpx-vp9 @ 0000018c0b89dc00] v1.12.0-37-g7b65e4698
Output #0, webm, to 'NUL':
  Metadata:
    title           : GGXXACPR_Win 2022.08.16 - <IP_ADDRESS>.DVR
    encoder         : Lavf59.26.100
  Stream #0:0: Video: vp9, yuv420p(progressive), 1280x720, q=2-31, 15998 kb/s, 60 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.36.100 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/15998000 buffer size: 0 vbv_delay: N/A
frame=  446 fps=222 q=0.0 size=       1kB time=00:00:00.00 bitrate=N/A speed=   0x

frame=  460 fps=221 q=0.0 Lsize=       1kB time=00:00:00.00 bitrate=N/A speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded
--- FFMPEG HAS EXITED ---
--- ENTERING PASS 2 ---
--- CREATING AVISYNTH PROXY ---
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, nut, from 'pipe:0':
  Metadata:
    encoder         : Lavf56.40.101
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 60 fps, 60 tbr, 61440 tbn
  Stream #0:1: Audio: pcm_f32le (PFD  / 0x20444650), 48000 Hz, stereo, flt, 3072 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> vp9 (libvpx-vp9))
  Stream #0:1 -> #0:1 (pcm_f32le (native) -> opus (libopus))
[libvpx-vp9 @ 000001c851f2eb40] v1.12.0-37-g7b65e4698
Output #0, webm, to 'C:\Users\oryx\Desktop\GGXXACPR_Win 2022.webm':
  Metadata:
    title           : GGXXACPR_Win 2022.08.16 - <IP_ADDRESS>.DVR
    encoder         : Lavf59.26.100
  Stream #0:0: Video: vp9, yuv420p(progressive), 1280x720, q=2-31, 15998 kb/s, 60 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.36.100 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/15998000 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: opus, 48000 Hz, stereo, flt, 64 kb/s
    Metadata:
      encoder         : Lavc59.36.100 libopus
frame=  460 fps=7.5 q=8.0 size=   12288kB time=00:00:07.67 bitrate=13124.3kbits/s speed=0.125x

frame=  460 fps=7.3 q=12.0 Lsize=   14751kB time=00:00:07.69 bitrate=15714.3kbits/s speed=0.122x
video:14696kB audio:49kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.044905%
--- FFMPEG HAS EXITED ---

Video converted succesfully!

--- Final file size is 14.4 MB ---

I can't reply the issue with a clip recorded by Nvidia Geforce Experience, the result is ok in my testing. Provide a clip in a free hosting service.

Test recording source
Test recording converted webm with desync audio

This is probably caused for an external library see issue here. The latest compatible versions still doesn't fix the problem with these specifics codecs.

I began to have the the same audio issue about a month ago maybe, but with any youtube video that comes in mkv, the desync does not happen if I convert said mkv file to mp4 or if it comes from youtube already in mp4.

This is probably caused for an external library see issue here. The latest compatible versions still doesn't fix the problem with these specifics codecs.

I'm not sure if I'm allowed to link files on here, but I tried copy-pasting the ffms2.dll file from my (really) old version of WebmForR... into your branch and... The audio desync problem seems to be gone? I tried it with that Street Fighter 5 video I linked above and it's all synced up fine, and I have the added benefit of converting to mp4 that this version has (along with generally being faster).
If anyone's interested in trying for themselves, backup your current version of ffms2.dll and try using this version instead;
https://drive.google.com/file/d/1sVNtVFyIAnGs1ugBrHO1vdLoK5kWX1nb/view?usp=sharing
The clips I make from this now all synced up;
https://drive.google.com/file/d/1bFvsgAoeGTggOGzNZ9af_-1vCAtrJHym/view?usp=sharing

I tried that  with the GTA video and crashed.

Getting an error too (not finding any audio tracks when opening the file) with the dll you uploaded.
Tried with a couple older ffms2.dll I found from other installs, and desync still occurs.

Guys, update to the latest version 3.18.6. All my test worked fine and the problem is solve.

Awesome, problem seems fixed on my side as well.
How did you resolve the issue?

Separating responsibilities. These third -party libraries are very inconsistent.
I proceed to close the issue.


TypeError: _dense() missing 2 required positional arguments: 'axes' and 'with_reshape'

I've installed ferminet on Linux with Python 3.11 today using the following commands:
python -m venv --system-site-packages ~/venv/ferminet
. ~/venv/ferminet/bin/activate
pip install -U "jax[cuda12_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
pip install -e .

But when I try to run example calculation as
ferminet --config ferminet/configs/atom.py --config.system.atom Li --config.batch_size 256 --config.pretrain.iterations 100

I'm getting the following output:
<...>
I1129 13:46:55.584216<PHONE_NUMBER>50240 train.py:384] Starting QMC with 1 XLA devices per host across 1 hosts.
converged SCF energy = -7.43242052759577  <S^2> = 0.75000054  2S+1 = 2.0000005
I1129 13:46:56.988999<PHONE_NUMBER>50240 train.py:584] No checkpoint found. Training new model.
I1129 13:47:01.371712<PHONE_NUMBER>50240 pretrain.py:344] Pretrain iter 00000: 0.0532642 0.890625
<...pretrain iterations...>
I1129 13:47:01.964662<PHONE_NUMBER>50240 pretrain.py:344] Pretrain iter 00099: 0.000944124 0.945312
Traceback (most recent call last):
  File "/home/ktokarev/venv/ferminet/bin/ferminet", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/mnt/storage/src/git/ferminet/bin/ferminet", line 39, in <module>
    app.run(main)
  File "/usr/lib/python3.11/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/usr/lib/python3.11/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
             ^^^^^^^^^^
  File "/mnt/storage/src/git/ferminet/bin/ferminet", line 35, in main
    train.train(cfg)
  File "/mnt/storage/src/git/ferminet/ferminet/train.py", line 852, in train
    opt_state = optimizer.init(params, subkeys, data)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/optimizer.py", line 1025, in init
    return self._init(params, rng, batch, func_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/utils/staging.py", line 277, in decorated
    outs = func(instance, *args)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/optimizer.py", line 993, in _init
    estimator_state=self.estimator.init(
                    ^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/utils/misc.py", line 342, in wrapped
    return method(instance, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/curvature_estimator/block_diagonal.py", line 390, in init
    self.finalize(func_args)
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/utils/misc.py", line 315, in finalize
    self._finalize(*args, **kwargs)
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/curvature_estimator/block_diagonal.py", line 375, in _finalize
    self._jaxpr = self._jaxpr_extractor(func_args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tracer.py", line 466, in get_processed_jaxpr
    closed_jaxpr, _ = retrieve(func_args)
                      ^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tracer.py", line 432, in retrieve
    processed_jaxpr = ProcessedJaxpr.make_from_func(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tracer.py", line 321, in make_from_func
    func = tgm.auto_register_tags(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 2080, in auto_register_tags
    func_graph, tagged_locations = _auto_register_tags(
                                   ^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 1849, in _auto_register_tags
    sub_graph, sub_tag_locations = _auto_register_tags(
                                   ^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 1915, in _auto_register_tags
    matches = find_layer_tags_and_patterns(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 864, in find_layer_tags_and_patterns
    match = match_pattern(
            ^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 767, in match_pattern
    if len(pattern.jaxpr.outvars) != len(root_eqn.outvars):
           ^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 478, in jaxpr
    return self.graph.jaxpr
           ^^^^^^^^^^
  File "/usr/lib/python3.11/functools.py", line 1001, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 488, in graph
    return make_jax_graph(
           ^^^^^^^^^^^^^^^
  File "/home/ktokarev/venv/ferminet/lib/python3.11/site-packages/kfac_jax/_src/tag_graph_matcher.py", line 355, in make_jax_graph
    closed_jaxpr, out_shapes = jax.make_jaxpr(func, return_shape=True)(*func_args)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: _dense() missing 2 required positional arguments: 'axes' and 'with_reshape'
--------------------

Same error reproduces for me with other calculations. I also tried switching used kfac_jax revision from the head of main branch to a few recent commits, with exactly the same error happening.
Any idea what's going on and how can I make it work?

This bug also popped up when I tried to run it on a server. It seems that problems are resulted from kfac optimizer. After I substitude it by adam, everything works fine. However, the same version of modified codes on my own server works perfect with kfac optimzer, which was deployed in September. I'm also trying to figure out what's wrong with the mess. It is puzzling for kfac lib hasn't be updated since April and I use version 0.0.6 consistently.

It is puzzling for kfac lib hasn't be updated since April and I use version 0.0.6 consistently.

By default FermiNet uses master of kfac-jax which is updated quite often (and, according to its git history, those updates sometimes break things in other projects). I've tried to use 0.0.6 but it just didn't work.

It occurs to me that kfac-jax has a recent commit, which seems not to be compatible with this version of ferminet. So you can just pull down a history commit of version 0.0.6  kfac-jax and mannualy install it via python -m build command. I fixed this problem by replacing the latest version with commit hash code "315f59e".

It occurs to me that kfac-jax has a recent commit, which seems not to be compatible with this version of ferminet. So you can just pull down a history commit of version 0.0.6 kfac-jax and mannualy install it via python -m build command. I fixed this problem by replacing the latest version with commit hash code "315f59e".

Hi @SeldonHarry
Thanks for the inspiring hint! I am also struggling with running the sample code, but with a different error. I am wondering if you have encountered this problem #88
And I can't appreciate more if you can share the environment configuration that you are using. Many thanks in advance.

I fixed this problem by replacing the latest version with commit hash code "315f59e".

I'm sorry but I can see no such commit in kfac-jax: google-deepmind/kfac-jax@315f59e

This is from a change in the KFAC API (specifically this commit). I've just push an update which handles this change. (An alternative, as @SeldonHarry suggests, is to ping kfac-jax to a version just before the API change.)
Please re-open if this doesn't fix things for you.

@dibbla I haven't met such error till now. But I guess that it's caused  by jax and jaxlib.


feat: ensure domain uniqueness

Toward https://github.com/snapshot-labs/workflow/issues/142
Together with https://github.com/snapshot-labs/snapshot-hub/pull/928
When saving space settings with domain, this PR will also save the domain in the domain column, and ensure that the domain is unique
See the hub PR for database update

Any idea about the test @wa0x6e

Any idea about the test @wa0x6e

was unable to fix the tests, they're failing because of other tests, since the database is shared among all integrations, which are run in parallel
Running the test individually work (use dotenv -e test/.env.test yarn jest test/integration/writer/settings.test.ts)
This test is disabled for now, so we can move on


Feature request: easy thread-local resources

(Related issue: https://github.com/rayon-rs/rayon/issues/493)
Many use-cases for threadpools have tasks that need to spawn onto the pool with access to non-Sync and/or non-Send resources but where successive tasks could reuse the resources. A canonical example would be a non-Sync database connection, such as diesel::pg::PgConnection.
A standard way of handling this is with std::thread_local, but it's inconvenient. threadlocal! can't statically guarantee that values will be initialized before use, so all threadlocals end up wrapped in an Option or a RefCell.
Rayon's threadpool could guarantee that a resource is initialized on a worker thread before the first task was spawned onto it, and so could do away with that ceremony.
I think the API might look like this:
struct ResourcedThreadpool<T, F>
where
F: Fn() -> T
{
    resource_factory: F
}

impl<T, F> ResourcedThreadpool<T, F>
where F: Fn() -> T
{
   pub fn new(resource_factory: F) {}
   pub fn install<OP, R,>(&self, op: OP) -> R
    where
        OP: FnOnce(t: &T) -> R + Send,
        R: Send
   {
   }
}

Note that either F would need to be Send + Copy or or T would be Send. I think both cases would have their use, and I'm not sure which would be better. If the API could allow either way to work that might be the best.

BTW - if you like something resembling this idea - I would be willing to contribute the patch (with guidance).

Can you sketch how you were thinking of implementing this? It's not obvious to me how you would avoid the caveats you mentioned about thread_local! -- we even use that for rayon's own needs.

From an API point of view, I think it would be fine to use thread_local! to implement this, as long as Rayon takes care of initialization and safe unwrapping.
I'd piggyback on this thread_local  by extending WorkerThread to WorkerThread and have it hold a T inside of it.
Before priming, T gets initialized here along with the rest of WorkerThread:
https://github.com/rayon-rs/rayon/blob/master/rayon-core/src/registry.rs#L739-L759
Then WorkerThread::execute passes &T to JobRef::execute and JobRef passes it along to on to execute_fn.

I suspect changing to a generic WorkerThread<T> would be really invasive, but maybe a type-erased Box<dyn Any + 'static> field would work. Your ResourcedThreadpool wrapper would have the type for a downcase_ref when needed. I guess I'm assuming that you would just read the thread-local again, rather than trying to propagate through JobRef etc.
Along that line, could it be done with a separate thread-local, entirely as an external utility? Maybe still shipped with rayon, if we find a nice API for it, but not necessarily affecting the deep internals. As an outside layer, it seems more feasible that this kind of thing could be composed, letting the rayon pool be used by any number of clients without getting stuck on a particular T globally.

Hi, is there any updates? Thanks!


fix: model enum declaration

I was hitting No ChainID defined in extension as declare enum only creates the types but not the actual const after typescript compilation.

Summary by CodeRabbit

Refactor

Updated the declaration of the ChainID enum to enhance compatibility and maintainability.

Oops, sorry I missed this when copied over.


Uptime monitor; for reference see Uptime Robot

Done as part of https://github.com/orgs/appifyhub/projects/1.


Improved Neuron Morphology Plugin - Orientation

User should be able to see the morphology as a whole (zoomed out). In addition, the user wants to have the morphology in the correct orientation.
Need some extra investigation, @adulbrich needs to check with scientists on Jira

From @jonathanlurie:

camera.up.negate() for the orientation
optional param while instantiating to choose distance between soma and camera

Can have a look at https://sentry.io/organizations/blue-brain-project/issues/1578992122/?project=1845930&query=is%3Aunresolved&sort=freq&statsPeriod=14d please

Dhanesh to check with Jonathan for camera distance


Salsa derived functions currently don't support unionall argument types (where clauses don't work)

I'm surprised we haven't noticed this until now, but simple where clauses don't work right now:
Salsa.@derived multi_method(db::AbstractComponent, ::Type{T}) where T = sizeof(T)

The reason is because we are currently evaluating all the argument types to figure out what type of dictionary and key type to create for this function. We are currently using the (typeof(f) and Tuple{Type}) to represent individual methods like this, and so we were calling evalfor each argument type in order to build that tuple, but if you attempt toeval(Type{T}), of course T` is not defined.
There are a few approaches to take that would fix this:

Move the Key construction into the function body, instead of in the macro body, where the types are available. But we were doing it in the macro body because constructing the Dict() itself seemed to be expensive, and we were hoping to precompute this in the macro.
We should consider simplifying how we use these types for keys:

Maybe we should stop using separate dictionaries for each derived function, and just use one big dictionary for all derived functions?
And then, we can also stop keying the derived values by DependencyKey(key=DerivedKey{F, Tuple{Int,Any}}, args=(2,"hi")) and instead just key it by F and the actual args themselves, something like: DependencyKey{Derived, F}(2,"hi").

I don't think we actually need to keep track of which method was called, because julia can track that itself! The only problem would be if a user defines a new, more specific method after starting to use a Salsa state, in which case the key would incorrectly carry over to the new function, but I don't think that's a supported mode anyway.

Discovered this in #10, while fixing #9.


Seeing compile warnings in bigmeta.c

I am seeing compile warnings from this test program:
bigmeta.c: In function ‘buildatts’:
bigmeta.c:137:9: warning: unused variable ‘attid’ [-Wunused-variable]
     int attid;
         ^~~~~
bigmeta.c: In function ‘buildgroup’:
bigmeta.c:152:40: warning: unused variable ‘attid’ [-Wunused-variable]
     int i, grpid, dimid, typid, varid, attid;
                                        ^~~~~
bigmeta.c:152:26: warning: unused variable ‘typid’ [-Wunused-variable]
     int i, grpid, dimid, typid, varid, attid;
                          ^~~~~
bigmeta.c:152:19: warning: unused variable ‘dimid’ [-Wunused-variable]
     int i, grpid, dimid, typid, varid, attid;
                   ^~~~~
bigmeta.c: In function ‘main’:
bigmeta.c:198:37: warning: unused variable ‘attid’ [-Wunused-variable]
     int grpid, dimid, varid, typid, attid;
                                     ^~~~~
bigmeta.c:198:30: warning: unused variable ‘typid’ [-Wunused-variable]
     int grpid, dimid, varid, typid, attid;
                              ^~~~~
bigmeta.c:198:23: warning: unused variable ‘varid’ [-Wunused-variable]
     int grpid, dimid, varid, typid, attid;
                       ^~~~~
bigmeta.c:198:16: warning: unused variable ‘dimid’ [-Wunused-variable]
     int grpid, dimid, varid, typid, attid;
                ^~~~~
bigmeta.c:198:9: warning: unused variable ‘grpid’ [-Wunused-variable]
     int grpid, dimid, varid, typid, attid;
         ^~~~~
bigmeta.c:196:9: warning: unused variable ‘stat’ [-Wunused-variable]
     int stat = NC_NOERR;
         ^~~~

There doesn’t appear to be anything serious reported by these warnings, and doesn’t really merit an open issue


update gfx942 bbs tn gridbased

[----------] Global test environment tear-down
[==========] 48206 tests from 13 test suites ran. (499036 ms total)
[  PASSED  ] 48206 tests.
hipBLASLt version: 1100
command line: ./hipblaslt-test


Hi,

Hi,
In short, my impression is that GM/T 0005-2012 testing is often done on post-processed output, not the actual physical entropy. Is this correct?
So to me it seems that the at least the "old" GM/T 0005-2012 an RNG test, not an entropy source test. I looked at the requirements and it seemed very similar to the obsoleted 800-22 which was a test covering also pseudorandom output. Entropy sources do not need to pass 800-22 either.
Anyway, in terms of timelines, it would seem sensible to look closely at the newer standard and see if it is compatible with AIS-31 and SP 800-90B standards (the old one is not really compatible with AIS-31/PTG.2 or 90B, as it does not seem to concern entropy. just "statistical randomness"). If it is not compatible and there is still a need for ISA support, it will need a separate instruction or a CSR interface. The seed entropy source is specifically an entropy source.
Cheers,

markku

Originally posted by @mjosaarinen in https://github.com/riscv/riscv-crypto/pull/142#issuecomment-988712228

Is this where you meant to leave this comment? ;)

No!


When I pull the a2d2 docker image from docker hub, I have a readme file in documentation, so that I know how to configure and run itage.yml

Adding a steps to fetch the readme.md and update it as description of docker image on docker hub,

@vdeshkar how is this tested?

@vdeshkar how is this tested?

@marianbuenosayres
As here is the changes of a workflow file, when we merge it then it will start build, and it will show its execution


Test the master branch compability with the TiDB 7.5/7.1/6.5

Test https://github.com/pingcap/tidb-dashboard/commit/473bf80ad4f2765884fbe0deb9f5c1c268c4cf07 with 7.5.0, 7.1.3, 6.5.6, all works fine.


Export named delete breaks

See https://github.com/sindresorhus/dot-prop/issues/32
When having an export named delete this module will cause the following error to be thrown by rollup:

Error parsing node_modules/dot-prop/index.js: Unexpected token (78:4) in node_modules/dot-prop/index.js

Maybe automatically renaming delete to _delete, or having a config option where one can specify a new name...

This is the same as #116 and is fixed in 6.0.0. Let us know if you still have problems, thanks


Small mistake in documentation

🔎 Search Terms
https://github.com/winstonjs/winston#usage
The problem
The comments about "Write all logs with importance level of error or less to error.log" is incorrect. It should rather be "Write all logs with importance level of error or higher to error.log"
Same applies to the next comment
This really gave me a hard time testing to find out the right way. Hope it doesn't trouble others
What version of Winston presents the issue?
v3.11.0
What version of Node are you using?
v18.18.0
If this worked in a previous version of Winston, which was it?
No response
Minimum Working Example
No response
Additional information
No response

Maybe it would be more clarifying to say
higher (i.e., error, fatal, but not other levels)
higher (i.e., fatal, error, warn, and info, but not trace)
just to be super explicit for folks who aren't familiar with standard logging levels.  PR welcome.

Closed via #2507


A problem using Grib.Api in Azure Functions project

When I call GribFile constructor at first time everything goes fine, but when I try iterate throught it application breaks down and exit.
  using (GribFile file = new GribFile(@"file_name.grib2"))             {                 var msg = file.First();             }
It all looks similar to issue #22

Your title mentions Azure Functions, does that mean that you don't have any problem when running your program on a local machine?
"Application break down" sounds like an unhandled expcetion, that causes the program to terminate.
If that is the case, try wrapping the code in side a Try/Catch to see which type of exception is thrown and what the exception message is.
What version of GribApi do you use?
Have you had a look at #52 ?
It also described a problem when running in Azure Functions.

When I use it in Console project everything works. I surround this code with Try/Catch but it doesn't throw any exceptions and application just break down (maybe exception throws on lower levels). I use version 0.7.1 and I also looked at #52 but there used version 1.0.0

Please try upgrading to newest beta to see if that solves your problem.
There once was a problem with uncatchable exceptions being thrown, but that should have been solved.

What is the value of GribEnvironment.DefinitionsPath?
Is the folder Grib.Api correctly copied from the nuget package to the code you deploy?
The message you see indicates that GribApi.NET could not locate the definitions folder.

The value for GribEnvironment.DefinitionsPath looks like string.Empty from the exception message. Yes I copy Grib.Api folder to the wwwroot folder on Azure Functions and Grib.Api folder contains definitions folder, but it all doesn't help, I still get the same mistake

Have you to explictly set GribEnvironment.DefinitionsPath to the absolute path pointing to Grib.Api/definitions?
GribApi.NET has some heuristics to search for the definitions folder, but it won't do an exhaustive search for it.

Yes, it helps. But now I face the same problem as here #52. As I understand the solution is to manually copy Grib.Api.Native.dll to bin folder?

Glad it helped.
To my knowledge there is no "official" way to make it work.
If it was me, I would add a link in the csproj file to the right Grib.Api.Native.dll to force it being copied to the root of the bin folder.

I hurried up with results(
           GribEnvironment.DefinitionsPath = @"D:\somepath\wwwroot\Grib.Api\definitions";             log.Info("defPath: " + GribEnvironment.DefinitionsPath);
when I run this code on AzureFunctions logs show me
2018-05-13T14:45:31.214 [Info] defPath:
and then I still get an exception about GribEnvironment::DefinitionsPath must be a valid path.
It looks like setter for GribEnvironment.DefinitionsPath doesn't work correctly

UPD: I achive running beta-4 on Azure Functions, but setter for DefinitionsPath is still doesn't work.

Could you put a few words on how you got it working?
It will help others with similar problems.


Adding Karma and related dependencies

After adding some *.spec.js test files just run 'npm test start' to run the tests.  May have to add test file locations in the karma.conf.js files list.

Can you close this and open it against dev instead, first? That way I don't need to merge from master to dev.

Reopened against dev


[Enhancement] Ignore the failed log for first retry for FE thrift client

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[ ] 2.5
[ ] 2.4
[ ] 2.3
[ ] 2.2

run starrocks_fe_unittest

[FE PR Coverage Check]
:disappointed: fail : 0 / 2 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/rpc/FrontendServiceProxy.java
0
2
00.00%
[51, 54]

run starrocks_admit_test


Rename Java package to better fit to our project

Change in decision of renaming packages. Will design our application in parallel.


chore: specify autora-core versions < 4

... because only version 3 includes the state objects we use here. A new update will be needed for core version 4.

Overtaken by #39


Mask RCNN - Normalization - mean & std

Hi,
I have trained my own dataset on Mask RCNN. Since my GPU is limited, i want to set a smaller resize by training and validation.
So, at the train pipe line, the resized image = 512, 512 (default 1333, 800)
At the val pipe line, the resized image = 695, 695.
I would like to know how can i adjust the "mean" and "std" by "img_norm_config". Is there a tutor for it?
thanks!

You can compare the two options：

Instead of calculating the mean and std, it is just using the imagenet value in the default config. Then use coco pre-training weights to fintune.
Iterate over the entire image dataset, then recalculate the mean and std. Finally use coco pre-training weights to finetune.

The image mean and std are inherits from the backbone model, which is pre-trained on  ImageNet.


Increase performance by storing response body separately

Context
If you have a file with multiple large multi line response bodies, it will cause php-vcr to become very slow
What has been done

On storing the requests, looks for the response body and puts it in a separate file
On loading, it will test if the body has been replaced with a file reference, and replaces the placeholder with the content fetched from the file

How to test

Make a test that requests multiple large response bodies

Todo

improve detection of placeholder?
This is basically a RFC PR, not per se final

Notes

caused a functional test of mine to go from 30s to 0.5s

just wanted to point out.... on the master branch (merged in 2018, no releases yet?) there is a perf improvement on the YAML storage that significantly helps out.  this should be ported over to JSON?

I have not tested it, is the performance increase so huge?
My approach is different tho, by storing the response body in a separate file, you don't need to loop over the many lines of a large multi line response.
Perhaps a combination of both would be even faster :)


Specify no recombination rate

Hello there,
I am working with genomic data of an ancient parthenogenetic nematode species. Therfore, its recombination rate should be close to 0 but we do not know for sure. That is why I was wondering if I can use your tool without specifiying a recombination rate?
If possible, how would I do that?
I appreciate your help.
Best,
Laura Pettrich

Hello Laura,
Can you tell me what you are trying to infer ?  As first analysis I would try to estimate the ratio of recombinaton  over mutation  (rho/theta) and make sure it is very small (in the parameters of eSMC or eSMC2 you can set Boxr=c(3,3) ).  If you are searching for selfing or signature of parthenogenesis it would decrease this value even more. If you don't observe a very small recombination rate, it might suggest that something else is going on.
From my side I never ran eSMC on data with recombination rate equal to 0. I will perform simulations and look at it. I will come back at you in couples of days with the results. In addition, we will soon submit ( ~ 2 weeks) a new paper and I will update the current version of eSMC2. You might want to wait for this new version as it offers new features which could help you with this problem.
Best,
Thibaut Sellinger

Hello Thibaut,
Thank you for your quick reply. It is really great to hear that you will update eSMC2 soon, looking forward to this since we think about implementing this tool in upcoming experiments. Right now I intend to test this tool in a small project and look at the data I get from two different species. So right now it is my first look at your tool.
What info would you need exactly? This nematode species is a really challenging case because it is not only pathenogenetic but it was also in a dormant stage for a very long time. So I intend to look especially at the signature of dormancy but the signature of selfing would also be intersting to infer.  Maybe this species has too complex life traits to infer its history at the moment but nevertheless we wanted to try.
Thanks a lot for offering to run simulations with the recombination rate equal to 0. In the meantime I will try to implement your suggestions.
Best,
Laura Pettrich

Hello !
Good News ! I did some simulations and eSMC2 should be able to tell you if recombination rate is null or not (I find a recombination rate of  ~  10^-17, convergence can be quite slow and take some time).  However, the problem is now that SMC methods do not perform good in absence of recombination events (so you can give up on recovering the demographic history if the recombination rate is really small   https://doi.org/10.1111/1755-0998.13416  ).
If you do not know the recombination rate (and have no idea of it's potential values), I would give up using eSMC2 to search for parthenogenesis or dormancy.  However if you have boundaries on the potential values of the recombination rates, I would uses those two values to run eSMC2 ( and to still estimate rho/theta). If in both cases, the rho/theta you find is smaller than what is expected, it means that a latent mechanism is diminishing rho/theta. It could  either be dormancy, parthenogenesis or both. Disentangling form both effect is the tricky part and cannot really help you. But this could help you to point toward a direction. if you have selfing (or parthenogenesis), i expect that diversity within individual will be less than diversity between individuals (you might want to read more about this). However, dormancy will simply increase diversity will not lead to such effect.
The best case would be if you had a sister species or population (and can assume they share recombination rate) and knew that the other population does not do dormancy or parthenogenesis. You could then simply infer in both case rho/theta, and impute the difference between the two estimated value to either dormancy and/or parthenogenesis and infer their rates (without requiring to actually know the recombination rate).
I will try to have the new version of eSMC2 uploaded within two weeks and will also update the tutorial for you so that you can run what I suggested.
I hope this will help you ; best,
Thibaut
PS: to calculate the ratio you can simply do : result = eSMC2(....) ;  rec/mu =  (result$rho/result$mu)


[WFLY-14940] Upgrade MicroProfile Health to 3.1 and SmallRye Health t…

…o 3.1.1
https://issues.redhat.com/browse/WFLY-14940

@jmesnil can you please take a look? Particularly not sure about the transformer model changes and tests if you can confirm that I got it right, please.

/retest

/retest

Added test cases for the empty-startup-checks-status flag.

/retest

@xstefank Possible clue why these continue to fail CI -- I've noticed that on the CI jobs these tests are the first ones to run in the testsuite/integration/manualmode surefire execution. When I ran this locally (and perhaps you too) they were in the middle.

@bstansberry thanks, they run first on my machine, also passing when they are the only ones run.

/retest


Compile-time KCL parser

Background
The KCL lexer and parser are both deterministic, pure functions. The lexer takes a Rust string and outputs a Vec. The parser takes that Vec and outputs a KCL Program.
Problem
In #922 we want to create a KCL stdlib. Right now that involves compiling KCL at runtime, and if it can't be compiled, we panic. It'd be better to compile KCL to its AST at compiletime. This way we could check at compile time that the KCL standard library is valid.
Solution
I suggest a kcl! macro like
let my_fn: Program = kcl! {
  fn cube = (pos, scale) => {
    const sg = startSketchAt(pos)
      |> line([0, scale], %)
      |> line([scale, 0], %)
      |> line([0, -scale], %)

    return sg
  }

  const b1 = cube([0,0], 10)
  const pt1 = b1[0]
  show(b1)
}

I think this would need to be a proc macro. It would take the text within the kcl!{ ... } block, put it into a Rust string, then run it through the KCL lexer and parser. If it succeeds, expand the macro into a KCL Program AST type. If it fails, compile error.

I would avoid the layer of abstraction and just go:
let my_fn: KCLProgram = kclCompile("...
...
...
");

It would make reasoning about any KCL code easier instead of guessing what the macro is doing, right?

We already have a function to compile KCL at runtime, which is your suggestion.
A macro would be more complicated, but it would let you parse KCL at compile time, not runtime.
This solves the problem of never compiling the WASM with some static KCL program embedded (e.g. in the standard library) which is parsed at runtime and, at runtime, discovered to be invalid. A macro would let you know the KCL is valid before users open the wasm library let alone run it.


Add .gitattributes file to use the right line endings

STATUS: not ready
This enforces the following EOL character in the following files:

LF in zsh related files: .sh, .zsh, .zsh-theme and _* (completion). This setting forces \n as a line ending, given it is the only acceptable one parsed by zsh. This is important for cygwin users, for instance.

Native EOL (default in core.eol setting). This will use the native EOL in all text files except above, regardless of the core.autocrlf setting which is considered harmful at this point. This means files like READMEs and such. I'm not 100% sure about this point, feel free to discuss it.

Binary files (as interpreted by git) will not be EOL-converted.

The current choice is to enforce LF on all files by default. This is probably wrong but it is temporary until we figure out the best possible choice.
Closes #3090.
Closes #4069.
Related issues:
#1363, #1483, #2770, #3079, #3595, #3687, #3963, #4402, #4665, #4783.
Related links:

http://blog.subgit.com/line-endings-handling-in-svn-git-and-subgit/
http://git-scm.com/docs/gitattributes

i think this is a good idea.
But I'm not a fan of setting native EOL. If I understand it correctly, that'll turn on line ending conversion for some files in the Cygwin environment, even if autocrlf is off. That'll mess stuff up, bc a lot of Cygwin tools expect Unix style line endings. I think this should be removed.
IMHO, enabling autocrlf in Cygwin is a user mistake, since Cygwin is a little schizophrenic about line endings. The best situation is for line conversion to not happen for any file under Cygwin. We're just working around that here, not supporting an environment where line ending normalization is needed or expected.
Since there's no non-Cygwin "native" Windows zsh AFAIK, we don't have to worry about supporting that situation, which is where cold conversion would actually be appropriate.
I'd consider going so far as to skip the *.sh pattern matching, and make .gitattributes a one-liner that disables text for every file in the repo. I think that's the most correct behavior under Cygwin.

The native default is for only READMEs and other txt files. This way Windows users could still correctly view and edit those files (for instance, if they are using notepad). This is a really fringe situation though, and I'll go back on my decision if needed.
Can you elaborate on how that will break stuff?

I had misunderstood the behavior at least partially. I think I get it better now after rereading the Git documentation some more.
On plain Windows, Git considers the "native" line ending to be CRLF, but under Cygwin it's LF. That makes me less worried; I had mistakenly thought it was using CRLF for "native" under Cygwin.
Sorry in advance for the long-windedness. Here goes...
First off, I don't think !eol works. The gitattributes syntax for unsetting a value is -name, not !name. But unsetting a value isn't the same as making it "unspecified"; it's the same as setting it to false. So I don't know if you can get the "use native line endings" behavior by unsetting eol. Though I'm not sure if that's because the -name syntax doesn't work to revert eol to "unspecified", or if it's because in the normal case, an "unspecified" eol is overridden by autocrlf=true. I've tested it out with some local repos on Cygwin, and neither !eol nor -eol seem to have any effect. eol=lf does, but that would force line normalization to Unix style under plain Windows too.
Potential Issues
There are a couple scenarios where I see potential problems:

Text processing of natural language text using Unix tools
Computer-readable text with other extensions
Files that really do need to be CRLF everywhere
It just complicates things

None of these are big cases, but they're a consideration. Probably as much as the case for CRLF conversion: any decent Windows text editor works fine with Unix mode files, and users advanced enough to be running Cygwin, zsh, and OMZ are advanced enough to have something better than Notepad.
Text processing of natural language text using Unix tools
This is when someone on Cygwin wanted to work with the human-readable markdown or txt files as text, but wanted to use grep, sed, zsh, pico or other Unix tools to do the work.
I don't think this is as big a problem as it used to be. I tested several of the text editors and text processing tools in current Cygwin, and they all seem to handle CRLF mode text just fine. Users may have to adjust the regular expressions they use with the tools, though.
But there are still edge cases here, and while it sounds exotic, I think I even have a live example: in #3913 (https://github.com/apjanke/oh-my-zsh/tree/histsubstr-update-2015_04_26), I wrote a zsh script to programmatically update a plugin from its upstream repo. Part of this involves adding OMZ-specific stuff to its README.markdown file. If normalization and "text file only" CRLF checkouts were happening, I think this means that I'd have a CRLF README file, and then the zsh script would append a few paragraphs of LF text to it, so you'd have a mixed-mode file. What happens then? Do I just have a locally-messed-up file that gets fixed upon checkin? Does it affect how the normalization happens?
Stuff involving asciidoc, ebook, and docbook processing might fall under this too, if we have plugins for those.
Computer-readable text with other extensions

Other shells: .bash, .ksh, .csh files
Scripts in other languages like .py, .pl, and .m files
Files with no extension that are external commands written as scripts with shebang lines
Data files like .1 roff-formatted man pages
Data files with no extension
Other file types we've overlooked

These are computer-readable scripts and should probably not be normalized, for the same reasons the .zsh and .sh files aren't. People could add other unforeseen extensions too.
The way the .gitattributes in the PR is written, it makes text=auto the default, and only excludes a couple specific .zsh-related extensions. This means any script-like stuff or text-based data format is going to be detected as text. This could be addressed by making "not text" the default, and only positively identifying known-text extensions like .txt and .markdown as text.
Examples:

The current PR doesn't include *.zsh-template, so I think the installer running under autocrlf would produce a broken ~/.zshrc.
plugins/gitfast/git-completion.bash
The chucknorris plugin has a text data file named chucknorris to be processed with strfile. strfile uses LF-only, so it produces a bad data file and the plugin breaks.
.conf files like plugins/tmux/tmux.extra.conf – I don't know if tmux likes CRLF files, but I wouldn't bet on it

Files that really do need to be CRLF everywhere
Some files really should be in DOS mode, regardless of your platform or conversion settings.

Windows-related files for plugins that integrate with native Windows applications might need config or script files to be in CRLF mode.
If we ever write tests, we might want test data to see if things work on CRLF data, like DOS mode text, or traffic from internet protocols like SMTP or HTTP.
There's probably some screwy X.509 certificate-related file format that specifies CRLF.

If text normalization is on, these will be CRLF in the author's working copy and test out okay, get normalized to LF on checkin (even if they're not running autocrlf), and then be in incorrect LF mode when checked out for other users who aren't normalizing.
It just complicates things
Aside from specific breakage, enabling text normalization just complicates things, and makes it harder to reason about what's going to happen between different users. This is exacerbated because we're not just using Git as source control for developers; we're using it as a distribution and deployment mechanism for users, too.
How do things behave when you're using a mix of Cygwin and native Git tools to work with a repo? That's not too fringe: I use both Cygwin command-line git and the Windows GUI SourceTree for working with OMZ repos.
What's the behavior under various configurations when users pick up an update with the new .gitattributes? What happens when we eventually revise .gitattributes and it goes out in an update? Which version controls the normalization of files in an update that include an update to the normalization definitions in .gitattributes? Are there cases where CRLF files in the working copy might be seen as local changes after a change to .gitattributes alters how they should be normalized, thus preventing future updates from running?
For files that need exceptions to this (e.g. as more computer-readable formats are found, or a particular file needs to be CRLF), will changes to plugins then be entangled with changes to the root .gitattributes?
What happens if you have your repo on a shared drive that's mounted across multiple platforms? What if it's in your Dropbox or Google Drive? What if you use VM file sharing to move files between your Windows and OS X VMs directly (say, while testing cross-platform behavior of a potential PR) instead of going through an intermediate Git repo? What if it's under a roaming profile on Windows? What about downstream repackagers who redistribute OMZ outside Git and aren't very sophisticated about this stuff?
And philosophically, actively supporting normalization like this just encourages people to use autocrlf with this or other projects, which imho complicates things for everyone.

Should this auto-close #3090?

I switched to LF on all files by default. Will refine when I have the time. I also added references to issues which would have been solved by this, to show that a solution is needed even if we have to check for all the edge cases you indicated. Notice that on Windows, the Github clients set core.autocrlf by default (though babun doesn't ship with it, which makes it not have that problem).

Hello, I just updated oh-my-zsh and got some CRLF errors for the second time this month.
Can we get this merged?
Going through the changed files and removing the CRs is really annoying.
Is there anything preventing this from being merged?

@a-p-o extensive testing and a check of all the edge cases made by @apjanke. I understand this is a very frequent issue but there isn't enough data to merge it. I'd rather instruct users on how to solve their errors than corrupting their repositories and forcing a reinstall.
Hopefully I can make a big enough test suite soon now that I'm testing docker images.

I've refrained from this approach due to the concerns expressed by Andrew above. A better approach is to fix the git configuration of the repository, setting up the appropriate value for the conflicting settings that users encounter.
These include:
core.eol = lf
core.autocrlf = false


Just Brilliant

Hats off to whoever created this.  You get my highest compliments.  It changed my thinking.
There is a huge problem with the MSM blackout of progressive news.  It is not like Tienamen Square in China, where there are literally no mentions of the massacre.  The web pages are there, just the MSM algorithms do not show us the pages.   How do we circumvent the the media blackout.  Well with RSS.
I have quite a few RSS feeds and a custom build feed reader, I curate and republish the news, but this is really the missing piece.
Okay, so where do I find your list of RSS feeds?
I run https://GreenMaps.US.  I have quite a large stack or RSS feeds.
I am not quite sure how I will interact with this project., but I am certainly going to be thinking about it.

Hi thank you very much
at the moment only I can add rss feed to the import. Can you shoot me the URLs so I can add them.
There is yet the API to do this (https://github.com/davidesantangelo/dato.rss/wiki/Feed)  in the future I will unlock it.
Keep an eye on the project.

I signed up to watch this project.  I recently started using Ruby, and prefer it to python, particularly for multiple processors.  But Since my servers are in Python, on the ZODB, I will use the Whoosh search library to index my RSS feeds.
How sad.


Unexpected mma -> mma layout conversion

I am trying to reimplement Praxis's dot product attention with a lazy broadcast prefix.
My code is the following:
import numpy as np
import torch
import triton
import triton.language as tl

def strides_length(shape):
  size = np.prod(shape)
  for s in shape:
    size = size // s
    yield int(size)

@triton.jit
def load_one_of(a, b, tile, threshold):
  return tl.load(tl.where(tile < threshold, a, b))

@triton.jit
def fused_attention_with_lazy_broadcast_prefix_kernel(
  # inputs
  Q, K_prefix, K_suffix, V_prefix, V_suffix, mask,
  # dimensions
  batch_size: tl.constexpr, seq_len: tl.constexpr, num_heads: tl.constexpr,
  head_dim: tl.constexpr, prefix_size: tl.constexpr, suffix_size: tl.constexpr,
  # outputs
  L, M, encoded,
  # block information
  block_q: tl.constexpr, block_k: tl.constexpr, block_d: tl.constexpr
):
  """Reimplementation of Praxis's lazy broadcast prefix attention.

  See https://github.com/google/praxis/blob/main/praxis/layers/attentions.py#L2068.
  """
  # note: block_d is here assumed to be == to head_dim

  head_stride = head_dim
  block_stride = num_heads * head_stride
  batch_stride = seq_len * block_stride

  block_index = tl.program_id(0)
  head_index = tl.program_id(1)
  batch_index = tl.program_id(2)

  # Load q tile
  q_tile = batch_index * batch_stride \
         + (block_index * block_q + tl.arange(0, block_q))[:, None] * block_stride \
         + head_index * head_stride \
         + tl.arange(0, block_d)[None, :]

  q = tl.load(Q + q_tile)

  # Prepare k and v tiles
  initial_kv_tile = batch_index * batch_stride \
                  + tl.arange(0, block_k)[:, None] * block_stride \
                  + head_index * head_stride \
                  + tl.arange(0, block_d)[None, :]
  kv_tile = initial_kv_tile

  nb_kv_prefix_elements = batch_size * prefix_size * num_heads * head_dim

  # Initialize local SRAM buffers
  m_i = tl.zeros([block_q], dtype=tl.float32) - float('inf')
  l_i = tl.zeros([block_q], dtype=tl.float32)
  acc = tl.zeros([block_q, block_d], dtype=tl.float32)

  for i in range(0, seq_len // block_k):
    start_k = tl.multiple_of(i * block_k, block_k)
    # -- compute qk --
    k = load_one_of(K_prefix + kv_tile, K_suffix + kv_tile, kv_tile,
                    nb_kv_prefix_elements)
    qk = tl.dot(q, tl.trans(k))
    # -- apply attention mask --
    # mask_tile =
    # -- apply row softmax --
    m_ij = tl.max(qk, axis=1)
    probs_ij_softmax_exponents = qk - m_ij[:, None]
    probs_ij = tl.exp(probs_ij_softmax_exponents)
    l_ij = tl.sum(probs_ij, axis=1)

    # -- construct new m_i and l_i --
    m_i_new = tl.maximum(m_i, m_ij)
    alpha, beta = tl.exp(m_i - m_i_new), tl.exp(m_ij - m_i_new)
    l_i_new = alpha * l_i + beta * l_ij

    # -- update accumulator --
    probs_scale = beta / l_i_new
    probs_ij = probs_ij * probs_scale[:, None]

    acc_scale = l_i / l_i_new * alpha
    acc *= acc_scale[:, None]
    v = load_one_of(V_prefix + kv_tile, V_suffix + kv_tile, kv_tile,
                    nb_kv_prefix_elements)
    acc += tl.dot(probs_ij, v)

    # -- update m_i and l_i --
    m_i, l_i = m_i_new, l_i_new

    # -- update kv tile --
    kv_tile += initial_kv_tile

  # Write back L, M, encoded
  lm_head_stride = seq_len
  lm_batch_stride = num_heads * lm_head_stride
  lm_tile = batch_index * lm_batch_stride \
          + head_index * lm_head_stride \
          + (block_index * block_q + tl.arange(0, block_q))

  tl.store(L + lm_tile, l_i)
  tl.store(M + lm_tile, m_i)
  tl.store(encoded + q_tile, acc)

def shape_matches(shape, target_shape):
  if len(shape) != len(target_shape): return False
  for component, target in zip(shape, target_shape):
    if target != -1 and component != target: return False

  return True

def fused_attention_with_lazy_prefix_broadcast(q, k_prefix, k_suffix,
                                               v_prefix, v_suffix, mask,
                                               block_q: int = 64,
                                               block_k: int = 64):
  """Implementation of FlashAttention with lazy prefix broadcast.

  The implementation is based on Pallas's MHA implementation, with handling for
  lazy prefix broadcast.

  See https://github.com/jax-ml/jax-triton/blob/mlir/jax_triton/pallas/ops/attention.py#L93
  and https://github.com/google/praxis/blob/main/praxis/layers/attentions.py#L2068.
  """
  batch_size, seq_len, num_heads, head_dim = q.shape
  assert shape_matches(k_prefix.shape, (batch_size, -1, num_heads, head_dim))
  assert shape_matches(k_suffix.shape, (batch_size, -1, num_heads, head_dim))
  assert shape_matches(v_prefix.shape, k_prefix.shape)
  assert shape_matches(v_suffix.shape, k_suffix.shape)

  block_q = min(block_q, seq_len)
  block_k = min(block_k, seq_len)
  grid = (triton.cdiv(seq_len, block_q), num_heads, batch_size)

  encoded = torch.empty_like(q)

  l = torch.empty((batch_size, num_heads, seq_len), device=q.device,
                  dtype=torch.float32)
  m = torch.empty((batch_size, num_heads, seq_len), device=q.device,
                  dtype=torch.float32)

  metaparams = dict(
      block_q=block_q,
      block_d=head_dim,
      block_k=block_k,
      num_warps=4 if head_dim <= 64 else 8,
      num_stages=2)

  fused_attention_with_lazy_broadcast_prefix_kernel[grid](
      q, k_prefix, k_suffix, v_prefix, v_suffix, mask,
      *q.shape, k_prefix.shape[1], k_suffix.shape[1],
      l, m, encoded,
      **metaparams)

  return encoded, l, m

Unfortunately, I encounter the following error when attempting to compile on A100 with Triton at HEAD---apparently, pretty much independently of the input shapes:
triton/lib/Analysis/Allocation.cpp:41: std::pair<llvm::SmallVector<unsigned int>, llvm::SmallVector<unsigned int> > mlir::triton::getCvtOrder(const mlir::Attribute&, const mlir::Attribute&): Assertion `!(srcMmaLayout && dstMmaLayout) && "Unexpected mma -> mma layout conversion"' failed.

I read from issue #1298 that this may happen when the optimizer doesn't do its job well. Is there a known workaround that might work in this case? If not, I'd be happy to help with debugging this if someone could point me in the right direction :)

A workaround is to store temporary values to the global memory and then reload from it.

Great, thanks! That unblocks me for the time being.

@bchetioui Can you show me final implementation code?

Sorry, I saw this and couldn't reply at the time---and then forgot about it. Unfortunately, I do not have code that I can share for this at the moment. Do you still need it?

A workaround is to store temporary values to the global memory and then reload from it.

i tried to use tl.save with tl.load to bypass this issue
it can compile and run
but it seems that the saving doesn't work properly
the stride info is not processed correctly so that some columns are missing


platform-existing: update docs for deploying on existing k8s cluster

Update docs for installing Kubeflow on an existing Kubernetes Cluster.
Signed-off-by: Yannis Zarkadas<EMAIL_ADDRESS>

This change is

/assign

@sarahmaddox thanks a lot for all the useful feedback!
I have incorporated the changes in a new commit, so that it's easier to check them.

/lgtm
/approve
Thank you!


[BUG] nodes_over_time not working

Hello,
I started working on TGNN recently and found this repo very useful!
I installed TGX from and not sure if the bug has been fixed in the latest release.
I am trying to use nodes_over_time on tgbl-flight dataset and got this error.

From dtdg.data.keys() we can see that it is because how discretize_edges works, the key of the dtdg.data is not necessary start from 0.

BTW, is there any function that turn a snapshot of temporal graphs into the the form of PYG Data object? Want to see static GNNs' performance on these dynamic datasets.

Hi,
Thanks for your interest in our work and reporting this issue.
did you load dtdg graph from your own dataset? did you obtain it by tgx.Graph.discretize function? If not, you can see our tutorial on how to load dataset.
The plotting function requires the snapshots to start from 0 and then increment 1 by 1, in addition, there should be no gaps in the timestamps. A simple solution for your issue above is to remap the integers to start from 0. I have wrote a helper function in tgx.Graph.shift_time_to_zero which will remap the snapshot ids to start from 0.
For your second question. it is non-trivial to turn a temporal graph into a static graph in PyG because edges can exist in multiple snapshots, nodes are different at each snapshot etc. What you can do however, is to construct TemporalData object in PyG.
You can achieves this quite easily by converting numpy array into torch tensor and then construct TemporalData. You can get the numpy array by using tgx.Graph.export_full_data.
data = TemporalData(
            src=src,
            dst=dst,
            t=ts,
        )

Best,
Andy

Thanks for the response!

issue closed


GitHub issues

Docs Build status updates of commit 8d1edd6:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/nuget-org/individual-accounts.md
:white_check_mark:Succeeded
View

docs/nuget-org/media/let-this-app-access.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/let-this-app-surface.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 3273fad:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/nuget-org/individual-accounts.md
:white_check_mark:Succeeded
View

docs/nuget-org/media/let-this-app-access.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/let-this-app-surface.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 5ab3aa7:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/nuget-org/individual-accounts.md
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-enter-code-2fa.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-register-2fa.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-sign-in-2fa.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 974d5cf:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/nuget-org/individual-accounts.md
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-enter-code-2fa.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-register-2fa.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-sign-in-2fa.png
:white_check_mark:Succeeded
View

docs/quickstart/includes/publish-api-key.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit f3f2383:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/nuget-org/individual-accounts.md
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-enter-code-2fa.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-register-2fa.png
:white_check_mark:Succeeded
View

docs/nuget-org/media/nuget-org-sign-in-2fa.png
:white_check_mark:Succeeded
View

docs/quickstart/includes/publish-api-key.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

#sign-off


NTR: Weep fluid collection (pouring)

Proposed Term Label
Weep fluid collection (pouring) [GENEPIO:0101003
Proposed Associated Field(s)
collection method
Definition/Description
A specimen collection process that is used to collect weep fluid via pouring from the source container into the sample collection device. (e.g. bag or tube)
Definition Source
No response
Guidance
No response
Alternative labels and/or synonyms
No response
Additional Comments (not an annotation)
No response
Optional - Contributor Attribution
No response

https://github.com/cidgoh/GRDI_AMR_One_Health/tree/v10.0.0


Add HR, Power, Cadence heat-point style options

Expose additional stream metadata for heatmapping.

Completed in build on 6/1 along with mapbox gl js v19 upgrade


Update Groq index.ts

Fixed Groq index.ts
💻 变更类型 | Change Type

[ ] ✨ feat
[x] 🐛 fix
[ ] ♻ refactor
[ ] 💄 style
[ ] 🔨 chore
[ ] 📝 docs

🔀 变更说明 | Description of Change
Updated Groq index.ts to fix the following error during the effort of adding support for Groq. If I am not mistaken, Groq.Combine points to CompoundedIcon in<EMAIL_ADDRESS>src/app/settings/llm/Groq/index.tsx:43:15 - error TS2339: Property 'Combine' does not exist on type 'CompoundedIcon'.

43         <Groq.Combine
                 ~~~~~~~

Found 1 error in src/app/settings/llm/Groq/index.tsx:43

📝 补充信息 | Additional Information

It appears I missed a file... Opened another PR #14 to fix it @arvinxx


[Migrator] Use Version::getCurrentLanguageVersion() as latest version

Instead of using hardcoded number, use version::Version::getCurrentLanguageVersion() as the latest supported Swift language version.

@swift-ci Please smoke test

Hm. Aren't 3 -> 4 and 3 -> 4.2 and 4 -> 4.2 all possible migrations, though?

@jrose-apple Currently, only 4.2 is supported as the migration target. We don't have the UI to specify the target in the first place.


[AVRO-3448][rust] Rust name/namespace spec adherence work

Description
In the Encoding and Decoding recursive flows, the schema definitions are being added to a map ad hoc. This means for input values that do not recurse to the schema definition needed by a later schema ref, there is an encoding error when both the input schema and input Value/bytes are valid.
Ex: The following schema fails to encode/decode when a is the first union variant (null)
{
            "type":"record",
            "name":"TestStruct",
            "fields": [
                {
                    "name":"a",
                    "type":[ "null", {
                        "type":"record",
                        "name": "Inner",
                        "fields": [ {
                            "name":"z",
                            "type":"int"
                        }]
                    }]
                },
                {
                    "name":"b",
                    "type":"Inner"
                }
            ]
        }

In the parsing, encoding, and decoding recursive workflow, the enclosing namespace needed to globally uniquely define a named schema type is not being passed recursively to help fully qualify a schema's name. This leads to the potential of ambiguous or undefined schema errors.
Ex: the following fails to parse when it should
{
          "name": "record_name",
          "namespace": "space",
          "type": "record",
          "fields": [
            {
              "name": "outer_field_1",
              "type": [
                        "null",
                        {
                            "type":"record",
                            "name":"inner_record_name",
                            "fields":[
                                {
                                    "name":"inner_field_1",
                                    "type":"double"
                                }
                            ]
                        }
                    ]
            },
            {
                "name": "outer_field_2",
                "type" : "space.inner_record_name"
            }
          ]
        }

Ex the following succeeds when it should fail
{
            "type":"record",
            "name":"outer",
            "namesapce":"outer.space",
            "fields":[
                {
                    "name": "field1",
                    "type": {
                        "type" : "record",
                        "name" : "middle",
                        "namespace":"middle.space",
                        "fields":[
                            {
                                "name":"middle_field_1",
                                "type": {
                                    "type":"record",
                                    "name":"inner",
                                    "fields":[
                                        {
                                            "name":"inner_field_1",
                                            "type":"double"
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                },
                {
                    "name":"field2",
                    "type":"inner"
                }
            ]
        }

The proposed solution here extends from the conversation has in the comments of https://github.com/apache/avro/pull/1582 by fulling resolving all the names to schema reference before all encoding and decoding workflows. In addition, this PR also updates the classes, and types that work with names to make them easier to work with. It uses these updated classes and types to passes a namespace type recursively so it can be used in fully qualifying the name of Named types in parsing, encoding and decoding workflows. The fully qualified name is ultimately what is used for schema lookup to avoid all ambiguity.
Breaking Changes

[ergo] Aliases moved out of the name struct to match with Java SDK (something suggested by @travisbrown with regards to another struct that I echo), and for clarity within this flow that uses names as a HashTable index.
[ergo]  encode_ref now includes the names lookup table and enclosing namespace input arguments. Can be replaces by the encode function which I believe was serving the same purpose)
[functional] users with incorrect schemas that were parsing for the wrong reasons will have errors (hopefully few)
[functional] users who were for some reason catching panics on the encode flow will no longer be able to catch their errors that way and must unwrap the Result (Compiler warning only to let them know)

Future work

Use the fulling indexed schema (now called resolved schema) to help speed up encoding and decoding workflows ( avoid indexing schemas every time)
Better type/name resolution error messages
~Avoid panics on encoding flow and set up AvroResult<()> returns with proper errors~
Value Resolution flow can make use of resolved schema with enclosing namespace (Its recursively adding schemas to hashmap). Believe it might have the same bug as the encoding flow.

Jira
https://issues.apache.org/jira/browse/AVRO-3448
Tests

My PR adds unit tests that are design to test taking the name/namespacing spec to the extreme.
I including many of the same schema in tests for the parsing/encoding/decoding workflows. The test dependency (have to parse a schema to encode with it, have to encode with a schema before decoding with it, etc)
All previous unit tests unchanged except for trivial constructor changes.

Documentation

I have added docs in some places where I believe the flow or use case to be non obvious
more can be added based on review feedback and what gets questions.

Thanks @jklamer !
I will try to review the PR soon! It is rather big so it may take some time.

@travisbrown You might want to review this PR too.

thank you for the review @martin-g !

Hey! Pushed some changes to remove the panics from the encode flow plus refactored the naming to match the other modules. Found some other future work that is relevant and related but in the interest avoiding scope creep I want to get some buy-in before I proceed.
Things that can be added to this PR that are significant:

Value Resolution flow has the same schema lookup strategy as the encoding, decoding, and parsing flows, which means it has the same potential namespacing issues. The flow can be changed to match the others. Better finding this now rather than later 🙃
Add in encoded size to the Ok() return from encode. This was discussed but would be a lot of code change not related to the original encoding panic bug that led to this PR.

I also added in more future work and breaking change (slight) to the PR's description.

@jklamer A small request: please don't use forced pushes because now I have to re-review everything. It is much easier to review only the new commits and Squash and merge at the end. Thanks!

Thank you for the contribution, @jklamer !


player-external-http uses content-type video/unknown

The option --player-external-http uses the content-type video/unknown. it would be better to use video/mp4 since a player might decide to download the file instead of playing it if the video type is now valid.
Even better would be some kind of type detection or to add an option where the content-type can be set.

it would be better to use video/mp4

Changing this is not really difficult. All necessary changes can be done in http_server.py.
  conn.send(b"HTTP/1.1 200 OK\r\n")
  conn.send(b"Server: Livestreamer\r\n")
- conn.send(b"Content-Type: video/unknown\r\n")
+ conn.send(b"Content-Type: video/mp4\r\n")
  conn.send(b"\r\n")

http debug: http: server='<IP_ADDRESS>' port=8080 file='/'
core debug: net: connecting to <IP_ADDRESS> port 8080
core debug: connection succeeded (socket = 2800)
http debug: protocol 'HTTP' answer code 200
http debug: Server: Livestreamer
http debug: Content-Type: video/mp4
core debug: using access module "http"