Feature Request: sort ships by various criteria

It would be nice if the dashboard had an option to sort ships by other criteria than alphabetical. Examples include:

boot date
startram status
running status (on / off)
disk usage
ram usage

Also in my opinion, boot date would be a better default than alphabetical. I often boot a new ship and forget the name and have to go back to my moon manager to check what it was named.

https://github.com/Native-Planet/GroundSeg/pull/639
Fixed version


Dataset release

Thank you for your interesting research,
I want to ask if it is possible for your team to release the dataset of the shot clips. I think it is important for the result visualization step.

The authors state, "We release a dataset of roughly 20k labeled match cut pairs for two types of match cuts: character framing and motion."
Yes, please! 🙏

the point is that I tried to find these films somewhere, but if found that there is very high chance, that timestamps won't be aligned


Add devfile description to the devfile CNCF page

Which area this documentation change is related to?
/area documentation
CNCF
Issue details
On the CNCF website the devfile description needs to be updated.
https://www.cncf.io/projects/devfile/

There is now a link in devfile.io referencing CNCF. We still need to update https://www.cncf.io/projects/devfile/

Can you be more specific on what needs to be updated? I mean what kind of description do you need?

@cdrage Can you clarify the request? All other CNCF projects only have a single-line description of the project on that link page and that's what we currently have.

You're right, the description is only regarding what sandbox level they are. I'll close this issue.


Correction for raw affiliation Nantes Université, Movement -Interactions -Performance, MIP, EA 4334, F-44000 Nantes, France.

Correction needed for raw affiliation Nantes Université, Movement -Interactions -Performance, MIP, EA 4334, F-44000 Nantes, France.
raw_affiliation_name: Nantes Université, Movement -Interactions -Performance, MIP, EA 4334, F-44000 Nantes, France.
new_rors: 03gnr7b55;04q2ax456
previous_rors: 03gnr7b55
works_examples: W4322009008
contact: 50e8246df367ca072597a744ebba08c8:d90df37269d8864a2df9bb3b16be22 @ univ-nantes.fr

This issue was accepted and ingested by the OpenAlex team on 2024-11-18. The new affiliations should be visible within the next 7 days.


Some questions about generating results

Hello, I am a newbie in this direction. Your article is very well written. I have a question about the generation of results, that is, about the implicit decoder, which is just a classifier, so how are the coordinates of the real points generated? thanks

The naive way, which is what we did, is to simply query every grid point under certain resolution. For example, say we set the resolution to be 64^3, then we just take all the 64^3 grid points in the voxel  and input to the decoder.
Of course, this would cause many unnecessary queries, so a more efficient way is proposed in OccNet. And I believe nowadays there are many other works that try to improve the efficiency of implicit network.

The naive way, which is what we did, is to simply query every grid point under certain resolution. For example, say we set the resolution to be 64^3, then we just take all the 64^3 grid points in the voxel and input to the decoder.
Of course, this would cause many unnecessary queries, so a more efficient way is proposed in OccNet. And I believe nowadays there are many other works that try to improve the efficiency of implicit network.

ok, I got it.

In other words, after the bounding box is generated, it is gridded, and then the coordinates of each voxel are calculated separately and then sent to the decoder, right? Does the final result need to be consistent with the number of points during training?Thank you for your patient reply.

In other words, after the bounding box is generated, it is gridded, and then the coordinates of each voxel are calculated separately and then sent to the decoder, right?
Partly right. Each part is generated separately in its own local space by the decoder using 64^3 points, so every part is within a 64^3 box. Then we use the predicted bounding box to transform each part to its correct position in the global space.

Does the final result need to be consistent with the number of points during training?
Not necessarily. The part autoencoder(implicit decoder) is trained under 64^3 resolution, but you can use any resolution in testing time, because point coordinates are normalized into [0, 1] before putting into network. This is essentially a good property for implicit neural representations, you can refer to DeepSDF, IM-NET, OccNET, etc.

In other words, after the bounding box is generated, it is gridded, and then the coordinates of each voxel are calculated separately and then sent to the decoder, right?

Partly right. Each part is generated separately in its own local space by the decoder using 64^3 points, so every part is within a 64^3 box. Then we use the predicted bounding box to transform each part to its correct position in the global space.

Does the final result need to be consistent with the number of points during training?

Not necessarily. The part autoencoder(implicit decoder) is trained under 64^3 resolution, but you can use any resolution in testing time, because point coordinates are normalized into [0, 1] before putting into network. This is essentially a good property for implicit neural representations, you can refer to DeepSDF, IM-NET, OccNET, etc.

ok,thank you very much


Fix the tests

This adds wget as a dependency in order to fix the CI

Sorry for taking so long to get to this.  Thanks for catching this! I had temporarily removed wget from the deps when testing a conda build but forgot to add it back. We will be replacing wget with parfive or urllib very soon, so as to avoid issues like this in the future.


slurm: <IP_ADDRESS> -> <IP_ADDRESS>

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/slurm/versions.
meta.description for slurm is: '"Simple Linux Utility for Resource Management"'.
Compare changes on GitHub

Checks done (click to expand)

built on NixOS
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sacct had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sacctmgr had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/salloc had a zero exit code or showed the expected version
/nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sattach passed the binary check.
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sbatch had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sbcast had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/scancel had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/scontrol had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sdiag had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sinfo had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/smap had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sprio had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/squeue had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sreport had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/srun had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sshare had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/sstat had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/strigger had a zero exit code or showed the expected version
/nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/slurmctld passed the binary check.
/nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/slurmd passed the binary check.
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/slurmdbd had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin/slurmstepd had a zero exit code or showed the expected version
3 of 22 passed binary check by having a zero exit code.
0 of 22 passed binary check by having the new version present in output.
found <IP_ADDRESS> with grep in /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>
directory tree listing: https://gist.github.com/d24827b3f6207c1f7b0f6bdd46357a8b
du listing: https://gist.github.com/a4146fa4aa7a9cd40091aea32115de0c

Outpath report (click to expand)

Outpath difference report
14 total rebuild paths
4 package rebuilds
4 x86_64-linux rebuilds
4 i686-linux rebuilds
2 x86_64-darwin rebuilds
4 aarch64-linux rebuilds
First ten rebuilds by attrpath
python27Packages.pyslurm
python36Packages.pyslurm
slurm
slurm-spank-x11

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS> \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A slurm https://github.com/r-ryantm/nixpkgs/archive/de625d4ae9be80cc99a0132deae7415d07612bad.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>
ls -la /nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>/bin

cc @jagajaga @markuskowa for testing.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: slurm
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: slurm
Partial log (click to expand)
/nix/store/5bpiax51vkz2kdalj526nddm1zw9k02n-slurm-<IP_ADDRESS>

Closing duplicate https://github.com/NixOS/nixpkgs/pull/49121 in favor of this PR.

@GrahamcOfBorg test slurm

Success on aarch64-linux (full log)
Attempted: slurm
Partial log (click to expand)
gzipping man pages under /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>/share/man/
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>/lib  /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>/bin  /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>/sbin
patching script interpreter paths in /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>
checking for references to /build in /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>...
moving /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>/sbin/* to /nix/store/w4r261rrgz7wn9b39ajzxww0rvvc4gzx-slurm-<IP_ADDRESS>/bin
shrinking RPATHs of ELF executables and libraries in /nix/store/d3phdx60nw0xvmpkgmi7a7s2ah3zklpk-slurm-<IP_ADDRESS>-dev
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/d3phdx60nw0xvmpkgmi7a7s2ah3zklpk-slurm-<IP_ADDRESS>-dev
checking for references to /build in /nix/store/d3phdx60nw0xvmpkgmi7a7s2ah3zklpk-slurm-<IP_ADDRESS>-dev...

@GrahamcOfBorg build python36Packages.pyslurm

Failure on x86_64-linux (full log)
Attempted: python36Packages.pyslurm
Partial log (click to expand)
INFO:
INFO: Cython version 0.28.5 installed
INFO:
INFO: Clean - checking for objects to clean
INFO: Clean - completed
INFO: Build - Found Slurm header in /nix/store/awgqzm0f0bf9xh5mdicb8wvf1vwmj2fs-slurm-<IP_ADDRESS>-dev/include
INFO: Build - Detected Slurm version - 0x120803 (18.08.03)
ERROR: Build - Incorrect slurm version detected, require Slurm-18.08.00 to slurm-18.08.00
builder for '/nix/store/wypl50vb6jd12lk5v54ar8q35syrvg31-python3.6-pyslurm-20180908.drv' failed with exit code 1
error: build of '/nix/store/wypl50vb6jd12lk5v54ar8q35syrvg31-python3.6-pyslurm-20180908.drv' failed

Failure on aarch64-linux (full log)
Attempted: python36Packages.pyslurm
Partial log (click to expand)
INFO:
INFO: Cython version 0.28.5 installed
INFO:
INFO: Clean - checking for objects to clean
INFO: Clean - completed
INFO: Build - Found Slurm header in /nix/store/d3phdx60nw0xvmpkgmi7a7s2ah3zklpk-slurm-<IP_ADDRESS>-dev/include
INFO: Build - Detected Slurm version - 0x120803 (18.08.03)
ERROR: Build - Incorrect slurm version detected, require Slurm-18.08.00 to slurm-18.08.00
builder for '/nix/store/jds087skiac2y68klzk7gklhnsf4pbgk-python3.6-pyslurm-20180908.drv' failed with exit code 1
error: build of '/nix/store/jds087skiac2y68klzk7gklhnsf4pbgk-python3.6-pyslurm-20180908.drv' failed

Success on x86_64-linux (full log)
Attempted: tests.slurm
Partial log (click to expand)
test script finished in 63.11s
cleaning up
killing node3 (pid 597)
killing node2 (pid 609)
killing node1 (pid 621)
killing control (pid 633)
killing submit (pid 645)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/wmk33rwpldkdjyyxnzsl37s7p2pqzwfg-vm-test-run-slurm

Success on aarch64-linux (full log)
Attempted: tests.slurm
Partial log (click to expand)
test script finished in 145.02s
cleaning up
killing control (pid 631)
killing node1 (pid 643)
killing submit (pid 657)
killing node2 (pid 670)
killing node3 (pid 683)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/6yaxfidxw9xrgr3v4adw1m10f84c0ph2-vm-test-run-slurm

Waiting for pyslurm upstream update to 18-08-03.


Nick: Watch spinandspark.in

Nick requests the watch of the watch_keyword spinandspark\.in. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

spinandspark\.in has been seen in 1 true positive, 0 false positives, and 0 NAAs.


Manage in Cloud Explorer does not show in VS 2022 like the page suggests

Manage in Cloud Explorer does not show in VS 2022 like the page suggests.
How can I get to "Cloud Explorer" ?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6e88ea08-81ef-2b5a-b9d6-539e85592672
Version Independent ID: f2d8b7b4-906a-d1f5-5201-fc22a7890955
Content: Quickstart: Create your first C# function in Azure using Visual Studio
Content Source: articles/azure-functions/functions-create-your-first-function-visual-studio.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

Hello @Ofer-Gal - Thank you for your feedback! We will review and update as appropriate.

Hi, got the same issue, no Cloud Explorer.

@Ofer-Gal @nailenTV Cloud explorer has been removed in VS2022 in favor of using Connected Services for Function Apps: https://docs.microsoft.com/en-us/visualstudio/azure/vs-azure-tools-resources-managing-with-cloud-explorer?view=vs-2022
I have assigned the issue to the doc author to update appropriately.

@SamaraSoucy-MSFT would you know what the replacement for Cloud Explorer is for Azure IoT Edge apps? They are containers that are connected to the Azure Container Registry through IoT Hub. But can't find a way to deploy an IoT Edge app built in Visual Studio 2022 with Cloud Explorer removed. There is no Connected Services node (it's likely only for Azure Functions) in my solution.

Thanks! We are updating this article now.


Removed dependency on itself.

This was causing go build build the app correctly. It was running the
vendored code instead of the code being changed. That was fun to figure
out.

Thanks @varikin!!! Not surprisingly (to you) this is my first time using vendored deps. Thanks for fighting through it and submitting this back.

No problem! I am just about month ahead of you with learning Go and Glide :)


RFC: Restore AudioCVT Interface (w/o CVec)

So in PR #269 AudioCVT was removed in a commit by @nukep
I have a patch that replaces this w/ an implementation that uses a caller allocated Vec, let me know what you think: drbawb/rust-sdl2@aa42ceeba3ca3af386a7b8e7a9d2c2c0c146e9bf
A couple key areas to review:

len, len_mult, and now len_cvt are int32 in SDL2; but interpreted as uint's in the bridge:

Are they guaranteed to never be negative?
Should we add a debug_assert! to that effect?

I set the length of the resultant Vec<u8> before checking the error code:

Is len_cvt well defined if the conversion failed?
Should we defer setting the length until after checking the return code?

Style nit: std::mem, std::raw are presently imported by the functions which use them; rather than at the module level.

I actually like this; it feels like it contains the "unsafe imports" to unsafe areas of the code.
However it does mean we have duplicate imports in several functions now...

Let me know if this looks worthy of a PR.

Those are some very excellent points.

It seems like they shouldn't be negative, although the SDL docs don't say so explicitly: https://wiki.libsdl.org/SDL_AudioCVT.

Because the size of c_int varies across platforms, maybe use std::num::cast()?

I'm not quite sure what happens if the len is within the bounds of c_int, but len_cvt is larger than c_int. SDL doesn't bounds-check it as far as I can tell: https://github.com/spurious/SDL-mirror/blob/7f5d524dca8bd2cfa1603c0ae2d6f52c18d53348/src/audio/SDL_audiocvt.c

It's probably safer to set the length on succeed.

The SDL docs say that len_cvt is set on successful return, but says nothing about failure. https://wiki.libsdl.org/SDL_ConvertAudio

It's semantically clearer anyway, as the vector is then always in a defined state and always with initialized values. Dropping a vector with uninitialized values seems like a Bad Thing to do.

I personally prefer that style of importing too, especially for "unsafe imports" like you said. I'm uneasy whenever I see something like use std::mem; at the module level (because namespace pollution and all).

A unit test would also be a good thing to have. I got stuck because I was getting some wonky results (for me it somehow changed new_buffer_expected or some other value (I don't remember), so it might have been a buffer overflow). Here's what I had at the time:
#[cfg(test)]
mod test {
    use super::{AudioCVT, AUDIOU8};
    #[test]
    fn test_audio_cvt() {
        use std::iter::repeat;

        // 0,1,2,3, ...
        let buffer: Vec<u8> = range(0, 255).collect().shrink_to_fit();
        // 0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3, ...
        let new_buffer_expected: Vec<u8> = range(0, 255).flat_map(|v| repeat(v).take(4)).collect();

        let cvt = AudioCVT::new(AUDIOU8, 1, 44100, AUDIOU8, 4, 44100).unwrap();
        let new_buffer = cvt.convert(buffer).unwrap();
        assert_eq!(new_buffer, new_buffer_expected);
    }
}

If the above issues are addressed, then it'll look good for a PR.

Hah, I'm starting to see why you ripped this out! See the latest here: drbawb/rust-sdl2@902d4725f4ac284ed2b3e6d97b3f0ddc4ad3f123.
After some cleanup and fixes: I ended up w/ a convert function that works for duplicating channels, e.g:
let cvt = AudioCVT::new(AUDIOU8, 1, 44100, AUDIOU8, 2, 44100).unwrap();
However, when quadrupling the stream, I see the 'buffer overrun' you brought up:
let cvt = AudioCVT::new(AUDIOU8, 1, 44100, AUDIOU8, 2, 44100).unwrap();
If I allocate a bunch of padding at the end of the vec (space for one additional channel to be precise): then the expected result buffer is undamaged, but the result buffer has the wrong result.  See this gist for the failed assertion.
It looks, to me, as though it duplicates the stream and then sets the 2 remaining channels to 0 or 1.
If I reserve only the space specified by len * cvt.len_mult then I see the "buffer overrun" behavior you're talking about; which is obviously very bad.
What's interesting, to me, is that the Vec should move elsewhere in memory when the capacity changes.
Yet it consistently overwrites new_buffer_expected; and it's always perfectly zero'd (on my machine.)
Very strange indeed; any thoughts before I throw this through valgrind a couple times?

Hmm; I just adapted your test-case to my branch that has the copy of the c_vec module, and I yanked out the original implementation of AudioCVT.
Again it works when duplicating channels; but w/ CVec it segfaults when quadrupling channels.

Hmm, this is bad; it could be a bug in SDL. I'll whip something up in C real quick and see if the problem occurs there as well.

Welp, it looks like it's an SDL bug.
Test case: https://gist.github.com/nukep/57aa85135f56bd838d37
My results: https://gist.github.com/nukep/0863a485356e4e22481c
It might be possible that the SDL folks already fixed this since the version on my machine. If I feel like it, I might checkout the latest SDL and test it with that.
But as far as I can tell, our code is correct.

Compiled your test w/ clang 3.5.0 and SDL 2.0.3 (which I believe is the latest stable?) -- I'm getting similar results.

Since this can be reproduced in C: I'm tempted to submit a PR to add CVT back in to the bindings, but perhaps we should mark AudioCVT#convert() as unsafe?
It's probably best that we get the public API back to the way it was before the FFI changes; but we should make callers aware of the safety issues we've uncovered.

It should at least be marked as #[unstable] IMO, but I don't know about marking it as unsafe. AudioCVT#convert() is only "unsafe" because of an underlying bug in SDL; the nature of the operation is not fundamentally unsafe, i.e. shouldn't require an unsafe { } block. Additionally, whether or not AudioCVT#convert() is bug-prone completely depends on which version of SDL is linked.
I absolutely agree, though; this issue should be documented to the caller.

#[unstable] sounds good to me (and we did just change the public API.)
I added a text annotation that points to this issue for anyone curious.
I'm going to wait to submit a PR until I can rebase this on top of #272

Closed by #275


[BEAM-1575] Adding validatesrunner test for metrics in sources

r: @chamikaramj
Adding a ValidatesRunner test that checks that  metrics work within sources.

The failure is due to lint failure, but it seems to be due to a different set of files (not affected by my change). I guess this is normal?

Coverage decreased (-0.003%) to 70.293% when pulling e20c4ead8513aacd7334d797f183590993f1cacc on pabloem:source-metrics-test into b47fd52cb87bccf821e8f66471b6e4d949fe9077 on apache:master.

This failure is from java. The Python tests pass.

@chamikaramj ping? : )

Retest this please.

Coverage decreased (-0.01%) to 70.284% when pulling e20c4ead8513aacd7334d797f183590993f1cacc on pabloem:source-metrics-test into b47fd52cb87bccf821e8f66471b6e4d949fe9077 on apache:master.

I have tested this on the dataflow service. Also I just relaunched the JEnkins tests.

Coverage decreased (-0.3%) to 70.046% when pulling e20c4ead8513aacd7334d797f183590993f1cacc on pabloem:source-metrics-test into b47fd52cb87bccf821e8f66471b6e4d949fe9077 on apache:master.

LGTM


Single point of entry docs webpage

http://www.mkdocs.org/#mkdocs

mkdocs seems to have an issue with combining code blocks inside lists.
Will also try https://github.com/dauxio/daux.io

http://sap.github.io/chevrotain/website/


Website on mobile freezes on load when there are too many tasks

On pc it's working fine, but on mobile it's like this: I can't change resin, check tasks done and even see map and domain. Check it on Chrome, Opera. And it's working like this when I log in to my account, without log in it's working fine.

Closing this issue due to the lack of information. I cannot reproduce this issue on my iPhone and I haven't received any similar reports recently, so this is assumed fixed.


Latest version of chef-vault needs newer gem version

This cookbook pins the chef-vault gem version to 2.2.2 in its attributes which is too old for the latest chef-vault cookbook (Error message below). Please remove this version pin...
       Recipe: chef-vault::default

           - install version 2.2.0 of package chef-vault

         ================================================================================
         Recipe Compile Error in /tmp/kitchen/cache/cookbooks/atlassian-common/recipes/default.rb
         ================================================================================

         NoMethodError
         -------------
         undefined method `vault?' for ChefVault::Item:Class

         Cookbook Trace:
         ---------------
           /tmp/kitchen/cache/cookbooks/chef-vault/libraries/helpers.rb:36:in `chef_vault_item'
           /tmp/kitchen/cache/cookbooks/atlassian-common/libraries/settings.rb:56:in `settings_merge_if_vault_exists'
           /tmp/kitchen/cache/cookbooks/atlassian-common/libraries/settings.rb:36:in `settings'
           /tmp/kitchen/cache/cookbooks/atlassian-common/recipes/java.rb:10:in `from_file'
           /tmp/kitchen/cache/cookbooks/atlassian-common/recipes/default.rb:16:in `from_file'

         Relevant File Content:
         ----------------------
         /tmp/kitchen/cache/cookbooks/chef-vault/libraries/helpers.rb:

          29:    # +node['chef-vault']['databag_fallback']+.
          30:    # @example
          31:    # item = chef_vault_item('secrets', 'bacon')
          32:    # log 'Yeah buddy!' if item['_default']['type']
          33:    # @param [String] bag Name of the data bag to load from.
          34:    # @param [String] id Identifier of the data bag item to load.
          35:    def chef_vault_item(bag, id)
          36>>     if ChefVault::Item.vault?(bag, id)
          37:        ChefVault::Item.load(bag, id)
          38:      elsif node['chef-vault']['databag_fallback']
          39:        Chef::DataBagItem.load(bag, id)
          40:      else
          41:        fail "Trying to load a regular data bag item #{id} from #{bag}, and databag_fallback is disabled"
          42:      end
          43:    end
          44:
          45:    # Helper method which provides an environment wrapper for a data bag.

@nelz9999 for review

:+1:


[Share] Shared prompt appears twice in "Shared with me" if to click on shared link

By user1 create a prompt in Recent -> share the link
By user2 click on the link -> shared prompt appears twice in "Shared with me" -> bug

Note: previously shared prompts are also appears twice in "Shared with me" -> bug

Note2: duplicates disappear from "Shared with me" after browser refresh

Note3: prompts inside folders do not appear twice (ok)

Verified this case on UAT successfully. Reported #979


M1. Change link management for signature

better to change to let it work till the document is not signed from one side and published on  chain.
It's still good to send by email for example, to share the draft before both sign it.

fixed.


Question: want to stop consuming messages upon fatal error

Hello gentlemen,
I'd like to pause message consumption in case of fatal problem with processing.
Is this possible as of now?
In my case processing depends on a third-party service, so in case of a serious issue with it would be nice just to stop processing until further notice.

Yes, you can add a custom ErrorHandler and call stop() on the message listener container based on the exception.

Hello, @stepio !
Let us know what you would like to hear else in this issue.
I find Gary's answer as good one and sufficient to close the ticket.
Thanks

Thanks for the quick reply, @garyrussell !
Hi, @artembilan , I haven't tried yet the proposed approach, but looks good - at least I know for sure that it's possible.
I haven't tried this yet, but at least I know exactly that it's possible.
As of now it's a bit unclear how to set this ErrorHandler. For example as of now I create a consumer factory using a code snippet below:
    public ConcurrentKafkaListenerContainerFactory<String, T> listenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, T> factory = new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(new DefaultKafkaConsumerFactory<>(consumerConfigs()));
        factory.setConcurrency(kafkaConcurrency);
        factory.setRecordFilterStrategy(new SimpleDeDuplicationStrategy<>());
        return factory;
    }

And then I specify the constructed containerFactory for the appropriate KafkaListener annotation.
Certainly I can dig into your code and find the right place, but if you have anything to add, I'll appreciate your comments.

Most probably I should add something like this:
        factory.getContainerProperties().setErrorHandler(new StoppingErrorHandler());

For your @KafkaListener case you should configure proper AbstractKafkaListenerContainerFactory @Bean:
@Bean
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>>
			kafkaAutoStartFalseListenerContainerFactory() {
	ConcurrentKafkaListenerContainerFactory<Integer, String> factory =
			new ConcurrentKafkaListenerContainerFactory<>();
	ContainerProperties props = factory.getContainerProperties();
        ...
	props.setErrorHandler(...);
        ...
	return factory;
}

because ErrorHandler now is a part of ContainerProperties

Oops! You have figured out that already :smile:

Thanks anyway! 👍 Now I know for sure

You are welcome!
Glad to see that issues have so simple solutions out-of-the-box. :wink:

Hi guys,
I am running into a similiar issue and I cannot figure out how to "stop the container" in the custom ErrorHandler. Can you provide an example of what the "new StoppingErrorHandler()" would be
new StoppingErrorHandler() ??
I am using spring-kafka 1.1.1-Release
Any info would be most welcomed.
thanks again for your work on this.

The AbstractMessageListenerContainer implements SmartLifecycle, therefore you can inject  AbstractMessageListenerContainer bean into your custom ErrorHandler and call its stop() whenever you need.

Note that you should execute the stop() on another thread - e.g.
Executors.newSingleThreadExecutor().execute(() -> container.stop());
to avoid delaying the shutdown (stop will wait for the listener thread to return to the container); the default wait time is 10 seconds before stop() gives up.

thank you both very much for your pointers.  sometimes u swirl around the solution and a nudge is necessary :)
@garyrussell I was thinking of decresing the shutdowntimeout but I'll go for yours.
One more thing:  I am not storing the offsets and I am using ACK manual_imm. Thus to my understanding, the incomming message that triggers the pause/custom_handling is "lost" to the current consumer i.e. once stopped, after resuming & starting the container, it will not consume it again, but will process with the next incoming msg. Is my understanding correct??
Is there any way of re-consuming this _last_message with the current (stopped & started) container ???
thanks again for your help

There is a boolean container property ackOnError (default true) which acks failed messages.
There's probably an argument for not defaulting this to true when using manual acks but currently it's true for all ack modes.
Setting it to false should work for you.

We are trying to do similar handling but unable to inject AbstractMessageListenerContainer and stop the container.
@stepio @artembilan @garyrussell
Could you share StoppingErrorHandler() code ?.
I am trying to inject the ConcurrentMessageListenerContainer created in the factory (  createContainerInstance(KafkaListenerEndpoint endpoint) ) into the ErrorHandler but a call to stop method on it is not causing the container threads to stops and we are seeing continuous  errors and finally OutOfMemory exception.
Thanks in advance for the help.

If you are talking about @KafkaListener(id ="foo"), you have to take a look to its JavaDocs:
/**
	 * The unique identifier of the container managing for this endpoint.
	 * <p>If none is specified an auto-generated one is provided.
	 * @return the {@code id} for the container managing for this endpoint.
	 * @see org.springframework.kafka.config.KafkaListenerEndpointRegistry#getListenerContainer(String)
	 */
	String id() default "";

And, therefore, inject KafkaListenerEndpointRegistry to your ErrorHandler.
Select the proper container from there by desired id, cast it to the Lifecycle and call its stop(), like Gary suggest above https://github.com/spring-projects/spring-kafka/issues/126#issuecomment-259138893

You can also stop() the KafkaListenerEndpointRegistry bean and it will stop all the containers.

@artembilan @garyrussell Thanks for your input. Suggested approach works.
Posting the code for reference.
ErrorHandler

@Component
public class StoppingErrorHandler implements ErrorHandler {

  @Autowired
  private KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;

  @Override
  public void handle(Exception thrownException, ConsumerRecord<?, ?> record) {
    kafkaListenerEndpointRegistry.stop();
  }

}

Factory configuration.

@Autowired
  private StoppingErrorHandler stoppingErrorHandler;

  @Bean
  public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>> kafkaListenerContainerFactory() {

    ConcurrentKafkaListenerContainerFactory<Integer, String> factory =
        new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory());
    factory.setConcurrency(1);
    factory.getContainerProperties().setPollTimeout(1000);
    factory.getContainerProperties().setErrorHandler(stoppingErrorHandler);
    return factory;

  }

@shashidarp Just for your info, if you are using batch processing of kafka listeners, this error handler might  not work.
You will need to do the following
factory.setBatchListener(true);
factory.getContainerProperties().setAckOnError(false);
factory.getContainerProperties().setAckMode(AbstractMessageListenerContainer.AckMode.BATCH);
//factory.getContainerProperties().setErrorHandler(stoppingErrorHandler);
factory.getContainerProperties().setGenericErrorHandler(stoppingErrorHandler);

Where the stoppingErrorHandler is implemented this way
@Component
public class StoppingErrorHandler implements BatchErrorHandler {

	private static final Logger LOGGER = LoggerFactory.getLogger(StoppingErrorHandler.class);
	@Autowired
	private KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;

	@Override
	public void handle(Exception thrownException, ConsumerRecords<?, ?> data) {
		LOGGER.error("Exception thrown while processing, going to stop");
		kafkaListenerEndpointRegistry.stop();
	}
}

@garyrussell  Could you please verify this approach?

That's correct, @shashidarp .
There is a validation code:
			Assert.state(ok, "Error handler is not compatible with the message listener, expecting an instance of "
					+ (batch ? "BatchErrorHandler" : "ErrorHandler") + " not " + errHandler.getClass().getName());

So, when you have AckMode.BATCH, you definitely have to use an instance of BatchErrorHandler.
And your configuration is correct as well. We have there this couple of methods:
/**
 * Set the error handler to call when the listener throws an exception.
 * @param errorHandler the error handler.
 * @see #setGenericErrorHandler(GenericErrorHandler)
 */
public void setErrorHandler(ErrorHandler errorHandler) {
	this.errorHandler = errorHandler;
}

/**
 * Set the error handler to call when the listener throws an exception.
 * @param errorHandler the error handler.
 */
public void setGenericErrorHandler(GenericErrorHandler<?> errorHandler) {
	this.errorHandler = errorHandler;
}

And, yeah, there is no specific option for the BatchErrorHandler.
I wonder if we could remove setGenericErrorHandler() altogether and change setErrorHandler() signature to the GenericErrorHandler. That would be legal in the current 2.0 version and will become useful option.

@pkfka ,
Please, stop commenting on old closed issues.
Looks like you have some questions, so consider to represent them in the Community forums.
We don't provide personal support from here.
Only bug reports and feature requests are valid for this project here in GitHub.
Thank you for understanding.


Windows CI?

The MorphicTestingFramework currently fails loading on Windows, leading to CI failures for ReactS, too. Is there any specific reason for testing on multiple OS for ReactS? Otherwise, we may just drop it. If there is a good reason I'll investigate fixing the MorphicTestingFramework :)

There is no super specific reason other than that I wanted to make sure that the code works on Windows and Linux. Given that The MorphicTestingFramework is a test-only dependency, and it only breaks on Trunk, it doesn't matter much I guess. However, generally speaking, I would like to keep testing on multiple OSs. I excluded the combination of Squeak Trunk and Windows for now though, so everybody should get fewer annoying emails :)


YouTube Video Downloader using Python

Hey, I would like to contribute YouTube Video downloader.
Please assign it to me.

Hi @ron2111, I am a new guy to open-sourcing but definitely not new to programming. I am a 2nd-year student and I would like to contribute to your repo. I can write the program in c, c++, or python.


Support SCIM for existing users

Description
WSO2 Identity Server does not support to plug a read-only user store and to retrieve users in that user store over SCIM.
Moreover, even a read-write user store is plugged the underlying immutable attributes cannot be mapped for scim claims like scim id, created date, modified date, etc. Specifically, this becomes useful when an AD is plugged and the immutable attributes of AD like objectGUID, whenCreated, whenModified needs to mapped with above claims.
The main cause for this seems to be that the ReadOnly attribute of a claim is not honoured. Thus, trying to write the value to the user store fails. This seems to be a bug and need to be fixed. Once this is fixed WSO2 Identity Server should be able to support SCIM for existing users if the SCIM meta claims are mapped properly.
Type
Improvement
Priority
High
Component/s
SCIM
Severity
Major
Estimated Complexity
Moderate
Label/s
Type/Improvement, Component/SCIM, Severity/Major, Complexity/Medium, Priority/High
Moved from https://wso2.org/jira/browse/IDENTITY-6932

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


ACM-13005 Turn off search queries on cluster details page

https://issues.redhat.com/browse/ACM-13005

/cc @jeswanke


The performance on changing tab is miserable. Huge lag after tapping the icon.

Fixed by switching to FlashList


DateTime VM implementation DST transitioning temporary solution

https://github.com/dart-lang/sdk/blob/fccdd5fb429002c6ba6a4f421a48f8465c287aec/sdk/lib/_internal/vm/lib/date_patch.dart#L333-L336
What I read online is this:

Not Always One Hour
Today clocks are almost always set one hour back or ahead.
However, on Lord Howe Island, Australia, clocks are set only 30 minutes forward from LHST (UTC+10:30) to LHDT (UTC+11) during DST.
30 and 45 minute time zones
Throughout history, there have been several variations, like half adjustments (30 minutes) or double adjustment (2 hours). Adjustments of 20 and 40 minutes have also been used.

I just stumbled about these lines and wanted to give a heads up - I guess with the growing popularity of dart because of flutter this might be something that should get a look from you guys. Feel free to close if this is not important :).

@lrhn - any sense for the best place to triage this (the impl. of dart:core DateTime)?

Moved to area-library library-core.

I believe that code has been updated to be more general. The only assumptions left are that time zone changes are at most two hours, and there is at most one time zone change in a single day.


npm run wdio - throws Must use import to load ES Module: throws

npm run wdio
of ES modules is not supported.
require() of /Users/testuser/git/component-testing-examples/react-typescript-vite/wdio.conf.ts from /Users/testuser/git/component-testing-examples/react-typescript-vite/node_modules/<EMAIL_ADDRESS>is an ES module file as it is a .ts file whose nearest parent package.json contains "type": "module" which defines all .ts files in that package scope as ES modules.

@howdoilearn thanks for raising an issue, mind providing a minimal reproducible example? It seems like a configuration issue.

I just followed the instructions at  - https://github.com/webdriverio/component-testing-examples
After I cloned the repo, I followed the instructions:-
"Switch to the desired example directory"
cd ./
I switched to - react-typescript-vite (there is no additional read me here so I followed the instructions in the main readme)
"Install dependencies" /*this is fine */
npm install
Run WebdriverIO tests:
This gives the "FileSystemPathService.js is an ES module file as it is a .ts file whose nearest parent package.json contains "type": "module" which defines all .ts files in that package scope as ES modules."
npm run wdio
It is related to Typescript  I am running component testing using wdio of js - for example there is no issue with the set up instructions and execution for - component-testing/svelte-example.js


Retry logic won't trigger; update while clause with <= instead of <

In TodoListController.cs, the retry logic never retries because retryCount gets set to 1, then the while clause evaluates to false.
// In the case of a transient error, retry once after 1 second, then abandon.
// Retrying is optional.  It may be better, for your application, to return an error immediately to the user and have the user initiate the retry.
bool retry = false;
int retryCount = 0;
do
{
    retry = false;
    try { ... } // omitted for brevity
    catch (AdalException ex)
    {
        // omitted
        if (ex.ErrorCode == "temporarily_unavailable")
        {
            // Transient error, OK to retry.
            retry = true;
            retryCount++;
            Thread.Sleep(1000);
        }
    }
} while ((retry == true) && (retryCount < 1));

Thanks for your feedback.
I've fixed the code.


Blog example fails due to ActiveRecord::RecordNotFound on the SchemaInfo

Hi All,
I've been experimenting with Camping and in all of them was stuck at the schema migration portion. I've recreated the Blog example from camping/examples and added in my Gemfile.lock file as well as the blog.db database. Please consult this repo for the environment gem versions.
I hope this is just user error and there is something wrong with the active record versions, I couldn't find a document that outlines a particular version dependency.
I greatly appreciate any clarification to running the example.
The resulting error is:
ActiveRecord::RecordNotFound at /
Couldn't find Blog::Models::SchemaInfo with 'id'=first
Ruby    /Users/xxxx/.rvm/gems/ruby-head/gems/activerecord-4.1.1/lib/active_record/relation/finder_methods.rb: in raise_record_not_found_exception!, line 320

rack requires Ruby version >= 2.2.2


Update 40

Adds an :update method to BlockScore::Base to add more of an ActiveRecord feel to the library.
In addition to the :update method, a protected :initialize_from method was added. This method will reinitialize the attributes of a given object from a provided values hash. The Stripe version of this
method updates the dynamically defined accessors and getters based on the values have changed. I have not done that because I do not think the value of an attribute changing is a positive signal as to whether or not a getter/setter will be called for that attribute in the future.
Also adds VCR cassettes for the update action, and changes the relevant spec from pending to active.
Fixes #40.
Bump version to 4.3.0 (Omit this commit if desired)
Minor version change due to addition of backwards compatible :update method.

I do not have permission to see what the Code Climate issue is, so I need some help there.

I do not have permission to see what the Code Climate issue is, so I need some help there.

same as flagged by hound


docs: Move manpages to separate toplevel

I belatedly noticed that the previous change effectively "corrupted" the docs site by trying to include these generated man pages.
I think we can figure out how to integrate them better, but for now move them to a separate toplevel.
While we're here add one link from the main docs to the manpages.

LGTM

I still haven't quite grokked how to build/view the docs locally, there's a big mess of ruby involved.


last of array is empty

on windows and mac
dart version:2.10.0
using the example code : https://pub.dev/packages/json_path/example
json path is : $['store']['book'][3]['price']
return empty.

@cacard thank you for reporting the issue. Sorry it took me long to notice it. It should be fixed in 0.0.2. Please confirm.


the flutter command doesn't respond to anything

I just updated my flutter, from version 0.9.4 to 0.10.2,
and I get a problem, that is when I do commands via CMD, for example the "flutter doctor" command. I have waited for quite a long time but still did not give any response, whether it was a successful message or an error message

Do you have an internet connection when you run the command?
Are you behind a proxy?

If anyone sees this again, it'd be worth taking a copy of the Flutter SDK before overwriting it - we may be able to figure out what went wrong by examining it.

It is happening to me right now

@thecodingpapa Could you make a copy of your Flutter SDK folder, then re-download it and see if that fixes it? If so, we should then examine the differences between them.

I missed this:

I was running "flutter update-packages --force-upgrade".

What's the reason you were running this? This command is used for updating Flutter's internal dependencies and shouldn't generally be run by users of Flutter (there's no guarantee that Flutter will work with newer versions of packages than it shipped with).

And It took so long and I just close the studio and restarted.

My  guess is that this terminated something in-progress and things in an inconsistent state.
I would delete the SDK and re-download it, and avoid running flutter update-packages --force-upgrade (and I think it's expected that it takes a while).

I am facing this issue whenever i switch from stable channel to master first of all it deletes my dart SDK and automatically start downloading it. Then flutter channel commands never respond, even no flutter command responds. it happened with me twice and always i have to re-download dart SDK.
Here you can see.

Some by running kill commands restart android studio it has started working again, but yes right after switching back to Stable branch it has re-installed dart SDK.

@ch-muhammad-adil It's expected behaviour for the SDK to be downloaded. Each version of Flutter may depend on a different version of the Dart SDK, so when you change channel, Flutter notices the local SDK does not match and fetches the correct one.
It's possible the other issues are related to Android Studio having Flutter processes running while you're switching channels, which might lock files. I would recommend closing any IDE/editor before changing channel to reduce the chance of it causing issues.

I'm having the same described problem w/ Flutter. As you suggested, i deleted the LockFile from the flutter folder and not worked. What can i do?

I had the same issue. Open CMD with Administrator's privileges. All commands started executing.

Open CMD with Administrator's privileges. All commands started executing.

This suggests you may have Flutter in a protected folder that requires elevated privileges. You should move it to somewhere that doesn't (and if required, reset the permissions to inherit from the new location).
There are some notes about this on the Get the Flutter SDK notes:

do not install Flutter in a directory like C:\Program Files\ that requires elevated privileges


Add FAQ entry for pipenv support

Adds a Tips and Tricks section to the readme that contains information on how to run the action against a pipenv based project. This example was pulled out of the similar PR on the pip-audit project (see https://github.com/pypa/pip-audit/pull/364)

CC @woodruffw

LGTM, thanks @ameily!


Support capsule geometry

There's a capsule geometry type on Ignition math now:
https://github.com/ignitionrobotics/ign-math/blob/ign-math6/include/ignition/math/Capsule.hh
It would be nice to add support for that in Ignition Rendering, together with the other currently supported geometry types, like sphere, cylinder, etc.

PR https://github.com/ignitionrobotics/ign-rendering/pull/200


[lit] reexport of html from lit fails because of side effects?

Description
I would expect to have a file with
export { html } from 'lit';

to also work if I execute it in node...
Steps to Reproduce

Open Stackblitz demo https://stackblitz.com/edit/node-uqy5mf?file=index.js
execute node index.js in the stackblitz terminal

Expected Results
A console log
Actual Results
❯ node index.js
Error: document is not defined
...

seems a side effect is doing something and fails

Is this about SSR?

sort of... consider something like this
// page.js
import { html } from 'lit';

export const content = () => html`<p>...</p>`;

export const foo = 'bar';

now I would like to get foo in a node process.
I would assume something like this to work
import { foo } from './page.js';

I would even assume this to work
import { foo, content } from './page.js';

I am fully aware that I won't be able to execute content() as that would "run" the function...
but in fact import { foo } from './page.js'; will already throw with the document error 😅

The core modules definitely require some kind of DOM shim in order to load.

this will run in node with the polyfills for ssr and will work fine
import { content } from './page.js';

ssrRender(content());

this will run in "pure" node and will error...
import { foo } from './page.js';

// encode and write foo to a file

I mean I can always import the polyfills, even though I don't use it in that case... but that feels dirty 😭

The core modules definitely require some kind of DOM shim in order to load.

why? I do not execute any of the functions... 🤔

the plot thickens... I now need to add those two import at the top to any file that at some point imports lit-html
// we load this before the global-dom-shim as otherwise prism thinks it's running in a browser 🙈
// we need to load the global-dom-shim as otherwise import { html } from 'lit'; breaks
import 'rehype-prism-template';
import<EMAIL_ADDRESS>
works - but feels quite wrong

Loading shim is not possible in all of the cases. Lit element is actually limiting itself because of idealistic view that all SSR types of things require to provide a shim. Just a simple typeof window === "undefined" will save so many problems if just this shim stand is let go. Currently, I am trying to use lit react component with Linaria(buildtime CSS extractor) but no luck and loading shim is not working either. So, if I want to use Linaria then I need to drop using Lit which I really like.
I genuinely dont understand why adding a simple check like typeof is so frowned upon in lit even if it is hurting lit element widespread adoption.

We've just published some pre-releases for Lit which includes https://github.com/lit/lit/pull/3156. This change means that Lit should now be importable directly from Node, without crashing, without the need to first load the global DOM shim.
You should be able to upgrade with:
npm i<EMAIL_ADDRESS>
Note this won't automatically enable any kind of actual SSR of Lit components out-of-the-box, since that requires specific integration with the framework/renderer -- but it should allow you to import without crashing Node.
Also note that this does not automatically globally define any DOM APIs like the global DOM shim does, apart from customElements which is shimmed as a no-op. So if you have any calls like window.addEventListener running at module initialization, that will still error.
If you have a chance to check out this pre-release, that would be great, and please comment here to let us know how that works for you!

@aomarks Amazing! I tried to use inside Nextjs this as soon as I saw but it broke on different error for me
<EMAIL_ADDRESS>(6:1220) @ c
ReferenceError: CSSStyleSheet is not defined

This import is coming from customElement()
customElement(lit/decorator.js) -> ClassDescriptor (base.js) -> Reactive Element(reactive-element.js) -> css-tag.js
Thanks for this feature.

sweet - for the simple test case
👉 index.js
console.log(`Hello from Node.js v${process.versions.node}!`);
export { html, css, LitElement, svg, unsafeCSS, nothing } from "lit";
console.log("✅ could export html via node");

node index.js

it works fine 💪
will check for Rocket tomorrow but I assume it will work fine 🤗

@aomarks Amazing! I tried to use inside Nextjs this as soon as I saw but it broke on different error for me
<EMAIL_ADDRESS>(6:1220) @ c
ReferenceError: CSSStyleSheet is not defined

This import is coming from customElement()
customElement(lit/decorator.js) -> ClassDescriptor (base.js) -> Reactive Element(reactive-element.js) -> css-tag.js
Thanks for this feature.

Thanks for the report @bugwheels94. Can you provide a little more context? What does your element definition look like? Are you using Lit SSR?

@aomarks Here is the file that I was trying to import in nextjs
File https://jsfiddle.net/prh5y4Lb/1/
This file is generated using rollup. The error comes in line 43 when using customElements.
No, I am not using Lit SSR. I am just using Nextjs

This file is generated using rollup. The error comes in line 43 when using customElements.

@bugwheels94 Ah I see. Rollup is going to select the default export condition by default. You'll want to set exportConditions: ["node"] in your nodeResolve settings to get the Node-specific build instead: https://github.com/rollup/plugins/tree/master/packages/node-resolve/#exportconditions. We don't include the required shims in the browser build, because we don't want to pay that byte cost in the browser.
Doesn't nextjs do its own bundling? I'm curious why you are bundling with rollup yourself? (I'm also quite new to nextjs)

@aomarks So, we are making a UI library using WC with Lit. We have many things like Linaria(CSS) with custom roolup plugins(for using linaria with constucted css)
Ohh! No issues. I will also make two builds for our ui library in browser and main field.
This is a bit new thing for ui so mentioning will help many like me in future. Thanks!
This change saved me so much trouble of dynamic import and all in Nextjs. Many thanks!

after some cleanup/fight with prism I can now confirm that this version 2.3.0-next.1 allows me to remove all the "nasty" workaround I have in Rocket for it 💪 - nice 🤗
any timeline for when this will be released? 😬

after some cleanup/fight with prism I can now confirm that this version 2.3.0-next.1 allows me to remove all the "nasty" workaround I have in Rocket for it muscle - nice hugs

Awesome! Thanks for checking it out.

any timeline for when this will be released? grimacing

Later today!

any timeline for when this will be released? grimacing

Later today!

Just released!
<EMAIL_ADDRESS>

Updated and used in rocket now 🎉
Sooo nice to get rid of this "workaround" 💪
thxxx 🙇♂
In case anyone is curious what it helped to clean up
https://github.com/modernweb-dev/rocket/pull/398


Versioning updates to documentation website

Closes https://github.com/weaveworks/weave-gitops/issues/3556

What changed?

[x]  Remove this Introduction page.
[x]  Remove this Architecture page.
[x]  Remove this Upgrading page.
[x]  Remove this Troubleshooting page.
[x]  Remove this Examples page.
[x] Old docs versions are sidelined off from the latest doc version, stored in their own section.
[x] Old docs versions retain the same styling as the user docs.
[x] The component tagging at the top right is gone.
[x] Deleted versions from versioned_docs only kept versions from 0.16.0 and above

Why was this change made?
See the discussion on the ticket

How was this change implemented?

How did you validate the change?
Local Testing

Release notes
Versioning updates to the documentation website

Documentation Changes

@Himangini any chance you could break up this PR into smaller ones? For example, 1 PR per version removed? It's quite difficult to review a PR with 1807 files changed.

@Himangini any chance you could break up this PR into smaller ones? For example, 1 PR per version removed? It's quite difficult to review a PR with 1807 files changed.

@yiannistri How about I separate out deleted folders in one/two PRs? There's not much to review for them.

I've had to use the sidebar/filter to hide deleted items and understand what was being deleted.

@Himangini Major thanks for this big step forward


copilot has lost ability to answer basic questions using @vscode and other

copilot seems to have lost its ability to answer basic questions, and keeps giving boilerplate answers that are in no way relevant.

issue occurs in both release version and nightly build.

Please try the latest insiders + latest pre-release as I cannot repo


Changed the $CONST keyword into $CONSTANTS for ubiquity purpose

Many thanks for submitting your Pull Request :heart:!
Please specify parts this PR updates:

[x] Specification
[ ] Schema
[ ] Examples
[ ] Extensions
[ ] Roadmap
[ ] Use Cases
[ ] Community
[ ] TCK
[ ] Other

What this PR does / why we need it:
Changes the $CONST keyword into $CONSTANTS for ubiquity purpose

That's fine, but it will break compatibility with the previous versions.

@ricardozanini indeed, that's why I wanted to squeeze it in before 0.9 which, as far as I'm aware of, not gave any of thoses

@fjtirado Replace all and that's done. Not such a huge compatibility problem imho. Plus, I don't see why you couldn't support both keywords before 1.0 if you don't want to go down that path.
The need for that change imo is ubiquity. The DSL is to be used by non-technical profiles too, and using abbreviations is not recommended, not even in dev, in your code. Ubiquity guarantees intuitive understanding of what you are looking at. Otherwise why not rename WORKFLOW to WF or SECRETS to SEC too then, if shortness is the goal. CONSTANTS is just one letter longer than WORKFLOW. Hell, we could even use CONSTANT if you prefer.

To be fair, const is a very well-known abbreviation for constants.

@ricardozanini I agree totally, not saying it's obscure terminology or whatever, but is less reader friendly imo, and less ubiquitous: it could stand for constraints too, for instance

The other terms we have been using are on point, perfectly explicit. Not that one, imo.

@cdavernas
Implementing $CONST in jsonpath properly was not straighforward. Although that is a proper language construct for a variable in jq, the proper language construct for jsonpath will be @.CONSTANT (the same applies for secret and workflow)
If you really want to have a valid expression without doing hacky string replamencent before evaluating it, I would argue that the spec should just define the magic word (WORKFLOW, CONST/CONSTANT, SECRET) and states that this should be variables of the expression.
And still not able to see whats the benefit of changing that, const is intuitive shortcut for constants, as C++ language demostrated
Also, another complaint I have is why there was not issue to discuss this change before actually opening the PR

Also, another complaint is of procedure: why there was no issue to discuss this breaking compatibility change before actually opening the PR?

@fjtirado We have been doing PR, then discussing on them for a while now, but ok. I don't think we have established any "procedure" as you say, especially if said PR concerns a 1mn "Replace All" kind of work.
I guess, however, that you are right: we should probably start with a GitHub conversation rather than with a PR.

Implementing $CONST in jsonpath properly was not straighforward

What do you mean by JsonPath? Are you using it for expression evaluation?
With  jq - which is the default expression language-, you can pass json arguments like, say: --argjson CONST {CONSTJSON} so in my use case it's an easy fix, really.

And still not able to see what's the benefit of changing that, const is intuitive shortcut for constants, as C++ language demonstrated

C++, C#, whatever, I agree with you, as I said to @ricardozanini, I'm not contradicting the fact that in our world (tech one, it is), const is a perfectly ubiquitous terminology; I'm just saying it might not be the case for a person working in, say business, and wants to use SW to orchestrate high level business tools. Furthermore, I believe that the IMO reasonable idea that a concept such as "constraints" might come into play at some point in time (spec or runtime side), thus becoming confusing.
Please understand I've no other interest than ubiquity here, it implies as well a (minor) change on Synapse, for example. So if you think we should not do that, and that's the consensus, then I'll close the PR 😉
But I'd like to hear what @antmendoza and @tsurdilo think about it before taking it one way or the other (or another 😜)

Oh ok! Yeah the jq vs jsonpath vs ... is yet another discussion, but yeah, of course, any and all implementations are/should be allowed. I really never intended to force the use of the $ prefix or anything like that.

Implementation wise to support constant rather than const just implies changing the value of a constant (no pump intended ;))
but to support both, implies changing the expression evaluator to add a new constant that is resolved the same way than the previous one. This implies at least three classes (the two languages expression implementations plus the one holding the constant) rather than one. Anyway, not a big deal, so if this goes forward, Ill probably do that to avoid changing all examples that currently use const.
But I think we need to define a procedure that, for every breaking compatibility change, albeit minimum one, we should open an issue to discuss first. In this particular case, it is just a question of taste, I prefer the existing word than the proposed one (const/constant refers to the same concept, there is not ambiguity when using the shorted one). And I also  feel that when changing something that is already there, it needs to be a good reason to do that (the existing one is either wrong, ambigous, error promt or any other reason I cannot foresee in this case).

@fjtirado @tsurdilo @ricardozanini Should we proceed with that PR, or just drop it alltogether?

@tsurdilo Not urgent at all, no. But I'd be happier tackling this, if ever, sooner than later, as it is a breaking change

I think we are missing a governance model to handle this kind of situation, like voting in a (sort of) controversial feature.

Agreed, let's then vote on that one on next meeting.
Created #644 to address future situations


Create a list of seen Ds as part of the normal pipeline

The discoverj script is unsuitable for that - the results of the plotalleles script are fine (plus an allele ratio filter). Possibly add a discoverd script. It will not really discover anything, only list the used Ds.

Use Martin’s script (D cores vs cd-hit)

This is actually two different tasks:

Determining which known D genes are expressed in the dataset
Discovering novel D genes

The first task has been solved by updating the count subcommand.
The second task has now also been solved by updating the discoverj script to also work for D discovery (it also needs to be renamed).


powerBI Warning -

this looks awesome - however, i get the following message after running update-DbcPowerBiDatasource- i have power bi installed –
PS C:\WINDOWS\system32> start-DbcPowerBi
WARNING: [14:28:17][Start-DbcPowerBi] .pbix not associated with any program. Please (re)install Power BI

Do you have the latest PowerBI installed from this location?
https://powerbi.microsoft.com/en-us/desktop/

thanks for the fast response - i did , but i took a powerbi class last month, and the instructor suggested installing from microsoft store as that will update the install monthly - i was testing brent ozars sp_blitz powerbi yesterday - worked well -

Are you able to open the PBIX manually from PowerBI Desktop?

yes, just did -that works well - very cool - looks like it cant find the pbix in ps - is there a config setting for this cmd?  thanks

I don't think so.  It seems like a file association issue with PowerBi and pbix files.  Will have to ask some of the others.  @ClaudioESSilva @potatoqualitee @SQLDBAWithABeard

thanks - its awesome - was able to publish this with no issues - very slick tool

Hi @seadba,
Cab you run the following instruction and let us know the output?
Get-ItemProperty "Registry::HKEY_Classes_root\.pbix"

nothing
PS C:\WINDOWS\system32> Get-ItemProperty "Registry::HKEY_Classes_root.pbix"
PS C:\WINDOWS\system32>

Thanks!
That is the way we are validating the path for PowerBI. Maybe the installation from the store keep the value on another key, we will take a look.

no worries - thanks for the fast response - this tool is great -
i tried this as well
PS C:\WINDOWS\system32> Start-DbcPowerBi -Path "C:\Program Files\WindowsPowerShell\Modules\dbachecks\1.0.0\bin\dbachecks
.pbix"
WARNING: [15:04:55][Start-DbcPowerBi] .pbix not associated with any program. Please (re)install Power BI

Yeah, that parameter is for selecting the Power BI file. But, we still validate if the registry key.  Because it is missing, you get the error message.

@claudio Windows apps are installed in a hidden directory under program files.

I verified on my machine that Microsoft Store apps are found under C:\Program Files\WindowsApps . Which this is a hidden system folder that normal users will not have access to by default.
I could find no registry key for the apps I have installed via the store.

Thanks for the confirmation @wsmelton!
Until we find a better way, I think we should use a -Force parameter or a -  SkipPBIValidation to ignore the actual validation.

Please see #311 and confirm if that's a valid way. I'm good with it.

i  just tested on a 2012 box with the regular download - works perfectly

excellent!


passing a list of filenames to dask.array.image.imread

-> in the aim to let the users specify the filenames they wish to work with.
A doctest illustrating such a use would be good, but I didn't find where the '2015-*-*.png' are located.
Feel free to complete it.
Patrick

FWIW it might be worth trying Dask-Image's imread, which has this functionality 😉

@jakirkham , @jrbourbeau
Not sure. the similar issue I have posted in dask-image one month ago about the same feature is still opened
(I don't use dask anymore, feel free to do the corresponding PR).
https://github.com/dask/dask-image/issues/191#issuecomment-785036372

Tried to click the URL, but am getting a 404. Can you please recheck the link?

I fixed the link above.


Wrong checksum in install instructions

I followed instructions for installing binaries from README:
export IMG_SHA256="c7db856ef0472df30115c3f76180a1c001d22406e22dbae60a648cc0afee15d3"
curl -fSL "https://github.com/genuinetools/img/releases/download/v0.5.2/img-linux-amd64" -o "/usr/local/bin/img" \
> && echo "${IMG_SHA256}  /usr/local/bin/img" | sha256sum -c - \
> && chmod a+x "/usr/local/bin/img"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   606    0   606    0     0   2818      0 --:--:-- --:--:-- --:--:--  2818
100 24.4M  100 24.4M    0     0  8733k      0  0:00:02  0:00:02 --:--:-- 10.9M
/usr/local/bin/img: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match

The hash I have is e6dd104cc9d76fe33e0638036f8f1f58d7ab4388162dda01a21cf74d31dee0ab. Multiple downloads make no difference. The downloaded binary works great (because img is an awesome project).
Maybe the hash needs updating?

Ah, I see this is already addressed by https://github.com/genuinetools/img/pull/188

fixed thanks!


Update layout of resources page based on updated wireframe

update layout
update venn diagram

see https://docs.google.com/document/d/15aX8-RcW8aAk6e6M5_N3JZ-b3D0b66R-sx1GWQLOFDY/edit and images in https://drive.google.com/drive/u/1/folders/14OfzztK2V-rqBhheoJzba4j0cxqzZwoV

made it as close as possible to the design


Version 5.0.0

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

See conda-forge/dash-feedstock#96.

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.
Here's what I've got...
For recipe:

requirements: run: dash == 2.0.0 should not contain a space between relational operator and the version, i.e. dash ==2.0.0

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Feature: Add a feature to the portals that issues a command when you go through a portal

Would it be possible to add a feature to the portals that issues a command when you go through the portal like /ungod for example?

I've put it on hold, as I think it'd be neat, but I have to think how to do it. This weekend a very large architecture change will be merged from dev, so maybe a week after that i'll look into it.

Alright sounds good, I was just wondering if anyone had even seen it or thought about it, thanks for getting back to me, I appreciate it.
I hope once you do look into this you'll be able to find a way to work it out, having this would help lots of people out, best of luck.

Ok, so I've been cleaning up the issues, and I know it's a pain, but could you file this under Portals? (This is the core issues page).
That's the only reason i'm closing it, just organization. Please reopen if you're still interested on the MV-Portals Issue page

No problem, already moved (:


[Ubuntu 22.04.1 LTS] No bspwm option when logging in.

Went through all instructions in readme file, after installation there are [xinit, Ubuntu, Ubuntu Wayland/Xorg, i3]. No bspwm option.

Did you use startx before? Check in /usr/share/xsessions/ if bspwm has an entry

@justKleb

the authentication keyring isn't unlocked

That should be fixed by #496.

and firefox + kitty terminal are missing

Could you perhaps elaborate a bit more on that?


split tables clients

Pull Request Checklist
Please leverage this checklist as a reminder to address commonly occurring feedback when submitting a pull request to make sure your PR can be reviewed quickly:
See the detailed list in the contributing guide.

[ ] C++ Guidelines
[ ] Doxygen docs
[ ] Unit tests
[ ] No unwanted commits/changes
[ ] Descriptive title/description

[ ] PR is single purpose
[ ] Related issue listed

[ ] Comments in source
[ ] No typos
[ ] Update changelog
[ ] Not work-in-progress
[ ] External references or docs updated
[ ] Self review of PR done
[ ] Any breaking changes?

Here's an alternate solution in PR (https://github.com/Azure/azure-sdk-for-cpp/pull/6135) which is almost identical, but

Makes the file name singular (table_client.hpp instead of tables_client.hpp)
Doesn't have a /internal/ folder with a standalone header file for constants that are in the _detail namespace. I don't believe leaf-node SDKs should have "internal" headers, nor _internal namespace. See  https://github.com/Azure/azure-sdk-for-cpp/issues/6120#issuecomment-2430407728
Kept the options with the client, rather than moving them to models.hpp.

Which one should we consider closing?


Easily get accession number to simplify XBRL and file extraction process

Currently we are scraping multiple pages to get to the final TXT file. We are making lots of unnecessary calls. The format of the URLs for both the TXT files and the XBRL files are of the same format. This is what a search looks like when searching for a certain filing type:

If you look at any row, you will notice that in the description there is an Accession Number. The URL for each filing is structured around the Accession Number like this:
https://www.sec.gov/Archives/edgar/data/.txt
OR (for zip file with all documents including relevant XML files containing XBRL)
https://www.sec.gov/Archives/edgar/data/-xbrl.zip
For example, here is a link for Apple's 2020 10-Q:
https://www.sec.gov/Archives/edgar/data/320193/000032019320000010/0000320193-20-000010.txt
You can see the URL pattern in the screenshot below.

I propose that we restructure our Network Client to only look for the Accession Numbers (by either using Regex or Beautiful Soup). This will eliminate a lot of unnecessary requests and will allow us to also add support for XBRL.
Adding support for XBRL will make parsing the 10-K and 10-Q data much easier.

We are actually using the xml format which gives all of the filing urls at once. This should not be an issue.


Update download script for February 2020 release

This PR updates the download script to pull the February 2020 G-NAF release. Tested locally successfully.

Hey John!
Thanks for submitting a PR :) Looks good!

My pleasure Matt.


Feature request: Show article number in what-if

In the title bar on what-if, there is currently no indication of the article number. But the comic view does.
Having it makes it easier to look up the article later.

I'm on it!

Most of the time it wouldn't fit into the title bar, so I added the numbers to the overview instead, hope that works for you too.

That does the job! I just wanted the reference whenever I stashed something to read later.
It's easier for me to type the url and number than google the title.


feat(electric): Add server-side validation of null values incoming from Satellite clients

Admitedly, we're relying on client-provided SatRelation messages to decide whether any given column is nullable or not. So it's not a good protection against malicious clients.
I wish we didn't have to pass nullability info via SatRelation message from the server to the client and back. But apparently that's what we have to do since we're not storing the history of schema changes for every individual client.
This has to be tagged as a feature because it changes the SatRelationColumn message in the protobuf definition.
This PR is based on the changes made in https://github.com/electric-sql/electric/pull/320. Related to VAX-813.

ignore my comment (😊) - i looked at the rest of that protobuf definitition and it seems my memory was at fault, not your code....


fix expected repoclone, but received: repoCloneLFS error

I am trying to find a way to test this, but can't deploy to staging at the moment due to #3533. @js-ts do you have a ready environment with gitlfs installed to test this? If not, don't worry about it and don't spend time on it as I believe we should get staging to work by end of the day

@wdbaruni I don't have any environments, but I think you can
Install GitLFS with a these commands in your existing environment
https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage
https://github.com/git-lfs/git-lfs/blob/main/INSTALLING.md#2-installing-packages
sudo apt-get install git-lfs
git lfs install

Tested in staging and working as expected using:
export BACALHAU_API_HOST=bootstrap.staging.bacalhau.org
bacalhau docker run \
--gpu 1 --memory 10Gb \
-w /inputs -p ipfs \
-i gitlfs://huggingface.co/databricks/dolly-v2-3b.git \
-i https://gist.githubusercontent.com/js-ts/d35e2caa98b1c9a8f176b0b877e0c892/raw/3f020a6e789ceef0274c28fc522ebf91059a09a9/inference.py \
jsacex/dolly_inference:latest \
 -- python inference.py --prompt "Where is Earth located ?" --model_version "./databricks/dolly-v2-3b"


create some performance metric measurements, notably CPU usage

Partly to assist with investigating the jagged leg movements, we need some measurements of how much CPU our various software tasks are taking, and what our total CPU usage is under various circumstances

Some thoughts:

do timestamp capture around task execution in loop(), and report on average and max
track how close to the scheduled millis() time the tasks are dispatched, and report on average and max
serialize anything that's async for priority control purposes (MQTT, network stuff?)
make the reporting interval a parameter
make the reporting available via MQTT
measure overhead as well for calculation/report, and MQTT overhead

There are a couple of reasons for splitting up long tasks dispatched from loop():

gets routines dispatched via millis() times started with less delay and more consistency
gets higher priority millis() tasks started before lower priority ones
generates finer granularity CPU usage information

Should we be measuring the CPU that network stuff runs on, or moving some of our tasks to it?

Initial CPU usage data is surprising. On an idle system, the percentage CPU used by the routines called in loop() is::
M) setup{7}> End of setup ############################################
  Flow: 0.00  Web: 0.00  Oled: 0.00  Mqtt: 0.00  1Sec: 0.00
  Flow: 0.00  Web: 98.44  Oled: 0.10  Mqtt: 0.50  1Sec: 0.05
  Flow: 0.00  Web: 98.46  Oled: 0.10  Mqtt: 0.50  1Sec: 0.04
  Flow: 0.00  Web: 98.45  Oled: 0.10  Mqtt: 0.50  1Sec: 0.04
  Flow: 0.00  Web: 98.45  Oled: 0.10  Mqtt: 0.50  1Sec: 0.04
  Flow: 0.00  Web: 98.46  Oled: 0.10  Mqtt: 0.50  1Sec: 0.04
  Flow: 0.00  Web: 98.45  Oled: 0.10  Mqtt: 0.50  1Sec: 0.04
  Flow: 0.00  Web: 98.45  Oled: 0.10  Mqtt: 0.50  1Sec: 0.04
  Flow: 0.00  Web: 98.49  Oled: 0.10  Mqtt: 0.47  1Sec: 0.04
  Flow: 0.00  Web: 98.55  Oled: 0.10  Mqtt: 0.40  1Sec: 0.05
  Flow: 0.00  Web: 98.56  Oled: 0.10  Mqtt: 0.40  1Sec: 0.04
  Flow: 0.00  Web: 98.56  Oled: 0.10  Mqtt: 0.40  1Sec: 0.04

Some thoughts:

the monitorWebServer() routine seems to be taking up about 98.5 percent of the available CPU.
loop() is a spin loop, executing at top speed so it might amplify some things
the numbers are derived from before and after timestamps using micros(). Once a second, the accumulated time is divided by 1,000,000, the number of microseconds in a second, converted to a percent, and displayed
because there's timing sensitive stuff dispatched from loop, we want to minimize the overhead, and the delays. One way to do this efficiently is to use a flag, of a millis() check to detect when work it to be done, and only do (and time) the work when it needs to be done.
we have some routine calls in loop() that combine checking for work to be done as well as executing them when needed. n this case the CPU measurement includes both the "check for work to be done" part plus actually doing the work
the monitorWebServer() function is function that makes calls to a localWebService object, which I think makes calls to a standard WebServer object. This may result in the check for work to be done being more complex and time consuming.
from a functionality point of view, we're doing more work than is practically needed. For example, checking to see if there's a new Broker IP doesn't need to be done at the loop() spin rate. Once a second would be more than sufficient, or react more quickly via a flag that says this needs to be handled.

Possible actions to address CPU utilization issues:

convert the first level object aaWebservice to functional code, if that's practical, so that simple status flags can be used to control task dispatching in loop()
modify the routine calls in loop() so activity is triggered either by a request flag, or by a millis() timer interval
modify loop() structure so that at most one routine runs per loop() cycle. This reduces delays for millis() based disppatching, and establishes task priorities, according to which appears earlier in the loop() checks for work to be done
break any long running tasks into smaller pieces that run sequentially to enable:

decreased delay in millis() based dispatching
better granularity in CPU usage measurements
better prioritization of tasks contending for CPU time

the do_flow routine is a good candidate for this fragmentation.

I changed the way monitorWebServer is called so it's based on a timer, and only done every 200 msec. The CPU stats now look like this:
M) setup{7}> End of setup #############################################################################
  Flow: 0.00  Web: 0.00  Oled: 0.00  Mqtt: 0.00  1Sec: 0.00
  Flow: 0.00  Web: 0.56  Oled: 5.28  Mqtt: 21.25  1Sec: 0.05
  Flow: 0.00  Web: 0.56  Oled: 5.27  Mqtt: 21.24  1Sec: 0.04
  Flow: 0.00  Web: 0.56  Oled: 5.29  Mqtt: 21.26  1Sec: 0.04
  Flow: 0.00  Web: 0.56  Oled: 5.28  Mqtt: 21.26  1Sec: 0.04
  Flow: 0.00  Web: 0.56  Oled: 5.26  Mqtt: 21.24  1Sec: 0.04
  Flow: 0.00  Web: 0.56  Oled: 5.26  Mqtt: 21.25  1Sec: 0.04
  Flow: 0.00  Web: 0.56  Oled: 5.29  Mqtt: 21.23  1Sec: 0.04
  Flow: 0.00  Web: 0.56  Oled: 5.27  Mqtt: 21.23  1Sec: 0.04

As expected, the monitorWebServer routine is now only occupying a small portion of the CPU. The other routines that are checked at the spin lock rate have gathered more of the CPU that's now available, but the timer driver oneSec routine stayed stable. So I should make the same fix to the other 2 routines dispatch mechanisms, and make the CPU usage output controllable via the trace mechanism. Then check to see if leg movements are still jagged.

Did timer dispatches for checkOledButtons() and checkMqtt()
added a total CPU number at start of line (an approximation, but useful)
made usage output controllable by trace 14 which is in oneSec() routine in main.cpp after loop()
tested on robot - leg movement still seems jagged to me

This got us down to expected CPU consumption on idle robot as these percentage usage numbers show:
M) setup{7}> End of setup ####################################################
M) oneSec{14}>   CPU Total= 0.00   Flow: 0.00  Web: 0.00  Oled: 0.00  Mqtt: 0.00  1Sec: 0.00
M) oneSec{14}>   CPU Total= 0.64   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.09
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.63   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.63   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.09

Tested by doing a move leg to angles command: MLA, 1, 0, 0, 0
which showed this usage:
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
  Hip angle, PWM= 0.00 295
 Knee angle, PWM= 0.00 290
Ankle angle, PWM= 0.00 300
M) oneSec{14}>   CPU Total= 0.80   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.18  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.64   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.10
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08

there's a bit of MQTT, but no flow, which is strange. We'll see what pushups does to CPU:
M) setup{7}> End of setup #####################################################
M) oneSec{14}>   CPU Total= 0.00   Flow: 0.00  Web: 0.00  Oled: 0.00  Mqtt: 0.00  1Sec: 0.00
M) oneSec{14}>   CPU Total= 0.74   Flow: 0.00  Web: 0.64  Oled: 0.00  Mqtt: 0.00  1Sec: 0.09
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.85   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.23  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.97   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.34  1Sec: 0.09
M) oneSec{14}>   CPU Total= 0.98   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.35  1Sec: 0.09
M) oneSec{14}>   CPU Total= 0.96   Flow: 0.00  Web: 0.54  Oled: 0.00  Mqtt: 0.33  1Sec: 0.09
L) do_flow{13}>  start of flow row # 0
L) do_flow{13}> start of flow row #1
M) oneSec{14}>   CPU Total= 37.23   Flow: 36.63  Web: 0.37  Oled: 0.00  Mqtt: 0.14  1Sec: 0.09
L) do_flow{13}>  start of flow row #  2
L) do_flow{13}>  start of flow row #  3
M) oneSec{14}>   CPU Total= 14.58   Flow: 13.93  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.10
L) do_flow{13}>  start of flow row #  4
L) do_flow{13}>  start of flow row #  5
M) oneSec{14}>   CPU Total= 15.31   Flow: 14.67  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  6
M) oneSec{14}>   CPU Total= 14.97   Flow: 14.34  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  7
M) oneSec{14}>   CPU Total= 17.49   Flow: 16.86  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  8
M) oneSec{14}>   CPU Total= 17.52   Flow: 16.89  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  9
L) do_flow{13}>  start of flow row #  10
M) oneSec{14}>   CPU Total= 17.54   Flow: 16.91  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  11
M) oneSec{14}>   CPU Total= 17.19   Flow: 16.56  Web: 0.53  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  12
M) oneSec{14}>   CPU Total= 15.28   Flow: 14.65  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  13
L) do_flow{13}>  start of flow row #  14
M) oneSec{14}>   CPU Total= 15.00   Flow: 14.36  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  15
L) do_flow{13}>  start of flow row #  16
M) oneSec{14}>   CPU Total= 14.98   Flow: 14.35  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  17
L) do_flow{13}>  start of flow row #  18
M) oneSec{14}>   CPU Total= 14.84   Flow: 14.21  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  start of flow row #  19
M) oneSec{14}>   CPU Total= 15.90   Flow: 15.27  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
L) do_flow{13}>  end of multi row flow processing
M) oneSec{14}>   CPU Total= 6.10   Flow: 5.47  Web: 0.54  Oled: 0.00  Mqtt: 0.01  1Sec: 0.09
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08
M) oneSec{14}>   CPU Total= 0.62   Flow: 0.00  Web: 0.53  Oled: 0.00  Mqtt: 0.00  1Sec: 0.08

Leg performance still seems jagged to me. Will have to analyse above numbers. May also add to CPU metrics to monitor how much timer driver task dispatches are delayed. I have a design for that on paper and it shouldn't be too hard to ad if we need to.

I'm about to upload the branch that implements CPU performance measurements, but I wanted to note some related things that snuck into the branch:

new method of dispatching tasks from loop() based on predefined macros. One method for tasks that are dispatched by timers, and another for tasks that are dispatched when some logical condition becomes true. A macro is also used to define all the associated variables used to track both CPU usage and dispatching latency for each task
along the way, I changed the way some routines were called, introducing timers rather than have them execute complicated checks on every loop() cycle. This appears to free up some CPU, but that's likely an illusion.
added a way to temporarily disable CPU performance number display, in case you want to examine the numbers without them moving, or to keep them out of the way of other messages going to the console. The disabling is controlled by GPio27, physical pin 23, acting as a capacitive touch sensor.
when the output is disabled, I turn on the onboard red LED on the Huzzah32, and I added software support for this LED in the process. It's debounced by a simple 4-state machine in the oneSec routine in main.cpp
as an exercise, I annotated a copy of the console log that was generated when an earlier version of the CPU measurement software was running, and the MQTT pushup script was generating messages. I'll put it in the performance section of Github

Related work remaining to be done includes:

documenting how to interpret the CPU usage display
documenting how to add either a timer or condition triggered task to loop()
breaking up the flow processing into smaller pieces for measurement purposes, and to get rid of a 340 millis deadspot
figuring out how to structure loop() so that at most one task runs per cycle, so high priority tasks have less latency
reducing the amount of asynchronous processing ( possibly MQTT command processing) that happens outside of the controls in loop().

I'm going to create an issue where I'll record topics I want to discuss / propose / explain so I don't forget about them.


Index silently continues to index after removal from schema if object already in DB

EdgeDB Version: 3.2+bb17a05
EdgeDB CLI Version: 3.4.0+160d07d
OS Version:

It took a while to figure out how to reproduce this, turns out it has to follow this exact order in which an object is inserted before an index is added. If an index is added and then an object is inserted, the index removal will work properly and Cost reverts to the pre-indexed number.
Steps to Reproduce:

Create schema with two types for comparison:

module default {
    type IndexedAndInserted {
        name: str;
    }

    type IndexedNotInserted {
        name: str;
    }
}

Use analyze to check Cost when ordering by a property:

edgedb> analyze select IndexedAndInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedAndInserted order by .name;

─────────────── Coarse-grained Query Plan ───────────────
       │ Time  Cost Loops Rows Width │ Relations
➊ root │  0.1 77.21   1.0  0.0    64 │ IndexedAndInserted
edgedb> analyze select IndexedNotInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedNotInserted order by .name;

─────────────── Coarse-grained Query Plan ───────────────
       │ Time  Cost Loops Rows Width │ Relations
➊ root │  0.0 77.21   1.0  0.0    64 │ IndexedNotInserted

Insert an object for one type, not for the other

insert IndexedAndInserted { name := "Some name" };

Check Cost again to be sure:

edgedb> analyze select IndexedAndInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedAndInserted order by .name;

─────────────── Coarse-grained Query Plan ───────────────
       │ Time  Cost Loops Rows Width │ Relations
➊ root │  0.0 77.21   1.0  1.0    64 │ IndexedAndInserted
edgedb> analyze select IndexedNotInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedNotInserted order by .name;

─────────────── Coarse-grained Query Plan ───────────────
       │ Time  Cost Loops Rows Width │ Relations
➊ root │  0.0 77.21   1.0  0.0    64 │ IndexedNotInserted

Add an index to property of both types:

module default {
    type IndexedAndInserted {
        name: str;
        index on (.name);
    }

    type IndexedNotInserted {
        name: str;
        index on (.name);
    }
}

Check Cost again. Both are improved but the type with an object in the database improves substantially compared to the other:

edgedb> analyze select IndexedAndInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedAndInserted order by .name;

────────────── Coarse-grained Query Plan ──────────────
       │ Time Cost Loops Rows Width │ Relations
➊ root │  0.0 1.02   1.0  1.0    64 │ IndexedAndInserted
edgedb> analyze select IndexedNotInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedNotInserted order by .name;

────────────── Coarse-grained Query Plan ──────────────
       │ Time Cost Loops Rows Width │ Relations
➊ root │  0.0 60.2   1.0  0.0    64 │ IndexedNotInserted

Now remove the index from both:

module default {
    type IndexedAndInserted {
        name: str;
    }

    type IndexedNotInserted {
        name: str;
    }
}

Check Cost again. The type without any object in the database reverts to previous non-indexed number, but the type with an object in the database is still showing indexed performance:

edgedb> analyze select IndexedAndInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedAndInserted order by .name;

────────────── Coarse-grained Query Plan ──────────────
       │ Time Cost Loops Rows Width │ Relations
➊ root │  0.0 1.02   1.0  1.0    64 │ IndexedAndInserted
edgedb> analyze select IndexedNotInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedNotInserted order by .name;

─────────────── Coarse-grained Query Plan ───────────────
       │ Time  Cost Loops Rows Width │ Relations
➊ root │  0.0 77.21   1.0  0.0    64 │ IndexedNotInserted

There's no bug here, just the vagaries of postgres's planner.
If you run \expand, it's clear that no index is being used:
edgedb> analyze select IndexedAndInserted order by .name;
────────────────────── Query ──────────────────────
analyze select ➊  IndexedAndInserted order by .name;

────────────── Coarse-grained Query Plan ──────────────
       │ Time Cost Loops Rows Width │ Relations
➊ root │  0.0 1.02   1.0  1.0    64 │ IndexedAndInserted
edgedb> \expand
──────────────────────── Fine-grained Query Plan ────────────────────────
 │ Time Cost Loops Rows Width │ Plan Info
 │  0.0 1.02   1.0  1.0    64 │ ➊ Sort sort_method=quicksort,
 │                            │ sort_space_used=25kB,
 │                            │ sort_space_type=Memory
 │  0.0 1.01   1.0  1.0    64 │ SeqScan relation_name=IndexedAndInserted

SeqScan, not an IndexScan.
When the query is actually run, it looks like the number of rows reported is the true number, and not the estimated number. If we run with execute := false, we can see what happened:
edgedb> analyze (execute := false) select IndexedNotInserted order by .name;
──────────────────────────────── Query ────────────────────────────────
analyze (execute := false) select ➊  IndexedNotInserted order by .name;

─────────── Coarse-grained Query Plan ───────────
       │ Cost Plan Rows Width │ Relations
➊ root │ 77.2      1070    64 │ IndexedNotInserted
edgedb> analyze (execute := false) select IndexedAndInserted order by .name;
──────────────────────────────── Query ────────────────────────────────
analyze (execute := false) select ➊  IndexedAndInserted order by .name;

─────────── Coarse-grained Query Plan ───────────
       │ Cost Plan Rows Width │ Relations
➊ root │  1.0         1    64 │ IndexedAndInserted

IndexedNotInserted estimated 1070 rows, and IndexedAndInserted estimated just 1, so that's why there was a cost difference. (Cost is the estimated number of "pages" that need to be read, so it's 1.0 for 1 row, even though the row doesn't take a whole page)
My guess here is that adding the indexes got postgres to update statistics on the tables, and also that if the stats say a table has 0 rows, postgres will assume it actually has some small-but-not-tiny number of rows (probably the stats are out of date, most tables have data), but if the stats say a table has 1 row it believes it
I also ran \di "edgedbpub".* from \psql before and after deleting the indexes, and can verify that the indexes were in fact deleted.


Fixes #1878 | Refactoring ExecutePlan#doExecutePlanLegacy()

What type of PR is this?

What does this PR do / why is it needed ?
Refactor for nits mentioned as part of Issue #1878.

Which issue(s) this PR fixes:

Fixes #1878
Other notes for reviewers:
Does this PR introduce a user-facing change? No

Any update/comments on this ?

This PR is stale because it has been open for 30 days with no activity. Please remove stale label or add any comment to keep this open. Otherwise this will be closed in 5 days.

Hi @academy-codex , thanks for your PR!
Unfortunately I don't have enough know-how on Legend Engine to review your PR. @akphi @epsstan @ivan-kyosev-gs - could you please review this PR ? Thanks!

I'll close this given that no response from project members.
Are external open source contributions being accepted at the moment for the project ?


Updated Markdown-it render options

Fixes #1429
Changelog Entry
Fix #1429. Changed Markdown-it options to render newline characters correctly, by @tdurnford in PR #1988
Description
When Markdown-it's breaks option is set to true, the renderer interprets all newline characters (\n) as line break elements (<br />), which is not the desired behavior for Web Chat. Setting the option to false changes the renderer's interpretation of a newline character to a new line within the paragraph tag unless it is preceded by two space characters. In that case, the renderer converts the newline character to a line break.

Specific Changes

Changed the Markdown-it breaks option from true to false in renderMarkDown

[x] Testing Added

lol just FYI you referenced the wrong PR ;)