Trouble importing FaceONNX detector model: Only tensor indices is valid

Describe the bug
Error during importing the detector model:
ERROR burn_import::logger: PANIC => panicked at /home/simon/Data1/GIT/Rust/burn/crates/burn-import/src/onnx/dim_inference.rs:820:14:
Only tensor indices is valid

To Reproduce

Download detector model from https://drive.google.com/file/d/1tB7Y5l5Jf2270IisgSZ3rbCQs0-pkNIQ/view?usp=drive_link (also attached down below)
Move the file to src/model/detector.onnx
Add this to build.rs

	ModelGen::new()
		.input("src/model/detector.onnx")
		.out_dir("model/")
		.run_from_script();

Build and see error

What I've figured out so far

There is something going on when updating the gather outputs: https://github.com/tracel-ai/burn/blob/main/crates/burn-import/src/onnx/dim_inference.rs#L812

Attachment:
face_detector_640.onnx.zip

As you correctly pointed out the issue is with the gather node.
We check for the inputs to make sure they are in accordance with the spec, but when checking for the indices we expect a tensor but your model uses a scalar (rank 0 tensor) - which is still valid.
I'll open a PR to fix that.

Also I just realized your model was generated with an older opset (v9) and we might not support all of the ops there. Right now the ops we have should support the latest ops at opset 13. Some also have support for older opset but not all (e.g., Slice which I see in your model is opset v1 and we don't parse the attributes).
The ONNX spec is large and our bandwidth for that is a bit limited.

We should probably fix this to support older opset.
As a work around, can you convert to a new opset? There are some tools to upgrade the onnx file.

I tried this and it did not help... That said from the comments my understanding is that the Only tensor indices is valid issue isn't due to an old opset, is it?

This dirty fix seems to be working:
diff --git a/crates/burn-import/src/onnx/dim_inference.rs b/crates/burn-import/src/onnx/dim_inference.rs
index 94d66d22..c404829c 100644
--- a/crates/burn-import/src/onnx/dim_inference.rs
+++ b/crates/burn-import/src/onnx/dim_inference.rs
@@ -808,11 +808,16 @@ fn gather_update_outputs(node: &mut Node) {
     };

     let indices_tensor = match &node.inputs[1].ty {
-        ArgType::Tensor(tensor) => tensor,
-        _ => panic!("Only tensor indices is valid"),
+        ArgType::Tensor(tensor) => tensor.clone(),
+        ArgType::Scalar(scalar) => TensorType {
+            elem_type: scalar.clone(),
+            dim: 0,
+            shape: None,
+        },
+        _ => panic!("Only tensor or scalar indices are valid"),
     };

-    if indices_tensor.dim != 1 {
+    if indices_tensor.dim > 1 {
         panic!("Gather: indices tensor rank above 1 not supported")
     }

but then I run into this:
ERROR burn_import::logger: PANIC => panicked at /home/simon/GIT/Rust/burn/crates/burn-import/src/onnx/op_configuration.rs:302:9:
Flatten: input tensor must have at least 2 dimensions (got 0)

(btw, is there a Discord server/Matrix space/whatever which would be more suitable for debugging this?)

As far as I can tell, according to the docs, the input tensor really must hast at least 2 dimensions which makes me ask how is it I am getting a 0D one...
https://onnx.ai/onnx/operators/onnx__Flatten.html

(btw, is there a Discord server/Matrix space/whatever which would be more suitable for debugging this?)

Absolutely! Come join us here.

I tried this and it did not help... That said from the comments my understanding is that the Only tensor indices is valid issue isn't due to an old opset, is it?

Yeah it's not just because of the opset. Right now the first errors you're seeing are due to the rank 0 tensor. When I checked your model last time the gather node used a single value to gather and then it was unsqueezed to a 1-dim tensor.
We currently don't support 0-dim tensors, so even if we try to work around it by returning a 1d tensor I don't think the rest of the codegen will match. We would have to add support for that.
Finally, you do have some ops that use an older opset which may not be supported (as pointed out in my last comment, some attributes which don't exist in new opsets are not parsed).

This part needs to be fixed:
We need to support Gather, Unsqueeze, and Concat operations on Shape

Used onnx-simplifier (https://github.com/daquexian/onnx-simplifier) which removes lots of these crazy gather reshape operation, which were due to lacking capabilities of the older ONNX OpSet versions.
Here is updated version:
face_detector_opt.onnx.zip
Output of the tool:
[burn]% onnxsim ~/Downloads/face_detector_640.onnx ./face_detector_opt.onnx
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃                    ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add                │ 3              │ 3                │
│ BatchNormalization │ 35             │ 0                │
│ Concat             │ 13             │ 5                │
│ Constant           │ 240            │ 111              │
│ Conv               │ 52             │ 52               │
│ Div                │ 2              │ 1                │
│ Exp                │ 1              │ 1                │
│ Gather             │ 8              │ 0                │
│ Mul                │ 4              │ 4                │
│ Relu               │ 37             │ 37               │
│ Reshape            │ 8              │ 8                │
│ Shape              │ 8              │ 0                │
│ Slice              │ 6              │ 4                │
│ Softmax            │ 1              │ 1                │
│ Sub                │ 1              │ 1                │
│ Transpose          │ 8              │ 8                │
│ Unsqueeze          │ 24             │ 0                │
│ Model Size         │ 1.5MiB         │ 1.3MiB           │
└────────────────────┴────────────────┴──────────────────┘

Now, there is still a slight issue with Slice operation which does not support attributes. I am working on this fix.

Used onnx-simplifier (https://github.com/daquexian/onnx-simplifier) which removes lots of these crazy gather reshape operation, which were due to lacking capabilities of the older ONNX OpSet versions.
Here is updated version:
face_detector_opt.onnx.zip
Output of the tool:
[burn]% onnxsim ~/Downloads/face_detector_640.onnx ./face_detector_opt.onnx
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃                    ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add                │ 3              │ 3                │
│ BatchNormalization │ 35             │ 0                │
│ Concat             │ 13             │ 5                │
│ Constant           │ 240            │ 111              │
│ Conv               │ 52             │ 52               │
│ Div                │ 2              │ 1                │
│ Exp                │ 1              │ 1                │
│ Gather             │ 8              │ 0                │
│ Mul                │ 4              │ 4                │
│ Relu               │ 37             │ 37               │
│ Reshape            │ 8              │ 8                │
│ Shape              │ 8              │ 0                │
│ Slice              │ 6              │ 4                │
│ Softmax            │ 1              │ 1                │
│ Sub                │ 1              │ 1                │
│ Transpose          │ 8              │ 8                │
│ Unsqueeze          │ 24             │ 0                │
│ Model Size         │ 1.5MiB         │ 1.3MiB           │
└────────────────────┴────────────────┴──────────────────┘

Now, there is still a slight issue with Slice operation which does not support attributes. I am working on this fix.

Ha, that's what I suggested on discord. Glad it cleared up the redundant stuff 🙂

Submitted a PR for the slice issue: https://github.com/tracel-ai/burn/pull/1989
But I stumbled another bug: https://github.com/tracel-ai/burn/issues/1983 which I am working to fix.
Overall, I think I am very close to fixing it. The good news is that all ops seems to be working.

I got the face-onnx building, however, there is a run-time error that's probably introduced recently when we were refactoring tensors. @laggui did some work in the Tensor random initialization. Maybe he knows what's going on.
I am attaching all my artifacts here in my repository, which will be hosted temporarily:
https://github.com/antimora/face-onnx
Please continue debugging. I think the bug is in the Burn's core.

I got the face-onnx building, however, there is a run-time error that's probably introduced recently when we were refactoring tensors. @laggui did some work in the Tensor random initialization. Maybe he knows what's going on.
I am attaching all my artifacts here in my repository, which will be hosted temporarily:
https://github.com/antimora/face-onnx
Please continue debugging. I think the bug is in the Burn's core.

I have filed the model initialization bugs here: https://github.com/tracel-ai/burn/issues/2043

@SimonBrandner has confirmed the transformed ONNX file works now.
This issue can be closed after #2008 is merged.

Just a note that while the original issue was circumvented with the help of onnx-simplifier, it does not always appear to remove the problematic gather node with indices being a scalar.
Another discord user faced the same issue, and onnx-simplifier did not change the gather op. In this case the node is not followed by an unsqueeze operation.

So for indices being a single scalar, we have two blockers:

there is no 0-dim tensor (scalar) support in Burn
the gather op with a scalar index removes a dimension, and there is no matching operation that performs this in Burn right now. We would need to do a sequence of tensor.select(dim, indices).squeeze(dim) (provided that the scalar is instead provided as [scalar] to make it 1d)

@laggui I have unassigned myself from this issue. As I recall, @SimonBrandner has linked an older version. So people who asked our help with FaceONNX is probably for newer versions.
I am closing it for now.


Program fails if /etc/machine-id is not available

Trying this on an embedded system (router) that does not have /etc/machine-id, the program fails.

Workaround is to just write your own random ID to that file, but we want to do something different in future.


Virtual contact sensor states reversed

SmartThings wants the contact sensor to default to closed and switch to open when tripped

This was resolved in dc64f5564c00e7ad07fdd706caa3e1ad67cf603e.  It's now configurable during the initial installation.


Fixed widget template path in custom layouts section

Modify this link to include the branch name, and possibly the page this PR modifies:
Branch
https://github.com/adaybujeda/ood-documentation/tree/custom_widgets_template_path
Page that modifies:
https://osc.github.io/ood-documentation/release-2.0/customization.html#custom-layouts-in-the-dashboard
Fixes to the template location for custom widgets.
As well as some configuration settings needed for the external location to work.

Thanks for the PR!


Add Morph Axis

This new custom axis is included in Kablammo (a private project yet) to modify the shapes of the glyphs in a playful and sort of  "random" way.

tag: "MRPH"
display_name: "Morph"
min_value: 1.00
default_value: 1.00
max_value: 4.00
precision: -2
fallback {
  name: "Phase 1"
  value: 1.00
},
fallback {
  name: "Phase 2"
  value: 2.00
},
fallback {
  name: "Phase 3"
  value: 3.00
},
fallback {
  name: "Phase 4"
  value: 4.00
}
description:
  "As the morph axis progresses, the letterforms transform"
  " into new variations, in ways specific to that typeface."

Morph would be the single axis of the project (no weight axis) thus the need for the 4 fallback positions.
On the private discussion about this axis @davelab6 said:

We also discussed that it will be very desirable to have the 4 named instances in the fvar table; since this is a single axis, strongly display, project, I think it worth exploring if we can establish a exception to the general fvar/stat table policy. @m4rc1e please discuss this with @vv-monsalve

On the private discussion about this axis @m4rc1e said:

Re fvar instances, no problem if you want to include all 4 styles as instances. I think for display families we should be making exceptions to our usual rule.

Inspecting this now I think MORP would be a better tag

That's ok with me

I'm risking a bike-shed debate here, but I think MRPH more closely hints (phonetically) at the word "morph"

I'm risking a bike-shed debate here, but I think MRPH more closely hints (phonetically) at the word "morph"

If one already knows the actual name of the axis, then yes. But I wondered before knowing what it is and what it does, what could be more descriptive once in the font name, e.g. Kablammo[MRPH] .ttf or Kablammo[MORP].ttf
That's why I commented it here so we can review/define it ;)


(Static Dashboard) Maps view, if I search for a map, I cannot duplicate in the UI

Related with https://github.com/CartoDB/support/issues/689
This bug should be fixed also in the Static Dashboard --> https://github.com/CartoDB/cartodb/milestone/144

This PR https://github.com/CartoDB/cartodb/pull/12952 fixes this issue for static dashboard. Well done, @rubenmoya! 😂  Closing it.


Carthage can't compile

Steps to reproduce the problem:

Create an iOS project
Create Cartfile and adds github "goktugyil/EZSwiftExtensions"
carthage update EZSwiftExtensions --platform ios

Using Carthage 0.25.0
➜  TestCarthage git:(master) ✗ carthage update --platform ios
*** Fetching EZSwiftExtensions
*** Checking out EZSwiftExtensions at "1.11"
*** xcodebuild output can be found in /var/folders/8b/dspfzsq952x7rzvsmlpv2vdc0000gn/T/carthage-xcodebuild.byR5gt.log
*** Building scheme "EZSwiftExtensions-iOS" in EZSwiftExtensions.xcworkspace
Build Failed
	Task failed with exit code 65:
	/usr/bin/xcrun xcodebuild -workspace /Users/carlbrusell/Projects/TestCarthage/Carthage/Checkouts/EZSwiftExtensions/EZSwiftExtensions.xcworkspace -scheme EZSwiftExtensions-iOS -configuration Release -derivedDataPath /Users/carlbrusell/Library/Caches/org.carthage.CarthageKit/DerivedData/8.3.3_8E3004b/EZSwiftExtensions/1.11 -sdk iphoneos ONLY_ACTIVE_ARCH=NO BITCODE_GENERATION_MODE=bitcode CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY= CARTHAGE=YES build (launched in /Users/carlbrusell/Projects/TestCarthage/Carthage/Checkouts/EZSwiftExtensions)

This usually indicates that project itself failed to compile. Please check the xcodebuild log for more details: /var/folders/8b/dspfzsq952x7rzvsmlpv2vdc0000gn/T/carthage-xcodebuild.byR5gt.log

The log is carthage-xcodebuild.byR5gt.log.txt

I guess here is the reason:
/Users/carlbrusell/Projects/TestCarthage/Carthage/Checkouts/EZSwiftExtensions/Sources/UIButtonExtensions.swift:16:47: error: Superfluous Disable Command Violation: SwiftLint rule 'function_parameter_count' did not trigger a violation in the disabled region. Please remove the disable command. (superfluous_disable_command)

@borkuslav thanks!
EZSwiftExtensions uses Swiftlint and it is violating one of the rules. We have to ways to deal with this:

Remove function_parameter_count which is being called but did not trigger any violation.
Disable superfluous_disable_command for the whole project.

Which path is the best one here?
I can manage to do a pull request with those changes.

These were already fixed in #444.
@Khalian we need to launch a new version so @icefall, @borkuslav and other Carthage users can use the app. Should we wait for any other PR or I can launch 1.12 right now?

@lfarah  Na. Go ahead.

Looks like this is already fixed


[duotone appearance]: tokens changes

After some research and discussion it was decided that the duotone appearance should be changed as described below:
Outline should change to 300 (Light theme)
Content should be changed to 600 (Light theme)

Reflect in Figma please, alongside the appearance matrix

in the process of changing the dueatone color I'm trying to make some order in the appearance map-color-names.

We are using same names for same purpose color - do we need it?
We need an additional name for the scale - darker than soft + faint, lighter than primary (which is also intermediate).

Tried to sum it up in visual:

We are using same names for same purpose color - do we need it?
We need the duplication for different usages
We need an additional name for the scale - darker than soft + faint, lighter than primary (which is also intermediate).
added only one additional name - pale


Revert "Use newer container-based builders in Travis and test JDK8 too"

This reverts commit 88a800dd10e7cf82216956481c8223acb5d47752.
Rationale is that the build is too hosed and the rationale for using the newer containers is speed. Once we get anything passing, we can look at the optimization of using container builds. In the mean time, it is arduous and time consuming to troubleshoot this.

cc @rtyler @abesto

ps JDK 8 happened to be in the same commit, but I've no strong feelings about that.


ldc: 1.24.0 -> 1.25.1 update (fixes broken) ldc and enable dub to build with alternative D compilers #115028

Motivation for this change
ldc: 1.24.0 -> 1.25.1 (fixes broken) ldc due to new llvm
dub: enable to build with alternative D compilers
fixes issue: https://github.com/NixOS/nixpkgs/issues/114875 (see discussion)
replaces pull: https://github.com/NixOS/nixpkgs/pull/115028 (as discussed)
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

re: dub, not sure about the changes to dub default nix between the two patches posted; implemented request without double checking and only tested  now (sorry) but: removing the -f flag (using only -r instead of -rf) as requested in #115028  SuperSandro2000 results in a build failure for: nix-shell -p dub --pure
(it builds when the -f flag is retained) should/may I revert? (i.e. reintroduce -rf)
see: https://github.com/NixOS/nixpkgs/pull/115028

ok, read the error message understood the reason for SuperSandro2000 request removing one non-existent file fixes the issue

ldc: ok
dub: ok sorted (SuperSandro2000 request complied with (s/-rf/-r/))

You seem to be force-pushing your branch incorrectly, because other people's commits should not be in this PR. Either drop those commits or rebase your two commits on top of master.

heh, yes i did a few things without being aware of what the consequences would be, pulling to master trying to rebase to my D-lang branch, and aborting that ... i should just have force pushed by branch

Result of nixpkgs-review pr 115133 run on x86_64-linux 1

3 packages failed to build:

Literate
lumpy
sambamba

5 packages built:

cheesecutter
dub
ldc
onedrive
rund

Failing packages are because dmd is failling to build (which we already know). I guess you can either remove/replace the dmd dependency, or just leave them as is (not really sure what the proper thing is to do here, heh).

Removing dmd from the list of buildInputs for sambamba seems to fix sambamba and lumpy:
diff --git a/pkgs/applications/science/biology/sambamba/default.nix b/pkgs/applications/science/biology/sambamba/default.nix
index 8802736e..96993da7 100644
--- a/pkgs/applications/science/biology/sambamba/default.nix
+++ b/pkgs/applications/science/biology/sambamba/default.nix
@@ -1,4 +1,4 @@
-{ lib, stdenv, fetchFromGitHub, python3, which, dmd, ldc, zlib }:
+{ lib, stdenv, fetchFromGitHub, python3, which, ldc, zlib }:

 stdenv.mkDerivation rec {
   pname = "sambamba";
@@ -12,7 +12,7 @@ stdenv.mkDerivation rec {
     fetchSubmodules = true;
   };

-  nativeBuildInputs = [ which python3 dmd ldc ];
+  nativeBuildInputs = [ which python3 ldc ];
   buildInputs = [ zlib ];

   # Upstream's install target is broken; copy manually

Similarly with Literate,
diff --git a/pkgs/development/tools/literate-programming/Literate/default.nix b/pkgs/development/tools/literate-programming/Literate/default.nix
index 87213b91..6174fc0c 100644
--- a/pkgs/development/tools/literate-programming/Literate/default.nix
+++ b/pkgs/development/tools/literate-programming/Literate/default.nix
@@ -1,4 +1,4 @@
-{ lib, stdenv, fetchgit, dmd, dub }:
+{ lib, stdenv, fetchgit, ldc, dub }:

 stdenv.mkDerivation {
   pname = "Literate";
@@ -10,7 +10,7 @@ stdenv.mkDerivation {
     sha256 = "09h1as01z0fw0bj0kf1g9nlhvinya7sqq2x8qb6zmhvqqm6v4n49";
   };

-  buildInputs = [ dmd dub ];
+  buildInputs = [ ldc dub ];

   installPhase = "install -D bin/lit $out/bin/lit";

I don't know either. Clearly dmd's build needs fixing.
We need the likes of @lionello and @ThomasMader
Thomas Mader has been a proponent of using Nix with D from before Nix* really had entered my consciousness  and Lio knows lots about D on Nix, (he must to do what he has done). There is also @Marler of rund fame. Ideally these three and others interested in D should agree the direction to take. I am sure there are others I do not know, (my interest has been narrow, I love my NixOS environment experience, and have my project that I wish to work on (without sacrificing being able to use NixOS)). I do not know Nix in a way that would allow me to continue the work on building D compilers using Nix in the way they have or even contribute particularly meaningfully and hope they continue to use D on Nix and do the heavy lifting ... it seems more manpower is needed just to keep things up to date based on the work already done. (that said, I am chuffed that the little dub default.nix fix might help any other D project on Nix, wonderful, I should have thought to complain sooner!).
There soon will be 3 D compilers with different strengths ... ideally any one could be chosen, and such flexibility should be preserved, but perhaps there should be a default. Though not much of an issue now I believe D libraries may need to be built by the same compiler as the calling program for it to build, which has implications for the storing of binaries for reuse. Which of the 3 should be the Nix default, there are arguments for each:

dmd is reference and very fast in its compilation, good also for scripting;
ldc produces cleaner faster binaries (than dmd) and has been this way for a while, it targets more architectures (llvm)
gdc/gcc will produce cleaner faster binaries (than dmd), and targets more architectures (gcc) ... it has been behind current dmd
My vote at least for the time being is for ldc as current default, to be reviewed in light of gcc once that is in full flight.
While on the subject Lio has dub2nix somewhere on github, hmm, ok it is here: https://github.com/lionello/dub2nix but it is not in nixpkgs and it should probably be at very least to make it more easily discoverable, I happened to be told of it by adisbladis on irc.freenode #nixos (or would not have thought to have looked for that specific name) ... dub2nix has also recently been discussed on the general D forum, so how to copy Lio and request that he adds it.

To the specific question at hand, matching the pattern I have a suggestion to use the dcompiler sequence currently in dub that I have pushed to branch D-lang-other at https://github.com/ralph-amissah/nixpkgs essentially:
for Literate:
- { lib, stdenv, fetchgit, dmd, dub }:
+ { lib, stdenv, fetchgit, ldc, dcompiler ? ldc, dub }:

and
- buildInputs = [ dmd dub ];
+ buildInputs = [ dcompiler dub ];

and for sambamba:
- { lib, stdenv, fetchFromGitHub, python3, which, dmd, ldc, zlib }:
+ { lib, stdenv, fetchFromGitHub, python3, which, ldc, dcompiler ? ldc, zlib }:

and
- nativeBuildInputs = [ which python3 dmd ldc ];
+ nativeBuildInputs = [ which python3 dcompiler ];

but I do not know whether this is the right or best way, or even as yet fully comprehend the implications of this choice for the future. Those who do, please chime in!
DMD in Nixpkgs needs to be updated, it is broken, that needs to be flagged ... an issue report should be filed

So I see two separate issues here:

DMD is currently (and has been for a while) broken on nixpkgs.
It may be desirable to pick, for each D package, which D compiler to use.

I think these can be tracked in a separate issue outside of this PR though, since the current PR doesn't actually create any new breakages in any packages. So this PR I think can (and should) be merged in.
@SuperSandro2000 thoughts?

@i077 well I bring up perhaps 3 issues only the first of which is immediately relevant:

(i) this patch (a) to update (thereby fixing) ldc and (b) making it possible to have a working dub, is ready to go, (unless there are other things that need to be done for the patch); rather nicely it appears to offer a fix for a few other D related packages in nixpkgs.

and as you mention (musings brought up by me, and not of direct relevance):

(ii) "DMD is currently (and has been for a while) broken on nixpkgs".
(iii) "It may be desirable to pick, for each D package, which D compiler to use". For which I wonder, whether down the line it might be possible to have a dcompiler option that can be set with the default there perhaps at ldc (or in future gdc), ...?

So is this (i) ready to go? or is there more that needs to be seen to?
@SuperSandro2000 @i077 thanks again


Crunchyroll subtitles (old format) decryption is broken

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x] I've searched the bugtracker for similar questions including closed ones

Question
Hello. Recently Crunchyroll changed encryption for their subtitles for RPC api, so decryption will always fail.
Code fragment:
https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/crunchyroll.py#L272

No problem on my side, maybe verbose log will be more helpful...

When getting the normal HTML frontend page, look for "subtitles":. It is followed by a JSON encoded array of subtitle entries with a language/title field and and an URL field.  It doesn't seem to require anything special beyond extracting the URL and maybe logging in originally.


[BUG] bundle format is ambiguous (could be app or framework)

Describe the bug
The compilation for "My Mac (Rosetta)" on MacBook M1 fails with "bundle format is ambiguous (could be app or framework)".
To Reproduce
I added the framework via manual install to my SwiftUI Multiplatform project.
It compiles for iOS, but fails for macOS.
Expected behavior
Compile.
Screenshots / Videos
If applicable, add screenshots to help explain your problem.
Environment:

OSX: macOS 12 Beta 3
Device: iOS 15.0 Simulator
SwiftUIPager version: 2.3.1

I found https://blog.inventic.eu/2015/03/os-x-codesign-failed-bundle-format-is-ambiguous-could-be-app-or-framework/ without trying that.
Does this framework use symbolic links?
I have some frameworks in use, but this would be the first with symbolic links. Are these necessary?

Hi @AshStefanOltmann . This framework shouldn’t have any symbolic link but I might be wrong as it uses a project file to build the xcframework, I could have added a symbolic link there by mistake.
Have you tried adding the xcframework with the latest (non-beta) macOS and Xcode? Is this something that just happens compiling for Rosetta? What about when compiling for arm64?
Just in case, I guess you’ve followed this tutorial to embed the framework. I wrote that a while ago, before the first M1 came out. I’m also thinking that this xcframework is compiled by using the latest Xcode (not the beta version). There might be something different in the compilation there. Something you can try is downloading the repo and executing ./scripts/build_xcframework.sh. Don’t forget to first use xcode-select -s <path/to/beta/>Xcode.app
I've never seen that kind of error 🤔

I Use Kotlin Multiplatform 1.5.21 to build my shared xcframework which does not support arm64 - this will change with recently released 1.5.30, but for now I can only test Rosetta - and I will need to provide a Rosetta installation for non-M1 user of my app anyway.
Also I do use async/await, so non-beta XCode can't compile my project right now.
I will try to build the framework myself as you suggested.

That did not work. I hope this helps you to get a clue what's wrong. I have no idea.
No error for iOS target, but I did not try to actually use the library so far.
dyld[10381]: Library not loaded: @rpath/SwiftUIPager.framework/Versions/A/SwiftUIPager
  Referenced from: /Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/MyApp.app/Contents/MacOS/MyApp
  Reason: tried: '/Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/SwiftUIPager.framework/Versions/A/SwiftUIPager' (code signature in (/Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/SwiftUIPager.framework/Versions/A/SwiftUIPager) not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)), '/Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/MyApp.app/Contents/MacOS/../Frameworks/SwiftUIPager.framework/Versions/A/SwiftUIPager' (no such file), '/Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/MyApp.app/Contents/MacOS/../Frameworks/SwiftUIPager.framework/Versions/A/SwiftUIPager' (no such file), '/Library/Frameworks/SwiftUIPager.framework/Versions/A/SwiftUIPager' (no such file), '/System/Library/Frameworks/SwiftUIPager.framework/Versions/A/SwiftUIPager' (no such file)

Same error as before.
It's the resign step that fails.
I have my own shared xcframework and also the Sentry.io framework in my project with "Embed & sign" - no issues there.
/Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/MyApp.app: replacing existing signature
/Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/MyApp.app: bundle format is ambiguous (could be app or framework)
In subcomponent: /Users/sol/Library/Developer/Xcode/DerivedData/MyApp-fdxfwjrcotubsrgvsztcciskrqlg/Build/Products/Debug/MyApp.app/Contents/Frameworks/SwiftUIPager.framework

@AshStefanOltmann the xcframework contains a binary for arm64 (the M1). If not supported, could this be the issue?
Is Flutter what you're using? I'm reading a lot of answers where people basically remove stuff from folders:
https://www.gitmemory.com/issue/flutter-webrtc/flutter-webrtc/421/735307303
https://www.gitmemory.com/issue/flutter/flutter/77173/790272366

@AshStefanOltmann the xcframework contains a binary for arm64 (the M1). If not supported, could this be the issue?

I don't know, maybe. Nice that you already provide one. I wait for Kotlin 1.5.30 to be broadly adopted until I can build my shared framework with it.

Is Flutter what you're using? I'm reading a lot of answers where people basically remove stuff from folders:

Nope, no flutter. I use Kotlin Multiplatform Mobile. I build my shared code module with that.
This means I have a native XCode project with a native SwiftUI where I want to include your framework.
Kotlin should have no effect here.

@AshStefanOltmann I cannot test it myself because I don't have a M1. Have you tried what those links I shared above suggest? Technically it's a matter of making sure that the macos-arm64 binary is removed from the binary

Did you fix the problem or is now a "won't fix" solution?

@AshStefanOltmann i hadn’t gotten any response in a while and that’s why I closed it 🤷🏻♂
Feel free to reopen and provide more information. I shared a possible solution in my previous comment

@fermoya Your solution of removing the macos-arm64 could only be a workaround for now. In the long-term you need to provide a universal XCFramework for iOS & macOS and both architectures. Otherwise there is unfortunately not much use for me.

@AshStefanOltmann xcframeworks contain all possible symbols for every single platform supported by Apple. macOS-arm64 refers to the M1 and M2 chips. You’re having problems with Rosetta, which simulates the old Intel chip.
this seems to me as a Kotlin multi platform problem. I’m giving support to the required platforms


Fix Mesh::ATTRIBUTE_UV_0 documentation

Comment incorrect suggests that texture is clamped outside of 0..=1 range, while it can actually be configured.
CC https://github.com/bevyengine/bevy/pull/11109

Looks like latest stable brought new clippy warnings. I'll try to fix.


Add a description field to tweaks

Supports Commonmark
Does not support GitHub @-mentions

Fixes #8

Coverage decreased (-0.7%) to 85.149% when pulling 4f725c0145fa5e6bf112c98eb0e5d5972699570e on description into 57380e3f36ccab1b55b1c40fa1587314fff74e50 on master.


Question about ADDB sample generation?

If ADDB consists of several tiers, how do you know where the bottleneck of performance of ADDB is, especially when you have multiple configuration files simultaneously? Or does it not matter?

It's hard to recognize where the bottleneck of performance of ADDB is. We will analyze their relationship by machine learning methodologies.
Thanks for asking!


Missing REST API descriptions in JSON

I have 2 REST services:

@RequestMapping(value = "/flights/{id}", method = RequestMethod.GET, params = {"view=registerHedging", "step=1"})
public  FlightData registerHedgingBackToStep1() {
}
@RequestMapping(value = "/flights/{id}", method = RequestMethod.GET, params = {"view=takeoff", "step=1", "type=dailyFlight"})
public  DailyData dailyDataBackToStep1(@RequestParam(required=true) String type) {
}

I'm only getting the API for the first REST services and the second one won't show up.

Yeah... that is a limitation in the current swagger specification. I did add an incubating feature (see #21) to help with that, but unfortunately swagger-ui doesn't play well with it yet.

@dilipkrish
"but unfortunately swagger-ui doesn't play well with it yet" - can you please elaborate ?
I've looked at the RFC. It seems like it should generate the output the way you've described.
Will enableUrlTemplating(true) generate that kind of output ? (and to my understanding it will not be readable by the Swagger UI ?)
What I'm trying to figure out is why params aren't fully supported in springfox.
Is it an issue of one or a combination of the these :

The current swagger spec (http://github.com/swagger-api/swagger-spec/blob/master/versions/2.0.md)
Swagger UI (http://github.com/swagger-api/swagger-ui)
Springfox.

Going through other issues, my issue seems similar to issue #750
Since I can't afford to have missing APIs in my UI, do you have any suggestions ?

This is now enabled via incubating feature to enableUrlTemplating via the Docket. Please provide feedback if you're using it.

wasn't sure where to leave feedback other than here. I've been using the enableUrlTemplating feature and it is totally awesome. Works in both swagger spec and swagger ui flawlessly. Thanks for the awesome work

@chrisdostert are you using it with featured turned on? If so would love to get your feedback

Yes, it's been working great so far. It's a very clever solution

@chrisdostert Dint think swagger-ui supported RFC 6570... parameterized query parameters in uri templates, in particular. Im surprised its working even!

Ah, I got so excited when all parameterized paths showed up I didn't actually submit a request. I just did and you are correct, swagger-ui does hydrate the parameterized query parameters but doesn't strip the variable template from the path when it executes a request. It's still much better than before in that all paths including those with parameterized query parameters show up and you can still successfully submit to paths without parameterized query parameters. We need to get swagger-ui updated to support this! : )

:+1: thanks for plus+1ing that PR!

I apologize if the right place to leave feedback is not on this closed issue.
Looking at the comment on https://github.com/swagger-api/swagger-ui/issues/1665 , I'm not really sure whether the right path moving forward is RFC 6570 since it seems like it will not get any swagger-ui support.
Looking at https://github.com/domaindrivendev/Swashbuckle/issues/142 it looks the approach SwashBuckle is taking is merging multiple operations into one leveraging optional parameters. Perhaps this approach could make sense for springfox as well?

I chatted with @webron a while back about this. His take on it is to try and discuss this as part of swagger spec next and decide on the way to go forward. So its not necessarily a dead end as his comment might suggest :) My hope is that we stick to standards when defining uris for services. The changes to support this in swagger ui has also been done and I've submitted a PR.
Having said, My opinion is that RFC 6570 is the way forward. I say this for a couple of reasons...

Service descriptions need to be deterministic. So just bunching a whole lot of parameters together and calling them optional isn't going to cut it. For e.g. lets say we have two operations findCustomerByNameAndEmail which has two parameters name and email and findCustomerByPhoneNumber  which has one parameter phone. It makes no sense to club them into one operation findCustomer with three optional parameters. How does the api description speak to developers in that case, which parameters should I provide?
When we design controller methods that reflect this expectation we would like to describe it via uri templates
Its not clear how these operations can be realized as distinct operations in implementation when one is doing swagger-definition first service design.
RFC 6570 is a standard and we want specifications to be built on standards.

Thanks @dilipkrish, I appreciate your prompt response.

It's very nice for one of my endpoints, but not great for the others, because it adds a bunch of information that is unnecessary when the multiple endpoints per uri/verb is not present. Would be great to be able to limit this per endpoint/resource class, as well has to have swagger ui respond correctly.

@orbfish Not sure what you mean. If its a suggestion, I'd recommend opening up a new feature request with more details and examples.

@dilipkrish thanks for the reply! I have added springfox-swagger-ui-rfc6570 instead of springfox-swagger-ui and enableUrlTemplating， but the Request URL is http://localhost:9000/api/statistics{?days}&days=5

You need to also use the appropriate swagger-ui.

please read https://stackoverflow.com/questions/33468129/springfox-404-error-when-testing-api-from-swagger-ui


AbstractColumnFamilyTemplate is not thread safe

activeSlicePredicate is mutated in setCount() and HSlicePredicate.count field is not volatile so other threads may not see mutation;
activeSlicePredicate is mutated in ThriftColumnFamilyTemplate.multigetSliceInternal(). This mutation may have side effects in concurrent execution environment;
activeSlicePredicate is mutated in addColumn() -> mutates HSlicePredicate.columnNames which is ArrayList instance (not thread safe);
batched, columnFactory, clock and exceptionsTranslator fields are not volatile so other threads may not see changes from corresponding mutation methods;

Also you may want to make all fields final (and the class abstract).

LOL at missing abstract!
Summary:

clock was fixed in #400
added clone() to HSlicePredicate as the modification of activeSlicePredicate in ThriftColumnFamilyTemplate is indeed ugly.
made relevant fields final
technically you are correct about exceptionTranslator and batched, but the design goal here is configure once and use and i don't see messing with those frequently from within an application as a common use case.
but in the same vein, i removed setColumnFactory as that should be final


Add F# template in samples

Is your feature request related to a problem? Please describe.
Right now there is no built in template for F# projects.
Describe the solution you'd like
A simple F# project which already has all necessary dependencies etc. included with sample code which creates an "empty" game.
Describe alternatives you've considered
Rewriting existing C# templates to F# everytime I want to create a new project in F#.

As soon as I got something working I'll try to contribute a F# template :)


Key saved on Phone confirmation

As a user, I am not sure if TIKI has saved the keys on my phone (no confirmation, or anything)

rolling into #18


Speed up pure Go version of Grain

The trick here is that negation turns a single bit into a mask: -0 is all zeros, while -1 is all ones.
The pure Go version is now faster than the assembler on amd64:
name           old speed      new speed      delta
Auth-8         88.7MB/s ± 1%  93.7MB/s ± 1%   +5.60%  (p=0.000 n=10+14)
AuthGeneric-8  69.0MB/s ± 3%  93.5MB/s ± 1%  +35.61%  (p=0.000 n=10+13)
Seal1K-8       44.5MB/s ± 3%  46.5MB/s ± 1%   +4.48%  (p=0.000 n=10+14)
Open1K-8       43.8MB/s ± 1%  46.1MB/s ± 1%   +5.20%  (p=0.000 n=10+15)
Seal8K-8       45.7MB/s ± 1%  47.5MB/s ± 1%   +3.93%  (p=0.000 n=10+15)
Open8K-8       44.8MB/s ± 1%  47.1MB/s ± 3%   +5.18%  (p=0.000 n=10+15)

If the assembler version of accumulate were removed, the function could be inlined for a further speed boost.

looks great, thanks so much!

If the assembler version of accumulate were removed, the function could be inlined for a further speed boost.

I'll look into this. I've been on the fence about the assembly implementation from the get-go because it's only marginally beneficial.


What even is this?

I've deleted the less kind words I initially wrote. Please can you read the comments below and look at addressing how you organise this repo.
Organising images by branch
This is the first time I've ever seen a repo organised by branch per project. Effectively this is a monorepo organised by branch. The branch per image thing is confusing at a minimum, but the monorepo aspect presents more challenges.
If someone wants to fork it to contribute back or clone it to use as a base to make their own image. Due to this being a repo of base images, you end up forking/cloning all of them. It would be far better to have a repo per image as the majority of folks forking or cloning would only ever use 1 image.
What is the process guideline for contributing back with this image?
Building the base image from tar files
You build the base image from tar files, which are based on Lambda Layers from my understanding. This means for me to understand what is in the docker image I have to clone the repo locally and untar the images. If I want to contribute back do I need to then rebuild that tar and include it in the repo? Or do I find a repo with the layer? Does a separate repo with these layers exist? How do you inspect changes and review PR's like that? You can't effectively handle contributions back when the base image is a set of tar files
Insecure Images and unneeded dependencies
I've just tried to build a Lambda Node.js container image, and while it completed successfully it spewed out a bunch of node.js errors, listed below

Sending build context to Docker daemon  950.8kB
Step 1/4 : FROM public.ecr.aws/lambda/nodejs:12
 ---> 9549f0a87034
Step 2/4 : COPY **/*.js **/*.json ./
 ---> 5753c7880774
Step 3/4 : RUN npm install
 ---> Running in 21b48a29cd7b
npm WARN deprecated<EMAIL_ADDRESS>This package has been deprecated and is no longer maintained. Please use @rollup/plugin-babel.
npm WARN deprecated<EMAIL_ADDRESS>"Please update to latest v2.3 or v2.2"
npm WARN deprecated<EMAIL_ADDRESS>Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated<EMAIL_ADDRESS>request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated<EMAIL_ADDRESS>fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated<EMAIL_ADDRESS>this library is no longer supported
npm WARN deprecated<EMAIL_ADDRESS>https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated<EMAIL_ADDRESS>request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated<EMAIL_ADDRESS>Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)

><EMAIL_ADDRESS>install /var/task/node_modules/node-sass
> node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.14.1/linux-x64-72_binding.node
Download complete
Binary saved to /var/task/node_modules/node-sass/vendor/linux-x64-72/binding.node
Caching binary to /root/.npm/node-sass/4.14.1/linux-x64-72_binding.node

><EMAIL_ADDRESS>install /var/task/node_modules/husky
> node husky install

husky > Setting up git hooks
git --version command failed. Got null.
husky > Failed to install

><EMAIL_ADDRESS>postinstall /var/task/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon:
> https://opencollective.com/core-js
> https://www.patreon.com/zloirock

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)

><EMAIL_ADDRESS>postinstall /var/task/node_modules/node-sass
> node scripts/build.js

Binary found at /var/task/node_modules/node-sass/vendor/linux-x64-72/binding.node
Testing binary
Binary is fine

><EMAIL_ADDRESS>postinstall /var/task/node_modules/husky
> opencollective-postinstall || exit 0

Thank you for using husky!
If you rely on this package, please consider supporting our open collective:
> https://opencollective.com/husky/donate

><EMAIL_ADDRESS>postinstall /var/task/node_modules/vuepress
> opencollective-postinstall || exit 0

Thank you for using vuepress!
If you rely on this package, please consider supporting our open collective:
> https://opencollective.com/vuepress/donate

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/rollup/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/@vuepress/core/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/watchpack-chokidar2/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/webpack-dev-server/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 2119 packages from 1636 contributors and audited 2127 packages in 50.811s

92 packages are looking for funding
  run `npm fund` for details

found 4 high severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details

Why are all of these unneeded modules being installed? I don't understand why my Lambda container now has Vuepress installed? What is using request that has been deprecated by it's authors? How do I fix or even evaluate the 4 high severity vulnerabilities highlight by the npm install process??
You do realise this will set off all kinds of security alarms when scanned by vendors, and leave your users having to explain to security teams why they're bringing an insecure container into their environment. When you surface partial information to users (ie through npm logs), you need to be able to provide the answers to the questions this information raises, or else you will erode trust.
Yes, this might all be safe because these deps get jettisoned or not used when run in Lambda, but 'trust us' isn't an answer, allow us to verify it, and allow us to fork and build a minimal version without unused deps if need be.
How do I make this simple change?
Coming back to why I originally came to this repo. I want to run a Lambda container using Node.js with more than memory than Node.js has access to by default to take advantage of the 10GB of memory available to Lambda now. To do this I need to start node with a simple feature flag. I am assuming somewhere in lambda-entrypoint.sh is the command to start Node.
All I want to do is clone the repo, add --max-old-space-size=8192 to the node command in lamdba-entrypoint.sh.  Where do I even start with this bonkers and opaque way of laying out this repo. I'm trying to add 26 characters to a single line. How do I do this?
My recommendation is this...

Move each base image to it's own repo. This will allow folks to fork and clone just the repo they need, either to contribute back, or to create their own versions of the image for their own purposes.
Put all layer code into those singular repos, or make them available in separate repos, don't hide them in tar files.

Where is the Customer Obession? Where is Simplify in Invent & Simply? Where are you Insisting on the Highest Standards?
Please stop building for yourselves. Build for us, your customers.

Hi Ant, thank you so much for your feedback.
The reasons behind the weird structure and contents of this repository is because this was planned to be an artifact store instead of a Git repository. This is also why there are no contribution guidelines (besides contributions to the documentation).
We are reviewing internally the feedback you provided on this regard and thinking on how to make this repository more useful for you.
Regarding the insecure and unneeded dependencies. From what I can see in the log of the docker build that you provided, these are dependencies being installed by node as they are declared by your package.json file. I am sure that the Lambda runtimes and base images do not depend on Vuepress or requests. You can see here the NodeJS runtime client dependencies. Could you try to run npm install in a different Docker image, or your local environment?
Regarding your question around changing the max-old-space-size. The base images already set this configuration based on the size configured for your Lambda Function sandbox when running inside the lambda environment. Currently is assigned 90% of that configuration. So, formally you do not need to modify the base image for this to work.
It is possible to see this by opening the file /var/runtime/bootstrap inside the Node base docker image.
In the following snippet I am showing only the relevant code:
$ docker run -it --entrypoint=bash public.ecr.aws/lambda/nodejs:12
bash-4.2# cat /var/runtime/bootstrap
[...]

if [ -n "$AWS_LAMBDA_FUNCTION_MEMORY_SIZE" ];
then
  new_space=$(expr $AWS_LAMBDA_FUNCTION_MEMORY_SIZE / 10)
  semi_space=$(expr $new_space / 2)
  old_space=$(expr $AWS_LAMBDA_FUNCTION_MEMORY_SIZE - $new_space)
  MEMORY_ARGS=(
    "--max-semi-space-size=$semi_space"
    "--max-old-space-size=$old_space"
  )
fi
[...]

NODE_ARGS=(
    --expose-gc
    --max-http-header-size 81920
    "${MEMORY_ARGS[@]}"
    /var/runtime/index.js
    )

if [ -z "$AWS_LAMBDA_EXEC_WRAPPER" ]; then
  exec /var/lang/bin/node "${NODE_ARGS[@]}"
[...]

You can use a Wrapper Script to modify the behaviour of this file, or you can replace it with yours if you need something more specialised.
I understand that navigating these files without any documentation is not a good developer experience so we will also discuss internally where we can add more documentation for this.

Chiming in with a similar experience - I arrived here while looking to understand what's inside that FROM public.ecr.aws/lambda/nodejs:14 at the top of my Dockerfile. Where can I see the Dockerfiles I'm deriving from? It's clear that this repo is not meant to answer that question.
I don't know how best to manage a large tree of Docker layers and tags, but I know other organizations do it in a way that the user of that image can find the upstream Dockerfiles. Here are a few examples I've worked with in the past:
https://github.com/dotnet/dotnet-docker
https://github.com/CircleCI-Public/circleci-dockerfiles
https://github.com/nodejs/docker-node
https://github.com/cypress-io/cypress-docker-images
So, in the long term, please figure this out!
In the near term, here are the approaches I'm aware of to understand e.g. lambda/nodejs:14 is doing:

shell into the docker image (e.g. docker run --entrypoint /bin/bash -it public.ecr.aws/lambda/nodejs:14) and poke around
clone a branch of this repo, unpack the tarballs, and look through the files

is there anywhere else I can go?

Here's a timely post from a team that maintains container images at Microsoft, which may offer some ideas:
https://devblogs.microsoft.com/dotnet/staying-safe-with-dotnet-containers/

I think it would be great if there was some way to understand how the binary artifacts in this repository were created, otherwise this repository is mostly useless (since we can get the same effect by pulling the finished Docker images and poking through them, and that's a whole lot simpler than this).
Here's an example (real) use case: We have base docker images (per programming language) that we use across many different execution environments (i.e. servers/VMs, long-running ECS services, Batch) and we would like to add Lambda to that list. For example, we may have a basic Docker image that includes tooling that everything needs, enterprise certificates, etc, then we build language-specific images on-top of that, such as a Java base image. Then we may further customize that to create Java-on-ECS and Java-on-Batch images (though more likely we just use the Java base image directly in most places).
How do we customize our Java base image to create a Java-on-Lambda base? I'm pretty sure the answer is "you can't do that". I mean, sure, we could create our own custom Lambda runtime from scratch, and AWS even provides most of the pieces if we can figure out how to wire it together, but that seems to be an exercise left to the reader. We could try to dissect what's going on in the existing image (which is of course made more difficult because it's composed by opaque blobs of compressed files) but then we are off in both undocumented and unsupported land.
In the end, we'll probably have to recreate any relevant customization on-top of the AWS-provided Lambda base images (which we can do, since we know what our own customizations are). It will work, but it's a lot of effort for something that should be simple, and that's effort that is only needed for Lambda and no other system. And even then, we're still stabbing-in-the-dark. It looks like cacerts is in /var/lang/lib/security/cacerts - but is it guaranteed to always be there? We need to run some code at startup - we can override the ENTRYPOINT and then exec /lambda-entrypoint.sh when we are done, but is that really the best way (and will it continue working into the future)?

Thank you for the feedback @abatkin. We don't have any short term plans to address at present, but we will consider.


Python related fixes for #355 and #359

Python related fixes for #355 and #359
The logic introduced allows pybind11 to try and use CMake if no explicit path was provided.
Along with that, the python import errors were caused by the respective modules not importing their required dependencies and were instead linked against them (this actually seems to fix some of the FLTO related errors we were typically having).
I also found that STRIDE_2D, STRIDE_3D and DEFAULT_STRIDE would cause an error when importing PyMaterialXRender to the tune of:
/build/inst/python/MaterialX/PyMaterialXRender.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN9MaterialX10MeshStream9STRIDE_2DE
The solution to this is to make them a read only property rather than linking to their pointers.

Apologies for the updates, we currently don't have a build system that deals with both py2 and py3 installs living alongside each other.

Em
======================================================================
FAIL: test_ReadXml (__main__.TestMaterialX)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "main.py", line 465, in test_ReadXml
    self.assertTrue(doc.validate()[0], filename + ' is not valid.')
AssertionError: False is not true : CustomNode.mtlx is not valid.

I'm not so sure what could be causing this?
I didn't actually change any of the logic involved in reading documents (that I'm aware of).

Thanks for putting this changelist together, @alister-chowdhury!  If the cause of the MacOS validation error isn't clear, we can take a closer look on our side.

@alister-chowdhury, how does this latest version look to you?  I've addressed the cross-platform compilation issues, but before merging I'd like to validate that it fixes the original issues you ran into.

@jstone-lucasfilm
So pybind11 is still found via cmake, however #359 is still a problem.
And importing PyMaterialXRender errors from undefined symbol: _ZN9MaterialX10MeshStream9STRIDE_2DE on import (because of the reverting of the changes in PyMesh.cpp).

@alister-chowdhury In my local environment, Windows 10 with Python 3.8, I'm now able to directly call import MaterialX.PyMaterialXGenGlsl, import MaterialX.PyMaterialXGenOsl, and import MaterialX.PyMaterialXRender without warnings or errors.
Would you be able to dig further into the errors that you're seeing locally, to see if there's a sequence of import calls that eliminates them?  My expectation was that the new calls to py::module::import would be sufficient to address them, but that doesn't appear to be the case.

@alister-chowdhury In case it helps to track down the issue you're seeing, I've verified that the master branch of MaterialX emits a Python error when I call import MaterialX.PyMaterialXGenGlsl without preparing it first with import MaterialX.PyMaterialXGenShader.  Running with this pull request, I get no errors when calling import MaterialX.PyMaterialXGenGlsl in a clean Python 3 session.

@jstone-lucasfilm I can confirm on linux / py3 I still get the error
PyMaterialXCore, PyMaterialXFormat and PyMaterialXGenShader import without complaint.
PyMaterialXGenGlsl, PyMaterialXGenOsl and PyMaterialXRender:
ImportError: PyMaterialXGenShader.cpython-37m-x86_64-linux-gnu.so.1: cannot open shared object file: No such file or directory
PyMaterialXRenderGlsl and PyMaterialXRenderOsl
ImportError: PyMaterialXRender.cpython-37m-x86_64-linux-gnu.so.1: cannot open shared object file: No such file or directory
When I order the imports as:
import MaterialX.PyMaterialXCore
import MaterialX.PyMaterialXFormat
import MaterialX.PyMaterialXGenShader
import MaterialX.PyMaterialXGenGlsl
import MaterialX.PyMaterialXGenOsl
import MaterialX.PyMaterialXRender
import MaterialX.PyMaterialXRenderGlsl
import MaterialX.PyMaterialXRenderOsl

The only error is:
ImportError: .../build/inst/python/MaterialX/PyMaterialXRender.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN9MaterialX10MeshStream9STRIDE_2DE
(From PyMaterialXRender).
I believe the "cannot open shared object file...." is coming from them being linked, and python/MaterialX/... is not going to be in LD_LIBRARY_PATH, so it errors (unless said .so has already been explicitly imported).
A especially inelegant solution would be to import them in the init.
I am not familiar with developing for OSX platforms, but I take it that they simply must be linked or strange problems propagate?

Thanks for those additional details, @alister-chowdhury, and those are very helpful in clarifying the picture.
It sounds like the py::module::import mechanism doesn't give us fully cross-platform behavior, so I would recommend that we remove these calls and simply require a manual import order for now.  We can follow up with the PyBind11 developers in the future for a more idiomatic way to accomplish this type of Python module inclusion.
Similarly, it sounds like it's not safe to bind constants such as STRIDE_2D, STRIDE_3D, and DEFAULT_STRIDE which are defined in shared headers.  I believe we can safely remove these Python bindings completely, as they're not likely to be required in Python scripts.

@jstone-lucasfilm would you like me to add imports ordered inside the __init__?
In regards to the STRIDE stuff, I originally opted to make it a read only property, so it acts as a const variable (from pythons point of view).

Would you like me to add imports ordered inside the init?

Since each application will have a different favored shading language (GLSL, OSL, and soon MDL), it's probably overkill to have the core init.py file bring all shader generators into the Python environment.  One option for the future would be to have a wrapper for each supported language, but for now I think it's most flexible to make the client responsible for import order, and we can create high-level wrappers for the most common inclusion patterns as they become clear.

In regards to the STRIDE stuff, I originally opted to make it a read only property, so it acts as a const variable (from pythons point of view).

That's an elegant solution, if access to those constants turns out to be useful.  But it's my sense that they were added just for the sake of completeness, and wouldn't be used in a practical Python script.  So for now, I'm thinking that we can take the simplest path and simply omit those Python bindings.

I think it could be possible to prevent import errors for most of the time, by simply importing PyMaterialXGenShader and PyMaterialXRender.
However, hopefully our build setup is an outlier and most people don't suffer from this :)

But it's my sense that they were added just for the sake of completeness, and wouldn't be used in a practical Python script. So for now, I'm thinking that we can take the simplest path and simply omit those Python bindings.

I expect so too, I'd imagine when you reach that level, you're probably going to go into C++.


rhino: 1.7R2 -> 1.7.15, various improvements

Remaining todos:

[x] Add update script
[x] Format with nixfmt-rfc-style
[x] Move it to by-name
[x] Add self to maintainers
[ ] Migrate to finalAttrs
[ ] Use stdenvNoCC
[ ] Figure out how to build rhino-all.jar
[ ] Test that this doesn't break LibreOffice Fresh

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Currently blocked. We need rhino-all for LibreOffice, but that hasn't been released yet. I've asked nicely for upstream to release a new version.


A translation error

I noticed a translation error at https://kubernetes.io/zh-cn/docs/reference/glossary/?all=true#term-containerd
. The brief description of each item will end with a “。”, but containerd does not have.

/language zh

/assign

/language zh

The English page also has this problem, we can update the English part first
https://kubernetes.io/docs/reference/glossary/?all=true#term-containerd
I feel that this punctuation problem does not affect reading, maybe this can be updated next time this page is updated by the way.
/language en

/close


Forbidden (401 - Missing Cookies) when making requests to APIs in SCM

Missing cookies server side.
Repro steps.

Create Azure App Service Web App
Add manual site extension with scmApplicationHost.xdt
Add aspnetcore3.1 sub application to SCM site
Make call to SCM Kudu API from sub application with propagated cookies
FedAuth* and LoginNonce* cookies are missing
Receive 401 Forbidden response

I manually output the cookies and headers from the server side and notice most of the cookies are missing in Request.Cookies while they are present in Chrome/Edge/Firefox.
Cookies in chrome developer tools

Headers as examined from the server side HttpContext:

Project structures.

Create template netcore3.1 app
Return headers to browser as string
Publish app to folder with scmApplicationHost.xdt

Point virtual directory to relative folder in C:\Home\SiteExtensions\MySiteExtension
Add this content with your app information to the scmApplicationHost.xdt
<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <system.applicationHost>
    <sites>
      <site name="%XDT_SCMSITENAME%" xdt:Locator="Match(name)">
        <application path="/sub-app-test" xdt:Locator="Match(path)" xdt:Transform="Remove" />
        <application path="/sub-app-test" applicationPool="%XDT_APPPOOLNAME%" xdt:Transform="Insert">
          <virtualDirectory path="/" physicalPath="%XDT_EXTENSIONPATH%\sub-app-directory" />
        </application>
      </site>
    </sites>
  </system.applicationHost>
</configuration>

The log/error given by the failure.

401 Forbidden when making requests to SCM app or deployed template app.

Could you even make a AJAX call to your own app (MySubApp)?  I would start with that first.  BTW, Kudu Debug Console or other extensions (such as /dev editor) do make AJAX calls to the SCM all the times (you can see from F12 -> Network).  Not sure why it would be any difference in your scenarios.

Hi
If the problem persists and is related to running it on Azure App Service, please open a support incident in Azure:
https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request
This way we can better track and assist you on this case
Thanks,
Joaquin Vano
Azure App Service


Completion of error handling

Would you like to add more error handling for return values from functions like the following?

malloc ⇒ dappendc
printf ⇒ handle_connection

Absolutely. Is that something you want to work on, or is this a suggestion?

I suggest to avoid ignorance of return values a bit more.
Would you like to detect every error situation as early as possible?

@elfring Ideally yes.

How do you think about to improve static source code analysis also for this software?

Are you interested to apply aspect-oriented software development?
Would you like to encapsulate error detection and corresponding exception handling as a reusable aspect in this software?

I don't know much about this but I would be open to hear more about it if you'd be willing to share why you think it is useful.

I suggest to read a bit more advice from linked information sources.


Add interstitial/intro/something else? to Meltano videos that say what Meltano is

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/957
Originally created by @emilie on 2019-09-05 14:39:33

This is especially relevant to the dogfooding videos. If someone is landing on the end-to-end process, it's great, but without a consistent intro, it's not totally clear what Meltano is, if you don't already know what it is.

View 5 previous comments from the original issue on GitLab


Update the molecule systemd configuration for cgroups v2

🗣 Description
This pull request updates the configurations of platforms in the molecule-with-systemd.yml file to support running on hosts using cgroups v2.

💭 Motivation and context
Currently our Ansible roles that need systemd-enabled images fail their molecule testing when run on Ubuntu 22.04 GitHub Actions runners. The following type of error is seen in the prepare step:
fatal: [debian12-systemd]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory.In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1670597193.4950671-6872-120647221743638 `\" && echo ansible-tmp-1670597193.4950671-6872-120647221743638=\"` echo ~/.ansible/tmp/ansible-tmp-1670597193.4950671-6872-120647221743638 `\" ), exited with result 1", "unreachable": true}

This is due to changes in behavior between cgroups v1 and v2. With these changes in place a role is able to run as normal.

🧪 Testing
Automated tests pass. I tested this previously in this run and I verified these changes allowed a role to clear the prepare step in this run in a downstream project.

✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[x] All relevant type-of-change labels have been added.
[x] I have read the CONTRIBUTING document.
[x] These code changes follow cisagov code standards.
[x] All new and existing tests pass.

Due to people being out of office, the somewhat critical nature of this pull request, and the constrained scope of tested changes we will be merging this pull request with only one team review.


DeveloperGuide: update "Setting up" section

Due to an ongoing issue1 with some of the newer versions of IntelliJ,
the IDE may report code errors for JAXB or JavaFx APIs depending on the
Intellij Edition being used, although the code can be build and run
successfully since we have already imported the dependencies through
gradle.
This may cause confusion amongst users.
Let's update the "Setting up" section to address this issue, and how
to resolve it.

Pull Request Test Coverage Report for Build 86

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.811%

Totals

Change from base Build 84:
0.0%

Covered Lines:
1433

Relevant Lines:
1544

💛  - Coveralls


Add an international landing page with info about the project: coronastatus.eu

Original idea from Gavin.
Use coronastatus.eu to point to the other sites and have documentation on how to join community and get a website for your country.

That sounds great, was thinking of exactly this!

Maybe we should show the aggregated data of all countries as well.

I've started on this: #194.

Here is a pull request in reference to this issue: https://github.com/BustByte/coronastatus/pull/560
It doesn't do this https://github.com/BustByte/coronastatus/issues/188#issuecomment-602112289 yet because each countries data is siloed


closing VimuxZoomRunner

How can I return back to vim when I close the zoomed tmux pane ? Right now I need to do q <bind> z <bind> k which seem like a lot of work.
Thanks

I don't quite understand what the situation is here. If you close the pane yourself doesn't it go back to VIM (or whatever the previous pane was) already? Is there something that still needs fixing in Vimux here or is this a moot point or a Tmux issue?


croutonxinitrc-wrapper: Kick xrandr harder

Workaround is racy, let's try multiple times until xrandr keeps
quiet.

+👟


Can we custom SoC or CPU and how

Though I know we can custom CPU itself and have read the README in soc/vexriscv, I have a few questions about it.
First, what does this line: PREREQUISITE 2: Ensure that you have the VexRiscv source submodule loaded.  in soc/vexriscv/README exactly mean?
Second, I'd like to add a co-processor to CPU through the AXI bus, in order to compare the performance between CFU and the co-processor. Is it possible to do that here?

Hi @limingxuan-pku!
I'm happy to hear that you are interested in customizing the VexRiscv.   The README may be a bit out of date; I will add some updates here.
PREREQUISITE 2: just means to load the git submodule for VexRiscv (and load it recursively).   The ./scripts/setup script will do that, or you could manually do it like this:
git submodule update --init --recursive third_party/python/pythondata_cpu_vexriscv

There are two levels of submodules; CFU Playground loads https://github.com/litex-hub/pythondata-cpu-vexriscv, and that in turn has https://github.com/SpinalHDL/VexRiscv as a submodule.
As loaded in CFU Playground, you will find the VexRiscv source code under ./third_party/python/pythondata_cpu_vexriscv/pythondata_cpu_vexriscv/verilog/ext/VexRiscv/src/main/scala/vexriscv/.
But, depending on what type of customizations you wish to do, you may be able to use a new mechanism recently added by @ShvetankPrakash.   Let me see if he has examples.

Customizing the CPU
I'll leave this example of the new functionality.   I just tested this on the Arty A7-35T board.   You can control each parameter independently:
make EXTRA_LITEX_ARGS="--cpu-variant=generate+bypass:false+csrPluginConfig:mcycle+dCacheSize:0+hardwareDiv:false+iCacheSize:8192+mulDiv:false+prediction:none+safe:false+singleCycleMulDiv:false+singleCycleShift:false" prog
make EXTRA_LITEX_ARGS="--cpu-variant=generate+bypass:false+csrPluginConfig:mcycle+dCacheSize:0+hardwareDiv:false+iCacheSize:8192+mulDiv:false+prediction:none+safe:false+singleCycleMulDiv:false+singleCycleShift:false" load

We don't have a reference manual for the parameterization, but the list of parameters and their default values is found here:  https://github.com/google/CFU-Playground/blob/main/soc/patch_cpu_variant.py#L148-L160

@limingxuan-pku @tcal-x yes that should work and I will follow-up soon with reference documentation to outline all the exact details and how to do this!

@limingxuan-pku the set of changeable parameters is determined by our top Vex generation script at https://github.com/google/CFU-Playground/blob/main/soc/vexriscv/src/main/scala/vexriscv/GenCoreDefault.scala.  If you are willing to start modifying that script, you are really unlimited in how you can modify the VexRiscv.

AXI
@limingxuan-pku I haven't really looked into this; but both LiteX and VexRiscv support AXI, so it should be possible to switch to an AXI-based SoC.   I don't think I'll have time to investigate this in the immediate future, howerver.
Possibly you could use a Wishbone-AXI bridge to attach your accelerator to the current Wishbone bus?

I did a quick experiment and found it is easy to switch the SoC to using an AXI bus.

That's really a good news! In a long run, I will try to attach my accelerator to the bus. Right now, I am going to focus on the development of CFU. Thanks for your reply and patience! :)


VS Code pre-release does not reload code on restart of the debugger

Discussed in https://github.com/microsoft/vscode-python/discussions/19600

Originally posted by Emergency5779 August  1, 2022
All of a sudden VS Code (preview) does not load changes made to the code, when restarting the debugger.
I have spent hours googling, but I have come up empty-handed. Can anyone enlighten me on what is going on. Is there a new config option to set whether the code is to be reloaded when restarting the debugger?
If I stop the execution of the script, and run the debugger again, the latest code changes are loaded.

@roblourens has something changed in Insiders that could explain this?

I hope it's related: Having the same issue on two different computers. Clicking restart would normally save everything and restarts with the changed code, but now it only restarts. Same with Java projects as well. Happens if I run both as Start Debugging and Run Without Debugging.

Same issue here, when debugging Python code clicking restart like @kuroodo shows used to auto-save my files now I need to manually save them before debugging it's a huge pain (mostly because I never had to do this so I never remember).
Version: 1.70.0 (user setup)
Commit: da76f93349a72022ca4670c1b84860304616aaa2
Date: 2022-08-04T04:38:16.462Z
Electron: 18.3.5
Chromium: 100.0.4896.160
Node.js: 16.13.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22000

@roblourens
I can easily reproduce this with js-debug in VS Code 1.70:

use this hello.js: setInterval(() => console.log("hello world"), 1000);

and create a node.js launch config for hello.js
set "Auto Save" setting to "off"
make sure to have "Save Before Start" enabled
F5
modify the "hello world" string
press Restart button

Observe:
hello.js is not saved; program logs the old message.
This problem does not occur in VS Code 1.70

Even reverting the fix for #149885 would be an option for the next recovery release because I suspect that more users are affected by the new problem than by #149885.

Update: My release version just updated, and now doesn't autosave before debug either.

Work around
Install multi-command extension
Then replace the hotkey for Debug: Restart in keybindings.json, in my case:
    {
        "key": "ctrl+shift+f5",
        "command": "extension.multiCommand.execute",
        "args":
        {
            "command": "multiCommand.restartDebug",
            "interval": 30,
            "sequence": [
                "saveAll",
                "workbench.action.debug.restart",
            ]
        }
    }

@weinand could you verify that this is fixed in insiders?

@andreamah sure

Thanks! Works for me as well


Fix bug when moving mount to a new path contained within it's current path

Fixes #2852

@jedesah Oh, actually, would you mind adding a test of the bug this fixes.

@wemrysi There is a property test that will fail occasionally when this scenario happens. I guess I could copy the test and make it not a property with a set of hardcoded values that would trigger this bug. I wish scalacheck (or another such library) offered a way to specify values that must be tested for the property to be validation.

@jedesah Can you copy the property test into an adjacent unit test with the failing generated values hard-coded?  I like to do that as a general principle when property tests have rare failures, just so it becomes a deterministic failure (at the cost of some redundancy).


Release tracking PR: hashes v0.14.0

In preparation for release add a changlelog entry and bump the version.
I'm not 100% sure that this release is API breaking, dependencies definitely changed. The rest might be only additive but I didn't bother looking exactly because I think its better to bump the minor version and err on the side of caution.
Note the hashes 0.13.0 dependency stays in the dependency graph because of secp, we can update secp after releasing hashes then update the secp dependency in rust-bitcoin thereby removing the hashes v0.13.0 dependency - phew.
Note we are right to release this immediately, the two open PRs (#2337 and #2541) that touch hashes only add a clippy attribute so can safely be ignored.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.913%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
19873

Relevant Lines:
23683

💛  - Coveralls

I tested this locally by doing:

Patch the manifest in secp to use this branch (keeping range dependency).
Bumped the secp version and then depended on the bumped version in rust-bitcoin on this branch to check it gets rid of duplicate hashes and hex deps.

Like its my job :)

Yep, #2384 at least is breaking, so we need to do a major bump.

Cool, thanks.

Fixed up the CHANGELOG, I had missed half the URLs - face palm.

My local CI is failing with some bizarre "file not found" from cargo doc. I can't reproduce outside of my CI environment. Investigating. But utACK.

it-list-files @
Cargo-minimal.lock
Cargo-recent.lock
base58/Cargo.toml
bitcoin/Cargo.toml
contrib/run_task.sh
hashes/CHANGELOG.md
hashes/Cargo.toml

nothing new or unusual there

I think what's happening is that cargo doc is tripping over itself trying to produce docs for both versions of bitcoin_hashes at once. I don't know why I can't reproduce this locally though and why it doesn't happen more often.

I added -j1 to the cargo doc thing and this fixed it. I'm not sure where the issue tracker is for cargo-doc.

In my environment the docs output from building different versions of a crate clobber each other so that might be happening while building with multiple threads? Not sure if that's a feature or a bug.

Tagged and published (hashes does not depend on internals so we don't need to do a release there)


resolvers management

I was wondering if you guys were thinking about how resolvers will be managed as the Namespaces grow in number and in size?
I was thinking you could do something like...

function generateRootQuery(Query) {
 return {
    Query
  };
}

function author(root, args){
      return Authors.findOne({
        firstName: args.firstName,
        lastName: args.lastName
      }, {
        fields: {
          'firstName': 1,
          'lastName': 1
        }
      });
    }
}

export const rootQuery = generateRootQuery({ author });

This would work well becuse you could then separate the query resolvers into modules, then do a rootQuery export.
Then you could combine resolvers as well
function rootResolver(Query, Types) {
   return Object.assign({}, {Query}, {...Types});
}

// used like

import rootQuery from '/imports/data/rootQuery';
import Author from '/imports/data/resolvers/Author';

export default rootResolver(rootQuery, {Author});

I dont know something like that?

Composing resolve functions is a great idea! Sashko and I have been toying with the idea a little bit, but I haven't implemented anything for it.
Composing resolvers in the way you propose is actually already possible right now, so I think it would just be a matter of making an 'official' recommendation of how people should do it.
Just one question: I don't really see any namespacing in your examples. Maybe we could add that with a simple function where you can give both the field and the resolve function a prefix or something?
Ideally, we'd like to be able to compose both type definitions and resolve functions, such that every type would be 'portable' and could be plugged into any schema.

Could I see a sketch of this?

Ideally, we'd like to be able to compose both type definitions and resolve functions, such that every type would be 'portable' and could be plugged into any schema.

Yes the above could already be done today and could be just shipped as utilities.

Should we move this to https://github.com/apollostack/graphql-tools? Seems more relevant to the server component!

yes def!


⚠ ClamAV has degraded performance

In 95b1313, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 86 ms

Resolved: ClamAV performance has improved in d23d551.


Image simulations of galaxies with high-resolution SEDs

Image simulations of galaxies with high-resolution SEDs
Use catalogs from lsstdesc-diffsky and DSPS to build image simulations featuring galaxies with high-resolution spectral energy distributions.
Contacts: @sidneymau
Day/Time: TBD
Main communication channel: #desc-wl-shear-color
GitHub repo: TBD (maybe https://github.com/LSSTDESC/chromatic-shear-bias)
Zoom room (if applicable): TBD
Goals and deliverable
Goals:

Learn how to use the lsstdesc-diffsky catalogs, including DSPS to build high resolution SEDs for appropriate galaxy catalogs
Create GalSim objects with these SEDs and other galaxy params
Draw images using GalSim of these objects

Deliverable:

Multiband image simulations of galaxies with high-resolution SEDs (e.g., g, r, i, images of a scene of simple exponential galaxies with high-resolution SEDs placed on a grid)

Resources and skills needed

NERSC access
GalSim

Detailed description
If the above is completed early, then I will propose a stretch goal of running a small campaign of image simulations and running metadetect to measure the multiplicative bias.

Hi Sidney, I worked with Andrew Hearin and the PZ group in creating an interface in RAIL to generate SEDs with DSPS. I might be of help


Can't see wifi

I ran through all of these steps and everything went through without issue. However, I'm unable to see the access point. When I open the serial connection to the device it looks like this:

I had the same issue and this was kinda annoying to figure out. Here is what I did to solve the issue. First if you are plugging the Cactus into a Windows machine, make sure that the drivers installed correctly (should say LilyPad something). Or the better option is plugging it into a power bank, it won't break it and it will not allow the Cactus to push a payload to a machine. Next you have to remember if you activated the auto payload function. If you did, then the access point is not going to be open properly until the payload is finished (sometimes it is 10 seconds after it is finished). If the auto payload is long, then you are going to have to wait it out. Next if you still can't see the access point from the Cactus, go to Add A Network on the device you want to connect to the Cactus. Type in the SSID and the WPA2 password. I have to add the network anytime I want to connect to my Cactus for some reason, none of my devices can see it until I point the device to the AP. I hope this helps your current situation. Good luck.
-theman23290


Usage of STOREFRONT_PROXY_URL

Heyho, im not sure but it seems like the env variable STOREFRONT_PROXY_URL is not used at all. Everywhere is called PROXY_URL.

good catch. do you want to open a PR?

Sure, should I change it all to STOREFRONT_PROXY_URL? Shopware is using PROXY_URL at the moment. Or should I change it to STOREFRONT_PROXY_URL here and in shopware/shopware?

i would name it PROXY_URL as it is in shopware itself. changing the env var is annoying 😅


Make adapter ready for Admin v3

For Admin3 some changes are needed: see http://forum.iobroker.net/viewtopic.php?f=24&t=10268

This issue was moved to iobroker-community-adapters/ioBroker.hid-community#1


Ensure codecov is run for master builds

Resolves https://github.com/ascoderu/opwen-cloudserver/issues/232

Codecov Report

Merging #234 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #234   +/-   ##
=======================================
  Coverage   96.78%   96.78%
=======================================
  Files          20       20
  Lines         996      996
=======================================
  Hits          964      964
  Misses         32       32

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d79b79a...56a2710. Read the comment docs.


Add Server Usage data

How busy are the servers?
https://cepdnaclk.github.io/maintenance/reports/server-gpu-util/plots/co/

@gihanjayatilaka Considering the availability of historical data, will it be possible to store the data in JSON format there like the below format? (This is the bare minimum, can accommodate any other changes too) We can use ChartJS to easily render the data in frontend
https://cepdnaclk.github.io/maintenance/reports/server-gpu-util/plots/{server}/{year}_{weekNo}.json
{
  "title": "Kepler GPU Utilization",
  "labels": [ "2023-01-01", "2023-01-02" ],
  "datasets": [
    {
      "label": "Server Utilization (%)",
      "data": [ 10, 35]
    }, {
      "label": "Memory used (MiB)",
      "data": [ 50, 456]
    }
  ]
}

Hey, just add a link to this webpage for now and create a new issue for the JSON approach.


Add operation_mode to KPOps config

The user should be able to determine KPOps's "mode".
KPOps should run in two modes:

Argo: The commands (deploy, destroy, reset, and clean) will only manifest Kubernetes resources. See #536
Manifest: only manifest the Helm charts
Default: How so far KPOps is handling the pipeline components

This can be passed the commands as an option --operation-mode or be configured through the config.yaml file

Closed with #541


write_release(release=Sys.time()) assumes that all Codera data engineers will work in GMT+2

query_params$release <- format(Sys.time(), "%Y-%m-%dT%H:%M:%S") assumes that the data engineer or computer that loads data into EconData will be set at GMT+2. Please edit so that the time zone is taken into account.
As an aside, I see that release has a slightly different meaning to the old tstamp. I take it that we will try to have small differences in time between our suppliers' publication and our load, when we don't specify the release parameter in write_release() (auto-etl/utils/cli_args.R doesn't have a release parameter for the automation), but the definition of this timestamp is clearly different when we retrospectively set the parameter.
The else part of the definition of release could be treated as a failsafe if we record tabular metadata for the supplier release timestamps (which calendar-releases can help with).

I have changed the code to
query_params$release <- format(Sys.time(),
  format = "%Y-%m-%dT%H:%M:%S",
  tz = "Africa/Johannesburg")

Which will account for the possibility that the person releasing the data is not using SAST.

We discussed the second part of this issue in a video call now:
The Oversight Dashboard will be inaccurate if a release is delayed by more than the period (assuming the latest period is not released), and the release parameter is not specified manually.

@byrongibby can you please open this issue again, because the timestamps in release for today were two hours ahead than what would have been expected (they were past 16:20 when I checked before it was 16:00, and our meeting was at 15:00). I tested
format(Sys.time(),
  format = "%Y-%m-%dT%H:%M:%S",
  tz = "Africa/Johannesburg")

in a terminal on my computer, and it seems to work fine, so @jacques-rs is the locale on your computer South Africa?
@jacques-rs please review my temporary branch auto-etl/tree/release and merge! My proposal provides some straightforward code to use the calendar start time as the release timestamp, so this whole issue of specifying the release parameter in auto-etl is automatically fixed.


Object type message deserialize issue

I have a message contract like
public class xxx
{
public object Payload { get; set; }
public Type PayloadType { get; set; }
}
which Payload is  a PayloadType type class instance.
At consumer side, when I opened ConsumeContext.Message.Payload, I got a wrong type 'JObject' instead of PayloadType .
And in my testing code
JsonConvert.DeserializeObject(message.Payload.ToString(), message.PayloadType);
solved the issue.
MT version: 3.5.7
VS version: 2013 ultimate update 5
.net version: .net framework 4.5.2

This is by design, unless you register your own JsonConvert for your object type.
MassTransit isn't designed to transport Object, by design.

Understood. Thanks for your reply :)


Update SafeMutex to implement IDisposable

The wrapped Mutex / CMutex class in SafeMutex is IDisposable, therefore SafeMutex should follow suit.

Can you remove the commented-out code, and change this to merge into dev instead of master?
Thanks!

Done


added nodepool loop var and -out param for terraform plan

added loop var for clarity and consistency, added -out for terraform plan to fix https://github.com/samsung-cnct/k2/issues/689

/lgtm. CI failed during e2e and we are still working to disable this stage for now. Therefore merging.


jsonrpcclient v4 causes some breaking changes

Since the jsonrpcclient v4 has some breaking changes the pycspr library needs to be updated.
Breaking changes:
https://composed.blog/jsonrpcclient-4-changes
I don't have the time right now to update all files but this is one example PR

How should I configure my localhost Casper network so the next error would be fixed when running pytest?


feat: bump runner version

There is a new runner version out so this bumps the runner
https://github.com/actions-runner-controller/actions-runner-controller/issues/680 also removes the deprecated env var entirely

@mumoshu Need help to merge this. :)


Fix set wrong month. Issue #14

When you try select date from next month, and next month do not contain selected date, it will set incorrect month value
#14

Thanks for this, sorry for the delay. Will release a new version with the changes.


refactor X-Forwarded-Proto

What does this PR do?
This code makes the X-Forwarded-Proto code path a tiny bit faster, more robust, and IMO more readable.
Motivation
I looked at the code and it took me a while to understand it, so I tried making it easier to understand
Additional Notes
The unified diff is pretty useless here, look at a split diff instead.

Thanks for your PR!
I agree with the refactoring of this part of the code, but one needs to understand that it changes the current behavior.
If you have another reverse-proxy in front of Traefik that configures X-Forwarded-Proto with "http" or "https" even if it's a websocket request, today, Traefik will change the X-Forwarded-Proto to "wss" or "ws", but with your change, the X-Forwarded-Proto would stay unchanged.
WDYT @SantoDE @jbdoumenjou @mpl ?

Yes, that's true. I didn't think of that when writing the description for the PR, definitely worth mentioning.
In my opinion, a proxy that has another proxy in front of it should not overwrite the X-Forwarded-* headers written by the first proxy. For all the other X-Forwarded-* headers, this is already done, and for the X-Forwarded-Proto header, it is also done, as long as it's not a websocket request. Special casing websocket requests here seems wrong to me.

I agree with your argument @jcgruenhage. I think, if there is another proxy in front which might to termination and sets the headers, the second proxy should not overwrite.
So I agree.

As long as the headers are put from a trusted source (or that we don't care), it seems logical to me not to overwrite them. So I also agree.
And I also think that if we apply it, we should clearly inform users of this "new" behavior.

@jbdoumenjou Anything I need to do for that? Include something for the changelog in my PR maybe?

@jcgruenhage , I am not sure, if we consider it as a bug fix, we have nothing to do.
@SantoDE @juliens do we considered this PR as a bugfix? If we do so, we have to rebase it to the v2.2 branch ;)

Oh, I'm not really sure on that. I could live with both to be honest. Wdyt @juliens?


No headtracking in full screen mode in VR browsers (OSX)

Test case:
http://destroy.vizor.io/fthr/elevator6/edit

Tested both FF nightly and Chromium VR.
Old Vizor version does work correctly, and so does mozvr.com so the problem is in the three build.


Performance metrics on intial synchronization and resynchronization processes

As we test and configure our Metadb Test instance, which is currently ingesting data from our FOLIO Test instance, we are wondering what performance metrics to expect for our synchronization and resynchronization procedures. Addition an example to the documentation on metadb at https://metadb.dev/doc/ showing estimated initial sync and resync times given X associated data volume would be very helpful.

I think there are too many variables to estimate this.  It might be possible very roughly as a DevOps activity using a similar source database, but I think it would be out of scope for this documentation.

@hutchinsonca


Posting new comments programmatically

Hello. I'm working on project with real-time commenting section using your library. So when one client posts a new comment via UI, I need to post this comment to other clients programmatically (using SignalR). Tell me, please, is it possible? I'm beginner in JavaScript and jQuery so I would be grateful for a little hint.
P.S. I've tried to add comment data manually with html, but it doesn't support all functionality.

This feature is not supported out-of-the-box. You can, however, copy the functionality from the postComment function:
'''
var commentModel = self.createCommentModel(commentJSON);
self.addCommentToDataModel(commentModel);
self.addComment(commentModel);
'''
self being the jquery-comments instance. You can get reference to this instance with jquery as follows
'''
$('.jquery-comments').data('comments')
'''

Thank you!


KAFKA-2837 Follow-up: Default max block to 60 seconds

Ping @ijuma for reviews.

LGTM

Merged to trunk.


Breadcrumbs don't work on file revisions

GitLens Version: 9.7.1

VSCode Version: 1.33.1
OS Version: Ubuntu 18.04

Steps to Reproduce:

breadcrumb crashes on file compare

Opened https://github.com/microsoft/vscode/issues/73594

Breadcrumb now mostly work for file revisions although I may work on removing the full path in a later GitLens version

I still having an issue related to breadcrumbs when I have GitLens enabled. I have to disable, reopen the file and the breadcrumbs will be working, but at least for me it's not possible to have both breadcrumbs and gitlens working together :(

@eb-jorgemaroto please open a new issue with more details (fill out the template) about what issues you are experiencing with breakcrumbs?


Provide documentations for K8S at the edge using K3s

This PR is presently blocked due to inability to accept connection on 514 for both TCP and UDP in single node K8s clusters


Support spread expressions for sets and arrays

I'll also add an update to the language docs in this pr.