Propagate gltf extras during object import to node->meta

This is useful for custom tagging objects with properties (for example in Blender) and having this available in the editor for scripting.
The current GLTF importer parses the extras section however does not use the values except for very specific ones. Add logic to propagate the parsed values all the way to the resulting Node.
It imports extras attached to nodes , meshes and materials (all in GLTF sense of the word)
From blender point of view, it will import custom properties attached to "Blender objects" (Mesh, Camera, Light, ...), "Blender mesh datablocks" and Materials.

Closes: https://github.com/godotengine/godot-proposals/issues/8271

Please open a proposal to discuss the need and use cases for this feature

Thanks, looks like there is one in https://github.com/godotengine/godot-proposals/issues/8271, so I'll try to reuse that one

I tested it with a sample scene and overall it works well but I noticed a few things:

materials don't get metadata
blender mesh objects with -colonly and -convcolonly lose the metadata (in my workaround I assign it to the resulting StaticBody3D)
metadata in mesh datablocks doesn't end in the corresponding ArrayMesh resource. It seems to be added to every MeshInstance3D using the mesh. This could create collisions if a meshinstance and its assigned mesh happen to have a custom property with the same name

We also worked on a similar pull request with meta here https://github.com/godotengine/godot/pull/39024

@noidexe I've uploaded new version

GLTF material extras gets set in Godot Material
collision node extras get set on StaticBody3D (does not support collider mesh extra CollisionShape3D)
MeshInstance3D gets the node extras and Mesh gets the mesh extra properties

Part of my goal is that meta import and export is symmetrical for gltf. Let me know if thats too much or you agree.

I was thinking the same thing. I wanted to add a test that creates a scene, serializes it to gltf, deserializes it and checks it has the right meta, but the fact the mesh re-generation is happening in ResourceImporterScene makes testing it hairy.
I also don't understand github too much, so thought I was not including it in this PR, but somehow my in-progress stuff got attached - ignore it for now, I'll try to figure out.

I feel bad for having you review this, apologies. I meant to say in my last comment that it needs more work - especially the test change, which I did not want to attach at all.
I guess I should mark it as draft?

I've added both import and export as discussed in https://github.com/godotengine/godot/pull/86183#issuecomment-1859073182 and added a test it works as expected.
I haven't found any import/export testing to take inspiration from - it might be my unfamiliarity with Godot, but the test has to instantiate way more of Godot that I would like - the whole import process is extremely intertwined with Editor/Engine (and their subsequent dependencies).
It's also not possible to do this in-memory, so a number of files (.gltf, .bin, .scn) get written and subsequently read from disk during the test run, which I'm not happy about.
Please take a look, cheers

@demolke Please review the CI failure and resolve all issues until the CI turns green. Only then is it ready for human review.

It's also not possible to do this in-memory, so a number of files (.gltf, .bin, .scn) get written and subsequently read from disk during the test run, which I'm not happy about.

It should be possible to append from buffer and generate_buffer if you want to go down that rabbit hole.

You need to delete your nodes at the end of the test, or they'll leak in this case, I suspect

It seems that there's a threading conflict with the embree raycaster, which happens in the importer, unsure how to fix this here, maybe we can turn it off for now in the test, but it could also indicate an issue in the code there

Now solved for this PR at least, please open an issue tracking this to ensure we don't forget about it, worth also investing upstream to see if this has been reported with embree directly

It's also not possible to do this in-memory, so a number of files (.gltf, .bin, .scn) get written and subsequently read from disk during the test run, which I'm not happy about.

It should be possible to append from buffer and generate_buffer if you want to go down that rabbit hole.

Unfortunately asserting the GLTFState is not enough - there are conversion steps that happen in editor/import/resource_importer_scene.cpp that are needed to verify it works correctly (when talking about meta/extras correctness), so one has to do real import.

Now solved for this PR at least, please open an issue tracking this to ensure we don't forget about it, worth also investing upstream to see if this has been reported with embree directly

Yes, I disabled lod generation :). I'll open an issue.

Filed https://github.com/godotengine/godot/issues/86306 for the LOD lock-inversion

In the future please apply suggestions as a batch, there's an option for it under the suggestion, to not make so many commits 🙂

Done, thanks a lot for the review.
I feel like most of the comments could have been prevented by having a much more opinionated clang-format and clang-tidy configuration (e.g. https://clang.llvm.org/extra/clang-tidy/checks/readability/identifier-naming.html#cmdoption-arg-PointerParameterPrefix). I'm used to not caring about C++ style at all since the presubmit will complain about it to me.

We don't use clang tidy like this, so it's up to everyone to just get used to the style, especially after looking at and recieving one review 🙂, we don't want to rely too much on format tools as they can mess with things and be confusing (and often they don't help unless you've set them up right, many PRs have whitespace issues despite them being in the format checks, so it's no guarantee anyway)
There's a balance between learning the style and formatting tools

There is probably a better space for this discussion, but I'm not arguing to reformat the whole codebase - there is clang-format-diff and clang-tidy-diff.
The style guide literally says
There are no written guidelines, but the code style agreed upon by the developers is enforced via the clang-format code beautifier, which takes care for you of all our conventions
Maybe it should say: some of our conventions :)
Yup, I had clang format errors until the github action ran the clang-format, but since I've been pretty well-behaved at running it locally :)

It also says:

Some of them are checked via the Continuous Integration process and reviewers will ask you to fix potential issues

So yes, discussion for elsewhere, but it's no problem, and it's a learning process and we simply can't enforce it all, we also don't run clang-tidy as part of the CI, as I said
So while it can improve, I don't think this is an area where much change will happen for the above stated reasons 🙂
In any case, feel free to discuss this in the chat
But also no need to feel bad or frustrated over reviews and not picking up on the style, it's a learning experience 🙂

@demolke this is awesome! Thanks for all the effort, cant wait for this to get merged <3

As per my previous pull requests, I agree that aligning gltf extras with node meta is a good idea. https://github.com/godotengine/godot/pull/40474 -- Maintainer review

How does this handle metadata attached to gltf nodes which are interpreted as skeleton bones?
Should it make a per-bone dictionary in meta that holds data per bone? Is there a naming convention for such metadata to prevent conflict with user-created metadata?
since it sounds like the main argument in favor of the proposal is preserving metadata from blender, I would like to copy how blender does these things and make sure we do not lose this data.
The other thing I would like to double-check is how data is preserved from the ImporterMeshInstnace3D into the MeshInstance3D after conversion..
One other case that is complicated is collision shapes such as -col since those are replaced outside the import system.
And on top of that there is the Advanced Import dialog, where nodes can be set as collision shapes. for some reason this is handled with a completely different codepath from -col, so this case also needs to be tested.
In an ideal world, there would be a way to guarantee 1-to-1 mapping between gltf nodes and godot nodes so we wouldn't have corner cases everywhere, but in reality this has all gotten quite complicated.

This part of the code

takes care of the MeshInstance3D conversion. You can see how it works in the test.
Bones in Godot are not nodes, so they can't have Node->meta and I consider it out of scope for this PR. The skeleton is a node and any extras set on nodes work.

I've added the bone functionality here https://github.com/godotengine/godot/pull/87150, however since it's changing the Skeleton3D datastructure it's probably better to have a separate discussion about the pros/cons there.

It would be great to get this change in... One of the things that has been holding me back is that there is no way to turn this feature off.
I would feel a lot nervous about this change if it were optional (gltf/convert_extras_to_meta importer and exporter option or something like that)... it might also be good to have it off by default to be on the safe side, but I don't have strong opinions about this.

I've added the option, but I think it should default to on - otherwise we won't get any feedback on it.

It should probably also be only a temporary option and if it's working as expected, it should get removed couple of versions down.

Thanks for taking a look, I've updated it except I did not call it gltf_extras, but just extras. If in the future another im/exporter wants to do a similar thing, it should just act on extras as well and not have it tied to the importer name.

@demolke: Thanks for taking a look, I've updated it except I did not call it gltf_extras, but just extras.

After thinking about it more, I agree that this is a good idea. If someone uses the .blend importer and that generates extras, the fact that the .blend file importer uses GLTF under the hood is an implementation detail, and the user ideally shouldn't know. So yeah, I like this PR as-is.

Hmm, I am worried extras is not something I think most people will understand. (It is a technical term in the gltf spec...)
Is there a better term to indicate it is imported? Or is extras ok

Hmm, I am worried extras is not something I think most people will understand. (It is a technical term in the gltf spec...) Is there a better term to indicate it is imported? Or is extras ok

In absence of other suggestions, I think extras is okay. We can't just call it imported_extras since they also get exported. ported_extras :).
The only other one I can see working is Imported/exported metadata, but it's quite long and since the main usage of this is from code it should be typo proof, so I would stick with extras.

I do like that extras is symmetrical on import and export.

https://github.com/godotengine/godot-proposals/issues/8271
https://github.com/godotengine/godot/pull/86183
@demolke
I think there should be settings for this. Here is a suggestion with two available flags:

One for enabling and disabling the copying of metadata from glTF files to nodes, materials, etc. (Remember, this should not interfere with manually defined metadata).

Another for automatically identifying properties recognized by Godot and applying them (using key and value) automatically where that metadata is defined. For example, if a light has "omni_range" as a custom property in Blender, in Godot this would be applied as the default value (still allowing manual changes), which would also work for "albedo_color" in materials, etc.

I am working on a proof of concept that explores this idea, but it could become a native feature in Godot if well-defined. I think these flags are really important because not everyone will make use of this feature (not everyone needs metadata to be stored).
godot-gltf-extras
Something interesting to mention is that properties applied automatically (such as uv1_scale, albedo_color) open up greater possibilities for Blender shaders to be "understood" by Godot, or conversely, for creating shaders in Blender that use custom property values as input (drivers) to simulate how Godot would handle certain data.
An example is 'Triplanar UVs,' which Godot does not inherently follow the same way a shader works in Blender, but this can be resolved with drivers, Blender's custom properties, and automatically applying metadata values.

@lgxm3z That will not be done. Extras will not be applied automatically. We already have a structured way to import lights, it's called KHR_lights_punctual. This is the data that will be applied automatically, not extras.
Please understand that the goal here goes beyond just making Godot understand Blender, or tailoring a model in Blender for Godot. The goal is to allow any GLTF file to come from anywhere and go anywhere. So you could not only use Blender, but any tool, and those assets could not only go to Godot, but any engine. The only way to do this is to standardize what that data looks like, which is what KHR_lights_punctual is for.

@lgxm3z That will not be done. Extras will not be applied automatically. We already have a structured way to import lights, it's called KHR_lights_punctual. This is the data that will be applied automatically, not extras.
Please understand that the goal here goes beyond just making Godot understand Blender, or tailoring a model in Blender for Godot. The goal is to allow any GLTF file to come from anywhere and go anywhere. So you could not only use Blender, but any tool, and those assets could not only go to Godot, but any engine. The only way to do this is to standardize what that data looks like, which is what KHR_lights_punctual and other standards are for.

I understand what you mean, but from my perspective, the glTF format provides the possibility to include extra properties, and it is the engine's job to read this format and import it in the best possible way. Godot already does things like transforming specific names into certain nodes (e.g., -vehicle to VehicleWheel3D, -convcolonly to ConvexPolygonShape3D).
https://docs.godotengine.org/en/4.1/tutorials/assets_pipeline/importing_scenes.html#create-collisions-col-convcol-colonly-convcolonly
Not only Blender but any program can export such extra properties in a glTF file. From my perspective, it would be interesting for Godot to offer the possibility of using this to its advantage since it treats each property uniquely. However, I understand that this is a sensitive decision and may diverge from the principles that Godot upholds.

@lgxm3z Godot already does things like transforming specific names into certain nodes (e.g., -vehicle to VehicleWheel3D, -convcolonly to ConvexPolygonShape3D).

These are existing hacks that were "grandfathered in". Node name suffixes are not as flexible, and they're not standardized across engines, and they aren't something we want to add more of in the future. They aren't actually even part of Godot's GLTF code, they exist in the scene importer in general, for all scene-imported file types including GLTF, FBX, Collada, etc.

@lgxm3z Godot already does things like transforming specific names into certain nodes (e.g., -vehicle to VehicleWheel3D, -convcolonly to ConvexPolygonShape3D).

These are existing hacks that were "grandfathered in". Node name suffixes are not as flexible, and they're not standardized across engines, and they aren't something we want to add more of in the future. They aren't actually even part of Godot's GLTF code, they exist in the scene importer in general, for all scene-imported file types including GLTF, FBX, Collada, etc.

Got it, and thanks for the explanation. I didn't know that Godot had 'changed its view' regarding these 'hacks'. It makes a lot of sense not to continue with these practices. Initially, I saw potential in this type of solution, but now I can see the problems it could cause and how certain features could be difficult to remove later.

Sorry i'm a beginner in all of this (github/source code) i want to try out this feature so i should probably compile godot from source but i still have to wait until it's merged right ?

See here for instructions

Should we also add it to skins, animations and textures?

In general I think it makes sense to import/export extras everywhere, but it doesn't need to happen all at once. If we miss anything, we can add it in later in another PR. So we could add it in this PR, or not, I'm good either way.

Needs rebase, I tried building it locally and it failed compiling the tests with:
In file included from ./core/templates/local_vector.h:35,
                 from ./core/object/message_queue.h:36,
                 from ./core/object/object.h:35,
                 from ./core/variant/binder_common.h:35,
                 from ./core/object/method_bind.h:34,
                 from ./core/object/class_db.h:34,
                 from ./editor/editor_paths.h:34,
                 from tests/test_main.cpp:36:
./modules/gltf/tests/test_gltf_extras.h: In function 'Node* TestGltfExtras::_gltf_export_then_import(Node*, String&)':
./modules/gltf/tests/test_gltf_extras.h:62: error: no matching function for call to 'ResourceImporterScene::ResourceImporterScene(bool, bool)'
   62 |         Ref<ResourceImporterScene> import_scene = memnew(ResourceImporterScene(false, true));
./core/os/memory.h:125:51: note: in definition of macro 'memnew'
  125 | #define memnew(m_class) _post_initialize(new ("") m_class)
      |                                                   ^~~~~~~
In file included from ./modules/gltf/tests/test_gltf_extras.h:39,
                 from ./modules/modules_tests.gen.h:42,
                 from tests/test_main.cpp:162:
./editor/import/3d/resource_importer_scene.h:309: note: candidate: 'ResourceImporterScene::ResourceImporterScene(const String&, bool)'
  309 |         ResourceImporterScene(const String &p_scene_import_type = "PackedScene", bool p_singleton = false);
./editor/import/3d/resource_importer_scene.h:309: note:   no known conversion for argument 1 from 'bool' to 'const String&'

(implies building with scons tests=yes)

Needs rebase, I tried building it locally and it failed compiling the tests with:

Done, thanks for taking a look.

Finally, I'm confused which objects support extras. In glTF, just about anything can have extras. In this PR, it's implemented for all types of nodes, but only meshes and material resources? Should we also add it to skins, animations and textures? Or should we add it only to nodes, not resources?

I think eventually yes, all the resources should have extras supported - adding it to node vs adding it to mesh (or other resource) are distinctly different use cases. You could put generic information on the "enemy" mesh re-used everywhere and have specific extras on the node.
Once this get submitted I would like to get consensus on the bones PR https://github.com/godotengine/godot/pull/86183#issuecomment-1859073182 and then I can look into the other types.

Thanks!

Great work i've tested it and it just works™ i just have one question, when i import a .blend there is a root node that is made with the name of the .blend, what is this root node in blender ?
I've tried adding custom properties to the : workspace, collection, scene and none of those correspond to the root node.

we force the name of the blend file into the root node name

So it does not correspond to anything on the blender side ? Then i should open a proposal to at least have the root node inherit the custom properties of the scene, because in an exported gltf you already have this data available.

@warcaninos That root node corresponds to the Blender scene itself. Blender scenes allow multiple root nodes, which is incompatible with Godot scenes that have exactly one root node.
The "proper" fix for this is to add a feature to Blender to allow it to optionally flag scenes as having a single root node. This is implemented in Godot here, it just needs Blender to add support for it. Meaning, you should open a proposal to Blender, not a proposal to Godot.

Unless there is something i missunderstood, when you export a gltf scene from godot it doesn't get the metadata as extra.

@warcaninos It only gets metadata named extras, not all metadata.

I've tried metadata with the name "extras" wich automatically get called "Extras" with values of type
string: test - Does not show up in the gltf
dictionary: string: Node, string: MeshInstance3D; Does not show up in the gltf
Array of dictionary with the same dictionary up here and it still does not show up in the gltf.
I've searched in the documentation but since this is a feature that is still in a dev branch there is nothing.

@aaronfranke I'm using 4.4 dev2 import extras does work so i assume exporting should work too.

Should i open an issue about this ?

@warcaninos I can't reproduce the problem. The scene in Godot:

The exported file (formatted with Prettier):

I switched to 4.4 dev 3  and i do not have this problem anymore. Thanks for your time.


Use azure-core-test optimized utilities

Description
Use optimized utility methods in azure-core-test for comparing binary data.
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

/azp run java - storage - tests


Updating to Protobuf 3.14

Hi all! Any chance you'd consider an update to protobuf 3.14?
[The Bazel Intellij/Android Studio plugin currently seems broken on a protobuf bug said to be fixed in 3.14 and lack of rules_proto support for 3.14 is preventing recommending an update to the latest rules_proto.]
Thanks!
Chris
(ex-Googler)
P.S. Friendly heads also that the commit/version hash in the README is way old! Might confuse some newcomers :)

Please update the commit/version hash in the README. Thanks!

Please update the commit/version hash in the README. Thanks!

This PR will updates to the newest protobuf version https://github.com/bazelbuild/rules_proto/pull/88


starting troubles

First off, i wanted to mention how awesome this project idea is and thank you for implementing it. I am just starting off with trying kappa and ran into the following issue when trying to run kappa for the first time. The readme seems to be out of date with the current commandline options. The commandline help doesn't really say what it expects for the "config" placeholder. I tried passing in the file name "kappa.yml" for the config option and ended up with the following error. Any thoughts/suggestions on what could be going wrong here?
(lambdaStats) osboxes@osboxes:~/projects/lambdaStats2> kappa --debug kappa.yml status
Traceback (most recent call last):
File "/home/osboxes/envs/lambdaStats/bin/kappa", line 155, in
cli(obj={})
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/click/core.py", line 664, in call
return self.main(*args, **kwargs)
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/click/core.py", line 644, in main
rv = self.invoke(ctx)
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
return callback(*args, **kwargs)
File "/home/osboxes/envs/lambdaStats/bin/kappa", line 98, in status
status = context.status()
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/kappa/context.py", line 173, in status
status['function'] = self.function.status()
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/kappa/function.py", line 192, in status
LOG.debug('getting status for function %s', self.name)
File "/home/osboxes/envs/lambdaStats/local/lib/python2.7/site-packages/kappa/function.py", line 38, in name
return self._config['name']
KeyError: 'name'

Did you install the latest stable version or the development version? Because in my case, a few days ago, I started off with the stable version, and tried following the ReadMe, but I was always getting errors. When I switched to the development version, then commands would work. The ReadMe is tailored towards the development version only it seems.

@guydaher that was it. the development version works as advertised. I glanced at the version number in the _version file and assumed they were they same... assumptions always get you in trouble :) Thank you. Appreciate it

Haha yes indeed. Glad I could help!

I've made a new release so the docs should match the latest release!


Implement questions up to 9h1

Resolves: #142

Missing CANNABUSE variable for checkbox 12r1e in the API

@ashleybot so the ol' "get to it after I finish up" caught up with me
There is a variable missing from the API that's present in the documentation that I mentioned in a comment. The CANNABUSE

Looks good! Noticed a couple things

[ ] CANNABUSE is missing UI behaviors

[ ] If BEAHALL has a value of 1 ("Yes") BEAHSIMP and BEAHCOMP are not required. I didn't see this mentioned as a requirement here --> https://github.com/naccdata/uniform-data-set/blob/main/forms/uds/b9/form_b9_ivp_questions_and_vars.csv but the question does read in a way that would make me think 12i1 and 12i2 would required a response when 12i is marked as "Yes". Either way, thought it was worth mentioning.

Other than that everything looks good!

@Oddvocado I think you're totally right with that second one, the sub-questions SHOULD be required. I also added a requirement for the pair above those 2 you pointed out as well.

[ ] Added CANNABUSE to the behaviors for BESUBAB. CANNABUSE should disable properly now when BESUABAB is 0 or 9
[ ] BEAHSIMP and BEAHCOMP are required if BEAHALL is 1
[ ] BEVPATT and BEVWELL are required when BEVHALL is 1


[Bug] Llama 405B FP8 causes OOM on 16xA40

Checklist

[X] 1. I have searched related issues but cannot get the expected help.
[X] 2. The bug has not been fixed in the latest version.
[X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
[X] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[X] 5. Please use English, otherwise it will be closed.

Describe the bug
I'm trying to run Llama-405B, FP8, on 4 nodes of 4xA40, each with ~44GB VRAM. Theoretically, speaking, this is plenty of VRAM for 405B, FP8, given that I only need 405GB VRAM total +/- 50GB during inference. However, I keep running into OOM issues, after all safetensors checkpoints have been loaded, which does not seem to make much sense to me.
I've tried to reduce cache, context length, etc, but that does not seem to have too much of an effect on the OOM.
Reproduction
On each of the nodes, I run the following command:
GLOO_SOCKET_IFNAME=eno12399np0 python3 -m sglang.launch_server --model-path meta-llama/Meta-Llama-3.1-405B-Instruct-FP8 --tp 16 --nccl-init-addr <IP_ADDRESS>:20000 --nnodes 4 --node-rank 3 --disable-cuda-graph --kv-cache-dtype fp8_e5m2 --chunked-prefill-size 1024 --mem-fraction-static 0.9 --disable-disk-cache

Environment
I have 4x4xA40 nodes, for distributed inference. They're linked by a 25GBe backbone. All of them have the same environment. Here is one of them detailed below:
Python: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
CUDA available: True
GPU 0,1,2,3: NVIDIA A40
GPU 0,1,2,3 Compute Capability: 8.6
CUDA_HOME: None
PyTorch: 2.4.0+cu121
sglang: 0.3.1
flashinfer: 0.1.6+cu121torch2.4
triton: 3.0.0
transformers: 4.44.2
requests: 2.32.3
tqdm: 4.66.5
numpy: 1.26.4
aiohttp: 3.10.5
fastapi: 0.114.2
hf_transfer: 0.1.8
huggingface_hub: 0.24.7
interegular: 0.3.3
packaging: 24.1
PIL: 10.4.0
psutil: 6.0.0
pydantic: 2.9.1
uvicorn: 0.30.6
uvloop: 0.20.0
zmq: 26.2.0
vllm: 0.5.5
multipart: 0.0.9
openai: 1.45.1
anthropic: 0.34.2
NVIDIA Topology:
        GPU0    GPU1    GPU2    GPU3    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      NV4     SYS     SYS     0,2,4,6,8,10    0               N/A
GPU1    NV4      X      SYS     SYS     0,2,4,6,8,10    0               N/A
GPU2    SYS     SYS      X      NV4     1,3,5,7,9,11    1               N/A
GPU3    SYS     SYS     NV4      X      1,3,5,7,9,11    1               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

ulimit soft: 1024

reduce --mem-fraction-static 0.9 ?

could you show me the full log?
reduce --mem-fraction-static 0.9 to prevent OOM instead of increasing it. The default value is 0.8 in this case.

I have same issue with DeepSeek-V2 and 4xL40S. I have 192GB VRAM Total and it crash on OOM during the load. I see it will use only one GPU. ExllamaV2 can utilize it much better than lVVM. I think lVVM is buggy.

see https://sgl-project.github.io/references/troubleshooting.html#cuda-out-of-memory


jquery plugins should not be declared inside $(document).ready(...)

For example, the $.fn.material_select() plugin is defined within a call to $.ready().
I will get an error if I try to execute the following:
...
<body>
    ...
    <script src="js/materialize.js"></script>
    <script>
        $("select").material_select();  // <--- ERROR: material_select not found
    </script>
</body>

This is bad.
All the materialize plugins should be defined once the materialize.js script has been parsed.

We'll look into moving our plugins. For now though, if you wrap your initializations within document ready's it should always work.

Fixed in e07cf3239d327140a8ee69917cc0cc1cd86e0803


Fixing incorrect ref counts

This PR fixes an issue where the generated QIR resulted in accessing released memory when arrays of pointers are updated in place and the updated item value is constructed in an inner scope.
Longer explanation looking at the test case:
function TestRefCounts(cond : Bool) : Unit {
    mutable ops = new Int[][5];
    if (cond) {
        set ops w/= 0 <- new Int[3];
    }
}

The first line gets translated into first creating and then populating the Int[][], After creation and populating it, the array and all its items have ref count 1. Then that array is assigned to the mutable variable. This assignment leads to the array and all its items having ref count 2, and an alias count 1.
Elaborating more on why the ref count needs to be 2 and can't remain 1, think of the case where the array assigned to the mutable variable has been passed in as an argument. Suppose that argument initially has ref count 1 and is "owned" by the calling function. Then if we kept the ref count at 1, and updated an item in ops, the old item's ref count would drop to 0, releasing it. Hence (assuming we can't know which items will be updated), we increase both the alias and the ref count when assigning to mutable variables for the array and all its item.
Continuing into the if-branch, we create a new array that has ref count 1. Upon assigning it to item 0 in the array, its alias count and ref count are increased. At the same time, the alias count and ref count of the old item are decreased. The ops array and all its items now have an alias count 1 and a ref count 2. The alias count of the old item is now 0, and its ref count is 1.
Now we exit the scope. When we exit the scope, the array value created inside the if-branch goes out of scope. Its ref count is hence decreased by 1. This now causes a problem when we exit TestRefCounts; upon exiting, we decrease the alias count and the ref count of ops and all its items by 1. The alias count of ops and all its items is then 0 (ok), and the ref count of all items except item 0 is 1, while the ref count of item 0 is 0.
And here is where things go wrong; we also release the initially created array (variable %0), which is correct, since the value goes out of scope. However, the original item at index 0 is no longer accessible by getting the 0-element pointer of %0 - instead of getting the old item that still has a ref count 1 that needs to be set to 0, we get the new item that has a ref count 0 already.
Now why is it relevant whether the ops array has been copied inside the if-branch or not? Suppose after the first line there is another variable defined that is bound to ops, such that the if-branch actually does create the copy. In that case when we exit TestRefCounts, accessing the item at index 0 of %0 indeed still accesses the old item, and everything works fine.
Whatever scheme we pursue to resolve the issue, it ultimately boils down to that we need to ensure that when %0 goes out of scope, we have a way to access all its items at the time of declaration. Thinking of queuing the unreferencing for each item separately when we increase the ref count for the array and all its items upon the initial assignment to ops, the problem is that iterating over the array means that the items are loaded inside a loop body, and I no longer have access to them when we exit the function. Hence, the simplest and I think cleanest solution I can come up with is that I inject an additional ref count increase for the new item and ref count decrease for the old item when an array item is modified in place.
I should say that the reason why I think the additional count change has to be exactly 1 is that I don't see a way for the array to be unreferenced more than once at the end of the scope while also not being copied; I believe the only way it would be unreferenced multiple times is if there is another alias for it, in which case the copy would get executed and everything should work.

I'd suggest, rather than thinking in terms of scopes, we try to outline the semantics of ownership when copy is involved. Following is the set of statements that I believe would define a correct model:

Arrays (and tuples) own their content with shared semantics, thus:

when arrays' ref count goes to 0 there must be code that would decrease ref-counts of its items
when item in an array is replaced there must be code that decreases ref-count on the items being removed and increases ref-count on the item being added

It might be desirable to elide the copy sometimes, thus have two different behaviours:

when array is actually copied, the new copied array has ref-count of one and owns the content, so there must be code that increases ref-counts of all items
when the copy is elided, the original array's ref-count is incremented but the content's ref-count shouldn't because there is no new owner of the content

Ideally, the runtime (which does the copy elision) would handle the different in behaviour re the ref-counts of the content, but because the runtime doesn't have type information, the task falls onto the Q# compiler while generating QIR.
I can see two possible solutions to this:

The compiler checks the returned pointer from the copy call and if the pointer matches the original array, the compiler infers that the copy was elided and doesn't increment ref-counts of the content.
The compiler checks the condition for copy elision (i.e. the alias count) and either does non-optimized copy (and increments the ref-counts of the content) or proceeds with updating the original array in place.

Personally, I like option 2 more because it makes the split of responsibilities between the compiler and runtime cleaner, but it does mean this extra handling will have to be done for all arrays/tuples whether or not they contain reference types.

Looks like I still need to know too much to understand the logic behind the original @bettinaheim 's explanation. At least looks like the currently existing logic of ref-counting (implemented in Q# compiler/runtime) significantly differs from what I would come up with based on my C/C++ way of thinking.
What @irinayat-MS is writing above seems to me very close to how I imagine the ref-counting.
Here is my understanding/vision in more details:
The original code can be written in any language (Q#, C/C++, whatever) but the same .ll can be generated from the original code regardless of the language. Now I will consider the original code step-by step (to be more precise - I'll consider what I would expect from the .ll generated from that original code), based on the QirArray data type in qsharp-runtime/src/QirRuntime/lib/QIR/qirTypes.hpp. Original:
function TestRefCounts(cond : Bool) : Unit {
    mutable ops = new Int[][5];
    if (cond) {
        set ops w/= 0 <- new Int[3];
    }
}

The fragment new Int[][5] heap-allocates an instance of QirArray and passes to its constructor
5 as a number of items,
and sizeof(void *) as an item size in bytes.
The constructor heap-allocates one more block - a block for 5 pointers - and saves the address of that block in the buffer member variable of the QirArray instance. Those 5 pointers are initialized to null.
The address of the just created instance of QirArray is saved in the .ll's internal variable %0, thus we get one reference to the QirArray instance (that is why the ref-count of this instance is set to 1 by the constructor).
The internal variable %0 is the .ll's abstraction. In real code the internal variable %0 can be implemented as a CPU register (in the particular case) or as a compiler-generated variable allocated on the stack (in the common case or if the value does not fit in the CPU register). Since the stack-allocated entities typically survive until the end of the scope only (until the }), I believe, that .ll assumes, that the internal variable %0 will only be valid until the end of the scope.
Then, during the fragment mutable ops = , the address is copied from the internal variable %0 to the variable ops, and we get 2 references to the same QirArray instance. Thus the ref-count for the QirArray instance is incremented from 1 to 2.
Then the execution enters the if and reaches the fragment new Int[3]. This fragment heap-allocates a second instance of QirArray and passes to its constructor
3 as a number of items,
and sizeof(int64_t) as an item size in bytes.
The constructor heap-allocates one more block - a block for 3 64-bit values - and saves the address of that block in the buffer member variable of the second QirArray instance. Those 3 64-bit values can be left uninitialized (or can be initialized to 0).
The address of the second QirArray instance is saved in the internal variable %1 (and we get one reference to the second QirArray instance), and the second QirArray instance's ref-count is set to 1 (by the constructor).
Then the fragment set ops w/= 0 <- (which in C++ I interpret like ops[0] =) copies the address from internal variable %1 to the ops[0] (to the first-most item of the heap-allocated block for 5 pointers). Thus we get a second reference to the second QirArray instance, and we increment the ref-count for this instance from 1 to 2.
Then the execution reaches the if's end of scope }, where the internal variable %1 goes out of scope and stop being valid (as described earlier for the internal variable %0). That is why the ref-count of the second QirArray instance is decremented from 2 to 1 (and only the ops[0] refers to the second QirArray instance).
Now we get to the second } - the end of the scope of the function TestRefCounts(). All the variables in of this scope - %0 and
ops[] go out of the scope - stop being valid. Since they all might have been allocated on the stack (in the real code in the common case), they go out of the scope (popped from the stack) in the reverse order of allocation. That is why ops[] goes out of the scope first. And the ref-count for the first QirArray instance is decremented from 2 to 1 (now only the %0 refers to the first QirArray instance).
Then the %0 goes out of the scope and the ref-count for the first QirArray instance is decremented from 1 to 0.
To be continues... temporarily saving the result.

For the future:
If the .ll files are written manually then would be nice to see the comments in high-level programming language or pseudocode :-)

For the future:
If the .ll files are written manually then would be nice to see the comments in high-level programming language or pseudocode telling what is going on in the code :-)

Unfortunately the tests currently compare what is in those ll files with what the compiler emits. It would be nice to insert comments into the emitted code, but I don't think I currently have a way to do that with the wrappers as is.

@irinayat-MS
The statements 1 and 2 you phrased are indeed consistent with how the ref counting works, though for the second bullet the implementation while equivalent in behavior currently deviates. Having the ref count change for the full array (meaning all items) inside a conditional block would prevent that these changes can be optimized out, but I would need to think more about whether there is a way to have the code be optimal when we conditionally update the counts. What the code is doing currently is indeed equivalent to that, it just reads slightly differently. I'd prefer to potentially doing further changes in that regard in a separate PR and preferable after we have a full set of integration tests.
@kuzminrobin I believe our conversation last week largely addressed your comment. Let me know if there was something we did not cover. Let me just very briefly post the examples we had talked about here:
function TestRefCounts(cond : Bool, arr : Int[][]) : Unit {
    mutable ops = arr;
    if (cond) {
        set ops w/= 0 <- new Int[3];
    }
    // do something
}

We had considered the case of a caller invoking TestRefCounts when a) the arr argument is assigned to a variable in the caller, b) when the array is created as part of the call, c) what one could do after the call invocation with arr, and what if ops were returned from TestRefCounts.

@kuzminrobin I believe our conversation last week largely addressed
@bettinaheim, my long comment was written before the discussion, sorry, I forgot to mention that.


Add a simple tag/keyword for each node, allowing searching related functions by different names

Feature/enhancement description:
A tag/keyword system on each node to allow searching related functions name from shader or even other programs.
For example, i was searching 'seamless' but there's a node called 'Make Tileable', if there was a keyword on it, it would be found by the keyword. Also for people coming from different tools, searching a node they are familiar and founding the related nodes for that function/action would cause a more seamless transition.
Also while we are at it, if you would implement a simple F1 help popup with some images and infos, for example, i'm not sure what each socket color means. Just a simple infographic/cheatsheet with main functions, symbology, would be great for new comers. Show it at first time and inform that we can press F1 to open it again.
Thank you for the tool

The tag system exists already (try to search "alveolus", for example). You can define your own tags for a node (just right click it in the library panel and select "define aliases").


Implement Day 5 Part 2

Problem description: https://adventofcode.com/2023/day/5#part2

This algorithm feels like it's going to be pretty simple. Just going to create another version of the getMinLocation function with a different loop.

Bahaha, well, I implemented it, but the test is taking forever and a day to run... Yes, on the sample data set. I don't think I'm going to attempt it on the real data set. Probably need to take a look into doing some optimizations, lol.

Oook, got the sample data to pass, and the test suite was only taking like 20ms to run the test on the sample data. But on the real data, it's taking forever...

 I feel like we're already pretty efficient with traversing the linked list structure we've created... The question is whether or not there's a way to do this which doesn't involve looping over every seed...
Well, considering that the seeds are really acting like any of the other nodes, we could turn this into a doubly-linked list and traverse backwards. That gives us the advantage of doing things in order (because we're looking for the lowest possible location values, not really the locations for every seed number). This meets the objective, but it just doesn't feel like it's in-line with the spirit of the problem...
Another approach to the problem could be memoization, though I doubt that would help much. The traversal is already pretty fast, we're just having to do a ton of traversals.
Of course, throwing more threads at it never hurts, but we'll come back to that after we try some other optimizations first.

First things first, let's try alleviating some of the GC work by declaring some reusable variables outside of the loops. The last test run took 9 minutes, so let's see what kind of improvement we get from this.

Ok, simply pre-allocating the variables didn't do much (as expected). We really just need an altogether new algorithm. Let's revisit that idea to walk the list backwards, starting from the minimum location and finding the first matching seed. Does that work?
Yeah, seems like it should work very nicely. Just sort the last MapCollection according to the destination, do a little while loop and walk backwards until a valid seed is found, then return it. We'll need to change our singly-linked list to a doubly-linked list, but that's no problem.

After having the opportunity to get some rest, I'm realizing that walking the tree backwards has one very important flaw. The "locations" represented in the last set of maps doesn't actually represent all possible locations... The actual set of locations could be more or less than the ones represented in the final map (due to the rule that if a new "destination" parameter isn't found for a particular set of maps, then the current "source" will be used as the "destination")...

Ok, after a bit of research, it seems the correct way to approach the problem is actually by mapping ranges instead of individual seed values. It kind of bothers me that I didn't see this from the beginning, as it's clearly staring me right in the face.
Approach: Walk over each of the seeds. Each time you find a match in the following category, make sure you use the "range" to determine what seed to start with on the next iteration. This is going to be a recursive algorithm. As you recurse, you're going to be continually adjusting the range for the previous source.
Termination condition: When you've found the "location" range (last node in our singly-linked list).
What are we returning in our recursive function? The final "destination" parameter found, and the "range" that determines how far to jump forward on the next iteration.


Typo fix

"pod" was being truncated to "po" in the names of the systemd unit files, e.g. pod-systemd-po.service

@gavincampbell the change LGTM, but you need to sign your commit.  We also appreciate a little note in the description if possible.  Please do git commit --amend -s and then repush.
Thanks!

 Then repush.
git push --force

Restarted the flaked tests. Thanks for contributing!

/ok-to-test
/approve

/hold cancel


Question

I'm kindly asking because I've never stumbled upon this problem.
First, why does PHP have the final keyword and why would anyone make their classes as final?
Second, I'd like to know some examples of actual use cases of this package, could you explain in more detail? I'm curious.

Hi @misterabdul,
The final keyword disables inheritance. This package removes the final keyword in specific vendor packages so classes that were previously marked final can be extended.

@mbabker

This issue should be re-opened, or converted to a discussion using the appropriate GitHub feature (presuming it was enabled on this repository).

No. I have answered the question.

You have managed to skirt around answering both questions posed by the OP with a purely philosophical answer.

No. That is objectively false. The answer I provided is exactly what the purpose of the package is. It is absolutely not "philosophical" by any means. The final keyword disables inheritance. This package removes the final keyword in vendor packages you define, so that you may extend classes that were previously marked final. That is literally and objectively its purpose.

So, beyond the philosophical "developers should not be impeded by the use of the final or private keywords in the programming language" position that you seem to think is the One Right Way™ to open-source code

You're making complete of assumptions of my thoughts, and also putting words in my mouth. I have never said or claimed any such thing.

, what is the practical use of a package like this? What are the real-world use cases for a package where creating a fork of the upstream source code to bypass the design decisions of the upstream vendor solves?

You've made your position clear on this, and I do not owe you any further explanation. If you do not have a use for this package, move along.

So, beyond the philosophical "developers should not be impeded by the use of the final or private keywords in the programming language" position that you seem to think is the One Right Way™ to open-source code

You're making complete of assumptions of my thoughts, and also putting words in my mouth. I have never said or claimed any such thing.

I'd say https://twitter.com/ste_bau/status/1393287684681043970 makes a pretty clear statement, but OK then.

, what is the practical use of a package like this? What are the real-world use cases for a package where creating a fork of the upstream source code to bypass the design decisions of the upstream vendor solves?

You've made your position clear on this, and I do not owe you any further explanation. If you do not have a use for this package, move along.

Actually, I have a legitimate use case for bypassing the final keyword, and have been using dg/bypass-finals for that purpose for a few years now.  You can take a look at the tests for https://github.com/Pagerfanta/doctrine-mongodb-odm-adapter/tree/4.x for that use case.  As much as I'd rather not need tooling like unfinalize or dg/bypass-finals, I also recognize that there are cases for it to exist (and in all honesty, because I'm not interested in testing the entire database interaction there but only that the adapter code is bridging through correctly, I'd rather get rid of the final keyword in this specific case than deal with a "pure" implementation that requires extra external resources).
Yes, my position is that unfinalize and dg/bypass-finals have no place in production code, but I'm also a pragmatist and if they're going to exist and be used, I'd rather see the authors convey clear and concise use cases for these tools instead of just encouraging folks to install them and blindly modify third party code.  To me, the way this package has been promoted comes across as "I want my third party code to follow my development philosophy of 'no final', and this is my way of accomplishing it", and I personally disagree with that as its main reason for existing.

I'd say https://twitter.com/ste_bau/status/1393287684681043970 makes a pretty clear statement, but OK then.

My original statement still stands. What you claimed is incorrect, and misconstrued.

Actually, I have a legitimate use case for bypassing the final keyword

As much as I'd rather not need tooling like unfinalize or dg/bypass-finals, I also recognize that there are cases for it to exist

Then what are you going on about?

To me, the way this package has been promoted comes across as "I want my third party code to follow my development philosophy of 'no final', and this is my way of accomplishing it", and I personally disagree with that as its main reason for existing.

Can't say anything about that. You've came up with your own hypothetical purpose for this package, and then disagree with it.
Let me be clear. I support developers doing whatever they please with open source MIT code. Whatever their philosophy is, I really don't care.

To me, the way this package has been promoted comes across as "I want my third party code to follow my development philosophy of 'no final', and this is my way of accomplishing it", and I personally disagree with that as its main reason for existing.

Can't say anything about that. You've came up with your own hypothetical purpose for this package, and then disagree with it.

Well then let me ask the pointed question, then.  Why is it so difficult to demonstrate practical use cases for a package like this beyond this marketing blurb:

Unleash the freedom lost with open source PHP packages marking classes and methods as final.

This is, to me, part of what creates the perception of "this exists only because I want it to".  I've watched enough arguments against final and private happen enough times to know that people have strong feelings about it, but I can't say I've ever personally seen anyone make a case for why those two keywords are so bad beyond the philosophical aspect of it.  To me, if it's really such a big issue that you've needed to write and release this tool, I would hope that there's some practical use case, even if it falls into the niche "I can't be bothered to set up a database to test a bridging layer" testing scenario I used.  At least dg/bypass-finals tries to present a primary use case, and it lives in the same space as this package as far as what problem it aims to solve.  And yes, I do realize that composer-patches is a thing; it's pretty much impossible to run a Drupal site without it.

Let me be clear. I support developers doing whatever they please with open source MIT code. Whatever their philosophy is, I really don't care.

Believe it or not, I actually have no issue with this package existing, or with folks using it.  Actually, I'm more bothered by the folks who are now shipping updates to their packages to force Composer to conflict their packages with this tool.  I just dislike the way this package has been presented (or the way some folks have spoken of this package or the people who don't 100% support this), and I don't think encouraging folks to blindly use it is in anyone's best interests.

Okay, I see.
I've done some Google searches and I think one reason the final keyword exists is this from the Open-Closed Principle:

Software Entities (Classes, Modules, Functions, etc.) should be open for Extension, but closed for Modification.

If that is the case, then I kind of agree this package should be used for testing purposes only (maybe to test private methods in the final class or something?). If this package is used just to remove the final in the production code, I think it just breaks the Open-Closed Principle.
I'd like to hear more opinions about this.

With due respect for my fellow devs, code should be used as one pleases.
As far as I know, there is no one true way of doing things in programming. That would be dogmatic. And principles like the Open-Closed don't exist in a vacuum. Context must be taken into consideration.
This package is not forced upon anyone. It is not for us to tell the author what is right or wrong.
They shared their work for free. And I'm grateful. Sometimes I just need to extend some small functionality and this is much more practical than forking and maintaining an entire repo.

mbaker ok here's one. I need a way to send emails to this address in laravel 9+
<EMAIL_ADDRESS>/vendor/symfony/mime is vehemently rejecting it in
if (!self::$validator->isValid($this->address, class_exists(MessageIDValidation::class) ? new MessageIDValidation() : new RFCValidation())) {
throw new RfcComplianceException(sprintf('Email "%s" does not comply with addr-spec of RFC 2822.', $address));
} despite it being a real email address and Docomo messing up by not following the RFC there is no way of getting around this. Another option would be to some      "exclude-from-classmap": [] sorcery and replace that file with my own version.


The cacerts file is a little out of date

Platform:
Any (checked Linux jdk8u242)
Architecture:
Any.
Please describe the issue that you are having with our binaries and if possible how to recreate it.
As noted on Slack: looking at the cacerts file, it's a little out of date, including at least one CA certificate that's expired, and missing a number of newer CA roots.
There's a static list of files that the regular openjdk build uses to create the cacerts file, which hasn't been updated for a couple of years. (The "official" openjdk downloads thus also have the same outdated cacerts file.)
I can't tell immediately whether the AdoptOpenJDK releases use the cacerts generated by the openjdk build or whether it drops in its own static one, but that static copy is also out of date. Updating that static copy from the Mozilla CA bundle is likely the easiest fix.

@ptribble Could you give an example of which CA is outdated and which CA expired, please?

Well, this is the one that's expired:
Issuer: CN=UTN-USERFirst-Object, OU=http://www.usertrust.com, O=The USERTRUST Network, L=Salt Lake City, ST=UT, C=US
Valid from: Fri Jul 09 19:31:20 BST 1999 until: Tue Jul 09 19:40:36 BST 2019
Overall, the cacerts file has 93 entries; the Mozilla bundle has 137 (and a number of the certs aren't present in the CA bundle; some have been withdrawn or marked as unwanted by Mozilla).
One, for example, that's missing:
Issuer: CN=COMODO Certification Authority, O=COMODO CA Limited, L=Salford, ST=Greater Manchester, C=GB
(There are some other Comodo ones, but not that one.)

Thanks!


Add Level to Value()

Class to allow fetching of levels and levelById - allow endusers to use this themselves to get data
Update Value to accept a levelId parameter
Update Value to handle levelId as if it were a timeseries Value
Create sample form to test various ways you might call a level into your report

Solves

#25

Need to test math comparison operators
Level vs Level
Level vs Timeseries
How should a single level be compared against multiple timeseries?
Should it only apply if the self.value is set?

Generally I think the same math follows, as already in value, if the LocationLevel is a single value it should be treated as a scalar, if it's a "time series" of values it should be treated like any other time series.
The concept of "Location Level" (or you may want to use "SiteReferenceMetaData" here, that's what we're going to call it in OpenDCS baring someone coming up with a better name) really just determines how it's looked up, changed by others, and when what should be presented, it's still ultimately a scalar value or time series of values.


Img URL doesn't work

Challenge Name

https://www.freecodecamp.com/challenges/use-the-bootstrap-grid-to-put-elements-side-by-side
https://www.freecodecamp.com/challenges/use-spans-for-inline-elements
Issue Description

Try to click, nothing happens except loading circle animation. Then try to open the link on the address bar, you'll get to "Not found URL".
Browser Information

Browser Name, Version:
Operating System:
Mobile, Desktop, or Tablet:

Your Code

Screenshot

@inovramadani Works here. Can't reproduce.

I think the server was down when I reported. I can see the image also now.
Thanks.
On Mon, May 8, 2017 at 11:13 AM, Ademola Adegbuyi<EMAIL_ADDRESS>wrote:

@inovramadani https://github.com/inovramadani Works here. Can't
reproduce.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/freeCodeCamp/freeCodeCamp/issues/14762#issuecomment-299768800,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AaOB1DOKsdAXza2jX-1ISjYVt5701Qjhks5r3pZ7gaJpZM4NS_6X
.

--
Best regards,
Iqbal

You can close the issue.
cc @Bouncey


Serialize arrays in a readable format instead of just counting the items.

We do currently have the problem that we are getting an exception where we can not really reproduce the issue but some of our users are running into the issue.
However, the raven client for PHP serializes arrays in a not very helpful way by just counting the array items, which leads to event data like this:

With the proposed changes this changes to (amongst other points where arrays show up):

Which is evidently much more helpful as we can see the user input and reproduce the problem now.
Would be awesome if we could put that into one of the next releases for the official client so we can switch back to that from our forked one.
I adjusted the tests accordingly, let me know if I should change anything else and keep up the good work!

There are already numerous attempts at refactoring this issue. We currently are working on the 2.0 branch for a new release, and we would like to address this issue thoroughly.
This approach seems dangerous and not practical to me, since it doesn't take into account the fact that an array can contain any number of things, especially objects, so this PR doesn't seem feasible to me.

@Jean85 You have a valid point, this is however the reason that I am using print_r and not var_export so one does not run into problems with circular references. I appreciate that object content may be serialized here as well, but how would this be a problem? Is your point that the request might be too big?
Regardless, do you have any advice on how we could analyze our issue differently in the meantime before 2.0 is finished?

The first thing that you can track down is which version of the client are you using, since #554 should have reduced this issue a bit. For anything else, there's #632 which would make the limit customizable.

We were in fact using an old version and the level was just cut off too early. Bummer I did not see this before trying to cook my own soup. Well, you never step learning I guess.
Thanks for the hint @Jean85 - I'm closing this PR as I agree it is nonsense then.


Create issue button is not using standard issue template language

This looks great! Can you make 1 minor tweak to your commit message? We use conventional commits to drive the changelog and to increment the version of the theme. Can you prefix your commit message with fix: ?
You can amend your commit message with:
git commit --amend
git push --force-with-lease

@jerelmiller I actually did this with the web editor—maybe just reject this PR and I'll re-submit with the correct commit syntax?

That works


Envoy Health Check: Both Failure and Success Counters Increased At The Same Time

Title: Envoy Health Check: Both Failure and Success Counters Increased At The Same Time
Description:
When we use Envoy TCP health check feature to check a ZooKeeper cluster, we noticed that both health_check.network_failure and health_check.success counters increased at the same time.
After checking the tcpdump outputs, we found that ZooKeeper cluster did send back the expected response back to Envoy and the ZooKeeper cluster also sent FIN immediately after that. We think that could be the reason why both failure and success counter increased at the same time.
Our solution is to set the reuse_connection to false.
Question: would it be better to only increase the failure counter under this situation to make it less confusing?
Admin and Stats Output Example:
cluster.name.health_check.attempt: 15600
cluster.name.health_check.failure: 15600
cluster.name.health_check.healthy: 0
cluster.name.health_check.network_failure: 15600
cluster.name.health_check.passive_failure: 0
cluster.name.health_check.success: 15596
cluster.name.health_check.verify_cluster: 0

Config Example:
healthy_threshold: 2
unhealthy_threshold: 2
interval: 5s
timeout: 1s
tcp_health_check:
    send:
        text: 72756F6B # ruok
    receive:
        - text: 696D6F6B # imok

cc @fishcakez and @JuniorHsu

Hi @Winbobob, I see, a network failure is recorded whenever we get LocalClose or RemoteClose and the expected_close_ is false. The reuse_connection to config expected_close_ sounds good to me. This scenario is like it is the peer doesn't want to reuse the the connection, while Envoy is configured to. Not sure if we have a better solution, and feel free to proposal, thanks!


running spec with ValidationError due to request timed out for Message returns exit code 0

we are struggling with an issue of false passing build although most tests were skipped
Expected behavior
failures due to errors in run such as described below should return exit code 1
there can be two options -

the described error below should set the test as failed instead of skipped ( it's not a "missing  implementation" error) + have option to mark tests as ignored / known issue not to fail a run
always return exit code 1 in case of real errors occurred

Actual behavior
running spec that contains error such as you see in the log below  returns exit code 0
while it's actually a real issue we need to address as QA team.

gradle -q -PgaugeTags=E2E -PgaugeEnv=ci gaugeTask
[ValidationError] /gauge/specs/ui/uspViewer.cpt:11 Request timed out for Message with ID => 3 and Type => StepValidateRequest => 'Successful login as username "admin" and password "admin"'
[ValidationError] /gauge/specs/ui/deviceViewer.spec:5 Request timed out for Message with ID => 2 and Type => StepValidateRequest => 'Go to the home page'

[0mSuccessfully generated html-report to => /gauge/build/reports/html-report/index.html
[0mSpecifications:	2 executed	2 passed	0 failed	5 skipped
Scenarios:	7 executed	7 passed	0 failed	16 skipped
Total time taken: 1m16.787s
Updates are available. Run gauge update -c for more info.
Gauge version
Gauge version: 1.0.4
Commit Hash: 3a9a647
Plugins
html-report (4.0.6)
java (0.7.1)
screenshot (0.0.1)

This issue is related to #523 Gauge exit status should be zero in case of skipped scenarios.
So Gauge return exit code non-zero only for failing tests while tests are skipped it return with 0 exit code.
You can use tags for marking specs/scenarios as a known issue and ignore them while running with
-PgaugeTags="\!known issue".
if you think this is not the right way to handle, update your comments here.
we can discuss it further.

Hi,
the linked issue #523 is not the same as what we are experiencing. our tests skipped due to connection time out and not parsing error which we need to be alarmed of while currently we see passing build
[ValidationError] /gauge/specs/ui/deviceViewer.spec:5 Request timed out for Message with ID => 2 and
in such cases i would expect the tests set to fail instead of skipped or return a new status which is not skipped or fail,maybe error? and this will return exit code 1.

@rinafrusman You are right this issue is not related to #523.
The issue here is Gauge ask the runner to validate the steps. To do so it sends a message to the runner and waits for a response. In your case for some reasons, the response is not coming with the specified timeout, hence the error Request time out. Now gauge interpret it as a validation error. By default, Gauge does not exit with a non zero code for validation errors.
Can you share a sample project where we can replicate this issue? Also please share your environment info like OS, CPU, gauge and it's plugin version (along with language's compiler version).

@rinafrusman any updates ?

Closing due to inactivity. Feel free to reopen.


Format of Date Properties in Page Attribute Display Block not working

I've recently updated a site from <IP_ADDRESS> to 8.1.0 and while nearly everything worked seamlessly after, the Page Attribute Display Block I was using for displaying the publishing date for news posts stopped regarding the Date format I entered under "Options" in the block editor.
So in spite of having this in settings:

The date still displays as this:

After looking through the source code for the Page Attribute Block, I see that in /concrete/blocks/page_attribute_display/templates/date_time.php there is a try {...} catch {...} clause that applies the format if it is set. Testing on my side, indicated that in spite of it being set, there's an exception being thrown on line 14 of that file. The exception message is Can't convert '' to a date/time iso format — so it appears to be just getting an empty string?
I'd troubleshoot more if I could, but I feel this may be better handled by people more familiar with the functions here. I'm happy to try anything that would assist in troubleshooting and reply back.
Thank you!

The issue is solved in the comment section here: https://documentation.concrete5.org/editors/in-page-editing/block-areas/add-block/page-attribute-display
Credit for finding solution is: magnolia4
//I've found the bug in templates/date_time.php in line 9:
echo $dh->formatCustom($format, $this->controller->getContent()); //wrong (current code)
echo $dh->formatCustom($dateFormat, $this->controller->getContent()); //right (corrected code)


40 GB model

Hi,
thanks for your nice repo. You mention 2 3090:
`The following hardware is needed to run different models in MiniLLM:

Model
GPU Memory Requirements
Compatible GPUs

llama-7b-4bit
6GB
RTX 2060, 3050, 3060

llama-13b-4bit
10GB
GTX 1080, RTX 2060, 3060, 3080

llama-30b-4bit
20GB
RTX 3080, A5000, 3090, 4090, V100

llama-65b-4bit
40GB
A100, 2x3090, 2x4090, A40, A6000

`
So when I try the 60B Version with 2 RTX 3090 I get an OOM - how can I use both GPUs?
Kind regards,
Dirk

Ah, sorry, I haven't implemented that feature yet :) happy to merge a PR if someone does that first

for what it's worth, i was running the 65B model on one a6000


API authentification

Introduce a handshake between the load balancer and the transcoder (based on a secret ?), actually someone can register as transcoder on your laod balancer, it sucks a bit for the security 😒

Maybe use a JWT to sign the request

JWT can be great solution to be sure the request wasn't edited but I don't see how it could help us to know if it's a "real transcoder" instead of someone trying to call the API?
It's necessary to have a secret shared between transcoders and the load balancer no?

I’m not sure but I think since JWT use a HMAC signature, we can sign a query with a shared secret, it prevent from sending the token directly in query, also we can force to have a date header to prevent someone reusing the query if he stole it with a MITM Attack

Has any plans been made on if this will be executed in the future? Even a simple password handshake would be nice in the interim.

It's on the roadmap, no ETA at the moment, feel free to make a PR 👌

I've been tinkering with something (yet unreleased) to this effect within for my Docker containers, wherein an environment variable is set containing a list of whitelisted transcoder IPs (TRANSCODE_WHITELIST=) that will populate iptables rules on startup. Perhaps something similar could be introduced within the load balancer? It could simply check all incoming transcoder connections against the whitelist before accepting them?

I think token authentification is a best solution to easily support upscale and downscale

Still goofing around so no PR yet, but here's rudimentary proof-of-concept for shared authentication: UnicornLoadBalancer, UnicornTranscoder.
A token can either be specified (env SERVER_AUTH) or it will be automatically generated at startup.


We need a way to clean the nuget/dnu caches on the CI

Who wants to implement this? Volunteers? :D

The CI shouldn;t be caching. We should just pass --no-cache to dnu restore on the ci. We also only cache if using http based sources (which we shouldnt; be on the ci).

Can we instead add an option to redirect where the cache gets stashed (via env variable)? We can simply move it to that location in that case.
Alternatively if we're good citizens and clear the cache ever so often %DNX_HOME%/temp/cache would be a nice location to relocate it. We have %DNX_HOME% redirected to a subdir of the working directory on the CI so it we wouldn't need to do anything.

Closing this one too based on @pranavkm's comment on #260.


[bug] image src  contains () can't load

If the data-src contains () , then backgroundImage cannot set url(src)
In your source code：
entry.target.style.backgroundImage = "url(" + src + ")";
if the data-src=“https://t1.27270.com/uploads/tu/201809/559/1 (2).jpg”  ，then the () Ambiguity
I tested it,
"url('" + src + "')"
this is okay

This was fixed with #428.


Register id provider refactor

In this PR I decouple IdentityAPI from the integrations protocol.
Also I'm moving the id provider into the register package

before merge this PR, update target to feat/register-entities

Pull Request Test Coverage Report for Build 222186359

0 of 0   changed or added relevant lines in 0 files are covered.
71 unchanged lines in 4 files lost coverage.
Overall coverage remained the same at ?%

Files with Coverage Reduction
New Missed Lines
%

internal/agent/event_sender_vortex.go
1
73.21%

pkg/integrations/v4/dm/emitter.go
6
82.17%

pkg/backend/identityapi/register_client.go
29
65.88%

cmd/newrelic-infra/newrelic-infra.go
35
8.51%

Totals

Change from base Build 208697048:
0%

Covered Lines:

Relevant Lines:
0

💛  - Coveralls


Improve AI handling of logical operators

This PR should address #213 and some minor code refactoring.

Hmm, so I've noticed that intermittently this test fails due to an unexplained 'type confusion':
Test Case 'ProgramBuilderTests.testTypeInstantiation' started at 2021-11-03 09:00:22.498
Fuzzilli/ProgramBuilder.swift:520: Assertion failed: Unexpected type encountered .integer | .object()
Current stack trace:

Any thoughts on what the root cause might be. Perhaps incorrect AI modelling somewhere?
I'll debug this further when I've got some time and put together a fix.

Ah yeah, I've noticed these before... I think the issue here is that generateType might generate e.g. a TypedArray from the JavaScriptEnvironment (that's correct), then generateVariable tries to call the corresponding constructor and attempts to obtain argument variables (that's also correct), but for TypedArrays, the constructor is defined to take .integer | .object(...): https://github.com/googleprojectzero/fuzzilli/blob/1c33ac3cefb0771978a3f8f1185ac77fca703b14/Sources/Fuzzilli/Core/JavaScriptEnvironment.swift#L390 (also correct), but generateVariable can't yet handle type unions (that is, I think, the issue). Feel free to open a separate issue for this :)


[feature request] Provide a way to mock backend responses when using viceroy as a test runner

When using viceroy as a test runner it would be helpful if there were a way to mock backend responses. The motivation for this is twofold:

Tests that make external network requests are inherently flaky due to transient network issues. It would be much better for reliability if backend requests in tests did not have to make an external request.
Writing unit-style tests is much easier if you can mock external dependencies. If your test is making external requests it's an integration test whether you like it or not. While integration tests are valuable, using mocks for unit tests allows writing tests for edge cases such as handling specific errors which are much more difficult to test against a real external service.

I don't know the viceroy codebase well so I can't offer useful implementation suggestions, but from a user's perspective I can imagine a few ideas of solutions that would be useful:

A hostcall that allows mocking requests for a backend, either by providing a handler routine in the wasm module or by directly providing a response that should be used.
A commandline argument to provide a handler routine in the wasm module to be used to handle requests for a specific backend, e.g. --mock-backend=my-backend=some_function_name.

Either of these would provide enough functionality to address the two points in the motivation list above.

Depending on the level of configurability needed, we could even do something like embedding https://github.com/facebook/starlark-rust to allow viceroy "scripts" to handle creating responses.

For the purposes of mocking backends for tests I would personally prefer a solution that allows using a handler function alongside the existing code in the module.


Add JarPluginManager, PluginLoader, AbstractPluginManager

This PR comes with many good things.
First of all, it comes with a builtin JarPluginManager (see #118). According javadoc:

It's a {@link PluginManager} that load plugin from a jar file.
Actually, a plugin is a fat jar, a jar which contains classes from all the libraries,
on which your project depends and, of course, the classes of current project.

So, your plugins are jar files that could be created using Maven Shade Plugin.
To add the possibility to create custom PluginManager more easy, this PR introduces new concepts:

AbstractPluginManager
PluginLoader

The current implementation of JarPluginManager is:
public class JarPluginManager extends DefaultPluginManager {

    @Override
    protected PluginRepository createPluginRepository() {
        return new JarPluginRepository(getPluginsRoot(), isDevelopment());
    }

    @Override
    protected PluginDescriptorFinder createPluginDescriptorFinder() {
        return isDevelopment() ? new PropertiesPluginDescriptorFinder() : new JarPluginDescriptorFinder();
    }

    @Override
    protected PluginLoader createPluginLoader() {
        return new JarPluginLoader(this, pluginClasspath);
    }

}

Coverage increased (+1.8%) to 15.176% when pulling d7f245e511f82b69925f90f378123cbedddfc92a on jar_plugin_manager into c82df37f676cb1c4ddd2ba8388deb2049bcb425a on master.


Update Golang 1.20

Tool name
Golang
Tool license
BSD 3-Clause "New" or "Revised" License
Add or update?

[ ] Add
[X] Update

Desired version
1.20.2
Approximate size
No response
Brief description of tool
Golang currently has 1.20.1 cached on the hosted runners, so the actions/setup-go action is picking 1.20.1 when using a go.mod file specifying 1.20.
Go 1.20.2 was released around a week ago which included various fixes (including security fixes). It would be good to have the cached version updated to 1.20.2 to pick up these fixes.
URL for tool's homepage
https://github.com/golang/go
Provide a basic test case to validate the tool's functionality.
No response
Platforms where you need the tool

[ ] Azure DevOps
[X] GitHub Actions

Runner images where you need the tool

[ ] Ubuntu 18.04
[ ] Ubuntu 20.04
[X] Ubuntu 22.04
[ ] macOS 11
[ ] macOS 12
[ ] Windows Server 2019
[ ] Windows Server 2022

Can this tool be installed during the build?
No response
Tool installation time in runtime
No response
Are you willing to submit a PR?
yes

@erik-bershel - that's great. Thanks for the prompt response!

Tool name
Golang
Tool license
BSD 3-Clause "New" or "Revised" License
Add or update?

[ ]  Add
[x]  Update

Desired version
1.20.2
Approximate size
No response
Brief description of tool
Golang currently has 1.20.1 cached on the hosted runners, so the actions/setup-go action is picking 1.20.1 when using a go.mod file specifying 1.20.
Go 1.20.2 was released around a week ago which included various fixes (including security fixes). It would be good to have the cached version updated to 1.20.2 to pick up these fixes.
URL for tool's homepage
https://github.com/golang/go
Provide a basic test case to validate the tool's functionality.
No response
Platforms where you need the tool

[ ]  Azure DevOps
[x]  GitHub Actions

Runner images where you need the tool

[ ]  Ubuntu 18.04
[ ]  Ubuntu 20.04
[x]  Ubuntu 22.04
[ ]  macOS 11
[ ]  macOS 12
[ ]  Windows Server 2019
[ ]  Windows Server 2022

Can this tool be installed during the build?
No response
Tool installation time in runtime
No response
Are you willing to submit a PR?
yes
https://github.com/actions/runner-images/issues/7281#issue-1623661041

Probably related: https://github.com/actions/runner-images/issues/7276

@stuartleeks Ubuntu images with Go 1.20.2 deployed.


struct perf_event_attr seems to be wrong

Hi,
Looking at the definition of perf_event_attr it seems wrong compared to what it should be
#[repr(C)] #[derive(Debug, Default, Copy, Clone)] pub struct perf_event_attr {     pub _address: u8, }
Where in fact it's
           struct perf_event_attr {
               __u32 type;                 /* Type of event */
               __u32 size;                 /* Size of attribute structure */
               __u64 config;               /* Type-specific configuration */

               union {
                   __u64 sample_period;    /* Period of sampling */
                   __u64 sample_freq;      /* Frequency of sampling */
               };

               __u64 sample_type;  /* Specifies values included in sample */
               __u64 read_format;  /* Specifies values returned in read */

               __u64 disabled       : 1,   /* off by default */
                     inherit        : 1,   /* children inherit it */
                     pinned         : 1,   /* must always be on PMU */
                     exclusive      : 1,   /* only group on PMU */
                     exclude_user   : 1,   /* don't count user */
                     exclude_kernel : 1,   /* don't count kernel */
                     exclude_hv     : 1,   /* don't count hypervisor */
                     exclude_idle   : 1,   /* don't count when idle */
                     mmap           : 1,   /* include mmap data */
                     comm           : 1,   /* include comm data */
                     freq           : 1,   /* use freq, not period */
                     inherit_stat   : 1,   /* per task counts */
                     enable_on_exec : 1,   /* next exec enables */
                     task           : 1,   /* trace fork/exit */
                     watermark      : 1,   /* wakeup_watermark */
                     precise_ip     : 2,   /* skid constraint */
                     mmap_data      : 1,   /* non-exec mmap data */
                     sample_id_all  : 1,   /* sample_type all events */
                     exclude_host   : 1,   /* don't count in host */
                     exclude_guest  : 1,   /* don't count in guest */
                     exclude_callchain_kernel : 1,
                                           /* exclude kernel callchains */
                     exclude_callchain_user   : 1,
                                           /* exclude user callchains */
                     mmap2          :  1,  /* include mmap with inode data */
                     comm_exec      :  1,  /* flag comm events that are
                                              due to exec */
                     use_clockid    :  1,  /* use clockid for time fields */
                     context_switch :  1,  /* context switch data */
                     write_backward :  1,  /* Write ring buffer from end
                                              to beginning */
                     namespaces     :  1,  /* include namespaces data */
                     ksymbol        :  1,  /* include ksymbol events */
                     bpf_event      :  1,  /* include bpf events */
                     aux_output     :  1,  /* generate AUX records
                                              instead of events */
                     cgroup         :  1,  /* include cgroup events */
                     text_poke      :  1,  /* include text poke events */

                     __reserved_1   : 30;

               union {
                   __u32 wakeup_events;    /* wakeup every n events */
                   __u32 wakeup_watermark; /* bytes before wakeup */
               };

               __u32     bp_type;          /* breakpoint type */

               union {
                   __u64 bp_addr;          /* breakpoint address */
                   __u64 kprobe_func;      /* for perf_kprobe */
                   __u64 uprobe_path;      /* for perf_uprobe */
                   __u64 config1;          /* extension of config */
               };

               union {
                   __u64 bp_len;           /* breakpoint length */
                   __u64 kprobe_addr;      /* with kprobe_func == NULL */
                   __u64 probe_offset;     /* for perf_[k,u]probe */
                   __u64 config2;          /* extension of config1 */
               };
               __u64 branch_sample_type;   /* enum perf_branch_sample_type */
               __u64 sample_regs_user;     /* user regs to dump on samples */
               __u32 sample_stack_user;    /* size of stack to dump on
                                              samples */
               __s32 clockid;              /* clock to use for time fields */
               __u64 sample_regs_intr;     /* regs to dump on samples */
               __u32 aux_watermark;        /* aux bytes before wakeup */
               __u16 sample_max_stack;     /* max frames in callchain */
               __u16 __reserved_2;         /* align to u64 */

           };

If it helps here some FFI someone wrote
https://docs.rs/perf/0.0.0/src/perf/ffi.rs.html
Also it seems like redbpf old sys module had it as well
https://docs.rs/redbpf/0.9.8/redbpf/sys/perf/struct.perf_event_attr.html

It seems like what's is missing is in bindings.h
#include <linux/perf_event.h>
But not sure how you want to solve this for __LIBBPF_SYS_NOVENDOR

libbpf is coming with its own set of Linux UAPI headers (in https://github.com/libbpf/libbpf/tree/master/include/uapi/linux). perf_event.h is not there, but we can add it there (and keep it in sync automatically). Will that help with SYS_NOVENDOR? @alexforster do you use libbpf's UAPI headers or rely on system ones?

While we are at it, I suggest we will also add
struct perf_sample_raw {
	struct perf_event_header header;
	uint32_t size;
	char data[];
};

struct perf_sample_lost {
	struct perf_event_header header;
	uint64_t id;
	uint64_t lost;
	uint64_t sample_id;
};

enum perf_event_type {
	PERF_RECORD_MMAP = 1,
	PERF_RECORD_LOST = 2,
	PERF_RECORD_COMM = 3,
	PERF_RECORD_EXIT = 4,
	PERF_RECORD_THROTTLE = 5,
	PERF_RECORD_UNTHROTTLE = 6,
	PERF_RECORD_FORK = 7,
	PERF_RECORD_READ = 8,
	PERF_RECORD_SAMPLE = 9,
	PERF_RECORD_MMAP2 = 10,
	PERF_RECORD_AUX = 11,
	PERF_RECORD_ITRACE_START = 12,
	PERF_RECORD_LOST_SAMPLES = 13,
	PERF_RECORD_SWITCH = 14,
	PERF_RECORD_SWITCH_CPU_WIDE = 15,
	PERF_RECORD_NAMESPACES = 16,
	PERF_RECORD_KSYMBOL = 17,
	PERF_RECORD_BPF_EVENT = 18,
	PERF_RECORD_CGROUP = 19,
	PERF_RECORD_TEXT_POKE = 20,
	PERF_RECORD_MAX = 21,
};

So one can use can use perf_buffer__new_raw and define a different algorithm for fetching i.e for batching

sgtm

libbpf is coming with its own set of Linux UAPI headers (in https://github.com/libbpf/libbpf/tree/master/include/uapi/linux). perf_event.h is not there, but we can add it there (and keep it in sync automatically).

This would be my preference. That way, if we ever go back to generating these bindings on user machines, users won't need to have Linux headers installed (same as with libbpf).

Will that help with SYS_NOVENDOR? @alexforster do you use libbpf's UAPI headers or rely on system ones?

Both – https://github.com/libbpf/libbpf-sys/blob/c1850cc107ea43ae9a103167d207ac5a770e5a5d/bindings.h#L1-L13
When "novendor" is unset, we rely entirely on libbpf's copy of UAPI headers. When it is set, we rely on the headers from the local machine.

libbpf is coming with its own set of Linux UAPI headers (in https://github.com/libbpf/libbpf/tree/master/include/uapi/linux). perf_event.h is not there, but we can add it there (and keep it in sync automatically).

This would be my preference. That way, if we ever go back to generating these bindings on user machines, users won't need to have Linux headers installed (same as with libbpf).

Done in https://github.com/libbpf/libbpf/pull/445.

Cool – any chance a new release is coming soon?

In a week or two, most probably. Still waiting for few more deprecation patches landing before cutting v0.7.

Any ETA when the next release will be?

@arilou, libbpf v0.7.0 was cut 10 days ago (https://github.com/libbpf/libbpf/releases/tag/v0.7.0).
@alexforster can you please create a new libbpf-sys release?

I gave it a try last week and I ran into an issue. For some reason, bindgen is now generating duplicate definitions of BTF_KIND_* preprocessor defines.
Example:
    --> src/bindings.rs:3157:1
     |
220  | pub const BTF_KIND_TYPE_TAG: u32 = 18;
     | -------------------------------------- previous definition of the value `BTF_KIND_TYPE_TAG` here
...
3157 | pub const BTF_KIND_TYPE_TAG: ::std::os::raw::c_uint = 18;
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `BTF_KIND_TYPE_TAG` redefined here
     |

I'm not sure why, but I'll continue digging.

It's this, probably: https://github.com/libbpf/libbpf/blob/master/src/btf.h#L385-L398
Not sure on how to work around this, but this is done intentionally for systems with older UAPI headers.

cc @mdaverde

@arilou Can you check this out and see if it works for you?
https://github.com/libbpf/libbpf-sys/pull/37
struct perf_sample_raw and struct perf_sample_lost aren't defined anywhere in libbpf or uapi. The only place they're defined is here, which bindgen can't see. However, we are now generating struct perf_event_header which should be enough for this use case, I think.

Thanks you @alexforster, it looks good, I understand perf_sample_raw and perf_sample_lost are currently not defined in the header files so bindgen cannot access them, but I was hoping that it makes more sense to move them to a header file, so once can implement there own drain mechanism.
I dont mind defining those structs manually for myself, as they wont change, but I figured it would make more sense to export them,

That sounds like a reasonable request, but since it's a change to the C API, I'll defer to @anakryiko

perf_sample_raw and perf_sample_lost are internal to libbpf, it's not part of any kind of UAPI. It corresponds to the way that perf subsystem lays out the data in perf buffer, but it's just a custom struct. So there is no need for libbpf to expose it as an API. Just define your own struct with the same layout.

Hi @anakryiko , thank you for the fast reply, I understand it's not a UAPI but libbpf exports today:
perf_buffer__new_raw
in which you can define an event_cb
I was under the assumption that the way you should work with the event_cp:
is something like
switch (e->type) {
case PERF_RECORD_SAMPLE:
perf_sample_raw *sample = (perf_sample_raw *)e;
...
break;
case PERF_RECORD_LOST:
perf_sample_lost *sample = (perf_sample_lost *)e;
....
break;
}
was there a different idea of how to use this API?

yes, but for perf_buffer__new_raw() user is expected to define their own data structure (or just not use any structure at all, just fetch necessary bytes). raw part means "advanced use case and user knows how to deal with it". So yeah, just go ahead and define your types, nothing wrong with that.
Also, I haven't double check and I haven't looked into that code for a while, the layout of that perf sample will depend on exact set of flags you specify when you call perf_event_open() syscall. So there is no single perf_sample_raw that would work for all cases.

thanks @anakryiko , will do that


Pod DNS resolution broken

Initially Flux setup failed because I was getting
2023-02-15T06:05:02.397Z error GitRepository/home-kubernetes.flux-system - Reconciler error failed to checkout and determine revision: unable to clone 'https://github.com/kmatasfp/k8s-external': Get "https://github.com/kmatasfp/k8s-external/info/refs?service=git-upload-pack": dial tcp: lookup github.com on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:48046-><IP_ADDRESS>:53: read: connection refused

confirmed that it was dns issue by running dig and nslookup from within a Pod I deployed for test purposes.
Even joined Discord to see if any information there and found this thread: https://discord.com/channels/673534664354430999/1074532630395105341
From there I got a link to recently merged PR https://github.com/onedr0p/flux-cluster-template/pull/627, If I removed the changes made there DNS resolution started to work again from  the Pods.
Host Details:
Ubuntu 22.04
resolvectl status

Current DNS Server: <IP_ADDRESS>
DNS Servers: <IP_ADDRESS>
DNS Domain: local.${SECRET_DOMAIN}

So the change might work for Fedora hosts, but not Ubuntu. Imho need to revisit this or provide a doc or have step in ansible playbook that during install is Ubuntu specific and makes sure that DNS works

Thanks for reporting this issue, I do have a feeling this needs to be documented more or maybe other people can share their experiences and what's causing the issue and how they fixed it.
There's not only a difference in ubuntu or fedora but also in the way people handle DNS at home. It's all over the place so it's hard to have one fix for all situations.
I'll go ahead and convert this into a GH discussion.
I suggest people post here their problems and what they did to mitigate DNS issues.


Animate everything

Transitions currently are a mess: map jumps (see also #136), options jump (#129), state changes in the opening hours editor are sudden, etc etc.
I'd like the app to feel smooth. For that we need to use Flutter animations, but that's too hard.

map jumps (see also #136)

There's #230, too.


The tests in challenge "Record Collection" is not accurately determining the result of the code entered.

Affected page
https://www.freecodecamp.org/learn/javascript-algorithms-and-data-structures/basic-javascript/record-collection
Your code
if(value.length === 0){
    delete records[id][prop];
    return records;
  }

  if(prop !== "tracks" && value.length > 0){
    records[id][prop] = value;
  } else if(prop === "tracks"){

    album = records[id];

    if(album.hasOwnProperty(prop)){

      if(value.length > 0){
        album[prop].push(value);
      }

    } else {
      album[prop] = [value];
    }
  }

  return records;

Expected behavior
I ran the code in the challenge webpage and it was stating that my solution was wrong. However, when I created a folder, an HTML page and ran the javascript by itself on the HTML page with the developer tools opened in chrome, the code executed just as it was explained under the hints the test is suppose to give you when you are wrong.
Screenshots
Image of chrome developer tools console results after calling "updateRecords(recordCollection, 5439, 'artist', 'ABBA');" followed by "console.log(updateRecords(recordCollection, 5439, "tracks", "Take a Chance on Me"));"
System (please complete the following information):

Device: Laptop
OS: Windows 10
Browser: chrome
Version: version of chrome? 91.0.4472.114 (Official Build) (64-bit)

Additional context
Add any other context about the problem here.
The files and file structure used to test the behavior against the challenge page goes as follows.
One folder called "wtf" was created. It stored a file named "wtf.html" and a folder called "js". The folder contained a file named "wtf.js".
wtf.html has the following code:
Image of HTML code because markdown isn't behaving
wtf.js has the following code:
`
var recordCollection = {
2548: {
albumTitle: 'Slippery When Wet',
artist: 'Bon Jovi',
tracks: ['Let It Rock', 'You Give Love a Bad Name']
},
2468: {
albumTitle: '1999',
artist: 'Prince',
tracks: ['1999', 'Little Red Corvette']
},
1245: {
artist: 'Robert Palmer',
tracks: []
},
5439: {
albumTitle: 'ABBA Gold'
}
};
function updateRecords(records, id, prop, value) {
if(value.length === 0){
  delete records[id][prop];
  return records;
}

if(prop !== "tracks" && value.length > 0){
  records[id][prop] = value;
} else if(prop === "tracks"){

  album = records[id];

  if(album.hasOwnProperty(prop)){

    if(value.length > 0){
      album[prop].push(value);
    }

  } else {
    album[prop] = [value];
  }
}

return records;

}
//console.log(updateRecords(recordCollection, 5439, 'artist', 'ABBA'));
updateRecords(recordCollection, 5439, 'artist', 'ABBA');
console.log(updateRecords(recordCollection, 5439, "tracks", "Take a Chance on Me"));
//updateRecords(recordCollection, 5439, "tracks", "Take a Chance on Me")
`

Thank you for reporting this issue.
This is a standard message notifying you that this issue seems to be a request for help. Instead of asking for help here, please click the "Get Help" button on the challenge on freeCodeCamp and choose the "Ask for help" option, which will help you create a question in the right part of the forum. Volunteers on the forum usually respond to questions within a few hours and can help determine if there is an issue with your code or the challenge's tests.
If the forum members determine there is nothing wrong with your code, you can request this issue to be reopened.
Thank you and happy coding.


[GSSOC'23]Adding hover in services section

I would like to contribute to this issue under GSSOC'23. Please assign me this issue.

Follow new issue template acc to guidlines, and this issue is not required RN


more doc updates

Current dependencies on/for this PR:

main

PR #60   👈

This stack of pull requests is managed by Graphite.


Update README.md

language name correction.
For Portuguese.
Font: https://en.wikipedia.org/wiki/Portuguese_language or
Font: https://www.wolframalpha.com/input/?i=Portuguese&lk=1&rawformassumption="ClashPrefs"+->+{"Language",+"Portuguese"}

sorry, I didn't see it.

@ccmawo

[Presenting the code to multiple developers with the purpose of reaching consensus about its acceptance is a simple form of software reviewing. ](https://en.wikipedia.org/wiki/Linus%27s_Law)

@EvertonMelo we practice holocracy, not consensus, as our decision making model.