[Feature-Request] Automatically set color cam lens position at RGB-depth alignment

Start with Why: Many users report that object/feature localization isn't accurate and that it's taking into account background as well. One reason for this is if the depth isn't aligned with the color stream. We have recently added optimization for rgb-depth alignment so it should be enabled at all localization demos/examples.
Move to What:  In firmware, automatically determine and set lens position for AF color camera when rgb-depth alignment is enabled. Currently, we just have the code below in the majority of code examples which isn't accurate. For OAK-D-Lite, it should be IIRC ~80, as the camera is different.
# For now, RGB needs fixed focus to properly align with depth.
# This value was used during calibration
camRgb.initialControl.setManualFocus(130)

How:  We could use getLensPosition, but not all cameras have this info in eeprom, so we would need to add some fallback logic, so ~15 LoC for each example. IMO we should do this in firmware, so whenever we have rgb-depth alignment enabled, we query getLensPosition, and if it's None we find out whether it's OAK-D (lensPos=130) or OAK-D-Lite (lensPos=~80) and set the focus of the color camera to the determined value.
This would reduce quite some complexity (~15 LoC) from the user which we are trying to achieve, so API is as easy to understand as possible.

I get the pain/why you describe. Its a real thing and points to a lack of scripter/dev knowledge and you are trying to bridge that gap of knowledge with an API.
My casual feedback...the change you describe is a substantial break in existing functionality/behavior. It could break apps that already use the alignment and focusing APIs. It also presumes that everyone wants to have the focus the same value as during calibration. I suspect most do (like you describe), but there are others that need the better/auto focus and give up a slight decline in alignment/depth.
The OP change would need to handle the fallbacks you describe + special cases like...
rgb.setAutoFocusMode(dai::CameraControl::AutoFocusMode::OFF);
rbg.setManualFocus(45);
stereoDepth.setDepthAlign(dai::CameraBoardSocket::RGB);
// what is the focus value here? is it 45? or is it the value you propose from eeprom/fallback?

To me it feels fragile, special cases, and breaks existing apps.
I believe this is a good candidate for the "DepthAI SDK". A python-only wrapper of functionality hiding the details/complexity of camera alignment.
As a C++ dev, I'm happy with the SDK as it is today on this topic. Nothing in the OP helps me and instead introduces behavior changes which cascade back to me as additional code changes and testing.
If there really must be a new C++ api, I prefer something new like setDepthAlignSetCalibrationFocus(CameraBoardSocket camera).
Technically, we could overload the existing setDepthAlign() with setDepthAlign(CameraBoardSocket camera, bool calibrationFocus = false), but this multiplies because there are two setDepthAlign()` signatures; causing 4 APIs (binary compatibilty) or 2 APIs (api compatibility).

### Auto-scale RGB image with each focus value before RGB-D alignment
Hi everyone,
I'm working on integrating the OAK-D pro camera into the robotic application. So it requires a lot in the accuracy domain. I feel the pain of lacking accuracy when having the Auto-focus feature. I have done some small experiments on the alignment and got a better accuracy result, at least in my test cases. I will explain my procedure till I figured out the RGB alignment error and the solution for it.
So in my setup for accuracy evaluation, I made a laser-cut plate, with locating holes on it to place 3d-printed ArUco targets. I also had my camera mounted on a mechanical stand. So that, I could get the GroundTruth in the 3D assembly file. After that, I wrote the script to run the camera and detected the ArUco targets and calculate the positions then compared them to the GroundTruth. The below image shows the data I've collected. The red dots are Target centers, and the green dots are projected Groundtruth.

My first approach is optimizing the extrinsic matrix of the camera wrt to its Ideal mounting position.
the optimized data will be from this..

to this..

the result I summarized are these charts below

Also, I used 3 approaches to calculate positions in this experiment. I used spatial calculate on the camera and spatial calculate on host. With the On-host method, I used 2 methods. The first one is by using HFOV, the formula that is provided by the Depthai example, and the last one is using Camera intrinsic matrix re-projection.

In my experiments on the calculation approach, the best way to calculate 3d position is by using the re-projection with the intrinsic matrix.
Finally, here is the stage I realized the RGB alignment error. I have skeptical about this, so I did this test.
I have my OAK-D pro captured images in 2 focus values and this is an overlapped image. You can see the shifting in the texture.

So I go into the next stage to evaluate my theory about focus scaling. I capture targets with multiple focuses, distances, and positions on the image frame. Focus ranges from 100 to 255

I wrote a script to capture the center of the red dot in 2D through multiple focus values.

After this, I concluded:

The auto-focus function will cause a scaling effect on images.
The further the object from the image center, the more offset it will be.
The scaling effect stays the same no matter how far the distance to the object is, as long as it still maintains the same 2D position on the image.
So I captured the whole range of focus values and wrote a script to match the scale of all of them.

After getting all scale values, I plotted on the chart (blue dots) and used the fit function to get the slope function for the Focus value and RGB scale value. Then I used the function to predict the scale value (orange dots)

I re-applied the scale value into the new set of images, and here is the result.

After all of these, I re-done my accuracy evaluation procedure, and here is the result.
OAK-D pro evaluate (with correctMat and scale value).xlsx
this is my first time commenting on Github, so please let me know if I said something wrong. Hope you could consider this and hopefully find something useful for applying to camera firmware

Great post!  Thank you for all the excellent data here.  We will digest and circle back tomorrow.  Thanks again!

@tachiuhy thanks for your data on this topic. :-)
I few questions, please.
I don't understand your first 5 charts about extrinsic matrix. Are you exploring where the 3 oak-d lenses are located (mono left, mono right, rgb)? How do these charts represent these three lenses? How does chart 1 lead to chart 2? What are charts 3,4,5 showing for these three lenses?
I remember a brief post from Lux , I think discord, about an idea of adjusting/scale based on rgb focus values. There were open questions on consistency across lenses of same model, across lenses across models, etc. It was interesting to see the loss of data on all sides when the scale+crop is applied in the last animated GIF. My instinct tells me that needs some thinking regarding sideaffects, repercussions.

Hi @diablodale, thanks for your attention to the post :)
I was thinking about explaining the charts since they lack descriptions and un-unique colors. So the 1st and the 2nd chart are 3D scatter that represent the 3D positions of the ArUco targets used in my test. The chart contains 3 axes for x y and z in millimeters. the blue dots are for the GroundTruth I got and the orange dots are for data I collected from the RGB-Depth alignment. (noted that, I assumed the camera coordinate system is at the front glass and on the optical axis of the RGB lens). The 1st chart is the raw data I got, and you can see it is very off from the GroundTruth. So I decided to move the whole dataset with a transformation matrix, which contains both translation and rotation parameters. That led to the second chart.
For the next 3 charts, I first did the subtraction for all x y, and z positions of GroundTruth and Captured data to get the delta. Those 3 charts are labeled as deltaX, deltaY, and deltaZ (millimeters). Those chart contain 2 lines. The "delta before" means the offset before applying the transformation, and the last one "delta after" is for offset after the transformation.
About the post you mentioned, yes I do think also. But I think there may be ways to calibrate this scale+crop factor at the factory-calibration level for each camera, like what you are doing for distortion and intrinsic parameters.
By the way, this scale+crop is not my final conclusion. Since I realized the fundamental of the focusing lens might not follow the PinHole intrinsic model, it should be the "Thin lens equation" model. I'm trying to do some tests for it. So the Scaling I've done might approx the true accuracy, but not yet to be fully accurate.

Hi both,
Sorry about the delay.  One of the engineers who works on this is on vacation this week.  And the other is down with COVID (and a nasty fever).
We'll hopefully circle back next week.
Thanks and sorry again about the delay,
-Brandon

Hi @saching13,
Thanks for your advice! :)

Yes, I set the focus to the fixed mode for the RGB lens and then captured it for accuracy evaluation.
I also set the depth alignment to the RGB sensor. And I put the camera's coordinate system in the 3D setup at RGB optical center.
Yes, I did turn on the subpixel mode when doing the capturing.

Actually, for the horizontal value of the chart you snipped, I should put from 100 to 255 since I capture images from that range. So for the data, you saw that being flat from 120-150 is actually from 230-255. And during my test, 255 is the value that the lens stays focused on the object at the nearest distance. With that said, it is very important to pay attention to the "Optical properties" of the RGB lens, in this case, it is optical magnification. I have some images you can see down below.
So, as you said, if we put the object really far away from the camera (and if you put the object around the canter of the frame), the shifting would not be much when changing focus. But if you can somehow move the object to the edge of the frame, you will see the rapid change when you change the focus. I mentioned this does not mean we will have to use multiple focus at all the time, but some applications would require different working ranges, and the current RGB alignment can not handle it, I think. Later I will try to capture something in the further distance to prove the point :)
For my robotics application for example. In some tasks, I have to change the camera to "extended disparity" mode to get the depth data from a closer distance, and since I have to detect stuff in RGB images, now I have to change the focus to nearer. And since I changed the focus to a nearer distance (let's say around 200), then now the optical magnification of the RGB lens system has been changed (larger magnification), and no longer matches the pre-defined intrinsic matrix/doesn't match the depth array anymore.
focus 100

focus 120

focus 150

focus 200

focus 255

overlapsed of 100 and 255

Hi, I have a few images capturing further distanced scene with specific focus values. You can see the shifting of the scene dramatically.
focus 100

focus 150

focus 200

focus 255

overlapsed of 100 and 150

I wonder if we can just do a scale-factor of the image in firmware to account for this.  Thoughts @saching13 ?

Thanks @tachiuhy .
Yeah for that close distance to align properly we need to figure a way to change the intrinsics as the lens position changes and then apply it to wrap engine for depth to change accordingly.
Quickest approach would be to use a FF with working range at closer distance.

Thanks for your help,
I will be keeping using my method for my application rightnow. Hope that my idea helps you somehow.
p/s: I love this new OAK-D pro camera :) so hope to see it get more updates and can go further!


feat(onyx-1087): new asset type 'additional files'

Jira
This PR adds new asset type "additional_file", which allows to upload any type of files (not limited to images).
Admins users can upload / download such files via admin interface:

This PR also adds Metaphysics support for new asset type - exposes new mutation arguments (s3Path, s3Bucket) to allow clients to specify where the file is. S3 location is not accessible to public users, therefore Convection pre-signs asset url and exposes it as a new documentPath property.
Deploy

[x] set AWS_UPLOAD_BUCKET env variable on staging / prod

For reviewers
Attaching here a couple of notes to help you to navigate through the code, since this PR includes multiple changes.
File upload
Main changes related to file upload are in this file: https://github.com/artsy/convection/pull/1487/files#diff-166a22f3aef8002c5064540e25e2c427bd29732202af971ffbb6e58238c6d366R1. For images, Convection uses our own https://github.com/artsy/gemini_upload-rails gem, which is a wrapper around Gemini and https://github.com/blueimp/jQuery-File-Upload. This gem dynamically prepares the form for s3 upload - it fetches upload configuration from Gemini and then submits the form using this jquery-file-uload library.
I do it similar thing for the new asset type, but there is no need to go to Gemini, so the form config is rendered on backend:

https://github.com/artsy/convection/pull/1487/files#diff-00832b81dd15c96731fd4ebd9edf330ec827b3add08b119506925c6b68a7aefcR1
https://github.com/artsy/convection/pull/1487/files#diff-bd72bc7edd5884dc9422088bb37aebfab182ba415384e2106ce6701ce473ce0dR1

Frontend passes additional_file_keys and names, which backend uses to create assets here: https://github.com/artsy/convection/pull/1487/files#diff-52d98c35c625099704543c387ac70e728efb04bfa7125d73bd81109907ae451fR3
Access to files
Files are then accessible via pre-signed urls (urls expire in some minutes for security reasons):

https://github.com/artsy/convection/pull/1487/files#diff-0545770a3fbf7606692cd849fe6ca5dd4b454b9a203b8588657abbe7f0ea30b8R57
https://github.com/artsy/convection/pull/1487/files#diff-412023f14b19c2efa5789da4580ddff970d21d92e54e7e40680b80b64ffac1fbR19

GraphQL
For "create asset" mutations, I made gemini_token optional (add added new s3 location fields):

https://github.com/artsy/convection/pull/1487/files#diff-1c8066499670f32295ab501950c5ace240dccb482738c1b403b7ac95d7974856R5
https://github.com/artsy/convection/pull/1487/files#diff-9792b956859f76409c9cf05fd151c6885a83789f032c35f9798525c779324a9cR5

To create an "additional_file" via GraphQL you can execute such mutation:
mutation {
  addAssetToConsignmentSubmission(input: { submissionID: 1, filename: "name.pdf", s3Path: "path", s3Bucket: "bucket"}) {
    clientMutationId
    asset {
      id
      submissionId
    }
  }
}

Assets are now filterable: https://github.com/artsy/convection/pull/1487/files#diff-124d27f404b8d8bd93de7d900554f4d59e4f749b277797cd315a63d9df024fbfR20
Since clients already do submission.assets and expect only images in response - I filter by images by default. It's possible to fetch additional files like this:
{
  submission(id: "246195") {
    id
    assets(assetType: [ADDITIONAL_FILE]) {
      id
      filename
      imageUrls
      documentPath
    }
  }
}

feedback question @olerichter00 @dblandin @joeyAghion: was "code walkthrough" in PR description useful (did it make code review easier) or I just waster mine and yours time?

Definitely not a waste of time! I found the notes useful to orient myself before jumping into the code. Very much a fan of thorough PR descriptions to offer that context.
Although once I jump to the diff mode, I'm unlikely to switch back to the PR description so I think file or line-specific "code tour" comments might have been more useful to have in the diff as comments rather than in the PR description.

feedback question @olerichter00 @dblandin @joeyAghion: was "code walkthrough" in PR description useful (did it make code review easier) or I just waster mine and yours time?

Always a good idea! And I agree with Devon, and maybe one entry point and links from comment to comment could make it easier to stay in the diff mode.


tekton: add tag to pipeline ref

to enable renovate to auto update the digest

OK to merge. Enterprise contract violations are in trustee-operator.


[Feature Request] Set higher frontend.persistenceMaxQPS default for dev server

Describe the solution you'd like
In 0.11 when we disabled search attribute cache (system.forceSearchAttributesCacheRefreshOnRead=true) and overrode QPS (frontend.persistenceMaxQPS=10000). When we re-disabled SA cache in 0.13.1 in #593, we did not re-override that max QPS setting and users are now hitting it. We should set it as it was in 0.11.

may need to bump history.persistenceMaxQPS as well if it is not already higher.


Cloudflare Data Connector - Parameterize blob download chunk size

Change(s):

Set log level to ERROR for transitive library charset_normalizer
Parameterize chunk size for the container client
Update AzureFunctionCloudflare/main.py in CloudflareConn.zip. Done via$ zip CloudflareConn.zip AzureFunctionCloudflare/main.py
updating: AzureFunctionCloudflare/main.py (deflated 68%)

Reason for Change(s):

When debugging issues, we encountered the logline Encoding detection on empty bytes, assuming utf_8 intention. at least once per processed file, but for large files, the line would be repeated hundreds of times. It did not seem to provide value, and made sifting through logs tedious.
We had multiple log files in the range of 3 to 5 mb in size, and the chunking behavior was not handling that well. Each time the client encountered these log files, we saw the exceptionException while executing function: Functions.AzureFunctionCloudflare Result: Failure
Exception: HttpResponseError: Download stream interrupted.
Stack:   File "/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py", line 461, in _handle__invocation_request
    call_result = await self._run_async_func(
  File "/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py", line 739, in _run_async_func
    return await ExtensionManager.get_async_invocation_wrapper(
  File "/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/extension.py", line 147, in get_async_invocation_wrapper
    result = await function(**args)
  File "/home/site/wwwroot/AzureFunctionCloudflare/main.py", line 55, in main
    await conn.process_blob(blob, container_client, session)
  File "/home/site/wwwroot/AzureFunctionCloudflare/main.py", line 101, in process_blob
    async for chunk in blob_cor.chunks():
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py", line 151, in __anext__
    self._current_content = await self._iter_downloader.yield_chunk(chunk)
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py", line 62, in yield_chunk
    return await self._download_chunk(chunk_start, chunk_end - 1)
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py", line 105, in _download_chunk
    chunk_data = await process_content(response, offset[0], offset[1], self.encryption_options)
  File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/aio/_download_async.py", line 27, in process_content
    raise HttpResponseError(message="Download stream interrupted.", response=data.response, error=error)

This was regardless of the concurrency being set to one or many files. The only remediation was to set the chunk size to be larger than the largest blob in the container.

Version Updated:

Required only for Detections/Analytic Rule templates
No

Testing Completed:

Yes

Checked that the validations are passing and have addressed any issues that are present:

No
I am unsure if that is required since this doesn't touch KQL or Yaml

@lwbrooke please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]

Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

Hi @lwbrooke,
I have reviewed the changes and it looks good to me. As it's a partner supported solution I can't directly merge this PR. Let me check with team on further steps and get back to you.
Thanks for the contribution.
Thanks


Standardise qualitative assessment of content items

We want to standardise the way we measure the quality of content during an audit so that all the content on GOV.UK is comparable.

 Simplify Audit class and questions

A follow up PR will be needed once this is deployed to remove any redundant database tables.


gyb: crashes on startup, complains about missing python module

Describe the bug
Package gyb crashes on execute, complaining of missing python module.
Steps To Reproduce
Steps to reproduce the behavior:
$ nix-shell -p gyb
[nix-shell:~]$ gyb --version
Traceback (most recent call last):
  File "/nix/store/akild48w36i3k7pc6z8nj0qbp51vmzy4-gyb-1.81/bin/.gyb-wrapped", line 87, in <module>
    import google.oauth2.service_account
ModuleNotFoundError: No module named 'google.oauth2'

Oddly enough, trying the import in the python repl works.
[nix-shell:~]$ which python
/nix/store/z7xxy35k7620hs6fn6la5fg2lgklv72l-python3-3.12.4/bin/python

[nix-shell:~]$ python
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google.oauth2.service_account
>>> google.oauth2.service_account
<module 'google.oauth2.service_account' from '/nix/store/hcryx5gig96q4f4dhx3aphyc14x6pj8z-python3.12-google-auth-2.30.0/lib/python3.12/site-packages/google/oauth2/service_account.py'>

Notify maintainers
@austinbutler
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
Reproduced bug on two machines:
Machine 1:
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.5.0, macOS 14.5`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.18.4`
 - channels(grace): `""`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/store/fwsfyb388gd6qfws95whlp77g3wymjxy-source`

Machine 2:
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.0-182-generic, Ubuntu, 20.04.6 LTS (Focal Fossa), nobuild`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.16.1`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/home/eecs/dinh/.nix-defexpr/channels/nixpkgs`

Add a :+1: reaction to issues you find important.

Weird. I'm just running into a different issue, but I can reproduce yours if I use an ephemeral shell.
> nix-shell -p gyb
[nix-shell:~]$ gyb --version
Traceback (most recent call last):
  File "/nix/store/akild48w36i3k7pc6z8nj0qbp51vmzy4-gyb-1.81/bin/.gyb-wrapped", line 87, in <module>
    import google.oauth2.service_account
ModuleNotFoundError: No module named 'google.oauth2'

However I first added it to systemPackages and it just worked.
> gyb --version
Got Your Back 1.81
jaylee.us/gyb
Jay Lee -<EMAIL_ADDRESS>Python 3.12.4 64-bit final
google-api-client 2.135.0
Linux-6.6.35-x86_64-with-glibc2.39 x86_64
Path: /nix/store/akild48w36i3k7pc6z8nj0qbp51vmzy4-gyb-1.81/bin
ConfigPath: /nix/store/akild48w36i3k7pc6z8nj0qbp51vmzy4-gyb-1.81/bin
OpenSSL 3.0.14 4 Jun 2024
gmail.googleapis.com connects using TLSv1.3 TLS_AES_256_GCM_SHA384

So maybe something else in my systemPackages is providing google.oauth2?
In any case, this didn't completely work. After setting things up, gyb tried to write to a json in the binary directory:
...
The authentication flow has completed.
Creating project "Got Your Back Project"...
Checking project status...
 enabling API drive.googleapis.com...
 enabling API gmail.googleapis.com...
 enabling API groupsmigration.googleapis.com...
 enabling API iap.googleapis.com...
 enabling API vault.googleapis.com...
Creating Service Account
ERROR: [Errno 30] Read-only file system: '/nix/store/akild48w36i3k7pc6z8nj0qbp51vmzy4-gyb-1.81/bin/oauth2service.json'

So after a lot of fiddling around with the Google API config, I managed to get a backup started.
The solution for the 'Read-only file system' error is to always explicitly specify the config directory with --config-folder and an appropriate path.
I'm not sure what makes gyb work when installed as a system package, but perhaps that will work for others.

Sounds like this is working as intended? 🤷


[BUG] github actions CI messed up w/ JDK versions intermittently

Describe the bug
https://github.com/NVIDIA/spark-rapids/actions/runs/5685693307/job/15411113728 (should be jdk8 but running w/ jdk11)
not sure about the root cause here, seems a github action bug.
I will try to setup an ENV to see if we could always repo and figure out the root cause

in https://github.com/NVIDIA/spark-rapids/actions/runs/5687245548/job/15415371715
all targeting jdk11, but its still showing error bootstrap class path not set in conjunction with -source 8
openjdk version "11.0.20" 2023-07-18
OpenJDK Runtime Environment Temurin-11.0.20+8 (build 11.0.20+8)
OpenJDK 64-Bit Server VM Temurin-11.0.20+8 (build 11.0.20+8, mixed mode)
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: /usr/share/apache-maven-3.8.8
Java version: 11.0.20, vendor: Eclipse Adoptium, runtime: /opt/hostedtoolcache/Java_Adopt_jdk/11.0.20-8/x64
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.[15](https://github.com/NVIDIA/spark-rapids/actions/runs/5687245548/job/15415371715#step:4:16).0-1041-azure", arch: "amd64", family: "unix"
ENV JAVA_HOME: /opt/hostedtoolcache/Java_Adopt_jdk/11.0.[20](https://github.com/NVIDIA/spark-rapids/actions/runs/5687245548/job/15415371715#step:4:21)-8/x64```

This might be because according to the doc https://github.com/actions/setup-java#install-multiple-jdks

All versions are added to the PATH. The last version will be used and available globally. Other Java versions can be accessed through env variables with such specification as 'JAVA_HOME_{{ MAJOR_VERSION }}_{{ ARCHITECTURE }}'.

we might need to set  JAVA_HOME=JAVA_HOME_{{ java-version }}_AMD64 for Maven to pick up the right one.

we are installing JDK in different github runners, so they should not meet this issue. But if there is a bug or other host-level caching strategy, it may cause the issue

I will keep monitoring the status for a couple of days

seeing sth new in https://github.com/NVIDIA/spark-rapids/actions/runs/5692968262/job/15431096263
  Trying to resolve the latest version from remote
  Error:
408 Request Time-out

  Your browser didn't send a complete request in time.

the error was saying request timeout, but I think there is some glitch w/ github CDH right now

Degraded performance incidents for GitHub services everyday
https://www.githubstatus.com/history

https://www.githubstatus.com/incidents/l59z35rhzdky another incident ongoing 🤷


https://itsyou.online/apidocumentation failed to load

saw this in chrome console
api-console-vendor.js:421835 TypeError: Cannot read property 'buffer' of undefined
    at api-console.js:1124
    at processQueue (api-console-vendor.js:423447)
    at api-console-vendor.js:423463
    at Scope.$eval (api-console-vendor.js:424699)
    at Scope.$digest (api-console-vendor.js:424515)
    at api-console-vendor.js:424738
    at completeOutstandingRequest (api-console-vendor.js:415069)
    at api-console-vendor.js:415457

From past experiences, this would indicate an error in the raml file


Remove Note on the Quick Reference Guide

The text "NOTE: This document forms part of a proposal by the Chronometric Age Extension Task Group to extend Darwin Core with vocabulary for chronometric ages. It is not yet ratified and may change as a result of recommendations received during the course of open public commentary." Should be removed and the document updated after ratification.

Completed with commit 56414338da3400e01faba88d0172618d7126365e.


fix(feature_flags): fix max_age type so mypy will not fail on strict …

In feature flags AppConfigStore, the max_age param is defined as optional with a default value but the type hint does not have optional. This caused me errors when enabling strict mypy configuration.

Codecov Report

Merging #1034 (f6804f3) into develop (00b5b5c) will not change coverage.
The diff coverage is 100.00%.

:exclamation: Current head f6804f3 differs from pull request most recent head e086247. Consider uploading reports for the commit e086247 to get more accurate results

@@           Coverage Diff            @@
##           develop    #1034   +/-   ##
========================================
  Coverage    99.96%   99.96%
========================================
  Files          119      119
  Lines         5365     5365
  Branches       612      612
========================================
  Hits          5363     5363
  Partials         2        2

Impacted Files
Coverage Δ

...da_powertools/utilities/feature_flags/appconfig.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 00b5b5c...e086247. Read the comment docs.

hey @ran-isenberg thanks for sending this over -- Could you provide more details on the mypy output as we don't use strict? An issue would be great!
The PR description is missing our template, are you happy for me to edit and add it? We can't accept otherwise (AWS OSS guidance).
I need to ponder on that a bit more for two reasons: 1/ Slight overhead on branching that wasn't needed before, 2/ We don't use strict mode in Mypy as it can be pedantic and increase the barrier to entry for new folks to contribute, which means we might end up making Mypy happy this time but not consistently over time.
Maybe there's another way we can make a compromise - please send the verbose/pretty output when you get a chance.
Thanks again!

Hey @heitorlessa ,  I stopped working on that mypy project, I'm unable to reproduce the issue at the moment. However, the fix does make sense. You have an argument with a default value, meaning it's optional but it's not marked as such.

Just had another look today with more time - thank you for the patience, though we must close it. I can't reproduce this issue with both mypy default config, our CI config, and with strict config.
I double checked the official Python typing docs that confirms why mypy passes:

Optional is an alias to Union[None, <wrapped type>]
Any default value that isn't None doesn't require Optional

Excerpt from the docs and tests below:

app.py
class A:
    # default value as 5
    def __init__(self, age: int = 5) -> None:
        self.age = age

a = A()

Mypy verbose output - default config w/ strict mode
mypy app.py --strict --verbose

LOG:  Mypy Version:           0.942
LOG:  Config File:            Default
LOG:  Configured Executable:  /Users/lessa/.local/pipx/venvs/mypy/bin/python
LOG:  Current Executable:     /Users/lessa/.local/pipx/venvs/mypy/bin/python
LOG:  Cache Dir:              .mypy_cache
LOG:  Compiled:               True
LOG:  Exclude:                []
LOG:  Found source:           BuildSource(path='app.py', module='app', has_text=False, base_dir='/private/tmp')
LOG:  Metadata fresh for app: file app.py
LOG:  Metadata fresh for builtins: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/builtins.pyi
LOG:  Metadata fresh for sys: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/sys.pyi
LOG:  Metadata fresh for types: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/types.pyi
LOG:  Metadata fresh for _ast: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/_ast.pyi
LOG:  Metadata fresh for _collections_abc: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/_collections_abc.pyi
LOG:  Metadata fresh for _typeshed: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/_typeshed/__init__.pyi
LOG:  Metadata fresh for io: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/io.pyi
LOG:  Metadata fresh for typing: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/typing.pyi
LOG:  Metadata fresh for typing_extensions: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/typing_extensions.pyi
LOG:  Metadata fresh for importlib.abc: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/importlib/abc.pyi
LOG:  Metadata fresh for importlib.machinery: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/importlib/machinery.pyi
LOG:  Metadata fresh for array: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/array.pyi
LOG:  Metadata fresh for ctypes: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/ctypes/__init__.pyi
LOG:  Metadata fresh for mmap: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/mmap.pyi
LOG:  Metadata fresh for os: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/os/__init__.pyi
LOG:  Metadata fresh for codecs: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/codecs.pyi
LOG:  Metadata fresh for collections: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/collections/__init__.pyi
LOG:  Metadata fresh for abc: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/abc.pyi
LOG:  Metadata fresh for importlib: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/importlib/__init__.pyi
LOG:  Metadata fresh for contextlib: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/contextlib.pyi
LOG:  Metadata fresh for subprocess: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/subprocess.pyi
LOG:  Metadata fresh for os.path: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/os/path.pyi
LOG:  Metadata fresh for posixpath: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/posixpath.pyi
LOG:  Metadata fresh for genericpath: file /Users/lessa/.local/pipx/venvs/mypy/lib/python3.7/site-packages/mypy/typeshed/stdlib/genericpath.pyi
LOG:  Loaded graph with 25 nodes (0.016 sec)
LOG:  Found 2 SCCs; largest has 24 nodes
LOG:  2 fresh SCCs (25 nodes) left in queue (and will remain unprocessed)
LOG:  Build finished in 0.021 seconds with 0 modules, and 0 errors
Success: no issues found in 1 source file


How to get all variables while breakpoint in loop

I have this class:
object UnderDebug{

  val d = "ffff"
  def main(args: Array[String]): Unit = {
    while (true) {
      val a = 5
      (1 to 10).foreach { index =>
        val f = index * a
        val g = index + a
        val z = f / g

        println(z)

      }
    }
  }
}

and I load the debuger:
object ReplDebuger {

  def main(args: Array[String]): Unit = {
    // Get the executing class name (remove $ from object class name)
    val klass = UnderDebug.getClass
    val className = klass.getName.replaceAllLiterally("$", "")
    // Add our main class to the classpath used to launch the class
    val classpath = JDITools.jvmClassPath
    val jvmOptions = Seq("-classpath", classpath)

    // Checks if the JDI is available in your runtime
    // classloader or by extracting it from tools.jar
    // found in JDK_HOME or JAVA_HOME
    println("JDI is available: " + JDITools.isJdiAvailable())

    // Loads the JDI from tools.jar and attempts to
    // add it to your system classloader
    println("Loaded JDI: " + JDITools.tryLoadJdi())

    val launchingDebugger = LaunchingDebugger(className = className,
      jvmOptions = jvmOptions,
      suspend = true // Wait to start the main class until after connected)
    )
    import org.scaladebugger.api.profiles.scala210.Scala210DebugProfile.Name
    launchingDebugger.start { s => println("Launched and connected to JVM: " + s.uniqueId)
      val pipeline =
        s.withProfile(org.scaladebugger.api.profiles.scala210.Scala210DebugProfile.Name)
          .getOrCreateBreakpointRequestWithData(JDITools.scalaClassStringToFileString(className), 12, CustomPropertyDataRequest("f"))

      pipeline.foreach{e =>
        val path = e._1.location.sourcePath
        val line = e._1.location.lineNumber

        println(s"Reached breakpoint for $path:$line")
        launchingDebugger.stop()
      }

    }

I do not get any values of the local variables: f and g
What I am doing wrong?

Sometimes the JVM transforms those variables into inline representations. Other times it turns them into methods that are invoked. What this says to me is that they are not avaiable as local variables, but maybe they are exposed in some other form. Have you tried connecting sdb to your code above and printing out variable information including the contents of this to see if you can find the variables?
The profile abstractions for Scala need some work to get them to be a bit cleaner.
You can look at this test, which debugs Variables.scala to see examples of how data is parsed. It uses the top frame of the thread to get a list of variables. Try doing that in your pipeline to print out the list of variables and see if "f" and "g" are there.

Thanks


What is the best way to train large models?

Hi all!
I was wondering how I could use this library with large models that don't fit on a single gpu?
For non-private fine-tuning I managed to get it working with FSDP and also I have put together a rough script that applies DP with FSDP, I have recently also written on the opacus forum here.
Do you know of any better way to do this? because when I try to call attach on the optimizer it throws an error saying that the FSDP layer is not supported, if it's helpful I can provide an MVP of the error.
Thank you a lot for your time,
Pier

For large models that do not fit on a single GPU, it may be the easiest for our library to work with DeepSpeed and FSDP. This requires some changes on both the DeepSpeed part and this library. I can share some thoughts here before we actually make the change.
One key observation is that DeepSpeed and fastDP both change the optimizer.step() function, though in different ways, so we want only one library to do the changes. Also have you tried mixed precision?

The best way to train large DP models is by ZeRO (e.g. in FSDP and DeepSpeed). We have developed this in an upcoming paper "Zero redundancy distributed learning with differential privacy" and will open-source the code soon. Stay tuned!


Spot instance tags issue

Flintrock version: Master version
Python version: 3.8.10
OS: Mac M1

Hi !
It seems that there is an issue regarding TagSpecifications in Make finding cluster nodes more reliable in the face of launch failures. As we are interested in this new feature, we tried to make a release in our forked version but we got this issue:
Parameter validation failed:
Unknown parameter in LaunchSpecification: "TagSpecifications", must be one of: SecurityGroupIds, SecurityGroups, AddressingType, BlockDeviceMappings, EbsOptimized, IamInstanceProfile, ImageId, InstanceType, KernelId, KeyName, Monitoring, NetworkInterfaces, Placement, RamdiskId, SubnetId, UserData

If we are not wrong aout this issue, it seems that this issue comes from TagSpecifications in common_launch_specs that is a parameter of request_spot_instances and not a key inside LaunchSpecification dict as you can see here. Also, we can't put ResourceType': 'instance' in client.request_spot_instances as explained in the documentation:
TagSpecifications (list) --
The key-value pair for tagging the Spot Instance request on creation. The value for ResourceType must be spot-instances-request , otherwise the Spot Instance request fails. To tag the Spot Instance request after it has been created, see CreateTags.

So, I think, we need to put ResourceType': 'spot-instances-request' if we want to tag the spot request instance (not spot instance), if not we can remove TagSpecifications from spot request instance.
Here is a possible solution in _create_instances:

Remove TagSpecifications from common_launch_specs
Add TagSpecifications=[{'ResourceType': 'spot-instances-request', 'Tags': tag_specifications}], in client.request_spot_instances
Add TagSpecifications=[{'ResourceType': 'instance', 'Tags': tag_specifications}], in ec2.create_instances
Add tags to spot instances using create_tags as last step inside if spot_price::

(ec2.instances
             .filter(
                Filters=[
                    {'Name': 'instance-id', 'Values': [instance.id for instance in cluster_instances]}
                ])
             .create_tags(Tags=tag_specifications))

We have done these modifications in our fork, so if you're okay with that, we can do the same here or if you have maybe a more elegant way to do that it could be interesting.
I hope it's clear and this helps !
Maxime

Thanks for the clear report. I appreciate it.
I did not think to test spot instances as I worked on #362. Apologies.
I would welcome a PR to fix this issue. However, I would not want to re-introduce tagging as a separate step, as you proposed in step (4) above. This recreates the original problem of cluster nodes becoming orphaned if something goes wrong between when they are launched and when they are tagged.
We want tagging to happen as part of the same AWS API call that creates the instances, whether they are spot or not.
Is that not possible with spot instances? i.e. Are tags for spot instance requests not automatically also tags for the spot instances themselves?

Hi,
Ok it's more clear now why you don't use create_tags anymore.
But when we tried to launch spot instances with TagSpecifications using request_spot_instances, the request is tagged but instances doesn't inherit from tags of the request. So it seems, it's not possible to tag spot instances during the same call that creates instances.

Looks like the boto3 method we use to launch spot instances is deprecated, and the solution is to migrate off that method.

We strongly discourage using the RequestSpotInstances API because it is a legacy API with no planned investment.

Oh actually I hadn't seen that ! If we follow AWS recommandation to launch spot instances Which is the best Spot request method to use?, it seems that we need to use create_fleet. I don't know exactly what is the impact of this change.. But regarding our Tag problem, it seems that create_fleet support instance tagging, so we could have a single AWS API call that launches the instances and tag them as described in the documentation:
TagSpecifications (list) –

The key-value pair for tagging the EC2 Fleet request on creation. For more information, see [Tag your resources](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-resources).

If the fleet type is instant, specify a resource type of fleet to tag the fleet or instance to tag the instances at launch.

If the fleet type is maintain or request, specify a resource type of fleet to tag the fleet. You cannot specify a resource type of instance. To tag instances at launch, specify the tags in a [launch template](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html#create-launch-template).

I thought create_instances is the API we need to use, actually. We are already using it to launch on-demand instances. It just needs different parameters to configure it for spot instances. That way we are using a single launch API across Flintrock. (Note that RunInstances from the AWS docs maps to create_instances in boto3.)
That was my plan, but I only took a quick look and plan to come back to this later. I think in the future we could migrate from create_instances to create_fleet if we want to support launching clusters with different master/slave instance types, but that's not required for now.
If you want to take a crack at this, go ahead. Otherwise, I will try to get to this next week.


Using Github enterprise, does not import github teams

What kind of request is this (question/bug/enhancement/feature request):
bug?
Steps to reproduce (least amount of steps as possible):
using Github enterprise

create a team in an organisation, call it Avengers
add some people

In rancher

setup authentication to use Github Enterprise
For Site Access, it does not matter the scope of lock down, we tried all 3 options

go to the cluster

add a project
try to add the Avengers group

Result:
the group does not appear in the possible type-ahead drop down. no group does
organisations and users do show
Other details that may be helpful:
this works as expected with Github.com
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): 2.3.2
Installation option (single install/HA): HA

solved it
we need to click the dropdown on the right


value::emplace takes arguments

This adds arguments to value::emplace

Closing this because the algorithm is less efficient than using the noexcept emplace with no args followed by a resize (e.g. for arrays)


Outputting NULL

The two files (Freenom.class.php and example.php) uploaded and tested, but only outputting "NULL"?

Hello,
I have checked the code and apparently there was a line that wasn't needed in Freenom.class.php at #343, in the JSON decoding function.
I have commented it and tested the example php.
Thank you for your feedback!


Example with dependency to mysql-connector-java which is licensed via GPLv2

The Community release of MySQL Connector/J 8.x is licensed via GPLv2:
https://github.com/mysql/mysql-connector-j/blob/release/8.x/LICENSE
Also the Azure Database for MySQL is based on the MySQL Community Edition

Azure Database for MySQL is a relational database service in the Microsoft cloud based on the MySQL Community Edition (available under the GPLv2 license) database engine, versions 5.6 (retired), 5.7, and 8.0.

Source: https://learn.microsoft.com/en-us/azure/mysql/single-server/overview
The example on this and other pages within https://learn.microsoft.com/en-us/azure/mysql are adding the dependency using this code within the pom.xml:
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>8.0.30</version>
</dependency>

If a developer is following this example, would this not put the whole project the developer is working on under GPL license?
In another example by Micrsoft the dependency is added differently:
<dependency>
	<groupId>mysql</groupId>
	<artifactId>mysql-connector-java</artifactId>
	<scope>runtime</scope>
</dependency>

Source: https://github.com/Azure-Samples/quickstart-spring-data-jdbc-mysql/blob/master/pom.xml
Would this solve the issue?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e49103a3-ef0f-f28a-05f7-5d8019845f9a
Version Independent ID: 50411faa-7eab-ad50-8a99-7406a9e71db9
Content: Quickstart: Use Java and JDBC with Azure Database for MySQL
Content Source: articles/mysql/single-server/connect-java.md
Service: mysql
Sub-service: single-server
GitHub Login: @jdubois
Microsoft Alias: judubois

@Nikemare Thank you for your feedback. We will review and make an update if appropriate.


How to use the BLIP synthetic caption?

I found that there is a content about blip caption in the introduction of pretrain data. I would like to ask how to combine the original CC3M data caption with the blip caption, and how to use it?
Thanks for your time!

Note that we only used the original CC3M data caption in our v0 release.  If you'd like to combine them, you may directly modify the annotation file, and include the response as "GPT".

I see. Thank you very much


fix(agora): make booklet link visible after deadline

:tada: This PR is included in version 1.33.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[Bug] Module "@heroicons/react/24/outline" has no exported member 'ChevronUpDownIcon'

I'm trying to use the ChevronUpDownIcon in v2.0.0 (formerly known as the SelectorIcon in v1.0.6), but when I try to import the icon into my React app I get a Typescript error that the icon is not exported from the @heroicons/react/24/outline module.
I get the same error when trying to import the icon from @heroicons/react/24/solid.
This is the only icon I've seen this issue with so far. I believe it might be an issue with the package.
Steps to Reproduce

Initialize a new React application
Install<EMAIL_ADDRESS>Try to import the ChevronUpDownIcon in your app

This is because you need to install the latest version of v2. The ChevronUpDownIcon was added back in v2.0.3. I recommend using the latest tag in NPM which will use a update your package.json to the latest version, use a semver caret, and install it (npm install @heroicons/react@latest).


Data Science with Python : Ensemble Methods

Welcome to 'DSWP' Team, good to see you here
This issue will helps readers in giving all the guidance that one needs to learn about Ensemble Methods. Tutorial to Ensemble Methods and how it's applied.
To get assigned to this issue, add your serial numbers mentioned in the spreadsheet of "Data Science with Python", the approach one would follow and choice you prefer (Documentation, Audio, Video). You can go with all three or any number of options you're interested to work on.
If you had referred any resources, add them up in "DS Resources". Similarly if you had used datasets, include them in "DS Datasets".
Domain : Machine Learning
Mentors Assigned : Shivang Dubey / Yash Goel
Points to Note :
The issues will be assigned on a first come first serve basis, 1 Issue == 1 PR.
"Issue Title" and "PR Title should be the same. Include issue number along with it.
Changes should be made inside the Datascience_With_Python/ directory & Datascience_With_Python branch.
Follow Contributing Guidelines & Code of Conduct before start Contributing.
This issue is only for 'GWOC' contributors of 'DSWP' domain.
All the best. Enjoy your open source journey ahead. 😎

I've made a video on exploratory data analysis in the past. Will this video mentioned below will be fine?
https://www.youtube.com/watch?v=q6oCG1aNVk8

S.no 609
I would like to work on documentation part of this issue.
Approach:
Explain what are ensemble methods. why do we need them.
What are decision trees? How Random forests work.
Some other types of methods like Voting Classifier, Bagging Classifier, Extra Tree Classifier
All this with the help of a dataset.

Issue is assigned to @nakshatra-garg for documentation.

I am bhaswati roy from dswp batch 19
I would like to contribute for this.

I think you can close this issue now? @prathimacode-hub

Its still open for audio or video @nakshatra-garg

Hello, I am Deepthi M with serial number:172, Batch-5. I would like to do audio on this issue.
My approach: a detailed explanation

Issue assigned to @deepthi1107 for audio contribution


pause showing toasts

Is it possible to pause showing toasts and keep them on the queue until for example the drawer is closed

No Option like that only way is to wait till the drawer is closed and then trigger the notification


[Feature] Support middleware as shared objects

Go 1.8 enables plugins (as shared objects), so instead of (or as an alternative) creating separate binary that represents regular functions but with additional middleware inside support middleware as *.so objects.
Required changes:

[ ] Provide unified interface for registering middleware from *.so
[ ] Make server to lookup specific directory (~/.middleware, etc.) for *.so files, try to load them and search for a functions

func AddMiddleware(server *Server) {
    server.AddMiddleware(&CustomMiddleware{})
}

inside each *.so object under ~/.middleware directory
Nice to have:

[ ] extract middleware framework from iron-io/functions into its own package and repo

iron-io/functions as dependency brings a lot code that might appear redundant for middleware development (see how may dependencies it brings at https://github.com/denismakogon/functions-keystonemiddleware/blob/master/glide.lock).

Refer to https://github.com/iron-io/functions/pull/431 for an historical perspective of such proposal.


More kbs to develop

@KataBoku @istvan-satmari I have tested this. An as far as claimed fixes are concerned, it works for me.
Please, @istvan-satmari , check it for yourself and merge as soon as possible.


Explicitly re-export models

Is your feature request related to a problem? Please describe.
Commonly installed python linter pyright complains about implicit reexport

While hera/workflows/models/init.py imports all the deeper models, it does not add them to an __all__ block.
Describe the solution you'd like
Ideally all the models are explicitly reexported, so linters dont complain about it
Describe alternatives you've considered
Perhaps there's a way of configuring pyright to not care about this, but it seems beneficial to fix the default behavior here. particularly in this case where the file seems to be autogenerated and defining the explicit reexport block ought to be relatively straightforward?

I'd be down to contribute the fix if interested. It's this file?

I'd be down to contribute the fix if interested. It's this file?

That's definitely the right place for it! Thanks for bringing this up!


Improvements: Add [AssetOnly] / [SceneOnly] attributes, add [SceneAsset] attribute

Add [AssetOnly] / [SceneOnly] attributes (as in Odin)
So if you have a slot in inspector is for a prefab, you can put only prefab from Project view there, and couldn't put a scene Gameobjects.
Conversely, if you have a slot for a scene objects only, you can't put assets from Project there.
Add serialization for Scene asset.
I want to have an ability to put a Scene in a slot. So I can load scene not by name but by scene asset reference directly.

There is already a pull request for this feature: #111


Support "State Objects"

With shader models 6.3 and later, applications have the convenience and flexibility of being able to define DXR state objects directly in HLSL shader code in addition to using Direct3D 12 APIs.
This is called "State objects" in HLSL, and slang doesn't seem to recognize them.
This is required for full support of WorkGraphs.
The syntax is
TYPE name = { field1, field2, ... };

where TYPE can be one of the followings,

StateObjectConfig
GlobalRootSignature
LocalRootSignature
SubobjectToExportsAssocation
RaytracingShaderConfig
RaytracingPipelineConfig
TriangleHitGroup
ProceduralPrimitiveHitGroup

One of the examples that uses GlobalRootSignature can be found from here.
GlobalRootSignature globalRS = { "UAV(u0)" };

we will need a solution to generalize this to all different targets. Also need to check how spirv/vulkan plans to represent this.

It turned out that this is not needed for supporting WorkGraph yet.
There is a proposal to add more nodes to a node array at runtime via "AddToStateObject()".
But it is not supported; in other words, it is not a part of SM 6.8.
See the document for more details if needed.

A proposal not supported yet is allowing the AddToStateObject() API that currently exists for DXR to be extended to support cheaply adding leaf nodes to an existing array in a work graph after the work graph has been created.


Performance regression on LatLonPoint#newPolygonQuery

Description
I just notice a big performance regression on polygon queries using LatLonPoint field in lucene geo benchmarks:

I checked and the regression was introduced by this change: https://github.com/apache/lucene/pull/1017.
My suspicion is that before this change, SpatialQuery was calling the method #getSpatialVisitor() once for the whole index but in the new version is calling it once per segment. This method might be expensive for LatLonPoint queries, threfore the regression.
@nknize FYI
Version and environment details
No response

Just seeing this. That's exactly what it would be! Snuck in one of those last commits on the long running PR. Thanks for refactoring and merging  @iverase!

Fix seems to bring performance back to previous levels:

Thanks for catching this @iverase and the quick fix, and the follow-on issue to better detect such regressions before release: #11827

What's annoying is how incredibly trappy this override logic is. That a method call literally moving from createWeight to getScorerSupplier results in a 72.2% regression even slipped by me before merge doesn't bode well for new committers. And then it sat in regression until an entire company was interested in releasing.
I wonder if we can do better? Like maybe figure out better guardrails in these methods? Perhaps by something as simple as a rename to signal one happens per segment? This isn't the first and will certainly not be the last time an expensive operation accidentally slips to a critical path. Any other ideas how to lower the bar here for new committers?


[FEAT] 868 Meta Response (dynamic input)

Pull Request Type

[x] ✨ feat
[x] 💄 style
[x] 📝 docs

Relevant Issues
This pull request resolves #868
What is in this change?

This change introduces a dynamic input method designed to enhance user experience by enabling intuitive and context-sensitive interactions. It leverages an autogenerated JSON meta object embedded within the response, guiding the front-end on how to render various input interfaces such as options, suggestions, or sliders, moving beyond traditional text inputs.
Key Components:

Meta Object Schema: Defines the structure for input types (e.g., options, range, multi-select checkboxes) and the data needed for rendering.
JSON Object Extractor: Extracts the meta JSON object from responses, converting it to an object passed to chatHistory.
Input Handler Module (DynamicInput): A flexible component that reads the meta object and dynamically changes the user input method based on its instructions.
Rendering Components: A set of components like OptionsSelect, containing dropdown menus, lists, buttons, etc., with more components to be added.
Settings: Enable the Meta response submenu from the chat settings, as shown in this Figma file: Figma Design.

The system is designed with modularity and scalability in mind, featuring a library of standalone input method components and a schema for the meta object that supports future expansion.
Example JSON Meta Object:
{
    "inputs": {
        "type": "options",
        "data": {
            "options": [
                {"label": "Restart Router", "value": "restart router"},
                {"label": "Check Service Status", "value": "check service status"}
                // Additional options...
            ],
            "label": "Select Server",
            "description": "List of servers as described"
        },
        "settings": {
            "allowMultiple": false,
            "displayType": "choose one: buttons/list/dropdown"
        }
    },
    "sentiment": "happy",
    "style": "text"
}

Screen shots:
Meta response settings area (in development ):

Chat area screen shots (Done ✓):

Additional Information
Add any other context about the Pull Request here that was not captured above.
Developer Validations

[ ] I ran yarn lint from the root of the repo & committed changes
[ ] Relevant documentation has been updated
[ ] I have tested my code functionality
[ ] Docker build succeeds locally

This is really great. I will say the only critique i have currently since this is in Draft is the UI. Once this is ready for review ill task our designer with overhauling this because some elements are not in spec with our design philiophsy and general design patterns. This is not something to overly concern yourself with as functionality is the most important thing for this PR.
Really awesome work @sherifButt 👏

Thank you @timothycarambat, I'm happy that you liked the "Meta response" feature, to be honest I developed the chat dynamic input element directly with no pre design, as the proof of concept for me was the functionality itself. There is some design involved on that part however it is minimal mostly copping existing elements from the app, and it is fully coded and actorly ready for review if you want. but I thought might be better to code the settings page before issuing a review? what you think?
Regarding the setting pages (second phase), I felt that I need a Figma touch, just to organise the settings menu fast and to imagine how to better control the feature,  just as a layout exercise. I didn't code the setting pages at all yet, however I'm planning to do so in the coming few days. as I'll need to introduce minor db Schema addition on the workspace_threads.
I have Figma design for the setting pages to share if you want to have a look.
I'm happy to adjust my designs to match your design philosophy as they are just to convey the ideas and to get feed back.

@sherifButt I do think that maintaining this feature set as a fork would be best as for certain there are plenty of people who use this branch. We did wind up going in another direction with the UI and while I can clearly see the implementation use-case I do think it falls out of scope for the core team for now.
I intend to revisit this PR in a few months when we have a proper plugin system that can extend the server and frontend UI and then can possibly re-factor or import the functionality you built here back into the main application or at least a way to make so easier without such a code change, but still have all the functionality.  Either way, It may be worthwhile to publish this branch as an image that others can leverage who need this. The merge conflicts as they exist now are not massive and can be rectified to keep the changes up to date and maintain this functionality for those that require it


NTUEEPLUS-22. Register Appearance.

What is this PR for?
The appearance of Register page is done.
What type of PR is it?
[Feature]
Todos
None
What is the Jira issue?
https://ntueeplus.atlassian.net/browse/NTUEEPLUS-22
Screenshots (if appropriate)

@featherchen  @vwvwMM  Please review for me.


Cayman PEPs

Ripe for automation:

Members of the House (legislativeassembly.ky) elected and the previous term
Under the Judicial Admin section has the various Judges: Cayman Islands Judicial & Legal Website | An official website of the Cayman Islands Government – Cayman Islands Judicial & Legal Information Website

Perhaps less ready for automation:

Board of Directors for the regulator which is a statutory Authority The Board of the Cayman Island Monetary Authority â€“ CIMA
List of government statutory authority and Government Companies: Statutory Authorities | Cayman Islands Government

Questions:

Members of the House (legislativeassembly.ky) elected and the previous term

We have only the following information:

Name
Position (e.g. Premier)
Full position (e.g. Premier and Minister for Sustainability and Climate Resiliency and Finance & Economic Development Member of Parliament for Newlands)

you want for all positions? Speaker of Parliament, Premier, Deputy Premier, Cabinet Ministers, Ex Officio Members, and the scrolling down on the first link Elected Members > Government Backbench Members, Opposition Members and Independent Opposition Member.
And we would add the Occupancy for the positions above terms.

Under the Judicial Admin section has the various Judges: Cayman Islands Judicial & Legal Website | An official website of the Cayman Islands Government – Cayman Islands Judicial & Legal Information Website

Are you interested only in the judges on https://www.judicial.ky/judicial-administration/judges, both current as retired ones? Which information to get? The only information that everyone have is the full name. Some have the nationality, some have the birth year, but there is no standard in how this information is displayed. Some are in a block of text and some are in a bullet list.

Board of Directors for the regulator which is a statutory Authority The Board of the Cayman Island Monetary Authority â€“ CIMA

I only see the full name and a block of text with extra information on each individual director page (e.g. https://www.cima.ky/ms-helen-dombowsky)

List of government statutory authority and Government Companies: Statutory Authorities | Cayman Islands Government

What to get, just the list of names, url's and tel? Some of the url's return 404 by the way.

Hey @aivuk :) I think if we pull in the judges and the House Assembly, that's enough for now. Names and positions will hopefully make this enough to reconcile with other data sources where we can then get more info (e.g. DOB). It's not great data, but we're playing for coverage here.

hello!

Assembly

Positions Member of parliament, Minister of X, Minister of Y, Speaker of Parliament, Premier, Deputy Premier

leader/deputy leader of opposition is nice to have but not essential
no need to capture their constituency
no need to capture back bench, ex officio, oppostion, independent, etc

yes occupancies to link the people and positions

Judiciary
Judges only, like Friedrich said

Only current, since it's not really shown in the data when they retired and we already know they're not current.
names, role.judge topic, sourceUrl is good yeah.

@pudo should judges get both the judge and the pep topic?

Done, closing.


LIME-713 - Added tests.

Proposed changes
What changed

Tests added to reflect current date validation implementation.

Issue tracking

LIME-713

Closing PR tests are testing hmpo form validator rather than DL CRI


Can't add comment

I'm calling createTorrent with options that include a comment field
I checked that this line runs--so it's setting torrent.comment
https://github.com/feross/create-torrent/blob/4df6bffeb346d38394d85b1e26330cbd3d24008c/index.js#L366
However, when I run parse-torrent on the resulting torrent file, the comment isn't there.

I can't reproduce this. It works fine for me.

Works fine from the command line:
$ create-torrent --comment hi index.js -o t.torrent
$ parse-torrent t.torrent | json comment
hi

So, this is probably a webtorrent-desktop specific issue.

Fixed in parse-torrent-file v3.3.8.


python: h5py update and new package pkgconfig

Since I think 2.4 h5py introduced a new way to configure mpi. Therefore,
the BuildFlags are removed.
I built h5py and h5py-mpi packages successfully. Not sure though whether
the mpi version does actually work correctly since I don't use it.

if h5py is used as a library,we should pull it into python-packages.nix

is it okay to have a named function in python-packages.nix, since we still want to have (at least) two different derivations, h5py and h5py-mpi.

Yeah we do the same for sqlalchemy
On Sun, 25 Oct 2015, 14:29 Frederik Rietdijk<EMAIL_ADDRESS>wrote:

is it okay to have a named function in python-packages.nix, since we still
want to have (at least) two different derivations, h5py and h5py-mpi.
—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nixpkgs/pull/10601#issuecomment-150923409.

Okay, I'll have a look at it.
Relevant issue regarding pkgconfig: https://github.com/matze/pkgconfig/issues/9

@domenkozar could you merge this?


chore: using workspace.package to centralize configs

when running cargo release [dry-run] we can see it works fine, changing only the root Cargo.toml
cargo release version alpha -v
   Upgrading workspace to version 0.1.0-alpha.14
[2024-08-12T10:58:24Z DEBUG cargo_release::ops::cargo] change:
    --- /home/heliofrota/Desktop/tc/trustify/Cargo.toml	original
    +++ /home/heliofrota/Desktop/tc/trustify/Cargo.toml	updated
    @@ -20,7 +20,7 @@
     ]

     [workspace.package]
    -version = "0.1.0-alpha.13"
    +version = "0.1.0-alpha.14"
     edition = "2021"
     publish = false
     license = "Apache-2.0"

   Upgrading trustify-cvss from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-migration from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-common from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-entity from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-module-storage from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-module-ingestor from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-test-context from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-auth from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-infrastructure from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-module-importer from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-module-ui from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-module-fundamental from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-module-graphql from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-server from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading trustify-trustd from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)
   Upgrading xtask from 0.1.0-alpha.13 to 0.1.0-alpha.14 (inherited from workspace)

CI is saying nothing about the build failure.
Running cargo xtask validate-openapi --export locally I can see this diff on openapi.yaml:

➜  trustify git:(ws-pgk-table) ✗ git status
On branch ws-pgk-table
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   openapi.yaml


Upgrade to 1.7.0: Config undefined results in error

https://github.com/poteto/ember-cli-flash/blob/e86f47dc963c5711d590a4ba12ca4d7a284fde4a/addon/services/flash-messages.js#L119
This property throws an error in my app's tests
Error: Assertion Failed: Cannot call get with 'flashMessageDefaults' on an undefined object.
It seems that resolving the config from the registry returns undefined in this version.

Seems to be a problem in unit tests only.  But we need to add a needs for 'config:environment' seems to fix the issue. Opened PR #290

#290 has been merged, does that help with this issue? Can this be closed, @mattgraham1995?

Yes that works.


Trigger registers accessibility in debug mode

Per the current definition, the trigger registers are always accessible in debug mode to allow a debugger with S-mode privilege to set up triggers. This scheme might introduce extra hardware complexity since the trigger registers bypass privilege-based CSR access rule checks in debug mode. There are other candidates for trigger usage:

The debugger sends a message to M-mode software to set up the trigger.
Add a set of shadow CSRs in S-mode to allow the debugger to set triggers.

It is desired to choose a scheme that balances hardware complexity and usability.

The first option, a "DBI" (debug binary interface), seems complex.  I'm not sure how the debugger could invoke M-mode to configure the triggers.  There are no traps from debug mode today, and the debugger can't set dcsr.prv=M when mdbgen=0.
The second option wouldn't be that hard.  There would be no new CSRs, just a set of indirect CSR indices that allow S-mode to access the trigger CSRs via siselect/sireg* (minus the m bits).  And there would be an Smstateen bit that controls S-mode access to these indirect CSRs.  There would be something similar for VS-mode.  There are several extensions that have already done something similar, so as long as the definition hews closely to those it should be non-controversial.  I could help with this part.

I'll add that the second option would also benefit in-target debuggers, eliminating the need for an SBI (and the associated overhead) to use triggers.

Regarding option 2, the overhead for debugger is to set debug privilege to S-mode (when halted in U-mode), and program the delegated trigger. But it isnt heavy.
It seems delegating trigger to S-mode could be a separate extension to debug spec and the main use case for it is to enable OS to set up triggers fast and external debug security also benefits from it.  Probably it's better to have another spec for it instead of adding it in external debug spec if we opt for this approach.

Yes, that would be great. Thank you for taking the initiative!

close the issue as Smtdeleg/Sstcfg proposed by Beeman was approved by as a fast-track: https://lists.riscv.org/g/tech-external-debug-security/topic/risc_v_tech_chairs_arc/108528941


Add delegating spender functionality to AccountAllowanceApproveTX

Problem
There is a concept, introduced in HIP-336, for a delegating spender account to be able to give allowances to other accounts (concerns only NFT allowances)
Basically, an account, which is given the allowance for all serials of a given NFT collection, can give allowances for single serial number NFT of the same collection to other spender accounts
Some additional context
Solution
Add the missing functionality to the AccountAllowanceApproveTX which sets the delegating_sender field of the NftAllowances proto. In that way, an account, which already has the approveForAll allowance to a NFT collection, can give individual allowances to other spender accounts
Alternatives
No response

Fixed by #1349


[Feature Request] Race weekend dates in lieu of Countdown?

Fantastic card 1stly.
I was wondering if there was the ability to just have the date of the race weekend instead of an active countdown?
I have had a look in the toggles, etc and it appears that its merely just an on/off
I have a card that I use which is for a different sport, and like the cleanness of having just the team picture and the date of the next game/etc.
I was hoping to recreate this with the track image and the date of the next race weekend (i guess whatever the countdown actually counts down too)
The race information is quite bulky, nice to have in a separate F1 card (which is what I have setup) but i like my home default card to be as sleek and to the point as possible, hence the suggestion/request
This could work like an on/off toggle much like the countdown timer.

Take a look at countdown_type in the documentation :)

This will still provide just a countdown.
What I was hoping for is for it to just provide a 'date' in that countdown section
so instead of 27D 16H....
It would just say the date&time like 26th August 2023 3pm

Oh right. You are look for the next_race card I guess?

Yeah but I find that the next race card information is huge with every bit about the race weekend. That is great for a dedicated card but as a quick home tab card it would be nicer to have just the race weekend date/time as a single line and the track

Hmm I don't see the added value for it right now. But it will go on the to do list if more people ask for it.
For now, you could use this sensor :
https://community.home-assistant.io/t/formula-one-card/476902/91?u=marcokreeft87
And show that value in a card?

I was thinking that you could use a bunch of the countdown code and have it display the date instead of the countdown. This could be a toggle to apply a conversion such that it would display the countdown information or the value that the countdown timer is actually counting down too.
You could then allow for the countdown to have a toggle for the 'track image' true/false
I will have a look and see what else is out there regarding ICS, etc. Just thought I would ask if it was going to be something somewhat easier to implement.
Again awesome card though

Unfortunately it's not that easy since the card is build for the countdown right now. I also try to keep it as clean as possible so this would probably be another card. I mean if it's now counting down, it's not a countdown card right?
But if someone else asks for this I will put it on the todo list.

@Joshndroid I could let it go. Check out version 1.8.2 :)

Love your work @marcokreeft87 !!


Allow compilation on older gcc versions

What problem is this PR intended to solve?
Compiling Nokogiri with older versions of GCC and centos causes the following error:
12:06:09 checking for gumbo_parse_with_options() in nokogiri_gumbo.h... yes
12:06:09 checking for xmlHasFeature()... yes
12:06:09 checking for xmlFirstElementChild()... yes
12:06:09 checking for xmlRelaxNGSetParserStructuredErrors()... yes
12:06:09 checking for xmlRelaxNGSetValidStructuredErrors()... yes
12:06:09 checking for xmlSchemaSetValidStructuredErrors()... yes
12:06:09 checking for xmlSchemaSetParserStructuredErrors()... yes
12:06:09 checking for rb_category_warning()... yes
12:06:09 checking for whether
12:06:09 -DNOKOGIRI_OTHER_LIBRARY_VERSIONS="\"libgumbo:1.0.0-nokogiri\"" is accepted as
12:06:09 CPPFLAGS... yes
12:06:09 creating Makefile
12:06:09
12:06:09 current directory:
12:06:09 /opt/dev/embedded/lib/ruby/gems/3.0.0/gems/nokogiri-1.16.0/ext/nokogiri
12:06:09 make DESTDIR\= clean
12:06:09
12:06:09 current directory:
12:06:09 /opt/dev/embedded/lib/ruby/gems/3.0.0/gems/nokogiri-1.16.0/ext/nokogiri
12:06:09 make DESTDIR\=
12:06:09 compiling gumbo.c
12:06:09 compiling html4_document.c
12:06:09 compiling html4_element_description.c
12:06:09 compiling html4_entity_lookup.c
12:06:09 compiling html4_sax_parser_context.c
12:06:09 compiling html4_sax_push_parser.c
12:06:09 compiling libxml2_backwards_compat.c
12:06:09 compiling nokogiri.c
12:06:09 compiling test_global_handlers.c
12:06:09 compiling xml_attr.c
12:06:09 compiling xml_attribute_decl.c
12:06:09 compiling xml_cdata.c
12:06:09 compiling xml_comment.c
12:06:09 compiling xml_document.c
12:06:09 xml_document.c: In function 'dealloc':
12:06:09 xml_document.c:77: error: #pragma GCC diagnostic not allowed inside functions
12:06:09 xml_document.c:78: error: #pragma GCC diagnostic not allowed inside functions
12:06:09 xml_document.c:90: warning: '__xmlDeregisterNodeDefaultValue' is deprecated
12:06:09 (declared at
12:06:09 /opt/dev/embedded/lib/ruby/gems/3.0.0/gems/nokogiri-1.16.0/ports/x86_64-linux/libxml2/2.12.3/include/libxml2/libxml/tree.h:684)
12:06:09 xml_document.c:93: error: #pragma GCC diagnostic not allowed inside functions
12:06:09 make: *** [xml_document.o] Error 1
12:06:09
12:06:09 make failed, exit code 2
12:06:09
12:06:09 Gem files will remain installed in
12:06:09 /opt/dev/embedded/lib/ruby/gems/3.0.0/gems/nokogiri-1.16.0 for
12:06:09 inspection.
12:06:09 Results logged to
12:06:09 /opt/dev/embedded/lib/ruby/gems/3.0.0/extensions/x86_64-linux/3.0.0/nokogiri-1.16.0/gem_make.out
12:06:09
12:06:09 An error occurred while installing nokogiri (1.16.0), and Bundler cannot
12:06:09 continue.
12:06:09 Make sure that `gem install nokogiri -v '1.16.0' --source
12:06:09 'https://rubygems.org/'` succeeds before bundling.
12:06:09
12:06:09 In Gemfile:
12:06:09   factory_bot_rails was resolved to 6.2.0, which depends on
12:06:09     railties was resolved to 7.0.8, which depends on
12:06:09       actionpack was resolved to 7.0.8, which depends on
12:06:09         actionview was resolved to 7.0.8, which depends on
12:06:09           rails-dom-testing was resolved to 2.2.0, which depends on
12:06:09             nokogiri

Have you included adequate test coverage?

No tests have been added; the solution implemented is similar to https://github.com/redis/redis/issues/5394 - and has been verified as working locally in a fresh docker container
Does this change affect the behavior of either the C or the Java implementations?

Impacts the C behavior

This looks like the right approach to me. Running the CI now.
I'm not sure how to tell which version of GCC support which pragmas where though.

I believe GCC versions prior to 4.6 are impacted - https://gcc.gnu.org/gcc-4.6/changes.html

New Languages and Language specific improvements
C family

...
Support for selectively enabling and disabling warnings via #pragma GCC diagnostic has been added. For instance:

#pragma GCC diagnostic error "-Wuninitialized"
 foo(a);			/* error is given for this one */
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wuninitialized"
 foo(b);			/* no diagnostic for this one */
#pragma GCC diagnostic pop
 foo(c);			/* error is given for this one */
#pragma GCC diagnostic pop
 foo(d);			/* depends on command-line options */

I aligned with the approach taken by Redis for simplicity; Although I think that this could be tightened up a bit to check for minor version too
# https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html
/* Test for GCC > 3.2.0 */
#if __GNUC__ > 3 || \
    (__GNUC__ == 3 && (__GNUC_MINOR__ > 2 || \
                       (__GNUC_MINOR__ == 2 && \
                        __GNUC_PATCHLEVEL__ > 0))

I'm happy to go with whatever check you think is best - I just know the currently suggested fix works for centos

I'd love to be able to test this manually, but I couldn't easily find a docker image with gcc 4.6 or earlier. Do you know of one you can point me at? The centos:centos6 image will no longer yum update, and the gcc:4.6 image has old CA certs so I can't git clone.
I mean, I'm happy to merge this without testing it, too, if you're sure this solves your problem. Let me know?

@flavorjones These replication steps should hopefully work

Git clone nokogiri on the host machine - or it can be git cloned inside the docker container
In the same directory run the docker container. It's centos 6 with gcc (GCC) 4.4.7

# Run the container from the same directory as nokogiri
docker run --rm  -i --tty -u jenkins --volume=$(pwd):$(pwd) -w $(pwd) rapid7/msf-centos6-x64-omnibus:2024_01 /bin/bash --login

# Install bundler
gem install bundler -v 2.3

# libyaml
sudo yum install -y libyaml-devel

# Bundle
bundle

# Compile nokogiri
bundle exec rake clean
bundle exec rake compile:nokogiri -- --use-system-libraries

Before
failure
$ bundle exec rake compile:nokogiri -- --use-system-libraries
....
/var/jenkins_home/.rvm/rubies/ruby-3.0.6/include/ruby-3.0.0/ruby/internal/arithmetic/int.h:155: warning: comparison is always true due to limited range of data type
../../../../ext/nokogiri/xml_document.c: In function ‘dealloc’:
../../../../ext/nokogiri/xml_document.c:77: error: #pragma GCC diagnostic not allowed inside functions
../../../../ext/nokogiri/xml_document.c:78: error: #pragma GCC diagnostic not allowed inside functions
../../../../ext/nokogiri/xml_document.c:93: error: #pragma GCC diagnostic not allowed inside functions
At top level:
cc1: warning: unrecognized command line option "-Wno-tautological-compare"
cc1: warning: unrecognized command line option "-Wno-self-assign"
cc1: warning: unrecognized command line option "-Wno-parentheses-equality"
cc1: warning: unrecognized command line option "-Wno-constant-logical-operand"
cc1: warning: unrecognized command line option "-Wno-cast-function-type"
gmake: *** [xml_document.o] Error 1
rake aborted!
Command failed with status (2): [/usr/bin/gmake]
/var/jenkins_home/.rvm/gems/ruby-3.0.6/gems/rake-compiler-1.2.5/lib/rake/extensiontask.rb:195:in `block (2 levels) in define_compile_tasks'
/var/jenkins_home/.rvm/gems/ruby-3.0.6/gems/rake-compiler-1.2.5/lib/rake/extensiontask.rb:194:in `block in define_compile_tasks'
/var/jenkins_home/.rvm/gems/ruby-3.0.6/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
/var/jenkins_home/.rvm/gems/ruby-3.0.6/bin/ruby_executable_hooks:22:in `eval'
/var/jenkins_home/.rvm/gems/ruby-3.0.6/bin/ruby_executable_hooks:22:in `<main>'
Tasks: TOP => compile:nokogiri => compile:nokogiri:x86_64-linux => copy:nokogiri:x86_64-linux:3.0.6 => tmp/x86_64-linux/nokogiri/3.0.6/nokogiri.so
(See full trace by running task with --trace)

After
success
$ bundle exec rake compile:nokogiri -- --use-system-libraries
cp tmp/x86_64-linux/nokogiri/3.0.6/nokogiri.so tmp/x86_64-linux/stage/lib/nokogiri/nokogiri.so
[jenkins@c4d13ce50246 nokogiri]$ echo $?
0
...

@adfoster-r7 Thank you for your help, and for submitting this. Looks good to me, will merge and get into the next patch release.


Start testing qvi_scope_ksplit().

@GuillaumeMercier qvi_scope_ksplit() will likely be a convenient call for your threading work. I still need to iron out some limitations, but for splits that have no more colors than pieces (a limitation that I need to lift, in general), this seems to be working okay.


Include "Mappy Breakpoints"

Even though I have not extensively used it in production yet, I really like the ideas and logic I saw in the Mappy Breakpoints mixin by @zellwk. Most problems / annoyances he describes in his blog post, I experienced myself.
I think it would be a great idea if you could join forces and include the mappy breakpoints approach in breakpoint. I would rather write  @include breakpoint(small large) than @include mappy-bp(small large) :-)

We've had the ability to have map based breakpoints for a while now (if I'm going to be specific, 2 years and 5 days) through respond-to. We abstracted out writing them as a map into function calls because we found that, once you started writing queries more advanced than a single number, or wanted to include or queries, it got a little bit weird to do by hand.
We have also had for almost Breakpoint's entire life the ability to output em based media queries. It's true that we don't support rems, but those aren't valid Media Query units and we're not in the base font size game, so we don't support them.
In terms of being able to do less than and greater than, it's something @codingdesigner and I have discussed. The issue we're currently facing with them (and with the rest of Mappy Breakpoints, specifically being able to combine media queries) is that Breakpoint allows for much much more than just min-width queries. How do complex queries get combined? What happens if there are competing definitions? What happens if there are too many definitions to resolve effectively? Or do we just restrict combining to only single number queries only when using a feature that supports min/max? If we can get those questions resolved, we're happy to add this functionality, but we've gotta figure that out first.

@Snugug I didn't realise there was a respond-to mixin. Glad I got to know it now.
By the way, Mappy breakpoints doesn't only work with width queries. It supports height queries and other properties as well. However, how the queries get combined may not be as comprehensive as what breakpoint has to offer.
Quick question: How do you calculate pixel to em based queries if you don't use a base for conversion?

@zellwk em based media queries are not based on your site's font size, but rather the browser's default 16px === 1em ratio that's used by all browsers. As such, we can use that for our conversion.

@Snugug what happens if users decide to change their font size such that 16px no longer equals to 1em? Say if they put font-size: 125% in the HTML.
It's been a big question I'm trying to answer when I wrote Mappy Breakpoints with a $base-font-size. Would be great to learn more, and see how to improve that.

It doesn't matter. It's not based on that. It's based on the internal browser core font size, what that 125% is a percentage of. What you're changing when you change that is the display font size, not the browser's inherent font size. The way that changes (and why em based media queries are so useful) is if a user goes into a browser's settings and changes the font size there.
To change that inherent size yourself, go into chrome://settings/ -> Show advanced settings… -> Web content -> Font size. That's also where you change your default serif, sans-serif, and monospace fonts.

Ahhhh. Okay i get it. This means I didn't have to use $base-font-size and rem in the first place. I could have just stuck with em!

Ay yo, good conversation (and old conversation; slowpoke isslow), but I wanted to point out that em font-size is actually based on the font size of the concerned element's parent container,* meaning that you can see fonts embiggen as nested elements set a multiple of em > 1 for their font size, contrasted to rem, which will always be relative to the browser's core font size, and will neither embiggen nor debigulate on further-nested elements.
*tested in Firefox but idk what Chrome thinks is the right way to render this fiddle. I guess I could check if I hadn't brew cask uninstalled Chrome a while ago. But then I guess I could just brew cask install it agai and in Chrome

@thepeoplesbourgeois you are not talking about media queries though, your example doesn't even use media queries


Username in liveshare not updating

Probably a duplicate
Describe what happened:
I changed my GitHub username a while back and recently saw that on live share the old one still shows up, see screenshot.
What was your system configuration?
Product and Version [VS/VSCode]: VSCode
OS Version[macOS/Windows]: Windows
Live Share Extension Version: v1.0.4673
Target Platform or Language [e.g. Node.js]: Not Applicable
Steps to Reproduce / Scenario:
I first deleted my live share user profile as I thought there might be a problem with it. After logging in again I quickly checked it and saw nothing out of the ordinary. Logging back into GitHub on vscode and granting live share access to my account and joining a new session the username was still showing up incorrectly
Logs
20210821_132024_16295520246310_VSCode.log
20210821_131002_Agent.log
Screenshots

Should be "(mutedev)" should be "(Myuuiii)".
Any help would be appreciated! ❤

Hello,
We have a fix rolling out for this shortly. Until then though, if you would like, I'm happy to manually update our db. If you'd like that, please email me with the details to<EMAIL_ADDRESS>

Alrighty thank you, I'll send an email

Closing as a duplicate of #4393.


Can traefik accesslog output timestamp with optional params: StartLocal/StartUTC

Do you want to request a feature or report a bug?
Feature
What did you expect to see?
Accross traefik raw access.log, there only one timestamp, it's UTC time. For example:
<IP_ADDRESS> - - [11/Aug/2020:09:41:31 +0000] "GET / HTTP/1.1" 404 19 "-" "curl/7.29.0" 1 "backend not found" "/" 0ms

Code: https://github.com/containous/traefik/blob/v2.2.8/pkg/middlewares/accesslog/logger_formatters.go#L25
What i want
I want raw accesslog use LOCAL time. Can traefik provide flags to adjust StartLocal/StartUTC in raw log?

args  add --accesslog.fields.names.StartUTC=drop  mount local  /etc/localtime
like
- mountPath: "/etc/localtime"
name: "timezone"
volumes:
- name: "timezone"
hostPath:
path: "/etc/localtime"

args add --accesslog.fields.names.StartUTC=drop , mount local file /etc/localtime
like
        - mountPath: "/etc/localtime"
          name: "timezone"
      volumes:
      - name: "timezone"
        hostPath:
          path: "/etc/localtime"

Ah..I know that this arg is not effect in v1.7.11. Thank you

v1.7 user "--accesslog.fields.names="StartLocal=keep StartUTC=drop""  you can  reference https://github.com/Lincoln-dac/learn-kubernetes/blob/master/traefik/traefik修改日志为中国时区

v1.7 user "--accesslog.fields.names="StartLocal=keep StartUTC=drop"" you can reference https://github.com/Lincoln-dac/learn-kubernetes/blob/master/traefik/traefik修改日志为中国时区

我们这边生产用的 1.7.11，现在做法是把那块代码改掉重新编译了一下，打了个 .1 的镜像出来


Delete symlinks whenever a git-hook is removed by composer

Close this because it's solved by commit 0a9728.


Newer versions of libcoro und libeio

I've inspected the changes to both libcoro and libeio and did not find anything incompatible or dangerous.
So why not, will land it. Thanks!


427 Make sure MonitorRegistryMetricPoller converts booleans into numerical values

Not sure if it is pointing to the right branch. I based it on version 2.12.14 because I use an older version of aws-sdk.

Closing old PR as it doesn't pass CI and this project is deprecated. Please re-open if there is still a need.