[SPARK-4176] [SQL] Supports decimal types with precision > 18 in Parquet

This PR is based on #6796 authored by @rtreffer.
To support large decimal precisions (> 18), we do the following things in this PR:

Making CatalystSchemaConverter support large decimal precision
Decimal types with large precision are always converted to fixed-length byte array.

Making CatalystRowConverter support reading decimal values with large precision
When the precision is > 18, constructs Decimal values with an unscaled BigInteger rather than an unscaled Long.

Making RowWriteSupport support writing decimal values with large precision
In this PR we always write decimals as fixed-length byte array, because Parquet write path hasn't been refactored to conform Parquet format spec (see SPARK-6774 & SPARK-8848).

Two follow-up tasks should be done in future PRs:

[ ] Writing decimals as INT32, INT64 when possible while fixing SPARK-8848
[ ] Adding compatibility tests as part of SPARK-5463

Merged build triggered.

Merged build started.

Test build #37552 has started for   PR 7455 at commit 7dd88ce.

Test build #37552 has finished for   PR 7455 at commit 7dd88ce.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #37620 has started for   PR 7455 at commit 43411f8.

Test build #37620 has finished for   PR 7455 at commit 43411f8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Waiting for merging #7441. Need to rebase against it.

Merged build triggered.

Merged build started.

#7441 was merged. Rebased this PR.

Test build #38534 has started for   PR 7455 at commit a543d10.

Test build #38534 has finished for   PR 7455 at commit a543d10.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Merged to master. Will update #7679 to include large decimal precision support.
@rtreffer I attributed this PR to you while merging. Thanks for the original contribution!

@liancheng thank you for your time reviewing and implementing this! :bow:


Consider using matrices for transformation methods in the Transform component

Summary
See if its worth it to use matrices in the Transform component that can be reused across calculations as long as properties like position, scale and rotation do not change. The current idea is to add 2 matrices to the Transform component: 1 local and 1 world matrix. Since the Transform component is only interested in 2d 3x3 matrices would suffice. Once the matrix is built transforming a vector3 or vector2 will consists of only some multiplications and additions.
At this point its not clear yet what the best way is to integrate this into the Transform API. Should it be used under the hood or be exposed as a separate property for 'batch' operations?
First focus on researching if its actually worth it and implement this as separate properties with no caching. The test code will handle the caching for now.
Analysis

Would avoid repeately calling expensive operations such as sin and cos. Most of the gains would be in batch operations. Though I suspect that if its reused even once it will outperform the current implementation.
Would avoid having to cast to Vector3 when transforming a Vector2.
Possibly cleanup the code a bit by moving some of the calculations out of the Transform component

Still skeptical as to whether this would pay off in the end when considering performance, but would be open to take a look at a prototype implementation and profiling results.
Some first thoughts:

Public API should remain the same at first.
There will need to be inverse matrices in addition to the regular ones to do inverse transform operations.
Transformation is still a 3D operation - even though rotation only happens around the Z axis (2D), we do have 3D coordinates for translation. That means we need a 4x4 matrix for transformation.
With between two and four 4x4 matrices added, memory footprint could become a concern. In addition to performance, memory size should be compared as well.

While a matrix-only implementation could be cleaner and more recognizable to contributors than the current specialized transformation code for Z rotation, scale and offset, that's still a lot of math and memory we don't actually need in our special case.

See if its worth it to use matrices in the Transform component that can be reused across calculations as long as properties like position, scale and rotation do not change.

If caching is the main concern of this issue, we could actually just cache the MathF.GetTransformDotVec results for XY rotation + scale, which is essentially a 2x2 matrix and handle the translation separately. This wouldn't require restructuring all internals to use matrices and could be some middle ground.
In this case too, memory footprint change and performance should still be evaluated.

In a console application that referenced duality I managed to get a tenfold increase in performance for big batches when using a 4x4 matrix. The break even point is around 6 reuses.
However keep in mind its currently creating alot of 4x4matrices (4 I believe) that are discarded as soon as it builds the composite matrix so I suspect this overhead could be reduced quite alot. This is the current code for building the matrix:
		public static Matrix4 BuildLocalTransformMatrix(Vector3 translation, float scale, float rotation) 		{ 			var rotationMatrix = Matrix4.CreateRotationZ(rotation); 			var translationMatrix = Matrix4.CreateTranslation(translation); 			var scaleMatrix = Matrix4.CreateScale(scale); 			return translationMatrix * rotationMatrix * scaleMatrix; 		}

@Barsonax Can you share the full code of the benchmark in order to interpret the results?

I have included 3 benchmarks: the 2 transformation benchmarks and a matrix build benchmark.
DualityMatrixTransform.zip
On my machine I get these results:

That looks pretty good - must admit, I wouldn't have expected the matrix variant to be so much faster for transformation. The build matrix example could likely be optimized quite a bit too, since the kind of matrix that is needed is limited and known in advance, so I'd be optimistic on that one.
On the skeptic side, we might need to build 2-4 matrices on transform changes, and another point remaining is the one of Transform memory overhead by the amount of matrices stored internally. Both of them could be solvable problems though.
We might find out more proceeding with a with a full prototype implementation. To get a solid side-by-side comparison, this could be done by copy-pasting the full Transform component into a matrix variant for testing, which can then be adjusted step by step while continuously being compared to the old version.

Could be worth to check if its worth just caching the sin and cos values instead of a full 4x4 matrix. Remember all we do here is caching the calculated values so a full matrix might be overkill for just that.

Sounds similar to what I meant by going with a MathF.GetTransformDotVec caching approach, but extended to storing only the sin and cos values, further cutting this down from 4 values to 2 values. At this point it would also cease to be a "proper math" cleanup issue and should be treated as a "performance vs. memory" tradeoff. Could be worth looking into it, need to get some data on how frequent the sin / cos values are used in real world scenarios.

Jup at this point this issue is petty much only for extra performance. I dont know how often ppl use these methods. 60 ns is still very fast so it might not be needed to do this optimization.
Still I was curious how far I could push it. Writing it all out like below results in 2.2 ns per operation on my machine:
`
public Vector3 NewTransformBenchmark2()
{
var lastX = _vector.X - _position.X;
var x = (lastX * _cachedCos - (_vector.Y - _position.Y) * _cachedSin) * _inverseScale;
var y = (lastX * _cachedSin + (_vector.Y - _position.Y) * _cachedCos) * _inverseScale;
var z = (_vector.Z - _position.Z) * _inverseScale;
return new Vector3(x, y, z);

}
`
Without crazy stuff like SIMD I dont think you can get much faster than this.

Added (way) faster transformation methods to Transform for:

GetWorldPoint vector2
GetWorldPoint vector3
GetLocalPoint vector2
GetLocalPoint vector3

Below are some benchmarks:


MAC Address should use Word64

On a 64-bit platforms, this makes MAC addresses representable in one word instead of two.

I've written a Word48 and am now using that.


Updated patterns to support base64 data urls

I had a use case where I'm using the slider in a preview before the user uploads files to the server, so I only have base64 data urls.
The loader would just hang because it wouldn't know what to do with them when I tried to navigate to the second image in the list.
I simply updated the pattern matching to understand the base64's prefix in addition to the file extensions it already supported.

URLs without extensions also don't work :(

@dankobaia I ran into that as well while developing my site.  Fortunately in my case I had full control over my images so I just switched to tacking on a file extension.  I couldn't think of a good way to update the slider to support them.
If you didn't have control over the images themselves, I wonder if you couldn't trick the slider by adding a query string or a hash to the end of them, so the browser won't actually care but the slider's regex will see it as an image (e.g.  https://test.com/image-with-extension?.jpg)

Can anyone merge this PR?


PCB review

upload of the reviewed PCB file

Hey @Samajadi great, thanks a lot!
Could you quickly summarise the key point of this change? Also, maybe a screenshot of the updated schematics/PCB would be great!
Do we need to adapt any screenshots that are linked in the main README file?


HTTP/S reponse record and replay

Provide support for HTTP/S response record and replay
Users have asked for the capability to record responses from external services, and then to use those responses as a mechanism to stub out the external service during the execution of a test run.
Proposed Record Mode Workflow

Put Martian into RECORD mode
Instruct Martian to record responses to requests that match a provided keyGen() function.
Provide Martian with the path to the db in which to save responses (bolt?).
As requests that match the provided keyGen() come across, Martian captures the response that comes back from the HTTP roundtrip.
For each response for request that matches the keyGen(), Martian adds a header with a known name (X-Martian-Key) where the value is a string representation of the output of keyGen().
While in RECORD mode, any response modifiers will be applied before capture, maybe. Depends on where on the stack the record/replay modifier lives.
Martian holds the responses in memory, and when Martian leaves RECORD mode, the responses are written to the filesystem to the provided path and filename.

Proposed Replay Mode Workflow

Put Martian into REPLAY mode and tell it where the response cache db and the keyGen() used in RECORD mode
Martian parses the response cache db and holds it in memory
As requests that match the provided keyGen() come across, Martian grabs any matching response from the cache db, deletes the X-Martian-Key header, and returns the cached response.
Martian applies any other modifiers on the stack - this is where you'd want to put modifiers that remove "dynamic" headers (datetime, etc.)
Martian returns the modified cached response.

Is this still planned? I think it'd be great for testing.

Yes - it's under active development.
Keep an eye out for it this quarter.
On Thu, Jan 19, 2017 at 10:19 AM bootstraponline<EMAIL_ADDRESS>wrote:

Is this still planned? I think it'd be great for testing.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/google/martian/issues/75#issuecomment-273855099, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AADI-Uz9eRpLjpAmFBCTNEWdB0lJVK6hks5rT6kegaJpZM4HjVnu
.

Awesome! Thanks for the info.

Is there an updated ETA?

An initial implementation is being worked on in the "caching" branch, for those interested.

It'd be helpful to demo a simple use case of record/playback for Java using okhttp. If this can be used as a wiremock alternative for Android testing, that'd be excellent.


installer for windows not working

I downloaded the file available on the download link for windows, unzipped via winrar, and tried to run.
When run, gtm.exe only flashes and then closes.
did I do something wrong?

I think you may have download the latest which doesn't have a Windows installer included.  I'm currently working on a Linux build issue and haven't included a Windows installer yet.  Try this release https://github.com/git-time-metric/gtm/releases/tag/v1.3.2 and run the installer gtm-v1.3.2-windows-installer.exe.

Just cut a new release, https://github.com/git-time-metric/gtm/releases/tag/v1.3.3

Thanks! Works like a charm!
Before closing, a question:
In Gitlab, there is the possibility of adding the estimated time and the time spent. Does the time information that the plugin provides integrate with Gitlab?

Unfortunately, I don't use GIRA. But thank's anyway.

You can do a pre-commit git hook and use gtm status --total-only to get the time spent and automatically add it to the git commit message.

Ah ok. Thanks! 😄


Release

Not sure why this is still open, but everything from here has been released long time ago.


How to use validation with TextBox?

Issue Type Question

Description

How to use validation with TextBox?
How to pass validations options to component?
Ex: https://js.devexpress.com/Demos/WidgetsGallery/Demo/Validation/Overview/jQuery/Light/

Hi
Currently, this is not supported, you can track this issue: #78.


Optimum SetUp of Filters in RS Viewer for Indoor Capture

Using a D415.
Running on Ubuntu 16.04
Issue Description
@RealSenseCustomerSupport
I'm struggling here trying to achieve an optimum configuration recording people walking indoor. In the range of 1 - 8m.
Basically i'm trying to setup the posto processing filters to record avoiding artifacts in the depth colorized stream.
Is there any tip to achieve the best result possible ?
Should i turn on any (which ?) filters like threshold, space, temporal ...  or others ?
Tried to use some of the default that come already with RS Viewer... but for poor results...
Thanks !

Though the D435 model is best suited to capturing motion thanks to its fast Global Shutter (compared to the slower Rolling Shutter on the D415), the D415 should still be able to handle human walking pace.
Intel's tuning guide for the 400 Series cameras advises that if the scene has bright light then you may be able to reduce motion artifacts by setting exposure manually to near to 1 ms.

@MartyG-RealSense
Thanks for the advise ... fot the time being i will continue capturing the pedestrian materials for my research using my D415 but i'm considering buying a D435 or other (Zed ?) Depth Camera for some further tests ... is there any recent comparison Matrix between Depth Cameras features that you would recommend as a reference ?
Also thank you for sending the link for the tuning guide ... really very useful ! Interesting reading with several tips !

Intel have a 'Which camera is right for you' guide that compares the models, including the forthcoming new L515 lidar camera.  Near the bottom of the page is a feature comparison chart.
https://www.intelrealsense.com/which-device-is-right-for-you/

@MartyG-RealSense
Thanks again for you comprehensive reply and for sending those extra partners links ...
L515 really seems that it's going to be very appropriate in my area of research (pedestrian tracking indoors...) right ?
Just one more related question: will this be back compatible (in terms of SDK / wrapper programing ...) with codes already developed for 415 or 435 ?

You are very welcome.  :)
L515 is best suited to use indoors or in controlled lighting conditions.  It is not vulnerable to motion blur and does not require recalibration during its lifetime.  So it does seem to be an appropriate choice for your application.
You may be interested in a recently released video demonstrating the new Cubemos Skeleton Tracking SDK for RealSense being used to track a mass crowd.  Cubemos is compatible with both the 400 Series cameras and L515.
https://youtube.com/watch?v=PENY6Q24c6A
Cubemos can be tried now with a 30 day free trial version.  A full licence is $75 USD in the official RealSense store.
Video
https://youtube.com/watch?v=PENY6Q24c6A
Information page with trial download button
https://www.intelrealsense.com/skeleton-tracking/
Store link
https://store.intelrealsense.com/buy-skeletal-tracking-sdk.html
In regard to L515's compatibility with the RealSense SDK: the RealSense SDK Manager has said that 99% of the SDK's functions will be supported for L515 out of the box at launch.  This is possible because the RealSense developers spent months adding support before the camera was even officially announced.
A set of new Visual Preset camera configurations for the L515 were also added in the latest 2.33.1 version of the RealSense SDK.
Existing programs for the 400 Series should be simple to convert.  Intel has said that the Cubemos software only took half an hour to get working on L515.

Thanks @MartyG-RealSense  !
This Cubemos SDK is in definitely impressive !!
In fact, what i need in my research is a subset of what this SDK seems to be able to provide ! I will experience the trial to check the capabilities... for sure.
Also, thank you for the explanations about the back compatibility of L515 for using code created for 400 series ! Your guidance is much appreciated Marty !

Hi @DavidCadtec I just wanted to add that the official RealSense blog has published a new article on skeleton tracking that contains a section speaking of the ease of converting 400 Series programs to L515.
https://www.intelrealsense.com/skeletal-tracking/

Super. Thanks @MartyG-RealSense  !!

Dear @MartyG-RealSense,
Since 2019 I have been developing my research system based on Cubemos Skeleton Tracker, and I just found out that currently Intel Realsense Camera is not supported by Cubemos SDK. How can I continue to use the Cubemos SDK?
I didn't have much time to start my research activities from scratch.
Thank you for your advise.

Hi @Basuki74  The Cubemos Skeleton Tracking SDK was a product of the Cubemos company and was withdrawn from availability, so there is no option to continue with an existing licence or download trial versions, unfortunately.  The best option for a quick transition for your research project is likely to be to switch it to the similar RealSense-compatible Nuitrack SDK skeleton tracking software if it is possible for you to do so.
https://nuitrack.com/

Dear MartyG,
Thank you very much for your quick response. Actually I've tried using
Nuitrack, but after I tried to implement it, the accuracy is less than
Cubemos. Skeletal joints generated by SDK are unstable for some distance
and also when the hands cover the body. Others, hand gesture cannot be
detected if the distance between humans and the camera is more than 1.5 m.
Those are many reasons why I want to continue using Cubemos.
I have also tried using MediaPipe, but as you know they can't support for
depth data.
I think Cubemos is very powerful for a skeletal joint tracing library.
Is there any problem in the future if I still continue for using Cubemos?
Best regards,
Dwiki
On Wed, May 25, 2022 at 4:26 PM MartyG-RealSense @.***>
wrote:

Hi @Basuki74 https://github.com/Basuki74 The Cubemos Skeleton Tracking
SDK was a product of the Cubemos company and was withdrawn from
availability, so there is no option to continue with an existing licence or
download trial versions, unfortunately. The best option for a quick
transition for your research project is likely to be to switch it to the
similar RealSense-compatible Nuitrack SDK skeleton tracking software if
it is possible for you to do so.
https://nuitrack.com/
—
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/5938#issuecomment-1136887893,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARQM27ZBLXODTU5EWQVCR7DVLXI3TANCNFSM4K5ENUQQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

My understanding is that once the time limit on a Cubemos SDK trial version expires then it is no longer possible to purchase a licence key.  The installer file for the Cubemos SDK program itself still has a working download for Windows and Linux though at the link below.
https://dev.intelrealsense.com/docs/skeleton-tracking-sdk-installation-guide
Pages 23-24 of the Cubemos manual have information about license keys.
https://download-skeleton-tracking-sdk.s3.eu-central-1.amazonaws.com/GettingStartedGuide_2.3.2.pdf

Thank you @MartyG-RealSense for your explanation.


[CI] Adding CPU docker pipeline

This patch adds the pipeline for CPU docker image. On each release(tag) it will build push the image to vllm/vllm-cpu
Fixes: #4771

@khluu
Hi Kevin, I have updated the code to use $RELEASE_VERSION. Could you please help to take a look? The CI failure seems not related with this patch(?)

I think the PR is ready but the thing is I'm not sure if BUILDKITE_TAG works here. For example, @simon-mo tagged a commit earlier today to release and this build was triggered: https://buildkite.com/vllm/release/builds/2257 but it didn't register any tag (you can see in the Environment tab for each step). Maybe we can just remove the if: build.tag ... line and have it use $RELEASE_VERSION as tag instead of $BUILDKITE_TAG. We can input $RELEASE_VERSION manually when triggering a new build on Release pipeline.

@khluu Hi Kevin, I have updated the code to use $RELEASE_VERSION. Could you please help to take a look? The CI failure seems not related with this patch(?)

I think the PR is ready but the thing is I'm not sure if BUILDKITE_TAG works here. For example, @simon-mo tagged a commit earlier today to release and this build was triggered: https://buildkite.com/vllm/release/builds/2257 but it didn't register any tag (you can see in the Environment tab for each step). Maybe we can just remove the if: build.tag ... line and have it use $RELEASE_VERSION as tag instead of $BUILDKITE_TAG. We can input $RELEASE_VERSION manually when triggering a new build on Release pipeline.

Yes let me test it on the Release pipeline then will merge!

@khluu did you forget to approve this?

oops yes

Hi @khluu @DarkLight1337 Thank you for the review and quick merge!
Thanks,
-yuan


fix(leadspace): properly point to defaultSrc for the Image

Related Ticket(s)
No related issue
Description
I can't seem to find the issue that this was related to, but wondering if this should get patched in. The Leadspace is pointing to image.default and not image.defaultSrc.
Changelog
Changed

Fix to Leadspace to properly point to defaultSrc

@Praveen-sr @chetankan fyi, this is getting hotfixed for the next release candidate, which affects the Leadspace.


pepare CHANGELOG for 1.32.0

What does this PR do?
Prep for 1.32.0

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-06-13T09:55:41.497+0000

Duration: 3 min 7 sec

:grey_exclamation: Flaky test report
No test was executed to be analysed.
:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run benchmark tests : Run the benchmark tests.

run jdk compatibility tests : Run the JDK Compatibility tests.

run integration tests : Run the Agent Integration tests.

run end-to-end tests : Run the APM-ITs.

run windows tests : Build & tests on windows.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)


Verify input {} against module?

I made a silly mistake in one of my terragrunt configs -- in this case using the terraform-aws-rds-cluster module
I had
input = {
   ...other inputs
   security-groups  = [
                          dependency.rds-security-group.outputs.security_group_id,
                          dependency.eks.outputs.worker_security_group_id
                        ]
  ...more inputs
}

The astute reader will notice that the correct variables.tf in terraform-aws-rds-cluster module is actually security_groups, ie with an underscore, not with a dash.
Now if you do it wrong, there is no warning, and since I did not set 'security_group', you get the default.
Would it be possible for terraform to check all the inputs against the module - and emit a warning if an unmatched input is used?  This would be a new feature.

Hi,
may be tricky to implement since it will require parsing all terraform files before launching and comparing variables and inputs

I think https://terragrunt.gruntwork.io/docs/reference/cli-options/#validate-inputs addresses the need here. If not, can you describe how we can improve that command (or what the bug is)? Thanks!

You are absolutely right, it's there.  How did I not see this?  Will close the issue.


enable fail_on_warning for the docs CI

RTD's sphinx extension released a new version since #4199, so let's try enabling the fail_on_warning setting again.

[x] Passes isort . && black . && mypy . && flake8

🙏


Create generic template to show blog post

After the post is created as in #145
We need a generic template to view the blog post.

What does Create generic template to show blog post #149 if defer from
Create template to list blog post #150? could you give me more details?

@fahad-albasrawie For this task, I mean to create a generic page that can show the blog post.
Blog post creation is defined in the task #145

I got it this way: You mean, Here is the list of blogs, When I click One blog post, it opens the post so it can be read, does this make sense?

@fahad-albasrawie this task is create page that would show the blog post when some click in the list.

Ig got it. Could you assign this task to me, sir?


add code for multivariate attack paper

Issue #, if available:
Description of changes:
Add the code for the paper Robust Multivariate Time-Series Forecasting: Adversarial Attacks and Defense Mechanisms.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Please tag this pr with at least one of these labels to make our release process faster: BREAKING, new feature, bug fix, other change, dev setup

Hi @Linbo-Liu,
it looks like you are including all of GluonTS again in your PR. Can you only add code that you are providing?

Hi @Linbo-Liu,
it looks like you are including all of GluonTS again in your PR. Can you only add code that you are providing?

Hi @jaheba,
The GluonTS package is modified. In order to successfully run the code, I have to include them here.

Then let's work on adding the changes that you need. But adding everything again is not feasible for us.

I have removed repeated files.

@Linbo-Liu style checks are failing, make sure to:

pull the latest changes (changes from dev were merged into this one)
have the latest black installed (pip install -U black)
run black src test
commit and push the changes that black applies


Include internal_login scope in the access token request using the refresh token grant type.

Purpose
Modifies the integration tests to cover the access token request with internal scopes using the refresh token grant type.
Related Issues

https://github.com/wso2/product-is/issues/15233

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3737289126
Status: success


PointSource result generator not working as expected

Depending on how the result generator is called, the same block might be yielded again (as opposed to the next block). This happens with PointSource, but not with TimeSamples, so there seems to be something off in the PointSource implementation of the result generator. See attached example. ("Output versions" differ when using p1, but do not and are correct with p2)
minimum_example_bug_PointSource.py

After line 29, add output print(i), line 25 has to be p=p1. Weirdly, during running through generator, values are different from after finishing. May be due to "try...except" statements in PointSource.
By Gert Herold on 2020-10-26T08:44:26 (imported from GitLab)


[SPARK-31595][SQL] Spark sql should allow unescaped quote mark in quoted string

What changes were proposed in this pull request?
def splitSemiColon cannot handle unescaped quote mark like "'" or '"' correctly. When there are unmatched quotes in a string, splitSemiColon will not drop off semicolon as expected.
Why are the changes needed?
Some regex expression will use quote mark in string. As config "spark.sql.parser.escapedStringLiterals" default value false allows us use unescaped string, we should process semicolon position correctly.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Added Unit test and also manual test.

Test build #121991 has started for PR 28393 at commit 562841b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26666/
Test PASSed.

Test build #121991 has finished for PR 28393 at commit 562841b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121991/
Test PASSed.

@xuanyuanking

@dilipbiswal @xuanyuanking Thanks for your advice, I have updated the code.

Test build #122169 has started for PR 28393 at commit 1947cbe.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26841/
Test PASSed.

Test build #122169 has finished for PR 28393 at commit 1947cbe.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122169/
Test PASSed.

Test build #122206 has started for PR 28393 at commit c20543d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26878/
Test PASSed.

LGTM
cc @maropu

Test build #122206 has finished for PR 28393 at commit c20543d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122206/
Test PASSed.

Test build #122329 has started for PR 28393 at commit d185264.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26999/
Test PASSed.

Test build #122329 has finished for PR 28393 at commit d185264.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122329/
Test PASSed.

cc @juliuszsompolski as well

thanks, merging to master/3.0!


208 adds cpu premerge tests github

Signed-off-by: Wenqi Li<EMAIL_ADDRESS>Fixes #208
Description
adds premerge github tests
tested: https://github.com/Project-MONAI/model-zoo/actions/workflows/premerge-cpu.yml
Please ensure all the checkboxes:

[x] Codeformat tests passed locally by running ./runtests.sh --codeformat.
[x] In-line docstrings updated.
[ ] Update version and changelog in metadata.json if changing an existing bundle.
[ ] Please ensure the naming rules in config files meet our requirements (please refer to: CONTRIBUTING.md).
[ ] Ensure versions of packages such as monai, pytorch and numpy are correct in metadata.json.
[ ] Descriptions should be consistent with the content, such as eval_metrics of the provided weights and TorchScript modules.
[ ] Files larger than 25MB are excluded and replaced by providing download links in large_file.yml.
[ ] Avoid using path that contains personal information within config files (such as use /home/your_name/ for "bundle_root").

/build

/build


Sunset PriceOverride discount type

Background
Closes #1415. In POS we sunset the 'Price override' functionality, in this PR. This discount type is no longer supported and needs to be removed from the UI extensions as well.
Solution
Removing the discount type from the Cart API.
🎩

n/a

Checklist

[x] I have :tophat:'d these changes
[ ] I have updated relevant documentation (currently working on this).

LGTM.
This is for unstable API version, when is the next stable version release? Do we need to do anything about that?


SDK converts Column Names to lower case

Why does the SDK convert the column names of the tables to lower case?
This creates problems if you want to have camel case names for the columns.

Hi @icsd
Sorry for late response. You can use camel case column names, just decorate your field with annotation for serialization:
@com.google.gson.annotations.SerializedName("ItemStatus")
private String status;

@icsd We haven't heard back from you. I'm closing the issue for now but please feel free to reach out to us if you have any questions. Thanks!

Hi @icsd ,
Could you please elaborate more what case exactly doesn't work for you?
For me, it looks like this is detail of implementation that shouldn't affect public API behaviour of the SDK.

It does not create a bug or issue..It is just a not convenient implementation because you can not have exaclty the same column names on the backend and the Android client.
For example when you parse a field (e.g. itemStatus) from a request you can have it declared
as @com.google.gson.annotations.SerializedName("itemStatus"), but when you make a query to the local DB and want this column you have to SELECT it as itemstatus because this is how it is normalized in the DB table from the Azure SDK.
In the iOS Azure SDK this column name normalization does not happen.
These are the reasons I was wondering why this happens.
Thank you for your response.

OK, got it. Thanks for reporting this. I agree that it should be consistent with other platforms.
Regarding reasons - it's really hard to say that now. It was added in 2014 and hasn't been changed since that.
I'm afraid we aren't going to change it because it's a breaking change and requires migration/renaming in existing DBs at least.
If it's important for you, you can try to drop this conversion in your fork, but since it's just inconsistency and doesn't cause any errors - I'd keep it as is in this repo.


feat(@angular/cli): support 5.0.0+ builds of @angular/service-worker

This will likely have issues on Windows until Angular 5.0.0-rc.3 is released, due to https://github.com/angular/angular/commit/621f87b2bde074d5317689f544f90714876dbbdc.

It throws an error when I don't have an ngsw.json file in src, is that intentional or am I doing something wrong? Perhaps it could warn the user and link them to an example file?
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
               ^
Error: ENOENT: no such file or directory, open '/usr/local/google/home/stephenfluin/workspace/devfest/src/ngsw.json'

I added an explicit error if the file is not found, @StephenFluin :)

we're on 5.0.0-rc.5, so this should work on Windows now. Hopefully this can be merged soon.

@alxhub  Why version 5.1.0-beta.2 is not supported ?? I used master build.

The installed version of @angular/service-worker is 5.1.0-beta.2. This version of the CLI
requires the @angular/service-worker version to satisfy >= 1.0.0-beta.5 < 2.0.0. Please upgrade
your service worker version.


vpnbypass action call violates dispatcher

Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: /usr/lib/lua/luci/dispatcher.lua:397: Access Violation
Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: The page at 'admin/services/vpnbypass/action/enable/' has no parent node so the access to this location has been denied.
Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: This is a software bug, please report this message at https://github.com/openwrt/luci/issues
Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: stack traceback:
Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: 	[C]: in function 'assert'
Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: 	/usr/lib/lua/luci/dispatcher.lua:397: in function 'dispatch'
Mon Oct 28 19:01:21 2019 daemon.err uhttpd[909]: 	/usr/lib/lua/luci/dispatcher.lua:144: in function </usr/lib/lua/luci/dispatcher.lua:143>

What version?

That's latest from souce 1.3.1-4
Only sorted when I changed the location to force on admin/services tab not admin/vpn

Did you reboot your system so that the ccache of LuCI is new?

Yeah rebooted and the issue persists up until I changed the entry to be admin/services, This problem is that you call the admin/services entry even if admin/vpn is in place

Could this be the root cause of your problem?
https://github.com/openwrt/luci/blob/7707b679e20df9806fbcbc58d12116f9ee9a3850/applications/luci-app-vpnbypass/luasrc/controller/vpnbypass.lua#L7-L10
If this is the problem we have to fix also the line https://github.com/openwrt/luci/blob/7707b679e20df9806fbcbc58d12116f9ee9a3850/applications/luci-app-vpnbypass/luasrc/view/vpnbypass/js.htm#L35 @stangri

That's exactly is the issue, You need to build url for admin/vpn as well and define " entry({"admin", "services", "vpnbypass", "action"}, call("vpnbypass_action"), nil).leaf = true " or " entry({"admin", "vpn", "vpnbypass", "action"}, call("vpnbypass_action"), nil).leaf = true " based on the "if statement"

@ananas77777 could you please try the updated luci-app-vpnbypass from my repo before I send the PR?
I don't have a device I can test 19.07/snapshot on. Thanks!

All good now, You can go ahead with the PR

@ananas77777 thanks for reporting this issue
@stangri thanks for the fast fix


Fix output for describering by fixing toString method of TokenRange

@tjake would you mind to give me high five here? This should be a no-brainer.


How to install in Home Assistant supervised installation?

Hi,
is there a way to install this in a supervised installation of Home Assistant?
Thank you so much!

Please someone convert this to an addon

Right now, Portainer is the best option to install this as a Docker container. You can use Portainer's Stack feature and use the docker-compose.yaml approach as described in the README.
However, the device must be fast enough to open a browser once a minute and render it. A Raspberry Pi might be too slow to do this once a minute, but it depends on how much to render on the lovelace page, how many addons are running etc.
For a Hass.io addon, I did create #1 quite a while ago as I agree this might be the most convenient approach.

Thanks, went the portioner route :)


Add a progressbar or copy-paste speed

When I measure, i can see a fantastic improve of speed, but it will be much more beautiful, if user can see speed of copying

This was discussed in https://github.com/SUPERCILEX/fuc/issues/14 and is now documented: https://github.com/SUPERCILEX/fuc/commit/95a2678b991e65b6d916c69131b3e80267a3f88c

This was discussed in #14 and is now documented: 95a2678

that's strange. I can't see --progress with cpz -h, and cpz --progress makes an Error. When I use just cpz from to, there aren't any difference between vanilla cpz and cpz with progress feature

Nope, you need to install the binary again but with the progress feature. It's not a flag in the binary itself.

Oh, just saw you tried re-install. Use --force as it suggests.

Oh, just saw you tried reinstalling. Use --force as it suggests.

And... nothing changes. All works excluding progress bar

There's no progress flag. Now the binary will always show progress.

There's no progress flag. Now the binary will always show progress.

But there aren't, even with large number of files. For example, in my screenshot I successfully copy windows folder to another folder. I works perfect and fantastic speed, but without progress bar

Hmmm, how long does the copying take? If less than a second, you probably won't see the progress, but it should definitely be visible with longer copies. Though it's also possible that this is just broken on Windows? I'm not sure.

Hmmm, how long does the copying take? If less than a second, you probably won't see the progress, but it should definitely be visible with longer copies. Though it's also possible that this is just broken on Windows? I'm not sure.

3 or 4 seconds. Btw, default windows cp used more than 600 seconds xD

Hmmm, how long does the copying take? If less than a second, you probably won't see the progress, but it should definitely be visible with longer copies. Though it's also possible that this is just broken on Windows? I'm not sure.

 It's unusual progress info btw =) Without copy speed or progress-bar
, just something like this one

Ok, can you try cargo install cpz --features progress --force --git https://github.com/SUPERCILEX/fuc

Ok, can you try cargo install cpz --features progress --force --git https://github.com/SUPERCILEX/fuc

Compiling fuc_engine v1.1.6 (C:\Users\krakotay\.cargo\git\checkouts\fuc-a8eff962d757a116\02d0910\fuc_engine)
error[E0425]: cannot find value `follow_symlinks` in this scope
   --> fuc_engine\src\ops\copy.rs:649:22
    |
649 |         #[cfg(unix)] follow_symlinks: bool,
    |                      ^^^^^^^^^^^^^^^ not found in this scope

error[E0425]: cannot find value `root_to_inode` in this scope
   --> fuc_engine\src\ops\copy.rs:650:22
    |
650 |         #[cfg(unix)] root_to_inode: Option<u64>,
    |                      ^^^^^^^^^^^^^ not found in this scope

error[E0554]: `#![feature]` may not be used on the stable release channel
 --> fuc_engine\src\lib.rs:1:12
  |
1 | #![feature(lazy_cell_into_inner)]
  |            ^^^^^^^^^^^^^^^^^^^^

error[E0635]: unknown feature `lazy_cell_into_inner`
 --> fuc_engine\src\lib.rs:1:12
  |
1 | #![feature(lazy_cell_into_inner)]
  |            ^^^^^^^^^^^^^^^^^^^^

Some errors have detailed explanations: E0425, E0554, E0635.
For more information about an error, try `rustc --explain E0425`.
error: could not compile `fuc_engine` (lib) due to 4 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `cpz v1.1.6 (https://github.com/SUPERCILEX/fuc#02d0910c)`, intermediate artifacts can be found at `C:\Users\krakotay\AppData\Local\Temp\cargo-installNGTSZh`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.```

Well, something like this one. And in the beginning `Installing cpz v1.1.6 (https://github.com/SUPERCILEX/fuc#02d0910c)`, why is 1.1.6?

Ok, can you try cargo install cpz --features progress --force --git https://github.com/SUPERCILEX/fuc

well, even i tried to build locally, i have this one. just with -F progress

Well this is icky, I guess the tracing attribute doesn't support cfg-ed args. Can you try installing again? Should be fixed.

nothing changed

Oh for that you just need to do cargo +nightly ... and rustup install nightly if you don't have it.

Oh for that you just need to do cargo +nightly ... and rustup install nightly if you don't have it.

it's already nightly
PS C:\Users\krakotay\Documents\fuc\cpz> rustc --version
rustc 1.81.0-nightly (fcaa6fdfb 2024-07-13)
PS C:\Users\krakotay\Documents\fuc\cpz> cargo build --release --features progress
   Compiling tracing-indicatif v0.3.6
   Compiling fuc_engine v1.1.6 (C:\Users\krakotay\Documents\fuc\fuc_engine)
   Compiling clap v4.5.9
error[E0425]: cannot find value `follow_symlinks` in this scope
   --> fuc_engine\src\ops\copy.rs:649:22
    |
649 |         #[cfg(unix)] follow_symlinks: bool,
    |                      ^^^^^^^^^^^^^^^ not found in this scope

error[E0425]: cannot find value `root_to_inode` in this scope
   --> fuc_engine\src\ops\copy.rs:650:22
    |
650 |         #[cfg(unix)] root_to_inode: Option<u64>,
    |                      ^^^^^^^^^^^^^ not found in this scope

For more information about this error, try `rustc --explain E0425`.
error: could not compile `fuc_engine` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
PS C:\Users\krakotay\Documents\fuc\cpz> cargo --version
cargo 1.81.0-nightly (154fdac39 2024-07-07)
PS C:\Users\krakotay\Documents\fuc\cpz>

Thank you, all works!

Yay!
btw, progress bar in rmz doesn't work...

Yeah, Windows progress support isn't going to be implemented because we use https://github.com/XAMPPRocky/remove_dir_all. Honestly not sure what that lib does extra so maybe I could just use rmz's non-linux implantation on Windows? Don't know.


DFS cycle detection

added DFS cycle detection for directed and undirected graph
added tests

Hey @bobluppes, thanks for your comments! Ill change according to them


Translate content

Hi! Can I translate this content to pt-br?
I really liked this content and wanted to show it to people who do not know how to communicate in English

yeah! nice idea, @daliannyvieira <3


Minimal implementation of admin roles

With @tiltec and @djahnie we discussed the admin role features that would be needed for implementing #324.
#356 discusses the wider topic of additional roles and is still relevant for the near future.
For #324 all that is needed is some set of users within the whole group that are able to edit the legal agreement. We realised we don't need to restrict other actions yet (e.g. ability to add stores).
In the future we would like to implement more non-hierarchical methods of managing admins but it is more realistic to start with a simple system.
We discussed whether having admins in a group should be optional or not, and realised if it was optional then it would be difficult to turn it on again in the future: if nobody is an admin, who has permission to enable admins? (this would be the case for existing groups, or new groups that initially selected not to have admins).
One option would be to have an owner of the group (by default the original creator, but could be reassigned) - but then we have just added an extra level of hierarchy and not solved anything.
Based on thoughts above we came up with the following concept:

each group must have at least one admin
if a group does not have an admin, then the oldest member of the group is promoted to admin (to comply with point 1)
only admins can edit the legal agreement
only admins can create other admins
only admins can remove other admins (they can remove the original admin of the group as all admins have equal powers)

This should allow us to implement the smallest possible admin system that allows us to continue with #324, whilst also leaving the future open to more admin-specific roles, and to build more mechanisms to create and remove admins.

Tiny clarification in case someone misunderstood:
The oldest member is the one who joined first. Age doesn't make any difference here... 😉

i am starting to feel uncomfortable with the word "admin" probably because of many bad hierarchie examples missuse of power from foodsharing and facebook groups.
isn't it more like you have a feature - and you assign people to this feature. and not having a role that has 234234 features.
or you have feature 1,2,3 and make a virtual role a that has feature 1 and a role b that has feature 1 and 2. and you assign the user to the role.
that is also something that got a bit to much on foodsharing. like you have the orgateam(role admin) and every "admin" function was put into that role. which made it less flexible to give one function to person without giving them all access.
How can we assign "powerfull" features (features that give power over other users) and prevent the missuse like on fs.de ?
is it "if you gain a privilege you lose another?" for example if i can ban a user from the group i am not allowed to do pickups ? just brainstorming to give an idea.

oh another thing is, having a legal agreement might require the user to have a valid address or sth. like that. (i don't know the exact law / legal binding), but remember sth. like that from fs.de

I agree, and don't think we actually need to call it "admin" in the software, but something more specific to the actual purpose we are addressing right now.
The role in question that we need is people that can modify the legal agreement, so it might be we could let that happen if users have the role "legal".
In general though, so far the feeling has been that we would like to leave it up to individual groups to decide how to organise themselves, but lean towards (and provide feature for) non-hierarchical approaches.
I've sketched out some implementation ideas in https://github.com/yunity/foodsaving-backend/issues/353 - and for that idea, it would just have some role name that we add to the users group membership:
[user] <-- [membership roles=(role1,role2,role3)] --> [group]

oh another thing is, having a legal agreement might require the user to have a valid address or sth. like that. (i don't know the exact law / legal binding), but remember sth. like that from fs.de

Maybe it should be more general than legal agreement then (esp as we don't know the laws in all the countries), and just be "terms of use" (role=terms, or maybe role=agreement) kind of thing (for a given group). And maybe they can set the text for the agreement and also the title of it (e.g. "Legal Agreement", or "Terms of use" or "Community Guidelines" - we would lose the multi language aspect there though, unless we just supported a standard set of agreement types)

And maybe they can set the text for the agreement and also the title of it

Yes, that's what the legal agreement proposal is about: #324
The name "legal agreement" wouldn't show up to the user like this.
Maybe an appropriate description would be: "terms you have to accept before you can join the group"?
And it can set by either anyone in the group or someone with the role "can-modify-terms".
This issue is about how to actually set roles in a group, coming from a group where everybody has all rights. Basically "bootstrapping group roles".

i would like to take a step back -
how do we now if it is anything "legal" or binding"
it could be a text like "You accept me as your lord and master and must do as i say". or
"if you accept this then you own me a cake! you willingly agree to this condition! "
It can never be evaluated if whatever text people are writing is legally appropriate or even valid.
So it is the featuer of a "welcome message that you either agree or decline"- declining making it unable for the user to join the group.
that would be the feature description.
Whatever the content is the user decides, but whoever is running the tool doesn't take any legal ownership / responsibility of the content of that message. and you could name it "friendship agreement, or "legal agreement" or whatever name you want or your imageniry lawyer tells you to.
maybe it would be usefull to avoid terms like "legally binding","terms of service", etc. that sound highly official and approved of some legal entity or including any kind of responsibility from the site owner.

I copied your comment and my proposal into #324

These are older thoughts, which led to newer thoughts. So these can be archived to not be distracting.


llvm_11: build from git

llvm is currenly built from individual tarballs instead of upstream monorepo
The flaws of this approach are:

there are boilerplate to reconstruct fragments of git monorepo in every subproject's .nix file. For example https://github.com/NixOS/nixpkgs/blob/9ddb9c0021ede97e981c3dee2250bbd1abc50c6f/pkgs/development/compilers/llvm/11/clang/default.nix#L14-L20
there are difficulties with cherry-picking upstream patches because directory structure is altered, so nixpkgs has to host .patch-files instead of using fetchpatch from upstream git repo: https://github.com/NixOS/nixpkgs/tree/9ddb9c0021ede97e981c3dee2250bbd1abc50c6f/pkgs/development/compilers/llvm/11
it is not possible to use bleeding edge features which are already on git's master but not yet in the latest tarballed release. chromiumDev often needs them. With git repo, llvm_11.override{src = ...} would work.

In upstream (and in some Linux distros) there are already LLVM 12 (12.0.0-rc2 tag) and LLVM 13 (master branch).
NixOS does not have to provide them out-of-the-box.
Just switching to them by .override{src=} would be very nixy.

related (attempt to customize LLVM https://github.com/NixOS/nixpkgs/pull/101786)

Also +1 for this btw. I'd be in favor of merging such an implementation as e.g. llvmPackages_git. That way it also wouldn't have to immediately replace the existing implementation (could be improved and extended incrementally) which should make the initial PR easier (focusing e.g. on just clang, lld, and their dependencies as well as a nice and clean diff). Unfortunately I don't have enough time to work on this myself (but I might be able to help a bit).
Documentation and references:

https://clang.llvm.org/get_started.html
https://llvm.org/docs/GettingStarted.html
https://llvm.org/docs/CMake.html
https://github.com/archlinux/svntogit-packages/blob/packages/llvm/trunk/PKGBUILD
https://wiki.archlinux.org/index.php/LLVM

i don't want to hijack this, but i think it's relevant: i arrived here while looking for a way to get a bleeding edge LLVM/Clang for a project that i'm trying to build (https://github.com/clasp-developers/clasp).
it would be very helpful for us if there were new hydra builds of llvm HEAD every couple of weeks.
sometimes we even need an LLVM with a patch of ours, and i'm looking into setting up a local nix cache for that, but it's not a pressing issue right now (we have no pending LLVM patches at the moment).
shall i hold my breath for an llvmPackages_git? :) or llvmPackages_13 that gets updated every once in a while to follow LLVM's git head?

shall i hold my breath for an llvmPackages_git? :)

Likely not, we're unfortunately lacking LLVM maintainers. It would certainly be nice though.

it would be very helpful for us if there were new hydra builds of llvm HEAD every couple of weeks.

An additional issue is to decide when to update llvmPackages_git. The best solution that I know of would be to automate this and base it on the version Chromium Dev is currently using:

Chromium ships a prebuilt clang binary. It's just upstream clang built at a known-good revision that we bump every two weeks or so. (Source: https://chromium.googlesource.com/chromium/src/+/master/docs/clang.md)

That way it would be clear when and to which revision to update LLVM and that revision should already be well tested (known-good).
See https://github.com/NixOS/nixpkgs/pull/101786/files#diff-8f208f3301c344e2c9bc9020c2b20c6585c642e8a9601a25669b2998da1c1247R43 for an example to extract this information and there are regular Chromium Dev releases: https://chromiumdash.appspot.com/releases?platform=Linux
I guess I could even get llvmPackages_git going based on llvmPackages_12 but we"d really need someone to actually maintain it (i.e. regularly update it, fix build failures/regressions, keep the Nix expressions maintained, etc.).

@primeos what's wrong with just pushing it, and then whenever bitrot happens wait for the opensource fairies to open a PR?
i'm rather new to nix, but i'd certainly keep an eye on it while working on clasp, and do my best to fix things if they break.
is there a way to mark a package "non-essential", or "optional"? i.e. failing to build shouldn't cause any blockage. llvmPackages_git would be a good candidate.

@primeos what's wrong with just pushing it, and then whenever bitrot happens wait for the opensource fairies to open a PR?

It's an option but the probability to get problematic regressions is much higher, people might disagree on which revisions to pick, when / how often to update, etc. That's why I'd prefer to stick to a clear plan (also e.g. our gn in Nixpkgs is based on master and IIRC it basically bitrots until something breaks and someone updates it) - it's not required but I fell like it'd work much better.

is there a way to mark a package "non-essential", or "optional"? i.e. failing to build shouldn't cause any blockage. llvmPackages_git would be a good candidate. (that is, unless it would be used for building chromium)

It would be non-essential by default so it should be fine. It might even be best to add the attribute via pkgs/top-level/aliases.nix so that nothing in Nixpkgs will "accidentally" depend on it (which would make updates much easier but we need to check if Hydra still builds it then).

I've drafted a PR for llvmPackages_git based on the upstream Git monorepo instead of the individual tarballs: #120780
It's far from perfect and a clean slate approach would be nicer but so far it doesn't seem realistic that someone will put in the effort. Therefore it seems like a better idea to start by copying llvmPackages_12 and improve / clean it up incrementally (with much less impact on Nixpkgs).

I absolutely agree we should have a derivation to track HEAD, but i think it should be built in the manner of the old style.
Until we are using CA derivations (https://github.com/NixOS/rfcs/pull/62) in Nixpkgs, we have no good way to filter stuff out of sources without polluting downstream hashes, so the current approach, as cumbersome as it is, is necessary to ensure individual packages can be patched whatnot nicely with minimal rebuilds.
(And to be clear, somehow filtering the source is a must because otherwise we could accidentally rebuild parts of old packages. The LLVM build system is pretty intense, and I absolutely don't trust it to ignore source code it shouldn't care about.)
I'm sorry if that's a bit of a bummer, it's precisely cause these things are annoying that I worked on https://github.com/NixOS/rfcs/pull/62 in the first place!

@Ericson2314 no sure what you mean here - could you e.g. give an example?

I absolutely agree we should have a derivation to track HEAD, but i think it should be built in the manner of the others.

AFAIK that's not possible (the tarballs are release assets, so we'd have to generate them manually for non-releases - which is of course possible but I don't see how this would be a good idea).

we have no good way to filter stuff out of sources without polluting downstream hashes, so the current approach, as cumbersome as it is, is necessary to ensure individual packages can be patched whatnot nicely with minimal rebuilds.

Upstream switched to the mono repository layout so I don't see why we should continue using our hacks for combining individual tarballs to fix builds. IMO we should avoid patches as much as possible and I don't really see the rebuilds problem (we can still use the patch phase as usual).

I absolutely don't trust it to ignore source code it shouldn't care about.

Maybe that's the actual problem (i.e. that we wouldn't know which packages we need to patch)? With upstream transitioning more and more to the mono repository layout I don't see how this'll be easily avoidable.
Anyway, should I read this as a veto against #120780 or not?

AFAIK that's not possible (the tarballs are release assets, so we'd have to generate them manually for non-releases - which is of course possible but I don't see how this would be a good idea).

It's possible to delete stuff at fetch time in
https://github.com/NixOS/nixpkgs/blob/50a11f4f4301b9b4cb1f3041fca4f2e71a73d4a5/pkgs/build-support/fetchzip/default.nix#L14
but that's quiet annoying, and I can't think of a time we bumped the fetched sources individually.
So I relent, it's fine to fetch everything at once and then split up.

Upstream switched to the mono repository layout so I don't see why we should continue using our hacks for combining individual tarballs to fix builds.

We can do src = runCommand "${pname}-src}" {} "take just what we need from ${src}";
Maybe we should also apply patches in the source post-processing derivations too.

IMO we should avoid patches as much as possible and I don't really see the rebuilds problem (we can still use the patch phase as usual).

Yes, per project patching is good.

Maybe that's the actual problem (i.e. that we wouldn't know which packages we need to patch)? With upstream transitioning more and more to the mono repository layout I don't see how this'll be easily avoidable.

Upstream still aims to support separate builds. I think they CI for it too. If we whitelist the source subdirs as described above to replicate how individual tarballs were combined previously, it should be fine.

Anyway, should I read this as a veto against #120780 or not?

Well I think there's still good stuff in there even if you do change to this approach?

We can do src = runCommand "${pname}-src}" {} "take just what we need from ${src}";

That seems like a feasible option. No sure if it's really worth it but at least it would require few extra effort.

Upstream still aims to support separate builds.

I didn't have that impression so far (they seem to increasingly hard-code paths according to the mono repository layout) but maybe it'll get better?

If we whitelist the source subdirs as described above to replicate how individual tarballs were combined previously, it should be fine.

Yeah, that should remain fine.

Well I think there's still good stuff in there even if you do change to this approach?

Not sure - https://github.com/NixOS/nixpkgs/pull/120780/commits/db1ce992c5138426ca3dc430a6bd52d749bf7efb is the relevant diff and might indeed safe some time. I guess we should then replace src (and ideally also sourceRoot but I don't see an elegant solution without modifying stdenv.mkDerivation) with some function that takes a list of relevant subdirectories.

I'm thinking let's land #111487 next and then redo #120780 in light of this and matching the general normalization I did in that.

Sounds good.

I'm happy to help out.

Thanks! That would be useful as I'm currently low on time and don't have much motivation to try out different approaches.
Tbh I'm already thinking about removing myself as LLVM maintainer as I was hoping to heavily refactor/modernize (/ incrementally rewrite) llvmPackages_git but after our discussion in https://github.com/NixOS/nixpkgs/pull/120506#issuecomment-827295512 I'm no longer too optimistic that this'll really help as we still depend too much on old/ancient LLVM versions and I'm starting to realize that some PRs might indeed want/need to modify all of them (I was hoping that for most PRs it would be enough to fix only the most recent LLVM version but for cross-compiling, etc. this obviously isn't enough).

I didn't have that impression so far (they seem to increasingly hard-code paths according to the mono repository layout) but maybe it'll get better?

I still see lots of conditional "is this a separate project" code and the beginning of CMake files and whatnot. I do fear most of the devs mainly do everything builds (even before the monorepo), so there is a bit of a bias for that, but as long as the official policy isn't changing we're good.

Tbh I'm already thinking about removing myself as LLVM maintainer...

Sorry to hear that, but I do empathize. I would be nice if we could retire old versions after. I think Darwin is supposed to finally leave 7 behind, which should help with that? (It was stuck there a while.)

this is somewhat offtopic here, sorry about that!
but is there a way to get llvmPackages_git binaries from hydra, without local recompilation?
this is what i'm doing in my default.nix for a project:
{}:

let

  # to use llvm from the unstable channel
  pkgs = import (fetchTarball https://github.com/NixOS/nixpkgs/archive/nixpkgs-unstable.tar.gz) {};
  #pkgs = import <nixpkgs> {};
  llvmPackages = pkgs.llvmPackages_git;

in

llvmPackages.clang.stdenv.mkDerivation {
  pname = "clasp";
  version = "unstable";

  src = ./.;

  nativeBuildInputs = with pkgs; [
    python git sbcl wafHook
    llvmPackages.llvm
    llvmPackages.clang
    libffi.dev boehmgc.dev
  ];

  buildInputs = with pkgs; [
    llvmPackages.libclang.out
    libffi boehmgc
    gmp zlib ncurses
    libelf libbsd boost
  ];

  buildTargets = "build_cboehm";
  installTargets = "install_cboehm";

  shellHook =
  ''
    alias ..='cd ..'
    alias ...='cd ../..'
  '';
}

Indeed, aliases aren't built on Hydra and there doesn't seem to be an existing way to bypass that in aliases.nix:
https://github.com/NixOS/nixpkgs/blob/41387135ddb01d84049d1931292cacabbde54a34/pkgs/top-level/aliases.nix#L6-L18
This may or may not be desirable but in any case llvmPackages_git isn't useful yet and updates require quite some effort ATM: #125698 (and I lack motivation to deal with some of those issues).


hanldeReminder:  label handling improment

Lables should NOT be removed and readded as this cuases too many log entries.
Cache existing labels and compare to desired labels. Removal and addition should only occure if theres a change needed.

fixed


fullscreen layout

I want to use vim-clap in a fullscreen layout. setting
  g:clap_layout = {'width': '50%', 'height': '100%', 'row': 0, 'col': 0}

doesn't work because the first result in the results list window will be covered by the prompt. I think maybe this could be fixed by carefully tweaking some of the layout arithmetic.

Try this one:
let g:clap_layout = {'width': '49%', 'col':'1%', 'height': '93%', 'row': '3%' }


On running commentsBox example encounter error " code 404, message File not found"

Following the instructions in the README file. Running the commentsBox app results in the following error
code 404, message File not found

Should be fixed by #96.


Copy Paste from the "box" in context menu.

It was very big negative surprise that such powerful and wonderful instrument have not such elementary things. It like make it without "Quit" option - just kill from manager task, All extensions that people did work very bad. 3 times copy is OK, 5 times - NO. Some nothing at all. Could you add normal copy paste context options from "box"?

On which OS are you?
Are you sayng there isn't a Quit option or it doesnt' work?
Some extensions are not always updated by their author. If you have troubles with them you should open issues on their repository.
Are you saing Ctrl / Cmd + C and Ctrl / Cmd + V don't work well?
There are already some issues about adding Copy / Cut / Paste to the context menu.

Duplicate of #12063


Log resolution failure

Implements #154
When resolution fails, this logs description of expected parameters, and lists all available registrations for given service type.
Example behaviour:
let container = Container()
container.register(Int.self, name: "A") { _ in 0 }
container.register(Int.self) { (_, a: String, b: Double) in 0 }
container.register(Double.self) { _ in 0 }

container.resolve(Int.self)

Swinject: Resolution failed. Expected registration:
	{ Service: Int, Factory: (Resolver) -> Int }
Available registrations:
	{ Service: Int, Name: "A", Factory: (Resolver) -> Int, ObjectScope: graph }
	{ Service: Int, Factory: ((Resolver, String, Double)) -> Int, ObjectScope: graph }

Disabling / customising of this feature can be done by changing Container.loggingFunction variable.

It's a very cool feature to have. Thank you for implementing it👍 As you told, it's helpful to notice the type mismatch problems!


Cleanup dependencies

Cleanup dependencies, and modernize all of them to current.
Change Item decoration used.
Text size alignment on product screens.

Looks good


Add the ability to use a different Jira project depending on Incident…

Adds the ability to use a different Jira depending on the incident type. Adds a JSON column to the incident_type DB table which allows you to place JSON data for the given plugin. The JSON first level keys are plugin slug.
Example data:
{
  "dispatch-jira-plugin": {
    "project_key": "TEST123",
    "issue_type_id": 10244
  }
}

Also add a name to the unique constraint from the previous db migration.

Updated to default dict {} for plugin_metadata.  Will PR generic Jira plugin in another PR, does that work?


User sessions started from non-wsl.exe sources (such as systemd services) don't have WSL_INTEROP

Version
Microsoft Windows [Version 10.0.22000.856]
WSL Version

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>-20220921-1-microsoft-custom-WSL2+
Distro Version
Debian bookworm/sid
Other Software
No response
Repro Steps

Ensure that your WSL distro has systemd-machined installed and enabled.
Start a WSL session.
At this point, in the shell created by wsl.exe, echo $WSL_INTEROP will return a value, such as /run/WSL/136_interop
Use sudo machinectl shell<EMAIL_ADDRESS>to enter a user session under systemd.

(This is used as a workaround to get a systemd user session created; see #8842 . But the same issue applies to any user session created under systemd/by non-wsl.exe means, including ssh, etc.)

At this point, echo $WSL_INTEROP will show nothing.
Attempt to run a Windows application, such as /mnt/c/Windows/notepad.exe

Expected Behavior
The Windows application should run as normal for interop.
Actual Behavior
The Windows application fails to run and exit code 1 is returned:
❯ /mnt/c/Windows/notepad.exe
[1]    31535 exit 1     /mnt/c/Windows/notepad.exe

Diagnostic Logs
No response

Thanks - I agree launching Windows processes from things launched from systemd would be a good enhancement and shouldn't be too bad to implement.  I'll see what I can do.

Voting for this enhancement - I am specifically prefer to use my XWin instead of wslg and till now i enabled it from systemd user unit. Unfortunately it stopped working when I moved to builtin systemd

Same problem in 68.2, I'm afraid. Both, in fact - no WSL_INTEROP in the systemd or systemd user session environment, and the binfmt is lost.

The solutions provided in #5065 are a workaround for my own systemd services


Media Notification & Lock screen notification Not working

In Android 10, media notification is not working, even on lock screen notification widget is not showing on Android 8.1.1.

For android 10 you need to add a permission:


Gravity machine  NOT building or working

Version of Advanced Rocketry
[AdvancedRocketry-1.11.2-1.4.0-90]
[insert here]
Have you verified this is an issue in the latest unstable build
Y
Version of LibVulpes
[LibVulpes-1.11.2-0.2.8-36-universal]
Version of Minecraft
[1.11.2]
Crash report or log (if applicable)
http://pastebin.com is a good place to put them
crash reports that are put in the issue itself are
hard to read
[N/A]
Description of the problem
[Build the Gravity machine on the moon. Does not build or work.

]

Is this still a issue? Please comment if it is and i will reopen if if you are unable? We are closing old issues to see what is still relevant and needs to be fixed.