❓ [Question] How to solve this warning: Detected this engine is being instantitated in a multi-GPU system with multi-device safe mode disabled.

❓ Question
I used Torch-TensorRT to compile the torchscript model in C++. When compiling or loading torchtrt model, it displays many warnings.
WARNING: [Torch-TensorRT] - Detected this engine is being instantitated in a multi-GPU system with multi-device safe mode disabled. For more on the implications of this as well as workarounds, see the linked documentation (https://pytorch.org/TensorRT/user_guide/runtime.html#multi-device-safe-mode)
WARNING: [Torch-TensorRT] - Detected this engine is being instantitated in a multi-GPU system with multi-device safe mode disabled. For more on the implications of this as well as workarounds, see the linked documentation (https://pytorch.org/TensorRT/user_guide/runtime.html#multi-device-safe-mode)
WARNING: [Torch-TensorRT] - Detected this engine is being instantitated in a multi-GPU system with multi-device safe mode disabled. For more on the implications of this as well as workarounds, see the linked documentation (https://pytorch.org/TensorRT/user_guide/runtime.html#multi-device-safe-mode)

What you have already tried
I found this link is useful, but it only provides Python API.
I checked the source code, but I still haven't figured out how to set up MULTI_DEVICE_SAFE_MODE in C++.
What can I do to address this warning?
Environment

Build information about Torch-TensorRT can be found by turning on debug messages

PyTorch Version (e.g., 1.0):
CPU Architecture: x86
OS (e.g., Linux): ubuntu18
How you installed PyTorch (conda, pip, libtorch, source): libtorch
Build command you used (if compiling from source):
Are you using local sources or building from archives:
Python version:
CUDA version: 12.2
GPU models and configuration: 1080Ti
Any other relevant information:

Moreover, when the model infers using forward function, there are these warnings:
WARNING: [Torch-TensorRT] - Using default stream in enqueue()/enqueueV2()/enqueueV3() may lead to performance issues due to additional cudaDeviceSynchronize() calls by TensorRT to ensure correct synchronizations. Please use non-default stream instead.
WARNING: [Torch-TensorRT] - Using default stream in enqueue()/enqueueV2()/enqueueV3() may lead to performance issues due to additional cudaDeviceSynchronize() calls by TensorRT to ensure correct synchronizations. Please use non-default stream instead.
WARNING: [Torch-TensorRT] - Using default stream in enqueue()/enqueueV2()/enqueueV3() may lead to performance issues due to additional cudaDeviceSynchronize() calls by TensorRT to ensure correct synchronizations. Please use non-default stream instead.

Hi @demuxin - thanks for the report - we likely need to add getter/setter methods to toggle this value in C++ as well:
https://github.com/pytorch/TensorRT/blob/07c5b07b5ca81165682be90049a6bc67cff9928e/core/runtime/runtime.cpp#L10
Similar to the following functions:
https://github.com/pytorch/TensorRT/blob/07c5b07b5ca81165682be90049a6bc67cff9928e/core/runtime/register_jit_hooks.cpp#L121-L122
We are aware of the default stream warning and are working on this. It should not have a substantial effect on inference from what I've seen

These may actually already be accessible in C++, prior to inference, could you try adding the line:
torch_tensorrt::core::runtime::set_multi_device_safe_mode(true)

Hi @gs-olive, this is not right, there is complie error:
error: ‘set_multi_device_safe_mode’ is not a member of ‘torch_tensorrt::core::runtime’
   27 |         torch_tensorrt::core::runtime::set_multi_device_safe_mode(true);

Does torch::ops::tensorrt::get_multi_device_safe_mode() exist, or does this also cause a compilation error?

This is a similar compilation error.
error: ‘torch::ops’ has not been declared
   24 |         torch::ops::tensorrt::get_multi_device_safe_mode(true);

I searched the libtorch and torch-TensorRT header files, and there are no functions related to multi_device_safe_mode.


[Planning] Integrate PDDLStream task and motion planner

A big reason for this repo existing is that it can be used to prototype task and motion planners.
One such planner which will be used as the main demo is PDDLStream (https://github.com/caelan/pddlstream).
We should integrate a few demos that use this planner as a regular PDDL planner (with only discrete variables), plus with the addition of continuous variables such as sampling placement poses and navigation plans (e.g., RRTs).
We will also have to figure out the necessary steps / instructions / scrips to ensure that PDDLStream works easily with this package.

I am excited for this. I am working on PDDLStream for my project as well.

Closed by #14 and #17.


cmd/link: command line too long running gcc

What version of Go are you using (go version)?

$ go version

go version go1.20.2 windows/amd64

Does this issue reproduce with the latest release?
Unsure how to test
What operating system and processor architecture are you using (go env)?
Windows 11 22H2, msys2 environment
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=G:\G
set GOENV=W:\Users\ender\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=W:\Users\ender\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=W:\Users\ender\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=W:\msys64-gtk4\mingw64\lib\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=W:\msys64-gtk4\mingw64\lib\go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=W:\msys64-gtk4\tmp\go-build3989973909=/tmp/go-build -gno-record-gcc-switches

What did you do?
I tried building gtkcord4. The build works fine right up to the link step, where it fails with W:\msys64-gtk4\mingw64\lib\go\pkg\tool\windows_amd64\link.exe: running gcc failed: fork/exec W:\msys64-gtk4\mingw64\bin\gcc.exe: The filename or extension is too long.. Running the linker manually with -v parameter shows that it's trying to pass over 1000 .o files with full path to gcc, resulting in command line that's almost 60 kB, while the command line length in Windows is limited to 32 kB.
What did you expect to see?
Successful link.
What did you see instead?
W:\msys64-gtk4\mingw64\lib\go\pkg\tool\windows_amd64\link.exe: running gcc failed: fork/exec W:\msys64-gtk4\mingw64\bin\gcc.exe: The filename or extension is too long.
Is there any way to tell link.exe to use a @response file instead?

Unfortunately getting the latest golang doesn't seem to work – when I ran go install golang.org/dl/gotip@latest the first time, it said it downloaded something, but running gotip afterwards resulted in -bash: gotip: command not found. I tried restarting msys2, now there's no output from go install, and still no gotip command.
Note that I'm using msys2, because the program I'm trying to build depends on GTK4, and I have no desire to compile GTK and its dependencies myself.
I'll test again once this lands in msys2, but from the description of 414654, it'll probably solve my problem.

gotip will be in $GOPATH/bin

OK, found it – for some reason $GOPATH points to my Windows user profile\go instead of inside msys2 root. Anyway, gotip install was successful.

Sorry, I wasn't clear – gtkcord4 built fine with gotip.

Sorry, I wasn't clear – gtkcord4 built fine with gotip.

Then I'll close this issue as done.


[SPARK-43353][PYTHON] Migrate remaining session errors into error class

What changes were proposed in this pull request?
This PR proposes to migrate remaining Spark session errors into error class
Why are the changes needed?
To leverage PySpark error framework.
Does this PR introduce any user-facing change?
No API changes.
How was this patch tested?
The existing CI should pass.

Merged to master.


Java 7

It can work on Android too, but the project was written in Java 8 (Stream API was used a lot)
Android supports Java 8 from 24 API (Android 7)
Android 4-6 are still very popular
Would it be hard for you to rewrite the code to Java 7?

I wrote this as a hobby project and I have no motivation to rewrite it in legacy code. Sorry!


Install failed: Fix image tag for helm install

Following https://istio.io/docs/setup/kubernetes/quick-start/#download-and-prepare-for-the-installation
When installing istio with helm, images can't be pulled because the tag has been changed from 0.8.latest to 0.8.0
❯ docker pull docker.io/istio/citadel:0.8.latest
Error response from daemon: manifest for istio/citadel:0.8.latest not found

This PR allows the image to be pulled correctly from docker.io
Proof: https://hub.docker.com/r/istio/citadel/tags/

Hi @veggiemonk. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: veggiemonk
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: greghanson
Assign the PR to them by writing /assign @greghanson in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/check-cla

See discussion here: https://github.com/istio/old_issues_repo/issues/387

@ymesika @gyliu513 The tag is now 1.0.0-snapshot.0, should this PR be updated ?

@veggiemonk: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ymesika @gyliu513 ping


gender neutral language

change "she uses" to "they use"

@emberstevens : Thanks for your contribution! The author, @mrbullwinkle, has been notified to review your proposed change.

This article was slated for retirement, and has now been retired and is no longer live.
#please-close


Update scalafmt-core, scalafmt-dynamic to 3.7.11

About this PR
📦 Updates

org.scalameta:scalafmt-core
org.scalameta:scalafmt-dynamic

from 3.7.2 to 3.7.11
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scalameta" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #105.


fix: add normalization in get_random_weighted_columns

Adds a normalization step in get_random_weighted_columns  (used when the arg use_max_modalities = False, which is how it is used in avatar).
Ensures that the max cumsum value is 1 and that all modalities can be sampled.

get_random_weighted_columns

there is another test: test_get_random_weighted_columns which uses 4 sets of parameters. This was not changed and still passes following the fix

get_random_weighted_columns

there is another test: test_get_random_weighted_columns which uses 4 sets of parameters. This was not changed and still passes following the fix

My bad, I was looking for a test_get_random_weighted_columns.py file


motion projection mode is optional and sensitive for the config

@CatoTH Can you plug-in the logic required by getSlide()?

@CatoTH @emanuelschuetze I added CRs as a dependency to motions -> in view motions one can now access CRs (Tobias: is this helpful for you in some other cases?)

@FinnStutzenstein Would the same be possible with the amendments to a motion? In that case, querying these in the motion detail view could be solved slightly more elegant...

This PR already works for amendments because an amendment has no CRs. Thr mode buttons are hide.

From my understanding, a motion that has amendments, but no CRs, should still have the buttons (contrary to a motion without none at all). For that, having the amendments in the ViewMotion would be necessary.


Implement a CONTRIBUTING.md file

See also https://opensource.guide/how-to-contribute/.

Done with f7791e6dee1cbe32728665c92b3dbf036cde7a82.


feat: openai completions polymorphic prompts

Summary
Add support for polymorphic prompts in OpenAI/GPT3/s Completions endpoint as described here: https://github.com/Azure/OpenAI/blob/ed700c41469a799e30baedbbc13c62aa634b777f/Reference/swagger_2021_11_01.json#L64
Tests
Added one test for each new prompt form, in OpenAISuite.scala
Dependency changes
N/A
AB#1790206

def setPrompt(v: String): this.type = setScalarParam(prompt, v)

I tried a much simpler change where I just overloaded setPrompt with Seq[String], Seq[Int] and Seq[Seq[Int]], but I couldn't get the new forms to work. Let me know if that's something I should look further into.

Refers to: cognitive/src/main/scala/com/microsoft/azure/synapse/ml/cognitive/OpenAI.scala:41 in a981b38. [](commit_id = a981b3891b963687960003c93359326dd4baa104, deletion_comment = False)

/azp run

Codecov Report

Merging #1510 (a981b38) into master (9657a53) will decrease coverage by 83.73%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master   #1510       +/-   ##
==========================================
- Coverage   84.30%   0.57%   -83.74%
==========================================
  Files         296     267       -29
  Lines       14906   13651     -1255
  Branches      717     722        +5
==========================================
- Hits        12566      78    -12488
- Misses       2340   13573    +11233

Impacted Files
Coverage Δ

.../microsoft/azure/synapse/ml/cognitive/OpenAI.scala
0.00% <0.00%> (-55.98%)
:arrow_down:

...m/src/main/scala/com/microsoft/lightgbm/SWIG.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...n/scala/org/apache/spark/ml/param/ModelParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...n/scala/org/apache/spark/ml/param/UDPyFParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...cala/org/apache/spark/ml/NamespaceInjections.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...cala/org/apache/spark/ml/param/DataTypeParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...ala/org/apache/spark/ml/param/ByteArrayParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...ala/org/apache/spark/ml/param/EvaluatorParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...la/org/apache/spark/ml/param/ParamSpaceParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...la/org/apache/spark/ml/param/TypedArrayParam.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 279 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9657a53...a981b38. Read the comment docs.

/azp run

/azp run

Switched to a different branch and checked in there.


Added gradient alias for nabla symbol

Personally, I really need a grad/gradient alias for nabla symbol, because it describes a gradient (∇f(...) is a gradient). I never heard nabla outside of searching "what is the symbol for gradient?". IMHO, a lot of folks will agree.
I'm not sure about the placement of the alias. We can change it, but I think it's in a good spot already.

physica package did this

Also, related discussion in https://github.com/typst/typst/issues/851, where it was last decided that we would keep with the nabla name. It seems like we completely forgot that we could have gradient as an alias... :facepalm:

One concern of mine is that I would rather do something like:
    gradient: nabla,
    nabla: '∇',

Because this makes more sense and reduces the repetitiveness of the raw Unicode characters in the file. It's much easier to read "gradient is the alias for nabla" this way, than "gradient is (the alias for) ∇".

I also want to note that for some reason the partial differential is shortened to diff, but integral is not shortened to int, which makes this not compatible with latex afaik, and also doesn't make much sense in general. So just from this, we can either add grad or gradient — both will "fit in". I don't remember, but latex probably has grad and I would agree that 4-letter abbreviation is long enough, especially when you have multiple symbols in one line. Maybe we should reconsider some long symbol names, or just add "an alias" int to integral (for example).

Seems like there is strong consensus for this, so I'm fine with it. Thanks!

Small correction to the small correction: a gradient does not need to be a vector. E.g., functional gradients

No wonder why Latex named the plain triangle symbol nabla instead of gradient, since there are close-but-different high level meanings.
IMO naming the plain triangle "gradient" is an overreach -- it's like Typst should not (and it does not) really name a square symbol the "d'Alembert operator". A square is a low-level name, a "d'Alembert operator" is a high level name in physics' context, and it may have other meanings in different contexts. But since the PR is submitted already I guess there's no much hurt to keep it.

naming the plain triangle "gradient" is an overreach

@Leedehai, do you suggest naming gradient the "non-plain triangle"? As in "bold triangle"?
Right now, Typst can only name symbols — Unicode chars. Maybe in the future if we can wrap nabla in strong and then name that gradient, maybe we can do that. But I don't think this is something we will see any time soon (if at all).
I have typesetted gradient a few times and I didn't see any "extra boldness" on the nabla. If I ever see it again, I will try and save that example for future references.

No, I mean, I actually prefer Typst doesn't add the name "gradient" to the symbol.

I actually meant I'd prefer Typst to not add "gradient" (or variations of it) name to that nabla symbol -- see my "IMO naming the plain... keep it" paragraph.

IMO naming the plain triangle "gradient" is an overreach -- it's like Typst should not (and it does not) really name a square symbol the "d'Alembert operator". A square is a low-level name, a "d'Alembert operator" is a high level name in physics' context, and it may have other meanings (and preferred looks) in different contexts.

I'd wager that gradients are much more common than d\Alembert operators. In the end, we just get back to the same thing we get at whenever we discuss math-related things, which is that we have to draw the line somewhere in regards to what gets included, and there is no line that will keep everyone happy. The current plan is to seek a line that keeps a reasonable amount of people sufficiently happy (as in, the stuff they need is conveniently available and sensibly so) by adding things that are "reasonably common" to std and moving the remaining stuff to packages (of which there is still quite a bit of work to do).
Also, relevant discussion: https://discord.com/channels/1054443721975922748/1184462259871821854/1184462259871821854

I can only add/emphasize one point: convenience. I think I'm a good example, since I don't care, and I can't remember that gradient uses nabla symbol. I don't do math, but on the occasions when I need to write a gradient, I will always say: "how does that symbol called again?" And instead of searching for the answer in a SE I can just happily use gradient and move on.
There is a certain threshold when some of the symbols/operators are too niche or used mainly only by mathematicians, in such cases those people probably know which operator uses which symbol, you know what I mean? For that, there is almost no benefit for majority (that doesn't use such symbols/operators) or for that small group (that already know the core Unicode/math chars that are used). And gradient is simply not niche at all + not everyone can/want to remember how to typeset one. That's all there is to it.


dbt-core 1.5.x support

Adding support for dbt-core 1.5.x
⚠ Breaks support for older versions of dbt-core. For older versions of dbt-core, users would have to install prior dbt-invoke versions. ⚠
Given the many changes in the dbt-core API, I'd argue that backwards compatibility increases code complexity of this project to the extend that it becomes to much of a burden to keep the code clean and simple. Also, this allows for a cleaner cut to switch to dbt-duckdb for testing.
Addresses https://github.com/Dashlane/dbt-invoke/issues/34
Changes:

migrating to dbt-duckdb for testing

running dbt project as part of the tests
fixing tests for new columns in seeds (dbt_scd_id and dbt_updated_at)

fixing issues with new json log formats
black replaced all single quotes with double quotes. I hope this is fine ;)

@robastel feel free to review.
Could you also remove the tests for dbt-core <1.5.x from the expected tests in case you agree to drop support for older dbt-core versions?

Thanks for submitting this pull request @ciklista !
I need a bit of time to review. It looked like a lot of changes, but I see much of it is just changing single quotes to double quotes, as you mentioned.
Black has an option -S for skipping string normalization, as they call it: https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#s-skip-string-normalization
For example, I use that option in the lint job here: https://github.com/Dashlane/dbt-invoke/blob/c749c72698083e39f2d85659ca225c99235b8968/.github/workflows/test.yml#L26C8-L26C8
Are you able to use that option to get rid of the quote changes (or at least separate them into their own commit) so that it is easier to review? I want avoid overlooking an actual logic change in the sea of quote changes.
Thanks again for taking this on!

Thanks for the feedback @robastel I reverted the quote changes.

Thanks for the feedback @robastel, I reverted the quotes changes.

Thanks again @ciklista, that definitely helped the readability for me. I've done a quick read through of the changes and plan to take a more thorough look in the coming days.

@robastel did you have the chance to look into the code? Are there any open questions?


Update app.yaml

This lab fails with the following error - ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Error(s) encountered validating runtime. Runtime python37 is end of support and no longer allowed. Please use the latest Python runtime for App Engine Standard..


Landscape

Hello! I've been commenting recently on a couple of issues, specifically, this branch should fix #103, #93  and #67, and hopefully not break too many unrelated things. This probably fixes #86 as well, I'm not quite sure what was going on with that issue. I intend to be responsive to any issues that arise, time permitting. That said, I've tested it under emulation for API 15, which works fine, and on my tablet, and my phone.
Thanks!

I just removed a couple of unrelated commits from the tip, they were, unfortunately, not as mature as I thought they were.

Tweaked one line in the layout refactor and forced the commit.

Hello @seizonsenryaku. First of all I don't want to come off as impatient, I don't mind how long this process may take (I've been to busy moving to work on this right now too). I would really appreciate knowing if there is an intention to move this work upstream. I love this launcher. I have a queue of commits so far, with no real end in sight.
What I'm working on:

Reducing maintenance/translation burden
Reducing compiled size
General updating where it makes sense
Coming up with a code style which fits this code base so the code can be uniform and more easily read
Reducing the code lint where it makes sense

What I have no intention of doing:

Adding bloat. I love the minimalism of this launcher, I like minimal code. At the same time, when features improve the usefulness, I would consider sending a PR as long as there's a good argument for it
I would be completely skeptical about adding permissions of any type as well, unless the feature would come with a completely optional popup and are well explained to the end user. There might be cases where it becomes completely necessary (the PR #98 comes to mind at the moment)

One thing I'll say is there is so much well thought out code in here. I am here to help clean it up in any way I can. I really don't want to fork it at all. If there's anything I can do to help move this forward, from changing my commit process (smaller targeted commits?) to code style (I try to finalize most fields and variables so it's obvious at a glance immutable status) or anything else, I'm all ears.
Thanks!

Latest push just added multi-window support.

I'm going to pull this pull request. Our code bases are going to fork too far to try to remerge without /some/ kind of cooperation or communication. At this point, any effort I try to keep this branch mergable is lost energy as long as there's no communication. I'm flexable and I'm willing to bend over backwards to be able to remerge, so feel free to shoot me an email at any time if you'd like to remerge.


[HUDI-4504] Disable metadata table by default for flink

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Sorry, @codope , i was still testing it, deciding to close it first before i complete the tests, maybe i would fix in in following RC if i found it is problematic.

https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=10449&view=results
The CI is actually green and i would merge it then ~


Pinch and zoom

Is it possible to add pinch and zoom to just image and not the whole shape?
And if yes please suggest me way to do it. I have tried it but it zoom whole shape and not the image inside it.

I created a PR in TouchImageView, https://github.com/MikeOrtiz/TouchImageView/pull/101 that does basically that same thing


[chef-client] Fix logging to /dev/null when log_file is set to that

Description

Issues Resolved

Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[ ] New functionality includes testing.
[ ] New functionality has been documented in the README if applicable
[ ] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

LGTM. Thanks


Vertical scroll doesn't scroll and gets disabled

Hi Team,
Vertical scroll doesn't scroll in browsers Google chrome, Firefox. This happens after the coding area is focused even for once.
Please, share a fix for this

Are you seeing this on http://codemirror.net/mode/javascript/ ? If not, try to find out what is different on your setup that might be causing it


Drop support for Ruby 2.3

What problem is this PR intended to solve?
Resolves #1886
Have you included adequate test coverage?
Not required.
Does this change affect the C or the Java implementations?
No, this doesn't.

@flavorjones requesting please to add to v1.11.0 at https://github.com/sparklemotion/nokogiri/milestone/18

Hi - thanks much for submitting this PR. Unfortunately, some of the files you're editing here are automatically generated by the dev dependency concourse-gem. I'll take the remaining changes through a manual merge, though.

@flavorjones No worries. Feel free to close this PR if that would be easier for you.

Manually merged after editing the commit to remove the concourse changes, and credited you in the CHANGELOG. Thanks!

Thank you for the mention @flavorjones.


Can't reach the web

Before you start
Please take a look at the FAQ as well as the already opened issues! If nothing fits your problem, go ahead and fill out the following template:
Describe the bug
Can't connect to the web
This site can’t be reached.
squoosh.app’s server IP address could not be found.
Expected behavior
to connect
Version:
macOS Mojave
Version 10.14.3
Additional context, screenshots, screencasts
It works on other devices on the same connection. It doesn't work on my laptop using diffrent wifis or browsers.

It works for me.
Is there any chance this is related to your ISP or your device's setup?

@TomasSestak What browser were you using?

Safari, Firefox, Chrome, also in private windows. It has to do something with my Mac because as I stated before it works on my phone or other devices even on the same network

Can you access http://www.spritecow.com/?

Can you access http://www.spritecow.com/?

yes, I can

Hm, so they're both hosted on Netlify, so it's not that.

Could it be the TLD?
@TomasSestak Can you reach https://web.dev?

Could it be the TLD?
@TomasSestak Can you reach https://web.dev?

Yes, I can

Could it be the TLD?
@TomasSestak Can you reach https://web.dev?

you might be right, https://get.app/ doesn't work for me

Does anybody know how to fix it?

Have you checked your hosts file for things that might block .app?

<IP_ADDRESS>	localhost
<IP_ADDRESS>	broadcasthost
::1             localhost
<IP_ADDRESS>	local.test
this is all I have there.

this showed up in Safari, but I can't seem to reproduce it

It might be from laravel valet I used to use, but I cant't find any records :/

I found it in etc/resolver, file called app, deleted, works now, thank you for help

Glad you figured it out!


Validation service

The validation service could either be run by the SBOM archival storage team or by another team in the organization that dictates policies on SBOM content.
The validation service could be implemented using Seedwing Policy, which has built in support for different SBOM formats and operations on SBOM documents.
The API will use the service to validate input.

Of the issues you've created so far, I believe I grok this one the most. Any thoughts on where the validation rules will live, who will define/maintain them, and how often?

I haven't given much thought about where they would live. We could make the API just take as argument a URL to a policy endpoint, where it would pass the input and be returned some seedwing rationale as output, and that's it. How seedwing itself gets deployed or managed would be up to whoever deploys the bombastic service and supporting services.
We could also bundle seedwing policies and embed seedwing itself, but I feel the flexibility of the first approach is more attractive right now.

Wondering if this would provide a good starting point: https://www.ntia.doc.gov/files/ntia/publications/sbom_minimum_elements_report.pdf

If it smells like policy, it's seedwing.  I'd vote for an independent seedwing service.

I had a look at this, and ran into the fact that we'd need to consume the stream in order to evaluate a seedwing policy.
Is there any obvious way around this? I was thinking if we implemented streaming support for seedwing, we could stream to seedwing and s3 at the same time, and somehow flip some pointer in s3 if policy evaluated well, or delete in s3.

Another option is to have the indexer eval the policy. That mitigates the stream consumption DoS risk, at the cost of not immediately failing the upload of an invalid doc. Plus we'd potentially store a crap-ton of invalid objects on S3 for some period of time. Maybe I'm talking myself out of this...
I don't love the simultaneous streaming to S3 & seedwing of a potentially invalid doc. Seems like a waste of resources if we immediately delete it from S3 -- we may even have to wait on the S3 upload to complete before deleting it.
Maybe we could introduce a feature of seedwing: "if this thing is valid, post it to this URL"?


Fix stencil color mask;

When calling lovr.graphics.stencil, the color mask is initially
disabled, and gets restored to its initial state afterwards.
However, when it's restored, it uses lovrGraphicsSetColorMask, which
just sets shadow state that doesn't make it all the way to GL until
another draw is done.
The consequence of this is that if you call .stencil and then don't do
a draw, any clears that happen will use the old (disabled) color mask,
preventing the color buffer from being cleared.
The solution here is to lower the color mask change down into opengl.c
where it can directly hit OpenGL.

Need to look over this to make sure there aren't any unintended consequences.

Doesn't work well in normal use. I'm testing with this code:
function lovr.draw()
  lovr.graphics.stencil(
    function()
      for x = -1, 1, 0.2 do
      for y =  0, 2, 0.2 do
      for z = -2, 0, 0.2 do
        lovr.graphics.box('fill', x, y, z, 0.05)
      end
      end
      end
    end)
  lovr.graphics.setStencilTest('less', 1)
  lovr.graphics.sphere(0, 0, -10, 5)
end

This is the correct render by the master branch:

This is render with this fix applied:

Both pieces of state needed to be updated.


LinBus is still not resolved by BOM

LinBus' groupId seems to be com.enginehub.lin-bus whereas the repository only includes the group org.enginehub (as well as com.sk89q - though that's not relevant). Therefor the most recent change does not seem to work.

Seems like the issue does not exist? At least linbus is properly resolved by FastAsyncWorldEdit-Core-Libs for me, so I'm gonna close this

https://discord.com/channels/268444645527126017/344128526435221505/1263321732777906268

Is it a not using Java 21 issue?

Mhm, that might be a good reason - I couldn't reproduce it so I linked the discord message

Was never going to work as expected - EH repo is added do API documentation


Support <reference> as the digest of manifest

In https://github.com/opencontainers/distribution-spec/blob/main/spec.md
<reference>  MUST be either (a) the digest of the manifest or (b) a tag
This PR is add support for the case of  (a) the digest of the manifest.
example:
ocipkg get quay.io/jitesoft/alpine:sha256:6755355f801f8e3694bffb1a925786813462cea16f1ce2b0290b6a48acf2500c

@termoshtt ocipkg is some great.
I did some patch allow ocipkg can load contain from oci image.
for some cases like https://github.com/comtrya/comtrya/issues/314.
Not sure what do you think about it.
Currently, ocipkg still missing something like:

get_image or pull_image into specific dir
handle multi-layer
handle docker v2 manifest
handle  the digest of manifest as    (this PR)

I would updated them later.

fixed https://github.com/termoshtt/ocipkg/actions/runs/9017870315/job/24802842566

Thanks PR.

get_image or pull_image into specific dir

Sounds good. It would be better to create a new API next to Artifact::unpack like unpack_on.


Pin maturin version in GitHub Actions

Description
This will fix the build fail for the release. I think it's better to pin down the version we will use here so that changes in Maturin don't break our pipeline. However I also think it's time to update the Maturin version we're using, so I created a separate issue for that: https://github.com/OpenMined/sycret/issues/39
Affected Dependencies
How has this been tested?
Ran locally
Checklist

[x] I have followed the Contribution Guidelines and Code of Conduct
[x] I have commented my code following the OpenMined Styleguide
[x] I have labeled this PR with the relevant Type labels
[x] My changes are covered by tests

Thanks @arturomf94! Does this PR also fix the current issue with the manylinux flag?

It should, momentarily. But I also raised this ticket: https://github.com/PyO3/maturin/issues/636. It has been fixed already so I will open a PR to update the maturin version.

I actually think it's better to merge this first, though, because we wouldn't want to rush into a newer version of maturin if we're not sure that it will work for us. Fortunately, maturin was working just fine before, so we should be ok with this fix and we can leave the update for later, whenever there's a release with the patch from https://github.com/PyO3/maturin/issues/636. What do you think?

I agree, thanks @arturomf94 (and nice for reaching out upstream). I'll merge!


Comments on version 0.0.2

Chapter 5 Some of the requirements are missing the RFC2119 keywords to indicate whether they are mandatory or optional.
ACPI_080 - implies that interrupt controllers may be either PLIC or APLIC. This statement may want to be expanded to include IMSIC and a corresponding RVI ACPI ID provided in section 6.2. In similar vein, section 8.3.6 "the MADT will also contain either PLIC or IMSIC/APLIC structures" would want to be written as "the MADT will also contain either PLIC, APLIC, or IMSIC structures"

@vlsunil do we need an IMSIC-specific CID for 6.2?

https://github.com/riscv-non-isa/riscv-brs/pull/163 for Chapter 5 improvements
Will send email to the mailing list to discuss the other points

#163 is merged.

Based onhttps://github.com/riscv-non-isa/riscv-brs/pull/166 and https://github.com/riscv-non-isa/riscv-brs/pull/164, we're good right? @ved-rivos


input seq has diff len for each patient

if each patient has a different len of observation, it will raise an error when making tensor in data_utils.py in the mix model.  We need to fix this.
Two proposals:

masking - need to change LSTM and TLSTM source code to include mask tensors and masking the gradients
padding seq for making tensor then add collate_fn to remove all paddings

proposed method for proposal 2:
d = load_pkl("static_seq_lable_test_set_mul.pkl")

nd = []
xnd = []

for i, each in enumerate(d[:5]):
    nd.append((each[0], each[1][:(i+1)*2, :], each[2]))
    xnd.append(each[1][:(i+1)*2, :])

nonseq, seq, label = [], [], []

for each in nd:
    nonseq.append(each[0])
    seq.append(each[1])
    label.append(each[2])

ds = TensorDataset(
                tensor(nonseq, dtype=float32),
                torch.nn.utils.rnn.pad_sequence([tensor(s,  dtype=float32) for s in seq], batch_first=True, padding_value=-100),
                tensor(label, dtype=long)
            )

def collate_fn(batch):
    tts = []

    # Gather in lists, and encode labels as indices
    for each in batch:
        n, s, l = each
        t = []
#         mask = ~each[0].eq(-100)
        for e in s:
            if e[0] == -100:
                continue
            t.append(e.numpy())
        tts.append((n, tensor(t), l))
#         ntt = torch.masked_select(each[0], mask)

    return tts

dl =  DataLoader(ds, sampler=SequentialSampler(ds), batch_size=1, pin_memory=True, collate_fn=collate_fn)


Implement mechanism for routing URIs exposed on objects

Implement mechanism for routing URIs exposed on objects (e.g. /path/to/object/ext:extension/), and proxied by API-X.
Simplifying assumptions:

Ignore selection criteria for the moment
assume that a request is routed to an extension based on URI matching alone, and that registering an extension at an exposed URI makes it available for all objects.

Depends on:

#5 (Extension registration/configuration model) so that a particular URI can be mapped to the extension that exposes it.  This information should be present in the registry entry for an extension.
#4 (Service invocation model)

Testing

Verify request is routed to service, response is routed back to caller

Implemented in #42


AWS-SDK-V3

I just couldn't get the import of credentials from the file to pass the tests and I haven't tried using it yet. But it's a first step

Any plan to merge this PR @benMain ?

The migration process is too risky and will create confusion for most users, so we never did this. Sorry.
https://github.com/deligenius/aws-sdk-v3-nest exists.


java.lang.ArrayIndexOutOfBoundsException: 8

http://pastebin.com/Gkqb1fzY

This is not a problem with my mod but instead one with the "CustomColors" mod

But I don't have this mod...

Do you have any mods that change/alter the sky in any way?

No, but..
Maybe, Optifine?
I have a resource pack that change sky..

Do you still get this error when you remove all other mods?

No, all working..

Then the problem is most likely not from my mod but instead a different one


Adapt view to Declarative Shadow DOM

According to https://github.com/Starcounter/RebelsLounge/issues/59
Follow the guidelines in the following three posts to adapt the views to use Declarative Shadow DOM:

Layout compositions for HTML partials
How to Apply New Shadow DOM Layouts to Existing HTML Templates
HTML View Guidelines

Implemented with https://github.com/StarcounterApps/Website/issues/80


Fix Exam Agent deployment

Description
This PR fixes exam agent deployment:

wrong parameter in deployment resource
wrong image name

/merge


Minor changes launch

Description
The user will now not be able to select knowledges which are striked off

Related Issues

Solution and Design

Test Plan

Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Docs update

Checklist

[x] My pull request is atomic and focuses on a single change.
[x] I have read the contributing guide and my code conforms to the guidelines.
[ ] I have documented my changes clearly and comprehensively.
[ ] I have added the required tests.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


API doc: BuildServiceProvider incorrect return type

BuildServiceProvider incorrectly states:
Returns
ServiceProvider
TheIServiceProvider.
See https://github.com/aspnet/DependencyInjection/issues/614

@dend @danroth27  see BuildServiceProvider /// return comments
Why is the API generating
Returns
ServiceProvider
TheIServiceProvider.
and not The IServiceProvider.

The API documentation reflects the true type that is used in code. The '/// comment is used as a description, where it was not modified to reflect the changes. The /// needs to be modified to include the correct type information, then packaged in an updated NuGet.

The API documentation reflects the true type that is used in code.

The 2.0 API is returning IServiceProvider, not ServiceProvider
@danroth27  Regarding the space, can you confirm that's what's in the XML embedded in the NuGet package used to generate the API docs?

@dend  should developers be instructed not to add  /// <returns> ? see BuildServiceProvider /// return

@Rick-Anderson If you take a look at the API, the docs reflect exactly what's in the source (and I am assuming the binary?):

The /// <returns> is not interpreted as the return type, but rather as a description for the return type, which is correctly displayed on docs.microsoft.com, as far as I can tell.

Ah our API docs are wrong 😄

@davidfowl I suspect you have in mind the triple-slash comment and not the presented return type? 😃

handled by https://github.com/aspnet/DependencyInjection/pull/615


Incorrect euler characteristics

I noticed that when I calculate the euler characteristics for the torus example data I get an euler characteristic of -119, whereas the correct value for a torus should be 0.
Here's some code to reproduce:
import vedo

mesh = vedo.shapes.Torus(r=50, thickness=10)
n_points = len(mesh.points())
n_edges = len(mesh.edges())
n_faces = len(mesh.faces())
euler_characteristic = n_points - n_edges + n_faces

Hi,
I think it's the way vtk builds the mesh, it creates additional edges that one can visualize (in magenta):
import vedo

mesh = vedo.shapes.Torus()
# mesh = vedo.shapes.Disc()   # ok
# mesh = vedo.shapes.Circle() # ok
# mesh = vedo.shapes.Cube()   # ok
# mesh = vedo.shapes.Sphere(res=(20,30)) # ok
mesh.clean()  # remove duplicate faces

bnds = mesh.boundaries()
mesh.lw(1).alpha(0.5).flat()

n_points = len(mesh.points())
n_edges = len(mesh.edges())
n_faces = len(mesh.faces())
euler_characteristic = n_points - n_edges + n_faces
euler_characteristic += bnds.npoints/2 +1 #only for torus correction..

# print(n_points, n_edges, n_faces, bnds.npoints)
print(euler_characteristic)

vedo.show(mesh, bnds, axes=1)

@marcomusy Thanks for looking into this - I also didn't know about the mesh.cleancommand. Thanks a bunch! That being said, would you think it desirable to replace the torus dataset by one without duplicate vertices?

Yes.. it could be done manually but it's a bit tricky... I need to think how to do it..


Turn EncodingWarning into errors and cleanup pytest.ini

Summary of changes
Attempt at superseding https://github.com/pypa/setuptools/pull/4234#discussion_r1511342695 w/o any potential breaking change
Essentially a band-aid for #3810 to make reading failed tests results more bearable.

Pull Request Checklist

[x] Changes have tests (these are the test changes)
[ ] News fragment added in newsfragments/. (no user-facing change)
(See documentation for details)

8 EncodingWarnings left with this, which I think is good enough for this PR https://github.com/pypa/setuptools/actions/runs/8148548361/job/22271588677?pr=4255

Yeah, a bit of a shame...
It might be the case there is a open()... read() ... exec() sequence somewhere or a ConfigParser.read() somewhere in distutils that fails only if SETUPTOOLS_USE_DISTUTILS=stdlib...
Worst case scenario, I was wondering if there is a way of introducing a  pytest.mark.filterwarnings("ignore::DeprecationWarning") decorator conditionally to SETUPTOOLS_USE_DISTUTILS in the complicated function.

Up to you, looks like the last ignore line I added works. Although it could hide other issues in that module.
I also didn't originally add a news fragment for this PR, but now that we're preventing more EncodingWarning from reaching user code, that's a beneficial user-facing change that's worth mentioning.

I think I understood the root of the problem: pytest.warns seems to reset the filter.
One last attempt of being more specific...

Now, that problem with stdlib's distutils is gone, but there seems to be another weird problem with pytest-mypy and pypy 😩.

This is the change for the windows errors:
From aee4012737a1e5cef3014908d4cd427017655b79 Mon Sep 17 00:00:00 2001
From: Anderson Bravalheri<EMAIL_ADDRESS>Date: Wed, 24 Apr 2024 17:20:09 +0100
Subject: [PATCH 1/2] Use UTF-8 in Windows tests

---
 setuptools/tests/test_windows_wrappers.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/setuptools/tests/test_windows_wrappers.py b/setuptools/tests/test_windows_wrappers.py
index 3f321386f..b27268935 100644
--- a/setuptools/tests/test_windows_wrappers.py
+++ b/setuptools/tests/test_windows_wrappers.py
@@ -110,7 +110,11 @@ class TestCLI(WrapperTester):
             'arg5 a\\\\b',
         ]
         proc = subprocess.Popen(
-            cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, text=True
+            cmd,
+            stdout=subprocess.PIPE,
+            stdin=subprocess.PIPE,
+            text=True,
+            encoding="utf-8",
         )
         stdout, stderr = proc.communicate('hello\nworld\n')
         actual = stdout.replace('\r\n', '\n')
@@ -143,7 +147,11 @@ class TestCLI(WrapperTester):
             'arg5 a\\\\b',
         ]
         proc = subprocess.Popen(
-            cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, text=True
+            cmd,
+            stdout=subprocess.PIPE,
+            stdin=subprocess.PIPE,
+            text=True,
+            encoding="utf-8",
         )
         stdout, stderr = proc.communicate('hello\nworld\n')
         actual = stdout.replace('\r\n', '\n')
@@ -191,6 +199,7 @@ class TestCLI(WrapperTester):
             stdin=subprocess.PIPE,
             stderr=subprocess.STDOUT,
             text=True,
+            encoding="utf-8",
         )
         stdout, stderr = proc.communicate()
         actual = stdout.replace('\r\n', '\n')
@@ -240,6 +249,7 @@ class TestGUI(WrapperTester):
             stdin=subprocess.PIPE,
             stderr=subprocess.STDOUT,
             text=True,
+            encoding="utf-8",
         )
         stdout, stderr = proc.communicate()
         assert not stdout
--
2.43.2

I am fixing the macos CI error in the upstram/main then rebasing this then merging.
Thank you vrey much @Avasam.


SMAC tuner with multiple trial concurrency

Describe the issue:
Hello!
I have been using nni for my project for a month and now I am trying to use multiple remote machines( 8 nodes with 8GPU) with trial concurrency set 8.
From what I've understood, SMAC tuner uses previous results to make surrogate model for hyperparameters. However, since there are 8 experiments running simultatneously on different nodes, I'm not so sure whether this surrogate model constructed by results obtained from all 8 nodes.
I think it either works as
a) The sinlge and only one SMAC uses hyperparameter and metric from finished trials of all 8 nodes to build surrogate model.
or
b) 8 SMAC tuners are constructed and each one uses hyperparameter and metric from 8 nodes, respectively.
Could you let me know how it is worked(a or b or in some different way)?
Thanks for your time!
Environment:

NNI version: 2.7
Training service (local|remote|pai|aml|etc): remote
Client OS: Linux
Server OS (for remote mode only): Linux
Python version: python 3.8.5
PyTorch/TensorFlow version: pytorch 1.10
Is conda/virtualenv/venv used?: Yes
Is running in Docker?: NO

Configuration:
experimentName: myexperiment
searchSpace:
my search space
trialCommand: my command
trialCodeDirectory:
trialGpuNumber: 8
trialConcurrency: 8
maxTrialNumber: myTrial
maxExperimentDuration: myduration
assessor:
name: Medianstop
classArgs:
optimize_mode: minimize
start_step : my_start
tuner:
name: SMAC
classArgs:
optimize_mode: minimize
trainingService:
platform: remote
machinelist:
- host: myhost1
- host: myhost2
- host: myhost3
- host: myhost4
- host: myhost5
- host: myhost6
- host :myhost7
- host: myhost8
Log message:

nnimanager.log:
dispatcher.log:
nnictl stdout and stderr:

How to reproduce it?:

hi @choant , it is option a)

@choant - I'm closing the issue as the question has been answered, feel free to reopen the issue if you need more discussions or clarifications. Thanks!


Update address block boundaries to match spec

Relates to email thread: "Alignment Issue - Hackney"
DVLA insert a long string - the "MDI" - just above the recipient
address, so it's important we reject letters with content that's
straying above this area. The example DVLA gave has quite a lot of
overlap - the entire first line of the address.
Two reasons for the overlap:

The boundary box we're using is out-of-date. This updates the box
to match the current spec 1.

We allow 1mm extra all around the address box 2, apparently to
avoid pixel-level validation issues.

I think it's the combination of these that's causing the problem,
but I don't want to open a can of worms by looking at the second
one. Hopefully correcting the raw bounding box will be enough to
start rejecting letters we're currently accepting.

I’m a bit worried that this is a breaking change for users of our API. In our specification we tell people that the top and bottom edges of the address area are 39.5mm and 66.3mm respectively:

Because of the 1mm buffer the this PR changes the boundary we check from 38.5mm to 39mm. So anyone falling foul of our check would still be falling foul of our published specification.
On the other hand, someone may have an integration that is working now and not overlapping the MDI. If we break the integration and they don’t have an easy way to make changes on their end then we could be disrupting important letters.

Update: we shouldn’t need to do this because we rewrite the address to put it in the proper position.
If DVLA are seeing an address in the wrong place it’s because it’s failed to be rewritten somehow.
See also https://github.com/alphagov/notifications-template-preview/pull/357

Thanks for your thoughts @quis ⭐.
Just a quick reply to say I've checked this locally to confirm it's not due to https://github.com/alphagov/notifications-template-preview/pull/603, which was deployed just before this letter was submitted. The issue predates the upgrade.

I've had a look into this and it turns out we don't always rewrite the address block. Specifically, when we get a RedactionException, which bubbles up into a redaction_failed_message in sanitise_file_contents:
        return {
            "recipient_address": recipient_address,
            "page_count": page_count,
            "message": None,
            "invalid_pages": None,
            "redaction_failed_message": redaction_failed_message,
            "file": base64.b64encode(file_data.read()).decode('utf-8')
        }

What happens to this? It gets received in two places:

In Admin, which ignores it entirely.
In Template Preview Celery, which still lets the letter through.

I'm struggling to see why we do this from the code history. I recall talking to @klssmith about whether we actually use the copies we make in (2), but not about why we let them through. @leohemsted do you know?
@quis I agree this PR is redundant, it's ☝ that's the problem. That's nice we don't need to worry about breaking things for users. I think it's worth adding a comment around those constants to indicate they're special: unlike the other constants, they don't need to conform to DVLA's spec because we rewrite that part of the page.
I've made a separate PR for that, so we can continue discussing ☝ here: https://github.com/alphagov/notifications-template-preview/pull/620

What we do is select all text in the address area.
We then delete the original text.
Then we slap a white box over the top just in case there's any non-text artifacts remaining, and re-write that original text in arial font size 8 aligned properly etc etc.
However, there a variety of ways this might fail. This happens regularly but infrequently (554 times in the last week). Mostly it fails because deleting the original text is really hard and we don't want to accidentally alter content elsewhere in the file, so we err on the side of caution.
Rather than rejecting the letter with a 400 error, we decided to just send through the letter without touching the address area as a lesser-of-two-evils decision. We made this call, partly since this was an additional sanitisation we added on top of our PDF letter flow when it was already live. (We didn't want to start failing letters that we previously accepted). It's really hard to explain this stuff to services - especially when we're often not sure why these letters are failing. These aren't services with deep and intimate knowledge of postscript and how to craft a letter - a lot of them are just letters that their existing case working system spits out that they have no control over the creation of, they just upload it to Notify.
We store the original file in case we want to debug why a given letter failed to redact. I suspect we looked at this a lot more when we were actively and frequently working on the redaction code.

I'm open to us doing something clever like:
do_margin_validation()
try:
  rewrite_address_block()
except RedactionException:
  # make sure that the MDI specifically won't be overwritten
  do_margin_validation_with_accurate_address_boundary()

That way, most letters will continue as normal. If they're slightly outside the boundaries of the address box, normally it doesn't matter as we fix the address box ourselves. However, if we can't fix the address box we can try again just to make sure it won't cover the MDI.

Thanks for the context @leohemsted ⭐.
For reference, I found these two commit messages helpful:

Transition to using a redactor.
How we got to the current state

That last commit also hints at this being the least worst option, like you said. I'm minded to write some documentation for this, as it's the overall process is complex and subtle.
However, I'm also wondering if we can just do a better job of redacting - ideally without the (somewhat) vendored code we have now. I had a go at this in https://github.com/alphagov/notifications-template-preview/pull/621 and the results look promising.
Some options:

Treat #621 as documentation and link to it in the README. Leave it at that for now. I'd like to clarify to DVLA that we won't actually be making any changes to this.

Write a card to look at #621 properly and, if it actually looks viable, turn it into a proper PR and carefully try deploying it.

Contact the Hackney service to see if they can provide a sample PDF that also fails redaction. We could use it for an xfail test to document the current state of the world.

I think I'll just do (3) as it's a quick win. What do you think we should do next?

Closing this as I think it's served it's purpose:

I'm in contact with the Hackney service and will look at a separate PR to add a test around the issue here.
We can use the draft PR above for further discussion about this - it's at least worth a bug card.


Kubernetes deployment script for sandbox-webhook

Description
Current deployment scripts of the sandbox webhook support deployment of the sandbox webhook  on docker, but cannot be used to deploy the services on Kubernetes.
Goals

[ ] Creation of Kubernetes deployment script for sandbox webhook.

Expected Outcome

Working deployment scripts for all the services and infra required for the sandbox webhook .
The services that need autoscaling should have horizontal pod scalers.
Services which needs to be exposed should have service deployment.
It must have a config file that will contain all configuration properties.
Ingres should be defined for accessing from outer world.

Acceptance Criteria

[ ] The script should be working properly on kubernetes.
[ ] The script should be tested on AWS.
[ ] The script should work with infra containers and managed services also.

Implementation Details
NA
Mockups / Wireframes
NA

Tech Skills Needed:
Docker
Kubernetes
Complexity
High
Category
Deployment
Sub Category
Deployment

We are looking at merging webhook to sandbox BPP, hence there is no fix required.  This ticket can be closed


Dotted style that I forgot about

whoops i totally missed this, this is the first time i've seen this PR

:) Don't worry. I'll make another PR soon, when options page is ready, otherwise always some folks would be unhappy.


Statement page fix

This PR updates the order for the sources so that the source badges in the statements view have the databases first/to the left and readers second/to the right.
The sources shown at the top are also dependent on which sources are actually present in the loaded statements.

The view after the PR:

Fixed:


Give more detail on what repair will do

When you click Repair you are presented with a warning... "Repairing visual studio will reset the environment...". It would be much more useful to explain the implications of that process in detail, rather than simply telling me where to find the button.
For example, will I need to repair other installs (like component packages, resharper, code rush, etc).  What files might it delete or reset? Is it worth backing up any of those?
Generally explain what impact it might actually have on me.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5fc5c36c-d8ef-6b2d-2c62-5243d1ac2ce1
Version Independent ID: 91c57e20-c75c-3c8d-e31f-727fa1e2af29
Content: Repair Visual Studio 2017 - Visual Studio
Content Source: docs/install/repair-visual-studio.md
Product: visual-studio-dev15
GitHub Login: @PoulChapman
Microsoft Alias: tglee

@apawsey Apologies that this GitHub Issue has taken us some time to respond to, but please be advised that this request, while currently placed in our backlog, is in our queue.

@tydunkel FYI, for consideration.

@apawsey can you please file the issue in our developer community since this is product feedback? This will allow the product team to track upvotes and copy to the backlog for updating.
In general, a repair will reset user settings of Visual Studio like window layouts. It will also clear per-user extensions added to that instance of Visual Studio.

Well no... I disagree.
The whole point here is that I went to the documentation to find out exactly what the repair process does, and the documentation is lacking. How is that product feedback? I don't want to change what repair does, just for you to explain it in the documentation better.

@apawsey sorry for the confusion. Are you requesting that the docs online be updated with more information on repair, or that the popup within the Visual Studio Installer be updated with more information?

Yes sorry, I wasn't being clear. My fault. See my edit above.

@apawsey thanks for the clarification! @TerryGLee FYI, doc update to clarify the implications of a repair operation.

@apawsey We've added a note to Step 2 on the Repair Visual Studio page that describes what Repair does; we hope that that additional information answers your inquiry. As such, we'll close the issue. However, you can still comment on the issue, and we can reopen it if need be.


terraform: 0.11.13 -> 0.11.14 & 0.12.0-rc1 -> 0.12.0

Motivation for this change

https://github.com/hashicorp/terraform/releases/tag/v0.12.0

https://github.com/hashicorp/terraform/releases/tag/v0.11.14

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

The builds on x86_64-darwin look to be failing due to some resource exhaustion instead of a broken build. It builds successfully on my Macbook so this should be safe to merge.


Fix alert and flash system.

Currently working on flash notice & alert system.
-- Flashes text, closes on x, and aligned right.
-- Missing colour and not properly inputting the needed bootstrap class.

Fixed.


Solana Python Transfer Help

Hey seveightech,
This isnt an issue but more of need in support of the solana transfer token code. I need some help preforming it on Raydium. Please do reach out to me on discord, my discord is @grinchify

Hello there @grinchify
I take it you want to be able to transfer a Solana-based token to Raydium and get the equivalent in another Solana-based token.
It has been a while since I touched my Solana-based project so I won't be of much help as things stand.
What I remember though is that you can only send directly to accounts using this code implementation. Raydium has made it such that, on their platform, upon receiving tokens they send the corresponding pair automatically as well. I'm not sure if the same would apply if you send tokens directly to the Raydium wallet through code, but you can give it a test,
Try sending some Solana or Solana-based token to the Raydium wallet dedicated to it and see if you automatically get the corresponding token back. If you do, then you can just proceed as such, if not, perhaps you can look into getting access to the Raydium API (I think there should be one) to facilitate this.


Techdocs mermaid plugin support not working

I have followed all the steps mentioned here https://github.com/backstage/backstage/pull/8804/files and https://backstage.io/docs/features/techdocs/how-to-guides#how-to-add-mermaid-support-in-techdocs

But during the generation of docs, it fails
ERROR    -  Config value: 'plugins'. Error: The "kroki" plugin is not installed

I added the plugin, but forgot to build the image. It is working as expected now. I will close this


subpatches throw warnings for commented-out lines in shaders

when I open the 'Manage Op' page for a subpatch containing a CustomShader that has the word 'uniform' in comments in the fragment shader, I get a warning:

the warning disappears when i remove these comments:

thanks fixed on dev.cables.gl shader warnings will only be shown for shader attachments.


Update CSV merger to inject Volume Replication Operator CRDs

Volume Replication Operator CRDs are not owned by default.
Since it is the part of ocs-operator bundle, we manually
create a mock CSV for volume replication operator and allow
ocs-operator to assume ownership of it's resources.

/hold waiting for @jarrpa

/cherry-pick release-4.10

These resources will be owned by Rook. No change required.


"mouse_entered" signal inconsistently affected by MOUSE_FILTER_PASS and MOUSE_FILTER_IGNORE

It seems that mouse_entered events are treated inconsistently by the mouse_filter setting. They don't honor the MOUSE_FILTER_PASS value, but do honor MOUSE_FILTER_IGNORE.
Given a scene made of two Button controls in a simple parent child relationship:
parent
  +-- child

Attach the following script to both controls:
extends Button

func _ready():
    mouse_filter = MOUSE_FILTER_PASS
    connect("pressed", self, "_on_pressed")
    connect("mouse_entered", self, "_on_mouse_entered")

func _on_mouse_entered():
    printt("entered", name)

func _on_pressed():
    printt("pressed", name)

My hope/expectation was that both the parent and child controls would receive the mouse_entered signal, since the mouse_filter is set to pass events up the hierarchy.  The pressed signals are there just to confirm the mouse_filter settings work in other cases.
It occurred to me that the documentation for mouse_filter refers to click events, so perhaps it is not supposed to affect mouse_entered behavior. However, if I change the code above to:
mouse_filter = MOUSE_FILTER_IGNORE

even the mouse_entered signals are ignored. So at a minimum it seems the effect of mouse_filter on the  mouse_entered signal is inconsistent.

I am seeing a similar issue. Here is a small example project that shows that mouse filter pass  on a child causes the mouse_exited signal to be emitted from the parent when the child is hovered.
Example.zip

It actually seems like MOUSE_FILTER_PASS is not implemented at all - several Controls set it, but it has no effect whatsoever in scene/main/viewport.cpp.

3.1.1 stable
Control node  (mouse -> pass) blocks InputEventMouseMotion to go to _unhandled_input
+
It seems that Control node blocks handled input on HTML5 builds as the _gui_input goes before _input.

Can confirm, PASS still doesn't let the events propagate - at the moment it does the same thing as STOP.


Cannot use behind a proxy

It will be good if proxy setting is supported.

https://github.com/koichik/node-tunnel

@hackjutsu
I think protocol socks5 is more commonly used in proxy, so I choose library proxy-agent instead.

https://github.com/hackjutsu/Lepton/pull/87


Implement perf tests

It looks like we still don't know the right way to measure perfomance of our cipher.
@kolobabka got an idea to use Spike to do this. Anyway, I'd leave this issue here and close after everything is done.

Closed with https://github.com/Exactlywb/grasshopper_riscv/commit/db5367e823baccbe52bbea1b6eefe0610dd4b468


Version v0.1.12

Comments on pull requests will not trigger Registrator, as it is disabled. Please try commenting on a commit or issue.


TextPainter - forced line-wrapping with bidi in text_painter_rtl_test failing

We are skipping these tests right now.

Hi @yjbanov
This seems like a duplicate of #13658, one of these should be closed.
Thank you


Schema with atoms as default value fails to start an OTP release app

To reproduce the bug you can do the following:
mix phoenix.new bug --no-ecto --no-brunch --no-html
cd bug
# add distillery and conform to the mix.exs
mix deps.get
mix conform.new
mix release.init
# add plugin Conform.ReleasePlugin to rel/config.exs
MIX_ENV=prod mix release --env=prod
_build/prod/rel/bug/bin/bug foreground

[info] Application bug exited: Bug.start(:normal, []) returned an error: shutdown: failed to start child: Bug.Endpoint
    ** (EXIT) bad start spec: invalid mfa: {{:__aliases__, [line: 102], [:Phoenix, :PubSub, :PG2]}, :start_link, [Bug.PubSub, [fastlane: Phoenix.Channel.Server, pool_size: 1, adapter: {:__aliases__, [line: 102], [:Phoenix, :PubSub, :PG2]}, name: {:__aliases__, [line: 94], [:Bug, :PubSub]}]]}
{"Kernel pid terminated",application_controller,"{application_start_failure,bug,{{shutdown,{failed_to_start_child,'Elixir.Bug.Endpoint',{start_spec,{invalid_mfa,{{'__aliases__',[{line,102}],['Phoenix','PubSub','PG2']},start_link,['Elixir.Bug.PubSub',[{fastlane,'Elixir.Phoenix.Channel.Server'},{pool_size,1},{adapter,{'__aliases__',[{line,102}],['Phoenix','PubSub','PG2']}},{name,{'__aliases__',[{line,94}],['Bug','PubSub']}}]]}}}}},{'Elixir.Bug',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,bug,{{shutdown,{failed_to_start_child,'Elixir.Bug.Endpoint',{start_spec,{invalid_mfa,{{'__aliases__',[{line,102}],['Phoenix',

Crash dump is being written to: erl_crash.dump...done

Shutting down..
Node is not running!

Commenting out the sections in the schema which have modules as default values mitigates the issue:
   # "bug.Elixir.Bug.Endpoint.pubsub.name": [
   #   commented: false,
   #   datatype: :atom,
   #   default: Bug.PubSub,
   #   doc: "Provide documentation for bug.Elixir.Bug.Endpoint.pubsub.name here.",
   #   hidden: false,
   #   to: "bug.Elixir.Bug.Endpoint.pubsub.name"
   # ],
   # "bug.Elixir.Bug.Endpoint.pubsub.adapter": [
   #   commented: false,
   #   datatype: :atom,
   #   default: Phoenix.PubSub.PG2,
   #   doc: "Provide documentation for bug.Elixir.Bug.Endpoint.pubsub.adapter here.",
   #   hidden: false,
   #   to: "bug.Elixir.Bug.Endpoint.pubsub.adapter"
   # ],

Just hit this as well. I've found that a significant amount of configuration properties don't seem to be handled properly with conform. Thankfully most of them don't actually need to be run-time configurable.
Are we doing something wrong?

Conform can only do so much to generate the config files for you - the idea was to give you a hefty head start, but that some manual intervention would always be required to get things to work the way you want them to.
Conform is intended to have some thought put in up front to build a schema that takes a simple, easy to use form (the .conf) and convert it to the far more complex datastructure required by the application. For simple types, including lists of simple types, as well as keyword lists, you can just use the auto-generated schema, but for things of a more complex nature, you will want to spend time defining types, transforms, etc., so that you take a simple expression, say "1 day", and convert that to some value like a Duration struct, or maybe a tuple like "{1, :day}". It is impossible for Conform to be able to do this for you for custom types like that, partly because rich datatypes can't actually be encoded in the simple language the .conf offers.
Over time I will be adding more warnings and informational messages to try and prod people in the right direction with this, so that this doesn't catch people off guard. Unfortunately it hasn't gotten as much love as exrm/distillery by me, because I use it the way I designed it, but didn't spend a great deal of time trying to make sure it worked magic out of the box. I am however hoping to get the UX of this project improved to the point where it's less painful to get started :)
I'm going to take a look at this example though, hopefully tonight or in the morning, and see if this is just something that should be working, but isn't, or just something that needs better information produced for the consumer.

Looks like this is working as of 2.3.3, which I'm about to push.


[feature] Support offline attestation verification: .sigstore file or persisted SET

This is a tracking issue and discussion for whether we should move to support the proposed Sigstore's attestation blob format when it is implemented
See https://docs.google.com/document/d/1gucjOA_bGyRjK6TeaOI-X5GIUv8WsPzeMDMkq25Kv4Y/edit#heading=h.we5fqok7jai5
https://github.com/sigstore/cosign/issues/2131
Describe the solution you'd like
This would require outputting .sigstore files instead of .intoto.jsonl attestation outputs for the blob builders and generators.
These .sigstore files would contain information like signing cert or pk and rekor SET for offline verification.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Some updates from the official sigstore bundle PR: the bundle would be a JSON that has the following format, with a sigstore bundle intoto type:
{
  "mediaType": "application/vnd.dev.sigstore.bundle.v1+json",
  "rekorEntry": {
    "logIndex": "123",
    "logId": "logid-123",
    "kind": "intoto",
    "version": "v0.0.1",
    "signedEntryTimestamp": "SET",
    "integratedTime": "1661340393"
  },
  "x509Cert": {
    "certificate": "PEM Encoded string",
    "chain": "PEM Encoded string"
  },
  "attestationDsse": {
    "payload": "eyJrZXkiOiJ2YWx1ZSJ9",
    "payloadType": "application/vnd.in-toto+json",
    "signatures": [
      {
        "sig": "aGVq"
      }
    ]
  }
}

The attestationDSSE would contain the intoto JSON file. So it a wrapper around the DSSE envelope, with the predicate still held inside an intoto attestation.
For multiple attestations, this bundle file would be concatenated as JSON lines.
The current bundle format is much more simple, but is more prone to error:
{
  "base64Signature": "MEQCIFWOIfIg0KvKk0iA7OZWrpRdQ/bb4gfk9+K6ZGcscsqpAiBKL0Z/sahv3EEadTXTVo1i+1OVC0BvLwsB7TPNQ1z4Nw==",
  "cert": "...",
  "rekorBundle": {
    "SignedEntryTimestamp": "MEYCIQCkyFReBLGFzzXjETXqxR+aK2xQVhVLooaAcB/Htp7oQwIhAMkpneby0tUljOM+wcYMDrlDbzFUFZNWzX90XKyS9B9N",
    "Payload": {
      "body": "...",
      "integratedTime":<PHONE_NUMBER>,
      "logIndex": 3433946,
      "logID": "c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d"
    }
  }
}

where the body is actually the rekor representation, so not the actual DSSE payload. So this format would require distributing both the .intoto.jsonl and this.

Filed https://github.com/slsa-framework/slsa-github-generator/issues/3750, will mark this as a dup. The bundle format should take precedence over individual files.


High code replication in (DataCumulant|ALS)(Sketch)? and loading functions of TextProcessor/LDAExample

Here's the output of a run of PMD's CPD on the project. It indicates that DataCumulant / DataCumulantSketch and ALS/ALSSketch should probably share some code through inheritance, and that the loading functions of TextProcessor, LDAExample, SpectralLDA should perhaps be shared through a singleton object or two.
<?xml version="1.0" encoding="utf-8"?>
<pmd-cpd>
   <duplication lines="188" tokens="1402">
      <file line="46"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/datamoments/DataCumulant.scala"/>
      <file line="48"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/datamoments/DataCumulantSketch.scala"/>
      <codefragment><![CDATA[    new DataCumulant(thirdOrderMoments, unwhiteningMatrix)
  }

  private def computeThirdOrderMoments(sc: SparkContext,
                                       alpha0: Double,
                                       dimVocab: Int, dimK: Int,
                                       numDocs: Long,
                                       firstOrderMoments: DenseVector[Double],
                                       documents: RDD[(Long, Double, SparseVector[Double])],
                                       tolerance: Double)
  : (DenseMatrix[Double], DenseMatrix[Double]) = {
    println("Start calculating second order moments...")
    val (eigenVectors: DenseMatrix[Double], eigenValues: DenseVector[Double]) = whiten(sc, alpha0,
      dimVocab, dimK, numDocs, firstOrderMoments, documents)
    println("Finished calculating second order moments and whitening matrix.")

    println("Start whitening data with dimensionality reduction...")
    val whitenedData: RDD[(DenseVector[Double], Double)] = documents.map {
      case (_, length, vec) => (project(dimVocab, dimK, alpha0, eigenValues, eigenVectors, vec)(tolerance), length)
    }
    val firstOrderMoments_whitened: DenseVector[Double] = whitenedData
      .map(x => x._1 / x._2)
      .reduce((a, b) => a :+ b)
      .map(x => x / numDocs.toDouble)
    println("Finished whitening data.")

    println("Start calculating third order moments...")
    val Ta: DenseMatrix[Double] = whitenedData map {
      case (vec, len) => update_thirdOrderMoments(
        dimK, alpha0,
        firstOrderMoments_whitened,
        vec, len)
    } reduce(_ + _)

    val alpha0sq: Double = alpha0 * alpha0
    val Ta_shift = DenseMatrix.zeros[Double](dimK, dimK * dimK)
    for (id_i <- 0 until dimK optimized) {
      for (id_j <- 0 until dimK optimized) {
        for (id_l <- 0 until dimK optimized) {
          Ta_shift(id_i, id_j * dimK + id_l) += (alpha0sq * firstOrderMoments_whitened(id_i)
              * firstOrderMoments_whitened(id_j) * firstOrderMoments_whitened(id_l))
        }
      }
    }
    println("Finished calculating third order moments.")
    val unwhiteningMatrix: breeze.linalg.DenseMatrix[Double] = eigenVectors * breeze.linalg.diag(eigenValues.map(x => scala.math.sqrt(x)))
    (Ta / numDocs.toDouble - Ta_shift, unwhiteningMatrix)
  }

  private def whiten(sc: SparkContext,
                     alpha0: Double,
                     vocabSize: Int, dimK: Int,
                     numDocs: Long,
                     firstOrderMoments: DenseVector[Double],
                     documents: RDD[(Long, Double, SparseVector[Double])])
  : (DenseMatrix[Double], DenseVector[Double]) = {
    val para_main: Double = (alpha0 + 1.0) / numDocs.toDouble
    val para_shift: Double = alpha0

    val SEED_random: Long = System.currentTimeMillis
    val gaussianRandomMatrix: DenseMatrix[Double] = AlgebraUtil.gaussian(vocabSize, dimK * 2, SEED_random)
    val gaussianRandomMatrix_broadcasted: Broadcast[breeze.linalg.DenseMatrix[Double]] = sc.broadcast(gaussianRandomMatrix)
    val firstOrderMoments_broadcasted: Broadcast[breeze.linalg.DenseVector[Double]] = sc.broadcast(firstOrderMoments.toDenseVector)

    val M2_a_S: DenseMatrix[Double] = documents map {
      this_document => accumulate_M_mul_S(
        vocabSize, dimK * 2,
        alpha0,
        firstOrderMoments_broadcasted.value,
        gaussianRandomMatrix_broadcasted.value,
        this_document._3, this_document._2)
    } reduce(_ + _)

    M2_a_S :*= para_main
    val shiftedMatrix: breeze.linalg.DenseMatrix[Double] = firstOrderMoments * (firstOrderMoments.t * gaussianRandomMatrix)
    M2_a_S -= shiftedMatrix :* para_shift

    val Q = AlgebraUtil.orthogonalizeMatCols(M2_a_S)

    val M2_a_Q: DenseMatrix[Double] = documents map {
      this_document => accumulate_M_mul_S(
        vocabSize,
        dimK * 2, alpha0,
        firstOrderMoments_broadcasted.value,
        Q,
        this_document._3, this_document._2)
    } reduce(_ + _)
    M2_a_Q :*= para_main
    val shiftedMatrix2: breeze.linalg.DenseMatrix[Double] = firstOrderMoments * (firstOrderMoments.t * Q)
    M2_a_Q -= shiftedMatrix2 :* para_shift

    // Note: eigenvectors * Diag(eigenvalues) = M2_a_Q
    val svd.SVD(u: breeze.linalg.DenseMatrix[Double], s: breeze.linalg.DenseVector[Double], v: breeze.linalg.DenseMatrix[Double]) = svd(M2_a_Q.t * M2_a_Q)
    val eigenVectors: DenseMatrix[Double] = (M2_a_Q * u) * breeze.linalg.diag(s.map(entry => 1.0 / math.sqrt(entry)))
    val eigenValues: DenseVector[Double] = s.map(entry => math.sqrt(entry))
    (eigenVectors(::, 0 until dimK), eigenValues(0 until dimK))
  }

  private def update_firstOrderMoments(dim: Int, Wc: breeze.linalg.DenseVector[Double], len: Double) = {
    val M1: DenseVector[Double] = Wc / len
    M1
  }

  private def update_thirdOrderMoments(dimK: Int, alpha0: Double, m1: DenseVector[Double], Wc: DenseVector[Double], len: Double): DenseMatrix[Double] = {
    val len_calibrated: Double = math.max(len, 3.0)

    val scale3fac: Double = (alpha0 + 1.0) * (alpha0 + 2.0) / (2.0 * len_calibrated * (len_calibrated - 1.0) * (len_calibrated - 2.0))
    val scale2fac: Double = alpha0 * (alpha0 + 1.0) / (2.0 * len_calibrated * (len_calibrated - 1.0))
    val Ta = breeze.linalg.DenseMatrix.zeros[Double](dimK, dimK * dimK)

    import scalaxy.loops._
    import scala.language.postfixOps
    for (i <- 0 until dimK optimized) {
      for (j <- 0 until dimK optimized) {
        for (l <- 0 until dimK optimized) {
          Ta(i, dimK * j + l) += scale3fac * Wc(i) * Wc(j) * Wc(l)

          Ta(i, dimK * j + l) -= scale2fac * Wc(i) * Wc(j) * m1(l)
          Ta(i, dimK * j + l) -= scale2fac * Wc(i) * m1(j) * Wc(l)
          Ta(i, dimK * j + l) -= scale2fac * m1(i) * Wc(j) * Wc(l)
        }
        Ta(i, dimK * i + j) -= scale3fac * Wc(i) * Wc(j)
        Ta(i, dimK * j + i) -= scale3fac * Wc(i) * Wc(j)
        Ta(i, dimK * j + j) -= scale3fac * Wc(i) * Wc(j)

        Ta(i, dimK * i + j) += scale2fac * Wc(i) * m1(j)
        Ta(i, dimK * j + i) += scale2fac * Wc(i) * m1(j)
        Ta(i, dimK * j + j) += scale2fac * m1(i) * Wc(j)
      }
      Ta(i, dimK * i + i) += 2.0 * scale3fac * Wc(i)
    }
    Ta
  }

  private def accumulate_M_mul_S(dimVocab: Int, dimK: Int, alpha0: Double,
                         m1: breeze.linalg.DenseVector[Double], S: breeze.linalg.DenseMatrix[Double], Wc: breeze.linalg.SparseVector[Double], len: Double) = {
    assert(dimVocab == Wc.length)
    assert(dimVocab == m1.length)
    assert(dimVocab == S.rows)
    assert(dimK == S.cols)
    val len_calibrated: Double = math.max(len, 3.0)

    val M2_a = breeze.linalg.DenseMatrix.zeros[Double](dimVocab, dimK)

    val norm_length: Double = 1.0 / (len_calibrated * (len_calibrated - 1.0))
    val data_mul_S: DenseVector[Double] = breeze.linalg.DenseVector.zeros[Double](dimK)

    var offset = 0
    while (offset < Wc.activeSize) {
      val token: Int = Wc.indexAt(offset)
      val count: Double = Wc.valueAt(offset)
      data_mul_S += S(token, ::).t.map(x => x * count)
      offset += 1
    }

    offset = 0
    while (offset < Wc.activeSize) {
      val token: Int = Wc.indexAt(offset)
      val count: Double = Wc.valueAt(offset)
      M2_a(token, ::) += (data_mul_S - S(token, ::).t).map(x => x * count * norm_length).t

      offset += 1
    }
    M2_a
  }

  private def project(dimVocab: Int, dimK: Int, alpha0: Double,
                      eigenValues: breeze.linalg.DenseVector[Double],
                      eigenVectors: breeze.linalg.DenseMatrix[Double],
                      Wc: breeze.linalg.SparseVector[Double])
                     (implicit tolerance: Double)
  : breeze.linalg.DenseVector[Double] = {
    var offset = 0
    val result = breeze.linalg.DenseVector.zeros[Double](dimK)
    while (offset < Wc.activeSize) {
      val token: Int = Wc.indexAt(offset)
      val count: Double = Wc.valueAt(offset)
      // val S_row = S(token,::)

      result += eigenVectors(token, ::).t.map(x => x * count)

      offset += 1
    }
    val whitenedData = result :/ eigenValues.map(x => math.sqrt(x) + tolerance)
    whitenedData
  }
}]]></codefragment>
   </duplication>
   <duplication lines="46" tokens="312">
      <file line="85"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/algorithm/ALSSketch.scala"/>
      <file line="90"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/algorithm/ALS.scala"/>
      <codefragment><![CDATA[  }

  private def simplexProj_Matrix(M :DenseMatrix[Double]): DenseMatrix[Double] ={
    val M_onSimplex: DenseMatrix[Double] = DenseMatrix.zeros[Double](M.rows, M.cols)
    for(i <- 0 until M.cols optimized){
      val thisColumn = M(::,i)

      val tmp1 = simplexProj(thisColumn)
      val tmp2 = simplexProj(-thisColumn)
      val err1:Double = breeze.linalg.norm(tmp1 - thisColumn)
      val err2:Double = breeze.linalg.norm(tmp2 - thisColumn)
      if(err1 > err2){
        M_onSimplex(::,i) := tmp2
      }
      else{
        M_onSimplex(::,i) := tmp1
      }
    }
    M_onSimplex
  }

  private def simplexProj(V: DenseVector[Double]): DenseVector[Double]={
    // val z:Double = 1.0
    val len: Int = V.length
    val U: DenseVector[Double] = DenseVector(V.copy.toArray.sortWith(_ > _))
    val cums: DenseVector[Double] = DenseVector(AlgebraUtil.Cumsum(U.toArray).map(x => x-1))
    val Index: DenseVector[Double] = DenseVector((1 to (len + 1)).toArray.map(x => 1.0/x.toDouble))
    val InterVec: DenseVector[Double] = cums :* Index
    val TobefindMax: DenseVector[Double] = U - InterVec
    var maxIndex : Int = 0
    // find maxIndex
    breakable{
      for (i <- 0 until len optimized){
        if (TobefindMax(len - i - 1) > 0){
          maxIndex = len - i - 1
          break()
        }
      }
    }
    val theta: Double = InterVec(maxIndex)
    val W: DenseVector[Double] = V.map(x => x - theta)
    val P_norm: DenseVector[Double] = W.map(x => if (x > 0) x else 0)
    P_norm
  }

}]]></codefragment>
   </duplication>
   <duplication lines="14" tokens="159">
      <file line="71"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/LDAExample.scala"/>
      <file line="46"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/SpectralLDA.scala"/>
      <codefragment><![CDATA[        .action((x, c) => c.copy(docConcentration = x))
      opt[Double]("topicConcentration")
        .text(s"""amount of term (word) smoothing to use (> 1.0) (-1=auto).
                  |  default: ${defaultParams.topicConcentration}""".stripMargin('|'))
        .action((x, c) => c.copy(topicConcentration = x))
      opt[Int]("vocabSize")
        .text(s"""number of distinct word types to use, chosen by frequency. (-1=all)
                 |  default: ${defaultParams.vocabSize}""".stripMargin('|'))
        .action((x, c) => c.copy(vocabSize = x))
      opt[Int]("libsvm")
        .text(s"""whether to use libsvm data or real text (0=real text, 1=libsvm data)
                 |  default:${defaultParams.libsvm}""".stripMargin('|'))
        .action((x, c) => c.copy(libsvm = x))
      opt[String]("stopwordFile")]]></codefragment>
   </duplication>
   <duplication lines="18" tokens="146">
      <file line="3"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/textprocessing/SimpleTokenizer.scala"/>
      <file line="1"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/LDAExample.scala"/>
      <codefragment><![CDATA[/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/**]]></codefragment>
   </duplication>
   <duplication lines="24" tokens="143">
      <file line="84"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/textprocessing/TextProcessor.scala"/>
      <file line="288"
            path="/home/huitseeker/tmp/SpectralLDA-TensorSpark/src/main/scala/edu/uci/eecs/spectralLDA/LDAExample.scala"/>
      <codefragment><![CDATA[        : RDD[(Long, breeze.linalg.SparseVector[Double])] = {
    val parsed = sc.textFile(path, minPartitions)
      .map(_.trim)
      .filter(line => !(line.isEmpty || line.startsWith("#")))
      .map { line =>
        val items = line.split(' ')
        val label = items.head.toDouble.toLong
        val (indices, values) = items.tail.filter(_.nonEmpty).map { item =>
          val indexAndValue = item.split(':')
          val index = indexAndValue(0).toInt - 1 // Convert 1-based indices to 0-based.
        val value = indexAndValue(1).toDouble
          (index, value)
        }.unzip

        (label, indices.toArray, values.toArray)
      }

    // Determine number of features.
    val d = if (numFeatures > 0) {
      numFeatures
    } else {
      parsed.persist(StorageLevel.MEMORY_ONLY)
      parsed.map { case (label, indices, values) =>]]></codefragment>
   </duplication>
</pmd-cpd>

Pinging @jill05 who recently worked on these and probably has fresh understanding of the code organization.

The plan is to first do the sketching-based functionality, then further refactor, polishing, add tests. We're not far away. Are you looking to use the code in any sort?

I understand this is work in progress, but the common parts revealed above do not seem to be related to sketching, IIUC. The sketching class could inherit those from a base trait, leaving you free to implement specific behavior incrementally.
You will also be free from the hassle of maintaining two versions of these basic functions. You will have the guarantee that these two versions of the same functions do not diverge without you realizing it.
I'm not trying to be stingy on code health or structure here. Simply suggesting there might be a less costly way of going forward.

Understanding your concern. Please bear with us.
As for the "least-costly way": my point of view and execution plan was to start with refactoring to a certain extent, then add in the new functionality (with replicating some code), then continue refactoring, pruning away redundant code, building a reasonable class hierarchy, adding tests, adding licenses.
The end goal was the same as you asked.
At this stage I chose to do so as this is also a piece of numerical software. There're small bits to move around to rationalise the code; and previously there was no test code -- If I first set out to rationalise the hierarchy I get entangled in small details without much advancing on a bigger plan, also with a risk to break things without being able to repair. Overall that's the approach I chose: to balance between numerical accuracy, code cleanness, speed, big picture, details, aesthetics.

It should be OK now for the current code base.


Multi-core Hash calculation

Hi,
First of all let me thank you for this amazing library, which is currently powering my own project https://github.com/julianxhokaxhiu/FFNx - This is a 32-bit modding platform loaded as DLL ( w/process injection ).
Through your library we were able to implement animated textures in a game which was never supposed to have them upscaled in the first place. Although as the game itself is developed around "old game logics" ( single core / single thread ), the issue we're currently facing is that when we use this library to hash some data, it can't really scale, and because of this we start to lose FPS.
But in the team, we were having some new ideas which might benefit your code as well, not only our case: Multi-core hashing.
How would this be implemented in practical ways?

Spawn a new thread that will be ready to listen for new hashes to calculate
On a new hash request the library will check which available cores we have, and pick one that is currently free
Throw the hash to that core and return back the result in sync

I know this is not really "async" per-se, but it would help in making those calculations faster as we can throw them to another core which the CPU might squeeze better, instead of parking the current core data to make room for the new calculations it has to do.
What do you think about this implementation? Does it make sense? How complex would it be to have it implemented on the core library code?
Thank you in advance,
Julian

Found a better alternative that fits better my project. Would be pointless to ask you to implement it, nevertheless, you might want to check https://github.com/Cyan4973/xxHash as it's really MUCH faster at least for us. You might get some insipiration from it, not sure.
Still thank you :) Cheers!


Automatically make highest tag the latest release

Maybe throw build notes in release?
Edit: Note that everything is tagged automatically, this is purely to get that nice github release UI

Thanks for this repo, it's just what's needed :-)
I'm converting my plugin's install-wp-tests.sh-based testing setup on Travis CI to use this package instead (along with wp-phpunit/wp-phpunit) and I'd like to retain the ability to lazily test against the latest version of WordPress without having to update the WordPress version number in my Travis config each time there's a new release.
Currently this is achieved by specifying the WP_VERSION=latest environment variable, which install-wp-tests.sh uses as an indication to check the w.org API to fetch the latest tag.
I'd therefore like to use a Composer dependency that looks something like roots/wordpress=latest which corresponds to a branch in this repo that always contains the latest stable release (ie. the highest tag). Is that what this issue is about? If so, a big +1 from me :-)

Actually, roots/wordpress=dev-master provides this from the look of things, but this requires minimum-stability: dev in the root, which I don't really want. Would a separate branch that tracked master should do the trick? Not sure.

More rubber ducking: roots/wordpress=* looks to be the answer.

Hey @johnbillion!
So this issue was for purely to make a github API call to map a git tag to a github release to get the nice UI. I updated the original post with details.
And yes dev-master is pinned at the latest release so people can track latest WP version for CI tests and stuff. To do this you would specify both "minimum-stability": "dev" and "prefer-stable": true
But yeah looks like roots/wordpress=* might be better anyway

@austinpray Can * results in the same behavior as latest and how to use nightly? I am using this with Github action and in a matrix, I don't want to index * instead use this latest. Overall need support for roots/wordpress="latest|nightly" so how can I do that?


#195 - Fix build artifact final name properties resolve

Following up on #195, this pull request proposes a simple naive solution to the issue

Hi @mabrcosta, I've already started analyzing this issue. It has to be done slightly differently.

this new parameter shouldn't have property attribute, changing it's default value will not work
since it's read-only, it should be hidden, no javadoc needed

    @Parameter( defaultValue = "${project.build.finalName}", readonly = true)
    private String finalName;

It should be used in two places, the second one is in projectAttachedArtifactFileName variable initialization below.
Can you implement these changes?

Hi @gslowikowski. I've implemented the changes you mentioned and updated the pull request.
Thanks for quickly analysing the issue.

Thank you @mabrcosta

@gslowikowski can we get a stable release (non SNAPSHOT) with this please ?


onActivated/onDeactivated not triggered when re-mount/unmount a v-if descendant component

Vue version
3.4.38
Link to minimal reproduction
https://play.vuejs.org/#eNqFU91u2jAUfhXPN4DUJRfdxdSFaIz1opu2Vv25s1SlziG4dezIdlIkxLv3OAkGWkiRkJLz/fj4nC9rOquqqKmBXtDEciMqRyy4ukqZEmWljSN/AaqZFA3cZ6YARxZGl2QUxe/q3mT0g6kk7mzQAF8clJXMHOAbwV8SROnusfeN02TnifwkDmJ6Rp3lWi1EET1brbDZtTdklOuyEhLMdeWEVpbRC9IiHsuk1K9/2pozNZxt63wJ/OVI/dmufI3RGwMWTAOMBsy1PXbw5d1/WOFzAEud1xLZA+AtWC1r32NH+1WrHNve47XdXrUjF6q4t5crB8puL+Ub9cxNy2cUhz0fuPqu3fPoW6tjaoNTPLKz04tfEwMLsukX3q+3x+Za5cKfm8n5Usg8pOI9sI0Fbs9idmRWkKn3HfsbTT7Ny4dzmq9iMWXUGzGKmel5T7VzWpGfXAr+0uN40JeOl969CseXSdzRDsPFFA7mWNtDk9HqQZW6Vg7yExPynN+QcScaPGWI9e9Tn9kpF6aCfDyekGnaRcAPW0uIpC7GjAYKoxOMAf69LHgOCQPpQBquPiQNpAPp3kRQPE2PSvdIQTwYk6tRSdwSCG9X93G7jw0Y/ynhRvF7iM6/080bGN+03A==
Steps to reproduce
Click 'Switch' button to unmount/remount the descendant component.
What is expected?
According to the documentation, onActivated/onDeactivated should get called when mount/unmount even on a descendant component.
What is actually happening?
onActivated triggered only on first mount, then click 'Switch' button to unmount/re-mount the component will trigger neither of onDeactivated and onActivated.
System Info
No response
Any additional comments?
No response

duplicate of https://github.com/vuejs/core/issues/10806
a workaround
<template>
    <KeepAlive>
        <ConditionalChild v-if="flag" />
    </KeepAlive>
    <button @click="flag = !flag">Switch</button>
</template>