Need a network level test for a client securely transacting with multiple peers

Description
This issue is being opened as a result of discussion here: https://github.com/hyperledger/fabric/issues/1991
The issue is that the enrollmentID and enrollmentSecret pair that is used to perform the enrollment with the membership service server is a one-time-use set of credentials. Once you have performed the enrollment one time on one peer, it will fail on subsequent enrollments. If the client must send their transaction to one of the other peer IP addresses, not the original peer IP that performed the user enrollment, that second peer will not have the appropriate user cert/tokens to transact. The current approach to transacting through multiple peers was to configure the application and peer nodes to replicate the location of the stored crypto materials among themselves.
We need a network level Behave test to verify the functionality above. @jeffgarratt please advice.
Describe How to Reproduce
If you log in through either the CLI or the REST API into one of the peer nodes, you will not be able to enroll through the same credentials on another peer. You will receive an error: "Identity or token does not match". That is expected, but the approach to replicate the crypto tokens between peers must be tested. It has been described, but not demonstrated/tested through Behave in our environment.

We should never replicate credentials across peers.  I don't think the use case is valid in real life because if multiple entities are hosting peers (which is how a blockchain should be deployed), then certainly you don't want the credentials for a user/client from one entity replicated to another entity's peer.
I also don't think that we should spend a whole lot of time at this point on the current REST implementation.  The right way to do this is going to be to go with managed / smart clients / wallets which might choose to expose the REST API.  I would have mentioned setting up the NVP as a "proxy" and adding the ability to specify the target peer, but NVPs as they exist today are not the strategic approach.
Note:  With the new NodeJS SDK people can build apps which can use the same credentials to speak to multiple peers and that is the way to handle this in the short term

@mastersingh24 thanks for this. I have also discussed with @jeffgarratt. We think you bring up a good point about the validity of the use case and more so, the fact that the current REST implementation will not be supported for much longer. I will go ahead and close for now. If that may change, we can always reopen.
Thanks.


improve admin approval flow

Admin receives an email with a new expense and hits approve and is directed to a url (https://opencollective.com/opensource/expenses/3561/approve) that only loads a word 'done'.
We should probably send them to their collective or their collective /expenses so they can see the expense approved or even a place to check out new collectives! to make this flow more meaningful.

will grab this one up

@FadySamirSadek Awesome! Let us know if you need any help setting up the api and frontend.

installing graphicsmagic manually using macports solved the issue


Only create kdb packages when kdb files are present

Due to the unguarded components line in the CMakeLists.txt file, the kdb packages are created with empty files even if the actual kdb files are not present, creating clutter and confusion for release management.
This PR makes the kdb package creation conditional on the files being present.

@junliume  Gentle ping for PR merge.


doesn't take context into consideration. gets confused easily.

Pls, help to remove this unwanted behavior, or suggest some ways to do it.
Thanks

Pls, help to remove this unwanted behavior, or suggest some ways to do it.
Thanks

@alan-ai-learner
You can try fine tuning it with custom dataset to improve the quality of responses. But don't expect it to be perfect. Can you tell your use case? So that I can suggest better option if I know?

I just want to make a chatbot, that is like, two friends were talking to each other, it not some domain based but a simple and usual conversation that will held between two friends, so that my aim.

If you have to use GPT, then best thing would be to fine-tune with super clean dataset.
If you always want clean responses, try rasa, it will for sure give you clean results.
It won't be as creative as GPT. But you can maybe integrate GPT inside rasa (I'm doing this).
For some basic intents, use your predefined responses, and for fallback_intents get GPT response. It will get heavier though.

@chiranshu14  thanks a lot for your response, btw my chatbot wasn't domain specific, I don't know any predefined intent as of now, I just need a chatbot that can do a casual conversation with the user and remember the context for a reasonable time.


WiFi credentials are not persistent after rebooting the Vision AI Dev Kit

Followed the instructions in step-3 of https://azure.github.io/Vision-AI-DevKit-Pages/docs/quick_start/ page.
WiFi Access Point credentials (SSID and Password) configured
Able to acess internet (Azure IoT) via this Access Point, everything is going well !!
Now power down the Dev Kit and power it up again
I expect the Dev Kit to automatically connect to the Access Point configured in the earlier steps. But it does not get connected !!
Only option is to press-n-hold the power button for 5 seconds to put the device to Configuration  mode and re-do the steps from 1 to 3

Why is the WiFi credentials not stored persistently across power cycles.

We are looking into it.

@anilvpr Can you try to reset your device once and verify.
You can follow this link- https://azure.github.io/Vision-AI-DevKit-Pages/docs/reset/

@anilvpr, are you on the latest firmware? Just a thought.

@v-prsasa Issue is resolved after device reset. Thank you.


Change delete choice warning

Change it to "This will delete ALL the cards below. Are you sure you want to proceed?"

Tested with commit 7af03df
Warning text when delete choice/main activity card is updated.


Solve #135

https://github.com/deuteronomy-works/pyffmpeg/issues/135
Changed shell parameter to False in subprocess.Popen to accept a path-like object
https://docs.python.org/3/library/subprocess.html#popen-constructor

Thank you very much @oladuwek

@oladuwek Please next time don't make the push to the main branch but to the develop branch


pass the coordiate to OnLong press callback

Looks good, tested, merging it :)

@all-contributors please add @AlhaythamElhassan  for code

@all-contributors please add @AlhaythamElhassan for code
sorry for extra ping, bot made a conflict 😞


test: use new member limits for basic example

Description
The minimum values required for read-only-replica for MySQL has changed, updated the basic example to reflect these new limits, which is also part of the unit test.
Release required?

[x] No release
[ ] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

The PR test does not run the TestRunBasicExample test that I'm trying to fix here, so I ran it manually and it has passed.
--- PASS: TestRunBasicExample (1349.00s)
PASS
ok  	github.com/terraform-ibm-modules/terraform-ibm-icd-mysql	1351.447s


termor adds 800kb to remix bundle

I investigated a quite impressive addition to the JS bundle by importing three tremor components.

I was surprised to see that the imported file<EMAIL_ADDRESS>is 1.2MB in size and seems to include several external libraries like react-dom.production.min.jsm
This seems unintended and explains the huge amount of bytes.

Hey thanks for sharing this, we'll investigate this further!

I got the same issue with Vite, tremor adding 1m1 MB to bundle. Contains 2 version of react-dom, which is crashing my (only) prod environment. Just spent a day in investigating to understand this 🙏

It contains another react-dom from 2.0.0

Thanks all for looking into this! Turns out react-transition-group had listed react-dom as a peerDep and we hadn't added it to our peerDeps. Should be fixed with v2.2.0

I just see a minor improvement from 800kb to 682kb.
The index.js files in dist still contain the code for classNames and react 16.
Also treeshaking fails for me using remix/esbuild.

should be fixed with v2.4.0


webpack clean without restore causes error

Problem
The webpack command presently ends with trying to start webpack (react-scripts start). If one only issues a clean -c, that causes an error 'react-scripts' is not recognized as an internal or external command because node_modules is now gone and needs restored.
Solution
If webpack command is issued with a clean, but not an accompanying restore, it should exit.

Published with v2.3.1


Expose camera authorization

In response to issue #134, I added the checkDeviceAuthorizationStatus method to the Camera reference API. Since it's not a value list, I opted to not add it to the constants var.

Thanks Brad, looks good.

Thanks!


Add copy+paste message for not sharing

We now have a specialized message for when you are no longer sharing
access instead of just telling the user they are no longer getting
access.
This will not work across restarts, that's fine.
Tested manually
Fixes #1592

:+1:


Upgrade to Python 3.11

This allows the use of NotRequired in TypedDict: https://docs.python.org/3/whatsnew/3.11.html#pep-655-marking-individual-typeddict-items-as-required-or-not-required
It is 10-60% faster
TaskGroup can be used instead of asyncio.gather: https://github.com/DigitalSlideArchive/ImageDePHI/blob/a937e55b454c0b0b01e7a90e8bb3e051f2949739/imagedephi/main.py#L64
Consider replacing asyncio.wait_for with asyncio.timeout(): https://github.com/DigitalSlideArchive/ImageDePHI/blob/a937e55b454c0b0b01e7a90e8bb3e051f2949739/tests/test_e2e.py#L52

@manthey says:

I don't think requiring Python 3.11 is a good idea yet. Some of our tile sources in large_image don't yet have wheels published for 3.11 (e.g., some of the dependencies for nd2 files and the tifffile library). And, our collaborators probably haven't upgraded to 3.11 anywhere yet, either, so it will add unnecessary complication to their use of it.

our collaborators probably haven't upgraded to 3.11 anywhere yet, either, so it will add unnecessary complication to their use of it.

Our end users should never need to have Python to use ImageDePHI. #17 and #18 should be the only supported approach to distributing to users.
Perhaps end users could try to directly run ImageDePHI using their own Python, but it will be a massive burden to us (both in terms of restricting our development freedom and in terms of debugging problems) if we support that.

Some of our tile sources in large_image don't yet have wheels published for 3.11 (e.g., some of the dependencies for nd2 files and the tifffile library)

We're not using large_image at present, and I think our approach should be to focus on deepening the feature set of the existing file support (TIFF / SVS) before trying to add other formats.
I don't expect that we'll get to supporting other formats for at least a few months, and we may decide on other alternatives to large_image to do so (indeed, 3.11 compatibility could be a factor in deciding what libraries to use).

This should allow a fix of #67.


compactor: 1.2.2 -> 1.2.3

Description of changes
change log: https://github.com/dns-stats/compactor/releases/tag/1.2.3
diff: https://github.com/dns-stats/compactor/compare/1.2.2...1.2.3

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 248267 run on x86_64-linux 1

1 package built:

compactor


Documentation should describe download destination argument

buildkite-agent artifact download <pattern> <destination> has a pretty counterintuitive algorithm for determining the destination path. Probably the documentation (same in v2) should be expanded more fully so people can use it.
In particular, the rules appear to be:

If destination ends in /, or if the last part of destination does not match the first part of path, then destination is joined with path
If the last part of destination matches the first part of path, then the last part of destination is removed before joining with path
If the the last n parts (counting from 0) of destination, reversed, match the 0th, 100th, 200th, …, 100 × n th parts of source, then those last n parts of destination are removed before joining with path

I created a test case to verify these rules:
a_test.go:
package a
import (
	"testing"
	"github.com/stretchr/testify/assert"
)
func TestGetTargetPath(t *testing.T) {
	// If the destination ends in /,
	// then no part of destination is stripped
	// before joining with path
	assert.Equal(t, "a/a", GetTargetPath("a", "a/"))
	assert.Equal(t, "c/a/a/b", GetTargetPath("a/b", "c/a/"))
	// if the last part of destination
	// does not match the first part of path,
	// then just join path to destination
	assert.Equal(t, "a", GetTargetPath("a", "."))
	assert.Equal(t, "a/b/c", GetTargetPath("a/b/c", "."))
	assert.Equal(t, "a/b/a/b", GetTargetPath("a/b", "a/b"))
	// If the last part of destination
	// matches the first part of path,
	// then remove the last part of destination
	// before joining with path
	assert.Equal(t, "a", GetTargetPath("a", "a"))
	assert.Equal(t, "c/a/b", GetTargetPath("a/b", "c/a"))
	assert.Equal(t, "lambda.zip", GetTargetPath("lambda.zip", "lambda.zip"))
	// when the path[0], path[100] matches
	// destination[-1], destination[-2]
	// then the last 2 characters of destination
	// are removed before joining path
	// This is a tricky one to use.
	assert.Equal(t,
		"a/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/b",
		GetTargetPath(
			"a/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/x/b",
			"b/a"),
		)
	// Gotcha: this is not what you want.
	assert.Equal(t, "a/lambda.zip/a/lambda.zip", GetTargetPath("a/lambda.zip", "a/lambda.zip"))
}

a.go:
package a

import (
	"fmt"
	"os"
	"path/filepath"
	"strings"
)

func GetTargetPath(path string, destination string) string {
	destinationPaths := strings.Split(destination, string(os.PathSeparator))
	downloadPaths := strings.Split(path, string(os.PathSeparator))

	for i := 0; i < len(downloadPaths); i += 100 {
		// If the last part of the destination path matches
		// this path in the download, then cut it out.
		lastIndex := len(destinationPaths) - 1

		// Break if we've gone too far.
		if lastIndex == -1 {
			break
		}

		lastPathInDestination := destinationPaths[lastIndex]
		if lastPathInDestination == downloadPaths[i] {
			destinationPaths = destinationPaths[:lastIndex]
		}
	}
	finalizedDestination := strings.Join(destinationPaths, string(os.PathSeparator))
	targetFile := filepath.Join(finalizedDestination, path)
	// targetDirectory, _ := filepath.Split(targetFile)
	return targetFile
}

func main() {
	path := os.Args[1]
	destination := os.Args[2]

	targetFile := GetTargetPath(path, destination)
	// Show a nice message that we're starting to download the file
	fmt.Printf("Downloading %s to %s\n", path, targetFile)
}

go.mod:
module a

go 1.16

require github.com/stretchr/testify v1.7.0 // indirect

To run the test, run:
go test *.go

Thank you for opening this issue, I’ve updated the artifact download CLI documentation to discuss the path matching logic and have opened a pull request to update the web docs too.


Refactor: Consistent package naming and structure

Which problem is this PR solving?
closes #408

Codecov Report

Merging #413 into master will increase coverage by 1.95%.
The diff coverage is 100%.

 I'll take care of it.


urllib2.HTTPError: HTTP Error 429: Too Many Requests

I'm using pytube with python 2.7 and got this error:
Traceback (most recent call last):
File "test.py", line 87, in
local_video = YouTube(url).streams.first().download(download_folder)
File "/usr/lib/python2.7/site-packages/pytube/main.py", line 88, in init
self.prefetch_init()
File "/usr/lib/python2.7/site-packages/pytube/main.py", line 96, in prefetch_init
self.prefetch()
File "/usr/lib/python2.7/site-packages/pytube/main.py", line 160, in prefetch
self.watch_html = request.get(url=self.watch_url)
File "/usr/lib/python2.7/site-packages/pytube/request.py", line 21, in get
response = urlopen(url)
File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 429: Too Many Requests

This is a common issue. Check out https://github.com/nficano/pytube/issues/601 for discussion on it. Setting your cookies with pytube/request.py should fix the problem.


use DeriveAnyClass to derive Enumerable where possible

All the (non TH) Enumerable instances in the examples use the same definition so this seemed like a better way to do it.

Cool! This seems more sensible.
The build failing is my fault probably - It looks like I've not configured cachix properly. :)

Redundant now that #3 is mreged


Fuzzy search for json type queries, eq and neq operators

Description
Fuzzy search will only take place for text type properties.
The contains type operator cannot make use of this since regex's are not well suited for this kind of query, and using a match query for the contains operator turned out to be far too promiscuous with the results.

Motivation and Context
The current list search functionality is overly restrictive, and does not all for slight misspellings, missing spaces, etc. (see related ticket). We need to add some fuzziness to the search fields to allow for more flexibility.
Notion

How Has This Been Tested?

Checklist

[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.

@io7m Bump

One more administrative comment:

The PR description doesn't really say much about what this PR is doing.
There's a small typo in the Motivation section: "...and does not all for..." -> "...and does not allow for..."


To Prod: Change to All Guides (Manually triggered)

Updated by github actions, triggered Manually.

close

not apply changes now
will be apply when guides need to be updated individually


Code blocks of unrecognized type should be rendered as preformatted

At the moment when a code block is fenced with an unrecognized type, the content seems to be rendered as normal text content.
Given markdowns ignoring of newlines (without two trailing spaces) this means multiple code lines are rendered in one line.
Triple-backtick code blocks should default to <pre>+<code> rendering when the code type is unrecognized.

Sorry, this was supposed to be a bug ticket rather than FR/enhancement.

It looks like the fallback is an adequate code block.
However, mermaid does not fall back, but renders as text instead.
Maybe the prism highlighting logic excludes mermaid? Or this addons parser?
This would be resolved by #20 mermaid support. Otherwise, it should render as a code block.


Update to requirejs 2.2.0

Tests fail. try updating them with the latest requirejs and test locally.


Build fails with "`.rodata' can not be used when making a shared object"

I used to build and debug my project totally fine, but after updating to newer version of vscode and cmake-tools extension, the build process will break at linking stage.
Here is the linking error:
***/external/freetype2/prebuilt/linux/64-bit/libfreetype.a(ftbase.linux64.o): relocation R_X86_64_32 against ".rodata" can not be used when making a shared object; recompile with -fPIC

No, I'm not making a shared object, so I don't need to re-compile freetype with -fPIC flag.
I tried both "make" and "Ninja" tools, both will fail.
Interesting part is, when I manually do a cmake like this:
mkdir build
cd build
cmake ../
make

It will compile and link just fine. I tried with Ninja tool, it works too. I don't know if the cmake-tools extension is adding a wrong flag to makefile that makes the linker think this is a shared-library-project...

What does your add_library() call look like?

Actually my project is Cocos2d-x game, there are lots of nested CMake files. I tried to find something useful for you but that's just too complicated.
Would it help if I include the build.ninja files? (The one that I manually created and compiles, and the one created with cmake-tool which fails)

Strangely, I got it working by commenting the
include(CMakeToolsHelpers OPTIONAL)

and adding :
include(CMakeToolsHelpers OPTIONAL)

at the top of my main CMakeLists.txt file.

Search your CMakeCache.txt for BUILD_SHARED_LIBS, both with and without using CMake Tools

@Arman92, I'm curious, what cmake_minimum_required(VERSION) do you have specified? If it's at least 2.6.0, that policy should be set.

this isn't a CMake-Tools issue,
you can fix it just adding  -fPIC option CXXFLAGS

It's a CMake Tools issue in that the behavior differs from the behavior when running CMake from the command line.

Sorry to vanish like this,
It's set to cmake_minimum_required(VERSION 3.4.1)
I didn't know if that policy is required, because my project was working fine before updating the CMake Tools extension

What about BUILD_SHARED_LIBS? Is that variable set in the cache? What is it set to when you run with and without CMake tools?

Closed.
If this issues still persists in the current 0.11.0-beta4, feel free to reopen this one or create a new one.


Would you ever consider adding a non anime movies section?

I noticed consumet indexes non-anime and korean drama content as well, such as american or european shows and movies.
Not sure if youd like to add it but i think a relevent section on the home page would be sweet. Ive been looking for an all in one, streaming site to self host, and most on github only do non anime or anime content, rarely both.

To my knowledge, Flix HQ was the sole source for movies and TV shows on Consumet. However, Flix HQ's team has altered the encryption method, and it was mentioned on the Discord server that Consumet will no longer support Flix HQ.
This does not imply that adding movies and TV shows support is impossible. Other sources exist, such as VidSrc Pro, among others, which unfortunately come with a significant number of ads. This is manageable for users with ad blockers, but for those unaware of such tools, the ads and redirects can be a source of frustration.
Anyways, I will try looking for some good sources. I am not sure whether I will find them or not so I can't promise anything at this moment.
(I will close this issue for now)

Okay!


tour: [REPLACE WITH SHORT DESCRIPTION]

Context: https://tour.golang.org/welcome/1
Change the title above to describe your issue and add your feedback here, including code if necessary

What is the issue you are having?

Can probably be closed now @adg @broady @rakyll @yangyangdeyi


Runtime error with tuberack

Hi,
I'm having a Runtime error at:   labware.load('opentrons-tuberack-15ml', '1') and I was wondering if anyone else had encountered similar errors.

Hi @Gylianne, sorry for the delay! I recommend that you:

Ensure your app is fully up to date
After you've checked the app's version, check that your robot is fully up to date
Make sure that all your robot's advanced settings are switched off (at the bottom of the robot settings page in the app)

If those initial steps don't help, please reach out to our support team via in-app chat or by going to https://support.opentrons.com. I'm going to close this issue as a support request for now, but will re-open and/or update it if we find any sort of underlying bug!


Create Project Outline

Create an outline of full project
DoD
-Outline of all MVP features presentable

Started Adobe XD outline of frontend and wireframing backend.

As a corporate procurement manager,
I want a web app that facilitates purchasing from commonly used vendors,
So that I can efficiently manage budgets, track spending, access a list of frequently purchased items, and generate and email purchase orders (POs).
Acceptance Criteria:
Budget Tracking:
The app should allow me to set budget limits for specific categories or vendors.
I should receive notifications when the budget limit for a category or vendor is close to being exceeded.
The app should provide visual representations of budget utilization.
Vendor Management:
I should be able to maintain a list of commonly used vendors with their contact information.
The app should allow me to categorize vendors for easier management.
Purchase Tracking:
Every purchase made through the app should be recorded and associated with the appropriate category and vendor.
I should be able to filter and sort purchases based on date, category, vendor, and amount.
Frequently Purchased Items:
The app should compile a list of frequently purchased items based on historical data.
This list should help me identify potential bulk purchase opportunities or negotiate better deals.
Purchase Orders (POs):
I should be able to create and customize purchase orders directly within the app.
POs should automatically include vendor information, item details, quantities, and pricing.
After creating a PO, I should have the option to review and edit it before finalizing.
Once finalized, the app should generate a printable version of the PO in PDF format.
I should be able to email the finalized PO directly from the app to the respective vendor.
User-Friendly Interface:
The app should have an intuitive and user-friendly interface to ensure easy navigation and usage.
Additional Notes:
Integrations with accounting and finance systems could enhance the accuracy of budget tracking.
The app should maintain data security and user authentication to protect sensitive corporate information.
Regular updates and improvements to the app should be part of its development roadmap.


nlohmann_json_schema_validator_vendor: 0.3.0-1 in 'iron/distribution.yaml' [bloom]

Increasing version of package(s) in repository nlohmann_json_schema_validator_vendor to 0.3.0-1:

upstream repository: https://github.com/open-rmf/nlohmann_json_schema_validator_vendor
release repository: https://github.com/ros2-gbp/nlohmann_json_schema_validator_vendor-release.git
distro file: iron/distribution.yaml
bloom version: 0.11.2
previous version for package: 0.2.4-3

nlohmann_json_schema_validator_vendor
* Switch to rst changelogs
* Contributors: Yadunund

manually updated doc and source versions to iron.


Overview on datasources

Detailed exploration of data sources: Gridmet, AMSR, MODIS, Snotel Importance and characteristics of each data source

look good, can you fix the QualityControl check?


Azure Policies and Remediation MMA

Syntax fixes -> checked
ARM API updates ->Fixed
No reference to 03.png and 04.png -> they are referenced

Hi Laura, great work. I have a few minor suggestions for revision to the readme. I also thought we might include a script that will help reduce time to complete lab by forcing policy evaluation. I have a PS script that can accomplish that (we could probably retrofit it for AZ CLI instead if we want). Would like your thoughts on whether it makes sense to include that or not.

Yes lets try the PowerShell script I did not want to mix AZ CLI and PowerShell in the mix, but let's discuss

Opening new PR


Always prefer window over this for global object detection.

Depending on how this module is consumed by applications, it might be wrapped by a module-pattern function. In this case this no longer refers to the global object, but to the module object, so the polyfill fails.
This change checks if window is present. If so, use that, since in this case, that definitely always refers to the global. To maintain compatibility (with node, or other non-browser envs), the existing this is used as a fallback.

Awesome, thanks for that! Would it be possible to npm publish this, too?


5 worker for staging

[x] check required ENV VARs
[x] single run of part the very first version of script

prefixed secrets:

not scheduled yet - on demand trigger

naive run of actual script - not great not terrible
email with confirmation delivered twice (different secretCode), only second works
some null in the database

with neverending retry (probably)
https://github.com/cesko-digital/pomahejukrajine-worker/runs/6056116883?check_suite_focus=true

github action 12 seconds even with downloading packages and build


persistent_id as string instead of int in tests of RedisResourceManager

Fixes #4

:+1

I don't understand what this changes try to fix. Because there is no previous test failure

@Maks3w It's only for clarification because of tests are often used to see how a system have to be used and at this point you could be thinking wrang that the persistent_id have to be an integer but it doesn't. On the same time the documentation was is wrong, too but in a very different way. see zendframework/zf2-documentation#1495

Then an expection is expected to be trown if value is invalid?

@Maks3w An integer isn't invalid it's simply an integer passed an function it expects a string so this will be automatically converted to a string. It will be invalid in PHP-7 only with strict_types=1

Do you think it could be merged?

There's nothing in the PR to prevent revert the change. May throw  Exception if type is invalid?

Sorry @Maks3w but I didn't understand

If previous value was wrong. Why test was working?. Should we add a check and throw exception if the value has the wrong type?
Anyway I rely on @marc-mabe to decide if this should be merged.

As @marc-mabe said in his last post, there's nothing wrong, because there is automatic conversion from int to string, but this way it's more clear that it's not an integer id, but a string.

@Maks3w the test works because it's not totally wrong. It simply passes an integer value to something that expects a string and the integer will be internally converted to a string.
It's like testing 'substr(1234,1,1)'.
But as sad before test are often used so see how to interact with a lib and with this in mind it's better to "fix" it as someone could wrongly think he needs to pass an integer value instead of a string.

Is it possible to merge so I can close the PR? Thanks

Thanks :)


Fix No GPU build

@tanmayv25 This is needed to fix the build for when TRITON_ENABLE_GPU=OFF. It was caused by your change here https://github.com/triton-inference-server/backend/pull/22

It was causing a failure for the ENABLE_GPU=OFF build here.
Some backend i.e. identity sequence and dyna_sequence need to be built with no GPU support.


Quest resource has no update method

ActiveQuestPool.update_objective calls the update method on Quest, however Quest has no update method. I suspect that Quest should have a default update method that simple sets objective_completed = true?
If this sounds correct, I can open a PR.

Oops, you're right, the method is indeed missing.
I'll merge your PR! Thanks!


RDF4J Implementation (based on Sesame impl)

This PR adds an rdf2go.impl.rdf4j artifact, which allows the use of Eclipse RDF4J (formerly known as Sesame 4.1) with RDF2Go.
Basically, I took the Sesame implementation, adapted all the dependencies, and took care of the API changes I could find (most of which were URI to IRI conversions).
Further improvements:

Uses RDF4Js Json-LD RIO implementation instead of a third party one, simplifying dependencies
The RIO packages are only test scoped now
Targets Java8 (since RDF4J uses the Optional<T> feature)

Unfortunately I needed to temporarily remove the .gitattributes file. Not sure what git's idea is about that. To me it looks as though everything was committed with CRLF in the last "Normalize line endings" commit, when it should have been LF.

Updated the PR with a version that avoids "Normalize line endings". Still, d58ab13 induces a whole-file recommit of the two relevant poms.

Thank you for your work! Give me a few days to take a look and I will merge the pull request.

Note that the first versions of Eclipse RDF4J are not technically released yet. The builds on Maven Central are all "Milestone" builds (M1/M2/etc.) that the Eclipse Foundation hasn't put through the full release procedure yet.

Thanks Peter. It will be some days until I fix #16 and get the old release pipeline back to work until I can put any of this on Maven Central myself. Hopefully, the Eclipse versions will be there then or I publish snapshots first.
As a last step I will later get rid of the rdf2go.impl.sesame artefact in favor of @wherget 's new rdf2go.impl.rdf4j which currently exists in parallel.


[Feature Request]: Leetcode Q-823

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Want to Add Solution Of Leetcode Q823 Binary Tree With Factors
Use Case
User will not have to search solution for this question somewhere else
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

@nishant0708 Please complete your task otherwise I will close it. and if you have any queries then ask me on our live chat for support: https://codeharborhub.github.io/community/supports#-how-to-get-support


[Integration] [Airflow] Expose the parent job to the Operator so that it can be passed down

In 1.10 we had lineage_run_id and lineage_parent_id macros that exposed generated job and run id for a given task to a downstream operator. Add macros to 2.3+ plugin so that we can reuse it there.


Register type definitions lazily

Pull Request Description

close #10923
Changelog:

update: registerTypeDefinitions registers constructors lazily

Important Notes

The improvement of time spent in the registerTypeDefinitions method (highlighted):
before

after

Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:

[x] The documentation has been updated, if necessary.
[x] Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
[x] All code follows the
Scala,
Java,
TypeScript,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
[x] Unit tests have been written where possible.
[ ] No regressions in benchmarks
[ ] Speedup in Startup benchmarks

Here is the benchmarks run, although I'm not sure how to compare it to the reference values https://github.com/enso-org/enso/actions/runs/12528446812

although I'm not sure how to compare it to the reference values

@Akirathan has provided the information here.

bench_download -h for documentation and usage

I've just tried:
enso/tools/performance/engine-benchmarks$ ./bench_download.py -s engine -b wip/db/10923-register-type-definitions-overhead develop

there are many slowdowns. Looks like allocations of atoms are slower than then used to be.
There is another set of benchmarks including startup benchmarks. Run following CI action and then generate comparition with:
enso/tools/performance/engine-benchmarks$ ./bench_download.py -s stdlib -b wip/db/10923-register-type-definitions-overhead develop

At least the startup benchmarks should show some improvement.

On my machine, after removing truffle boundaries b94f81fe92a727695d46e5b02f2abe57b0d29859 it is:
[info] Benchmark                                    Mode  Cnt   Score   Error  Units
[info] AtomBenchmarks.benchGenerateList             avgt    5   4.502 ± 1.034  ms/op
[info] AtomBenchmarks.benchGenerateListAutoscoping  avgt    5  10.528 ± 0.146  ms/op
[info] AtomBenchmarks.benchGenerateListQualified    avgt    5   4.560 ± 0.379  ms/op

Before:
[info] Benchmark                                    Mode  Cnt   Score    Error  Units
[info] AtomBenchmarks.benchGenerateList             avgt    5  20.476 ±  1.154  ms/op
[info] AtomBenchmarks.benchGenerateListAutoscoping  avgt    5  32.763 ± 19.428  ms/op
[info] AtomBenchmarks.benchGenerateListQualified    avgt    5  19.932 ±  2.811  ms/op


Protocol package should use ContractKit for tooling and testing

Expected Behavior
Please describe the behavior you are expecting
Current Behavior
What is the current behavior?

@yorhodes how urgent do you think this one is?

@yorhodes how urgent do you think this one is?


Appended "Component" to all hand written components

Reason for the change is that the Metadata component was clashing with a symbol found in some Xbox libs. Wanted to append Component for other naming reasons as well.
If there are other ideas for what to name the components feel free to comment.
Primary Reviewers:
@improbable-valentyn

Do the component classes need to be accessible through blueprint? otherwise I would probably consider wrapping the classes in a namespace rather than renaming them

I agree with Daniel that we should put them in a namespace (e.g. SpatialComponents).

After discussing this with Dan, I think a better thing to do right now is to prefix the components with Spatial (e.g. SpatialPosition, SpatialEntityAcl etc).
We should gather opinions from the rest of the team about the use of namespaces and then, if there is a decision to use a namespace for anything that isn't a UClass, apply that to everything that applies.

Ok, will change the PR to reflect that.


Address already in use

Hi, I am trying to run validate.rs using
cargo +nightly run --release --bin minionn-accuracy -- --weights PATH/TO/MODEL/WEIGHTS --images /home/USER/research/delphi/rust/experiments/src/validation/images --layers 0;
however, I keep getting this error:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 98, kind: AddrInUse, message: "Address already in use" }', experiments/src/lib.rs:67:44
Do you know as to what might be causing this problem and how I can fix it?
Thanks in advance.

I'm also getting this:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 111, kind: ConnectionRefused, message: "Connection refused" }', experiments/src/lib.rs:54:47

Hi, the first error is occurring because the server is trying to bind to port 8001 but another process is already using that port. I would check to see which process is using the port and kill it (e.g. if you're on Linux you can do something like this to find the PID, and then run kill {PID}). This will probably fix both issues (the second error is the client failing to connect to the server).

Thank you for your response. I actually did kill it. It turned out that the same process was using it. But still I get the "connection refused" error. Any idea what could be preventing the client to connect to the server?

 Usually these error messages are due to a firewall being mis-configured, but I believe this test just uses the local loopback address so that shouldn't be interfering. Are you running this inside a container? That could also cause issues.
In the meantime I would try a pgrep minionn-accuracy to make sure there aren't any stray processes. If there aren't then I would do something like nmap -Pn <IP_ADDRESS> or nmap -Pn <IP_ADDRESS> and check that a service is running successfully on port 8001.

Thank you very much! It has something to do with my system. I'm gonna leave it for later haha. It runs just fine in another system (on Chameleon). As for my system, I did as you instructed but yet couldn't figure out what was the problem. Is a conda environment considered as a container? Anyways, I deactivated it and still encountered the same problem.

Hm no, conda shouldn't affect things at all


HPMA115S0 dust sensor library broken

Dust sensor code library currently doesn't read data from sensor.. unsure if it's not structured properly or not actually transmitting the parameters to configure the sensor.. Needs analysys

Hi I just want to get the data from the sensor Do you have code about it?
I also couldn't find the sensor library.

ı use to raspberry pi 3

For Raspberry Pi 3, you should focus on the other https://github.com/dnear1/HPMA115S0-RaspberryPi-Python
This uses the uart available on the header
These 4 pins.


Using sample code, browser can not show Google map at all.

Hi! This is supposed to be a quick test of Mplace.js, but it didn't work.
Please help!
ERROR: Uncaught TypeError: Cannot read property js?key=MyGoogleAPIKEY&libraries=geometry
'idle' of undefined
at Dc (js?key=MyGoogleAPIKEY&libraries=geometry:49)
at new Mc (js?key=MyGoogleAPIKEY&libraries=geometry:49)
at Object.addListener (js?key=MyGoogleAPIKEY&libraries=geometry:116)
at Object.addListenerOnce (js?key=MyGoogleAPIKEY&libraries=geometry:117)
at Maplace.Load (maplace.js:1111)
at map.html:12
Script:

HTML
I have a div block with id=map

Is the div block before or after the js include?

I place the div block after the js include as follow:
After seeing your reply just now, I placed div block before and it also didn't work.
I'm using the Chrome browser and  installed XAMPP(Apache/PHP/MySQL) on a Mac calling this map.html from localhost.
I've tested Google tutorial and it showed up the map correctly.
https://developers.google.com/maps/documentation/javascript/adding-a-google-map
What could be wrong?  Thanks a lot! 👍
      </script>
HTML

Check if you have two lots of 'new Maplace' and just initialise once

The default div delector as per documentation is gmap, and not map.
Please change your code as follow:

Also, make sure you call the object instantiation [...] new Maplace() [...] after the div has been placed in the DOM. Your example would not work anyway.
The div block has to placed before the js. Alternatively you can use the standard jQuery helper for when the DOM is fully loaded:
$(function() {
  // handler for .ready() called
});


ApplicationMessageReceived handler triggering with duplicate messages

Describe the bug
If you subscribe to a topic with Qos of AtMostOnce and then subscribe to another topic.
The callback for the first topic is triggered twice.
This multiplies with the number of subscribes done.
It just adds it to the already existing subscribes and sends it once more to the broker.
Which project is your bug related to?

ManagedClient (V3.0.8)

To Reproduce
Steps to reproduce the behavior:

Subscribe to topic A
The message for topic A is received
Subscribe to topic B
The message for topic B is received and the message for topic A is received a second time

Expected behavior
Do not trigger the received handler again if there if there is a subscribe to a different topic.
It seems like there is a new message received for that first topic but they are the same.
(Tested with retained messages)
Additional context / logging
First subscribe to BMS/MyMES/Events & BMS/MyMES/Events/+ :
[MqttClient.MqttChannelAdapter] [Verbose]: TX (44 bytes) >>> Subscribe: [PacketIdentifier=1] [TopicFilters=BMS/MyMES/Events@AtMostOnce,BMS/MyMES/Events/+@AtMostOnce]
Second subscribe to BMS/MyMES/Get/Groups/MachineType :
[6] [MqttClient.MqttChannelAdapter] [Verbose]: TX (79 bytes) >>> Subscribe: [PacketIdentifier=2] [TopicFilters=BMS/MyMES/Events@AtMostOnce,BMS/MyMES/Events/+@AtMostOnce,BMS/MyMES/Get/Groups/MachineType@AtMostOnce]

I see that there has been some work done and that this might be a duplicate report but we are in desperate need of this fix. Any eta on a release date?

This issue is a duplate of #569
Pull request #776 fixes this issue.
When will this be released?

@cosmanovich True, I will close this one.

@BmsVisionJo You could use the develop branch and build the project on your own for the moment if it's urgent.


NEXT-37303 - Fix main branch CI part two

https://youtu.be/aWfYxg-Ypm4?si=lmBm32kqvF9WWAW5

next try 😂 one of the best vids on yt 😅

Yes, I like that one a lot 😂


cannot load such file -- winrm-fs (LoadError)

Operating System Version: Ubuntu 17.10
Provider (VirtualBox/VMWare): VirtualBox
Vagrant Version: 1.9.1
Packer Version: 1.0.2
Is the issue reproducible or intermittent?

Description of the issue:
vagrant up
causes the following error: /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- winrm-fs (LoadError)
Had to install winrm and winrm-fs plugins
Which gave another error: WinRM::WinRMWebService (NameError)
According to this site, one of the gems broke a reference: https://github.com/NetDocuments/rd-winrm-plugin/issues/29
Fix:
vagrant plugin install winrm --plugin-version 1.8.1
vagrant plugin install winrm-fs

Scrap that last fix, it then failed when doing the WEF
Downloaded Vagrant 2.0.2 from: https://www.vagrantup.com/downloads.html
Debian 64bit
vagrant plugin update
In before next error :-D

@bacardiandwatermelon did Vagrant 2.0.2 fix the issue?

Sorry for the delay..
Just to confirm, Ubuntu 17.10 comes with version 1.9.1 https://launchpad.net/ubuntu/artful/+source/vagrant Ubuntu users will need to get an updated version as specified in your minimum requirements :-) The 2.0.2 deb from vagrantup.com looks to have done the trick, got all the VMs up and running. I haven't had much of a chance to play with the lab yet though, so I assume everything is all good :-)

@bacardiandwatermelon to resolve this I added a check to the build scripts that will ensure the installed version of Vagrant is at least 2.0.0 and give the user a warning if not: https://github.com/clong/DetectionLab/blob/master/build_vagrant_only.sh#L25-L28


Database API architecture.

What is wrong
It seems our database wrapping architecture is starting to break down in some ways.  Recent work that @carver has been doing has uncovered a number of needed units of functionality that have caused the complexity level of some of the database wrappers to grow beyond what may be reasonable to maintain.
How can it be fixed.
Not clear.
Whatever our solution needs to have the following functionality (please add to this list)

batching of writes
journaling with checkpoints
trie functionality (state roots)
applying only additive changes (don't apply deletes)

https://github.com/ethereum/py-evm/blob/0cd3ebac9c7c336b07f2f52c52b069fbe400bcef/eth/db/batch.py#L48

ability to read through to the underlying database (@carver should clarify this one)
generation of diffs

https://github.com/ethereum/py-evm/blob/0cd3ebac9c7c336b07f2f52c52b069fbe400bcef/eth/db/journal.py#L235

atomic reads and writes

https://github.com/ethereum/py-evm/blob/0cd3ebac9c7c336b07f2f52c52b069fbe400bcef/eth/db/atomic.py#L49

Things that I think might be code smells or in general result in increased complexity.

Functionality flags like apply_deletes or read_through_deletes.

These require us to test all combinations of various flags.

Ability to access the underlying wrapped databases.

If we stick with the wrapping API it should be hard to get at the underlying database.

Things that I think might help us test these better.

Simple DSL for declarative testing.
State machine based hypothesis tests to better define and test the rules for each database.

Ideas:

Does a middleware pattern work for this?  It would keep each database from having access to the underlying database directly, allowing us to only expose it via a well defined API.

applying only additive changes (don't apply deletes)

(to be explicit: also applies updates, although that happens quite rarely since most data is content-addressed)

ability to read through to the underlying database (@carver should clarify this one)

This prime example when this comes up is when you are mutating a trie with pruning on. The pruning deletes all nodes no longer referenced when the trie is updated. This is good for all the intermediate tries, but bad for the trie at the state root before the updates began. We want to always be able to continue to reference those. So we want to have some concept of only deleting intermediate nodes.

One option is to make sure to squash_changes() for all changes to the trie
Another is to add a HexaryTrie.batch_update() that only starts regenerating hashes after all the new data changes. This would be a potentially large performance boost, and would also give us the concept of dropping all intermediate trie nodes for free.

Ability to access the underlying wrapped databases.

If we stick with the wrapping API it should be hard to get at the underlying database.

Yeah, a few of these APIs have evolved to the point where they aren't really a wrapper anymore, it's just a database that has a spigot to dump into another database on demand. (eg~ BatchDB.commit_to())

Does a middleware pattern work for this?  It would keep each database from having access to the underlying database directly, allowing us to only expose it via a well defined API.

I think the only way it helps is by showing that we can't (easily) get it done without direct access to the underlying, so maybe we should drop the wrapping concept altogether.


Angular 17 Module-federation migration issue

Hi...
I'm trying to upgrade a project that is using module federation plugin form Angular 16 to Angular 17.
I've completed all steps, but when I try to start the dev server compilation fails with this error.
`
./node_modules/.pnpm/@angular-architects+module-federation@17.0.7_@angular+common@17.2.1_@angular+core@17.2.1/node_modules/@angular-architects/module-federation/src/index.js:1:0 - Error: Module parse failed: 'import' and 'export' may appear only with 'sourceType: module' (1:0)
File was processed with these loaders:

./node_modules/.pnpm/@angular-devkit+build-angular@17.2.0_@angular+compiler-cli@17.2.1_@angular+service-worker<EMAIL_ADDRESS>./node_modules/.pnpm/@ngtools+webpack@17.2.0_@angular+compiler-cli@17.2.1_typescript@5.3.3_webpack<EMAIL_ADDRESS>./node_modules/.pnpm/source-map-loader<EMAIL_ADDRESS>You may need an additional loader to handle the result of these loaders.

export * from '@angular-architects/module-federation-runtime';
`

I did upgrade angular, typescript, eslint, etc..
Does anybody faced this issue?
Thanks and best regards.

I've had this same issue today. Using 17.0.6 works so maybe an issue with the latest release.

I haven't seen any blog post about support for Angular 17.2. On the other hand, I have not anything about Module/Native federation being broken with Angular 17.2 either.

Thanks for pointing this out. It was due to wrong meta mata in the package.json. It's fixed now.


Add some project info

Sure, I will get the project back. I was unsure how to tag it since their website doesn´t give much info on what DH stuff they do, but I can just mark it as TEI XML.


ECO mode writeable

Hi again,
not sure it's done correctly (not a developer and ChatGPT is able to do mistakes very well ;-) ) but it works for me :

adapter restart removed after 3rd attempt connection failure (it was not restarting it was terminating adapter)  - seems iObroker do not have "restart" option
added ECO mode to be writable
added checks so that it's set only if eco_editable is set to TRUE
shortened io-package.json history, even in my case I haven't seen any errors

Please cherry pick or reject, up to you - I will be not touching this for some moment now ;-)

but it is not very elegant. In my opinion, it would be better to perform the check initially when initializing the states. The state should only be made writable if its is supported by the device. Can you implement it in this way?

Unfortunately this is outside my limits ;-) I'm using iOBroker almost just because of HaasSohn adapter , which feeds my Homey Pro 2023 via MQTT... enabling ECO mode from time to time, so I thought I will have a look. But continue developing something I do not understand so much, it might be dangerous ;-)

the test breaks since the version in package.json needs to be bumped to v1.0.14 (i.e., it is inconsistent to the version in io-package.json.

Will fix, I was wondering but haven't seen the error/cause....


Linting with Vale

Linting with Vale and the RedHat style on push and on pull requests.
See: https://vale-at-red-hat.github.io/vale-at-red-hat/docs/end-user-guide/using-vale-github-action/

Implemented by other means in #138


+1 475 NPA XXXX

Country/region affected (e.g., "US"): US/Connecticut
Example number(s) affected ("+1<PHONE_NUMBER>"): +1 475 NPA XXXX
The phone number range(s) to which the issue applies ("+1 555 555-XXXX"): +1 475 NPA XXXX
The type of the number(s) ("fixed-line", "mobile", "short code", etc.):
The cost, if applicable ("toll-free", "premium rate", "shared cost"):
Supporting evidence (for example, national numbering plan, announcement from mobile carrier, news article): http://www.allareacodes.com/475

Duplicate of #1527.


python311Packages.exrex: add dontWrapPythonPrograms

Description of changes
$ ./result/bin/exrex -h
usage: exrex [-h] [-o FILE] [-l N] [-c] [-m N] [-r] [-s] [-d DELIMITER] [-v] REGEX

exrex - regular expression string generator

positional arguments:
  REGEX                 REGEX string

options:
  -h, --help            show this help message and exit
[...]

Fixes https://github.com/NixOS/nixpkgs/issues/271896

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 272421 run on x86_64-linux 1

9 packages built:

python310Packages.exrex
python310Packages.exrex.dist
python310Packages.myjwt
python310Packages.myjwt.dist
python311Packages.exrex
python311Packages.exrex.dist
python311Packages.myjwt
python311Packages.myjwt.dist
sherlock


AttributeError: 'AlexaNotificationService' object has no attribute 'email'

Yesterday evening it worked but now its broken again.
What I did yesterday:

Removed integration
Removed Alexa Media Player in HACS
Restart HA
Add Alexa Media Player in HACS
Add Integration
Give my Email, Password, did not give a include or exclude.
3 Devices where add: "This Device" "Peter's Alexa Apps" and "Peter's Echo Dot"
All entities are correct.

Today whole day no notifications at all, but when sunset start first notify must start. And didn't hear anything and saw this error in log.
Created a automation (when turn on a switch it must notify something)
###########################################################################################
#
# PLAY TEST AND SEE IF THINGS WORK OR NOT
#
###########################################################################################

- alias: "Test - Notify Alexa On"
  trigger:
    platform: state
    entity_id: switch.newkaku_014e5906_a
    to: "on"

  action:
    - service: notify.alexa_media
      data:
        data:
          type: announce
        message: "test"
        target: media_player.peter_s_echo_dot

System details

Home-assistant (version): 0.109.2
Alexa Media Player Add-On: Installed: v2.6.1

Logs
2020-05-04 22:01:41 ERROR (MainThread) [homeassistant.components.automation] Test - Notify Alexa On: Error executing script. Unexpected error for call_service at pos 1: 'AlexaNotificationService' object has no attribute 'email'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 153, in _async_step
    self, f"_async_{cv.determine_script_action(self._action)}_step"
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 623, in _async_call_service_step
    *self._prep_call_service_step(), blocking=True, context=self._context
  File "/usr/src/homeassistant/homeassistant/core.py", line 1247, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1282, in _execute_service
    await handler.func(service_call)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 119, in async_notify_message
    await notify_service.async_send_message(**kwargs)
  File "/config/custom_components/alexa_media/notify.py", line 209, in async_send_message
    self.email
AttributeError: 'AlexaNotificationService' object has no attribute 'email'

Additional context
Yesterday did some tests, notify worked, but now get this error again.
No change on the Add-On...

That was fixed in 2.6.1. Make sure you're actually on it. The line it's reporting isn't in the code for 2.6.1.
Duplicate of #666

Thanks, removed integration again, removed HACS again. restart HA, add HACS, add integration and did the test. Alexa now say the message. I'll keep a eye on it.


Import Nexus IQ to DefectDojo not same result

Bug description
I scanned  an example project :  https://github.com/WebGoat/WebGoat.git , using Nexus IQ and imported the result file to DefectDojo , but didn t get exactly the same result.
Steps to reproduce
Steps to reproduce the behavior:

Scan WebGoat project with Nexus IQ
Generate File
Import file to DefectDojo
See DefectDojo report
See Nexus Iq report

Expected behavior
I expected to see the same number of issues in DefectDojo and Nexus IQ
Deployment method (select with an X)

[ ] Docker Compose
[ ] Kubernetes
[ ] GoDojo

Environment information

Operating System: [e.g. Ubuntu 18.04]
DefectDojo version (see footer) or commit message: v. 2.37.1 ( release mode )

Sample scan files
trigger: none
pool: agents-dev
variables:

name: nxiqUrl
value: 'https://xxx.xxxx.xxxx'
name: nxiqAppId
value: 'webgoat'
name: ddProduct
value: 'testProduct'
name: ddProductType
value: 'testProductType'
name: ddEngagement
value: 'Nexus IQ API'

jobs:

job: build
displayName: build
container:
image: testing/maven:3.8.3-openjdk-17
endpoint: test-dev
steps:

task: CmdLine@2
inputs:
script: 'git clone -b v2023.4 --single-branch https://github.com/WebGoat/WebGoat.git'

task: Maven@4
inputs:
mavenPomFile: 'WebGoat/pom.xml'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
javaHomeOption: 'JDKVersion'
mavenVersionOption: 'Default'
mavenAuthenticateFeed: false
effectivePomSkip: false
sonarQubeRunAnalysis: false

task: PublishPipelineArtifact@1
inputs:
targetPath: '$(System.DefaultWorkingDirectory)/WebGoat/target'
artifact: 'webgoat'
publishLocation: 'pipeline'

job: scan_nxiq
displayName: Scan Nexus IQ
dependsOn: build
container:
image: testing/openjdk:11
endpoint: test-dev
steps:

task: DownloadPipelineArtifact@2
inputs:
buildType: 'current'
artifactName: 'webgoat'
targetPath: '$(System.DefaultWorkingDirectory)/WebGoat'

task: NexusIqPipelineTask@1
inputs:
nexusIqService: 'iq-dev'
organizationId: 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'
applicationId: 'appId'
stage: 'Build'

job: defectdojo
displayName: DefectDojo Import
steps:

task: AzureKeyVault@2
inputs:
azureSubscription: 'test2-dev'
KeyVaultName: 'test-dev'
SecretsFilter: 'NexusIQUserCode, NexusIQPassCode, dd-api-key'
RunAsPreJob: true

task: CmdLine@2
displayName: Generate Nexus IQ Report JSON File
inputs:
script: |
npm install minimist
node ./generate-report-data.js
--user-code $(NexusIQUserCode)
--pass-code $(NexusIQPassCode)
--nxiq-url 'https://xxxx.xxxxx.xxxx'
--public-app-id 'appId'

task: CmdLine@2
displayName: Import to DefectDojo
inputs:
script: |
curl -X POST "https://xxxx.xxx.xx/api/v2/import-scan/"
-H "accept: application/json"
-H "Content-Type: multipart/form-data"
-H "Authorization: Token $(dd-api-key)"
-F "product_type_name=$(ddProductType)"
-F "active=true"
-F "verified=true"
-F "engagement_name=$(ddEngagement)"
-F "minimum_severity=Info"
-F "product_name=$(ddProduct)"
-F "scan_type=Sonatype Application Scan"
-F<EMAIL_ADDRESS>-F "auto_create_context=true"
cat nxiq-report.json

Screenshots
If applicable, add screenshots to help explain your problem.

Hi @benamar19. Thank you for reporting the issue. Unfortunately, it is quite hard to find an exact bug and make any adjustments in parsers without testing data.
Can you please provide your nxiq-report.json report file?

Hi  @kiblik , Thanks , The Nexus Iq is joined here since it s too long .
nexus-iq-report.json

For Sonatype Application Scan is severity calculated based on following code:
https://github.com/DefectDojo/django-DefectDojo/blob/7533448196bbd775c7e66b7e36c98166fa000803/dojo/tools/sonatype/parser.py#L78-L86
I suppose it follows https://help.sonatype.com/en/security-policies.html (calculation looks same to me).
Can you please provide example of some exact finding which is not calculated correctly? As I do not have access to your system, I'm not able to identify it.
Or if is severity in Nexus Iq calculated differently, please explain how.
I see that total number is different (180 != 183). Can you identify which finding is it, please?

The 3  issues that was reported by Nexus Iq  but not by DefectDojo are:
1 - org.springframework.boot : spring-boot-jarmode-layertools : 2.7.1    :   Violation of Component-Similar
Unknown modification to component is in violation for the following reason(s):
Match state was 'Similar'
Coordinates were org.springframework.boot : spring-boot-jarmode-layertools : 2.7.1 (do not match org.eclipse.* : * : * : * : *)
2 -       org.jruby : jruby-stdlib : <IP_ADDRESS>   :    Violation of License-Threat Not Assigned:
License threat group has not been assigned is in violation for the following reason(s):
Found licenses that are not assigned to any license threat group ('PSF')
3 -   webgoat-2023.4.jar    :     Violation of Component-Unknown
Unknown 3rd party component is in violation for the following reason(s):
Match state was 'Unknown'
Component does not contain proprietary packages
Data Source has support for Identity


Maven - Plugins and Dependencies

Description
Maven
Explanation about maven and its plugins and dependencies and their use and how to add them.
Domain
Backend Dev (Java)
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign


Provide more details on access rule evaluation

Hi @MarkvanMents, I have  responded. Shall I make changes or do you want to?

Hi @MarkvanMents, I have responded. Shall I make changes or do you want to?

Perhaps better if you make the changes? Especially the one about when the object is retrieved and how the access rules are then applied? Then I'm sure it is technically correct.
But let me know in the unlikely event that you are struggling to find the words 😀.

Nice suggestion @MarkvanMents. Can we merge it?

Merging.
Thanks for all your work.


No License Intentional?

Is it intentional that you have not chosen a license for this repository?
Citation of GitHub's documentation about open source licensing:

You're under no obligation to choose a license. It's your right not to include one with your code or project, but please be aware of the implications. Generally speaking, the absence of a license means that the default copyright laws apply. This means that you retain all rights to your source code and that nobody else may reproduce, distribute, or create derivative works from your work. This might not be what you intend.

The reason why I'm asking is that I'm thinking about porting this project to the Python 3 Script Hook for GTA V. But without your permission, I have no right to do so (the same applies to many others, of course).

I wanted to use a MIT license but I keep forgetting to upload it. I'll do it right now.

Alright, thanks for the quick reply. :)

I've decided with the GPL license.


Add instructions on managing a separate mount namespace

In short, there are some real-world benefits to putting kubelet and the container runtime (and any mountpoints created by these or their children) in a separate mount namespace from the parent OS.  This document outlines why one might want to do so, how to do so, and what the downsides are.
After some initial discussion around this idea in sig-node, @SergeyKanzhelev had the great idea of documenting this optional mode of operation of a kubernetes cluster here instead of https://github.com/kubernetes/community/pull/5724 where it was originally targeted.

/sig node

I agree with

We (SIG Docs) can help turn this into a task page or a blog article. Right now it's not quite one thing or the other.

This would make a good blog post. Additionally, the instructive bits could be extracted to form a task page.

If you'd like me to prepare a draft of that, let me know @lack :)

If you'd like me to prepare a draft of that, let me know @lack :)

@geoffcline Sure!  I'd appreciate any help you can offer, getting this to fit into the website standards and conventions!

@lack I would love to see this land as a blog or a task. Given the age of the PR with no action I'm going to close it, but feel free to reopen / edit as you see fit. I really like the content you've created and would hate to see it not land somewhere.
/cc @geoffcline
/close


TypeError: load_dataset() got an unexpected keyword argument 'test_with_fake_data'

Environment info
Operating System: ubuntu 14.04 and Mac OSX
Pip install: 0.9.0
In 'tensorflow/tensorflow/examples/skflow/' folder, when run 'text_classification*.py',
the code
'''
dbpedia = learn.datasets.load_dataset(
'dbpedia', test_with_fake_data=FLAGS.test_with_fake_data)
'''
cause a bug
'''
TypeError: load_dataset() got an unexpected keyword argument 'test_with_fake_data'
'''
It is because that if user install tensorflow by pip, the definition of 'load_dataset()' in 'tensorflow/tensorflow/contrib/learn/python/learn/datasets/init.py' is
'def load_dataset(name):',
but in current git version, the definition of 'load_dataset()' is
'def load_dataset(name, size='small', test_with_fake_data=False):'.

@ziaoang Yeah you are using the old version. Please install the latest version.

Do you mean I should re-install tensorflow via source code instead of pip?

I re-install tensorflow via source code, and the example runs normally. Thank you @terrytangyuan .


Reduce GIF frame window from 30ms to 20ms

Pull request checklist:

[x] CHANGELOG.md was updated, if applicable

Description
Currently the check for the next animated frame is every 30ms which happens to be too slow for the gif standard, the maximum speed of a gif is 20ms per frame.
An edge case would be when a gif only has 2 frames

Frame Number
Frame delay

1
20ms

2
20ms

The total frame duration is 40ms
In the current implementation, if we start at a 0 offset

Offset
Offset mod 40
Time Passed
Frame Displayed

0
0
0ms
1

33
33
30ms
2

66
26
60ms
2

99
19
90ms
1

132
12
120ms
1

As you can see the frames are duplicated when they shouldn't be.
Its like a train station with trains ever 30ms and we are only ready to board every 20ms, and since we are impatient we miss our train.
Perhaps a smarter way of detecting which frame to select is a more performant approach however I haven't thought of how to do that. Or we could just say that 30ms is the min frame timing instead of 20ms
fixes https://github.com/SevenTV/chatterino7/issues/85
This one works and has been tested on windows by myself, I haven't tested osx because i dont have a mac.
Please can someone else confirm that this works on windows, linux and osx.

Seems fine on Windows, I can slightly see the speed difference on old emotes, but without lag I doubt others will be able to notice
will need the changelog again

The slight difference is likely actually lag on windows. Since the faster emote timing is now closer to how browsers are implemented. Try comparing the emotes in chatterino vs the raw url in a browser tab. @Felanbird

Perhaps @pajlada we can write a test to test the interval at which the timer fires. Which can help mitigate this problem. I would do it but i dont know how the testing works here.

The vcpkg config change is because that dep is missing when you try install via vcpkg. Probs unrelated to this pr.

Twitch Channel
Emote Name
Link to emote

pajlada
WAYTOODANK
https://betterttv.com/emotes/5ad22a7096065b6c6bddf7f3

pajlada
Ninja
https://betterttv.com/emotes/5e7f2cc0d6581c3724c16b84

pajlada
pajaAAAAAAAAAAA
https://betterttv.com/emotes/5e8b989d09989e5cdff54e2e

pajlada
pajaHop
https://betterttv.com/emotes/56eddabbf854c2297286e5e8

pajlada
AlienPls
https://betterttv.com/emotes/5805580c3d506fea7ee357d6

pajlada
ppHopper
https://betterttv.com/emotes/5b0da25016252035ad06900e

pajlada
pajaSWA
https://betterttv.com/emotes/5738dc6ad7a0b60d147270a0

When comparing timings please compare between this commit and the master commit and also look at the browser since this change should try and make it as close to how its implemented in the browser as possible. (the master commit might be wrong)

@TroyKomodo Merge it with the latest main branch

Does this actually fix the chatterino7 issue? It seems a bit faster but is still not the required speed to match (the thumbnail/the website listing)
I can't show it in a gif b/c the thumbnail is 2 fast 2 furious

Currently the min allowed is 20ms on our fork people have been asking me to allow less than 20 because of webp allowing 1ms frame timings and stuff like that. I haven't allowed it yet. However we might revisit this change when we get round to the c2 c7 merge.


UI for adding rights statement shells

One major piece of functionality for this application is creating rights statement shells, which can then be associated with groupings. This should include all fields in the Rights Shell and Rights Granted models. Eventually there will be logic to make certain fields inactive based on basis type.

Quick wireframes:

Quick wireframes:


feat(pkg/kube): add openapi validation for k8s objects

Add back OpenAPI validation for kubernetes objects.
Fixes: #6382

Seems fairly straightforward. LGTM!


prefooter, git-info.html, Analytics alt image src

This is a Bug Report
Analytics link on content pages.
Problem:
In the prefooter, there is an "Analytics" link that is displaying the alt text. Investigate if the query
for the image source is correct. I am not familiar with the functionality or history of the code in the prefooter.

Proposed Solution:
To be investigated.
Page to Update:
Investigate the query from the partial, layouts/partials/git-info.html

# In repo root
$ rg '\?pixel'

content/zh/docs/tutorials/stateful-application/cassandra.md
807:[!Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/cassandra/README.md?pixel)]()

layouts/partials/git-info.html
7:

Point being, there's a reference in content/zh/docs/tutorials/stateful-application/cassandra.md:807 as well.

Visiting one of these analytics links, i.e. [1], yields a page that displays
Go 1.9 is no longer available. Please refer to https://goo.gl/aESk5L for more information.

[1] https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/concepts/services-networking/ingress.md?pixel

Thanks @nicolaiskogheim . Yes, I agree.

/kind bug

/assign

/close


[Bug] PARSING FAILED when "(=)" in Footnote

This is a BUG[^Bug]

[^Bug]: Below

    (`=`)

If you place this construct and activate Dataview Plugin.
And delete the = and retype it or if you delete the ) and retype it will give this message...
Dataview (inline field ''): Error:
-- PARSING FAILED --------------------------------------------------

Got the end of the input

Expected one of the following:

'(', 'null', boolean, date, duration, file link, list ('[1, 2, 3]'), negated field, number, object ('{ a: 1, b: 2 }'), string, variable

if you deactivate the dataview plugin the error remains, if you deinstall dataview the error will disappear, if you reinstall the plugin the error will not appear, if you edit the text the error will reapear.
its very odd
the error did appear with a lot of plugins, i have installed all footer* plugins.
i did uninstall and reinstall all of them ... the only one that affects the error message is dataview plugin... i did test it also in a empty new test vault ... the only and single plugin that affects the error is the dataview
Expected behavior
No Error Message
Desktop (please complete the following information):

OS: [Windows]
Obsidian Version [e.g. 0.12.15]
Dataview Version [e.g. 0.4.10]

Additional context
In Autoupdate or Install from list i do not see a version 0.4.2 of dataview

That's because this is the default inline dataview syntax - = <query> will evaluate a dataview expression inline. In this case, there is no query to evaluate, so it shows an error.
I will improve this behavior with a better error message (or not do anything if the query is empty).

Note you can change the dataview inline prefix to something else in the settings, which also solves this particular issue.

Note you can change the dataview inline prefix to something else in the settings, which also solves this particular issue.

oofff this was a good hint, i did change the inline prefix to DATAVIEW= and not i can also write = as inline code.
Thanks a lot

I think this issue is closed as we found some solution (see last two posts above).
May this behavior will get later improved, how ever, until now we have this workaround you can change the dataview inline prefix to something else than a single "=" in the settings as in example "DATAVIEW=" to then use a single "=" as inline code.
Thanks to all and good luke


Add option to update BepInEx packs < v5.4.23 before installing

Motivation
BepInEx versions prior to v5.4.23 come with doorstop v3 or earlier, and do not support the --doorstop_enabled command line flag which provides a convenient way to create shortcuts to run games either modded or vanilla, which is particularly useful for Steam games where simply running the game's .app directly to bypass the run_bepinex.sh script means losing out on in-game Steamworks features such as the Steam overlay, achievements, and so on. Moreover, some games restart themselves with Steam when they detect the missing Steamworks API, leading them to boot up modded when we wanted vanilla.
Doorstop's wiki claims that since v2.3 it has supported the --doorstop_enable flag, but I have had no luck getting this to work with BepInEx v5.4.22, and the run_bepinex.sh script of BepInEx v5.4.22 includes no mention of the flag. Older doorstop also makes use of the DOORSTOP_ENABLE environment variable, however BepInEx v5.4.22's run_bepinex.sh always sets this to TRUE.
As such, gib currently foregoes offering the vanilla shortcut for Steam games when it detects the BepInEx pack does appear to include support for the --doorstop_enabled flag, instead advising the user that if they would like a vanilla shortcut, they should finish the installation of the current pack to ensure any game-specific customisations are installed, and then to use gib again to update BepInEx to the latest v5 release by downloading it and running gib again on it.
This is quite tedious.
Instead, when detecting a lack of support for the --doorstop_enabled flag, gib could offer to download the latest official release of BepInEx 5 (or falling back to a known good v5 release on error) and merge its contents with the provided BepInEx pack, automatically taking care of the issue for them.

Interestingly, it seems some BepInEx packs out in the wild contain support for the --doorstop_enable flag of earlier doorstop versions. So, we should probably also add support for this, but if support for neither flag is found, then offer to update to latest BepInEx v5.


Consider removing IValueProvider and only using IBindingSourceValueProvider or another variant

Having the IValueProvider AND IBindingSourceValueProvider interfaces made me believe I could build a value provider that sat next to the other value providers and then modify values (a value not from a source). Seeing that all of our value providers are binding source value providers and the infrastructure around them expects a certain behavior, having the IValueProvider interface only adds confusion.

Unfortunately not high enough value here.


Is using non bundled zeromq still supported on windows ?

Hi,
While trying to update pyzmq to 16.0.0 (freom 15.2.0), I noticed that when using bundled zmq sources, pyzmq builds libzmq as a python extension. When using the --zmq=<path to zmq lib>, this extension is not built as expected, but  the zmq.__init__ seems to assume libzmq to be available as a python extension, and otherwise does not load the dll at all, thus causing import failures.
If the intent is to still support this method, would it make sense to simply build that extension in both bundled and non bundle modes (in the non bundled mode, the extension would of course link to the externally provided zmq).

I don't think the extension has a non-bundled mode. Either there is a bundled Extension or there isn't an Extension. If zmq is external, the Extension is skipped.
I'm happy to support building with non-bundled windows zeromq, I just don't have any experience/expertise for maintaining it.
16.0 has some changes to simplify loading the bundled libzmq, I don't think it was intentional for this to result in breaking things. But now it makes sense that the switch to always import _libzmq would preclude that.
What would be 100% super awesome would be if someone could add an AppVeyor build that installs pyzmq with libzmq already on the system. That probably isn't going to be me, though.

dll bundling has changed and uses standard DLL loading thanks to delvewheel. No need for libzmq as an extension anymore, most of the time.


feat!: add parameter projectId to filter deployments correctly

close #3

If still relevant for you https://github.com/marketplace/actions/get-vercel-preview-url-by-project-id

Thank you for your reply.
I made my own one base on your source code. Thanks again.
https://github.com/justincase-jp/vercel-preview-url-alias


Python 3 only support

Codecov Report

Merging #65 into master will decrease coverage by 0.84%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #65      +/-   ##
==========================================
- Coverage    77.3%   76.46%   -0.85%
==========================================
  Files          26       26
  Lines         608      565      -43
==========================================
- Hits          470      432      -38
+ Misses        138      133       -5

Impacted Files
Coverage Δ

django_nyt/consumers.py
0% <ø> (ø)
:arrow_up:

...o_nyt/migrations/0004_notification_subscription.py
100% <ø> (ø)
:arrow_up:

django_nyt/migrations/0001_initial.py
100% <ø> (ø)
:arrow_up:

django_nyt/decorators.py
70.96% <ø> (-1.76%)
:arrow_down:

django_nyt/migrations/0006_auto_20141229_1630.py
100% <ø> (ø)
:arrow_up:

django_nyt/tests/test_views.py
100% <ø> (ø)
:arrow_up:

django_nyt/forms.py
100% <ø> (ø)
:arrow_up:

django_nyt/migrations/0008_auto_20161023_1641.py
100% <ø> (ø)
:arrow_up:

...grations/0007_add_modified_and_default_settings.py
100% <ø> (ø)
:arrow_up:

django_nyt/subscribers.py
0% <ø> (ø)
:arrow_up:

... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f4ea22...869e5b5. Read the comment docs.


Add ROI tables & a Fractal task

@tibuch @imagejan Super cool library for converting MD images into OME-Zarrs! 👏🏻
I would like to add Fractal support to use this library. That means 2 things:

Add ROI tables to the MD conversion (because Fractal relies on ROI tables for downstream processing)
Add actual Fractal wrapper tasks

This PR contains a proposal to add both things to this repository, as well as a few minor changes I'll go in detail about below (and that we can also move to a different PR, if that's preferred).

ROI tables
We typically rely on 2 types of ROI tables being created by the OME-Zarr converter: Whole-well ROI (trivial, but allows us to use ROI-based loading in a general manner) and FOV ROIs (describing which region of the full image belong to which field of view from the microscope).
To add FOV tables, this needs to be integrated into the core calculations of loading the images (because that's when the decision is made on how to place the FOVs in the well image). Thus I have integrated this into the core functionality in MetaSeriesUtils and now a few additional functions return a roi_tables dictionary (keys: str = table_name, values: pd.DataFrame actual table).
The actual table writing is optional, the parsing functions now just return a dictionary with tables that can be written to an AnnData table in the OME-Zarr. I also included a helper function to actually do this in Zarr.py as write_roi_table.
Unfortunately, this is does mean that it's a breaking change to the write_cyx_image_to_well & write_czyx_image_to_well  API in that it now returns an additional output (the roi_tables dictionary). I have updated all tests, examples and usage of this function in this package. But if e.g. the prefect tasks depend on this repository, it would require a small adaption in those scripts (e.g. just adding an , - to the outputs of write_cyx_image_to_well & write_czyx_image_to_well to ignore those roi_tables.
All other changes should be transparent to the usage of this library, as far as I can tell.
Including the ROI tables writing in the main package (and being able to read them in the tests) does mean that anndata would become a core dependency.

Fractal tasks
I included 2 Fractal tasks in a subpackage. These tasks allow users to parse 2D, 3D or mixed MD images into an OME-Zarr & write the ROI tables.
We could also have these tasks in a separate package if you prefer. It should not interfere with anything in the main package though.
The tasks are in their sub package and running them in Fractal comes with an extra dependency on fractal-tasks-core (we're just refactoring this to be much lighter) & pydantic. This dependency is only needed when installing it to run in Fractal. Thus, they are in an extra fractal-tasks.
In addition, there is a small json manifest for the tasks, some tests to check their output & a small example in the examples folder using your example data from resources.

Minor suggested changes
a) Changes to build_omero_channel_metadata: I added a check that start and end are not the same value (leads to issues in napari otherwise). Also, proposing to switch to 0.999 percentile rescaling to make the visualization a bit less harsh by default (alternatively, we could expose the percentiles as an option the user can set)
b) dimension_separator: I hard-coded the dimension_separator = “/” now. I don’t fully understand how it picks the defaults. But sometimes, it picks “.” as a dimension separator for me, and that does not work with napari-ome-zarr for viewing the images. We can also make this an option of course to parse through everything.

Summary
I'm proposing:

Modify the API of write_cyx_image_to_well & write_czyx_image_to_well slightly to also output ROI tables
Add a anndata dependency to the package
Add 2 fractal tasks to a subpackage
Add an extra for fractal-tasks with their additional dependencies
Change the behavior of build_omero_channel_metadata slightly
Hard-code the dimension_separator to /

Fixed the pre-commit issues now. It looks like one of my new test for the Fractal tasks seems to fail in the CI, but it's running locally. Will need to look into this afterwards.

Hi @jluethi,
Thank you for opening this PR. Extracting the information of the individual fields of view during the data parsing and handing it back makes a lot of sense. I also like adding support to write tables to AnnData following the emerging NGFF spec.
However, I would like to keep the core processing functionality strictly separated from any workflow orchestration tool.
Could you refactor the PR such that only core functionality is added to faim-hcs? We could also schedule a pair-programming session where we go over the PR together.
Cheers!

Hey @tibuch
Thanks for the feedback. Yes, I can refactor it that way and make a new repository for the actual task, which will depend on the faim-hcs repository. Are there plans to deploy this repository to pypi for easier dependency handling?
I'll ping you for a pair-programming session to go over the refactored PR once I have it ready.

Hey @tibuch
I went through all the suggested changes from our meeting on Monday (cleanup, using the output of compute_z_sampling for the Z extent of the ROIs, parsing dimension separator & pyramid settings through all the functions, refactoring the ROI table creation & writing a bit to reduce redundancies).
I have one additional suggestion to add to this PR (see below) that came up. Other than that, it would be ready from my side.

During the work on pyramid level specification, I noticed that it's hard to solve https://github.com/ome/napari-ome-zarr/issues/84 with just the two existing parameters. Thus, I'm suggesting we add an additional parameter there:
Add a lowest_res_target parameter to the chunk size calculation in _compute_chunk_size_cyx to replace the check on whether the calculated resolution should be the lowest resolution created.
Reason: Chunk size of 2048 is a good target for most resolutions. But with the current setup, a 384 well plate would get chunks of 1024 each at the lowest resolution pyramid level. For 24 wells, that leads to a plate image of 24'576 => to large for most GPUs to display in napari (see  https://github.com/ome/napari-ome-zarr/issues/84)
⇒ A user should be able to overwrite this, i.e. set it to 512 or 256 to enforce that low resolution pyramids are created and it shouldn’t relate to optimal chunk sizes at higher resolutions.
By default, the behavior will remain as it was before (I think that’s a reasonable default for anything below 384 well plates)
There is some interaction with max_levels (as before): Whichever condition is hit first limits the number of pyramid levels. That can be a bit confusing, but also desired flexibility. I currently still kept it in (behavior as before).

Thanks @imagejan ! I'll update the PR to address these docstring & pytest suggestions.
Regarding the main questions:

Shouldn't the column naming (x_micrometer etc.) be a bit more flexible and not hard-code the unit?

Yes, this should eventually become more general. The first version of Fractal ROIs has these hard-coded column names though and this PR adds support for such Fractal ROIs. There should be more generic ROIs eventually, but I'm not fully sure what direction they'd go (maybe the shapes variant of spatial data regions, see https://spatialdata.scverse.org/en/latest/design_doc.html#regions-of-interest ? Or some other broader approach?).

Why did you remove the optional inclusion of the z_position from the metadata here? That's unrelated to the topic of this pull request, isn't it?

I assume you mean include_z_position in get_well_image_CYX?
Agreed, I didn't remove it at first. During a review with @tibuch , we noticed that the variable is not used anymore in the get_well_image_CYX function since https://github.com/fmi-faim/faim-hcs/commit/ba0f7ff9eff2669f564029ab1debb96eede099ba (it was left in the docstring, but removed from the actual function). Plus, I'm not sure what the z_position would be in 2D only images anyway. @tibuch suggested removing it as part of cleaning up this function. We can also add it back if there's a case to be made for what z_position should be in the 2D case and if we add back use of the include_z_position variable within the get_well_image_CYX function.

Would be great if you could run the tests in the github CI again. Had some issues running them locally with the test data (yay Airport wifi? Didn't manage to check before boarding)
I think I've addressed all the issues mentioned above. Let me know if you'd be happy with fixing the AnnData version for the time being, given that the write_elem import comes from the experimental _io subpackage.

Tl;dr: It wasn't a local CI issue. The same bug gets reproduced locally as in the Github CI. It seems to be an fsspec issue (see below for my investigation on it).
Looks like this affects ome-zarr-py as well, see: https://github.com/ome/ome-zarr-py/issues/302
Their error messages are very similar to ours here.
I suggest we fix fsspec to fsspec==2023.6.0 (the one before the fsspec==2023.9.0 that breaks ome-zarr-py atm) until we know how to address this.

The very confusing thing: Commit c471dd1003bf4f0d2b33cd33ff3c41a8fc59d587, which passed the CI above (the one before my latest changes) now also fails when I run the test this way. Even the current main branch seems to fail with the same error as above, at least in my local testing. Thus, it appears that some dependency changed since to trigger this.
I checked ome-zarr (I had it working in 0.7.1 before, now it's at 0.8.0. But tests in main still fail with it).
Also checked Zarr (I had it working with zarr-2.13.6, now it's at 2.16.1. But test still fail with 2.13.6)
Also checked fsspec (I had it working with fsspec==2023.5.0, now it's at 2023.9.0) => downgrading fsspec to 2023.5.0 made tests pass again locally. Independent of the ome-zarr & zarr version above

The most recent commit fixes fsspec<=2023.6.0 and limits anndata to anndata>=0.8.0,<=0.9.1.
Would be great to run the online CI again with this, locally it's working again. And that would stop potential anndata changes to interfere with faim-hcs for the moment

@jluethi is there somewhere a Fractal ROI Table spec to which we could link? Maybe even a versioned spec?
Otherwise this looks good to me. I think we can merge it.
@imagejan is there something you would like to see addressed?

We're working on a formalized Fractal ROI table spec, it's planned for October. I made another reminder issue here for it: https://github.com/fmi-faim/faim-hcs/issues/41


Add more python versions to Travis CI

This should help support a few more versions of Python. I'm currently having issues on 3.8.2 and I wonder if it's something on my end.

And bingo, it's not just my install! It looks like https://github.com/alvinwan/timefhuman/blob/e3bace6a283ba85024f2361c66ba5e9a365e3710/timefhuman/categorize.py#L119
is crashing if monday is 4 days away from the 31st day of the month. That's a pretty wacky and intermittent bug if I'v ever seen one! :D

Ipython debug log

[ins] In [3]: import timefhuman

[ins] In [4]: timefhuman.timefhuman('october 27th')
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-4-ed924c5d5f0d> in <module>
----> 1 timefhuman.timefhuman('october 27th')

~/.local/lib/python3.8/site-packages/timefhuman/main.py in timefhuman(string, now, raw)
     56         now = datetime.datetime.now()
     57
---> 58     tokens = timefhuman_tokens(string, now)
     59
     60     if raw:

~/.local/lib/python3.8/site-packages/timefhuman/main.py in timefhuman_tokens(string, now)
     73     """Convert string into timefhuman parsed, imputed, combined tokens"""
     74     tokens = tokenize(string)
---> 75     tokens = categorize(tokens, now)
     76     tokens = build_tree(tokens, now)
     77     return tokens

~/.local/lib/python3.8/site-packages/timefhuman/categorize.py in categorize(tokens, now)
     30     tokens = convert_words_to_numbers(tokens)
     31     tokens = convert_day_of_week(tokens, now)
---> 32     tokens = convert_relative_days_ranges(tokens, now)
     33     tokens = convert_time_of_day(tokens)
     34     tokens = maybe_substitute_hour_minute(tokens)

~/.local/lib/python3.8/site-packages/timefhuman/categorize.py in convert_relative_days_ranges(tokens, now)
    117             (("yesterday",), DayToken.from_datetime(now - datetime.timedelta(1))),
    118             (("weekend",), DayRange(saturday, saturday + 1)),
--> 119             (("weekdays",), DayRange(monday, monday + 4))):
    120         for keyword in keywords:
    121             tokens = [replacement if token == keyword else token \

~/.local/lib/python3.8/site-packages/timefhuman/data.py in __add__(self, other)
    179         """
    180         assert isinstance(other, int)
--> 181         return DayToken(self.month, self.day + other, self.year)
    182
    183     def __radd__(self, other):

~/.local/lib/python3.8/site-packages/timefhuman/data.py in __init__(self, month, day, year)
    135
    136         assert month is None or 1 <= month <= 12
--> 137         assert day is None or 1 <= day <= 31
    138
    139     def combine(self, time):

AssertionError:

[ins] In [5]: %debug
> /home/dominik/.local/lib/python3.8/site-packages/timefhuman/data.py(137)__init__()
    135
    136         assert month is None or 1 <= month <= 12
--> 137         assert day is None or 1 <= day <= 31
    138
    139     def combine(self, time):

ipdb> day
34
ipdb> up
> /home/dominik/.local/lib/python3.8/site-packages/timefhuman/data.py(181)__add__()
    179         """
    180         assert isinstance(other, int)
--> 181         return DayToken(self.month, self.day + other, self.year)
    182
    183     def __radd__(self, other):

ipdb> self.day
30
ipdb> other
4
ipdb> l
    176         >>> d1 = DayToken(3, 2, None)
    177         >>> d1 + 3
    178         3/5
    179         """
    180         assert isinstance(other, int)
--> 181         return DayToken(self.month, self.day + other, self.year)
    182
    183     def __radd__(self, other):
    184         """
    185         >>> d1 = DayToken(3, 2, None)
    186         >>> 3 + d1

ipdb> up
> /home/dominik/.local/lib/python3.8/site-packages/timefhuman/categorize.py(119)convert_relative_days_ranges()
    117             (("yesterday",), DayToken.from_datetime(now - datetime.timedelta(1))),
    118             (("weekend",), DayRange(saturday, saturday + 1)),
--> 119             (("weekdays",), DayRange(monday, monday + 4))):
    120         for keyword in keywords:
    121             tokens = [replacement if token == keyword else token \

ipdb> monday
3/30/2020
ipdb> monday + 4
*** AssertionError
ipdb> monday + 1
3/31/2020
ipdb> monday + 2
*** AssertionError

I think Travis is complaining because of the intermittent issue you've discovered. I'll fix that test just to make sure. 😂


Feature/mm 42, 53 | poll adjustment and added tests for Overview

this contains updates for three different jiras. The reason being that they are so interconnected it would be difficult to pull them in seperatley


HHH-11596 - @OneToOne association with @JoinTable ignores optional attribute

https://hibernate.atlassian.net/browse/HHH-11596

Applied upstream, thanks. I also added another test for optional=false.


fix:  value  is same as message in  setFieldError

setError is setFieldError bound to specific field. Therefore the setError value should have same type as setFieldError message

Yup