WIP: integration testing via pytest-docker

No rush here either - I don't have this at a place it's ready to merge yet but I would love to get your input on the direction to take. So far I have a very basic integration system set up but it needs a lot of polish added.
Running list of goals:

[X] Ability to test against multiple netbox versions in a single pytest run
[ ] ~Ability to automatically determine what versions of netbox to test against~

~Do you know what to test against here? Off the top of my head I would think to scrape the latest verisions from the netbox releases github page and pull the latest version of each of the 3 most recent major/minor combos.. so like right now it would be v2.10.2, v2.9.11, and v2.8.9. Thoughts?~

[X] Ability to override the default automatically chosen netbox versions to test against via pytest args
[X] Clean up any templated docker-compose yaml files after the pytest run completes? Or perhaps clear them before a new run?
[X] Clean up any leftover docker containers on an unclean pytest exit? Or clean them before a new run?
[ ] ~Add a pytest flag to disable all integrations tests? Should the default be enabled or disabled?~
[X] Add a pytest flag to skip cleaning up the containers after a run so they can be poked at manually
[ ] Add devicetype library integration and auto-import a few device types to the containers
[ ] Clean up docker networks and volumes in addition to containers

Is there anything else that should be done for a basic integration suite feature add?
The short version of what this does is that when pytest runs it clones the netbox-docker repo and then reads the docker-compose.yml file included in that repo as a base config but then renames some things in there (mostly prefixing names with the netbox version for that group of containers, and putting them all in a dedicated network for that netbox version) so that we can spin up multiple versions of netbox at once via writing out multiple slightly altered docker compose files based off of that base config. Is there a better way to test against multiple versions at once?
Right now with me testing against 2 versions of netbox the total run time is about 2 and a half minutes. Adding multiple netbox versions does not really add much additional time, just load on the system since most of the time is spent populating the database and caching static files from what I've seen so far.
This is just a jumping off point, let me know what you think. Thanks :)

[ ]  Ability to automatically determine what versions of netbox to test against

Do you know what to test against here? Off the top of my head I would think to scrape the latest verisions from the netbox releases github page and pull the latest version of each of the 3 most recent major/minor combos.. so like right now it would be v2.10.2, v2.9.11, and v2.8.9. Thoughts?

netbox-docker should let us pull the latest minor release by just specifying the major release number, e.g. v2.9 will pull the v2.9.11 image. At least, I'm pretty sure. I think I'd rather manually specify which releases we're testing against. That way when a new version comes out we don't have the potential to randomly start failing tests. It's nice to know, but probably frustrating on a PR that deals with something else entirely.

[ ]  Ability to override the default automatically chosen netbox versions to test against via pytest args
[ ]  Clean up any templated docker-compose yaml files after the pytest run completes? Or perhaps clear them before a new run?
[ ]  Clean up any leftover docker containers on an unclean pytest exit? Or clean them before a new run?
[ ]  Add a pytest flag to skip cleaning up the containers after a run so they can be poked at manually

Yep, those all make sense, will have to familiarize myself with pytest-docker before I can help here.

[ ]  Add a pytest flag to disable all integrations tests? Should the default be enabled or disabled?

Could just provide a more specific path to the unit tests if you wanted to bypass, e.g. pytest tests/unit.

Is there anything else that should be done for a basic integration suite feature add?
This is just a jumping off point, let me know what you think. Thanks :)

This is a great start, had no idea we could do all this in pytest. I'd like to convert at least a few of the fixture-based tests over to this to get a better idea of how we'd use it in practice. Particularly curious about how we can best handle dependencies. Like how testing devices requires us to add a device_type which, in turn, requires manufacturer etc.

netbox-docker should let us pull the latest minor release by just specifying the major release number, e.g. v2.9 will pull the v2.9.11 image. At least, I'm pretty sure. I think I'd rather manually specify which releases we're testing against. That way when a new version comes out we don't have the potential to randomly start failing tests. It's nice to know, but probably frustrating on a PR that deals with something else entirely.

Ah yeah great point I totally understand that. So then right now we would hard code the short version 2.8, 2.9, and 2.10 only and let the containers spin up to whatever the latest patch version is automatically correct? That works just fine as I'm only testing the major/minor version at the moment anyway because of the version reporting pre-2.10.

Add a pytest flag to disable all integrations tests? Should the default be enabled or disabled?

Could just provide a more specific path to the unit tests if you wanted to bypass, e.g. pytest tests/unit.

Ah I meant mostly as a way to single out and run only all of the fast-running unit tests to make sure your code at least has no major holes in it before running a much longer (comparatively - there are no tests yet but I'm sure the integration suite will just get longer and longer as time goes on) integration test. It's been handy to me a few times in other repos but that may just be my style, if you don't see a need than that's good with me :)

Particularly curious about how we can best handle dependencies. Like how testing devices requires us to add a device_type which, in turn, requires manufacturer etc.

Yes! I wanted to bring up the dependency issue myself as well once you had looked this over. There is the https://github.com/netbox-community/devicetype-library repo that we could import a few device types from to test with after the containers are spun up.. manufacturers are literally just a name so those can be read from the yaml config for each of the device types we would load as well and populated at the same time. And we could pin a version of that repo to use as well (bumping occasionally) so that no failures are introduced spontaneously from external updates. This solves for servers and switches, but I haven't seen any sort of patch panels included in the repo yet.. I may be wrong though. We could just send up a PR to add a few of those there or perhaps keep a few yaml models in this repo.. we wouldn't need many. Now that I think about it they may not allow patch panels in that repo because of how much of a gray area  they are implementation-wise - one setup may not have the same needs as another.
Thank you for checking this out, I'll go ahead and get started on the points you OK'd above.

So then right now we would hard code the short version 2.8, 2.9, and 2.10 only and let the containers spin up to whatever the latest patch version is automatically correct?

Yep, I think the API only makes breaking changes between major releases. If not, we can revisit pinning to minor releases.

Ah I meant mostly as a way to single out and run only all of the fast-running unit tests to make sure your code at least has no major holes in it before running a much longer (comparatively - there are no tests yet but I'm sure the integration suite will just get longer and longer as time goes on) integration test. It's been handy to me a few times in other repos but that may just be my style, if you don't see a need than that's good with me :)

I follow. I was just saying we can do that now if we provide more specific path to pytest. Though running the old fixture-based tests is a little tough going that route, but those are hopefully on the way out anyways. :)

Yes! I wanted to bring up the dependency issue myself as well once you had looked this over. There is the https://github.com/netbox-community/devicetype-library repo that we could import a few device types from to test with after the containers are spun up.. manufacturers are literally just a name so those can be read from the yaml config for each of the device types we would load as well and populated at the same time. And we could pin a version of that repo to use as well (bumping occasionally) so that no failures are introduced spontaneously from external updates. This solves for servers and switches, but I haven't seen any sort of patch panels included in the repo yet.. I may be wrong though. We could just send up a PR to add a few of those there or perhaps keep a few yaml models in this repo.. we wouldn't need many. Now that I think about it they may not allow patch panels in that repo because of how much of a gray area they are implementation-wise - one setup may not have the same needs as another.

Yeah, think I like that approach. We should still test adding a device-type from the API, but it'll probably get old having to make all the ones we'll need for some of the interface and cable tests.

So then right now we would hard code the short version 2.8, 2.9, and 2.10 only and let the containers spin up to whatever the latest patch version is automatically correct?

Yep, I think the API only makes breaking changes between major releases. If not, we can revisit pinning to minor releases.

Ah I meant mostly as a way to single out and run only all of the fast-running unit tests to make sure your code at least has no major holes in it before running a much longer (comparatively - there are no tests yet but I'm sure the integration suite will just get longer and longer as time goes on) integration test. It's been handy to me a few times in other repos but that may just be my style, if you don't see a need than that's good with me :)

I follow. I was just saying we can do that now if we provide more specific path to pytest. Though running the old fixture-based tests is a little tough going that route, but those are hopefully on the way out anyways. :)

Yes! I wanted to bring up the dependency issue myself as well once you had looked this over. There is the https://github.com/netbox-community/devicetype-library repo that we could import a few device types from to test with after the containers are spun up.. manufacturers are literally just a name so those can be read from the yaml config for each of the device types we would load as well and populated at the same time. And we could pin a version of that repo to use as well (bumping occasionally) so that no failures are introduced spontaneously from external updates. This solves for servers and switches, but I haven't seen any sort of patch panels included in the repo yet.. I may be wrong though. We could just send up a PR to add a few of those there or perhaps keep a few yaml models in this repo.. we wouldn't need many. Now that I think about it they may not allow patch panels in that repo because of how much of a gray area they are implementation-wise - one setup may not have the same needs as another.

Yeah, think I like that approach. We should still test adding a device-type from the API, but it'll probably get old having to make all the ones we'll need for some of the interface and cable tests.

Gotcha, sounds good then. I have most of the changes done locally now I just need a little more time to clean it up before sending. After that set goes in I'll add a basic setup of the devicetype library integration as well to get some initial devices populated in the containers.

Gotcha, sounds good then. I have most of the changes done locally now I just need a little more time to clean it up before sending. After that set goes in I'll add a basic setup of the devicetype library integration as well to get some initial devices populated in the containers.

Basic pytest args should work as expected now.. currently netbox 2.8 does not spin up correctly though via netbox-docker. I'll have to look in to that if you want to test that far back. I also added a comment about the container  cleanup hacks only being needed until https://github.com/avast/pytest-docker/pull/33 gets resolved.

Basic pytest args should work as expected now.. currently netbox 2.8 does not spin up correctly though via netbox-docker. I'll have to look in to that if you want to test that far back. I also added a comment about the container  cleanup hacks only being needed until https://github.com/avast/pytest-docker/pull/33 gets resolved.

Nice, looks good!

currently netbox 2.8 does not spin up correctly though via netbox-docker

hrm, what's it say? Was able to find a tag for it on docker-hub. I feel like we probably should test at least that far back. 2.9 is only ~6mo old. Once this is settled though, we probably should come up with some kind of policy around minimum supported version.

Nice, looks good!

currently netbox 2.8 does not spin up correctly though via netbox-docker

hrm, what's it say? Was able to find a tag for it on docker-hub. I feel like we probably should test at least that far back. 2.9 is only ~6mo old. Once this is settled though, we probably should come up with some kind of policy around minimum supported version.

It's an error in the "netbox" container when running the setup so I assumed it was an issue in the 2.8 tagged image. I'll dig a bit. Last lines:
▶ Running the startup script /opt/netbox/startup_scripts/240_virtualization_interfaces.py
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/shell.py", line 92, in handle
    exec(sys.stdin.read())
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/runpy.py", line 280, in run_path
    run_name, mod_spec, pkg_name).copy()
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "../startup_scripts/__main__.py", line 25, in <module>
    runpy.run_path(f.path)
  File "/usr/local/lib/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/local/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/netbox/startup_scripts/240_virtualization_interfaces.py", line 4, in <module>
    from virtualization.models import VirtualMachine, VMInterface
ImportError: cannot import name 'VMInterface' from 'virtualization.models' (/opt/netbox/netbox/virtualization/models.py)

It's an error in the "netbox" container when running the setup so I assumed it was an issue in the 2.8 tagged image. I'll dig a bit. Last lines:
▶ Running the startup script /opt/netbox/startup_scripts/240_virtualization_interfaces.py
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/shell.py", line 92, in handle
    exec(sys.stdin.read())
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/runpy.py", line 280, in run_path
    run_name, mod_spec, pkg_name).copy()
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "../startup_scripts/__main__.py", line 25, in <module>
    runpy.run_path(f.path)
  File "/usr/local/lib/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/local/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/netbox/startup_scripts/240_virtualization_interfaces.py", line 4, in <module>
    from virtualization.models import VirtualMachine, VMInterface
ImportError: cannot import name 'VMInterface' from 'virtualization.models' (/opt/netbox/netbox/virtualization/models.py)

Ah yeah, getting the same thing. VMInterface wasn't added until 2.9, and I couldn't find where netbox-docker says for how long they maintain backwards compatibility with their initializers. Guess that settles how far back we test. :rofl:

I asked how far back they want to test in the NTC slack.. if I can't get an anser there I'll open an issue and ask directly in the repo. First iteration of defice imports works now.. let me know what you think. I added a basic test as well.

I asked how far back they want to test in the NTC slack.. if I can't get an anser there I'll open an issue and ask directly in the repo. First iteration of defice imports works now.. let me know what you think. I added a basic test as well.

There are many options for how to set up testing, i only guessed for the test i added in the last change. If you want to change it feel free

There are many options for how to set up testing, i only guessed for the test i added in the last change. If you want to change it feel free

Oh man I messed up that rebase..will fix.

Oh man I messed up that rebase..will fix.

OK.. from talking in the NTC channel one of the maintainers wrote this in response to my post:

it would be nicer if we could add logic to support older versions from the latest snapshot though.

This would add too much overhead on our (the maintainers) side, so this is unlikely to happen (unless someone starts to pay us for this perhaps :wink: )
But you can check out the relevant version of the repository (the release notes state which versions of Netbox Docker work with which version of Netbox) and re-build the container to get updated dependencies. See https://github.com/netbox-community/netbox-docker/wiki/Build

How would you feel about checking out multiple copies (different tags) of the netbox-docker repo so we could run as many versions as we want? It would need a little more code and maintenance but I think the flow would be fairly simple still. Just a thought.

OK.. from talking in the NTC channel one of the maintainers wrote this in response to my post:

it would be nicer if we could add logic to support older versions from the latest snapshot though.

This would add too much overhead on our (the maintainers) side, so this is unlikely to happen (unless someone starts to pay us for this perhaps :wink: )
But you can check out the relevant version of the repository (the release notes state which versions of Netbox Docker work with which version of Netbox) and re-build the container to get updated dependencies. See https://github.com/netbox-community/netbox-docker/wiki/Build

How would you feel about checking out multiple copies (different tags) of the netbox-docker repo so we could run as many versions as we want? It would need a little more code and maintenance but I think the flow would be fairly simple still. Just a thought.

How would you feel about checking out multiple copies (different tags) of the netbox-docker repo so we could run as many versions as we want? It would need a little more code and maintenance but I think the flow would be fairly simple still. Just a thought.

Yeah, saw the reply in the NTC slack. That makes sense. We'll probably need to map repo & NetBox version together at some point anyways. So yeah, we could use it, but I don't think we have to have it immediately.

How would you feel about checking out multiple copies (different tags) of the netbox-docker repo so we could run as many versions as we want? It would need a little more code and maintenance but I think the flow would be fairly simple still. Just a thought.

Yeah, saw the reply in the NTC slack. That makes sense. We'll probably need to map repo & NetBox version together at some point anyways. So yeah, we could use it, but I don't think we have to have it immediately.

Woohoo! Well I had some time and decided to just do it. Checking it out side by side works with a few minor tweaks. The test run time is up to about 4 minutes though, maybe we should bump the max timeout pytest-docker will wait for the containers to spin up from 5min to maybe 8?

Woohoo! Well I had some time and decided to just do it. Checking it out side by side works with a few minor tweaks. The test run time is up to about 4 minutes though, maybe we should bump the max timeout pytest-docker will wait for the containers to spin up from 5min to maybe 8?

Woohoo! Well I had some time and decided to just do it. Checking it out side by side works with a few minor tweaks. The test run time is up to about 4 minutes though, maybe we should bump the max timeout pytest-docker will wait for the containers to spin up from 5min to maybe 8?

Oh snap, nice work! Yeah, bumping the timeout makes sense. I wonder if, ultimately, we'd be better off putting these in their own workflow :thinking:. Anything specific you want help on?

Woohoo! Well I had some time and decided to just do it. Checking it out side by side works with a few minor tweaks. The test run time is up to about 4 minutes though, maybe we should bump the max timeout pytest-docker will wait for the containers to spin up from 5min to maybe 8?

Oh snap, nice work! Yeah, bumping the timeout makes sense. I wonder if, ultimately, we'd be better off putting these in their own workflow :thinking:. Anything specific you want help on?

Ah yeah I gotcha. To split out to its own workflow all we would need is some args to select only unit vs integration tests but that's doable, I've don it before. Right now I think it's fully functional.. I'd like your input on current implementation of the setup and actual tests and to know what style or content you want in the docstrings for args/returns, I'll check out the rest of the codebase for examples in a bit. I think it's just about ready now.
I'll bump the timeout for container generation as well for now.

Ah yeah I gotcha. To split out to its own workflow all we would need is some args to select only unit vs integration tests but that's doable, I've don it before. Right now I think it's fully functional.. I'd like your input on current implementation of the setup and actual tests and to know what style or content you want in the docstrings for args/returns, I'll check out the rest of the codebase for examples in a bit. I think it's just about ready now.
I'll bump the timeout for container generation as well for now.

Ah yeah I gotcha. To split out to its own workflow all we would need is some args to select only unit vs integration tests but that's doable, I've don it before.

Seems to work now with pytest tests/integration, don't think we need an arg, but it wouldn't hurt.

Right now I think it's fully functional.. I'd like your input on current implementation of the setup and actual tests and to know what style or content you want in the docstrings for args/returns, I'll check out the rest of the codebase for examples in a bit. I think it's just about ready now.
I'll bump the timeout for container generation as well for now.

Ok, yeah, made a quick pass at it. I might have to stew on the functional approach you have in test_dcim a little more. My fear is that it'll quickly get out of hand with a multitude of tests that do a lot of the same thing (e.g. making devices), and it might be difficult to tell exactly what's under test. OTOH, we might run into the same problem either way.

Ah yeah I gotcha. To split out to its own workflow all we would need is some args to select only unit vs integration tests but that's doable, I've don it before.

Seems to work now with pytest tests/integration, don't think we need an arg, but it wouldn't hurt.

Right now I think it's fully functional.. I'd like your input on current implementation of the setup and actual tests and to know what style or content you want in the docstrings for args/returns, I'll check out the rest of the codebase for examples in a bit. I think it's just about ready now.
I'll bump the timeout for container generation as well for now.

Ok, yeah, made a quick pass at it. I might have to stew on the functional approach you have in test_dcim a little more. My fear is that it'll quickly get out of hand with a multitude of tests that do a lot of the same thing (e.g. making devices), and it might be difficult to tell exactly what's under test. OTOH, we might run into the same problem either way.

Seems to work now with pytest tests/integration, don't think we need an arg, but it wouldn't hurt.

Can you select only unit tests currently though? They are pretty spread out, unless you do a glob match of pytest tests/test_* tests/unit, that's totally fine as long as you just expect the normal behaviour of running pytest to just run all tests.

My fear is that it'll quickly get out of hand with a multitude of tests that do a lot of the same thing (e.g. making devices), and it might be difficult to tell exactly what's under test. OTOH, we might run into the same problem either way.

Yeah, I worry about it also. I had just left a comment about using dependencies via pytest-depends as an alternative, you just have to be careful about not modifying the items between the dependent tests. Maybe tests that would modify items would use classes with dedicated objects and tests that only need to read objects could use commonly created objects via dependencies?

Seems to work now with pytest tests/integration, don't think we need an arg, but it wouldn't hurt.

Can you select only unit tests currently though? They are pretty spread out, unless you do a glob match of pytest tests/test_* tests/unit, that's totally fine as long as you just expect the normal behaviour of running pytest to just run all tests.

My fear is that it'll quickly get out of hand with a multitude of tests that do a lot of the same thing (e.g. making devices), and it might be difficult to tell exactly what's under test. OTOH, we might run into the same problem either way.

Yeah, I worry about it also. I had just left a comment about using dependencies via pytest-depends as an alternative, you just have to be careful about not modifying the items between the dependent tests. Maybe tests that would modify items would use classes with dedicated objects and tests that only need to read objects could use commonly created objects via dependencies?

Can you select only unit tests currently though? They are pretty spread out, unless you do a glob match of pytest tests/test_* tests/unit, that's totally fine as long as you just expect the normal behaviour of running pytest to just run all tests.

I think that's fair, and yeah, it'd depends on deprecating the fixture based tests for these so that eventually we only have unit and integration in tests.

Can you select only unit tests currently though? They are pretty spread out, unless you do a glob match of pytest tests/test_* tests/unit, that's totally fine as long as you just expect the normal behaviour of running pytest to just run all tests.

I think that's fair, and yeah, it'd depends on deprecating the fixture based tests for these so that eventually we only have unit and integration in tests.

Hey I hope things are going well :) Is there anything else you need done on this at this time or are you just holding on it for testing? I only ask because I should have some spare time this weekend to put in if needed. Thanks

Hey I hope things are going well :) Is there anything else you need done on this at this time or are you just holding on it for testing? I only ask because I should have some spare time this weekend to put in if needed. Thanks

:wave: I don't think so, think it's a great foundation. I've been meaning to spend some time on it to see if we could generate tests similar to how we do with the fixture-based ones (i.e. test each endpoint with a CRUD request). Wanted to do that today, but it doesn't look like that's going to happen. I'll see if I can't take a swing at that over the weekend or sometime next week.

:wave: I don't think so, think it's a great foundation. I've been meaning to spend some time on it to see if we could generate tests similar to how we do with the fixture-based ones (i.e. test each endpoint with a CRUD request). Wanted to do that today, but it doesn't look like that's going to happen. I'll see if I can't take a swing at that over the weekend or sometime next week.

Oh no rush, I was just checking in. Don't rush it on my part as I won't be able to extend the much in the next few weeks anyhow. Sounds good, if I can help set up a few tests closer to how you want just let me know. Have a good weekend!

Oh no rush, I was just checking in. Don't rush it on my part as I won't be able to extend the much in the next few weeks anyhow. Sounds good, if I can help set up a few tests closer to how you want just let me know. Have a good weekend!

Didn't make it very far into my idea before realizing it's probably too involved to get done on this PR. I'll CC you when I get a chance finish it out.
Many thanks for tackling this!

Didn't make it very far into my idea before realizing it's probably too involved to get done on this PR. I'll CC you when I get a chance finish it out.
Many thanks for tackling this!

Ah right on, no worries thank you for the assist! So just to confirm, I should wait to actually build any tests until you get a solid framework pattern going right?

Ah right on, no worries thank you for the assist! So just to confirm, I should wait to actually build any tests until you get a solid framework pattern going right?

Probably, just so we're not duplicating effort, but it might be a few weeks before I can get to it. The idea was that we could basically setup session-scoped fixtures of anything that would get used more than once (e.g. Site, rack(s), devices, etc.). Then we'd go through each endpoint methodically by running create, retrieve, update, and finally delete operations on each. Obviously for stuff we've created in the fixtures we wouldn't use another create on, but that also means coordinating our tests so that we deleted things in the right order. There's a few implementation details like that I still need to get into, but that's the gist. I just figured a more methodical approach would mean better coverage and fewer redundancies when we did identify gaps. You're a lot more familiar with it than I am though; so does that make sense?

So essentially there would be thousands of fixtures then right? Like server_device_connected_to_x_type_switches? That would work but it might just get hard to manage on the long run. From my testing of our use cases in netbox I think there are so many edge cases it may be easier to have isolated spin up/spin down tests for those but honestly I'm not sure and it would be great to just see where this goes. It's not like it would be super hard to change in a few months if it needs tweaking, and any tests are better than no tests so run with it and when you have a setup you like I'll start adding to it :) Thanks and have a good week!

So essentially there would be thousands of fixtures then right?

:grimacing:  I hope not, maybe at most a few dozen. Not looking to test each device-type, just a more fundamental test of whether we're interacting with the endpoint properly. A cursory example would be where we create a couple devices as session-scope fixtures. Then when we got to say the cables test, we'd just create one between the two existing devices, retrieve it and update it with something, then delete it when we're done.

Right on :) Sounds good to me!


Generate formulas automatically from the component graph

Closes #102

For me looks good.
Waiting for more formulas :)

I will add some more tests as well, to this PR.

This is ready for review again.

Ah, this looks very nice with the new exceptions and one class per formula :heart:


pkg/validator: target value is not empty by default

The consistency make the code more predictable.
We want all the values to be non-empty (commit, kernelPath, etc.).
If "empty" is an option - use validator.AnyOk(validator.EmptyStr(target), otherCheck(target)).

@dvyukov please review


[#24789] Spot fix fullvalue wrapping for SDF.

Discovered there's an inconsistency with how FullValue wrappings occur between SDF phases and DataSource decoding. Undiscovered due to fusion hiding the inconsistency.
Discovered while adding SDF handling to the replacement runner tracked in https://github.com/apache/beam/issues/24789.
Ideally, there would be no such distinction, but would entail a major consistency cleanup in the execution layer. For now, we wrap if needed, but cleaning up the inconsistency would improve SDK throughput.
This is a re-do of #24790 since that pr was targeted incorrectly.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @damccorm


[Frontend] Add compatibility in the frontend for macOS and Linux.

Context: There are some small differences between macOS and Linux in the frontend.
Description of the Change: Use .dylib instead of .so. Split -Wl,rpath across multiple options. Remove --no-as-needed as it is the default behaviour on linux and unavailable on macOS.
Benefits: Portability.

[sc-43392]


Vertical Federated Learning RFC

Motivation
XGBoost 1.7.0 introduced the initial support for Federated Learning. However, only horizontal federated learning is supported. Training samples are assumed to be split horizontally, i.e. each participant has a subset of samples, but with all the features and labels. In many real world applications, data is split vertically: each participant has all the samples, but only a partial list of features, and not all participants have access to the labels. It would be useful to support vertical federated learning in XGBoost.
Goals

Enhance XGBoost to support vertical federated learning.
Support using NVFlare to coordinate the learning process, but the design should be amenable to support other federated learning platforms.
Efficiency: training speed should be close to traditional distributed training.
Accuracy: should be close to centralized learning.

Non-Goals

Initially we will assume the federated environment is non-adversarial, and will not provide strong privacy guarantees. This will be improved upon in later iterations.
We will not support data owners dropping out during the learning process.

Assumptions
In vertical federated learning, before model training, the participants need to jointly compute the common IDs of their private sets. This is called private set intersection (PSI). For XGBoost, we assume this is already done, and users may rely on some other library/framework for this step.
Similar to horizontal federated learning, we make some simplifying assumptions::

A few trusted partners jointly train a model.
Reasonably fast network connection between each participant and a central trusted party.

Risks
The current XGBoost codebase is fairly complicated and hard to modify. Some code refactoring needs to happen first, before support for vertical federated learning can be added. Care must be taken to not break existing functionality, or make regular training harder.
Design
LightGBM, a gradient boosting library similar to XGBoost, supports “feature parallel” distributed learning.

Conceptually, feature parallelism is similar to vertical federated learning. A possible design is to first enhance XGBoost distributed training to support feature parallelism, and then build vertical federated learning on top of it. This would benefit the wider user community, thus greatly reducing the risks involved in refactoring XGBoost’s code base.
Feature Parallelism
XGBoost has an internal training parameter called DataSplitMode, which can be set to auto, col, and row. However, it’s currently not exposed to the end user, and can only be set to row for distributed training.
In order to support column-based data split for distributed training, we need to do the following:

When initially loading data, support splitting by column. To keep it simple, we can have all workers keep a copy of the labels (and other things like weight and qid, effectively the MetaInfo object). The resulting DMatrix needs to keep track of which features belong to which worker.
When generating the prediction, participants need to work collaboratively: the worker owning the feature used to split a node needs to collect the left and right splits and broadcast the results. A naive implementation may incur too much communication overhead, but there is prior work to encode partial predictions in bitsets to make the process more efficient (see paper).
The worker owning the label should calculate the gradients and broadcast them to other workers.
When finding the best split, each worker finds the best local split based on the features it owns, and then performs an allreduce to find the global best split. Workers don’t need to access histogram from each other. The worker owning the feature for the best split then broadcasts the split results to others.

We may also want to consider implementing LightGBM’s voting parallel approach (paper) for more efficient communication.
Vertical Federated Learning
Assuming feature parallelism is implemented, vertical federated learning is a slight modification:

When loading data, no need to split the columns further, since we assume each worker only has a subset of the features.
We can no longer share labels between workers.
Communication needs to switch to the federated communicator.

Federated Inference
In horizontal federated learning, since each participant has all the features and labels, trained models can be shared to run inference/prediction locally. In vertical federated learning, however, this is no longer feasible. All the participants need to be online and work collaboratively to run inference. For batch inference, a federated learning job can be set up (for example, using NVFlare) to produce predictions. For online inference, we would need to set up services at participating sites to jointly produce online predictions, which is out of the scope of this design.
Alternatives Considered
It may be possible to implement vertical federated learning without first adding support for column data split mode in distributed training. However, since this would require extensive refactoring of the XGBoost code base without producing any benefits for users not using federated learning, it may be too risky. For very wide datasets (relatively many features with moderate number of rows), column data split may be a useful feature for distributed training.

Thank you for the detailed RFC! I'm sure there are lots of excitement coming. I tagged the RFC as part of 2.0. Let me know if there's anything I can help.

Adding a task list to keep track of the progress:

[ ] Column-wise data splitting
[ ] Column-wise data splitting for external memory (this seems to require extra work)
[ ] Prediction with column-wise split data
[ ] Gradient/hessian calculation with column-wise split data
[ ] Split finding with column-wise split data
[ ] Vertical federated learning (needs further breakdown)

hello, Mr.rongou  @rongou . I have much experience on the  implementation vertical federated XGB/LGB. But my former version was coded by python and scala. I was trying to implement it based on lightgbm source codes, also added some c_api for python wrapper, but this work is very hard for me because of poor c++ coding experience. So how can I contribute my experience with you or your group to implement this work based on xgb src?

Hi @peiji1981 , thank you for the offer! Do you have a paper or document on your work?


'k8s' key updated multiple times in policy package

terrascan version: 1.2.0
Operating System: macOS

Description
While registering the cloud provider, key 'k8s' is updated multiple times resulting in kustomize as iac type for cloudType=k8s
What I Did
Found this while debugging the code

This issue no longer exist. This issue was resolved in #576


Improved license error content

Table of Contents
Link to GitHub issues it solves
closes #29
Description
The error shown on failure was improved
Contributing has more information and tips for a great pull request.

@zeobot set lc to pass


Create reference for all available functions

So far, all of the functions one can use to change key behaviour (around, tap-hold-next-release, ...) are mentioned in the tutorial.  However, it may be nice to have a sort "quick reference guide" for all functions and what they do.  This would improve discoverability considerably I think.

I think making them into a man page would be very useful. However, I think it won't be so convenient for Windows users!

I like the idea, but instead of a man page the central point of this documentation should be on Github somewhere. Then, if someone wants to turn this into a man page, be my guest.

I think this issue should be closed after b0d4bec44be459f8c7b052fe7e9a6bd09413ce34 and 22a79f4e3658fbe3f672e3696a021104afaac221.


Update script to audit multiple Spark versions

This fixes https://github.com/NVIDIA/spark-rapids/issues/465.
Added profile to run against different versions.

I think reviewers were auto added as there was a change in pom file in api_validation repo.
@tgravescs - It would be great if you could take a look and comment if this approach if fine.

build

Commits need to have signoff.

build

build


Add Logging around sent notifications in the API

Motivation
There are no log statements when we emit notifications from the API.
Scope of change

AllNotifiers - wrap the individual notify IO with a monitor function

final case class AllNotifiers(...) extends Monitored {
...
  def notify(notifier: Notifier, notification: Notification[_]): IO[Unit] =
    monitor(notifier.getClass.getSimpleName) {
      notifier.notify(notification).handleErrorWith { e =>
        IO { logger.error(s"Notifier ${notifier.getClass.getSimpleName} failed.", e) }
      }
  }
...

Hi! This looks like you have a fairly solid understanding of what you need done here. Mind if I take this one?

Go for it @ericcheatham, let us know if you have any questions!


There is no remote client ip information on UvTcpConnection #2202

Hi @arontsang - we're currently not taking any changes to the corefxlab release/2.1 branch as that branch was only used (temporarily) for building and releasing System.IO.Pipelines  for .NET Core 2.1 Preview 1 only.
I'm closing this PR against corefxlab release/2.1, but we'd be happy to review other PRs to the corefxlab master branch, instead.
Thanks and sorry about the branch confusion.


Handle extensions

Implement the mechanism described in RFC 8285.  Support only a one-byte header, as according to RFC this form is preferred and it must be supported by all receivers.

I think we should also update VAD to reflect these changes


CPU mode?

Hi is there a CPU mode only flag available?

@Sugarbank Thanks for your interests about our work.
Yes, we provide the option for running in CPU mode:
th test_wct.lua -gpu -1

Could someone provide model snapshots for CPU? (or can they be converted without cuda support?

@jjmontesl
Hi, do you mean that you cannot load my model trained under GPU? Did you install cudnn?

Hello. Thanks for responding. I don't have a Cuda GPU, hence I cannot
install cudnn and wanted to run from CPU.
When loading models, torch complains because the models are serialized
including Cuda classes that are not available in my runtime. The issue
seems well known (https://groups.google.com/forum/#!topic/torch7/ugBCwaoXw_s
).
I am entirely new to Torch and maybe this has an easy solution, but I have
since tried the Tensorflow implementation you referenced which doesn't
raise this issue.
Best,

Hi guys...seems with cpu flag it's still requiring cudnn.....
module 'cudnn' not found:No LuaRocks module found for cudnn
no field package.preload['cudnn']
no file '/Users/sugarbank/.luarocks/share/lua/5.1/cudnn.lua'
no file '/Users/sugarbank/.luarocks/share/lua/5.1/cudnn/init.lua'
no file '/Library/StylaRender/Deployment/torch/install/share/lua/5.1/cudnn.lua'
no file '/Library/StylaRender/Deployment/torch/install/share/lua/5.1/cudnn/init.lua'
no file './cudnn.lua'
no file '/Library/StylaRender/Deployment/torch/install/share/luajit-2.1.0-beta1/cudnn.lua'
no file '/usr/local/share/lua/5.1/cudnn.lua'
no file '/usr/local/share/lua/5.1/cudnn/init.lua'
no file '/Users/sugarbank/.luarocks/lib/lua/5.1/cudnn.so'
no file '/Library/StylaRender/Deployment/torch/install/lib/lua/5.1/cudnn.so'
no file '/Library/StylaRender/Deployment/torch/install/lib/cudnn.dylib'
no file './cudnn.so'
no file '/usr/local/lib/lua/5.1/cudnn.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'error'
...er/Deployment/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
test_wct.lua:9: in main chunk
[C]: in function 'dofile'
...ment/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x010eec09c0
Any ideas?

@Sugarbank Sorry for the late reply. Since I use the GPU during the training, the decoders require the cudnn for the loading.
I need to explicitly transfer the models to float so that you can load in CPU model. Please find them here.

Hi guys, still getting a cudnn error when using the CPU models....any ideas?
module 'cudnn' not found:No LuaRocks module found for cudnn
no field package.preload['cudnn']
no file '/Users/sugarbank/.luarocks/share/lua/5.1/cudnn.lua'
no file '/Users/sugarbank/.luarocks/share/lua/5.1/cudnn/init.lua'
no file '/Library/StylaRender/Deployment/torch/install/share/lua/5.1/cudnn.lua'
no file '/Library/StylaRender/Deployment/torch/install/share/lua/5.1/cudnn/init.lua'
no file './cudnn.lua'
no file '/Library/StylaRender/Deployment/torch/install/share/luajit-2.1.0-beta1/cudnn.lua'
no file '/usr/local/share/lua/5.1/cudnn.lua'
no file '/usr/local/share/lua/5.1/cudnn/init.lua'
no file '/Users/sugarbank/.luarocks/lib/lua/5.1/cudnn.so'
no file '/Library/StylaRender/Deployment/torch/install/lib/lua/5.1/cudnn.so'
no file '/Library/StylaRender/Deployment/torch/install/lib/cudnn.dylib'
no file './cudnn.so'
no file '/usr/local/lib/lua/5.1/cudnn.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'error'
...er/Deployment/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
test_wct.lua:9: in main chunk
[C]: in function 'dofile'
...ment/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x010a7969c0


Image repository: check operator for harbor

@DaOsie can you please comment here if you were researching something in here?
@toszo @pprach can you please tell me if that task should be in this repo?

@mkyc Yes, this one as well should go to Build System repo.


Oxford 2016 CI tutorial readiness

See what you think, @jchiang87 - not sure if the use cases are what you are looking for, but I littered the code with "BUG" markers for things to do better, and put in a bunch of possible failure modes in the docstrings to aid test writing. Notice how I am not employing test-driven development! You may want to branch off and reset things to be better for TDD? Not sure. Anyway, I hope this helps! For a complete working example, try
python python/desc/slcosmo/SLCosmo.py

You should get something like
Posterior mean H0 =  69.9996918047

Thanks, @drphilmarshall !  I just started going through this, but it looks like a lot of your BUGs are just right for doing the TDD I had in mind.  I'll study things some more and will make issues for each one.  I think we will have a nice handful of exercises for the tutorial.

Great! Have fun :-)


Include both symbol span and whole definition span for tsserver definitions

TypeScript Version:  3.4.0-dev.201xxxxx

Search Terms:

definition
definitionandboundspan

Problem
VS Code uses typescript's definition requests to render small previews of a declaration when you hover over a symbol and hold cmd. However because TS currently only returns the symbol span with definitions, we cannot always guess the range to preview correctly (see https://github.com/Microsoft/vscode/issues/72017)
For example, for the code:
function foo() {
	// noop
}

foo()

the definitionAndBoundSpan result for the invocation of foo is:
[Trace  - 11:35:52 AM] Sending request: definitionAndBoundSpan (12). Response expected: yes. Current queue length: 0
Arguments: {
    "file": "untitled:^Untitled-1",
    "line": 5,
    "offset": 3
}
[Trace  - 11:35:52 AM] Response received: definitionAndBoundSpan (12). Request took 25 ms. Success: true
Result: {
    "definitions": [
        {
            "file": "untitled:^Untitled-1",
            "start": {
                "line": 1,
                "offset": 10
            },
            "end": {
                "line": 1,
                "offset": 13
            }
        }
    ],
    "textSpan": {
        "start": {
            "line": 5,
            "offset": 1
        },
        "end": {
            "line": 5,
            "offset": 4
        }
    }
}

To make our UI more reliable, we need both the symbol span (covering just foo in the line function foo() {) as well as entire declaration span (covering all of the function declaration)
Proposed api
Keep the existing behavior of definitionAndBoundSpan, but tack on additional definition start/end information to each returned span:
    interface DefinitionInfoAndBoundSpan {
        definitions: ReadonlyArray<FileSpan & { definitionStart?: Location, definitionEnd?: Location }>;
        textSpan: TextSpan;
    }

This should be a backwards compatible change
Related Issues:

#8808

Which one is which in this API? Does the original FileSpan stay the same?

Yes, the existing FileSpan data would remain unchanced and track the symbol range. The added definitionStart and definitionEnd properties would track the range of the whole definition

@mjbvz Does this mean we need to add this to everything related to defintions.
Eg.
Definition = "definition",
        /* @internal */
        DefinitionFull = "definition-full",
        DefinitionAndBoundSpan = "definitionAndBoundSpan",
        /* @internal */
        DefinitionAndBoundSpanFull = "definitionAndBoundSpan-full",
        Implementation = "implementation",
        /* @internal */
        ImplementationFull = "implementation-full",
       TypeDefinition = "typeDefinition",
        References = "references",
        /* @internal */
        ReferencesFull = "references-full",

Ideally yes. I believe that VS Code can use this information in our UI for implementations and type definitions as well.
The definition one is the most important to us since it is the most noticeable in the UI

@mjbvz Few more question.
Do we want this only for funciton or any declaration name?
Do we want to call this contextSpan instead of definitionSpan?
Eg. for import { x as B } from "a" I think the whole declaration makes sense rather than sending you span for complete import declaration rather than just the span for x as B.

Yes, we would like this for all declarations including: variables, classes, and interfaces.
I agree that definitionSpan may not be the clearest name. I find contextSpan is a little vague too but would be fine with that. Maybe declarationSpan?

Your proposal about the handling of the import also makes sense to me


Fix dict default values for integers.

Fix dict default values for integers
Description
We cannot provide a default (zero) implementation for every possible sierra type. This PR implements a default for integer types which are known to be valid for dicts. Since we don't know exactly which Sierra types are valid for dict values we cannot be sure that we're currently supporting all of them.
Checklist

[ ] Linked to Github Issue
[ ] Unit tests added
[ ] Integration tests added.
[ ] This change requires new documentation.

[ ] Documentation has been added/updated.

Codecov Report
Attention: 25 lines in your changes are missing coverage. Please review.

Comparison is base (1c61305) 77.51% compared to head (431a788) 77.48%.

Files
Patch %
Lines

src/types.rs
59.01%
25 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #435      +/-   ##
==========================================
- Coverage   77.51%   77.48%   -0.04%
==========================================
  Files         109      109
  Lines       31098    31166      +68
==========================================
+ Hits        24106    24149      +43
- Misses       6992     7017      +25

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Update .travis.yml

Add markdown link check

ugh. this isn't worth it.
doesn't handle relative links or jekyll config like {{ site.links.twitter }}


Added Sort and Filter to the API layer and to the UI

Created the backend logic to allow sorting and filtering by Reward on the backend.
Changes are mainly located in the packages/react-app/src/pages/api/bounties/index.ts file. I decided to split Filter and Sort functionality, so that future maintainers can manage them independently.
You can test the changes using postman or by hitting the api directly, some sample queries:
# status Open, no text filtering, at least 1000 bank reward in descending order
http://localhost:3000/api/bounties?status=Open&search&gte=1000&sortBy=reward&asc=0

# status Completed, no text filtering, less than 1000 bank reward
http://localhost:3000/api/bounties?status=Completed&search&lte=1000

# status all, no text filtering, at least 1000 bank reward and up to 4000 bank
http://localhost:3000/api/bounties?status=&search&gte=1000&lte=4000&sortBy=reward

Changelog:

Added sample queries to the mongo/bounties/bounty-queries.js
Created type declaration files for accepted query filters and sorts
handleFilters function was getting very large by the end, so I split that up into separate functions.
Added less than or equal to, greater than or equal to and sort functionality to the query

Just as a comment, text search works but if seeding the DB you need to create an index. I should add that to the onboarding documentation.

Added the React changes, you can now:

Sort by reward only
Sort ascending and descending
Sort by a range

Using the UI
@amaredeus the latest 2 commits contain the UI changes, building on the initial API changes


fix: Fix docker rmi in CI workflow on main

docker rmi failed on main
The images were pushed successfully but the docker rmi $(...) command failed because no images where found by $(). It is either that the image is no longer there (but why?) or the expression $(...) is incorrect (but why since it works locally?).

> nx run openchallenges-mariadb:publish-and-remove-image

"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.

Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]

Remove one or more images

 >  NX   ERROR: Something went wrong in run-commands - Command failed: docker rmi $(docker images --filter=reference="ghcr.io/sage-bionetworks/openchallenges-mariadb:*" --quiet) --force

Docker version on GH runner
Run docker --version
Docker version 20.10.25+azure-2, build b82b9f3a0e763304a250531cb9350aa6d93723c9


Generate test coverage data

Shouldn't be too hard, and the coverage should be high!

Fixed by #10


troubles with java.time.LocalDate

How can I do to solve this problem:
could not find implicit value for parameter rw: fabric.rw.RW[java.time.LocalDate]

Can you show the code that's firing this?

thank you !
import com.outr.arango.{Document, DocumentModel, Id, Index}
import com.outr.arango.Field
import com.outr.arango.DocumentRef
import fabric.rw._
import java.time.LocalDate
import shared.mytypes.{ItemRecipe, Recipe}

case class RecipeScarango (
                            folio: String = "",
                            date: LocalDate = LocalDate.now(),
                            _id: Id[RecipeScarango] = RecipeScarango.id()
) extends Document[RecipeScarango]

object RecipeScarango extends DocumentModel[RecipeScarango] {

    override implicit val rw: RW[RecipeScarango] = RW.gen

    override def indexes: List[Index] = Nil
    override val collectionName: String = "tblrecipe"
}

Your problem is that there's no built-in RW support for LocalDate. You can create your own if you like. However, I recommend changing date to Long and using System.currentTimeInMillis().  LocalDate is significantly heavier than Long.

Ok, thank you.


About the waymo percetion v1.4.0 dataset used in your work.

Hi,
Thanks for your excellent work! But the waymo percetion v1.4.0 dataset is too large, so could you explain the specific sequences used in your work for training and testing? Or have you used all the training, validation and testing datasets?
Thanks for your attention.

Thanks for your issue, we used sequences<PHONE_NUMBER>2863604198,<PHONE_NUMBER>939952000 and<PHONE_NUMBER>967750070 of validation dataset to test the model in this repo.
best

Hi, thanks for your response. But I met a problem again, that is "KeyError: "No object named 'MaskFormer' found in 'META_ARCH' registry!"". It seems that the MaskFormer isn't registered in the META_ARCH. The truth is that only the below models are registered. Did you meet the same question?

I think this could be helpful https://github.com/LightwheelAI/street-gaussians-ns/issues/1#issuecomment-2127463228

I think this could be helpful #1 (comment)

Oh, thanks for your help.

感谢您的问题，我们使用验证数据集的序列<PHONE_NUMBER>2863604198、2094681306939952000 和<PHONE_NUMBER>967750070 来测试此 repo 中的模型。 最好

感谢您的问题，我们使用验证数据集的序列<PHONE_NUMBER>2863604198、2094681306939952000 和<PHONE_NUMBER>967750070 来测试此 repo 中的模型。 最好
This link cannot download data

感谢您的问题，我们使用验证数据集的序列<PHONE_NUMBER>2863604198、2094681306939952000 和<PHONE_NUMBER>967750070 来测试此 repo 中的模型。 最好

感谢您的问题，我们使用验证数据集的序列<PHONE_NUMBER>2863604198、2094681306939952000 和<PHONE_NUMBER>967750070 来测试此 repo 中的模型。 最好

This link cannot download data


Add HTML reports to Cobertura

Coverage remained the same at 64.828% when pulling c9f6fc63d011e74bbd58c8e6c3aa4d0f98e90539 on cobertura-html-juha into f4e0f5a9567643af1f11326885672b38a7963941 on master.

Coverage remained the same at 76.471% when pulling 699c5d925c148a01e5d9b984a04fc01748213a2a on cobertura-html-juha into 69e07e19ad400b7ee60832e3fc045bc0e5b2ca59 on master.

LGTM.


WIP: Custom themes

Please review and test in windows
https://github.com/hydralauncher/hydra/assets/169095248/115fef74-5ca3-4c8d-946b-cb697fa21307

looks pretty cool! How do you even setup your computer to look like that?

looks pretty cool!

Thanks man. I use arch with hyprland, I can send my dotfiles if you want

looks pretty cool!

Thanks man. I use arch with hyprland, I can send my dotfiles if you want

Sure man!

looks pretty cool!

Thanks man. I use arch with hyprland, I can send my dotfiles if you want

please, also want it, its so cool! looks like a linux porn thread heheh

Example of error handling and opening folder


Unable to add repository

Under the latest HA version (2024.9.2) I am unable to add the repository.
I'm getting a generic "not a valid add-on repository" message and nothing in the HA logs from what I can find.
The screen no longer seems to ask for the repository type.
Am I doing anything wrong here?

Today I installed HA for the first time. And only later did I learn what HACS is. Now I know.

No problem at all.


Add infrastructure to enrich errors with hints

Followup for #1456.
Goals during implementation:

As little changes to existing code as possible, as few changes in signatures as possible

I did refactor some test code however - feel free to critique!

Incremental shifts are possible - any existing code must be easily enriched with hints without huge changes
Hints always belong to errors, and there can be as many as preferred
Hints are currently codespan_reporting labels, but this can theoretically be changed in the future
The typ-file based tests can be extended with expected hints and they fail if a hint is not emitted or not annotated

Hint validation (for un-annotated output) can be disabled to avoid cluttering duplicated hints in technical tests

I did evaluate some more complex options (like a new result type) but nothing was convenient, really. There are some drafts in my fork but I don't think they're worth looking at.
What can be done after this:

All "did you" messages can be converted into hints
Richer hint output in the cli could be an option, but this might require moving away from codespan_reporting or simply abusing it by printing an additoinal information output as hint, which will then be unrelated to the error, at least technically.

Thanks, looks good!


Update README.md

IBM Cloud toolchain: Delivery Pipeline deployed orders to nyu-devops, including commit 345c2daf3bdf5a093f00d7631aa4777aabb20697


There is some shivering while flipping in mozila

Here's a vertical image slide that flips smoothly in Chrome but vibrates most of the time in mozila. Is this a normal bug in Mozilla?
http://ibc-live.herokuapp.com/

Any follow up on this? I'm having the same issue. Works great in Chrome, Safari even Edge 😆  (the old one). But in firefox, swiper just shakes and shivers a lot.


Create _index.md

👍👍👍

Hi, Jack. How to see what I've created in the web browser? Just like you did in the tutorial video.
I saw on your browser address localhost:xxx something

@andrecahyadi you will have to follow the instructions in the readme.md file in this repository. Make sure you have git and hugo extended edition installed locally on your machine before you follow the repository instructions once you have set them up.
Then you can run a hugo server in your terminal and it should point a localhost adress at port 1313

@PardusEidolon Hi, Jack. I've managed to install Hugo extended in my laptop.
I'm able to run a sample website using command "hugo server" and open using web browser (localhost:1313).
I have installed git as well and still have no idea on how to install PostCSS and clone the github as mentioned in the readme.md, haha.
I'm using windows 11 64 bit

I also forgot to mention that you needed Node.js and Npm then you can run an npm install postcss-cli and npm install autoprefixer.
I also recommend that you get git lfs just incase, because sometimes the website doesn't recognise the images stored in the repository that you clone. If you run into an error like that running git lfs pull seems to do the trick otherwise throw up an issue and label it a bug and ill sort it out for you.


Add get storage at

Changes proposed in this pull request:

Added implementation for eth_getStorageAt (https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_getstorageat)

@gnosis/mobile-devs

Codecov Report

Merging #35 into master will increase coverage by 0.11%.
The diff coverage is 83.33%.

@@             Coverage Diff              @@
##             master      #35      +/-   ##
============================================
+ Coverage     62.21%   62.33%   +0.11%
  Complexity      152      152
============================================
  Files            54       54
  Lines          1334     1346      +12
  Branches        220      223       +3
============================================
+ Hits            830      839       +9
- Misses          404      406       +2
- Partials        100      101       +1

Impacted Files
Coverage Δ
Complexity Δ

...ava/pm/gnosis/ethereum/rpc/EthereumRpcConnector.kt
0% <ø> (ø)
0 <0> (ø)
:arrow_down:

...main/java/pm/gnosis/ethereum/EthereumRepository.kt
62.96% <0%> (-2.43%)
0 <0> (ø)

.../java/pm/gnosis/ethereum/rpc/models/RpcRequests.kt
87.8% <90%> (+0.3%)
0 <0> (ø)
:arrow_down:

...va/pm/gnosis/ethereum/rpc/RpcEthereumRepository.kt
97.08% <91.66%> (ø)
9 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7482fc9...bc7d850. Read the comment docs.


[BUG] "keyInfo is not in PEM format or in base64 format" when upgrading to v5

Because of this commit https://github.com/node-saml/node-saml/commit/8920013542bca7337a1330e8e73ceb7263114ee8 some of our certificates which contained spaces caused the error TypeError: keyInfo is not in PEM format or in base64 format, while it was working fine in previous version.
As this breaking change was not explicitly stated in the changelog, I figured out that I'd log this bug in case someone else searches for it. The solution is to clean up the base64 of idpCert before passing it to node-saml (so no spaces, correct "=" padding, etc) - I don't know if node-saml should defensively perform the clean up too, up to you (let me know if you'd like a PR for this, happy to do it).

note that I choose the bug category, though it isn't really a bug,  because I did not know what else to choose - since the breaking change happened in a major release, this is mostly for documentation purpose.

I'd be happy to see what a PR like this would look like. Were the spaces something that were generated by some tool you used or were they are artifact of some other processing you did with them?

So I don't really know how those spaces appeared tbh, the certificates are provisioned by our customers and it seems some of them have somehow the end of line of the base64 turned to spaces, and we did not validate enough so we accepted those certificates and stored them as is (which worked fine until the update, so we never realized).
For the "fix" I don't imagine anything too fancy one of:

keyData = keyData.replace(/ /g, ''); // drop only spaces
keyData = keyData.replace(/\s/g, ''); // drop all white-spaces
keyData = keyData.replace(/[^A-Za-z0-9+/=]/g,''); // drop all non base64 character

One potential issue (we did not encounter, but I think could happen) would also be missing padding, which probably used to work, but would be blocked by the new stricter regexp. I don't know if it would be a good idea to handle that too.

Wouldn't that affect the header line as well?

No because the failing assertion is after checking for PEM headers https://github.com/node-saml/node-saml/blob/72a39dcf46a93a8679123db78b22ff82d6dd4096/src/crypto.ts#L63 so that's were I would do the replace
Our certificate are in base 64 without the PEM headers. When there are PEM header, it seems that the regexp doesn't validate the base 64 at all (.*) so it would also have avoided us the issue

We had the same problem. Our key files had Bag attributes and Key attributes blocks before begin private key line. Removing those lines fixed the issue.

Thank you @srd90 for this research. It does seem that, while we could allow for more lenient whitespace handing, it cautions that this "may run the risk of accepting text that was not intended to be accepted in the first place". However, "the choice of parsing strategy depends on the context of use"; we can pick and still be compliant. It seems we have picked, and we are compliant, and any cert used which has extraneous whitespace is actually not from a compliant generator.
The quote in that Stack Overflow answer where it says "Furthermore, parsers SHOULD ignore whitespace and other non-base64 characters" is referring to data before the encapsulation boundaries.

Data before the encapsulation boundaries are
permitted, and parsers MUST NOT malfunction when processing such
data.  Furthermore, parsers SHOULD ignore whitespace and other non-
base64 characters and MUST handle different newline conventions.

Unless I'm missing something, the approach currently implemented is the most correct one and there isn't a compelling reason to change.
Here are some relevant snippets:

New implementations SHOULD emit the strict format (Figure 3)
specified above.  The choice of parsing strategy depends on the
context of use.

Most extant parsers ignore blanks at the ends of lines; blanks at the
beginnings of lines or in the middle of the base64-encoded data are
far less compatible.  These observations are codified in Figure 1.
The most lax parser implementations are not line-oriented at all and
will accept any mixture of whitespace outside of the encapsulation
boundaries (see Figure 2).  Such lax parsing may run the risk of
accepting text that was not intended to be accepted in the first
place (e.g., because the text was a snippet or sample).

Generators MUST wrap the base64-encoded lines so that each line
consists of exactly 64 characters except for the final line, which
will encode the remainder of the data (within the 64-character line
boundary), and they MUST NOT emit extraneous whitespace.  Parsers MAY
handle other line sizes.  These requirements are consistent with PEM
[RFC1421].

While I appreciate the strict format, it would be really helpful to have an error message which actually tells, which key/cert is falsy.

That seems very reasonable @KiwiKilian . Can you put up a PR for that?

I'm not sure I understand the problem. My application pulls the settings from the database including the IdP-supplied certificate stored as a text field. Upon upgrading to v5 I'm now getting this error message. Here's an example of the cert I'm using which is now rejected unless I remove all line breaks in the base64 string:
-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJALmVVuDWu4NYMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX
<-- more base64 encoded lines here -->
99CJ25RkAcSobWNF5zD0O6lgOo3cEdB/ksCq3hmtlC/DlLZ/D8CJ+7VuZnS1rR2n
aQ==
-----END CERTIFICATE-----

This format is widely used so I don't understand the need to enforce the strict certificate parsing.

I was facing similar issue after upgrading to v5.0.0. Solution was to update private key to start with
-----BEGIN PRIVATE KEY-----

My private key file was having extra properties e.g.
Private-Key: (2048 bit, 2 primes)
modulus:
publicExponent:
privateExponent:
prime1:
prime2:
exponent1:
exponent2:
coefficient:

I deleted all the above sections and then it worked fine.  Below is an example of how the content of files are loaded in nodejs:
  decryptionKey = fs.readFileSync(path.resolve("./privkey.pem"), "utf8"),
  idpCert = fs.readFileSync(path.resolve("./idp.crt"), "utf8");

Hope this helps someone landing on this issue.


Fix broken url in docs

Signed-off-by: Saintmalik<EMAIL_ADDRESS>Summary
Easy navigation and easy reading for others
/area documentation
/kind

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/lgtm
@saintmalik  Thank you for the fix!!!

LGTM label has been added.  Git tree hash: 139574e2317983a1c553afd4df645c4f8aef8482

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Rajakavitha1
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Rajakavitha1]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/approve

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Rajakavitha1
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Rajakavitha1]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr, Rajakavitha1
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Rajakavitha1,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Write new Notify Program

the email notifications assume that it can pass off the mail via loopback.  Allow it to do SMTP Auth

Changing this issue to build a new notify program.     the old search did both the pull/update and notifications.  These functions needed to be uncoupled.

DiningNotify is now part of the Main repo.   closing ticket

DiningNotify is now part of the Main repo. closing ticket


Close mic when not in use

Presently, once the mic is used once, the mic for this app keeps listening, which puts the orange dot,

on a Mac.
Even if YakGPT isn't using misusing the data, I can't use the orange indicator to know if my other apps are using the mic.

Very good point @owenbrown! This is something I noticed as well. I will noodle on this this afternoon.


Improve user service

This PR addresses a couple of issues with the user service, including a bug that is probably related to caching user entities.
Checklist

[x] I tested all changes and all related features with different users (student, tutor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[x] Server: I followed the coding and design guidelines.
[x] Server: I added multiple integration tests (Spring) related to the features (with a high test coverage)
[x] Server: I added @PreAuthorize and check the course groups for all new REST Calls (security)
[x] Server: I implemented the changes with a good performance and prevented too many database calls
[x] Server: I documented the Java code using JavaDoc style.
[x] Client: I followed the coding and design guidelines.
[x] Client: I added multiple integration tests (Jest) related to the features (with a high test coverage)
[x] Client: I added authorities to all new routes and check the course groups for displaying navigation elements (links, buttons)
[x] Client: I documented the TypeScript code using JSDoc style.
[x] Client: I added multiple screenshots/screencasts of my UI changes
[x] Client: I translated all the newly inserted strings into German and English

Motivation and Context
When users get instructor or tutor rights, they need to logout and login again, most probably because of caching issues.
Description
I fixed some caching issues and refactored the code so that it is easier to understand and cleaner.
This also fixes some smaller issues when deleting users and adds a TODO to cleanup all user related
Steps for Testing
You need 2 users (A, B) and 1 course (C)
C: Course
A: Instructor in C
B: Normal user (who has not any instructor rights yet)

Log in to Artemis with both users in two different browsers
User A: Navigate to Course Administration and grant B instructor rights in C
User B: Refresh the page and check that you can see course administration now (without logout and login)
User A: Remove B from the instructors in C
User B: Refresh the page and check that you cannot see course administration now (without logout and login)

Please also test other related use cases:

Course registration for students on TS1 and TS2 should work as before
Login and logout should work as before
Registering a new user account (TS2) should work as before

@FuchsDominik @stefanwaldhauser @TobiasPr I just tested it again locally and it works like a charm. I cannot reproduce your error. Can you please double check and help me to reproduce the issue?

Ok, I can now reproduce the issue. The problem is the authentication mechanism for roles using
@PreAuthorize("hasAnyRole('TA', 'INSTRUCTOR', 'ADMIN')")
To make this check super fast (and to prevent database access), the authentication seems to be retrieved from the bearer token (I need to double check this), which would basically mean that there is currently no other way than logging out and logging in again to get the new rights.

Ok, I can now reproduce the issue. The problem is the authentication mechanism for roles using
@PreAuthorize("hasAnyRole('TA', 'INSTRUCTOR', 'ADMIN')")
To make this check super fast (and to prevent database access), the authentication seems to be retrieved from the bearer token (I need to double check this), which would basically mean that there is currently no other way than logging out and logging in again to get the new rights.

Yes, this is the case and there seems to be no easy way to break this :-(
We basically would need to regenerate the token of the user, but the question would be when this regeneration should happen. For performance reasons, we cannot re-generate it all the time.


Crypto-App Added

🛠 Fixes Issue (#221)
Added Crypto-App
👨💻 Changes proposed
Added crypto-app in this project.

✅ Check List (Check all the applicable boxes)

[ ] My code doesn't break any part of the project (Zero Octave-Javascript-Projects).
[ ] This PR does not contain plagiarized content.
[ ] My Addition/Changes works properly and matches the overall repo pattern.
[ ] The title of my pull request is a short description of the requested changes.

📷 Screenshots

@Astrodevil
No .DS_store isn't necessary ,I will remove it


Pre/Post Scripts are not executed

Hi! I am currently defining pre scripts on yaks version YAKS 0.4.0-202104200030. As already discussed in some earlier issues I know that these script are executed on my local host. Yet I struggle to execute a single script. To break the problem down I post a small example where the script is no executed.
[start.sh]
yaks test -s yaks.config.yaml hook.feature
[hook.feature]
Feature: Hello
Scenario: Print hello message
Given print 'Hello from YAKS!'
[yaks.config.yaml]
pre:

script: prehook.sh
run: sudo mkdir hook_directory

[prehook.sh]
sudo mkdir hook_directory
When executing this yaks test I still can not find the directory "hook_directory". Are there limits to the capabilities of these scripts?

The scripts are run locally as you have mentioned so I would expect the directory to be created on your local host. In case you need to create the directory inside of the test runtime POD you would need to run a custom Cucumber step as part of the test that creates the directory.

Hi Christoph!
I figured out why the script was not executed. I had to change the filename yaks.config.yaml to yaks-config.yaml. After that change the scripts ran again. I that this is a bug because I copied the "yaks.config.yaml" straight out of your documentation -->yaks test --settings yaks.settings.yaml camel-route.feature

yeah config file needs to be yaks-config.yaml


Fix documentation : Add proper casting to Post object using json() method

Problem
Using the code is causing build error as below.

Reason
In the definition, we can see the .json method don't take any type nor any parameter. Instead it is needed to cast the response object manually.

json(): Promise<JSON>;
Solution
Manually cast the response object as Post like below
const post = await c.req.json() as Post
It will solve the build error.
Special note
We also need to create the model file and implement methods to properly solve the other build errors.

Hi @foyzulkarim !
According to Web Standard, json() can not have generics. But, in Cloudflare Workers environments, we could pass the generics like json<T>() for adding types of the body.
const post = c.req.json<Post>()

With @cloudflare/workers-types, type hints are enabled. The definition of json<T>() is here:
https://github.com/cloudflare/workers-types/blob/master/overrides/http.d.ts#L2
So, it's OK to write down c.req.json<Post>().

We also need to create the model file and implement methods to properly solve the other build errors.

I know it, the code is just for explanation, so I think it does not have to work well.

BUT, it is indeed difficult to understand. I will make the website for Hono next time, so I will try to explain it clearly then. I'll close this PR. Thank you for noticing them.

@yusukebe thanks for your explanation. The title is mentioning as Practical example, not the Cloudflare example. If it is not too much trouble, please add this small note in the readme somewhere. Otherwise whenever a new visitor will come,  everyone of them will face this error.

Yes, I'll do it. Thanks again for your concern.

Thanks for your awesome work. 🥇


Fix undefined track function in mocha tests

The master builds have been failing with a few errors:
TypeError: Cannot read property 'track' of undefined

It's likely due to the change in https://github.com/artsy/force/pull/6249/commits/cafff1a63b61bafddf649380234ad004716f535b. This fixes it by mocking it in tests.

@emmadickson I assigned this to you since @eessex is OOO. Let me know if I can help clarify anything! Thanks! 🙏


it is not necessay to loadFromMemcached  in createSession

From<EMAIL_ADDRESS>on February 22, 2012 11:20:53
I don't understand, loadFromMemcached has bean invoked in findSession methed. why we still invoked loadFromMemcached in createSession.
if we need invoked createSession,it explain the sessionId is not in memcached.
so it is not necessay to load the session From Memcached in createSession.
Original issue: http://code.google.com/p/memcached-session-manager/issues/detail?id=122

From martin.grotzke on February 22, 2012 05:11:54
This was implemented similarly in a previous version of tomcats PersistentManager (which I used as blueprint together with DeltaManager when I started this project), but AFAICS you're right, that the invocation of loadFromMemcached from createSession can be omitted, as all createSession invocations before also invoke findSession (checked with current tomcat6/tomcat7 versions).
Thanx for spotting this!
Status: Accepted
Labels: -Type-Defect Type-Enhancement

From martin.grotzke on March 03, 2012 16:15:46
Changed.
Status: Fixed
Labels: Milestone-1.6.2


Problems resuming after network disconnected and then reconnected

Got this exception when coming back to my terminal running rainbowstream after resuming my laptop:
[@kramimus]: -- Heartbeat Timeout --
You can use switch command to return to your stream.

Also, I generally have problems making my stream resume after resuming my laptop.  Often the stream never resumes and I have to exit rainbowstream and start it again, even if I enter "switch mine".  It would be nice if it could just retry connecting after a few minutes so I would not have to do anything.
Version 1.2.5

I see. I will look into this further and add a patch when have enough time . Thanks :+1:

Also got this bug, after fiddling with the time format and Unicode symbols in rainbow_config.json. Replaced it with the default config file and it went back to normal.
 Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rainbowstream/rainbow.py", line 2080, in stream
    except (URLError, ConnectionResetError):
NameError: global name 'ConnectionResetError' is not defined


doc updates for #1

While it was a disastrous miscalculation to exclude ES6/ES7/Mongodb+Postgresql support from the initial release of noop3, I will wait patiently until a more robust buildchain is developed to facilitate the necessary transpiler plugins to generate the code. Consider those bullet points deferred until the next major release when I'm assuming both grunt and gulp will be leveraged via a massive, unreadable npm start one-liner.
No narwhal's is inexcusable, though, bordering on xenophobic. For that reason, I've updated the documentation to include an additional narwhal example. It's simple, familiar, and I think it will all help us learn not only something about noop3, but about ourselves.
Finally, to show there are no hard feelings on the narwhal thing, I've also include here the song you requested. I believe it not only suits this change, but also captures the essence of this interaction, specifically as it relates to professional interaction in our field today.  Thank you!
And sorry about the miscalculations. I'm still a noop noob.


Update set-up-cost-visibility-for-kubernetes.md

Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66b115cc38c4bf462052d858--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66b12d8729cce200abef9f74--harness-developer.netlify.app


Parallel workers

I want to perform queries by upto 128 number of parallel/concurrent workers. Can you recommend some specific configuration please? I want to measure the QpS (queries per second) value of many triple stores for parallelism (querying by more than one user at a time).

Parliament has a many readers/single writer model of concurrency.  In other words, if Parliament receives several requests at once, and all of them are read-only requests (such as a query), then Parliament will execute those requests in parallel.  Whenever Parliament receives a request that requires writing (such as an insert, SPARQL Update, create/drop graph, etc.), that request will wait until all other executing requests are finished, and then it will execute.  And while it is executing, all other requests will wait in the queue until it is finished.
This model works well in most situations, where the number of concurrent requests is not terribly large and read-only requests dominate.  A dozen or two concurrent queries is not uncommon in my experience, and does not cause problems.  I would expect that Parliament will handle 128 concurrent read-only requests correctly, but the performance may not be as high as one would like.  Mainly this is because all those requests ultimately are reading from the same I/O device.  Therefore, caching becomes the key issue for concurrency levels this high.
Parliament uses four pools of memory:

The Java heap (to configure, see "Parliament's Startup Scripts " in Section 2.2)
Berkeley DB's cache (to configure, see "Parliament’s Main Configuration File" in Section 2.2, under the key bdbCacheSize)
The C++ heap (not configurable)
The operating system's demand-paged mappings of memory-mapped files (not configurable)

Achieving the right balance between these pools of memory is tricky, and I'm afraid I don't have any concrete advice, especially in the case of 128 concurrent requests.  However, there are a few things I can say:

Don't make the Java heap too big.  Many Java programmers are used to setting their Java heap as big as possible, but for Parliament, the Java heap is not as important as the Berkeley DB cache and the memory-mapped files.
Parliament currently has a bug (issue #29) that applies the Berkeley DB cache size on a per-graph basis.  So if you have 100 named graphs, each graph gets its own cache.  I plan to fix this bug in the next release, so that there is one Berkeley DB cache across all the graphs.
The guidance in the User Guide for the size of the Berkeley DB cache is probably too low.

As before, please leave this issue open as a reminder to me to improve the documentation.

Documentation improvements incorporated on 11/1/2021, to appear in version 2.8.0.


Nuxt link outer element

Is there any way to use Active Class on Outer Element?
In plain router-link we can do it by simply add tag attribute on router-link.
But it doesn't work on nuxt-link

Hi @saeidakbari,
Can you show us some code snippet about it?
I can reproduce your issue, tag attribute works fine.

@alexchopin
when i add tag attribute nuxt-link won't make <a> tag
here is my code :

and here is inspect element :

@alexchopin
I fixed it
sorry for my mistake
i wrap all span tag inside a tag and its worked
i will close issue.


Investigate custom shader support

At the moment, all objects are rendered using a standard set of shaders, but this is limiting.
The good thing about the limits are that it means people new to game dev don't need to worry about how shaders and rendering work. The drawback is the people willing to roll up their sleeves to do more are unable.
One big question is what language would the shaders be written in? Would we need to do something custom (I believe Godot does this)? It would need to work for WebGL 1 and 2, and potentially other renderers in the future.
One of the requirements that custom shaders will force is the ability to swap shaders on a per object basis, which is useful in general even if we decided custom shader weren't themselves a good idea.

Closing - will be in the next release.


Current song's album not being displayed and being cut off of at the bottom.

Hi everyone,
I noticed for the current song, the artwork for the album is not being displayed. Also, I noticed that the image is cut off at the bottom.
I think this would be a good first issue for newcomers to fix up! :)
Here's the problem for Desktop:

I am unable to reproduce this on a few different OS's and browsers. Can anyone confirm that this is still an issue? I think it may be resolved.

@moT01 I can't reproduce anymore.

https://github.com/freeCodeCamp/coderadio-client/pull/135 seems to have resolved the issue.


Eval Error

Hello, during the evaluation on my dataset, it encounters this error. It stops and reports an error after processing a few images, and it only continues when I remove those images. This process repeats itself. Can you help me with this?
Traceback (most recent call last):
File "eval_mixNet.py", line 259, in
main(vis_dir)
File "eval_mixNet.py", line 231, in main
inference(model, test_loader, output_dir)
File "eval_mixNet.py", line 65, in inference
for i, (image, meta) in enumerate(test_loader):
File "D:\anaconda\envs\mixnet\lib\site-packages\torch\utils\data\dataloader.py", line 633, in next
data = self._next_data()
File "D:\anaconda\envs\mixnet\lib\site-packages\torch\utils\data\dataloader.py", line 677, in _next_data
data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
File "D:\anaconda\envs\mixnet\lib\site-packages\torch\utils\data_utils\fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\anaconda\envs\mixnet\lib\site-packages\torch\utils\data_utils\fetch.py", line 51, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\OCR_AICS\MixNet\dataset\Total_Text.py", line 134, in getitem
data = self.load_img_gt(item)
File "D:\OCR_AICS\MixNet\dataset\Total_Text.py", line 108, in load_img_gt
polygons = self.parse_mat(annotation_path)
File "D:\OCR_AICS\MixNet\dataset\Total_Text.py", line 61, in parse_mat
pts = np.stack([x, y]).T.astype(np.int32)
File "<array_function internals>", line 200, in stack
File "D:\anaconda\envs\mixnet\lib\site-packages\numpy\core\shape_base.py", line 464, in stack
raise ValueError('all input arrays must have the same shape')
ValueError: all input arrays must have the same shape

Hello, there seems to be an error while reading the annotation.
Maybe the annonation of these images doesn't match the format.
I guess it could start by checking the annotations for these images or try removing the part of the code that reads the annotations.


showRationale after

is it possible to show the rationale,first time user presses deny ?!?

@s1mar you can call on your side as you like, it's not supported on library side though.


Fully populate the ds3_error returned

when retry_count > client->num_redirects
*** No errors detected
==8301==
==8301== LEAK SUMMARY:
==8301==    definitely lost: 0 bytes in 0 blocks
==8301==    indirectly lost: 0 bytes in 0 blocks
==8301==      possibly lost: 0 bytes in 0 blocks
==8301==    still reachable: 91,618 bytes in 17 blocks
==8301==         suppressed: 0 bytes in 0 blocks
==8301==
==8301== For counts of detected and suppressed errors, rerun with: -v
==8301== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Bump

As an end user how does this change how I treat a redirect retry limit?

if you encounter this situation, you will no longer get a seg fault if you try to access ds3_error->error, ds3_error->error->http_error_code, or ds3_error->error->code


test: make sure we test port correctness for internal services

Hi @achanda could you please rebase the PR and address the errors in the CI? Please remember to always run make test and make checks to check that the code is in good shape.

Thanks @mnencia I forgot about this PR 🤦 Fixed everything now

/ok-to-merge unit tests only change


Update test_sql_database.py

In the previous function, db_chain.run is used to execute the chain, but in line number 79, only db_chain is called, which can lead to code inefficiency because it doesn't actually execute the chain. To ensure consistency, it's important to use the .run method consistently to execute the chain in all relevant parts of the code.

From: Shorthills AI @.>
Sent: Monday, October 16, 2023 6:42 PM
To: shorthills-ai/langchain @.>
Cc: Vishal Chand @.>; Author @.>
Subject: Re: [shorthills-ai/langchain] Update test_sql_database.py (PR #89)
@ShorthillsAI commented on this pull request.
How did you reproduce this error, does the function not work without a run function, by chance?
Please elaborate on the issue, replicate the steps, and fix test result here
—
Reply to this email directly, view it on GitHubhttps://github.com/shorthills-ai/langchain/pull/89#pullrequestreview-1679956558, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCML6B3S525MY52OEXBJ4SDX7UXDJAVCNFSM6AAAAAA6B5Z5J6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMNZZHE2TMNJVHA.
You are receiving this because you authored the thread.Message ID: @.***>


List of party pets

Please start a list of pets who will come to the party. Name the file party-pets.md.

I will write a list of pets who will come to the party.