Enhance Presentation server to use valid Coco Pharmaceutical userids

Thanks @davidradl :)

@mandy-chessell There seems to be some inconsistencies with the user definitions. I had copied a subset of the UI chassis UI users into the presentation server users. I was looking at Faith Broker's userid in the Egeria code base and see:

open-metadata-resources/open-metadata-deployment/sample-data/coco-pharmaceuticals/vars/users.yml
Has username: fbroker

/Users/davidradley/egeria/open-metadata-resources/open-metadata-samples/open-metadata-security-samples/src/main/java/org/odpi/openmetadata/metadatasecurity/samples/CocoPharmaServerSecurityConnector.java
Has final String faithBrokerUserId    = "faithbroker";

src/main/java/org/odpi/openmetadata/userinterface/uichassis/springboot/auth/db/DemoInMemoryInitializingBean.java
addUser("faith","Faith Broker ", "admin","/resources/img/faith.png",Role.DATA_ANALYST);
addUser("faithbroker", "Faith Broker", "admin", "/resources/img/user.png" );

org/odpi/openmetadata/accessservices/governanceprogram/samples/leadership/GovernanceLeadershipSample.java
private static final String faithBrokerUserId  = "faithBroker";

presentation-server/nodejs/src/server/db/users.js Has a subset of the UI chassis users
addUser("faith","Faith Broker ", "admin","/resources/img/faith.png",Role.DATA_ANALYST);
addUser("faithbroker", "Faith Broker", "admin", "/resources/img/user.png" );

I assume that the userids exposed in the UIs are not the valid ones. Is the users.yml list the definitive user definitions? Or is UI Chassis user list the definitive ones and the short versions of the user is e.g. faith should be removed?
At the moment Faith's userid is defined as fbroker, faith, faithbroker or faithBroker.
I was hoping to fix the presentation server part of this in this issue as I can test that and raise issues against the other areas as appropriate.
fyi @planetf1.

I would suggest fixing all the inconsistencies in the same PR
IMO 'faithbroker' is correct - - I seem to remember when I added to the UI I left the existing fbroker in place (in case ING were using it) . I would suggest copying in bogdan/chris to check no undue impact from the other changes

Also to be made consistent - the ldap definitions in ./open-metadata-resources/open-metadata-deployment/sample-data/coco-pharmaceuticals/ldap which use another form for example:
 dn: uid=Faith_Broker,ou=People,dc=example,dc=org
 objectclass: top
 objectclass: person
 objectclass: organizationalPerson
 objectclass: inetOrgPerson
 cn: Faith Broker
 gn: Faith
 sn: Broker
 uid: Faith_Broker
 userPassword: password

@planetf1 . So your suggestion is that list in src/main/java/org/odpi/openmetadata/userinterface/uichassis/springboot/auth/db/DemoInMemoryInitializingBean.java
is accurate?
Apart from we need to remove
 addUser("faith","Faith Broker ", "admin","/resources/img/faith.png",Role.DATA_ANALYST);         addUser("callie","Callie Quartile ", "admin","/resources/img/callie.png",Role.HR);         addUser("zach", "Zach Now", "admin", "/resources/img/user.png" );         addUser("steves", "Steve Starter", "admin", "/resources/img/user.png" );         addUser("terri", "Terri Daring", "admin", "/resources/img/user.png" );
I notice the list has
addUser("designa", "designa", "admin", "/resources/img/user.png" );
which I am not sure what the long named version is
and
addUser("stewFaster", "stewFaster", "admin", "/resources/img/user.png" );
this has capitalization which does not match the rest of the userids.

@cmgrote and @bogdan-sava we are looking to standardize the coco users in the code base. If we standardize on something based on the y last update - would there be implications for you?

I attempted to make that file accurate when I last edited it but that was probably 1.5 years ago, so ids may have been changed since. The reason for opening #1140 was to get things consolidated - though there have been changes since then.
I think the most important aspect is consistency. Beyond that I would go with

all lower case userids
generally of form  like faithbroker
using only the set defined in the coco pharma persona documentation

The security plugin will also refer to explicit users so needs to form part of the evaluation too - this can be found at 9:56]
/Users/jonesn/src/egeria/open-metadata-resources/open-metadata-samples/open-metadata-security-samples/src/main/java/org/odpi/openmetadata/metadatasecurity/samples CocoPharmaServerSecurityConnector.java - interestingly this does use 'stewFaster'
I think any reasonable proposal is ok as long as it aligns all these pieces - our notebooks, security plugin, UI #1, UI #2, documentation/readmes, any presentations, ldap definitions. The most likely correct sources are probably the security plugin tbh as they follow the coco pharma persona docs very closely
Also I noticed open-metadata-implementation/adapters/open-connectors/data-store-connectors/file-connectors/csv-file-connector/src/test/resources/SimpleColumnsWithColumnNames.csv uses the same set.

See also #3450 (for UI #1)


Distribution Support Request

Details of the Distribution

Name of the Distribution:
Version of the Distribution:
Package Types

[ ] Deb
[ ] RPM
[ ] Tar.gz
Snap - Please file issue in https://github.com/powershell/powershell-snap.  This issues type is unrelated to snap packages with a distribution neutral.

Processor Architecture (One per request):
The following is a requirement for supporting a distribution without exception.

[ ] The version and architecture of the Distribution is supported by .NET Core.

The following are requirements for supporting a distribution.
Please write a justification for any exception where these criteria are not met and
the PowerShell committee will review the request.

[ ] The version of the Distribution is supported for at least one year.
[ ] The version of the Distribution is not an interim release or equivalent.

Progress

[ ] An issues has been filed to create a Docker image in https://github.com/powershell/powershell-docker

For PowerShell Team ONLY

[ ] Docker image created
[ ] Docker image published
[ ] Distribution tested
[ ] Update packages.microsoft.com deployment
[ ] Lifecycle updated
[ ] Documentation Updated

/cc @TravisEz13 and @adityapatwardhan


feat: switch store from rocksdb to redb

Opening this for discussion, not sure if we want to ever merge this. You can find a description of redb at https://github.com/cberner/redb.
Some pros & cons

Redb is written in Rust and performs quite well according to benchmarks.
Redb is much less mature than Rocksdb, and unclear how well it will scale in the TB+ scale.
Redb is considerably simpler & easier to work with, and has proper transactions, allowing us to remove the hacky in memory locks
Redb compiles a lot faster (binary size is smaller as well, but I haven't measured by how much)

Do we have any benchmarks for the database that can be compared?

We have some in the iroh-store/benches, which currently suggest ~20% reduced perf in some cases, but I am not very confident in them. Need to look at some other benchmarks as well

Since redb does not guarantee any data format compat, at the very least we would have to check in cargo.lock and basically tell people to do a full blown import/export to/from car or whatever whenever the data format changes.
Does this require nightly? I get this error:
error[E0658]: use of unstable library feature 'map_first_last'

I was running with stable, so just a pretty new one probably

Code looks reasonable, and the build times are quite good. I have got to play around with the thing in isolation to say more.
Why does it have an (optional) dependency on a python interpreter?
https://github.com/cberner/redb/blob/master/Cargo.toml#L23

That’s a library to build python apis, I supect because they expose a python api

Since redb does not guarantee any data format compat, at the very least we would have to check in cargo.lock and basically tell people to do a full blown import/export to/from car or whatever whenever the data format changes.

We can just use a fixed version eg =1.2.3, no need for Cargo.lock.
The author said they are quite close to a stable file format, but in any case, as long as there is no actual changes in the format, we can upgrade and test manually when a new version is released. And if the format changes we can provide a migration using import/export functionality if needed.

Not saying that this is a showstopper in general, but on this branch the add performance is totally... bad.
For adding a large file I get about 40 mb/s, whereas with rocksdb I get more like 400 mb/s.

we should figure it if we can improve that, that is very bad :/

It is due to the transactions committing immediately fully to disk. If you set the transaction mode to "eventual" you get back the expected performance.
As I said on discord, you can't treat a database system that does not have a WAL the same way as one that works with a WAL and expect similar performance.
For the ipfs add thing you would just do transactions with mode eventual and then one with mode immediate at the very end, that should take care of it.

Hello! The code that was previously hosted in this repository has been moved to n0-computer/beetle. beetle is in maintenance mode, but will still accept PRs.
If you are still interested in getting your PR merged, please re-open your PR on n0-computer/beetle.
Check out our blog post for more info on our new direction for iroh: https://n0.computer/blog/a-new-direction-for-iroh/


fix:GML2:readFlatCoordinates can return the format of (x,y) or (y,x)

Fixes #15617

Thanks, @jacklinna

Also, the tests show that the fix might not be as straightforward as you thought:
Chrome Headless <IP_ADDRESS> (Linux x86_64) ol.format.WFS when parsing mapserver GML2 polygon creates a polygon for My Polygon with hole FAILED
	Error: expected [ 47.003018, -0.768746, 0.532597, undefined ] to sort of equal [ 47.003018, -0.768746, 47.925567, 0.532597 ]
	    at Assertion.assert (/home/runner/work/openlayers/openlayers/node_modules/expect.js/index.js:96:13)
	    at Assertion.eql (/home/runner/work/openlayers/openlayers/node_modules/expect.js/index.js:230:10)
	    at Context.<anonymous> (/tmp/_karma_webpack_683077/webpack:/ol/test/browser/spec/ol/format/wfs.test.js:141:1)


Project review (opened new issue, because reply was misplaced in another issue) : Ayesha_Danny_Suzana

All the requirements for the project are met  ✔
Good commit history ✔
Your README file is professional ✔
Follow GitHub workflow ✔
Issues:
-I suggest using img tag instead of object tag for images, because currently, object tag is not supported by all browsers you need to add browser support to provide support for more browsers and the validity of the document as well.
-Use of Classes art1, art2, art3,…, I suggest using id attribute for that cause class IDs are only used when one element on the page should have a particular style applied to it. However, a class can be used to identify more than one HTML element.
Acknowledgments:
-Appropriate use of semantic tags
-Clearly written code
-Well structured
Originally posted by @Suzi216 in https://github.com/BucurEva87/Portofolio/issues/9#issuecomment-1181414054

HTML Object element is supported in all major browsers
Switched from classes for article elements and div elements (.designX) to ids.


[checks] Align TFSEC checks to Secuirty Hub findings

Target Provider
AWS
Target Terraform Block type

[ ] module
[ ] resource

Target Terraform Type(s)
All supported resources that have an AWS Security Hub check
Description of check
AWS Security Hub implements security checks for CIS, AWS Best Practice and PCI-DSS.
It would be very beneficial if TFSEC development could prioritise adding support for each the Security Hub checks, starting by covering off the critical, then highs, mediums etc.
Relevant links
https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards.html
CIS controls - https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-cis-controls.html
PCI-DSS controls - https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-pci-controls.html
AWS Best Practice controls - https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-fsbp-controls.html
Anything Else
I've used TFSec in a number of production environments and whilst it's been great to 'shift security to the left' for some quick wins, there are plenty of things showing up in Security Hub that TFSec hasn't caught, and it would be great to close that gap.
For example this check isnt' picked up by TFSec so I've only found it afterwards, and there are many others so this is just an example - https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-fsbp-controls.html#fsbp-lambda-1

@markrossatos  - really keen to get on this. Once we have some bandwidth we'll attack this aggressively


An attachment cannot be view the first time it is clicked

In order to view the attachments of a feature, it's currently necessary to:

identify the feature
view its list of attachments
tap to individually request viewing at least one attachment
exit the view to return to the map
identify another feature or otherwise deselect the current feature
re-select the same feature from step 1 and again commence with step 2

Observations:

thumbnails will be created
attachments themselves will be viewable

This might be the product of attempting to view the attachment before the file has finished downloading.
I'll look into it.


[Launcher] 'Search the web' option after search results

Summary of the new feature/enhancement
Say a user wants to quickly search the internet about 'pineapples' and but also has a folder named 'Pineapples'. After the first result which would be the search result for the folder, an addition result can be 'Search the web for: pineapples'. This would open up the default browser (or a new tab) and paste the query in the search box.

+1 on this one! Specifically being able to go "Google " or "Bing " or even "[Google,Bing] Maps ".


ananicy-rules-cachyos: 0-unstable-2024-05-28 -> 0-unstable-2024-06-07

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for ananicy-rules-cachyos is: CachyOS' ananicy-rules meant to be used with ananicy-cpp
meta.homepage for ananicy-rules-cachyos is: https://github.com/CachyOS/ananicy-rules
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0-unstable-2024-06-07 in filename of file in /nix/store/gjhkf2h983m0h0ffqwch6khl6vl30xrj-ananicy-rules-cachyos-0-unstable-2024-06-07

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

ananicy-rules-cachyos

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/gjhkf2h983m0h0ffqwch6khl6vl30xrj-ananicy-rules-cachyos-0-unstable-2024-06-07 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A ananicy-rules-cachyos https://github.com/r-ryantm/nixpkgs/archive/98ab6d76b07718b32eaf9c9957cf4bca6c997c46.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/98ab6d76b07718b32eaf9c9957cf4bca6c997c46#ananicy-rules-cachyos

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/gjhkf2h983m0h0ffqwch6khl6vl30xrj-ananicy-rules-cachyos-0-unstable-2024-06-07
ls -la /nix/store/gjhkf2h983m0h0ffqwch6khl6vl30xrj-ananicy-rules-cachyos-0-unstable-2024-06-07/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

1 package built:

ananicy-rules-cachyos

Maintainer pings
cc @Artturin @diniamo @johnrtitor for testing.

@NixOS/nixpkgs-merge-bot merge


E01 us06 t07/e02 us02 t02  branch

CSS is absolutely horrible, but considering time we have left, that's what's delivered

User ID is impossible to get for a new user creation with current Back end capabilities @BOPS38IMT, waiting for a fix


Any reason why the very similar function reset_prometheus is copy&pasted multiple times in your tests?

Let's say that the canonical version is here: https://github.com/trallnag/prometheus-fastapi-instrumentator/blob/c0813def118754db109a435d154351a680eede8e/tests/helpers/utils.py#L16-L28
I see pretty much the same code again in other test files, such as:

https://github.com/trallnag/prometheus-fastapi-instrumentator/blob/c0813def118754db109a435d154351a680eede8e/tests/test_expose.py#L61-L74
https://github.com/trallnag/prometheus-fastapi-instrumentator/blob/c0813def118754db109a435d154351a680eede8e/tests/test_instrumentation.py#L26-L39
https://github.com/trallnag/prometheus-fastapi-instrumentator/blob/c0813def118754db109a435d154351a680eede8e/tests/test_instrumentator_expose.py#L15-L28

It needs to be refactored. I will work on it. Thanks for your announcing.


Support testing macos/windows

Currently gh actions only use linux to run tests and it'd be beneficial to enable macos and windows getting better test coverage.
I experimented enabling tests on macos/windows are here's some notes.

On windows there's some things to change

We need graal 22.2.0 due to https://github.com/oracle/graal/issues/4502.
Line breaks cause issues with checkstyle and spotless. Works by making git to checkout without modifying line breaks and then spotless needs to be configured to use unix style breaks.
Quite a bit of trouble with gradle build not exiting most likely due to https://github.com/gradle/gradle/issues/3987. As tests run underlying project tests via exec task(by running gradle command), theory is that the process causing build to hung up is a gradle daemon. I mostly got build working by making sure that all tests are using main wrapper which limits created daemons as gradle version would be same. Disabling daemon on gradle run for tests might also fix this issue on windows.

On both macos and windows

postgres and mysql test will fail as those were made to work only on linux runners as you get failure to launch docker container for database.
Tried @EnabledOnOs(OS.LINUX) on those tests which doesn't work on native tests due to
Error: Classes that should be initialized at run time got initialized during image building: org.junit.jupiter.api.condition.OS was unintentionally initialized at build time.

I have a PR for this issue.


Add activation scripts to workaround problem in relocatability

Fix https://github.com/conda-forge/libignition-rendering4-feedstock/issues/25 .
The activation scripts are far from perfect, I have opened https://github.com/conda-forge/libignition-rendering4-feedstock/issues/26 to track that.
Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

Testing a local version of this package on ign-sensors's examples/save_image, the image is correctly generated on Windows:

Thanks for this @traversaro! Should we use $PKG_NAME instead of hard coding the name though so it's less work when updating versions?


Ediatable configs without risk of versioning + Getting started instructions in README

Pull Request
Description

Moves the existing default configs to configs.example directory
Users are now expected to copy this to configs directory (which is gitignored to prevent versioning)
Adds instructions for new users to PVNet to help get up and running, including advice on sourcing datasets for use with PVNet

How Has This Been Tested?
Ran through new instructions from scratch to ensure accuracy

[x] Yes

If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?

[ ] Yes

Checklist:

[x] My code follows OCF's coding style guidelines
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[x] I have checked my code and corrected any misspellings

@dfulu can you have a look at this?


Change the tier of block blob that has snapshots

As per documentation, 'You cannot currently change the tier of a block blob that has snapshots'.
I tried to change the access tier from hot to cool of block blob that has snapshot and i am successfully able to change the tier through portal.
It seems to be document is not updated as per latest changes.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 30c49a6b-e628-4aab-dc7f-fc88264b88f0
Version Independent ID: 3da34196-ac73-5fd9-c473-51a07226f83a
Content: Hot, cool, and archive access tiers for blobs - Azure Storage
Content Source: articles/storage/blobs/storage-blob-storage-tiers.md
Service: storage
Sub-service: blobs
GitHub Login: @mhopkins-msft
Microsoft Alias: mhopkins

@shwetamathur7 Thank you for bringing this to our notice, our team will look into it and update the document accordingly.

@shwetamathur7 Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

@shwetamathur7 Thank you for calling this to my attention. I've removed that sentence from the topic. The update should be published in the next day or so.
Thanks!
~ Mark
#please-close


Grid display with draggable

Is it not possible to use grid display with this?
I tried and every new object goes to the next row instead of filling in the column.
When I remove the draggable tag it displays normally.

i looking for the same thing. do you have a solution?

i found this, maybe we can use with it.
https://ej2.syncfusion.com/vue/documentation/diagram/grid-lines/

Could writer provide one example like vue.draggable ?
Thanks


Update libnetwork to fix port binding issue

This updates balena-libnetwork to a version that should fix some port binding issues that may happen after balenaEngine or device crashes. Specifically, this balena-libnetwork version cherry-picks this unmerged upstream patch (with minor changes to make it compatible with recent Moby versions).
I cannot comment on the precise details, but this patch essentially changes the order of initialization of some network-related components in order to avoid getting into a inconsistent state.
Fixes #272 (at least shall fix some of its occurrences)
Testing
Tested for regressions: Engine unit tests and integration tests passing. Tried it in a meta-balena branch; all tests passed. Also did some manual testing on a Pi 3.
Testing for effectiveness is another story. We don't have a reliable way to reproduce the issue, so I created a version of the Engine meant to crash at a point that triggers the issue. Now, I cannot tell for sure this is reproducing exactly the same case we are seeing in practice, but to me the symptoms look close enough to give a good confidence this is a step in the right direction.
I'll describe in details what I did to reproduce the issue and test the patch because this might be a good future reference should other similar issues appear (or this one re-appear).
First, based on this analysis we see that the issue happens when the Engine crashes at a more or less specific point. I tried to locate such point; not sure I found it exactly, but I found something -- and then added some code that allows us to force a crash right there:
diff --git a/daemon/container_operations.go b/daemon/container_operations.go
index af69e0474b..029d8176a6 100644
--- a/daemon/container_operations.go
+++ b/daemon/container_operations.go
@@ -825,6 +825,14 @@ func (daemon *Daemon) connectToNetwork(container *container.Container, idOrName
               return err
       }

+       ///////////////////
+       _, err = os.Stat("/mnt/data/crash-the-engine.please")
+       if err == nil {
+               os.Remove("/mnt/data/crash-the-engine.please")
+               panic("Geronimoooooo!")
+       }
+       //////////////////
+
       if !container.Managed {
               // add container name/alias to DNS
               if err := daemon.ActivateContainerServiceBinding(container.Name); err != nil {

For the test itself, I prepared two Engine versions: one containing the patch we are testing (balena-engine-patched), another containing the "crash code" above (balena-engine-crashable). I copied both to the data partition of a Pi 3, so that  I can symlink /usr/bin/balena-engine to either of them as needed. And then:

First let's reproduce the issue. Initial state: running balena-engine-crashable (but not forcing a crash yet!), user service (container) running, all nice and fine.
Run ps aux | grep proxy, check the PIDs. In my case, 2216 and 2226.
touch /mnt/data/crash-the-engine.please
Restart the service.
Engine crashes, service doesn't restart.
But we get our stale balena-engine-proxy processes holding the ports. Check with lsof -nP -iTCP -sTCP:LISTEN and ps aux | grep proxy. Notice these are new processes (PIDs 2984 and 2993 in my case) created while bringing up the service again, before the forced crash.
reboot
We get balena-engine-proxy processes even before we try to start the service (IIUC, they are created as the Engine initializes the network subsystem; it's basically trying to restore the pre-reboot state.)
As the device tries to start the service, we get the error we were looking for: "Failed to allocate and map port 80-80: Bind for <IP_ADDRESS>:80 failed: port is already allocated". Service remains in the "Installed" state.
Now let's test the patch. Redo steps 1-6.
Replace the Engine with the patched version: mount -o remount,rw /, cd /usr/bin/, ln -nfs /mnt/data/balena-engine-patched balena-engine.
reboot
The service starts normally, no port binding issues at all!

So, looks like the patch helped, Q.E.D. :slightly_smiling_face:
Side note: If we reboot again between steps 9 and 10 , the service starts successfully. In this case, we apparently don't create balena-engine-proxy processes before attempting to start the service. I don't know why this happens -- why does this second reboot (apparently) makes the internal state consistent again?

Very clever test case reproduction, looks good to me!


Improve error message when connecting Dataverse account

Steps

Log in to osf.io and head to one of your projects.
Navigate to User Settings and click "Connect an Account" next to the Dataverse add-on.
Without selecting a Dataverse repository, click "Save."
Notice the error message.

Current Message
"Sorry, but there was a problem connecting to that instance of Dataverse. It is likely that the instance hasn't been upgraded to Dataverse 4.0. If you have any questions or believe this to be an error, please contact<EMAIL_ADDRESS>Suggested Change
Make the error message match other error messages when required fields are left empty.
Change to: "This field is required" or "Please select a Dataverse repository."
Screenshots
Current Dataverse error message for required field

Uploader widget required field

Onboarder project creation required field

I can work on this! Thanks!

Done!


Fix Wownero sync issue on linux desktop

Add Desktop platform check to store() delay logic

Converted to draft because sync taking forever--not successful yet


use apply and ignore flags together

It should error out

This is my fault so I'll asign it to myself. Unless you are doing it... @rrd108

@David-Pena Why did ypu removed that error? I can't see the reason from the code, but I guessyou had a reason :smile:

@David-Pena Why did you removed that error? I can't see the reason from the code, but I guessyou had a reason 😄

I was stuck receiving the error of both being applied when trying some changes of the new logic, I couldn't understand the hierarchy of the command line and config file, so I was kind of stuck with that. That's why I deleted for the moment to fix it at the end.
But then I realized that even without that check the logic is still working because of the --apply/--ignore grouping per rule/ruleset changes. So the real question is, is it really necessary that check? Meaning, is is really necessary to not allow apply and ignore?
Maybe adding some tests or finding edge-cases where the current logic without the check fails will helps us decide. But I'm all ears @rrd108 🙌

I was stuck receiving the error of both being applied when trying some changes of the new logic, I couldn't understand the hierarchy of the command line and config file, so I was kind of stuck with that. That's why I deleted for the moment to fix it at the end.

I see. My test was crappy that is why it passed. My bad.
The hierarchy is the following (maybe we should state it more clerly in the docs?)

The analízer has its own defaults (weekest)
If it finds a config file the values in it will overwrite the defaults
If the user provided some cli flags it will overwrite the previous ones. (strongest)

But then I realized that even without that check the logic is still working because of the --apply/--ignore grouping per rule/ruleset changes. So the real question is, is it really necessary that check? Meaning, is is really necessary to not allow apply and ignore?

Using both apply and ignore will be confusing.
What is the expected behaviour of this?
yarn analyze -i=rrd,vue-strong -a=rrd

As the ignore flag states use everything but the ignored and apply means only the selected one(s) - it will end up check nothing.
I can imagine only one scenarion when using both can be useful, when I want to check one ruleset and ignore one rule from that ruleset.
What do you think?

I created a PR for this #198


Bug webpage freezes when running a test

I'm doing some practice at the link below but when I pressed "Run The Test" button the webpage froze
https://www.freecodecamp.org/learn/responsive-web-design/basic-css/understand-absolute-versus-relative-units

Browser Name: Firefox
Browser Version: 70.0
Operating System: Linux Mint 18.3 (Sylvia)

When it happens: The issue happens if the test isn't done correctly, that is, if the padding of .red-box is not equal to 1.5em. If you set the padding to anything other than 1.5em or don't set the padding at all, the page freezes with some process running on an infinite loop probably (my laptop's fan started to turn on full speed).
Can be reproduced on the following browsers:

Browser Name: Google Chrome
Browser Version: 78.0.3904.70 (Official Build) (64-bit)

Browser Name: Chromium Browser
Browser Version: 78.0.3904.70 (Official Build) snap (64-bit)

Browser Name: Brave Browser
Browser Version: 0.69.135 Chromium: 77.0.3865.120 (Official Build) (64-bit)

Browser Name: Firefox
Browser Version: 70.0 (64-bit) Canonical
Operating System: Ubuntu 19.10

Definitely a regex issue -- pasting it into regex101.com results in catastrophic backtracking. My first guess is the capture group (\s|.)*? just goes to infinity, happy to take a closer look.

When it happens: The issue happens if the test isn't done correctly, that is, if the padding of .red-box is not equal to 1.5em. If you set the padding to anything other than 1.5em or don't set the padding at all, the page freezes with some process running on an infinite loop probably (my laptop's fan started to turn on full speed).
Can be reproduced on the following browsers:
* Browser Name: Google Chrome

* Browser Version: 78.0.3904.70 (Official Build) (64-bit)

* Browser Name: Chromium Browser

* Browser Version: 78.0.3904.70 (Official Build) snap (64-bit)

* Browser Name: Brave Browser

* Browser Version: 0.69.135 Chromium: 77.0.3865.120 (Official Build) (64-bit)

* Browser Name: Firefox

* Browser Version: 70.0 (64-bit) Canonical

* Operating System: Ubuntu 19.10

I already made sure I put the padding with 1.5em within the read-box css class

When it happens: The issue happens if the test isn't done correctly, that is, if the padding of .red-box is not equal to 1.5em. If you set the padding to anything other than 1.5em or don't set the padding at all, the page freezes with some process running on an infinite loop probably (my laptop's fan started to turn on full speed).
Can be reproduced on the following browsers:
* Browser Name: Google Chrome

* Browser Version: 78.0.3904.70 (Official Build) (64-bit)

* Browser Name: Chromium Browser

* Browser Version: 78.0.3904.70 (Official Build) snap (64-bit)

* Browser Name: Brave Browser

* Browser Version: 0.69.135 Chromium: 77.0.3865.120 (Official Build) (64-bit)

* Browser Name: Firefox

* Browser Version: 70.0 (64-bit) Canonical

* Operating System: Ubuntu 19.10

I already made sure I put the padding with 1.5em within the read-box css class

Then there might be another issue, but when I tested, the issue only happened when the padding wasn't equal to 1.5em. If it was equal to 1.5em, the test passed and the modal with the save progress and go to next challenge came up and worked correctly.

This could be helped with #36793 landing, but the current automated challenge test suite needs to be replaced for the PR to be accepted.
A potential solution is coming, but time estimates are long.

@moT01 Why did PR #37575 close this issue?

Not sure, go give it a test. My browser crashed on production and didn't when looking at this PR.

@RandellDawson #37575 fixes a bad capture group in the regex for the solution test that was capable of causing stack overflow


Add support for Orange pi plus 2e

connects to https://github.com/resin-os/resin-os-device-support/issues/24

The actual supported device is Orange Pi Plus2 which is a little different from Orange Pi Plus2e.
Probably the image works with Orange Pi Plus2e also but we need to test this assumption to be sure. We don't have a Orange Pi Plus2e board.

@spanceac according to this forum post the Orange Pi Plus2 image works on Orange Pi Plus2e.


auth WindowsAD timeout

Describe the Bug
Add a login auth on windows ad
Steps To Reproduce
New-PodeAuthScheme -Basic | Add-PodeAuthWindowsAd -Name 'Login'
Expected Behavior
Add-PodeRoute -Method Get -Path '/api/user/:username/invite' -Authentication 'Login' -ScriptBlock {
#Some stuff here for this route
}
Platform

OS: Windows Server 2019 domain joined
Browser: Firefox and Chrome
Versions:

Pode: 2.4.2
PowerShell: 5.2

Additional Context
start pode on the shell with

start pode -verbose

Nothing happen until the client received a timeout...
Probably I miss something... thanks for your help

Hi @phand00,
I'm guessing the timeout is from when the /api/user/:username/invite route is called?
If your AD happens to have a lot of groups, try using -NoGroups on Add-PodeAuthWindowsAd:
New-PodeAuthScheme -Basic | Add-PodeAuthWindowsAd -Name 'Login' -NoGroups

This will stop Pode from getting all the groups a user is a member of, and should speed up the auth :)
Hope that helps!

Hi Badgerati,
I confirm that the -NoGroups works. But, it also means i won't be able to check the groups of the user? Is there a way to handle or optimize the request done by the WindowsAD module?
I just don't understarnd why it's taking so long to retreive all the groups of the user and if I do a simple ldap query or net user /domain I was able to retreive if pretty fast. I know i have thousands of groups in our ad. But each user a finite number of groups. I test it with a user that only has 10 groups and I have a timeout.
Thanks for your help
Thanks

Hi @phand00,
It's because the query at the moment is recursive. The user might only be in 10 groups directly, but those groups can be in groups, and they can be in groups, etc. So it depends how deep that tree goes.
The query being used is the fastest I could find, so if there's a way to optimise it then suggestions are welcome!
$query = "(member:1.2.840.1135<IP_ADDRESS>1:=$($DistinguishedName))"
$Connection.Searcher = New-Object System.DirectoryServices.DirectorySearcher $Connection.Entry
$null = $Connection.Searcher.PropertiesToLoad.Add('samaccountname')
$Connection.Searcher.filter = $query
$groups = @($Connection.Searcher.FindAll().Properties.samaccountname)

I have noted down a third option for groups though, which is to only get the direct groups - because that would be much faster.
At the moment there's no way to customise it, without editing the module.

Hi Badgerati,
I would say that in powershell we can do something like this and it's pretty fast: https://www.sysadmins.lv/blog-en/efficient-way-to-get-ad-user-membership-recursively-with-powershell.aspx
I see that you hare using ldap query and not AD cmdlet, but we can probably do the same loop validation as the powershell example with ldap query.
Like you suggest in your previous post, I would also suggest a -Recursive or -RecursiveGroup or -NestedGroup parameter. Leaving the -Group non recursive might be a nice addon too.
And finally. Adding something like -Properties and being able to supply additional attributes to return into the user hash table might be very useful in the scriptblock section. For example, we might add -Properties extensionattribute14 and in the scriptblock we can validate the content of the hash key $user["extensionattribute14"] for a specific value?

Hi @phand00,
That's actually the way the old logic use to work before I found the member:1.2.840.1135<IP_ADDRESS>1 shortcut; the recursive method like that works, but it was actually much slower! It could be different with the ActiveDirectory module, but as you've spotted Pode uses the DirectoryServices methods. The reason for using the DS methods was so people didn't have to install RSAT as a requirement.
The switch would have to be -DirectGroups, and leave current groups logic as recursive. Changing the existing logic to direct instead of recursive could break anybody who currently relies on that implementation. (I can note down to switch this logic to direct first in a future v3.0 release though).
Yep, totally agree with -Property! :)
I think there could be a couple ways of speeding this up then:

Add a -DirectGroups switch to get groups non-recursively
Check if the ActiveDirectory module exists when Add-PodeAuthWindowsAd is called (on Windows). Or, similar to how there's an -OpenLDAP switch, have an ADModule switch.
a. If it existing (or switch supplied), Import-Module (and export for Pode, if auto-imports restricted)
b. Flag that we're using the AD module
c. When it comes to authenticating a user, instead of using the DS methods, use the AD module

Combination of the two should help to speed things up 🤔

ok
I'll check for this in futur release. As of now, I'll implement in my own scriptblock a manual check with ad cmdlet.
Thanks a lot for your help.
And keep up the good work. Pode is a fantastic tool with a lot of potential.

@Badgerati I have an idea, although full disclosure, I don't use this feature at all, yet.

Option 4: Cache user-group memberships periodically, using a timer-based task. Increase the number of threads on the Pode server, to free up additional runspaces for executing the background task and responding to API routes. Calls to the API route reads from JSON cache instead of directly from Active Directory. Provide a feature flag to enable or disable caching of user-group memberships.

@pcgeek86 Ooo, it'd probably have to be a schedule so it doesn't impact anything else; I can see it taking quite some time to run for some domains!
But as an optional switch, it should be feasible 🤔  I'll have a play with the idea when I get chance, thanks!

Hi @phand00,
I've done some work on this, and in the above commit there are now -DirectGroups and -ADModule switches available on Add-PodeAuthWindowsAd.
I've done some initial testing with them, and each seems to work. When you specify -ADModule this will auto-import the AD module for you as well.
So:
New-PodeAuthScheme -Basic | Add-PodeAuthWindowsAd -Name 'Login' -DirectGroups
New-PodeAuthScheme -Basic | Add-PodeAuthWindowsAd -Name 'Login' -ADModule
New-PodeAuthScheme -Basic | Add-PodeAuthWindowsAd -Name 'Login' -ADModule -DirectGroups

# etc


GET 500 when creating "Conversational Core" bot (new creation)

Using new creation experience, select "Conversation Core".  Eventually get a "500" error in Composer.  In the command prompt, it's from one of the GETS.
Looks like:
GET /api/status/6ec66669-0c29-49e1-acb9-1a923f13b15b 500 1.124 ms - -
Version
Latest from main.  Running local build.
Commit: ca41b20
Browser

[ ] Electron distribution
[ ] Chrome
[ ] Safari
[ ] Firefox
[x ] Edge

OS

[ ] macOS
[x] Windows
[ ] Ubuntu

I also get this on Mac
GET /api/status/85baaa02-c76d-4b13-a732-df91a9dc9072 500 0.675 ms - -

I'm also getting this on Mac running Version:  1.3.1-nightly.217012.67a05de

This is due to the fact that the conversational core packages required by this module are published in a private feed that Composer cannot access.
Once they are published publicly, this should work.

@tracyboehrer @mewa1024 @sangwoohaan as these components are now all public,  this should now be fixed.  can you please give it another try?

@tracyboehrer  can you confirm this is working in the nightly build?

@benbrown I can confirm that as of Friday with a build from main this now works.  I assume because the packages were published.

thanks!


Add IPFS and Redis in Advanced TOC

Issue
Add IPFS and Redis in Advanced TOC: https://www.manual.grid.tf/advanced/advanced.html

Issue fixed when #361 is merged

Closing this as #361 is merged.


Add basic latency benchmarks in CI

We could show how a PR impacts

fill time
seal time
the root hash specifically

#15 includes a benchmark for the signing of the block.

Adding benchmarks for root hash in https://github.com/flashbots/rbuilder/pull/292


An error was encountered while running DADA2 in R (return code -6)

Hi there, have had a few new issues with latest version and now am stuck on the final issue.
nextflow.config:
process {
  executor = 'slurm'
  queue = 'defq'
}
params.dada2_cpu="256"
params.vsearch_cpu="256"
params.cutadapt_cpu="256"

// CPU limit if using local executor
process {
  name = "Local"
  cpus = 32
}

// Set cpus and memory. Process in "main.nf" will
// contain label corresponding to one of these unless
// the CPUs can be specified via specific parameters
// (e.g. --vsearch_cpu)
process {
  withLabel: cpu_def {
    cpus = 32
    memory = 128.GB
  }

  withLabel: cpu8 {
    cpus = 64
    memory = 256.GB
  }

  withLabel: cpu32 {
    cpus = 256
    memory = 1024.GB
  }
}

profiles {
  standard {
      params.enable_conda = false
      params.enable_container=false
      singularity.enabled = false
      singularity.automounts = false
      docker.enabled = false
      podman.enabled = false
      shifter.enabled = false
      charliecloud.enabled = false
  }
  conda {
    conda {
      useMamba = false
      conda.enabled = true
      // Allow longer conda creation timeout
      createTimeout = '24 h'
      cacheDir = "$HOME/.conda/envs"
    }
    params.enable_conda = true
    params.enable_container=false
    singularity.enabled = false
    singularity.automounts = false
    docker.enabled = false
    podman.enabled = false
    shifter.enabled = false
    charliecloud.enabled = false
  }
  singularity {
    singularity.enabled = true
    singularity.autoMounts = true
    singularity.cacheDir = "$HOME/.conda/envs/singularity"
    params.enable_container=true
    docker.enabled = false
    podman.enabled = false
    shifter.enabled = false
    charliecloud.enabled = false
  }
  docker {
    singularity.enabled = false
    singularity.autoMounts = false
    docker.enabled = true
    params.enable_container=true
    podman.enabled = false
    shifter.enabled = false
    charliecloud.enabled = false
  }
}

// Generate report
report {
  enabled = true
  file = "$params.outdir/report.html"
  overwrite = true
}
// Timeline
timeline {
  enabled = true
  file = "$params.outdir/timeline.html"
  overwrite = true
}
// DAG
dag {
  enabled = true
  file = "$params.outdir/dag.html"
  overwrite = true
}

First Issue: Error executing process > 'pb16S:cutadapt (1)'

Error output:
Caused by:
  Failed to create Conda environment
  command: conda env create --prefix ~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160 --file ${pb_16S_nf_base}/env/qiime2-2022.2-py38-linux-conda.yml
  status : 120
  message:

(message is literally several empty lines).  The job took a long time on this step, perhaps over 2 hours, and I remember seeing createTimeout = '2 h' in nextflow.config.  I updated that to 24 h and restarted.  Then it only took about 20 minutes and got past this step and on to the second issue.

Second Issue: Error executing process > 'pb16S:merge_sample_manifest' [csvtk: command not found]

Error output:
Caused by:
  Process `pb16S:merge_sample_manifest` terminated with an error exit status (127)

Command executed:

  csvtk concat -t samplefile.txt > merged_sample_file.txt

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 2: csvtk: command not found

Work dir:
  ${name}_working/work/73/981c2869a4629afcd01ea635a6ff2b

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

Even though csvtk is already installed in the pb-16S-pbtools environment, that was not sufficient.  I had to create and activate new conda env for pb-16S-nf and installed csvtk into there,
$ conda create -n pb-16S-nf
[...]
$ conda activate pb-16S-nf
$ conda install -c bioconda csvtk
[...]
The following NEW packages will be INSTALLED:

  csvtk              bioconda/linux-64::csvtk-0.25.0-h9ee0642_0
[...]
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
$

That version of csvtk is newer than the one in pb-16S-pbtools env so not sure if that helped or what, the error indicates csvtk could not even be found, even though it's at ~/.conda/envs/pb-16S-pbtools-79aaa66f28337ea289e1eec2f1ea2739/bin/csvtk.  Either way, upon restarting, this step was completed, leaving us at the final issue.

Third Issue: Error executing process > 'pb16S:dada2_denoise (1)'

Full command and parameters:
$ alias pb_16S_nf="nextflow run ${pb_16S_nf_base}/main.nf -profile conda"
$ mkdir ${name}_working && cd ${name}_working
$ pb_16S_nf  --input ../${name}_inputs/${name}_samples.tsv  --metadata ../${name}_inputs/${name}_metadata.tsv  --outdir ../${name}_outputs

N E X T F L O W  ~  version 23.01.0-edge
Launching `${pb_16S_nf_base}/main.nf` [berserk_linnaeus] DSL2 - revision:<PHONE_NUMBER>
Only 1 sample. min_asv_sample and min_asv_totalfreq set to 0.

  Parameters set for pb-16S-nf pipeline for PacBio HiFi 16S
  =========================================================
  Number of samples in samples TSV: 1
  Filter input reads above Q: 20
  Trim primers with cutadapt: Yes
  Forward primer: AGRGTTYGATYMTGGCTCAG
  Reverse primer: AAGTCGTAACAAGGTARCY
  Minimum amplicon length filtered in DADA2: 1000
  Maximum amplicon length filtered in DADA2: 1600
  maxEE parameter for DADA2 filterAndTrim: 2
  minQ parameter for DADA2 filterAndTrim: 0
  Pooling method for DADA2 denoise process: pseudo
  Minimum number of samples required to keep any ASV: 0
  Minimum number of reads required to keep any ASV: 0
  Taxonomy sequence database for VSEARCH: ${pb_16S_nf_base}/databases/GTDB_ssu_all_r207.qza
  Taxonomy annotation database for VSEARCH: ${pb_16S_nf_base}/databases/GTDB_ssu_all_r207.taxonomy.qza
  Skip Naive Bayes classification: false
  SILVA database for Naive Bayes classifier: ${pb_16S_nf_base}/databases/silva_nr99_v138.1_wSpecies_train_set.fa.gz
  GTDB database for Naive Bayes classifier: ${pb_16S_nf_base}/databases/GTDB_bac120_arc53_ssu_r207_fullTaxo.fa.gz
  RefSeq + RDP database for Naive Bayes classifier: ${pb_16S_nf_base}/databases/RefSeq_16S_6-11-20_RDPv16_fullTaxo.fa.gz
  VSEARCH maxreject: 100
  VSEARCH maxaccept: 100
  VSEARCH perc-identity: 0.97
  QIIME 2 rarefaction curve sampling depth: null
  Number of threads specified for cutadapt: 256
  Number of threads specified for DADA2: 256
  Number of threads specified for VSEARCH: 256
  Script location for HTML report generation: ${pb_16S_nf_base}/scripts/visualize_biom.Rmd
  Container enabled via docker/singularity: false
  Version of Nextflow pipeline: 0.5

[ab/b27986] process > pb16S:write_log                   [100%] 1 of 1 ✔
[90/07120c] process > pb16S:QC_fastq (1)                [100%] 1 of 1 ✔
[e9/f2bd61] process > pb16S:cutadapt (1)                [100%] 1 of 1 ✔
[20/cd7cfa] process > pb16S:QC_fastq_post_trim (1)      [100%] 1 of 1 ✔
[0b/6dd519] process > pb16S:collect_QC                  [100%] 1 of 1 ✔
[ee/440c69] process > pb16S:prepare_qiime2_manifest (1) [100%] 1 of 1 ✔
[0d/e4a74a] process > pb16S:merge_sample_manifest       [100%] 1 of 1 ✔
[f7/776596] process > pb16S:import_qiime2 (1)           [100%] 1 of 1 ✔
[d9/c83ad6] process > pb16S:demux_summarize (1)         [100%] 1 of 1 ✔
[d1/3ac7f3] process > pb16S:dada2_denoise (1)           [100%] 1 of 1, failed: 1 ✘

Error output:
Error executing process > 'pb16S:dada2_denoise (1)'

Caused by:
  Process `pb16S:dada2_denoise (1)` terminated with an error exit status (1)

Command executed:

  # Use custom script that can skip primer trimming
  mkdir -p dada2_custom_script
  cp run_dada_ccs.R dada2_custom_script/run_dada_ccs_original.R
  sed 's/minQ\ =\ 0/minQ=0/g' dada2_custom_script/run_dada_ccs_original.R >     dada2_custom_script/run_dada_ccs.R
  chmod +x dada2_custom_script/run_dada_ccs.R
  export PATH="./dada2_custom_script:$PATH"
  which run_dada_ccs.R
  qiime dada2 denoise-ccs --i-demultiplexed-seqs samples.qza     --o-table dada2-ccs_table.qza     --o-representative-sequences dada2-ccs_rep.qza     --o-denoising-stats dada2-ccs_stats.qza     --p-min-len 1000 --p-max-len 1600     --p-max-ee 2     --p-front 'none'     --p-adapter 'none'     --p-n-threads 256     --p-pooling-method 'pseudo'

Command exit status:
  1

Command output:
  dada2_custom_script/run_dada_ccs.R

Command error:
  Plugin error from dada2:

    An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

  Debug info has been saved to /tmp/qiime2-q2cli-err-bp4mht2v.log

Work dir:
  ${name}_working/work/d1/3ac7f378a25410b1e8b34a1bf06dab

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

First of all, there was no /tmp/qiime2-q2cli-err-bp4mht2v.log to inspect, and when I checked the other files, there was not anything that helped me get past this:
$ cat /tmp/qiime2-q2cli-err-bp4mht2v.log
/tmp/qiime2-q2cli-err-bp4mht2v.log: No such file or directory

$ cd ${name}_working/work/d1/3ac7f378a25410b1e8b34a1bf06dab
$ ls -lah
  0 Feb 13 18:15 .command.begin
209 Feb 13 18:55 .command.err
378 Feb 13 18:55 .command.log
169 Feb 13 18:16 .command.out
12K Feb 13 18:15 .command.run
718 Feb 13 18:15 .command.sh
  0 Feb 13 18:16 .command.trace
  1 Feb 13 18:55 .exitcode
271 Feb 13 18:40 dada2.log
 71 Feb 13 18:16 dada2_custom_script/
 88 Feb 13 18:16 run_dada_ccs.R -> ${pb_16S_nf_base}/scripts/run_dada_ccs.R*
145 Feb 13 18:16 samples.qza -> ${name}_working/work/f7/7765961d4d1af2b8d075f2978db691/samples.qza

$ cat dada2.log
R version 4.1.3 (2022-03-10)
Loading required package: Rcpp
DADA2: 1.22.0 / Rcpp: <IP_ADDRESS> / RcppParallel: 5.1.5
1) Removing Primers
2) Filtering
.
3) Learning Error Rates
4592234917 total bases in 3216241 reads from 1 samples will be used for learning the error rates.
$

The rest of the data was already shown:

.command.sh == Command executed:
.exitcode == Command exit status:
.command.out == Command output:
.command.err == Command error:
.command.log == Command output: + Command error:

So any ideas what the real issue is or where else to check?
Thanks

Hi @AGI-chandler , thanks for sharing your observations. Looks like Conda is messing up some of those packages. I did not change the environmental YAML file in the new version so I'm not sure what happened there. I typically very much prefer docker/singularity to avoid issues like this but glad to know that you got it working and thanks for sharing those steps.
For the denoise issue, can you check if there's anything inside ".command.err" and ".command.out"? The "/tmp/qiime2-q2cli-err-bp4mht2v.log" file might be on the compute node that runs the job if you're using the cluster mode, but otherwise I've seen in the past QIIME 2 has this weird issue of deleting the tmp file after it fails and asking users to check...
Can you also try running on the test samples just to rule out any weird issue?
Thanks.

The second issue is a bug I missed out asking the processes to use the correct environment. Will submit a pull request in a bit.

Hi @proteinosome thanks for your suggestions so quick tonight!  That's good to know regarding second issue.  I am used to the old way of installing apps system-wide for multi-user environments so unfortunately sometimes with these newer applications like nextflow, stuff is only accessible from the root account and not able to be run from regular user account.  I also did find lots of duplicate data in /root/.nextflow, /root/.conda, /root/nf_conda, ~/.nexflow, ~/nf_conda, and ~/.conda/envs so I did uninstall and clear those out in case it caused any conflicts or confusion.  Now the environment feels much cleaner to me and I configured pb-16S-nf to use ~/.conda/envs.
We did try docker originally but that was not compatible in our cluster environment in rootless mode.  I believe we need rootless mode so other users can use docker as well.  I haven't used singularity before but if it is something that can run from user account with slurm then I can check into it for you.
Regarding the issue, there was nothing extra from .command.out or .command.err as those were already fully printed under Command output: and Command error:....
$ cat .command.out
${name}_working/work/d1/3ac7f378a25410b1e8b34a1bf06dab/dada2_custom_script/run_dada_ccs.R

$ cat .command.err
Plugin error from dada2:

  An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-bp4mht2v.log

and you were right the temp file was hiding on one of the nodes!  but, the contents are not anything promising:
double free or corruption (out)
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_ccs.R /tmp/qiime2-archive-rujplarq/0ac7d821-4908-477f-9a64-f5b412d9cb07/data /tmp/tmprq5m3s7r/output.tsv.biom /tmp/tmprq5m3s7r/track.tsv /tmp/tmprq5m3s7r/nop /tmp/tmprq5m3s7r/filt none none 2 False 0 0 2 2 1000 1600 pseudo consensus 3.5<PHONE_NUMBER>

Traceback (most recent call last):
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 361, in denoise_ccs
    run_commands([cmd])
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada_ccs.R', '/tmp/qiime2-archive-rujplarq/0ac7d821-4908-477f-9a64-f5b412d9cb07/data', '/tmp/tmprq5m3s7r/output.tsv.biom', '/tmp/tmprq5m3s7r/track.tsv', '/tmp/tmprq5m3s7r/nop', '/tmp/tmprq5m3s7r/filt', 'none', 'none', '2', 'False', '0', '0', '2', '2', '1000', '1600', 'pseudo', 'consensus', '3.5', '256', '1000000']' died with <Signals.SIGABRT: 6>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/site-packages/q2cli/commands.py", line 339, in __call__
    results = action(**arguments)
  File "<decorator-gen-191>", line 2, in denoise_ccs
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    outputs = self._callable_executor_(scope, callable_args,
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/site-packages/qiime2/sdk/action.py", line 391, in _callable_executor_
    output_views = self._callable(**view_args)
  File "~/.conda/envs/qiime2-2022.2-py38-linux-conda-0f9d5d3ab8a678e45d02e511ce6c0160/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 370, in denoise_ccs
    raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code -6), please inspect stdout and stderr to learn more.

SIGABRT?  That's not good.  Yes in the system log it has:
Feb 13 18:54:58 n012 systemd[1]: Started Process Core Dump (PID 4138769/UID 0).
Feb 13 18:55:00 n012 systemd-coredump[4138770]: Core file was truncated to<PHONE_NUMBER> bytes.
Feb 13 18:55:04 n012 systemd-coredump[4138770]: Process 4121883 (R) of user 10063 dumped core.

Stack trace of thread 4121883:
#0  0x00007ffff756237f n/a (n/a)
Feb 13 18:55:04 n012 systemd[1]<EMAIL_ADDRESS>Succeeded.

and I have this file but not sure what to do with it:
323M Feb 13 18:55 core.R.10063.cd89d332292b4d7796dc20df80e22b7a.4121883.1676339698000000.lz4

I'll be looking into that next...

I looked back at the log you showed above again and it seems like this is a run of one sample with 3 million reads? That might be way too many reads for one single sample and crashes it. Can you try downsampling the FASTQ and see if that gets it through?

I could, for testing purposes, but the 3 million reads came from your machine and all of the data needs to be processed, somehow, I'm sure.  How should I downsample it and/or what extra options should I add to the nextflow command?
I figured out how to analyze the crash dump from previous iteration, but couldn't because it was truncated at 2 GiB: expected core file size >=<PHONE_NUMBER>88, found:<PHONE_NUMBER>.  So I set the max size to 150 GiB and restarted the pipeline to see if the new dump can be analyzed at least.

and we're just in time for the coredump, my modifications helped... here is the stack trace if it helps:
                #0  0x00007ffff756237f raise (libc.so.6)
                #1  0x00007ffff754cdb5 abort (libc.so.6)
                #2  0x00007ffff75a54e7 __libc_message (libc.so.6)
                #3  0x00007ffff75ac5ec malloc_printerr (libc.so.6)
                #4  0x00007ffff75ae390 _int_free (libc.so.6)
                #5  0x00007fffe03d7335 _Z12dada_uniquesSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS5_EES_IiSaIiEES_IbSaIbEEN4Rcpp6MatrixILi14ENSC_15PreserveStorageEEESF_iiibdidddbidiibbbibbibb (dada2.so)
                #6  0x00007fffe03d237f _dada2_dada_uniques (dada2.so)
                #7  0x00007ffff79ef1ea R_doDotCall (libR.so)
                #8  0x00007ffff79efc36 do_dotcall (libR.so)
                #9  0x00007ffff7a2dda1 bcEval (libR.so)
                #10 0x00007ffff7a48560 Rf_eval (libR.so)
                #11 0x00007ffff7a4a2bf R_execClosure (libR.so)
                #12 0x00007ffff7a4b08e Rf_applyClosure (libR.so)
                #13 0x00007ffff7a37d41 bcEval (libR.so)
                #14 0x00007ffff7a48560 Rf_eval (libR.so)
                #15 0x00007ffff7a4a2bf R_execClosure (libR.so)
                #16 0x00007ffff7a4b08e Rf_applyClosure (libR.so)
                #17 0x00007ffff7a37d41 bcEval (libR.so)
                #18 0x00007ffff7a48560 Rf_eval (libR.so)
                #19 0x00007ffff7a4a2bf R_execClosure (libR.so)
                #20 0x00007ffff7a4b08e Rf_applyClosure (libR.so)
                #21 0x00007ffff7a48791 Rf_eval (libR.so)
                #22 0x00007ffff7a49087 forcePromise (libR.so)
                #23 0x00007ffff7a49368 getvar (libR.so)
                #24 0x00007ffff7a31a02 bcEval (libR.so)
                #25 0x00007ffff7a48560 Rf_eval (libR.so)
                #26 0x00007ffff7a49087 forcePromise (libR.so)
                #27 0x00007ffff7a49368 getvar (libR.so)
                #28 0x00007ffff7a31a02 bcEval (libR.so)
                #29 0x00007ffff7a48560 Rf_eval (libR.so)
                #30 0x00007ffff7a4a2bf R_execClosure (libR.so)
                #31 0x00007ffff7a4b08e Rf_applyClosure (libR.so)
                #32 0x00007ffff7a37d41 bcEval (libR.so)
                #33 0x00007ffff7a48560 Rf_eval (libR.so)
                #34 0x00007ffff7a4a2bf R_execClosure (libR.so)
                #35 0x00007ffff7a4b08e Rf_applyClosure (libR.so)
                #36 0x00007ffff7a48791 Rf_eval (libR.so)
                #37 0x00007ffff7a4cd94 do_set (libR.so)
                #38 0x00007ffff7a48a99 Rf_eval (libR.so)
                #39 0x00007ffff7a7d336 Rf_ReplIteration (libR.so)
                #40 0x00007ffff7a7d6f9 R_ReplConsole (libR.so)
                #41 0x00007ffff7a7d7a9 run_Rmainloop (libR.so)
                #42 0x000055555555505d main (R)
                #43 0x00007ffff754e493 __libc_start_main (libc.so.6)
                #44 0x000055555555508d _start (R)

It says I need to install debuginfos if that isn't enough info...

HI @AGI-chandler I hear you. Unfortunately it's not usual for 16S sample to have that high of a depth. Even for a species with 0.01%, sequencing 100k depth would already give you on average 10 reads. 3 millions are overkill in most scenario unless there's a very very specific need. I think the softwares are just not designed to handle that amount of reads per sample.
I would first maybe downsample to 100k just to see if we can get the sample to analyze finish. The other approach is to split the 3 million reads into "chunks", so maybe 100 sets of 30k reads each, and run them through the pipeline (set "condition" to be identical for all 100 samples). The pipeline will be able to make use of information from these "chunks" for discovery purpose so you won't miss much by doing that. The researcher will just have to merge the results in the end. Does that make sense?

Just let me know how to do it and I'll pass the info along to the team

You can use seqkit split2 -s 50000 -O OUTPUTDIR SAMPLE.fastq.gz which will split SAMPLE.fastq.gz into multiple parts containing 50,000 reads each into the folder OUTPUTDIR. Then, create one sample manifest with each sample pointing to a different parts output by seqkit, e.g.:
sample-id    absolute-filepath
sample1    /path/to/part1.fastq.gz
sample2    /path/to/part2.fastq.gz

And your metadata file:
sample_name    condition
sample1    replicate
sample2    replicate

And run the pipeline on the new sample manifest and metadata file.
Thanks.

Hi @proteinosome, than you for helping us troubleshoot the issues. I work with @AGI-chandler
The issue with the number of reads per sample may be easy to solve, once we figure out how to set up the metadata.
We are analyzing data from a single Sequel II SMRT cell, with about 4.6 million HiFi reads (input, from the sequencer). There are 96 samples, all different from each other, and we don't know anything about them. This is why we treated them as a single "condition" per your documentation.
Besides downsampling (which is not the real answer in our case I think), how else can we parse data from a cell that has 96 different samples, all related to the same condition/experiment?
Soon we will sequence another set of 192 samples, obtained with PacBio's 24 x 8 primer set.
Looking forward to your opinion,
Dario

Hi @dcopetti did you barcode the samples? You need to demultiplex them first so you have 96 FASTQs.

Hi Chua,
Yes, the amplicons were generated with asymmetric barcodes as documented here:
https://www.pacb.com/wp-content/uploads/Procedure-checklist-Amplification-of-bacterial-full-length-16S-rRNA-gene-with-barcoded-primers.pdf
We used the set 13-24 as reverse primer.
No, we did not demultiplex the samples before running the pipeline. The test samples we run (16 samples, rev primers of columns 1 and 2) did the demultiplexing within it, we used the whole HiFi_reads as the input file.
are we missing something?

Hi @dcopetti , thank you for the clarification (Sorry for my weird response timing, I'm based in Singapore). The pipeline expects demultiplexed input. The sample sheet should contain one line per sample, each pointing to the sample's FASTQ. You will likely need to rerun your test samples, too, if you did not demultiplex. The "demultiplexing" step in this pipeline is used to trim the forward and reverse primers, but it does not look for samples barcodes. Do you know how to demultiplex PacBio's HiFi reads?

Hi Chua,
We found the issue, it was between the chair and the keyboard :-)
We will run the pipeline soon
Thanks for your support!

Glad to hear that! I will close this issue for now, then. Feel free to open another issue if you run into one (hopefully you don't!).


[package upgrades] Add fast-path package upgrade CLI command

This adds a fast-path upgrade for the Sui CLI I specifically didn't want to add it to the transaction builder/json-rpc API at the moment since we will probably want to to think that API through quite a bit more so we don't have to make too many breaking changes to it (and to allow custom upgrade policies).
Test Plan
Added a new test (there's some nastiness around published-at that's needed in there sadly), and exercised the upgrade path locally in the CLI as well.

Should be all fixed up now (apologies for the original issues!)


Fixes to compile ros indigo on Gentoo.

Requires overlays ros and Gentoo-ROS-overlay (https://github.com/steup/Gentoo-ROS-Overlay)

Our CI enforces alphabetical ordering: https://travis-ci.org/ros/rosdistro/builds/58676112#L192

I was a little too fast with making this pull request. Other than the ordering there is a one more problem with dependencies between urdfdom and urdfdom_headers... but I'll be back :-)
Btw I have a few questions:

What should I do with this pull request?

close it and open a new fixed one?
add new commits with the fixes?
rebase content of the first commit?

Can I run the checks that travis does locally?
At three places (libvtk-java, python-cairo and python-vtk) I'm using use: as in ros-infrastructure/rosdep#176, although It probably doesn't do anything. The problem is that there are no separate packages for language support of libraries and AFAIK the only way to get this is with changed use flags. Is it ok to keep them in the yaml files?

What should I do with this pull request?

I would close this, then when you're ready you can reopen it and update the branch it points to with the correct content.

Can I run the checks that travis does locally?

Yes, just do nosetest -s in the root of the repository.

At three places (libvtk-java, python-cairo and python-vtk) I'm using use: as in ros-infrastructure/rosdep#176, although It probably doesn't do anything. The problem is that there are no separate packages for language support of libraries and AFAIK the only way to get this is with changed use flags. Is it ok to keep them in the yaml files?

Yes, if you plan to eventually update rosdep to support them.


Add sampling

Training model rn to see if its even slightly sensible

let's get it actually working before we merge into main haha

let's get it actually working before we merge into main haha

Requirements is missing imageio

Requirements is missing imageio

This should be ready to merge now, but i'm just going to train some small models to make sure everything's working.

This should be ready to merge now, but i'm just going to train some small models to make sure everything's working.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 6 committers have signed the CLA.:x: sid:x: kingoflolz:x: ConnorJL:x: leogao2:x: lucidrains:x: sdtblcksid seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Compiling issue

Hello,
I installed the module '0.1.0' on Magento 2.4.3 p-1
Getting an error when compiling setup:di:compile
Impossible to process constructor argument Parameter #0 [ <required> Magento\Elasticsearch\Mode   l\ResourceModel\Fulltext\Collection\SearchResultApplierFactory $elasticSearchResultApplierFacto   ry ] of Amasty\Shopby\Model\ResourceModel\Fulltext\Collection\SearchResultApplier class
 Class Magento\Elasticsearch\Model\ResourceModel\Fulltext\Collection\SearchResultApplierFactory   does not exist
We have Amasty_LayeredNavigation installed on the site.

Hello,
You must take a look deeper in this module and validate that it doesn't require fulltext indexation from search engine to work correctly.
It looks like it does, so you cannot disable elasticsearch.

Thank you!


1.2.2

1.2.2

Coverage remained the same at 97.59% when pulling 4be2881245e736ac14203ca18237b23d1a3cd826 on development into 5da0d8679228c142003403654a3032ec6bb2dedc on master.


Use goreleaser for release

One liner to make the release: just type goreleaser.
This tool is pretty common now. https://goreleaser.com/
For local dev, we can also use it and do goreleaser --snapshot
Example output: https://github.com/guillaumerose/machine-driver-libvirt/releases/tag/0.12.12

No consensus on this. Closing it.

No consensus on this. Closing it.


Updated Symantec Endpoint Protection TemplateSpec to 2.0.1

Change(s):
See guidance below
Reason for Change(s):
This PR is part of template spec solution migration for solution Symantec Endpoint Protection.
See guidance below
Version Updated:
2.0.0
See guidance below
Testing Completed:
The link for test results is - Link
See guidance below
This is a 3P solution.

Hi @v-sabiraj
Can you please get confirmation on this PR merging or closing and act accordingly.
Thanks

Hi @v-sabiraj
Can you please get confirmation on this PR merging or closing and act accordingly.
Thanks

@v-atulyadav There are conflicts please resolve them. Thanks

@v-atulyadav: There are conflicts in this PR please resolve them. Thanks

Closing this PR as new PR# 6093 has already raised and merged


Huge number of notes (2400) - replication close

Hello
begining of sync goes well and then stops
05/01/2024, 12:03:29->↑5900 (115891) ↓0 05/01/2024, 12:03:32->↑5902 (115889) ↓0 05/01/2024, 12:03:33->Replication closed 05/01/2024, 12:03:34->↑5938 (115853) ↓0 05/01/2024, 13:37:35->OneShot Sync begin... (sync) 05/01/2024, 13:37:36->Looking for the point last synchronized point. 05/01/2024, 13:37:36->Replication activated 05/01/2024, 13:37:37->↑50 (115796) ↓0 05/01/2024, 13:37:40->↑51 (115795) ↓0 05/01/2024, 13:37:41->↑68 (115778) ↓0 05/01/2024, 13:37:43->↑100 (115746) ↓0 05/01/2024, 13:37:47->↑102 (115744) ↓0 05/01/2024, 13:37:51->Replication closed 05/01/2024, 13:37:52->↑103 (115743) ↓0
I missing a lot of notes ...
Thanks for help

The situation is unfortunate.

Is it reproducible?

Or is it out of memory on the CouchDB server side?
Volume usage is at 100%,
iCloud Drive is being used.
etc.

Please provide additional information about your environment.

In reference to New Issue inquiry? · vrtmrz/obsidian-livesync · Discussion #327, etc.
This will encourage more people to respond. It may increase the number of people willing to respond.
That is all. I wish you good luck.

docker	-	obsidian-livesync	running	12 hours
CPU 0,2
MEM 1.55G
DISK 138G used on	165G available
no icloud drive used (on windows for testing)
Photon os latest
Compose
version: "2.1" services:   couchdb:     image: couchdb:latest     container_name: obsidian-livesync     user: 1000:1000     environment:       - COUCHDB_USER=zzzzzzzzzzzzzzzzzzzzzzzz       - COUCHDB_PASSWORD=xxxxxxxxxxxxxxxxxxxxxx     volumes:       - ./data:/opt/couchdb/data       - ./local.ini:/opt/couchdb/etc/local.ini     ports:       - 9584:5984     restart: unless-stopped
local.ini
`[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size =<PHONE_NUMBER>
[chttpd_auth]
require_valid_user = true
authentication_redirect = /_utils/session.html
[httpd]
WWW-Authenticate = Basic realm="couchdb"
bind_address = <IP_ADDRESS>
enable_cors = true
[cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true
headers = accept, authorization, content-type, origin, referer
methods = GET,PUT,POST,HEAD,DELETE
max_age = 3600`

Thank you for opening this issue!
At first sight, I was very surprised to see that there was a tremendous number of chunks waiting to be sent.
May I ask the total file size of your vault?
And, If you have not configured an enhancing chunk size, please try setting 'Enhance chunk size' to 100 and rebuild the databases once. It works to reduce the number of chunks to 1/100.
To rebuild both databases,  we can place 'redflag3.md' on top of the vault (which has the fine notes and is backed up), and restart obsidian, instead of using 'Hatch'.
I hope that this information will work for us.

2gb I already worked with the chunks

I am so relieved to hear that! Sorry that your confusion had not been solved earlier!
I close this issue. Thank you for your patience!


Update to androidx support libraries

Depends on https://github.com/lkorth/device-automator/pull/3
Update to androidx support libraries.

This can be rebased against master now to drop the SDK and gradle upgrades from this branch since they were merged into master.
Breaking changes are fine on 0.x versions, but I'm fine creating a branch and final release of 0.x and then merging this and creating a 1.0.0.

Rebase is done 👍  I'm not partial to either versioning approach, so either is fine with me.

1.0.0 was just released.


Current version triggers no matching package found error

Hi!
I was updating my project current dependencies and I've found that the new version triggers this error. Any ideas of what may be failing?
endroid/installer 1.0.4 requires composer-plugin-api ^1.1 -> no matching package found.
Complete log track:
Installation request for endroid/installer 1.0.4 -> satisfiable by endroid/installer[1.0.4]. - endroid/installer 1.0.4 requires composer-plugin-api ^1.1 -> no matching package found. Problem 2 - endroid/installer 1.0.4 requires composer-plugin-api ^1.1 -> no matching package found. - endroid/qr-code 3.2.12 requires endroid/installer ^1.0.3 -> satisfiable by endroid/installer[1.0.4]. - Installation request for endroid/qr-code 3.2.12 -> satisfiable by endroid/qr-code[3.2.12]. Potential causes: - A typo in the package name - The package is not available in a stable-enough version according to your minimum-stability setting

Hoi @juanantoniodelgado looks like you are using an old version of composer. Please update composer and let me know if that helped. You can use composer self-update to update to the lastest version.


cosmetic changes in drivers/pci and drivers/sensors

Summary

drivers/pci: fix style issues in function headers
drivers/sensors: fix style issues in function headers

Impact
only cosmetic style changes
Testing
none

related issue https://github.com/apache/nuttx/issues/12133


import of DB doesn't work

Describe the bug
Install mysql, install antares, create new empty DB , import previously dumped DB. It now shows a modal that is doing something, however upon completion there is no trace of any imported data.
To Reproduce
See description above. If not clear I can add video.
Expected behavior
DB import works and i can see the imported tables and data.
Screenshots
Don't have atm, but can attach later.
Application (please complete the following information):

App client [e.g. MariaDB]
Installation source: [e.g. exe]

Environment (please complete the following information):

OS name: [e.g. Windows 10]
DB name  [e.g. MariaDB]

Additional context
Add any other context about the problem here.

Hi Danny,
Could you provide me a sql with a dump for which you get the problem?


look into derive-builder

alot of the code contained in this repo targets building structs for requests parameters
https://github.com/colin-kiegel/rust-derive-builder represents a path for replacing that with something generated which should result in a smaller more maintainable code base

Is this still the plan, @softprops?
If so I might just look into this.

It would likely save a lot of manually written "builder" code at the moment for building up requests.  I was deferring this until I had a few things worth making a breaking change for to make a breaking change worth it. A new version of hyper was just released so it now may be a fine time.

Still no sign of a tool to mechanically check if a change is breaking, right? I'm wondering how much can be converted to derive-builder without breaking compatibility.

Trimming issues. Closing for now. Will reopen when there's more motivation


RESTEASY-2734 The current WADL GRAMMAR generation logic incorrectly clean the schema types

Hey @liweinan,
Could you give me a quick hint about what problem you are solving?
Thanks,
Ron

@ronsigal This PR is to address two issues:

The previous ResteasyWadlDefaultResource  is not abstract, so its path can't be customized by users.(and the document is updated to address this: https://github.com/resteasy/Resteasy/pull/2589)
The previous implementation will cleanup the GRAMMAR section incorrectly which is bug that has to be fixed.


No interfaces obtained via qemu-agent

System Information
Linux distribution
Ubuntu 20.04
Terraform version
1.0.6
Provider and libvirt versions
0.6.2

Checklist

[ ] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?

[ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
[ ] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
[ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation

[X] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question
Setup
Terraform:
cloudedge.txt
Cloudinit:
network_config_static.txt
cloud_init.txt
Steps to Reproduce Issue
I want to create VMs and expose them to the local network via a network bridge br0. However, using Terraform the VMs are stuck on the following (from TF_LOG=DEBUG terraform apply)
2021-10-06T13:45:01.302Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] sending command to qemu-agent
2021-10-06T13:45:01.307Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] qemu-agent response: {"return":[{"name":"lo","ip-addresses":[{"ip-address-type":"ipv4","ip-address":"<IP_ADDRESS>","prefix":8},{"ip-address-type":"ipv6","ip-address":"::1","prefix":128}],"statistics":{"tx-packets":6,"tx-errs":0,"rx-bytes":416,"rx-dropped":0,"rx-packets":6,"rx-errs":0,"tx-bytes":416,"tx-dropped":0},"hardware-address":"00:00:00:00:00:00"},{"name":"eth0","ip-addresses":[{"ip-address-type":"ipv6","ip-address":"fe80::5054:ff:fe7e:962e","prefix":64}],"statistics":{"tx-packets":29,"tx-errs":0,"rx-bytes":27610,"rx-dropped":5,"rx-packets":224,"rx-errs":0,"tx-bytes":7838,"tx-dropped":0},"hardware-address":"52:54:00:7e:96:2e"}]}
2021-10-06T13:45:01.307Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] Parsed response {Interfaces:[{Name:lo Hwaddr:00:00:00:00:00:00 IPAddresses:[{Type:ipv4 Address:<IP_ADDRESS> Prefix:8} {Type:ipv6 Address:::1 Prefix:128}]} {Name:eth0 Hwaddr:52:54:00:7e:96:2e IPAddresses:[{Type:ipv6 Address:fe80::5054:ff:fe7e:962e Prefix:64}]}]}
2021-10-06T13:45:01.307Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] Interfaces obtained via qemu-agent: []
2021-10-06T13:45:01.307Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] no interfaces could be obtained with qemu-agent: falling back to the libvirt API
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] Interfaces info obtained with libvirt API:
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: ([]libvirt.DomainInterface) {
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: }
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local:
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] ifaces with addresses: []
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] 52:54:00:7E:96:2E doesn't have IP address(es) yet...
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [DEBUG] IP address not found for iface=52:54:00:7E:96:2E: will try in a while
2021-10-06T13:45:01.308Z [DEBUG] provider.terraform-provider-libvirt-local: 2021/10/06 13:45:01 [TRACE] Waiting 10s before next try

Resulting in
Error: Error: couldn't retrieve IP address of domain.

The VMs have qemu-guest-agent running on them, which gives back this list of network interfaces available on the VM, but this list is missing something / incorrect according to Terraform. The VM ends up having no IP (using virsh console to login to the VM, followed by `hostname -i```.

Additional information:
I have tried many permutations of the given terraform and cloudinit file, including other terraform-provider-libvirt versions, other operating systems for the VM, using static IPs and / or dynamic IPs via DHCP. At this point I am out of ideas and would like some help in figuring out what goes wrong.
I am able to create VMs with static IPs and a network bridge using the Qemu CLI (without Terraform).

Hi,
i think i have the same problem. I try to deploy a debian vm with this plugin but it runs into a timeout while trying to get the ip of the vm. On my dhcp server i can see that the vm successfully obtained an address but terraform isnt recognizing it.
my tf file:
terraform {
  required_version = ">= 0.13"
    required_providers {
      libvirt = {
        source  = "dmacvicar/libvirt"
        version = "0.6.11"
      }
    }
}

resource "libvirt_domain" "virt-machine" {
  count  = var.vm_count
  name   = format("${var.vm_hostname_prefix}%02d", count.index + var.index_start)
  memory = var.memory
  vcpu   = var.vcpu
  autostart  = var.autostart
  qemu_agent = true

  cloudinit = element(libvirt_cloudinit_disk.commoninit.*.id, count.index)

  network_interface {
    bridge         = var.bridge
    wait_for_lease = true
    hostname       = format("${var.vm_hostname_prefix}%02d", count.index + var.index_start)
  }

  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }

  console {
    type        = "pty"
    target_type = "virtio"
    target_port = "1"
  }

  disk {
    volume_id = element(libvirt_volume.volume-qcow2.*.id, count.index)
  }

  graphics {
    type        = "spice"
    listen_type = "address"
    autoport    = true
  }
}

while terraform tries to obtain the ip of the vm, i can already find it via virsh:
virsh qemu-agent-command nomad01 '{"execute": "guest-network-get-interfaces"}'
{"return":[{"name":"lo","ip-addresses":[{"ip-address-type":"ipv4","ip-address":"<IP_ADDRESS>","prefix":8},{"ip-address-type":"ipv6","ip-address":"::1","prefix":128}],"statistics":{"tx-packets":0,"tx-errs":0,"rx-bytes":0,"rx-dropped":0,"rx-packets":0,"rx-errs":0,"tx-bytes":0,"tx-dropped":0},"hardware-address":"00:00:00:00:00:00"},**{"name":"ens3","ip-addresses":[{"ip-address-type":"ipv4","ip-address":"<IP_ADDRESS>","prefix":24},{"ip-address-type":"ipv6","ip-address":"fe80::5054:ff:fe62:d25f","prefix":64}],"statistics":{"tx-packets":56,"tx-errs":0,"rx-bytes":70690,"rx-dropped":5,"rx-packets":743,"rx-errs":0,"tx-bytes":8065,"tx-dropped":0},"hardware-address":"52:54:00:62:d2:5f"}**,{"name":"docker0","ip-addresses":[{"ip-address-type":"ipv4","ip-address":"<IP_ADDRESS>","prefix":16}],"statistics":{"tx-packets":0,"tx-errs":0,"rx-bytes":0,"rx-dropped":0,"rx-packets":0,"rx-errs":0,"tx-bytes":0,"tx-dropped":0},"hardware-address":"02:42:3e:b2:81:46"}]}

in my cloud init file i also install, start and enable the qemu-guest-agent.service.
before terraform aborts i can connect to the vm via ssh and verify that its running:
$ sudo systemctl status qemu-guest-agent.service
● qemu-guest-agent.service - QEMU Guest Agent
     Loaded: loaded (/lib/systemd/system/qemu-guest-agent.service; static)
     Active: active (running) since Sun 2021-10-10 11:35:23 CEST; 23min ago
   Main PID: 455 (qemu-ga)
      Tasks: 2 (limit: 529)
     Memory: 2.3M
        CPU: 5ms
     CGroup: /system.slice/qemu-guest-agent.service
             └─455 /usr/sbin/qemu-ga

Oct 10 11:35:23 nomad01 systemd[1]: Started QEMU Guest Agent.

and here is the output from the terraform apply command:
2021-10-10T10:07:14.825Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] waiting for network address for iface=52:54:00:62:D2:5F
2021-10-10T10:07:14.826Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] Not implemented
2021-10-10T10:07:14.826Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] no interfaces could be obtained with qemu-agent: falling back to the libvirt API
2021-10-10T10:07:14.826Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] Interfaces info obtained with libvirt API:
2021-10-10T10:07:14.827Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: ([]libvirt.DomainInterface) <nil>
2021-10-10T10:07:14.827Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11:
2021-10-10T10:07:14.827Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] ifaces with addresses: []
2021-10-10T10:07:14.827Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] 52:54:00:62:D2:5F doesn't have IP address(es) yet...
2021-10-10T10:07:14.827Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [DEBUG] IP address not found for iface=52:54:00:62:D2:5F: will try in a while
2021-10-10T10:07:14.827Z [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/10/10 10:07:14 [TRACE] Waiting 10s before next try

I found this issue here: https://github.com/dmacvicar/terraform-provider-libvirt/issues/710 its closed because there was already a fix for a problem with channel bindings.

@mxkle , but this version would not work with terraform 1.x

Is there a version that supports the qemu-agent and works with terraform 1.0.x?

@everflux, no wait for https://github.com/dmacvicar/terraform-provider-libvirt/pull/ 873 is reviewed and merged in.

The PR is reviewed but it looks like it will not be merged, it is 1,5h years old and conflicts with the current main.
I can understand the requested changes by @dmacvicar but I am afraid that it won't materialize.

I have manually rebased and committed #873
I don't like the 120M image for the test, so the test is pending and I opened #913 to revisit it.


Integrate RDARSAT data

Issue Description
RADARSAT is a canadian SAR sensor that can be used for various geospatial analysis  like flood delineation
References
https://registry.opendata.aws/radarsat-1/
Dataset ID
No response
Screenshots
No response

https://aws.amazon.com/marketplace/pp/prodview-gfw3itcvzk2ca#overview


Custom launchSettings.json profile causes "Unable to configure HTTPS endpoint. No server certificate was specified"

Description
When creating a new standard blazor serverside (with docker) project, and adding the following to launchSettings.json:
"DockerTest": {
      "commandName": "Docker",
      "launchBrowser": true,
      "launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}",
      "publishAllPorts": true,
      "useSSL": true,
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "TEST"
      }
    }

I basically copied the entry for Docker, renamed Docker to DockerTest and added the environmentVariables
Then selecting that particular profile and run it, I get the following exception:

Happens both in Chrome and Firefox
App runs fine if I use the default Docker profile.
I tried to do what the exception said; and this is what the developer console spit out:

Configuration
I'm running .net 5.0.300 on windows 10, 20H2 build 19042.985
Visual Studio 2019 - 16,10,0
System.InvalidOperationException
HResult=0x80131509
Message=Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found or is out of date.
To generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.
For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
Source=Microsoft.AspNetCore.Server.Kestrel.Core
StackTrace:
at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, Action1 configureOptions)    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions)    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.<BindAsync>d__2.MoveNext()    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.<BindAsync>d__0.MoveNext()    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<BindAsync>d__32.MoveNext()    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<StartAsync>d__291.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.AspNetCore.Hosting.GenericWebHostService.d__31.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Extensions.Hosting.Internal.Host.d__9.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.d__4.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.d__4.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
at Admin.Program.Main(String[] args) in C:\Devops\ST-Test\Admin\Program.cs:line 16
This exception was originally thrown at this call stack:
Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions, System.Action<Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionAdapterOptions>)
Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions)
Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBindContext)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(System.Collections.Generic.IEnumerable<Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions>, Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBindContext)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
...
[Call Stack Truncated]

I did some experimenting and since you're switching the environment to one that is not from the predefined 3: Development, Staging, Production; it's somehow not able to pick up the trusted cert. I assume there's some piece of configuration that needs to be done for the new environment.
Is using your custom environment something that's necessary for you? I assume that if you switch back to "Development" it will work.
@Pilchie My knowledge here is lacking. Is this something unsupported? Or is there anything the customer can do to make it work?

@javiercn any ideas here?

@Pilchie It's a combo between the custom environment and Docker.
The way docker works from VS requires the environment to be Development. Otherwise we don't try to probe the filesystem for the developer certificate (for security reasons).
When you use a custom environment you need to setup the certificate explicitly via config.
We have docs here and here on how to do this setup.


Various single line fixes

Change v-btn in template

The first 3 commits are just about template, but the last one probably breaks alacarte at the moment.

Danke


Error in layouts with middleware

I'm using a layout that relies on the state of the vuex:
<template>

{{user.name}}

    <nuxt/>

</template>

Middleware auth check state.currentUser and redirect to login page if it's null
export default function ({ store, redirect }) {
  console.log('\nCheck user data ...\n')
  if (!store.state.currentUser) {
    return redirect('/login')
  }
}

So, now, if I'm not authorized and trying to load a page with this template, I get an error in the server console:
Check user data ...

[Vue warn]: Error in render function:
(found in <Sidebar> at /run/media/alex/LinWin/Develop/fb-scaner/resources/layouts/sidebar.vue)
TypeError: Cannot read property 'name' of null
    at Proxy.render (resources/layouts/sidebar.vue?7fdc:2:0)
    at VueComponent.Vue._render (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue/dist/vue.runtime.common.js:3605:22)
    at renderComponent (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:6698:25)
    at RenderContext.renderNode (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:6681:7)
    at RenderContext.next (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:169:14)
    at RenderContext.next (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:178:12)
    at cachedWrite (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:30:9)
    at renderElement (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:6732:5)
    at renderNode (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:6685:7)
    at renderComponent (/run/media/alex/LinWin/Develop/fb-scaner/node_modules/vue-server-renderer/build.js:6704:3)

The code of the page itself is very simple.
<template>

    Home page

</template>

Yes, this or the same error will reproduce with use store state as function.
All components on which use a state as a function will be broken and I don't know how to fix that. Maybe guys know how to fix that?

Hi, @cawa-93!
I think nuxt render layouts despite any middlewares. In my project, I just add v-if for components that used in layouts.

@KonstantinVlasov However, in the console I see a message about the execution of the Middleware. And after the error, there is a redirect. But I do not understand why an error occurs, if redirection is to be performed prior to rendering.

As I understand its prior to page rendering, but not layout rendering. Looks like it is intended. But maybe not :)

@cawa-93 are you using 2 different layouts, one for logged-in and one for logged-out users?

@cawa-93 This commit fix the issue: https://github.com/nuxt/nuxt.js/commit/ce69c3be0f544afa9b36b23b9e106631c9c51f70
It will be in the next release (alpha4).

@Atinux Yes In alpha4, no problem was observed. Today, I used alpha5 and the problem returned again.

Closing it since rc3 is out

 Test layouts middleware