ROCM version cannot be built

We are using Sirius on the ROCM platform with release v6.3.4.
However, ROCM version doesn't seems to be able to build successfully.
The build fail logs are appended below...
Also, we would like to ask if there is some comprehensive documentation on Sirius that covers
a list of unit tests that needs to be passed, including the hand to hand steps to run them, and what to be expect int the correct output?
[ 33%] Building CXX object src/CMakeFiles/sirius.dir/SDDK/wave_functions.cpp.o [ 33%] Building CXX object src/CMakeFiles/sirius.dir/Hubbard/hubbard_generate_atomic_orbitals.cpp.o [ 32%] Building CXX object src/CMakeFiles/sirius.dir/Band/diag_pseudo_potential.cpp.o [ 33%] Building CXX object src/CMakeFiles/sirius.dir/Potential/xc.cpp.o [ 20%] Building CXX object src/CMakeFiles/sirius.dir/SDDK/wf_trans.cpp.o [ 33%] Building CXX object src/CMakeFiles/sirius.dir/dft_ground_state.cpp.o [ 33%] Built target HIP_TARGET_random.cu In file included from /home/sarahhung/Sirius/SIRIUS/src/SHT/sht.hpp:39:0,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/hubbard_orbitals_descriptor.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom_type.hpp:35,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom_symmetry_class.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom.hpp:29,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/unit_cell.hpp:29,                  from /home/sarahhung/Sirius/SIRIUS/src/radial_integrals.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/radial_integrals.cpp:1: /home/sarahhung/Sirius/SIRIUS/src/SDDK/linalg.hpp: In member function ‘void sddk::linalg2::trmm(char, char, char, ftn_int, ftn_int, const T*, const T*, ftn_int, T*, ftn_int, stream_id) const [with T = double; ftn_int = int]’: /home/sarahhung/Sirius/SIRIUS/src/SDDK/linalg.hpp:214:87: error: too many arguments to function ‘void hipblas::dtrmm(char, char, char, char, int, int, const double*, const double*, int, double*, int)’              gpublas::dtrmm(side, uplo, transa, 'N', m, n, alpha, A, lda, B, ldb, sid());                                                                                ^ In file included from /home/sarahhung/Sirius/SIRIUS/src/SDDK/GPU/gpublas_interface.hpp:34:0,                  from /home/sarahhung/Sirius/SIRIUS/src/SDDK/linalg.hpp:30,                  from /home/sarahhung/Sirius/SIRIUS/src/SHT/sht.hpp:39,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/hubbard_orbitals_descriptor.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom_type.hpp:35,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom_symmetry_class.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom.hpp:29,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/unit_cell.hpp:29,                  from /home/sarahhung/Sirius/SIRIUS/src/radial_integrals.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/radial_integrals.cpp:1: /home/sarahhung/Sirius/SIRIUS/src/SDDK/GPU/hipblas_interface.hpp:232:13: note: declared here  inline void dtrmm(char side, char uplo, char transa, char diag, int m, int n, double const* alpha,              ^~~~~ In file included from /home/sarahhung/Sirius/SIRIUS/src/SHT/sht.hpp:39:0,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/hubbard_orbitals_descriptor.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom_type.hpp:35,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom_symmetry_class.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/atom.hpp:29,                  from /home/sarahhung/Sirius/SIRIUS/src/Unit_cell/unit_cell.hpp:29,                  from /home/sarahhung/Sirius/SIRIUS/src/radial_integrals.hpp:28,                  from /home/sarahhung/Sirius/SIRIUS/src/radial_integrals.cpp:1: /home/sarahhung/Sirius/SIRIUS/src/SDDK/linalg.hpp: In member function ‘void sddk::linalg2::trmm(char, char, char, ftn_int, ftn_int, const T*, const T*, ftn_int, T*, ftn_int, stream_id) const [with T = std::complex<double>; ftn_int = int]’: /home/sarahhung/Sirius/SIRIUS/src/SDDK/linalg.hpp:251:81: error: too many arguments to function ‘void hipblas::ztrmm(char, char, char, char, int, int, const hipDoubleComplex*, const hipDoubleComplex*, int, hipDoubleComplex*, int)’                            reinterpret_cast<acc_complex_double_t*>(B), ldb, sid());
\

@AdhocMan Simon, can you please have a look? As a side notice, I suggest to work with the latest develop branch.

The ROCm support at moment is still experimental. The initial implementation was done with for an older release of ROCm and it may require some changes now. Due to the missing functionalities, only a very limited subset could be run anyway (mainly the FFT part, which has now been separated into a new library SpFFT). Tests are available in the verification folder, but it is not yet possible to run them with ROCm.
The plan at moment is to wait for the missing parts before working on the ROCm integration again.
In particular, what we are missing is:

Complex number support in rocBLAS
Eigensolver
Additional LAPACK functionality such Cholesky factorization and inversion of triangular matrices

The last two points can be provided by the MAGMA library, once it supports ROCm.

Ahhh I see. Can I assume that there is a stable version of Sirius that can be built and run successfully with some older version of Rocm?
If so I, would you kindly provide the version tags for both Sirius and Rocm as I would like to try with that combination first.

There is no stable support for ROCm. The experimental support was added with commit 7e886cc3b1f3239b38a9e1fba296f13d9dcfaf64 and targeted ROCm 2.1 or 2.2. However, you will not be able to use most functionality.
I recommend you wait with testing, until ROCm is fully supported (which depends on the missing features mentioned before).

Ok I understand!
Thanks for the clarification!


scheme callable is horrible UX

Enhancement Proposal
right now, in ingress v2, we need to pass scheme in as a callable to avoid timing bugs, as our charms decide what scheme to use based on the presence/config of other relations.

It's partially implemented in #232
https://github.com/canonical/traefik-k8s-operator/pull/232/files#diff-739e50d32a5a65d89cde876926a12946a218106c7b90a8800e9eeeb3d45d15a7R677

dropping as obsolete and an alternative is already implemented


chore(docs): restructuring of API documentation

fix: Only commit changed files.. dont commit new files
fix: Add new files
fix: Add awesome pages

Types of changes
This PR restructures how the documentation is built to minimise the amount required to be defined by a user

[ ] Hot fix (emergency fix and release)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Documentation (change which affects the documentation site)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Release (develop -> main)

Quality assurance (all should be covered).

[x] My code follows the code style of this project.
[x] Documentation for my change is up to date?
[x] My PR meets testing requirements.
[x] All new and existing tests passed.
[x] All commits are signed-off.

Summary
Key links:

Sonar coverage

Before you merge

Ensure it is a 'squash commit' if not a release.
Ensure CI is currently passing
Check sonar. If you are working for a fork a maintainer will reach out, if required.

@jpower432  / @degenaro a first pass review here would be helpful. All the descriptions (Where BANANAS has been put to have a key to replace) need to be updated.
Highlights:

Tree structure automatically creates the website
No need to update mkdocs.yaml to add a file. In 90% of locations this will happen automatically.

@butler54 I see you've already delivered. I pulled develop branch and doc serves works for me. Hopefully Reference -> API will be improved with future code restructuring.


[Feature request] Add option to not register session when opening one-tab-group homepage. Plus a bug report.

It would be great to have an option in the settings section, to not automatically save the current windows as a new session when we click on the extension icon (opening the extension homepage).
As a side note, it would also be great to have the option to open the one-tab-group homepage on browser opening in a single window (preferably the first window) also. For now you give the option to open it automatically when opening a new window only. In other words, having two separate toggle options in the setting for those two behaviour.
Also, as a bug report to this last option you already have that I talked about ("Automatically open the extension homepage when opening a new window" toggle). When you have the option turn on and you exit the browser with multiple windows; when reopening the browser, you have multiple one-tab-group homepage tab opening in the same window. In other world you quit chrome with 2 windows open, when reopening chrome, their 2 one-tab-group homepage tab that open in one of the two windows that just open (the last one).

Same sentiment. It's a hassle to remove the newly created session all the time.
When multiple windows are restored from a browser restart, One Tab Group opens as well, and creates a new session for each window.
The behavior makes it increasingly inconvenient to delete all the new sessions created.


Full uninstall does not work

🧐 Have you read the FAQ and viewed similar issues?

[X] I have read the FAQ
[X] I have checked older issues, open and closed
[X] I did a clean install of SpotX as described in the FAQ

Сountry of your account
Indonesia
What is your Spotify Plan?
Free
Do you have Spicetify installed?
No
ℹ Computer information
- Spotify: 1.2.01165
- Windows: Windows 10 21H2
Device name	XXXXXXXXXXXXXX
Processor	Intel(R) Core(TM) i3-6006U CPU @ 2.00GHz   1.99 GHz
Installed RAM	4.00 GB (3.89 GB usable)
Device ID	XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Product ID	XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
System type	64-bit operating system, x64-based processor
Pen and touch	No pen or touch input is available for this display

- PowerShell: 5.1.19041.2364

📝 Description
Full Uninstall does not work
https://github.com/amd64fox/Uninstall-Spotify
PC Username theres space
📸 Screenshots

thanks for the feedback, fixed


[druid] fire up groupby query instead of TopN when using having clause

https://github.com/airbnb/superset/pull/1694


[e2e]: container schedule main default slsa3

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/5792327409
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.container.schedule.main.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.container.schedule.main.default.slsa3.yml
Trigger: schedule
Branch: main
Date: Tue Aug  8 03:18:13 UTC 2023

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/5804453250
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.container.schedule.main.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.container.schedule.main.default.slsa3.yml
Trigger: schedule
Branch: main
Date: Wed Aug  9 03:19:03 UTC 2023
Tests are passing now. Closing this issue.


Makes the deathgasps more labored.

About the PR

The death gasps are more labored and all species share the same deathgasps.
Why / Balance

The old death gasps weren't impactful, and were to similar to regular gasps.
Technical details

Media

https://github.com/space-wizards/space-station-14/assets/125223432/43c085bd-a44f-480c-b50f-deebb240484d

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

🆑 Redfire1331

tweak: Death gasps are more impactful.

I'm not going to lie, that sounds terrible. It reminds me of a burping pig and is just... terrible...


Shouldn't border-radius classes (rounded-*) replace element's default border-radius?

Let's say we have a button:
<button class="btn btn-primary">Test</button>

It will be applied the .btn's default border-radius, which is a .25rem radius for every corner:
    border-radius: .25rem;

Let's say we only want the left side corners to be rounded. So we do:
<button class="btn btn-primary rounded-left">Test</button>

Nothing changes and every corner (even from right side) is still rounded. It happens because .rounded-left is:
    border-bottom-left-radius: .25rem;
    border-top-left-radius: .25rem;

Shouldn't we add:
    border-bottom-right-radius: 0 !important;
    border-top-right-radius: 0 !important;

Onto .rounded-left so it will change the element's default border radius to be controlled by rounded-* classes?
Or it's meant to be done somehow else?

That's an intresting point and usecase of the rounded corner utility classes. I am unsure of what's the expected result, if an utility class should overwrite the defaults styles in this case, but if it's ok with you I'll create a PR today to test this idea.
What do you think?

You can do this by stacking rounded-0 and rounded-left on the button
<button class="btn btn-primary rounded-0 rounded-left">Test</button>
https://codepen.io/morrissey-ingenious/pen/NvZRzg

@morrissey-ingenious that works but the idea of resetting styles to set them again is in my opinion a "hack", not intuitive enough.

@andresgalante I would rather not see your PR going in as won't allow things such as rounding everything but the top right corner see my codepen above it has been edited.
<button class="btn btn-primary rounded-0 rounded-left rounded-bottom">Test</button>

@morrissey-ingenious you have a really good point by introducing this solution it generates other problems.
Also, I guess this gets twisted once someone wants to do [top-right + bottom left] or [top left + bottom right]
I'll leave it open and let @mdo decide.
Thanks a lot for your help, I honestly haven't consider the case you are showing.

imho, the utility classes should only do what their name says and have no hidden side effects.
Otherwise it'd be hard to consistently draw a line somewhere for the other utility classes; e.g.: why wouldn't ".pl-1" reset all other padding values then, too?
Following this scheme, adding the 4 utility classes would be ok but I'd reject your existing PR.

You guys are right, I've close the PR. Thanks for the feedback.

Negative, we'll leave as-is.

@rediche this is probably best solved by using your own custom classes now

why is rounded-0 set to be more specific then the individual rounded classes? rounded-bottom should override rounded-0. the only class rounded should be more specific than, of the 5 available, is rounded (and maybe rounded-circle, but that's debatable)
this makes it impossible to use the built in classes to override things like the default button styles to only have the bottom of the button rounded
forcing custom css to do something that would otherwise be obvious seems like something bootstrap shouldn't be aiming for


CreateSharePointFarm Object reference not set to an instance of an object.

CreateSharePointFarm  Object reference not set to an instance of an object.
Details of the scenario you tried and the problem that is occurring
Scenario: Prerequisites + binaries + Language packs installed + but farm creation object fails with  Object reference not set to an instance of an object.
Verbose logs showing the problem
If i use different order it works, if i restart the configuration with force it can create the farm it only fails if everything executed in one pass.
{"time": "2019-07-11T22:31:32.525-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPProductUpdate]InstallCULD] Target Values: Ensure=Present; SetupFile=C:\SPINSTALL\Updates\wssloc2019-kb4462221-fullfile-x64-glb.exe; Verbose=True"},
{"time": "2019-07-11T22:31:32.525-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Test     ]  [[SPProductUpdate]InstallCULD]  in 0.5320 seconds."},
{"time": "2019-07-11T22:31:32.525-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Skip   Set      ]  [[SPProductUpdate]InstallCULD]  "},
{"time": "2019-07-11T22:31:32.525-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Resource ]  [[SPProductUpdate]InstallCULD]  "},
{"time": "2019-07-11T22:31:32.525-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Resource ]  [[File]InstallCULD]  "},
{"time": "2019-07-11T22:31:32.525-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Test     ]  [[File]InstallCULD]  "},
{"time": "2019-07-11T22:31:32.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[File]InstallCULD] The destination object was found and no action is required."},
{"time": "2019-07-11T22:31:32.541-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Test     ]  [[File]InstallCULD]  in 0.0160 seconds."},
{"time": "2019-07-11T22:31:32.541-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Skip   Set      ]  [[File]InstallCULD]  "},
{"time": "2019-07-11T22:31:32.541-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Resource ]  [[File]InstallCULD]  "},
{"time": "2019-07-11T22:31:32.541-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Resource ]  [[WaitForAll]Waitforallbinaries]  "},
{"time": "2019-07-11T22:31:32.541-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Test     ]  [[WaitForAll]Waitforallbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[WaitForAll]Waitforallbinaries] Remote resource '[SPProductUpdate]InstallCULD' is ready"},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Test     ]  [[WaitForAll]Waitforallbinaries]  in 0.3280 seconds."},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Skip   Set      ]  [[WaitForAll]Waitforallbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Resource ]  [[WaitForAll]Waitforallbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Resource ]  [[File]Allbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Test     ]  [[File]Allbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[File]Allbinaries] The destination object was found and no action is required."},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Test     ]  [[File]Allbinaries]  in 0.0000 seconds."},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Skip   Set      ]  [[File]Allbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Resource ]  [[File]Allbinaries]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Resource ]  [[xPendingReboot]BeforeWizard]  "},
{"time": "2019-07-11T22:31:32.869-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Test     ]  [[xPendingReboot]BeforeWizard]  "},
{"time": "2019-07-11T22:31:32.900-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[xPendingReboot]BeforeWizard] No pending reboots found."},
{"time": "2019-07-11T22:31:32.900-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Test     ]  [[xPendingReboot]BeforeWizard]  in 0.0310 seconds."},
{"time": "2019-07-11T22:31:32.900-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Skip   Set      ]  [[xPendingReboot]BeforeWizard]  "},
{"time": "2019-07-11T22:31:32.900-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Resource ]  [[xPendingReboot]BeforeWizard]  "},
{"time": "2019-07-11T22:31:32.900-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Resource ]  [[SPFarm]CreateSharePointFarm]  "},
{"time": "2019-07-11T22:31:32.900-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Test     ]  [[SPFarm]CreateSharePointFarm]  "},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Testing local SP Farm settings"},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Getting the settings of the current local SharePoint Farm (if any)"},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Detected installation of SharePoint 2019"},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] This node has never been connected to a farm"},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Current Values: AdminContentDatabaseName=; CentralAdministrationAuth=; CentralAdministrationPort=; CentralAdministrationUrl=; DatabaseServer=; Ensure=Absent; FarmAccount=; FarmConfigDatabaseName=; IsSingleInstance=Yes; Passphrase=; RunCentralAdmin="},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Target Values: AdminContentDatabaseName=SP_MyDev_Central_Admin; CentralAdministrationAuth=NTLM; CentralAdministrationPort=22641; CentralAdministrationUrl=https://cenadmin.sky.net; DatabaseServer=sqlserver; Ensure=Present; FarmAccount=System.Management.Automation.PSCredential; FarmConfigDatabaseName=SP_MyDev_Config; IsSingleInstance=Yes; Passphrase=System.Management.Automation.PSCredential; RunCentralAdmin=True; ServerRole=Custom; Verbose=True"},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ End    Test     ]  [[SPFarm]CreateSharePointFarm]  in 0.2350 seconds."},
{"time": "2019-07-11T22:31:33.135-5:00", "type": "verbose", "message": "[sharepointserver]: LCM:  [ Start  Set      ]  [[SPFarm]CreateSharePointFarm]  "},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Setting local SP Farm settings"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Getting the settings of the current local SharePoint Farm (if any)"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Detected installation of SharePoint 2019"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] This node has never been connected to a farm"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Server not part of farm, creating or joining farm"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Executing as the local run as user NA\xsSP19MYCenAdminDev"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Loading module from path 'C:\Program Files\WindowsPowerShell\Modules\SharePointDSC\<IP_ADDRESS>\Modules\SharePointDsc.Farm\SPFarm.psm1'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Exporting function 'Get-SPDscConfigDBStatus'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Exporting function 'Get-SPDscSQLInstanceStatus'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Exporting function 'Add-SPDscConfigDBLock'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Exporting function 'Remove-SPDscConfigDBLock'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Importing function 'Add-SPDscConfigDBLock'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Importing function 'Get-SPDscConfigDBStatus'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Importing function 'Get-SPDscSQLInstanceStatus'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Importing function 'Remove-SPDscConfigDBLock'."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Detected Version: SharePoint 2019 - configuring server as Custom"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] The SharePoint config database 'SP_MyDev_Config' already exists, so this server will join the farm."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] The database exists, so attempt to join the server to the farm"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] The server will attempt to join the farm now once every 60 seconds for the next 15 minutes."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Leaving BeginProcessing Method of Connect-SPConfigurationDatabase."},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Entering function SPMountSPConfigurationDatabase.InternalProcessRecord"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Creating connection string for config db SP_MyDev_Config server sqlserver"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Creating connection string for admin content db admincontentdatabase server sqlserver"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Using NTLM for sql connection string"},
{"time": "2019-07-11T22:31:34.541-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Openning configdb so that I can join it at server sqlserver database SP_MyDev_Config in farm mode"},
{"time": "2019-07-11T22:31:35.310-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Now joining to farm at server sqlserver database SP_MyDev_Config"},
{"time": "2019-07-11T22:31:35.388-5:00", "type": "error", "message": "Object reference not set to an instance of an object."},
{"time": "2019-07-11T22:31:35.388-5:00", "type": "verbose", "message": "[sharepointserver]:                            [[SPFarm]CreateSharePointFarm] Leaving ProcessRecord Method of Connect-SPConfigurationDatabase."},
Related uls log entries:
07/11/2019 16:28:15.11	wsmprovhost.exe (0x134C)	0x1770	SharePoint Foundation	Unified Logging Service	b6ig	Medium	Data collection is not enabled for tag '837l' in the configuration or configuration file not available. Not reporting to Watson	ca51ef9e-6060-0000-0000-051399302e77
07/11/2019 16:28:15.11	wsmprovhost.exe (0x134C)	0x1770	SharePoint Foundation	General	8e2s	Medium	Unknown SPRequest error occurred. More information: 0x80020009	ca51ef9e-6060-0000-0000-051399302e77
07/11/2019 16:28:15.11	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	Server Help	9uyz	Medium	InstallOneHelpCollection, for HC using SpTimer, with guid 42187cdd-d275-43d5-8a31-d9b359b2886e	ca51ef9e-605f-0000-0000-0bb7f51fb4ae
07/11/2019 16:28:15.11	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	PowerShell	6tf2	High	System.NullReferenceException: Object reference not set to an instance of an object.     at Microsoft.SharePoint.PowerShell.SPCmdletInstallSPHelpCollection.InternalProcessRecord()     at Microsoft.SharePoint.PowerShell.SPCmdlet.ProcessRecord()	ca51ef9e-605f-0000-0000-0bb7f51fb4ae
07/11/2019 16:28:15.11	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	PowerShell	91ux	High	Error Category: InvalidData    Target Object  Microsoft.SharePoint.PowerShell.SPCmdletInstallSPHelpCollection  Details  NULL  RecommendedAction NULL	ca51ef9e-605f-0000-0000-0bb7f51fb4ae
07/11/2019 16:28:15.12	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	PowerShell	6tf0	Medium	Leaving ProcessRecord Method of Install-SPHelpCollection.	c935c4f9-382d-000e-59c8-35c92d38d501
07/11/2019 16:28:15.12	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	PowerShell	6tf0	Medium	Entering EndProcessing Method of Install-SPHelpCollection.	c935c4f9-382d-000e-59c8-35c92d38d501
07/11/2019 16:28:15.12	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	PowerShell	6tf0	Medium	Leaving EndProcessing Method of Install-SPHelpCollection.	c935c4f9-382d-000e-59c8-35c92d38d501
07/11/2019 16:28:15.12	wsmprovhost.exe (0x134C)	0x1360	SharePoint Foundation	PowerShell	6tf0	Medium	Entering BeginProcessing Method of Initialize-SPResourceSecurity.	ca51ef9e-6061-0000-0000-0edd4c7f97e2
07/11/2019 16:28:15.12	wsmprovhost.exe (0x134C)	0x1770	SharePoint Foundation	General	aix9j	High	SPRequest.SetHttpParameters: UserPrincipalName=, AppPrincipalName= ,bstrHttpMethod= ,bstrRequestDigest= ,flags=1 ,gTranLockerId=00000000-0000-0000-0000-000000000000 ,bIgnoreTimeout=False ,bstrUserLogin= ,bstrUserKey= ,ulRoleCount=0 ,bstrRoles= ,bWindowsM
Suggested solution to the issue
Doing a forced reboot before farm creation fix the issue like this:
Script ForceReboot
{
TestScript = {
return (Test-Path HKLM:\SOFTWARE\MyMainKey\RebootKey)
}
SetScript = {
New-Item -Path HKLM:\SOFTWARE\MyMainKey\RebootKey -Force
$global:DSCMachineStatus = 1
        }
        GetScript = { return @{result = 'result'}}
        DependsOn = @('[WaitForAll]Waitforallbinaries')
    }

    xPendingReboot BeforeWizard
    {
        Name = 'AfterBinaries'
        SkipCcmClientSDK = $true
        DependsOn = "[Script]ForceReboot"
    }

The DSC configuration that is used to reproduce the issue (as detailed as possible)
 SPInstallPrereqs SharePointPrereqInstall
        {
            IsSingleInstance     = "Yes"
            InstallerPath        = "C:\SPINSTALL\SharePoint Server Enterprise\" + "PrerequisiteInstaller.exe"
            OnlineMode           = $false
            SQLNCli              = "C:\SPINSTALL\SP_Prereqs_2019\" + "sqlncli.msi"
            Sync                 = "C:\SPINSTALL\SP_Prereqs_2019\" + "Synchronization.msi"
            AppFabric            = "C:\SPINSTALL\SP_Prereqs_2019\" + "WindowsServerAppFabricSetup_x64.exe"
            IDFX11               = "C:\SPINSTALL\SP_Prereqs_2019\" + "MicrosoftIdentityExtensions-64.msi"
            MSIPCClient          = "C:\SPINSTALL\SP_Prereqs_2019\" + "setup_msipc_x64.exe"
            WCFDataServices56    = "C:\SPINSTALL\SP_Prereqs_2019\" + "WcfDataServices56.exe"
            MSVCRT11             = "C:\SPINSTALL\SP_Prereqs_2019\" + "vcredist_x64.exe"
            MSVCRT141            = "C:\SPINSTALL\SP_Prereqs_2019\" + "vc_redist.x64.exe"
            KB3092423            = "C:\SPINSTALL\SP_Prereqs_2019\" + "AppFabric-KB3092423-x64-ENU.exe"
            DotNet472            = "C:\SPINSTALL\SP_Prereqs_2019\" + "NDP472-KB4054530-x86-x64-AllOS-ENU.exe"
            Ensure               = "Present"
            # DependsOn            = "[Package]InstallVisualC2017Plus"
            SXSPath              = "C:\SPINSTALL\SP_Prereqs_2019\"
            PsDscRunAsCredential = $credsLocalAdmin
        }
        File PrerequisiteInstaller
        {
            DestinationPath      = "C:\TEMP\SPDSC\01prereqDone.txt"
            PsDscRunAsCredential = $credsLocalAdmin
            Contents             = "DSC Done"
            Ensure               = 'Present'
            Type                 = 'File'
            Force                = $true
            DependsOn            = '[SPInstallPrereqs]SharePointPrereqInstall'
        }
        SPInstall SharePointInstall
        {
            IsSingleInstance     = "Yes"
            BinaryDir            = "C:\SPINSTALL\SharePoint Server Enterprise"
            ProductKey           = $configdata.farm.LicenseKey
            Ensure               = "Present"
            DependsOn            = "[SPInstallPrereqs]SharePointPrereqInstall"
            PsDscRunAsCredential = $credsLocalAdmin
        }

        $waitnodes = ($configdata.allnodes | Where-Object { $_.NodeName -ne '*' }).NodeName
        #$waitnodes = ($configdata.allnodes | Where-Object { $_.NodeName -and $_.NodeName -ne '*' }).NodeName

        $lastdir=$null
        foreach ($dir in $(get-childitem $configdata.farm.ScriptConfig.LangPackInstallFolder -Directory))

        {

            $lastdir = $dir.Name.Replace("-", "")
            SPInstallLanguagePack $lastdir
            {
                BinaryDir            = "C:\SPINSTALL\LanguagePacks" + "\$($dir.name)\"
                DependsOn            = '[SPInstall]SharePointInstall'
                PsDscRunAsCredential = $credsLocalAdmin
            }

        }

        SPProductUpdate InstallCULI
        {
            SetupFile            = "C:\SPINSTALL\Updates\" + (Get-ChildItem $configdata.farm.ScriptConfig.UpdatesFolder"sts*.exe").Name
            Ensure               = "Present"
            PsDscRunAsCredential = $credsLocalAdmin
            DependsOn            = '[SPInstallLanguagePack]'+$lastdir
        }

        SPProductUpdate InstallCULD
        {
            SetupFile            = "C:\SPINSTALL\Updates\" + (Get-ChildItem $configdata.farm.ScriptConfig.UpdatesFolder"wssloc*.exe").Name
            Ensure               = "Present"
            PsDscRunAsCredential = $credsLocalAdmin
            DependsOn            = "[SPProductUpdate]InstallCULI"
        }

        File InstallCULD
        {
            DestinationPath      = "C:\TEMP\SPDSC\04CUinstall.txt"
            PsDscRunAsCredential = $credsLocalAdmin
            Contents             = "DSC Done"
            Ensure               = 'Present'
            Type                 = 'File'
            Force                = $true
            DependsOn            = '[SPProductUpdate]InstallCULD'
        }

        WaitForAll Waitforallbinaries
        {
            ResourceName         = '[SPProductUpdate]InstallCULD'
            NodeName             = $waitnodes
            RetryIntervalSec     = 60
            RetryCount           = 120
            PsDscRunAsCredential = $credsLocalAdmin

        }

        File Allbinaries
        {
            DestinationPath      = "C:\TEMP\SPDSC\02binariesDone.txt"
            PsDscRunAsCredential = $credsLocalAdmin
            Contents             = "DSC Done"
            Ensure               = 'Present'
            Type                 = 'File'
            Force                = $true
            DependsOn            = '[WaitForAll]Waitforallbinaries'
        }
        xPendingReboot BeforeWizard
        {

            Name             = 'BeforeWizard'
            SkipCcmClientSDK = $true
            DependsOn = '[WaitForAll]Waitforallbinaries'

        }

        #Findig the first CA host server where we are going to create all shared object
        $FirstCAServer = ($configdata.allnodes | Where-Object { $_.roles.Contains("CA-PRIMARY") -and $_.roles.Contains("APP") } | Select-Object -First 1).NodeName
        $joiningnodes = ($configdata.allnodes | Where-Object { $_.NodeName -ne $FirstCAServer -and $_.NodeName -ne '*' }).NodeName
        $waitnodes = ($configdata.allnodes | Where-Object { $_.NodeName -ne $Node.NodeName -and $_.NodeName -ne '*' }).NodeName

        if ($node.NodeName -eq $FirstCAServer)
        {

            SPFarm CreateSharePointFarm
            {
                IsSingleInstance          = "Yes"
                Ensure                    = 'Present'
                FarmConfigDatabaseName    = $configdata.farm.ConfigDB
                DatabaseServer            = $configdata.farm.ConfigDbServer
                FarmAccount               = $credsLocalAdmin
                Passphrase                = New-Object System.Management.Automation.PSCredential ('Passphrase', (ConvertTo-SecureString $Passphrase -AsPlainText -Force));
                AdminContentDatabaseName  = $configdata.farm.CentralAdminContentDB
                RunCentralAdmin           = $Node.roles.Contains("CA-PRIMARY")
                CentralAdministrationPort = $configdata.farm.CentralAdminPort
                CentralAdministrationUrl  = $configdata.farm.CentralAdminUrl
                CentralAdministrationAuth = 'NTLM'
                ServerRole                = "CUSTOM"

                PSDSCRunAsCredential      = $credsLocalAdmin
                DependsOn                 = @('[WaitForAll]Waitforallbinaries', '[xPendingReboot]BeforeWizard')
            }

The operating system the target node is running
Win 2016 RTM and latest update aswell repro
OsName               : Microsoft Windows Server 2016 Standard
OsOperatingSystemSKU : StandardServerEdition
OsArchitecture       : 64-bit
WindowsBuildLabEx    : 14393.2430.amd64fre.rs1_release_inmarket_aim.180806-1810
OsLanguage           : en-US
OsMuiLanguages       : {en-US, de-DE, es-ES, fr-FR...
Version of SharePoint that is used (e.g. SharePoint 2016)
2019
Version and build of PowerShell the target node is running
To help with this information, please run this command:

Name                           Value

PSVersion                      5.1.14393.2430
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.2430
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
Version of the DSC module that was used ('dev' if using current dev branch)
3.4 and 3.5 repro aswell
{854F30E3-A455-11E9-9676-001DD8B7313D}-0.details.json.txt

My customer got this exact same issue last week as well (SP2019 on Win 2016)! We could not find the root cause of this issue, so I suggested to create a support case since this looks like an issue in the Config Wizard while creating the farm. With the vacation period coming up, I am not sure if they will do that right away or after they come back from vacation.
The thing they found was that the Config DB had been created, but the AdminContent wasn't. When deleting the Config DB and rerunning the DSC config, the farm was created just fine.


aws-sdk/client-route-53  TS1005, '}' expected

Hello,
Since aws-sdk-js-v3 uses smithy to generate types, i'm guessing this missing closing curly brace is related to this project, so i'm cross-referencing the issue raised there.
Sorry if this is unrelated.
https://github.com/aws/aws-sdk-js-v3/issues/2921

I believe this has since been fixed.


Add support for Goodcell 3-phase 22kW EV charger

Pre-eliminary support for the Goodcell 3-phase 22kW EV charger.
A few issues remain:

the two other phases current and voltage are not visible.
the energy reset does not work (might also not be implemented correctly for all I know on the charger)

You have multiple unnamed sensors with the same class (for at least voltage and current).  These need names to distinguish them from each other (max is one unnamed sensor of each class, and one unnamed sensor without a class).  Basically each entity needs to be uniquely identified by (entity type)_(name or class)  ie. name is used if available, or class if unnamed.

You have multiple unnamed sensors with the same class (for at least voltage and current). These need names to distinguish them from each other (max is one unnamed sensor of each class, and one unnamed sensor without a class). Basically each entity needs to be uniquely identified by (entity type)_(name or class) ie. name is used if available, or class if unnamed.

Ofc, this is so obvious now. Well it was getting late last night. Anyway updated my PR with names as I can now see all different phases. I do not know if it is needed to mark them optional, as that might make this also work with the 1-phase models.  It seems there might be something I can do with the constraints too, so current can be adjusted to max 16A for the models not going to 32A.

Rebased and pushed some small tweaks.
Could not figure out how to use dps 113 to select dps 114 or 115 for the current setting. (As in dps 114 if dps 113 says Max16A, and dps 115 if it is Max32A), If you can guide me on how to do it I can implement and fix that. Are there other changes needed still otherwise?

Something like:
  - entity: number
    name: Set charge current
    category: config
    icon: "mdi:ev-plug-type2"
    dps:
      - id: 115
        type: integer
        name: value
        unit: A
        range:
          min: 6
          max: 32
        mapping:
          - constraint: max_current
            conditions:
              - dps_val: Max16A
                value_redirect: value_alt
                range:
                  min: 6
                  max: 16
      - id: 114
        type: integer
        name: value_alt
        unit: A
        range:
          min: 6
          max: 16
      - id: 113
        type: string
        name: max_current

Updated as per your suggestion, but I am not sure it works (did not seem to switch to pds 114 when I changed the Max16A condition to Max32A to trigger it....
Otherwise it pretty much works only exception is that energy reset button does not seem to work, but it is not supported in the app. So I guess it is not supposed to work. Should I remove that?
Otherwise anything I should do before you consider this done?

Energy reset probably only affects balance_energy, which is probably only relevant in charge_energy charging mode.  I don't think it affects the total_forward_energy, or the charge_energy_once (which I think is the last/current charge only)


Stars lose color if first component instance is hidden (display:none)

If the component is used multiple times, all stars lose their gradient as soon as the first component instance is set to display:none.
I made a minimal example: REPL
If you click the first button, all stars lose their color, but they are still there.
The other two don't impact the overall behaivor, as it should.
I am working on a solution.

I just noticed I used svelte-star-rating instead of svelte-stars-rating.
There is noch issue, sorry for the inconvenience.

🤷♂


New version: Google.Chrome.Canary version 107.0.5298.0

Result: Installation Successful

Manifest
Add/Remove Programs

Name
Google Chrome Canary
Google Chrome Canary

Version
107.0.5298.0
107.0.5298.0

Publisher
Google LLC
Google LLC

ProductCode

Google Chrome SxS

Auto-updated by vedantmgoyal2009/winget-pkgs-automation in workflow run 3032
Microsoft Reviewers: Open in CodeFlow

@wingetbot run


Using the code without GPU

I want to use the Flowformer on a system without GPU but the code uses CUDA. Can someone explain what I have to change to run the train.sh?

Thanks for your attention, but I believe our code is unsuitable for running without GPU :thinking:.

Oh that's a bummer. I guessed it was because on the KITTI 2015 benchmark FlowFormer is listed with CPU 2,5GHz usage.

Sorry for the mistake :face_with_head_bandage:. The information on KITTI has been updated


optimize connect propTypes useage

Needs refinement. Maybe only pass props specified in propTypes to the component.... This would definitely make it more predictable.
Also possibly check for propTypes on the component?

Scratch that. This NEEDs to be its own package. Not directly related to state management.


pkg/report: better handling of numbers in titles

We've recently got a fallout of 50 gvisor bugs of the form:
panic: unable to find an index for ID: 84
panic: unable to find an index for ID: 4
panic: unable to find an index for ID: 7
panic: unable to find an index for ID: 47
panic: unable to find an index for ID: 57
...

We have logic to replace numbers/pointers in titles with NUM/ADDR, but it did not kick in in this case. I think we need to make number replacement more aggressive to prevent such cases in future.
Now that we have AltTitles machinery, maybe we could use AltTitles for this. Namely, we could keep original title "panic: unable to find an index for ID: 7", but add a "panic: unable to find an index for ID: NUM" alt title. It should also make them all merged together, but will effective bug title unmodified. Need to think about pros and cons.
I think in lots of cases replacing varying parts in the actual title makes titles neater and shorter. E.g. we had this one "BUG: rwlock bad magic on CPU#0, kworker/0:18/18158,<PHONE_NUMBER>9f2a8d" which was turned into "BUG: rwlock bad magic on CPU, kworker/0:LINE/NUM, ADDR".

I've taken all currently opened bug titles, removed duplication indicators (e.g. (2), (3), (4), etc. from their ends) and ran them against (\W)(\d+)(\W|$). It found these matches:
BUG: bad host security descriptor; not enough data (4 vs 5 left)
BUG: please report to<EMAIL_ADDRESS>=> prev = 0, last = 0 at net/dccp/ccids/lib/packet_history.c:LINE/tfrc_rx_hist_sample_rtt()
upstream test error: BUG: program execution failed: executor 0: exit status 67
WARNING: CPU: 1 PID: 8448 at net/ma
syzkaller: testing failed: failed to run ["go" "test" "-short" "./..."]: exit status 1
usb-testing boot error: kernel panic: VFS: Unable to mount root fs on unknown-block(0,0)
upstream boot error: kernel panic: VFS: Unable to mount root fs on unknown-block(0,0)

Replacement with ${1}NUM${3}  would give
BUG: bad host security descriptor; not enough data (NUM vs NUM left)
BUG: please report to<EMAIL_ADDRESS>=> prev = NUM, last = NUM at net/dccp/ccids/lib/packet_history.c:LINE/tfrc_rx_hist_sample_rtt()
upstream test error: BUG: program execution failed: executor NUM: exit status NUM
WARNING: CPU: NUM PID: NUM at net/ma
syzkaller: testing failed: failed to run ["go" "test" "-short" "./..."]: exit status NUM
usb-testing boot error: kernel panic: VFS: Unable to mount root fs on unknown-block(NUM,0)
upstream boot error: kernel panic: VFS: Unable to mount root fs on unknown-block(NUM,0)

So

It seems that we can let number replacement be more agressive not just for panic: unable to find an index for ID:, but for all bugs.
It is a pity that Go's regexp engine does not support look ahead/behind - it would otherwise be possible to write a simple regexp that would avoid the (NUM,0) problem from above.

Using (\W)(\d+) instead of (\W)(\d+)(\W|$) leads to conflicts with HEX number replacement - it begins to fire on titles like WARNING: Unsupported flag value(s) of 0x%x in DT_FLAGS_1. and tries to replace it with WARNING: Unsupported flag value(s) of NUMx%x in DT_FLAGS_1..

It found these matches:

This set of changes looks small enough to just change titles w/o any special measures.
But is it all kernels? Or just upstream linux?
There are some on gvisor that involve slice sizes as [40:50]. I assume these should change as well, but you don't list them here.

It is a pity that Go's regexp engine does not support look ahead/behind

Let's apply it multiple times. We need to replace things like [40:50].

Or just upstream linux?

That was for Linux, yes.

[40:50]

That would turn to [NUM:LINE] now, which is not a reasonable change. Probably we need to slightly change the regexp for LINE as well, e.g. maybe demand that there must be an alphabet character before the :.


help

I want to test some other softwares, what can I do to make CodeAlchemist support them? like:
1）https://github.com/Moddable-OpenSource/moddable
2）https://github.com/jerryscript-project/jerryscript
3） https://github.com/pcmacdon/jsish

@DaramG

Add engine to Engine type in https://github.com/SoftSec-KAIST/CodeAlchemist/blob/master/src/Common/Conf.fs#L8
Update code for new engine type. Compiler will warn you where you should update.
Prepare conf files for each engine and run the fuzzer.


Nesting components with Svelte 5 snippet parameters with the same name can cause ESLint errors

Describe the bug
When components in nested scopes use the same snippet names, ESLint's no-shadow rule detects that the inner scope is shadowing the snippet name from the outer scope.

While the example shows a contrived use-case, I ran into this pretty quickly once I started creating reusable UI components. I've temporarily disabled ESLint's no-shadow rule, but of course that isn't ideal.
Reproduction
Example with ESLint configuration on GitHub
Example with the same code on REPL

Logs
[{
	"resource": "/Users/trevorparscal/Sites/svelte-issue/src/routes/+page.svelte",
	"owner": "eslint4",
	"code": {
		"value": "no-shadow",
		"target": {
			"$mid": 1,
			"path": "/docs/latest/rules/no-shadow",
			"scheme": "https",
			"authority": "eslint.org"
		}
	},
	"severity": 8,
	"message": "'icon' is already declared in the upper scope on line 16 column 12.",
	"source": "eslint",
	"startLineNumber": 22,
	"startColumn": 13,
	"endLineNumber": 22,
	"endColumn": 17
},{
	"resource": "/Users/trevorparscal/Sites/svelte-issue/src/routes/+page.svelte",
	"owner": "eslint4",
	"code": {
		"value": "no-shadow",
		"target": {
			"$mid": 1,
			"path": "/docs/latest/rules/no-shadow",
			"scheme": "https",
			"authority": "eslint.org"
		}
	},
	"severity": 8,
	"message": "'label' is already declared in the upper scope on line 13 column 12.",
	"source": "eslint",
	"startLineNumber": 27,
	"startColumn": 13,
	"endLineNumber": 27,
	"endColumn": 18
}]

System Info
System:
    OS: macOS 15.0.1
    CPU: (8) arm64 Apple M1
    Memory: 66.11 MB / 16.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 20.12.2 - ~/.nvm/versions/node/v20.12.2/bin/node
    npm: 10.5.0 - ~/.nvm/versions/node/v20.12.2/bin/npm
    bun: 1.1.12 - ~/.bun/bin/bun
  Browsers:
    Brave Browser: <IP_ADDRESS>
    Chrome: 126.0.6478.127
    Safari: 18.0.1
  npmPackages:
    svelte: ^5.0.0-next.1 => 5.0.0-next.264

Severity
annoyance

I think that rule correctly reports shadowing, why do you think it's not ideal?

Oh I see. So snippet tags define variables, but users don't intend variable definitions, and no-shadow rule doesn't seem to work well when slot names are intended.
I think we need to create a rule that extends no-shadow rule, and the new rule will exclude snippet tags used only with slot names from reporting.


Improve error messages for incorrect zero argument signatures

Which issue does this PR close?

Closes https://github.com/apache/datafusion/issues/13763

Note this is a small improvement, before this PR:

https://github.com/apache/datafusion/pull/13871

Rationale for this change
As described on https://github.com/apache/datafusion/issues/13763
In version s 43.0.0 and earlier of DataFusion, a TypeSignature::Any(0) means your udf could be called with zero arguments:
SELECT my_udf();

However in version 44.0.0 you get a very confusing error that says zero arguments are not supported but then gives you a candidate function is the one it just said was not supported:
SELECT my_udf();

Error during planning: example_union does not support zero arguments.
No function matches the given name and argument types 'my_udf()'.
You might need to add explicit type casts.

Candidate functions:
example_union()

What changes are included in this PR?
Improve error message to direct people to nullary
Are these changes tested?
Yes by CI

Are there any user-facing changes?

here is a PR to just update the signatures:

https://github.com/apache/datafusion/pull/13881

Thank you for the review @jayzhan211


updated dependencies, and removed fsharp.core from packages

Yeah fair enough. It wasn't so much around the packages being official, but more around it not seeming too reliable or predictable. I know what you mean about reproducible though. I'll definitely be keeping an eye on this, because I would also like a world where everything is constrained!

Perhaps @pblasucci can weigh in, as I think it was his request this change satisfied.

@panesofglass I'm happy to relate my experiences...
@haf I tried combining suave, freya, and fszmq in the same project. This included the appropriate binding redirects of FSharp.Core.dll (which I nearly always reference from NuGet in all my projects). Everything compiled fine. But on startup, freyaRouter would raise a TypeInitializationException indicating problems with types from FSharp.Core. I don't recall exactly the error, but it was something around resolving a tuple. Now, if I omitted using freyaRouter, I could still use other parts of Freya and everything worked fine. However, after @kolektiv released the update, I could start using freyaRouter again and now everything just works. So, I'm not sure if anything else changed (there are a lot of little moving parts to keep track of afterall). But at a minimum, it seems omitting the dependency on FSharp.Core.dll helped.
Personally, I suspect this is actually an issue with FSharp.Core, as the same binding redirects which resolve compile-time issues really ought to hold at run-time. Thus free library and application authors from having to deal with these shenanigans. Additionally, I'm reviewing the other libraries I maintain to see if there are more places where I can drop the explicit dependency. My feeling is, in general, libraries should have as few dependencies as possible.
Anyway, I'm not sure if this helps. But that's the state of things.

@pblasucci Thanks for telling us about it. My reasoning goes F# Core is a dependency, so it should be explicit to make things easier; at some point in the future libs will start referencing F# 4.0 or F# 5.0 and then some will work well with others but crash in some combinations, all without being able to debug it.
If you still have the broken runtime commit of your code; perhaps we could present it to the F# team for review so they can take action accordingly?

@haf makes a good point. Also, once we start doing things for .NET Core, we will have to make NuGet package references, so we will need to make these explicit dependencies.


Pretty-print SVG file (Issue #33)

Fixes #33.
This formats all rendered SVG files nicely, using 4 spaces per indentation (which templates currently have). It's not optional at the moment, it just does this with all rendered SVGs.
I'm not sure how to write a test for this.
lxml supports pretty-printing, but I could not use that because it removes comments in the XML, which would break styling.
I made another pull request for this because I deleted my repo; I forgot I still had a pull request open. :(

Thanks for the pull request, but I'd rather not make this the default behavior since indentation increases file size. However I'll keep this in mind in case a debugging option is added to the command line interface. Sorry.


how to add arbitrary binary to the image?

how to add arbitrary binary to the image? is this supported with apko? or should i use sth like melange to package them first?

Yeah, using melange is the recommended solution. This way we can ensure reproducibility and keep a full SBOM.
I think we will support adding files especially for things like config in a later version, but apk's will remain the recommended approach for most things.


website download area down

I cant access the download area it seems the site is down.

Here?
If so, seems like it's just you.

Re-open with more details if there's still an issue reaching our download page @jmartiuk5, closing this for now.

@nodejs/build I think this might be a DigitalOcean backup thing, it happens around this time each week and the OP was a week ago. Remember that we pass-through for download tarballs so CloudFlare doesn't keep it up when the server is having trouble serving requests. The backup is COW but I wouldn't be surprised if all the writing to the log files fill up the available buffer space or something like that because the backups take a long time.
I'm hesitant to just disable backups without having a more solid backup solution for this server. We have most of it in Ansible but there is state on there that would be nice to keep, particularly logs, but there's likely lots of other things that would be nice to node lose in the case of catastrophic failure.


Js cleanup named pipes

@paetling PR does:

Remove wait on pid, which was causing to block on background threads.
Don't send output of always to a log file.  I don't think we need to do this since that's the main command that gets run on the container (just use dusty logs)

I do not really understand your argument for the second part. Can't you also look at logs for once?  I guess since it does not always run it is not worth saving?

My argument is that once output is worth saving.  always output is not

I like the once always distinction. I think we mention in our logs somewhere about them. Worth a quick glance and correction if it is there.

just the logs thing and then :ghost: :cow:

Closes https://github.com/gamechanger/dusty/issues/380


Consolidate client subclasses

Porting #461 (merged to shared-threadpools branch) over to master/trunk.
Rebasing and adding release notes.

Codecov Report

Merging #479 (8efa995) into master (5da7989) will increase coverage by 0.09%.
The diff coverage is 97.05%.

:exclamation: Current head 8efa995 differs from pull request most recent head c5abf5b. Consider uploading reports for the commit c5abf5b to get more accurate results

@@            Coverage Diff             @@
##           master     #479      +/-   ##
==========================================
+ Coverage   83.61%   83.70%   +0.09%
==========================================
  Files          16       17       +1
  Lines        2642     2657      +15
==========================================
+ Hits         2209     2224      +15
  Misses        433      433

Impacted Files
Coverage Δ

dwave/cloud/client/hybrid.py
100.00% <ø> (ø)

dwave/cloud/client/qpu.py
100.00% <ø> (ø)

dwave/cloud/client/sw.py
100.00% <ø> (ø)

dwave/cloud/utils.py
89.65% <80.00%> (ø)

dwave/cloud/__init__.py
96.42% <100.00%> (+0.97%)
:arrow_up:

dwave/cloud/client/__init__.py
100.00% <100.00%> (ø)

dwave/cloud/client/base.py
91.21% <100.00%> (ø)

dwave/cloud/concurrency.py
93.75% <100.00%> (ø)

dwave/cloud/exceptions.py
100.00% <100.00%> (ø)

dwave/cloud/solver.py
81.27% <100.00%> (ø)

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5da7989...c5abf5b. Read the comment docs.


Highlight group debugPC seems to be broken

With nvim-dap I could barely see the text when stopped.
It uses highlight group debugPC by default - https://github.com/mfussenegger/nvim-dap/blob/master/lua/dap.lua#L233
Overriding it to different group works fine!
I tried overriding it in dap's  config in setup function as well as according to explaination here - https://github.com/mfussenegger/nvim-dap/issues/76#issuecomment-738817965, nothing changes and still see the same thing.
With Light Theme,

With Dark Theme,

Never mind, I was overriding it to ColorColumn in dapui config.


[mkldnn-v1.0] Use memcpy instead of set_handle with unidirectional 1-layer RNN

Description
Fix the issue filed by https://github.com/apache/incubator-mxnet/pull/16555#issuecomment-547000155. Previously, we don't need Concat to concatenate pieces of weights to a continuous space with unidirectional 1-layer RNN. Thus, mkldnn::memory::set_data_handle(ptr) is directly used to fetch the weights, and ptr is the pointer of weights shifted from NDArray. When it runs into gru mode, a swap mechanism of gates is used to swap weights of the reset gate and the update gate. If we use set_data_handle here, it will crash the naive memory from NDArray.  As a result, the gru operator swaps the naive memory every iteration when is_train=True. It will then give inconsistent results. We should use memcpy here.
Checklist
Changes

[x] Move set_data_handle to memcpy.

@ciyongch @pengzhao-intel @TaoLv

This change will impact performance. Why v0.20 doesn't have this issue? Is there any solutions other than memcpy?

This change will impact performance. Why v0.20 doesn't have this issue? Is there any solutions other than memcpy?

v0.20 swaps the GRU memory on native NDArray space and then swaps back to the original layout. It may have more overhead over the first iteration. Thus, v1.0 uses a different mechanism that directly adjusts mkldnn::memory. Previously, we use OpenMP for-loop to apply the swap and copy. I have had an experiment measuring the performance of the operator from stl and the one wrapped by OpenMP. It seems that std::memcpy needs another ~4 milliseconds (clx-8280) to finish the operation with 1e7 long float array. It may not severely impact the performance in the inference procedure, as we copy and swap the memory only once at the first iteration. Honestly, it is not true in forward_training and backward. There are lots of memcpy to deliver the gradients of weights. I will try to simplify and accelerate them in this PR.

@zixuanweeei take a look for the CI failure

IMPERATIVE.FullyConnectedOp failed again. @pengzhao-intel


Remote policyLocation with Basic-Auth

Hi,
does it look feasible to add support for Basic-Auth configuration when the policyLocation is retrieved from a remote location?
Background: We are using spring-boot with centralized configuration management via spring-cloud-config. The latter allows for serving "raw" files via it's interface. Think jolokia-access.xml served from the config server.
spring-boot allows setting the servlet init parameters. Thus the following setup:
jolokia.config.policyLocation=http://configserver/*/*/*/jolokia-access.xml

Unfortunately our config server is guarded with HTTP Basic-Auth and so the RestrictorFactory will get a 401 response and fail. Also the Java UrlConnection doesn't like URL tricks like http://user:pass@host so this isn't an option either. One would have to set the Basic-Auth headers itself.
A setup of the config server without Basic-Auth and jolokias Restrictor configured from remote works like a charme.
The config-server way would be my ideal solution instead of distributing a jolokia-access.xml with every application. (Be it by dependency, a spring boot starter or a build step for making sure the file is within the deployable artefact.)

I could image to provide a functionality like this by parsing the URL for user & password and the setting the Authorization: header accordingly on our own. This would save some new configuration keys from which it already have too much (so that I always hesitate a bit to introduce new keys).
However, I wonder whether this really make sense when used together with plain http://. After all jolokia-access.xml is about security and if you send it over http this can be easily forged with a man-in-the-middle attack. And since the password is send via plain text as well, this could be grabbed, too.
IMO it only makes sense to either use SSL (and then with server cert validation), or to put the restrictions really into the jar where it does not need to be loaded over the net during runtime.
BTW, you can easily write your own restrictor, too, which can do what you want. It depends on the agent your are using how to fire it up. You simply specify a classname with restrictorClass which needs to be in the classpath. You need to implement Restrictor and provide a constructor which takes a Configuration as single argument.
You could easily inherit from PolicyRestrictor which takes an InputStream with its constructor. This is probably not well suited for inheritance, but I'm open to refactor this class to make it easier to inherit from. Happy also about any PR ;-)

Hi Roland,
thanks for the quick reply.
And indeed I forgot to mention the little detail that the config-client uses an SSL connection to the  config-server. The URL I posted was just local evaluation.
Writing my own Restrictor or shipping the jolokia-access.xml would mean a dependency (.jar) every deployable would have to attract. Dunno yet, if this fits our idea of a "shared-nothing" approach. Allthough this stance is arguably a bit troublesome in regards to a common set of basic functionality.
Also thanks for your proposal to parse the URL and apply Basic-Auth setup to the UrlConnection. I can't promise anything yet, but will come back to you as soon as we've decided on how we're going to proceed.
Thanks!
Michael

Hi again. Decision made.
I'd dare an implementation for the feature against 1.3.x/master if that is ok with you.

Yeah, master is fine. As I said, would love an implementation without any extra configuration option (by using the standard URL encode credentials).
thanks !

As @rhuss mentioned, org.jolokia.server.core.service.api.Restrictor is a great extension point and Spring (Boot) is a great platform to wire any kind of simple and complex applications.
Once we'd add support for basic auth credentials encoded within the URI for policy file location, someone may ask for digest auth, mutual TLS or OAuth2 Bearer token authentication.
So it's recommended for such complex (as yours) setup to implement your own Restrictor.


Deploy an instance of Backstage

Instance of backstage running on CloudPlatform https://backstage.io/ Names something like moj-developer-portal or something better and more sensible.

https://developer-portal.apps.live.cloud-platform.service.justice.gov.uk


When Trying to Register ASDK Deployment getting failure message (1904)

VERBOSE: 2019-05-27.13-36-21: Connection to AzS-ERCS01 successful
VERBOSE: 2019-05-27.13-36-21: Verifying stamp version.
VERBOSE: 2019-05-27.13-36-31: Running registration actions on build 1.1904.0.36. Cloud Id: xxxxxxxx
-xxxx-xxxx-xxxx-xxxxxxxxxxxx, Deployment Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
VERBOSE: 2019-05-27.13-36-31: Creating Azure Active Directory service principal in tenant 'xxxxxxxx
-xxxx-xxxx-xxxx-xxxxxxxxxxxx' Attempt 0 of 3
VERBOSE: Looking up shared vhd product drive letter.
WARNING: Unable to find volume with label Deployment
VERBOSE: Get-Package returned with Success:True
VERBOSE: Found package Microsoft.AzureStack.Solution.Deploy.Security with version 1.1904.0.36 at C:
\NugetStore\Microsoft.AzureStack.Solution.Deploy.Security.1.1904.0.36\Microsoft.AzureStack.Solution
.Deploy.Security.nuspec.
VERBOSE: type added
VERBOSE: 2019-05-27.13-41-52: *** WARNING ***
WARNING: 2019-05-27.13-41-52: Creation of service principal failed:
Action plan execution failed. Error: Execution timeout
VERBOSE: 2019-05-27.13-52-36: *** End WARNING ***
VERBOSE: 2019-05-27.13-52-36: Waiting 10 seconds and trying again...
VERBOSE: 2019-05-27.13-52-46: ************************ Error ************************
VERBOSE: 2019-05-27.13-52-46: Action plan execution failed. Error: Execution timeout
VERBOSE: 2019-05-27.13-52-47: at Start-ActionPlanInstance, C:\Program Files\WindowsPowerShell\Modul
es\Microsoft.Azurestack.Activation\Microsoft.Azurestack.Activation.psm1: line 426
at New-AzureBridgeServicePrincipal, C:\Program Files\WindowsPowerShell\Modules\Microsoft.Azurestack
.Activation\Microsoft.Azurestack.Activation.psm1: line 179
VERBOSE: 2019-05-27.13-52-47: *********************** Ending registration action during New-Service
Principal ***********************
VERBOSE: 2019-05-27.13-52-47: Logs can be found at: C:\MASLogs\Registration\AzureStack.Activation.S
et-AzsRegistration-2019-05-27.log and \AzS-ERCS01\c$\maslogs
VERBOSE: 2019-05-27.13-52-47: Removing any existing PSSession...
Action plan execution failed. Error: Execution timeout
At C:\AzureStack-Tools-master\Registration\RegisterWithAzure.psm1:1249 char:13
    $servicePrincipal = Invoke-Command -Session $PSSession -S ...
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo : OperationStopped: (Action plan exe...ecution timeout:String) [], Ru
ntimeException
FullyQualifiedErrorId : Action plan execution failed. Error: Execution timeout

@joseq-terawe This is a known issue, and is being discussed Here on the Azure Stack MSDN Forums.

This issue is being caused due to an improper timeout on the service principal creation, and all we need to do is increase the timeout.
To mitigate this issue, you will need to open up the file at "C:\AzureStack-Tools-master\Registration/RegisterWithAzure.psm1" in an editor with ADMINISTRATOR PERMISSIONS, I prefer PowerShell ISE as it gives line numbers.
Line#1249 needs -TimeoutInSeconds 600 added to the "New-AzureBridgeServicePrincipal" cmdlet, and should look like this:
$servicePrincipal = Invoke-Command -Session $PSSession -ScriptBlock { New-AzureBridgeServicePrincipal -RefreshToken $using:RefreshToken -AzureEnvironment $using:AzureEnvironmentName -TenantId $using:TenantId -TimeoutInSeconds 600}
We are fixing this in the Registration Script as well, but for now the manual workaround is the best.

Fix for this has been merged. Please let us know if that doesn't resolve the problem.


Add autofix for D300

Summary
Add fix for D300
Test Plan

cargo test and manually

Didn't want to commit with your automerge on, just in case 😅


Handle channel back-pressure

As mentioned in amqplib docs, channels mimic stream.Writable behaviour when being written to: a call to publish or sendToQueue can return false, which would mean that internal buffer has been filled and you should wait for drain event before sending more data.
Nodejs docs for stream.Writable also strongly recommend respecting stream back-pressure to avoid memory issues (like clogging application memory with unsent data by continuously reading it from source stream).
This request implements it.
There are some points worth mentioning:

There are no changes for ConfirmChannels because for them next will only be called after getting a response from broker which would imply that the data has been sent.
While this implementation mimics the example from nodejs docs, it implies that send operation will not finish (next will not be called) until the channel is drained. This makes sense because we want to make sure the data has been actually sent before calling it a success.
I haven't touched any tests and yet they are still green after this change. This kinda worries me, because the logic has been changed. I didn't come up with a strait-forward way to cover this behaviour with tests.

Hi @Gaen,
Thanks for the PR. First glance looks good, but I'll have a proper look this evening.

I haven't touched any tests and yet they are still green after this change. This kinda worries me, because the logic has been changed. I didn't come up with a strait-forward way to cover this behaviour with tests.

From what I see, the logic is only different if the publish / sendToQueue returned false. Unless we can find a way to fill the buffer, I don't see how this can be tested.

Hi @Gaen,
I've been thinking about your suggestion and have some concerns. In the event of amqp returning false the correct behaviour is to wait for a drain event before attempting to publish more messages. Rascal doesn't currently do this, and your PR attempts to resolve this by not executing the callback until the drain even has been emitted, however this does not necessarily mean that the application using Rascal will slow down. If it is publishing messages in response to an external source (e.g. an incoming message or HTTP request) then it will continue telling Rascal to publish, and Rascal will continue to send messages to the queue / exchange, then block the callback until drain is called.
One solution would be for Rascal to queue messages, but I'm not keen on this, since it just creates a different set of problems, that would ultimately need to be reported to the calling application. Another solution is for Rascal to emit an event, when amqplib reports the write buffer is full to propagate the 'drain' event e.g.
const stats = { received: 0, sent: 0, errors: 0 }
readable.on('data', (message) => {
  stats.received++;
  const publication = await broker.publish('some-exchange', message, options);
  publication.on('success', () => {
    stats.sent++;
  }).on('error', () => {
    stats.errors++;
  }).on('saturated', () => {
    readable.pause();
  }).on('drain', () => {
    readable.resume();
  });
});

What do you think?

this does not necessarily mean that the application using Rascal will slow down

Good point.
In general, stream api leaves it to the app to decide to either pause on stream saturation or keep sending data and allow stream to buffer it. If we had a straightforward pipe of streams - this logic would propagate transparently through that pipe. Even amqplib with its internal multiplexing mumbo-jumbo in the end terminates with exactly one downstream socket. However, that is not the case with Rascal because of its pooling. As I understand it, there are some weird cases possible, like we have several connections to rabbitmq node (or even cluster?) and when we publish a message and one connection gets saturated, we could try another instead of just waiting for drain. With this approach Rascal indeed would have to have its own multiplexer which would handle this logic while behaving like stream (having own buffer, emitting drain on reaching watermark size etc). With that implemented, Rascal would be able to expose drain event.
I really hope that there is some less over-engineered way to do this.

A bit of background on Rascal's pooling...
Rascal creates a channel pool using a shared connection for each "publication". By default the pool size is 1. The history behind this feature is that Rascal intentionally opened and closed a channel for each publication to limit splash damage from an error on one channel interfering with another publish operation. The problem with this approach is that performance suffered, so we added a channel pool which could keep the channel isolation, but alleviate the problem of opening an closing channels.
However channels share a connection and while an application could configure multiple connections to the same vhost it's probably quite rare. Even when using a RabbitMQ cluster, the way I believe the protocol works WRT to publishing is that clients will connect to a single host, and the broker will mirror messages between queues.

Still thinking about this. I'm considering making this a developer responsibility by changing the code from
      publishFn(channel, buffer, publishConfig, function(err) {
        removeListeners(channel, errorHandler, returnHandler);
        returnChannelFn(channel);
        if (err) return emitter.emit('error', err, messageId);
        emitter.emit('success', messageId);
      });

to
      publishFn(channel, buffer, publishConfig, function(err, busy) {
        removeListeners(channel, errorHandler, returnHandler);
        returnChannelFn(channel);
        if (err) return emitter.emit('error', err, messageId);
        emitter.emit('success', messageId, busy);
      });

And having the code forward the drain event through the emitter. This way the developer can check the busy flag and apply back pressure to the source of the message until the drain event is received.

I implemented in a slightly different way to previously discussed...
While channel.publish returns false when saturated, this seems to affect all channels for the connection and so it doesn't really make sense to handle this on a publication by publication basis. Instead I've bubbled busy and ready events up to the broker, but pass the vhost name to the event handler, hopefully allowing you to apply back pressure
Rascal.Broker.create(Rascal.withDefaultConfig(config), function(err, broker) {
  if (err) throw err

  const stream = random.createRandomStream()
    .on('error', console.error)
    .on('data', data => {
      broker.publish('demo_pub', data, function(err, publication) {
        if (err) throw err
        publication.on('error', console.error)
      })
    })
    .on('end', () => {
      console.log('end');
    })

  broker.on('busy', function(vhost) {
    console.log(Date.now(), `Pausing vhost: ${vhost}`);
    stream.pause();
  });

  broker.on('ready', function(vhost) {
    console.log(Date.now(), `Resuming vhost: ${vhost}`);
    stream.resume();
  });
})

https://github.com/guidesmiths/rascal#flow-control

Bah. Looks like the pooling was handing out the same channel each time, so my statement about the connecting being saturated rather than the channel was wrong. I'll have another look.

I've gone with a hybrid of your original and the event emitting approach.
Now when channel.publish returns false I delay returning the channel to the pool until the channel drain event is received.
Additionally when requests to the channel pool start being queued (because channels are slow to be returned) I emit a busy event, containing details such as the vhost name and pool statistics. When the channel request queue drops back to zero I emit a 'ready' event with the same details.
By listening to busy and ready the developer can apply / release back pressure to the message source.
To get this working I had to update to the latest version of generic-pool, and read on their GitHub issues there might be a memory leak / some performance problems, so I'm going to leave running overnight before publishing to npm.
I would also like to make the pool details configurable, allowing developers to set min/max pool size etc.

Released as<EMAIL_ADDRESS>https://github.com/guidesmiths/rascal#flow-control
I've made the pooling much more configurable, so if you previously specified pool sizes you'll have to update your config. Also the pools are not created automatically but on first use, so there may be a very slight delay when publishing your first message (can be changed by specifying autostart in the pool config), but otherwise the changes are backwards compatible.


Attempt to port doctr to generator rendering

(See https://github.com/mindee/doctr/discussions/1000 for background)
A problem with generators was missing __len__(). However, we know the prospected length already, so we can create a wrapper object that behaves like an iterator but also implements __len__().
Backwards-compatible pypdfium2 4 port on the way.

Ah, the CI results show that quite some more changes may be necessary (I only ran tests/common locally, so didn't notice).

Looks like CI has halted itself?

So far, among the 6 failing checks, only the style failures seem valid. The others are (I think) unrelated to my changes, but 10 checks are still in progress.

Not sure if the remaining checks will ever finish? Some steps are running since >5h already...

I've just merged the devel branch, so pypdfium2 v4 will be released soon (probably next monday).
This means at least part of the patch should be interesting.

Hi @mara004 :wave:
Thanks for the PR :)
I'm not sure if the changes will add any real value/improvement as long as we end up being forced to cast to a list.
(Otherwise we would have to refactor all the logic for processing PDF files).
Also ping @frgfm  for a second opinion.

Just in case you intend to merge this, please squash the commits first. Thanks!

Another thing: pypdfium2 doesn't permit rendering in parallel with buffer/bytes input anymore due to unresolved random segfaults. The tests expect this to work, though. Should we handle these input types by saving to a tempfile in PdfReader?

Hey everyone :wave:
My apologies, I didn't come back to this notification before now! Glad to see you around @mara004 :smile:
A few high-level notes:

I fully agree with the proposal of moving list to generator (that will save RAM on the end device and I think we could make it compatible with the overall lib with a bit of planning)
interface-wise, moving from functional to class version is alright in my opinion

However, I suggest that if we make those changes we check if harmonizing this would make sense. I mean:

for other inputs than PDFs (images, etc.), does it make sense to have a generator
functional vs class API for the whole io module
The few challenges I imagine: generators are actually helpful if the inputs are file paths (otherwise it's the same for the RAM), and we should check performance impact on the transformation/preprocessing stage.

What do you think?

Thanks for the response and the interesting considerations! A few comments/clarifications:

I'm not sure how doctr handles images, but as I guess compressed inputs (e. g.  JPEG, PNG) need to be decompressed for processing, I imagine doing that in step-by-step fashion would be good, too.
File paths/handles are certainly more memory efficient than a bytes object. However, I believe delaying rendering/decompression is a logically independent memory advantage.

hi @mara004 :wave:,
for the moment i would suggest to do only minimal changes without breaking our IO api to be compatible with your latest releases >=4.0.0.
If have tested it shortly:
def read_pdf(
    file: AbstractFile,
    scale: float = 2,
    rgb_mode: bool = True,
    password: Optional[str] = None,
    **kwargs: Any,
) -> List[np.ndarray]:
    """Read a PDF file and convert it into an image in numpy format

    >>> from doctr.documents import read_pdf
    >>> doc = read_pdf("path/to/your/doc.pdf")

    Args:
        file: the path to the PDF file
        scale: rendering scale (1 corresponds to 72dpi)
        rgb_mode: if True, the output will be RGB, otherwise BGR
        password: a password to unlock the document, if encrypted
        kwargs: additional parameters to :meth:`pypdfium2.PdfDocument.render`

    Returns:
        the list of pages decoded as numpy ndarray of shape H x W x C
    """

    # Rasterise pages to numpy ndarrays with pypdfium2
    pdf = pdfium.PdfDocument(file, password=password, autoclose=True)
    renderer = pdf.render(pdfium.PdfBitmap.to_numpy, scale=scale, rev_byteorder=rgb_mode, **kwargs)
    return [img for img in renderer]

but it blocks all which is not a Path or str:
if not isinstance(self._input, (Path, str)):
     raise ValueError("Can only render in parallel with file path input.")

Sure, you can do that. (However, I would still appreciate if we can finish the generator work some day. :) )

but it blocks all which is not a Path or str:

Yes, unfortunately I had to make that change due to random segfaults when rendering in parallel with bytes/buffers (https://github.com/pypdfium2-team/pypdfium2/issues/164). This is strange because it works perfectly without multiprocessing, and I'm afraid I was unable to locate the cause so far. These failures only appeared after I updated my Python/OS, on the previous install and GH runners the test suite had always passed...

Sure, you can do that. (However, I would still appreciate if we can finish the generator work some day. :) )

but it blocks all which is not a Path or str:

Yes, unfortunately I had to make that change due to random segfaults when rendering in parallel with bytes/buffers (pypdfium2-team/pypdfium2#164). This is strange because it works perfectly without multiprocessing, and I'm afraid I was unable to locate the cause so far. These failures only appeared after I updated my Python/OS, on the previous install and GH runners the test suite had always passed...

Sure i will create an issue to keep it on track :+1:
Mh the problem is also if i try to deactivate the parallel rendering with n_processes=1 it has no effect which is currently a blocker because it seems there is no way to work with bytes inside render ? :sweat_smile:

Mh the problem is also if i try to deactivate the parallel rendering with n_processes=1 it has no effect which is currently a blocker because it seems there is no way to work with bytes inside render ? 😅

Well, you can use the page-level renderer which is not parallelized, something like [page.render(...) for page in pdf].
However, that is not recommendable. I would write a tempfile instead, will be much faster than bytes without concurrency.
Also previously, where (unsafe) parallel bytes rendering was possible, I doubt if it was good since you then have the pdf in memory n_processes+1 times, which can be suboptimal for big pdfs.


Maintenance/Support

Hey! I was looking for an HTML parser for a React Native project and reached here which seems to be the most recent parser library I found. Does this package work with Expo/React Native?

Yes.


TypeError: 'type' object is not subscriptable

Getting following error after installing.
Steps to reproduce:

python3
from globox import ObjectDetectionEval

ObjectDetectionEval is the old package name, you can either import everything from globox with the wildcard:
from globox import *

Or selectively import classes and functions:
from globox import AnnotationSet


feat: support sha1 integrity hashes

pnpm also supports sha1 hashes for integrity checks (see here: https://github.com/pnpm/npm-registry-client/blob/6d9f7ec580979e97c3fbc3fa17ea26f2a410d96f/lib/publish.js#L88). This PR adds support for them.

The error you had usually means that there's a dependency in package.json which isn't present in the lockfile, so it doesn't get downloaded
Are there other hash algorithms that are supported by pnpm? If yes, it might make sense to add support for those as well


Unmaintainable README-language.md

Having a readme for every language is simply unmaintainable - every time a readme is updated, it would need to be updated in every file, whereas I, for instance, don't know all the languages for which README.md exist in this repo :p.
I see two approaches that could be taken here:

keep only the English version of the README.md file
automate translation of the README.md file

The automation could even be accomplished via pre-commit task that would send a request to GPT API if README.md was changed and commit the newly created files for other languages. This could be also accomplished in a GitHub workflow.
Nevertheless, is it necessary to have the readmes in any other language than English? There are browser extensions that allow translation of the website content, so I don't think individual users would have a problem with that.
Hence, I suggest to keep only the English version of README.md

Hi, i can work automating the translation process. Could i get this issue assigned to me?

Hi, i can work automating the translation process. Could i get this issue assigned to me?

Can you share your thoughts on What would be your plan of addressing this issue? Will you use openai GPT or deepl or google translate?

automating

Theres a free tool called argos translate.
Heres an example of a workflow
name: Translate README

on:
  push:
    branches:
      - main
    paths:
      - 'README.md'

jobs:
  translate:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Install Argos Translate
      run: |
        pip install argos-translate

    - name: Translate README to Spanish (as an example)
      run: |
        argos-translate-translate --from-lang en --to-lang es README.md README_es.md

    - name: Commit translated README
      run: |
        git config --local user.email<EMAIL_ADDRESS>        git config --local user.name "GitHub Action"
        git add README_es.md
        git commit -m "Added Spanish translation for README" || echo "No changes to commit"
        git push

Hi, i can work automating the translation process. Could i get this issue assigned to me?

Can you share your thoughts on what would be the plan of addressing this issue?
Will you use openai GPT or deepl or google translate?
If its gpt, whose key are you going to use and who is going to be paying for it?
My personal opinion is that automating this process inside this repo would only bring down the value of the project as it is unnecessary, thus generating additional maintenance costs.

Im confused about how this would bring down value to the repository? It would be a low overhead cost and appeal to a more global audience

Hi, i can work automating the translation process. Could i get this issue assigned to me?

Can you share your thoughts on what would be the plan of addressing this issue? Will you use openai GPT or deepl or google translate? If its gpt, whose key are you going to use and who is going to be paying for it?
My personal opinion is that automating this process inside this repo would only bring down the value of the project as it is unnecessary, thus generating additional maintenance costs.

Update, the Argos translate thing did not translate text very well between languages. What is your thoughts? DeepL has a free tier. We could even use Google translate?