prevent memory test failing due to earlier test leaking memory

What does this PR do?
This PR unblocks master failing in a flaky test.
I observed the other day that the test
tests/trainer/loops/test_evaluation_loop:test_memory_consumption_validation

is failing when running together with other tests, but never failing alone.
When running all tests together and monitoring nvidia-smi, we can see that early on after a few tests the GPU memory goes up to 500MB but never drops back down even when running CPU tests.
This indicates that there is a process hanging and CUDA memory leaking from one test to the next.
This PR attempts to prevent the above test failing due to too much memory being allocated, by measuring only the residual that the current test is using since the start.
Findings
These tests leave some memory behind:
ERROR tests/trainer/test_trainer.py::test_gradient_clipping_fp16 - ValueError...
ERROR tests/tuner/test_scale_batch_size.py::test_auto_scale_batch_size_trainer_arg[power]
ERROR tests/tuner/test_scale_batch_size.py::test_auto_scale_batch_size_with_amp

ValueError: Test left CUDA memory allocated: 6.0 MB

Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃

The memory test isn't on the bugfix branch so changing the milestone :)


So I've got ST-01 and I've managed to connect it to the server.
Not sure If I don't understand how it works with ST-901, but for example, when I trigger shock alarm on the device, I get TEXT message but nothing on the server/web shows that device is in alarm. I've tried to set Vibration alarm, but it does not receive anything.
Similar to Main power off alarm.  When I switch it off, I get Text with alert but nothing to show.
Is it just that this is not implemented ?  I would be happy to work on some implementations for those.
Also it seems like on the map it does not show latest location, I mean time on the Pin on the map, only shows first connection ( I am stationary), while "last connection" on device page, shows later time. Does that mean position wasn't updated ?
Another question ;) - how messages work ? I only seem to get response for RCONF but for anything else it does not show response. Even for rconf it seems like sometimes it does get it sometimes it doesn't

Hello! The device alarms haven't relation with the platform alarms.
The platform alarms are calculated on their own from the position reports sent by the device.
The connection timestamp should be relative to the last time the device has sent a location to the server, regardless of whether it is correct or there is movement.
About messages, the platform receives a connection, reads the content and sends the requested message and waits again for the response in the next message.
You can enable server debug and check the log 8091-connection.log to view all platform-device communications :)

Thanks, I was actually looking into logs and I can see that message with location was sent, but  the timestamp on the map is not changing.  I will play with it further.
Do you think it would be possible to read decide states somehow ? So to receive alerts from the device ? It must be sending something, since the Chinese server reads it. I guess I can debug messages and see what's coming, however currently in the log I've only seen location or RCONF responses from device

Another question ;) Is that the whole message that device sends ?
This one for example was send on ACC ON
[2024-01-29 21:18:09.726 +00:00] [INFO] [65b81611b123f] [CONNECTED] {"id":"65b81611b123f","timestamp":1706563089,"server_port":8091,"client_ip":"xx.xxx.xxx.xx","client_port":30211,"valid":true}
[2024-01-29 21:18:10.605 +00:00] [INFO] [65b81611b123f] [READ] *HQxxxxxxxV1,191237,A,5129.3905,N,00308.7766,W,000.00,000,290124,FBFFF9FF,234,15,24607,781090#
[2024-01-29 21:18:10.720 +00:00] [INFO] [65b81611b123f] [WRITE] *HQ,xxxxxxx,V4,V1,20240129211810#

And this one on POWER OFF
[2024-01-29 21:16:36.938 +00:00] [INFO] [65b815b4e4f0e] [CONNECTED] {"id":"65b815b4e4f0e","timestamp":1706562996,"server_port":8091,"client_ip":"xxx.xx.xxx.xxx","client_port":41554,"valid":true}
[2024-01-29 21:16:38.794 +00:00] [INFO] [65b815b4e4f0e] [READ] *HQ,xxxxxx,V1,191237,A,5129.3905,N,00308.7766,W,000.00,000,290124,FBFFF9FF,234,15,24607,781090#
[2024-01-29 21:16:38.910 +00:00] [INFO] [65b815b4e4f0e] [WRITE] *HQ,xxxxxx,V4,V1,20240129211638#
[2024-01-29 21:17:44.619 +00:00] [INFO] [65b815b4e4f0e] [READ] null

I can't really see anything that indicates alarm type except that FBFFF9FF could be different, on shake it was FFFFF9FF

Yes, these is the status field:

Here the protocol:
SinoTrack Protocol  (1).pdf

Great stuff, That's what I was looking for

You should configure devicd timezone with UTC:

[2024-01-29 21:16:38.794 +00:00] [INFO] [65b815b4e4f0e] [READ] *HQ,xxxxxx,V1,191237,A,5129.3905,N,00308.7766,W,000.00,000,290124,FBFFF9FF,234,15,24607,781090#

UTC TIME 21:16:38 - DEVIDE TIME 191237

I've set that now, but even that time says V1,212332,A,5129.3947,N,00308.7776,W and message (connection) was received on 21:46, Map still shows

Yes, map positions are filtered by signal and distance from previous position, but connection and messages are managed before those filters.

That's what I thought - makes sense. Cool Cool, thank you. I will see if I can implement some alerts detection


Retries should not fail a subgraph

We have seen subgraphs fail with Subgraph instance failed to run: Transport("Unexpected response status code: 429 Too Many Requests"), but this should never be a fatal error and always be retried. The block stream will retry indefinitely, so my guess is that this happens for calls in the instance manager.

@leoyvens @otaviopace will this be solved by the in-progress retry mechanism?

@azf20 yes, we can consider this closed!

I'll close once it's deployed 😉

This was deployed 🙏


Roslyn analyzer CompilationEndAction action is never called in VS Code

I wrote an analyzer that uses a compilation end action. While it works fine in Visual Studio, the end action never seems to get called in Visual Studio Code even with solution-wide analysis enabled.
public override void Initialize(AnalysisContext context)
{
    ...
    context.RegisterCompilationStartAction(ctx =>
    {
        ...
       // MyAction never gets called
       ctx.RegisterCompilationEndAction(cctx => MyAction(cctx.Compilation));

I used the following components in VS 2019 to build the analyzer:

Microsoft.CodeAnalysis.CSharp 3.11 (NuGet)
Latest MSBuild for VS 2019.

Edit: I initially had filed an issue in the Roslyn repo: dotnet/roslyn#67294 since this also appears to happen with JetBrains Rider but they believe this is an IDE-specific issue since all they do is provide the APIs and it's up to the particular IDE or environment to implement calling the relevant actions.
 The |INFO|ROSLYN| and |DEBUG|ROSLYN| entries are from my analyzer. The analyzer logs Compilation start in RegisterCompilationStartAction and it should log Compilation end. when the action registered in RegisterCompilationEndAction executes but this never happens.
Starting OmniSharp server at 3/15/2023, 10:50:59 PM
    Target: c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.DevEx.TestProjects.sln

OmniSharp server started with .NET 7.0.100
.
    Path: c:\Users\Allister\.vscode\extensions\ms-dotnettools.csharp-1.25.4-win32-x64\.omnisharp\1.39.4-net6.0\OmniSharp.dll
    PID: 25888

[info]: OmniSharp.Stdio.Host
        Starting OmniSharp on Windows 10.0.19044.0 (x64)
[info]: OmniSharp.Services.DotNetCliService
        Checking the 'DOTNET_ROOT' environment variable to find a .NET SDK
[info]: OmniSharp.Services.DotNetCliService
        Using the 'dotnet' on the PATH.
[info]: OmniSharp.Services.DotNetCliService
        DotNetPath set to dotnet
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
        Located 1 MSBuild instance(s)
            1: .NET Core SDK 7.0.100 17.4.0 - "C:\Program Files\dotnet\sdk\7.0.100\"
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
        Registered MSBuild instance: .NET Core SDK 7.0.100 17.4.0 - "C:\Program Files\dotnet\sdk\7.0.100\"
[info]: OmniSharp.WorkspaceInitializer
        Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpFormattingWorkspaceOptionsProvider, Order: 0
[info]: OmniSharp.Cake.CakeProjectSystem
        Detecting Cake files in 'c:\Projects\Stratis.DevEx\tests\testprojects'.
[info]: OmniSharp.Cake.CakeProjectSystem
        Did not find any Cake files
[info]: OmniSharp.MSBuild.ProjectSystem
        Detecting projects in 'c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.DevEx.TestProjects.sln'.
[info]: OmniSharp.MSBuild.ProjectManager
        Queue project update for 'c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Stratis.CodeAnalysis.Cs.GuiTestProject1.csproj'
[info]: OmniSharp.Script.ScriptProjectSystem
        Detecting CSX files in 'c:\Projects\Stratis.DevEx\tests\testprojects'.
[info]: OmniSharp.Script.ScriptProjectSystem
        Did not find any CSX files
[info]: OmniSharp.WorkspaceInitializer
        Configuration finished.
[info]: OmniSharp.Stdio.Host
        Omnisharp server running using Stdio at location 'c:\Projects\Stratis.DevEx\tests\testprojects' on host 22564.
[info]: OmniSharp.MSBuild.ProjectManager
        Loading project: c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Stratis.CodeAnalysis.Cs.GuiTestProject1.csproj
[info]: OmniSharp.MSBuild.ProjectManager
        Successfully loaded project file 'c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Stratis.CodeAnalysis.Cs.GuiTestProject1.csproj'.
[info]: OmniSharp.MSBuild.ProjectManager
        Adding project 'c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Stratis.CodeAnalysis.Cs.GuiTestProject1.csproj'
[info]: OmniSharp.MSBuild.ProjectManager
        Update project: Stratis.CodeAnalysis.Cs.GuiTestProject1
[info]: OmniSharp.Roslyn.CSharp.Services.Diagnostics.CSharpDiagnosticWorkerWithAnalyzers
        Solution initialized -> queue all documents for code analysis. Initial document count: 3.
[INFO] Initialize called on thread id 14.
2023-03-15 22:51:10.5154|INFO|ROSLYN|Stratis.CodeAnalysis.Cs initialized from entry assembly OmniSharp, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=7e5bf7b3b470b509 with log file C:\Users\Allister\AppData\Roaming\StratisDev\Stratis.CodeAnalysis.Cs.4205.log...
2023-03-15 22:51:10.5464|INFO|ROSLYN|Loading existing global configuration file...
2023-03-15 22:51:10.5635|INFO|ROSLYN|Debug mode enabled.
2023-03-15 22:51:10.5635|INFO|ROSLYN|Loaded 1 section(s) with 2 value(s) from global configuration at C:\Users\Allister\AppData\Roaming\StratisDev\Stratis.CodeAnalysis.Cs.cfg.
2023-03-15 22:51:10.5635|INFO|ROSLYN|5 existing log file(s) found.
2023-03-15 22:51:10.5635|INFO|ROSLYN|Initialize called on thread id 8.
2023-03-15 22:51:10.5635|INFO|ROSLYN|Runtime already initialized.
2023-03-15 22:51:10.5635|INFO|ROSLYN|Initialize called on thread id 6.
2023-03-15 22:51:10.5635|INFO|ROSLYN|Runtime already initialized.
2023-03-15 22:51:10.5835|DEBUG|ROSLYN|Compilation start...
2023-03-15 22:51:10.5835|DEBUG|ROSLYN|Compilation start...
2023-03-15 22:51:10.5835|DEBUG|ROSLYN|Compilation start...
2023-03-15 22:51:10.5835|INFO|ROSLYN|Loading analyzer configuration from c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\stratisdev.cfg...
2023-03-15 22:51:10.5835|INFO|ROSLYN|Loading analyzer configuration from c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\stratisdev.cfg...
2023-03-15 22:51:10.5835|INFO|ROSLYN|Loading analyzer configuration from c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\stratisdev.cfg...
2023-03-15 22:51:10.5835|INFO|ROSLYN|GUI enabled for compilation, registering action to send compilation message...
2023-03-15 22:51:10.5835|INFO|ROSLYN|GUI enabled for compilation, registering action to send compilation message...
2023-03-15 22:51:10.5835|INFO|ROSLYN|GUI enabled for compilation, registering action to send compilation message...
2023-03-15 22:51:10.8526|DEBUG|ROSLYN|Using namespace Stratis.SmartContracts at c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Test.cs: (0,6)-(0,28).
2023-03-15 22:51:10.9158|DEBUG|ROSLYN|Class Test with attributes  declared at c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Test.cs: (4,4)-(15,5).
2023-03-15 22:51:10.9158|DEBUG|ROSLYN|Class Test declared at c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Test.cs: (4,4)-(15,5).
2023-03-15 22:51:10.9358|DEBUG|ROSLYN|Constructor for type Test declared at c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Test.cs: (7,8)-(14,9).
2023-03-15 22:51:10.9973|DEBUG|ROSLYN|Variable b of type byte[] declared at c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Test.cs: (11,19)-(11,26). Base type: System.Array. Enum type: False. User struct: False. Primitive type: False., Array type: True. Element type: byte.
2023-03-15 22:51:11.0003|DEBUG|ROSLYN|Method invocation Stratis.SmartContracts.SmartContract.SmartContract(Stratis.SmartContracts.ISmartContractState) at c:\Projects\Stratis.DevEx\tests\testprojects\Stratis.CodeAnalysis.Cs.GuiTestProject1\Test.cs: (8,12)-(8,25)

Steps to reproduce

Ensure the "Analyze Open Documents Only" OmniSharp setting is off.
Open a .NET project with a omnisharp.json that enables Roslyn analyzers and uses a Roslyn analyzer that registers a compilation end action.
Ensure compilation end action for analyzer runs under Visual Studio.

Expected  behavior
Compilation end action should run in VS Code OmniSharp.
Actual behavior
Compilation end action never runs in VS Code OmniSharp.

@dibarbet I confirmed that this still repros with the latest C# extension even after enabling full solution analysis. There seem to be few issues here:

The below code skips computing workspace diagnostics for open documents: https://github.com/dotnet/roslyn/blob/f5b6c715a742c56b7cc672e47385508fb4df98cc/src/Features/LanguageServer/Protocol/Handler/Diagnostics/WorkspacePullDiagnosticHandler.cs#L122-L128
This means any non-local diagnostics, such as compilation end diagnostics, are never reported for open documents.

The below code marks diagnostics from DocumentDiagnosticHandler as superseding the ones froom WorkspacePullDiagnosticHandler: https://github.com/dotnet/roslyn/blob/f5b6c715a742c56b7cc672e47385508fb4df98cc/src/Features/LanguageServer/Protocol/Handler/Diagnostics/DocumentPullDiagnosticHandler.cs#L43-L47

I tried to fix this with below changes locally:

Do not skip open documents in WorkspacePullDiagnosticsHandler in ShouldSkipDocument method pointed in 1. above
Remove the Supersedes property setting in DocumentPullDiagnosticHandler
Additionally, also mark the diagnostics from WorkspacePullDiagnosticHandler with Supersedes property set to DocumentDiagnosticIdentifier
Now the compilation end diagnostics do get computed and reported for open documents by WorkspacePullDiagnosticsHandler, but they still do not seem to be reported in VSCode (both in Problems window, and squiggles in the editor). So, something else is filtering them out - I am presuming this is some caching at Editor layer that is causing this. @dibarbet I am not familiar with the LSP contracts here and Editor assumptions/caching logic to diagnose this further. Does the information provided above strike something to you about the possible causes for this?

@mavasani I think there are two problems here

Workspace pull diagnostics generally is not supposed to report diagnostics for open documents.  You could be hitting one of the conditions listed here causing the document diagnostics to always win - https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_diagnostic
Even if you get the workspace diagnostics to successfully report for open documents, remember that a diagnostic report for a single URI is all or nothing - not additive.  So the workspace diagnostics would replace the entire document diagnostics.

I think the only way that this is going to work is for the document pull diagnostics to report these diagnostics.  However I understand that could be slow...

I think the only way that this is going to work is for the document pull diagnostics to report these diagnostics. However I understand that could be slow...

Yes, we definitely do not want to go that route, that can make the squiggles and problems window refresh for active document to become extremely slow. I see couple of options here:

Call it out as a known limitation that compilation end diagnostics will not show up during background analysis, even with FSA enabled. User is expected to run an explicit build command to see these diagnostics in the problems window, and once fixed would need to rerun the build to see them cleared. They will be truly build-only diagnostics in every sense.
Create a new diagnostic source for reporting non-local live diagnostics. We are already going to need 2 sources - build-only and live for #5728. This will be a third source for non-local live diagnostics, and I presume it would be okay for refresh of this source to be slow, as the live source for open documents will still be refreshing majority of diagnostics pretty fast.

I am fine with either stance here.

@mavasani

Create a new diagnostic source for reporting non-local live diagnostics.

I actually really like this idea - and it should actually be possible to do it entirely server side in LSP.  We'd want to confirm with the vscode team, but I think that the identifier here - https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#diagnosticOptions is the diagnostic source.  I'm pretty sure with dynamic registration we can register two sources from the server (through https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#client_registerCapability).
Then when the client calls back into us, it will tell us which source is being used https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#documentDiagnosticParams
So we'd have a regular one and then compilation end for document.  We could consider improving it even more to have a compiler source, analyzer source, and compilation end source.

Fixed with https://github.com/dotnet/roslyn/pull/69541


Fix number overflow in the disk space detection. Closes #379

Damn, this was the cause?
🤦♂

BTW, build failed due to #373 so I have retriggered


error to Execultar gulp command

Error: watch ENOSPC
at exports._errnoException (util.js:746:11)
at FSWatcher.start (fs.js:1172:11)
at Object.fs.watch (fs.js:1198:11)
at Gaze._watchDir (/home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/gaze.js:289:30)
at /home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/gaze.js:358:10
at iterate (/home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/helper.js:52:5)
at Object.forEachSeries (/home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/helper.js:66:3)
at Gaze._initWatched (/home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/gaze.js:354:10)
at Gaze.add (/home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/gaze.js:177:8)
at new Gaze (/home/gerardo/projetos/laravel-angular-cms/backend/node_modules/gaze/lib/gaze.js:74:10)

Hello, what npm version you are using?

<EMAIL_ADDRESS>,
I am developing a similar project with this , and practically , his is the one who found, you know each other?

Will test npm 3 version soon.
This project own myself. i don't know others. If you would like to develop together . contact me.

yes , I want to work on this project , I hope can make this project grow
My idea is to create a front end page for demo
and sorry for any errors do not speak English errors are graces of the translator google


stream claims it was aborted even though it was not

Describe the bug
When using 4.0 beta.4, when I do a stream chat completion, after the for await loop the AbortController behaves as if it was aborted, even though I am quite certain it was not.
This is signal.reason:
aborted DOMException [AbortError]: This operation was aborted
at new DOMException (node:internal/per_context/domexception:53:5)
at AbortController.abort (node:internal/abort_controller:342:18)
at Stream.[Symbol.asyncIterator] (/app/node_modules/openai/src/streaming.ts:69:12)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at streamChatCompletion (/app/src/streamChatCompletion.ts:39:36)
Is it possible that finishing the stream is handled as if it was aborted?
To Reproduce
Stream a chat completion, check the result of the corresponding abortcontroller.signal
Code snippets
No response
OS
Manjaro
Node version
node 18
Library version
v4 beta.4

@DerKorb it's my understanding that the response has to be aborted to avoid hanging. For example, if you remove this line & then run the examples/demo.ts script it will hang indefinitely.
I may be incorrect though & there could be a way to avoid hanging without explicitly aborting.
I'm curious what your use case is for checking if the stream was aborted or not?

I solved this now by checking whether the finish_resean is null.
I have a multi agent setup where agents communicate with each other. I implemented an abort button to stop generation, but then the aborted response would still trigger the next generation from the next agent.

I find it odd, that a successfully completed stream counts as aborted.

Thank you @DerKorb ! We have a fix that will be released in the next version of the beta.

This should be fixed in v4.0.0-beta.5, please give it a try!


Add support for migrations within compute_ctl

Problem
It's easy bungle stuff with regards to permissions or other stuff. Fixing it is hard because we don't have a way to run a query on everyone's computes exactly once. Any migrations we currently do makes it so that we have to maek sure everything is ok to be run many times.
Summary of changes
Adds a new SEQUENCE that keeps track of the migrations we've done, and executes these migraitons.
Checklist before requesting a review

[x] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

@kelvich , you wanted to review this PR

cc @ololobus to be aware = )

also tests are broken

wonder if there is an easy way to feature-flag this and gradually roll it out. or at least region by region.

See for compute feature flags

https://github.com/neondatabase/cloud/pull/9171
https://github.com/neondatabase/neon/pull/6016

I'm going to get use of them in https://github.com/neondatabase/neon/pull/6008

Added support for these feature flags in https://github.com/neondatabase/cloud/pull/9450, will merge this PR once that one finishes


rhel: Fix ovn-controller-vtep systemd-unit parameter

In patch [1] running OVN services with supplying user was added.
Since ovn-controller-vtep is started directly from systemd unit,
unlike other OVN services (whitch use ovn-ctl script), user parameter
should be passed as --user, not --ovn-user.
 I backported directly  - https://github.com/ovn-org/ovn/commit/18d3c8e2a0c3b16e7286ccd2f7d56dfd3a645488
Closing the PR.
Thanks


Crash Report

Hi Cherry! 👋 Just got a random mod loader crash so I'm sending the report your way. I've never had the watcher crash before so I'm not sure what happened... only thing I've been doing differently is launching the game directly from the exe (so I can avoid hotfix 18 for awhile, lol) and it was otherwise working fine all day.
The watcher crashed immediately on game load and then again when I tried to quit it from the toolbar. Ended the task, restarted the watcher, reloaded the game and now it works perfectly fine, so... ????? ¯_(ツ)_/¯
YABG3ML Version: Watcher, v0.2.2
Game Version: BG3 DX11, Patch 6, v<IP_ADDRESS>63283
If there's other logs or info that you might need, just let me know!
crashreport.zip

Could you create some reproducible steps for me so I can try to get the crash too?

Oh god, I'm so powershell illiterate so I don't know if I did this right. I'm trying the debug build and now the watcher exe creates a blank terminal window. Am I supposed to see something there? The exe does work fine though. I was able to generate the log_output at least!
The only thing I could think that I did differently was that I'm in the middle of updating armor stats in a pak, and I think I didn't re-export my mod list prior to launching, like I usually do. I tried to recreate that on both the v0.2.2 and debug watcher builds, but no crash. I have no idea what I did to crash it!
log_output.txt

Oh god, I'm so powershell illiterate so I don't know if I did this right. I'm trying the debug build and now the watcher exe creates a blank terminal window. Am I supposed to see something there? The exe does work fine though. I was able to generate the log_output at least!

It's alright! I'm very certain the crash is completely fixed now anyways, so you don't need to bother with these steps or with trying to reproduce it. I had enough info from the log to figure it out.
But to answer the questions

Am I supposed to see something there?
If you did the > log_output.txt, then it's normal to see no output, yes!

Thanks again for your crash report! I'll release a new version shortly after I'm done with a few maintenance things.

Thank YOU for the speedy fix!! Looking forward to the new version!


Font size configuration support

It's totally possible that I've missed it, but I didn't see any way to configure font sizes. If it already doesn't exist, is it on the roadmap/something you want to support at all? If so, I'd be happy to contribute to the feature, maybe with the smallest amount of direction 🙂

Do you mean a global font size? You should be able to zoom in your terminal, e.g. kitty uses control + and control - for this.
I don't think you can have different font sizes simultaneously, in case that's what you're asking for, as the terminal has a number of rows and every character fits in only one. So you can't have one row with font size 12 and the next with 15 AFAIK.

Ahh, yeah I didn't consider terminal limitations on granularly assigning font sizes. That makes sense. Thanks! Was just a nice-to-have anyways (I wanted to make the title bigger)

Ah yeah, that would be nice but unfortunately it's a general terminal limitation.


⚠ Bofrost.lu has degraded performance

In a0248e1, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 968 ms

Resolved: Bofrost.lu performance has improved in f323b90 after 6 minutes.


Can we integrate Pyleecan with Scilab/XCOS?.

Dear Pierre,
Just curious do you have MTPA, FOC, or DTC options/libraries /Algo available as a functionality/feature?
Can we integrate Pyleecan with Scilab/XCOS?.
If yes please share the procedure to integrate with a few examples...

Hello,
Regarding MTPA there are few functions and tests within pyleecan that will be improved by #558. Otherwise we have few scripts/Tests that uses EEC (Electrical Equivalent circuit) but most of the time simulation are sinusoidal in pyleecan. The exception is PWM that is available for PMSM so harmonic current is also possible (and enforced harmonic current shouldn't be too hard to implement).
There is no coupling with Scilab/XCOS but it could be interesting to investigate that. If you have some time to work on the subject I should be able to guide you.
Best regards,
Pierre

Dear Pierre,
Sure. Let me set up some modules with Scilab/XCOS and get back to you.
Any update/ feedback on FOC, DTC , etc .
Further, do you have support to generate SVM ( 3 or multistage)?.


Test applying Certificate.Verify() to leaf certificate

https://golang.org/pkg/crypto/x509/#Certificate.Verify
https://pkg.go.dev/crypto/x509#VerifyOptions

From what I can tell at a glance it doesn't appear to be too difficult to work in. This should probably be either optional or at least opt-out as not every certificate tested will have a proper signature (e.g., certificates for some network appliances such as storage arrays).

Initial prototyping work suggests that we're going to want to allow the user to specify custom paths to root certificates, intermediate certificates.
I tested against a site that uses a custom CA and the initial verification failed (as expected), but once I provided the CA public key the verification was successful (and I had hoped).

The work for GH-211 is applicable here; adding support for validation is useful, but it should probably be opt-in with explicit notation whether validation is enabled.

for lscert, it can be provided via the summary
for check_cert, it can be listed "below the fold" in the LongServiceOutput details

For the latter, the output currently provided by plugin in the atc0005/check-vmware project can serve as inspiration. In that output, whether the server cert is ignored is explicitly noted.

Potentially relevant from https://go.dev/doc/go1.18#crypto/x509

Certificate.Verify now uses platform APIs to verify certificate validity on macOS and iOS when it is called with a nil VerifyOpts.Roots or when using the root pool returned from SystemCertPool.
SystemCertPool is now available on Windows.
On Windows, macOS, and iOS, when a CertPool returned by SystemCertPool has additional certificates added to it, Certificate.Verify will do two verifications: one using the platform verifier APIs and the system roots, and one using the Go verifier and the additional roots. Chains returned by the platform verifier APIs will be prioritized.
CertPool.Subjects is deprecated. On Windows, macOS, and iOS the CertPool returned by SystemCertPool will return a pool which does not include system roots in the slice returned by Subjects, as a static list can't appropriately represent the platform policies and might not be available at all from the platform APIs.
Support for signing certificates using signature algorithms that depend on the MD5 and SHA-1 hashes (MD5WithRSA, SHA1WithRSA, and ECDSAWithSHA1) may be removed in Go 1.19.

See also:

https://github.com/atc0005/check-cert/issues/359

Note to self: Pretty sure I have a WIP private branch with initial skeleton details in place.
When I pick back up on this, console "The Go Workshop" title, somewhere around page 511. Potentially relevant to:

#209


New representation for scope handler tests

The problem
We need a better way to represent our test cases for scope handlers, because today we end up needing to record a bunch of commands to test edge cases, and the tests aren't easy to read.  In addition, once #1523 is merged, we expect that most scope type development will be done by viewing the output of the scope visualizer, so we'd like test cases that mirror the scope visualizer as closely as possible
The solution
We'd like an ascii representation that is readable by humans and parsable by machine.  The idea is as follows:

We intersperse comment lines among the source code that point out regions in the line above
The comment lines begin with the language's comment token (eg //), followed by !
Each source code line is indented by extra whitespace at the beginning to allow the comment lines to be able to point at the start of the line above if necessary.  If we didn't do this, then a source code line could begin at  the same spot as the // on the line below, so we couldn't point out the beginning of the line.
The content range is marked by {^^^}
The domain is marked by [---]
The removal range is marked by (xxx)
The iteration scope is marked by <***>
We need to figure out interior / leadingDelimiter / trailingDelimiter

Would like to avoid unicode if possible
Here are suggestions from chatGPT:

The interior is marked by /###\
The leadingDelimiter is marked by «@@@»
The trailingDelimiter is marked by §$$$¶

If any range is equal to contentRange, it does not get a line, as that is the implicit assumption
In each of the above, the surrounding brackets are outside of the range, so that we can handle empty ranges by using eg {}
Each range type (eg content, domain, removal, etc) gets its own comment line
If there are overlapping ranges, the range with later start or earlier end gets its own set of lines after the other scope, beginning with //!2 (or //!3, etc).  Note that ranges are overlapping even if they are 1 character apart, because otherwise there's no room for the brackets
The start of the file will contain metadata lines to indicate scope type being tested and language id, eg //!! scope: {type: namedFunction}
 //!! languageId: typescript, plaintext

Note that multiple examples of the same scope type could be present in one file, so that we don't need a million test files like we have today
Note that we don't handle indicating isPreferredOver; need special tests for that

Components

[ ] Automatically generate test case from source code.  Can probably steal some code from #1523
[ ] Parse output to run as a test.  Before running the scope handler, it should remove all the special comment lines and dedent every line by the correct amount.  Note that it should be able to only look at the delimiters (eg {}), as the internal markers (eg ^^^) are just for readability

Considerations

[ ] In addition to tests that just check the dump of all scopes, we want to automatically test edge cases, because in reality scopes aren't used by dumping everything; they iterate from a position.  Can handle this by taking the dumped scopes and creating a dummy scope handler that just yields them all, so they'll get filtered by BaseScopeHandler, and then comparing the output of this scope handler to the real scope handler at various corner cases, eg start position at beginning / end of various scopes and iterating in various directions with various hints.  Should be pretty performant because there won't be that many test cases and it's not doing a full command, just iterating scopes

Examples
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//!                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
//!  ----------------------------------------------------------------------------

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!        ^^^^^^^^^^^^
//!  ----------------------------------------------------------------------------

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!1 {^^^^^^^^^^^^^^^^
        function bbb() {
//!2   {^^^^^^^^^^^^^^^^
          const ccc = "ddd";
        }
//!2  ^^^}
      }
//!1  ^}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!1   ^^^
//!1  [------
         ccc: "ddd",
//!2     ^^^
//!2    [-----------]
       },
//!1   --]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!1  ^^^
#!2     ^
#!3      ^^^
#!4         ^
#!5          ^^^

Here are some alternative representations to consider:

Merge ranges onto one line
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  ------------------xxx^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^-

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  ------^^^^^^^^^^^^----------------------------------------------------------

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!1 {^^^^^^^^^^^^^^^^
        function bbb() {
//!2   {^^^^^^^^^^^^^^^^
          const ccc = "ddd";
        }
//!2  ^^^}
      }
//!1  ^}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!1   ^^^
//!1  [------
         ccc: "ddd",
//!2     ^^^--------
       },
//!1   --]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!1  ^^^
#!2     ^
#!3      ^^^
#!4         ^
#!5          ^^^

Always surrounding brackets
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!                      {^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^}
//!                   (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx)
//! [----------------------------------------------------------------------------]

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!       {^^^^^^^^^^^^}
//! [----------------------------------------------------------------------------]

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//! {^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^}

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!  {^^^^^^^^^^^^^^^^
        function bbb() {
//!2   {^^^^^^^^^^^^^^^^
          const ccc = "ddd";
        }
//!2  ^^^}
      }
//!   ^}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!   {^^^}
//!   [------
         ccc: "ddd",
//!2    {^^^}
//!2    [-----------]
       },
//!    --]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!  {^^^}  {^}
#!2    {^}  {^^^}
#!3     {^^^}

Internal markers on every line of range
value, name, statement, and token
(as above)
namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!  {^^^^^^^^^^^^^^^^
        function bbb() {
//!   ^^^^^^^^^^^^^^^^^^
//!2   {^^^^^^^^^^^^^^^^
          const ccc = "ddd";
//!   ^^^^^^^^^^^^^^^^^^^^^^
//!2  ^^^^^^^^^^^^^^^^^^^^^^
        }
//!   ^^^
//!2  ^^^}
      }
//!   ^}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!   {^^^}
//!   [------
         ccc: "ddd",
//!    -------------
//!2    {^^^}
//!2    [-----------]
       },
//!    --]
     }

No internal markers
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!                      {                                                      }
//! [                                                                            ]
//!                   (                                                         )

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!       {            }
//! [                                                                            ]

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//! {                                                                            }

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!  {
        function bbb() {
//!2   {
          const ccc = "ddd";
        }
//!2     }
      }
//!    }

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!   {   }
//!   [
         ccc: "ddd",
//!2    {   }
//!2    [           ]
       },
//!      ]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!  {   }  { }
#!2    { }  {   }
#!3     {   }

Using `.` to indicate whitespace
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  ....................{......................................................}
//! [............................................................................]
//!  .................(.........................................................)

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  .....{............}.........................................................
//! [............................................................................]

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//! {............................................................................}

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!  {................
        function bbb() {
//!2  .{................
          const ccc = "ddd";
        }
//!2  ...}
      }
//!   .}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!  .{...}..
//!  .[......
         ccc: "ddd",
//!2 ...{...}.......
//!2 ...[...........]
       },
//!  ....]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!  {...}..{.}..
#!2  ..{.}..{...}
#!3  ...{...}...

Using `.` only where necessary
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  .                 . {.                                                    .}
//! [.                 .                                                       ..]
//!                   (.                                                       .)

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  .    {.          .}                                                        .
//! [.                                                                          .]

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//! {.                                                                          .}

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!  {.
        function bbb() {
//!2   {.
          const ccc = "ddd";
        }
//!2    .}
      }
//!   .}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!   {. .}
//!   [.
         ccc: "ddd",
//!2    {. .}      .
//!2    [.         .]
       },
//!     .]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!  {. .}. {.}
#!2    {.} .{. .}
#!3     {. .}

Using markers only where necessary
value
//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  .                 . {^                                                    ^}
//! [-                 .                                                       .-]
//!                   (x                                                       x)

name
//!! scopeType: {type: name}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!  .    {^          ^}                                                        .
//! [-                                                                          -]

statement
//!! scopeType: {type: statement}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//! {^                                                                          ^}

namedFunction
//!! scopeType: {type: namedFunction}
//!! languageId: typescript

      function aaa() {
//!  {^
        function bbb() {
//!2   {^
          const ccc = "ddd";
        }
//!2    ^}
      }
//!   ^}

collectionKey
//!! scopeType: {type: collectionKey}
//!! languageId: typescript

     const aaa = {
       bbb: {
//!   {^ ^}
//!   [-
         ccc: "ddd",
//!2    {^ ^}      .
//!2    [-         -]
       },
//!     -]
     }

token
#!! scopeType: {type: token}
#!! languageId: plaintext

     foo.bar.baz
#!  {^ ^}. {^} .
#!2    {^} .{^ ^}
#!3     {^ ^}

A few questions:

should we put leading / trailing delimiter on same line as content range? They never overlap
should we show removal range if it's just equal to trailing ?? leading?
should we show leading / trailing / removal when it's just token delimiter ranges (ie whitespace)? On the one hand we could do that to write nice test cases for token removal, but on the other hand I worry it will lead to lots of noise for scopes that just get token removal by default, esp if removing them doesn't make much semantic sense. But for those where we actually want those token removal semantics it might be nice to have that tested 🤔. Maybe syntactic scopes could opt in / out of token removal behaviour?

cc/ @AndreasArvidsson @phillco

Please make an example of that I think it sounds reasonable
I believe so. It's not always obvious if it's going to be trailing or leading for a particular case and we might change that algorithm in the future.
I guess we could start with them and if it becomes too noisy we could remove them in the future?

Please make an example of that I think it sounds reasonable

//!! scopeType: {type: value}
//!! languageId: typescript

     const removalRange = getRelatedRange(match, scopeTypeType, "removal", true);
//!                    xxx^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//!  ----------------------------------------------------------------------------

I believe so. It's not always obvious if it's going to be trailing or leading for a particular case and we might change that algorithm in the future.

Mixed feelings.  It will be extra noise I think, and while I do agree it can be unintuitive when you're just using Cursorless, in this case the leading / trailing are marked quite clearly visually.  But easiest to just show them for now, and then see how annoying it is.  If it feels like noise, I think just having a check if target.removalRange.isRangeEqual(target.contentRange.union(target.leadingDelimiter ?? target.trailingDelimiter ?? target.contentRange) is easy to do, and should be fairly intuitive

I guess we could start with them and if it becomes too noisy we could remove them in the future?

Sounds good

Yeah fair. I'm happy with that

Any thoughts about what to do about tabs that are part of ranges? There's no right answer about how many dashes/whatever to use under them. (Can discuss over voice if the answer is long.)

Yes tabs continue to be the bane of our existence in cursorless. See eg

https://github.com/cursorless-dev/cursorless/issues/1586
https://github.com/cursorless-dev/cursorless/issues/1741

I have no idea how to handle them tbh 😅. For some situations we could possibly read the tab width setting, as suggested here, but I don't think that helps us in this situation, as different users might have different settings, and we're baking in one setting to the recorded test.
My instinct is to avoid tabs wherever possible in our tests, but that will potentially cause skew from real usage in the case of Go, where I'm guessing most users will be using tabs due to gofmt
Very much open to ideas here

The only "works always" fix that I am aware of is to fight tabs with tabs: allows tabs anywhere in ranges as a no-op, and put them below any real tabs so that the alignment is always preserved. But that creates all sorts of knock-on problems.
Let's kick around a few ideas over voice sometime.

lolsob https://stackoverflow.com/questions/68333368/tab-stops-are-messed-up-after-rarer-unicode-characters

terrible idea: replace tabs with:
U+2409 ␉ SYMBOL FOR HORIZONTAL TABULATION

not that it helps us, but elastic tab stops are nice:
https://nick-gravgaard.com/elastic-tabstops/
https://pkg.go.dev/text/tabwriter

Another annoying question. (Sorry, thinking through all this stuff a bit.) How will this representation represent comment scopes if/when they exist?

FWIW, I think the "fixed but inconvenient length" problem (how do you keep symbol-by-symbol alignment in the face of grapheme clusters) may be easier: use the correct number of zero-width spaces to preserve alignment.
And yes, there's a opposite type of this category of problem, which is zero-width spaces in content. Maybe we can assume nobody uses those. 😅

Another annoying question. (Sorry, thinking through all this stuff a bit.) How will this representation represent comment scopes? The scope system will pick up the annotations as comments themselves...

That's what the ! is for in //!

I have an idea for a really simple format to test scopes

Folder name denotes language. If all tests in this folders should be the same language there's really no point duplicating the language as a field in the fixture.
Fixture file name contains facet id from: https://github.com/cursorless-dev/cursorless/issues/2010
Scope is derived from facet id
Fixture file only needs example code and the range representations described in this issue

example
typescript/ternary.condition.txt
;;!! true ? 0 : 1
;;!  ^^^^

Me and @josharian just had discussion about my post above. We are both in agreement that third and more verbose format is the way to go. We came up with a few thoughts.

Only one range per header. If we need multiple content ranges we would have multiple content headers. This means that there is never range interleaving and there is no need to denote the start or end of a range.
The only part that's actually gonna be parsed/read from these files are the code at the top and to the delimiter we could have a custom format for the rest of the file if we like. If we in the future need a parsable format we could switch to something like txtars


regex pattern on textinput

Hi,
I have tried below regex pattern and its not worked. From the user entered text I want to extract only the number.

Thanks...

Hello @BeheraMadan - This looks very similar to another issue: https://github.com/microsoft/BotFramework-Composer/issues/6928
Is that another account of yours by any chance?

Yes this is account is specific to project and other is personal.

@BeheraMadan

Did you intend this as a how-to question or something else?
If you intend this as a how-to question, what is the question?
Why didn't you follow Chris's instructions after he closed your other issue?
Did you get my message that I sent you 7 hours ago on Teams?

@BeheraMadan, closing due to inactivity.  Please reactivate if this is still an issue.


Check panics for webpack-tests

Panics for each webpack-test should be checked no matter it's enabled or not.

Bump

Checked by @GiveMe-A-Name , see also https://github.com/web-infra-dev/rspack/issues/8445


ENH: Simplified API for Data Conversions

SciKeras manipulates input data (X and/or y) to make it agree with what Keras expects and to allow users to implement multi input/output models.
I'm opening this issue to centralize discussion surrounding the data transformation API. As discussed in #78 and #79, this API is convoluted and could use improvement. In principle, this is because we do not know anything about the loss function until the Model is compiled, but determining what loss to use may require information about the input data. We need to break this cyclical dependency if we want a clear API. I can think of several ways to do this:

interpret loss=None as "please select an appropriate loss for me given this input data". Data reshaping to match the loss function (ex: one-hot encoding for categorical_crossentropy) will only happen when SciKeras is allowed to determine the loss function.
KerasClassifier/Regressor (but not BaseWrapper) only work with the new model compiling API (#66). This allows the user to select the loss function but we can use information from that to manipulate the input data since we have it before calling model_build_fn.

Both of these initiatives could help address https://github.com/adriangb/scikeras/pull/66#pullrequestreview-495121341.
Neither of these options will likely work for advanced cases (like custom losses or multi input/output models) but in those cases, the users should be advanced enough to handle customizing the data pre/post processing themselves.
Separately, I feel that we should combine {pre,post}process_{X,y}, _check_output_model_compatibility and _utils.LabelDimensionTransformer into a single transformer. We can make these transformers __init__ params for the wrappers (which would allow on-the-fly overriding) or class attributes (which would be similar to the current subclassing interface).

class DataTransformerBase(ABC, TransformerMixin, BaseEstimator):

    def get_meta_params(self) -> Dict[str, Any]:   # allows retrieval of `n_classes_` and such
        return {...}

    def fit(self, x: ndarray) -> None:
        return

    def transform(self, x: ndarray) -> Union[ndarray, Iterable[ndarray], Dict[ndarray]]:
        return x

    def inverse_transform(self, x: Union[ndarray, Iterable[ndarray], Dict[ndarray]]) -> ndarray:
        return np.column_stack(x)

class KerasClassifierFeatureTransformer(DataTransformerBase):

    def __init__(self, loss: Union[Dict[str, Loss], Union[Loss, Iterable[Loss]]]):
        self.loss = loss

    def fit(self, x: ndarray) -> None:
        ...

    def transform(self, x: ndarray) -> Union[ndarray, Iterable[ndarray], Dict[ndarray]]:
        ...

    def inverse_transform(self, x: Union[ndarray, Iterable[ndarray], Dict[ndarray]]) -> ndarray:
        ...

KerasClassifier(
    model=get_model,
    loss=None,
    feature_transformer=KerasClassifierFeatureTransformer
)

I'm open to any input on these intiatives.

I thought about this more, and think the API on master is decent for the user:

Data processing functions ({pre,post}process_{X,y}) are public methods of BaseWrapper (allows the user to customize them for their specific problem).
BaseWrapper: no processing (which enables users to handle their own data and do their own processing).
KerasClassifier and KerasRegressor have special handling for their particular problems (like binary/multiclass/etc, which enables Scikit-learn compatibility).

Some internal refactoring to make the logic simpler might be warranted. This might involve making Keras-specific data processing transformers public, as discussed below.

I feel that we should combine {pre,post}process_{X,y}, _check_output_model_compatibility and _utils.LabelDimensionTransformer into a single transformer.

How far could that go? Wouldn't the transformer depend on the loss function? I'm imagining a refactoring like this:
class KerasClassifier(BaseWrapper):
    def preprocess_y(self, y):
        ests = {"binary": BinaryClassifierTransformer, "multiclass": MultiClassifierTransfomer, ...}
        est = ests[target_type]()
        return est.fit_transform(y), est.meta_params()

If implemented, I think these transformers should be public in case users want to use them.

How far could that go?

The hope would be to encapsulate all data conversions into a single transformer. {pre,post}process_{X,y} conceptually map 1:1 with transform and inverse_transform and in fact are largely based on calling a series of transformers. They also throw values back to the fit/main loop just so that they can be saved and then passed to the counterpart, while if the transformers were classes instead of functions, they'd be able to store that state internally.

Wouldn't the transformer depend on the loss function?

Yes, but I feel that this problem (i.e. the circular dependency) is the same for the current function-based implementation and a transformer-based implementation. I think that regardless of which solution we end up with, we need to figure out how to solve this problem.
I envision something like this:
transformers.py (or something like that; see bottom of https://github.com/adriangb/scikeras/issues/83#issue-708598300 for more detail)
class KerasClassifierFeatureTransformer(DataTransformerBase):

    def __init__(self, loss):
        self.loss = loss   # compiled loss function/functions

    def fit(self, x: ndarray) -> None:
        ...

    def transform(self, x: ndarray) -> Union[ndarray, Iterable[ndarray], Dict[ndarray]]:
        ...

    def inverse_transform(self, x: Union[ndarray, Iterable[ndarray], Dict[ndarray]]) -> ndarray:
        ...

in wrappers.py:
class KerasClassifier(BaseWrapper):
    def fit(X, y, ...):
        ...
        X, y = self._validate_data(X=X, y=y, reset=reset)  # line ~ 607 in master
        self.data_transformer_ = self.data_transformer()  # data_transformer is a class attribute? and __init__ param?
        X = self.data_transformer_.fit_transform(X)
        self.loss_ = ...  # compile losses here? only if user gave loss=None? otherwise don't reshape data?
        self.target_transformer_ = self.target_transformer(loss=self.loss_)
        y = self.target_transformer_.fit_transform(y)
        ...

    def predict(X):
       X = self.data_transformer_.fit_transform(X)
       y = self.model_.predict(X)
       return self.target_transformer_.inverse_transform(y)

Got it. I see what you're saying now. Why aren't Scikit-learn pipelines a suitable replacement for {pre,post}proceses_X?
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.neural_network import MLPRegressor
from sklearn.datasets import make_regression

X, y = make_regression()
X[:, 0] += 400

est = Pipeline([
    ("normalize", StandardScaler()),
    ("keras", MLPRegressor()),
])
est.fit(X, y)

This would also work with post-processing to, as long as the user defined transform or fit_transform. Here's an example with images:

Autoencoder example
from sklearn.preprocessing import MinMaxScaler, FunctionTransformer
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
X, y = load_digits(return_X_y=True)

normalize = MinMaxScaler()  # normalize all pixels to (0, 1)
est = Pipeline([
    ("normalize", normalize),
    ("autoencoder_standin", PCA()),
    ("unnormalize", FunctionTransformer(normalize.inverse_transform)),
])
est.fit(X, y)

I'm not integrating SciKeras in this example, or showing the Keras autoencoder implementation – autoencoders only require transform = predict in their class implementation.

Why aren't Scikit-learn pipelines a suitable replacement for {pre,post}proceses_X?

I don't fully understand how your examples would replace {pre,post}proceses_X, unless we had KerasClassifierTargetEncoder (say that's the transformer that absorbs what {pre,post}process_y does) like this:
est = Pipeline([
    ("normalize", KerasClassifierTargetEncoder()),
    ("keras", MLPRegressor()),
])
est.fit(X, y)

If I understood correctly, I'd say the main issue with that is that the wrappers need access to meta parameters / context of the transformer, eg. to get the number of classes (which would normally be needed to build the output of the model).
But more generally, the output from {pre,post}proceses_y can be a list of arrays, or even lists of arrays of different length, depending on the model's output[s].* If we pass some funky data like this into fit it will end up in _validate_data which will either raise an error or coerce the data back into a single numpy array. So clearly this has to be done after that step.

users with their own model know what loss function they're minimizing

I generally agree. I guess we could just look at the value of loss!=None but if users want to build models that accept multiple types of target (like most sklearn classifiers do) then this would break (since they would want the loss function to be determined based on the type of input). This is a hard one...
* Keras supports a lot more data formats and target formats than ScikitLearn. The default implementation in {pre,post}proceses_{X,y} is converts common ScikitLearn task-types to what a basic Keras model implementing them would expect, thus easing the learning curve for ScikitLearn users. But eventually these users may want to try more advanced stuff, or even stuff that ScikitLearn doesn't really support. Then they can override these methods and output whatever format their model needs. Thus the output from {pre,post}proceses_{X,y} should be expected to be arbitrary data and it should be fed into the model untouched (except for that annoying windows dtype bug we did a workaround for).

I don't fully understand how your examples would replace {pre,post}proceses_X, unless we had KerasClassifierTargetEncoder (say that's the transformer that absorbs what {pre,post}process_y does) like this:
If I understood correctly, I'd say the main issue with that is that the wrappers need access to meta parameters / context of the transformer, eg. to get the number of classes (which would normally be needed to build the output of the model).

I'm not saying anything about processing y for now; I'm only concerned with the processing of X.* I think {pre,post}process_X can be replaced with Scikit-learn pipelines if the population of meta is handled elsewhere. Here's the implementation I'm thinking of:
class BaseWrapper:
    def array_meta(self, X, y):
        return {"X.dtype": X.dtype, ...}

    def _build_keras_model(self, X, y):
        meta = self.array_meta(X, y)
        ...
        model = final_build_fn(..., meta=meta)
        ...
        return model

    def preprocess_y(self, y):
        ...  # still required

    def postprocess_y(self, y):
        ...  # still required

    preprocess_X = postprocess_X = None  # not required

Why isn't that valid? It'd still allow the user to customize metadata via the same method (subclassing). If https://github.com/adriangb/scikeras/issues/83#issuecomment-701137200 is incorporated into the documentation, it'd encourage the use of Scikit-learn transforms.

I think that if anything, we can check self.model_.loss, that way if users compile a model within build_fn they won't get an error, and we don't break any APIs

:+1:
*I specifically chose KerasRegressor in https://github.com/adriangb/scikeras/issues/83#issuecomment-701137200 towards that purpose but didn't clearly express that fact.

I see. But what problem does that solve?
preprocess_X is not meant to be a replacement for Sklearn pipelines/transformers. It is different for two reasons:
(1) Outputs are explicitly inputs to Keras Model's fit and predict, which are different from valid Sklearn Xs.
(2) B

Going back to the discussion of how to break the dependency between the target encoding and loss function, I'm thinking the way to go is to use the loss parameter as follows:
(1) loss={known_loss_fn}. KerasClassifier uses this to encode the input. The values in meta (y_shape_, etc.) correspond to this final encoding. We check that the actual loss function (model_.loss) matches the user-inputted loss function and warn if it does not (no error, that would break the old API and prevent porting a lot of code).
(2) loss!={known_loss_fn}. Do not transform the input to match the loss function. If model_.loss turns out to be a known loss function, show a warning (no error, that would break the old API and prevent porting a lot of code).
Do you think this sounds reasonable @stsievert ?

To be clear, this discussion if for KerasClassifier and BaseWrapper/KerasRegressor aren't relevant, correct? And the relevant code is in the _check_output_model_compatibility and preprocess_y KerasClassifier methods?
Why is it necessary to make the processing code specific to each loss function? Why does some of that happen in _check_output_model_compatibility instead of in preprocess_y? Here's the code I'm thinking of:
https://github.com/adriangb/scikeras/blob/b8f1b63febfb8dfb3c29a3bd6055fc430c8f9f3b/scikeras/wrappers.py#L1051
That's the only line of code I'm seeing that ties the encoding to the loss function. Is that correct?

If I were developing SciKeras, I'd first try to make KerasClassifier a class that a user could feasibly write. I'd probably make some public utilities to make doing basic data manipulation data easier. I'm be inclined to write some code like this:
import scikeras.utils as utils

class KerasClassifier(BaseWrapper):
    def array_metadata(self, X, y):
        n_classes = len(self.encoder_.classes_)
        ...
        return {"X.dtype": X.dtype, ..., "n_classes": n_classes}

    def preprocess_y(self, y):
        transformers = {
            "binary": utils.KerasBinary,
            "multiclass": utils.KerasMultiClass,
            "multilabel-indicator": utils.KerasMultiIndicator,
            "multiclass-multioutput": utils.KerasMultiClassMultiOut,
        }
        self.encoder_ = transformers[target_type]()
        return self.encoder_.fit_transform(y)

    def postprocess_y(self, y):
        return self.encoder_.inverse_transform(y)

I think all the logic in KerasClassifier.preprocess_y could be hidden in utils.Keras*. Then if a user wants to use other parts of KerasClassifier (i.e, the score or predict_proba methods) they only have to write this class:
class CustomClassifier(KerasClassifier):
    def preprocess_y(self, y):
        self.encoder_ = CustomEncoder()
        return self.encoder_.fit_transform(y)

Why is it necessary to make the processing code specific to each loss function?

The practical reason is that that is what the original wrappers did: https://github.com/tensorflow/tensorflow/blob/22d6b01650422e88de48f9baf7a2eb16e6f894a4/tensorflow/python/keras/wrappers/scikit_learn.py#L159. I believe that the intention was to make it easy for users since in sklearn world targets aren't usually one-hot-encoded and they might be confused for multilabel-indicator. But perhaps this is an API that we should break? It doesn't seem that bad to do a single if check in preprocess_y (which is what the implementation would look like if we use the __init__ loss param instead of model_.loss) (more below)

Why does some of that happen in _check_output_model_compatibility instead of in preprocess_y?

Because previously there was no loss parameter, so the only reliable way to to check the loss function was model_.loss, which you don't have until after preprocess_y.


Service registry cleanup

This addresses a few things:

Renaming serviceregistry.ServiceRegistry to serviceregistry.ProviderID. This avoids name stutter and also clarifies the intent of the value.

Introduce new interface serviceregistry.Instance, which includes all of the fields needed by the aggregate.Registry (clusterID, providerID, model.ServiceDiscovery, model.Controller).

Made several controllers implement the new serviceregistry.Instance interface. This is important because kubernetes has its own clusterID that must be set. Allowing the controller to implement the interface means that clusterID only has to be set once.

Moved aggregate.Registry to serviceregistry.Simple. This seemed to be a more natural place for it since we now have the interface. It's odd to need the aggregate package in order to manually construct a simple serviceregistry.Instance.

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_istio
f250856cbe6e6ca018b4c84b8451d58f0dd5829c
link
/test gencheck_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_istio
f250856cbe6e6ca018b4c84b8451d58f0dd5829c
link
/test gencheck_istio

unit-tests_istio
f250856cbe6e6ca018b4c84b8451d58f0dd5829c
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lint_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lint_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test lint_istio

pilot-multicluster-e2e_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test pilot-multicluster-e2e_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test pilot-multicluster-e2e_istio

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lint_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test lint_istio

pilot-multicluster-e2e_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test pilot-multicluster-e2e_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test lint_istio
/test pilot-multicluster-e2e_istio

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-multicluster-e2e_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test pilot-multicluster-e2e_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-multicluster-e2e_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test pilot-multicluster-e2e_istio

lint_istio
cfe22f36f969daa41bb49bfff11cdfdf59b2ef82
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-multicluster-e2e_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test pilot-multicluster-e2e_istio

lint_istio
cfe22f36f969daa41bb49bfff11cdfdf59b2ef82
link
/test lint_istio

gencheck_istio
4044c89e6e05e01b3af213e6611f5a210c44a73b
link
/test gencheck_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-multicluster-e2e_istio
c191cc49dd6abd15c86fbaee9a52a223e1bb5821
link
/test pilot-multicluster-e2e_istio

gencheck_istio
4044c89e6e05e01b3af213e6611f5a210c44a73b
link
/test gencheck_istio

lint_istio
4044c89e6e05e01b3af213e6611f5a210c44a73b
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_istio
8d57d1ca8950f507b5789b30f4d757107dc9483a
link
/test gencheck_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test gencheck_istio


Make *Spec Types Real Types instead of Interfaces

on behalf of @lmb
Make *Spec regular structs instead of interfaces, and make GetELFSections return a "list of specs" (CollectionSpec? not sure). This will allow users to load from ELF, but then tweak the programs if they desire.

As I said in #20 I don't know if making them structs buys us that much other than letting library users do relocating themselves. I think we could solve that by having a parse parameter that delays relocation, and a method on the specs that allow them to specify different relocation parameters, then initiate relocation (it already is based on symbol name anyways).
As far as having a "list of specs". I think that's a good idea, the two argument thing is awkward, and we could probably put the higher level relocation functionality on that.

Maybe you're right, and I'm missing something that requires an interface. I usually reach for an interface if I want to enable multiple implementations of something, which doesn't seem so apply to the *Spec.
What functionality does the interface give us?

Nah, you're right. I think I had an early version of the code needing interfaces, but the elf refactor I did got rid of the need. We can get rid of the interfaces.


Adjust Bosnia and Herzegovina to expect 6-digit numbers, not 7

In response to #342
Since I don't really know anything about how phone numbers in Bosnia and Herzegovina are formatted, I've looked up some companies and pulled their contact numbers, stuck them into the plausibility_spec.rb and countries_spec.rb and then found none to work. (Note that I've anonymised these numbers into simple sequences, not sure if real numbers in the specs are considered a problem...)
From what I've come up across while looking into this I think the numbers should be in the form of leading 0, 2-digit area code, 6-digit number, so I think the countries.rb definition simply had the wrong length implied by the previous split.

Coverage increased (+0.002%) to 98.091% when pulling 1653c02082d0aba0566917b0828337c7ae0e3b70 on DMA57361:master into e7532f9cd8d61600b3288ecb7977e1dcf7e907d9 on floere:master.


Epic: Create a possibility to add a Course as a copy or link to a Cooperation

[ Epic ] Create a possibility to add a Course as a copy or link to a Cooperation
Add a possibility to add a course to a cooperation by two ways: by linking and as a duplicated copy. Currently, the system only supports adding courses as a duplicate copy.
Link: https://excalidraw.com/#json=aErJ68u6pn5EHYV7vbLVh,wrp8pBjiHa23ImjpGYXgkA
It is a complex task that needs to be divided into smaller tasks on both frontend and backend sides.

This epic was successfully accomplished


CaseView: respect visibility condition

Added logic for the tab component to respect the visibility conditions as configured in Pega.
Note that this logic patch should also be applied to SDK-A and SDK-WC when the change is made to SDK-R

There are some lint errors that need to be fixed before we try to merge this change. It may be easier to make the change in a new branch/commit/PR than to edit this git "patch". If we do that, we can decline the PR.

Hi @JEAxero , these linting errors even occur with a fresh install of the ReactSDK. Once those are fixed I will resubmit the request

We can decline this PR since it was superceded by PR 146

Closing. This fix was handled in another PR.


OccupancyGrid display ignores timestamps

The OccupancyGrid plugin (MapDisplay) is transforming the map using a time stamp of ros::Time() instead of current_map_.header.stamp. If this was intentional, perhaps we can add an option to use the header's time stamp? I'd be happy to submit a PR.

I'm not sure if it was intentional or not. @hersh @DLu can either of you think of a reason the current time was used for the transform in the MapDisplay rather than using the timestamp in the header? Either way, having an option (which defaults to the current behavior) would not do any harm.

When the OccupancyGrid represents a grid map, I would assume that header.stamp would correspond to the map-creation time, which for the typical localization scenario would lie far in the past and outside the TF buffer, so I think the current behaviour as a default makes sense for that scenario, but I understand that for different uses of OccupancyGrid this might make less sense.

Nothing comes to mind.

Closed with https://github.com/ros-visualization/rviz/pull/1066


The attribute maxEvictableIdleTimeMillis does not exist in DruidDataSourceFactory

Therefore, if tomcat is used, maxEvictableIdleTimeMillis cannot be configured in server.xml

You're right and thanks for feedback.
And any one can process it?

I'd like to fix this, please assign it to me @kimmking


[Improvement][C++] Use inherit to implement EdgesCollection

Proposed changes

replace the variant of EdgesCollection with inheritance to implement polymorphism of different adj list type edges collection.
Add cpp build process to JAVA CMakeList to make JAVA SDK base on a specific version of cpp SDK, not compile with the latest commit. That would help the breaking change in C++ SDK not to infect JAVA SDK imediately.

Types of changes
What types of changes does your code introduce to GraphAr?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA
[x] Lint and unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if appropriate)

Further comments

I think the best way to adapt this change is getting EdgesCollection from factory function directory.
For details, you need:

Remove 4 supper classes of EdgesCollection;
Remove create function in EdgesCollection;
Write a new factory function in util/GrapharStaticFunctions.java to create EdgesCollection, which maps ConstructEdgesCollection in C++;
Add annotations for EdgesCollection:

For class: @FFIGen @FFITypeAlias @CXXHead
For return objects: @CXXReference or @CXXValue

Modify related usage in test.

If there are new problems, please reach out to me.

@lixueclaire any comments?


tests: bump cOS image version

Needed to unbreak tests. The image which is used to run upgrades needs to have the GRUB menu changes introduced lately
Signed-off-by: Ettore Di Giacinto<EMAIL_ADDRESS>

@davidcassany merging as it is a really minor change and master is broken otherwise.. :man_shrugging:


Underscores in numeric literals.

Since Python 3.6 underscores are allowed in numeric literals.
The underscore cannot be used at the beginning or end, nor can it be
duplicated. The change follows exactly the specification in Python
documentation referenced in the comment above it.

Cc: @brammool for inclusion in runtime

It should be possible to see Python2 scripts properly highlighted, I have several laying around (to be converted some day).
Is there an easy way to switch to that, after seeing some errors?

@brammool
I hope you had a chance to see my reply to the review related to this topic. I already explained my rationale and reasoning there.
I see several options here:

Use the old version of the syntax file before PR #7855 where I already dropped Python 2 keywords/builtins. This is what I mentioned in that reply above. This approach is the least burden for me as a maintainer, but I am advocating it also because this is the exact approach Python Foundation has taken with the end of support for Python 2.
Introduce a setting in the file that would produce two effectively different syntax highlighting approaches. I could do this, but this adds an additional burden to maintain and unnecessarily complicates the code logic with too many if branches, for a diminishing benefit of a dwindling Python 2 code base.
Have a separate Python 2 syntax highlight file, based on the version of my file before PR #7855 that users can instruct Vim to use when editing their old Python 2 code. Is there an easy way to do this in Vim? If it is, this is perhaps the best compromise between the first and second approach above. Setting one option to change highlighting from Python 3 to 2 (second item above) is not much different to setting another option to load a different syntax file (if possible). The latter has the benefit of keeping two separate approaches as separate files and leaving Python 2 file to slowly deprecate.

Let me know what you think and the way you want to proceed.
Considering that we already dropped syntax highlighting for Python 2 keywords/builtins, perhaps we could commit this and I would have a chance to prepare other important Python 3 updates, while also working on the compatibility layer in the way we agree on. If you prefer that we make the decision and implement the compatibility change first, that's also OK. The users who need this feature can always apply this patch directly to their syntax files until then.
Thanks!

Vim has always followed the philosophy to support as many users (and uses cases) as we can, with limited extra work.  Dropping support for Python 2 syntax completely goes against that.  Those files will still exist and occasionally users will open them and look though them.  If it's riddled with error highlighting that is not nice.
I can understand that Python has moved on to Python 3 and no longer wants to spend time on old stuff.  So, what we can do is introduce the "python2" filetype.  Then we can use the previous syntax that was for Python 2 only.  It doesn't need to be maintained (except when someone spots an error).  And we don't do file type detection, the user has to manually set the filetype to "pyton2".  That way we don't need to mix two filetypes in one script.
How does that sound?

@brammool

How does that sound?

That sounds fantastic. That's exactly the third item I was proposing above as the best compromise.
I just didn't know if it would be acceptable to you and if there's an easy setting in Vim (turns out that new syntax type is certainly such a setting).
I think we're on the same page here.
How do you propose we proceed?

I could prepare a variant of my syntax file before PR #7855 with only Python 2 relevant syntax in it as python2.vim. I would be able to do that this weekend at the earliest. I could name it python2 and send it in a new PR, or
We could just use the version before PR #7855 directly as python2.vim, but the disadvantage is that it has straddling support for Python 2/3 mixed in it at that point in time.

Zvezdan Petkovic wrote:

How does that sound?

That sounds fantastic. That's exactly the third item I was proposing
above as the best compromise.  I just didn't know if it would be
acceptable to you and if there's an easy setting in Vim (turns out
that new syntax type is certainly such a setting).
I think we're on the same page here.
How do you propose we proceed?

I could prepare a variant of my syntax file before PR #7855 with
only Python 2 relevant syntax in it as python2.vim. I would be able
to do that this weekend at the earliest. I could name it python2 and
send it in a new PR, or
We could just use the version before PR #7855 directly as
python2.vim, but the disadvantage is that it has straddling support
for Python 2/3 mixed in it at that point in time.

It would be good to just have Python 2 support.  But I suppose the
latest Python 2 version did also include some (future) Python 3 items,
thus we should not go back to before Python 3 existed.  I think the goal
should be to keep it simple and avoid highlighting Python 2 specific
items as errors.
--
Bumper sticker: Honk if you love peace and quiet.
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///                                                                      \
\\        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

why is this still open? Can this be merged?

why is this still open? Can this be merged?

@89z Bram wanted to ensure Python 2 compatibility, while this and one previous change (see comments above) dropped support for Python 2. I can extract the last version of the syntax file that supports both Python 2 and 3, make Python 2 keywords and builtins the default and submit it as a separate Python 2 syntax file. Then this would probably be accepted by Bram for merge.

@brammool The PR ##12066 creates Python 2 syntax file.
Can we proceed with the commit of this change so that I can continue with the updates of other syntax features for Python?

@brammool I see that the content of PR #12066 has been committed in dd60c365cd2630794be84d63c4fe287124a30b97.
Can we proceed with committing this change now?

OK, I'll include this now.  And update the Last Change date in the header.

@brammool
I don't see this merged into the main branch yet.

I need to send out a runtime files update.  I still have items in my inbox that I need to go through...


Discord authenticated context mapping

When trying to map the verified property on the user json object this always resulted in false because discord serializes true as "True" added a tolowerinvariant to counter this.

Thanks for the patch


Left align "schedules" page, "update schedule" page, and "create schedule" page.

When you log in, you get an additional menu option for Schedules.
Currently, in the component javascript/src/main/components/Schedule/ScheduleTable.js, the fields for name and description are centered.
It would be preferable if both the data in the table, and the table headers for these two fields were left aligned.
This can likely be done by simply adding the field align: "left" to the javascript object that defines the table structure, and headerAlign: "left" to affect the headers.
Documentation: https://react-bootstrap-table.github.io/react-bootstrap-table2/docs/column-props.html#columnalign-string-function
This image shows how it looks now.

User Story

As a logged-in user
I can see schedule data left aligned
so that the information is easier to understand and edit

Acceptance Criteria

[ ] Data and header for name column are left aligned
[ ] Data and header for description column are left aligned

Implementation Todos
In javascript/src/main/components/Schedule/ScheduleTable.js find this code and add in the align: "left" and headerAlign: "left" to the object for nameand the one fordescription`.
 const columns = [{
    dataField: 'id',
    text: 'id'
  }, {
    dataField: 'name',
    text: 'Name'
  },
  {
    dataField: 'description',
    text: 'Description'
  },

Testing:
For this change, it may not be necessary to add additional tests, but existing tests should all pass.
However, if you do want to try to write tests for it, this may help:
https://medium.com/@the_teacher/how-to-test-a-react-components-css-styles-with-react-testing-library-rtl-styled-components-43f744334528

Made a new branch: 5pm-1-leftAlign


Cannot start new entries from Todoist

Please follow this guide when creating a new issue. This speeds up the process of replicating the issue and finding a solution.
Environment details

Operation system: Ubuntu 16.04 Google Chrome Version 54.0.2840.100 (64-bit)
Extension version:  1.0.15

Expected behavior

Click on the button of a Todoist task in the list
set details
click start
timer starts.

Actual behavior

Click on the button of a Todoist task in the list
set details
click start
timer does not starts but the button in the task row displays a green Top timer button.

Steps to reproduce the behavior
Additional details

Extension background page shows an error: background.js:545 POST https://www.toggl.com/api/v9/workspaces/1023529/time_entries 400 (Bad Request)
Response value is Conflict with project/workspace: project needs to be in the same workspace as the time entry
I can start tasks from the button in the browser bar as usual.

We are aware of this issue. We'll try to push out a release with the fix today.


[Question] Blockchain Index

Is blockchain loading restricted to a single core?

If so we should fix this.

These labels are quite annoying, we need action not labels.

did you make a pull request? lol

Nou

is this resolved now after pull #706  ?

@jrsikors That merge adds a printf for the load progress, not multi-thread/core support.

we're working on a total rebase in another repository right now, and it will have multi core/thread support, as well as a slew of new rpc commands ;]


chore: bump to Bazel 7.2.1

Getting strange failures on attempted bumps to 7.2.1 and 7.3.1:
(04:00:05) ERROR: /home/runner/work/bazel-lib/bazel-lib/e2e/copy_action/~/.cache/xdg-cache/bazel/_bazel_runner/62e983f9dcfd129cff412924cb4ab3e8/external/bazel_tools/tools/cpp/runfiles/BUILD:3:11: Compiling ~/.cache/xdg-cache/bazel/_bazel_runner/62e983f9dcfd129cff412924cb4ab3e8/external/bazel_tools/tools/cpp/runfiles/runfiles.cc failed: (Exit 1): gcc failed: error executing CppCompile command (from target //~/.cache/xdg-cache/bazel/_bazel_runner/62e983f9dcfd129cff412924cb4ab3e8/external/bazel_tools/tools/cpp/runfiles:runfiles) /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++14' -MD -MF ... (remaining 16 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
~/.cache/xdg-cache/bazel/_bazel_runner/62e983f9dcfd129cff412924cb4ab3e8/external/bazel_tools/tools/cpp/runfiles/runfiles.cc:17:10: fatal error: tools/cpp/runfiles/runfiles.h: No such file or directory
   17 | #include "tools/cpp/runfiles/runfiles.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Getting strange failures on GitHub Actions ephemeral runners compiling tools/cpp/runfiles/runfiles.cc on attempted bumps to 7.2.1 and 7.3.1

Resolved by not setting XDG_CACHE_HOME env.
Setting XDG_CACHE_HOME breaks the build as of Bazel 7.2.0 due to
Bazel on Linux and BSD now respects the XDG_CACHE_HOME environment variable instead of assuming that ~/.cache/bazel is writable. (#21817)

https://github.com/bazelbuild/bazel/releases/tag/7.2.0
See https://bazelbuild.slack.com/archives/CA31HN1T3/p1718650523156059 for more details.


Huge CPU usage

I have used Node.js (0.10.33) quite long time with 'longjohn' in the clustered application. No any bad affect of 'longjohn' was detected there. The application worked stable and used about 20% of CPU time.
Recently I have updated my Nodejs engine up to latest version (0.12.5) and correspondingly did update for all dependencies. The 'longjonh' was updated to 0.2.7 version. But, unfortunately, after restart my application began to eat up to 95-100% of CPU time. The commenting of 'var longjohn = require('longjohn');' fixed the situation and now it's using no more than 20% of CPU time.
Can you please explain what is happening? Is it my fault? Or there is some bug?

it is not recommended to use longjohn in production

Since 0.2.5 (after my pull request was merged), longjohn creates stack trace for EVERY wrapped callback.

I have already sensed this:(
Besides, it contains the memory leak - the memory usage was grown and grown until I commented the usage of 'longjohn' .

My team ran into the same issue while using longjohn during automated tests on Node v0.10.38. v0.2.4 works great, v0.2.7 causes a bunch of our tests to timeout. An example being a test that took 119 ms on v0.2.4 now takes 1580 ms on v0.2.7.

@jeffcharles, @shunanya, please, check with this.

That seems to perform much better. Test times are back to what they were on v0.2.4.

Hey Sergey. Could you send me an email? I'd love to chat about giving you
commit/push creds on longjohn.
On Tue, Jul 7, 2015 at 10:15 AM, Sergey Volkov<EMAIL_ADDRESS>wrote:

@jeffcharles https://github.com/jeffcharles, @shunanya
https://github.com/shunanya, please, check with this
https://github.com/drauggres/longjohn.
—
Reply to this email directly or view it on GitHub
https://github.com/mattinsler/longjohn/issues/46#issuecomment-119217920.

@mattinsler, thank you for your trust. This PR is last, so there is no need in commit/push creds for me.

OK, sounds good.
On Thu, Jul 9, 2015 at 12:13 PM, Sergey Volkov<EMAIL_ADDRESS>wrote:

@mattinsler https://github.com/mattinsler, thank you for your trust.
This PR is last, so there is no need in commit/push creds for me.
—
Reply to this email directly or view it on GitHub
https://github.com/mattinsler/longjohn/issues/46#issuecomment-120052882.

Hey Sergey. Good job.
Since you already upgraded the code, could you please change correspondingly the version number in the package.json so that longjohn can be updated automatically?

@shunanya, done.


Feature: Add Doubly Linked List program in Java (Insertion [start, end, mid], Deletion [start, end, mid], Traversal)

@akgmage, can you assign this to me


feat: allow .env files in project directories to configure environment

Since one of the complexities of managing the xs-dev environment is making sure certain environment variables are sourced in the correct shell profile, maybe per-project configuration can use a conventional .env file to load into the process before certain commands like xs-dev run.
Along with existing configuration variables like UPLOAD_PORT, there could be a DEPLOY_TARGET or DEPLOY_DEVICE setting for the default --device (or platform target) for running projects.

I think a .env file is going to go a long way in improving build configurations with minimal effort. It could be mostly self documenting by naming convention.
I would probably prefer having default settings behind the scenes and then being able to override with a .env, than a .env per project.


Users - Password reset by email

This may need to be split into a couple tasks, But we need an endpoint that processes a request for a new password, sends an email with a link to the user, possibly with a json token in the query string of the link for verification, then another endpoint that accepts this token and an update password.

Have made a suggestion of how this can be implemented. Probably needs to raise issued on the frontend board: for a resetPassword page, if its not made.


Expose port to enable Calls connectivity

Summary
PR adds the necessary configuration to expose the WebRTC (udp based) port (8443) that Mattermost Calls uses for connectivity. Since we are going GA it makes sense to make this change as it removes some burden on the users given the feature is supposed to be enabled by default now.
I tested both cases (with and without nginx) by following https://docs.mattermost.com/install/install-docker.html#deploy-mattermost-on-docker-for-production-use and got everything working correctly once configured accordingly.

World class PR description 🎖


Broken github setup

Hi,
For whatever reason, it no longer works to point to https://github.com/wendellpiez/JATSKit/raw/develop/oXygenJATSframework.xml from the Add-ons panel under Options / Preferences in oXygen. Nothing happens when I select Apply, and the framework does not load.
I can, however, see the zip file at https://github.com/wendellpiez/JATSKit/archive/develop.zip as referenced.
But I seem to have broken the framework xml file somehow. (Radu, any idea?)
Thanks, Wendell

My colleague Alex will look into this today and add his analysis results here.

Hello,
Adding the URL inside preferences is just the first step. Afterwards you have to manually install the add-ons, from the Help menu using the "Install new  add-ons..." action.
Which Oxygen version are you using? Anyway, in the later versions of Oxygen you don't have to go to Preferences at all. Just go directly to Help->"Install new add-ons..." and paste that URL inside the text field on the upper part of the dialog. The two frameworks will appear (JATS and JATSKit) and you can proceed with their installation.
Best regards,
Alex

Alex helped me out ...


GitHub style admonitions not rendering

Looks like GitHub-style admonitions such as this one are not rendering nicely:

I thought that we had implemented this at some point, so maybe a regression?
Would be good to document this style as well and say that it's good for GitHub readmes.

forgot to add that specific type: https://github.com/nf-core/website/blob/main/src/components/octokit.js#L83