Pylance Language Server gets "stuck" with bad cache, won't reparse changed file fully, requires language server restart

Environment data
VS Code Help Info:
Version: 1.62.1 (system setup)
Commit: f4af3cbf5a99787542e2a30fe1fd37cd644cc31f
Date: 2021-11-05T10:57:55.946Z
Electron: 13.5.2
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Windows_NT x64 10.0.18363

Language Server version: v2021.11.1 (from extension window)
OS and version: See above
Python version (& distribution if applicable, e.g. Anaconda): 3.7.9

Expected behaviour
Pylance to continually, and correctly, update its analysis as files are edited.
Actual behaviour
Pylance linting regularly get confused or "stuck", and is partially giving results of both old and new file changes. It appears to get "stuck" with bad/partial parsing feedback and will not reanalyze a file upon save or even close and reopen of a file (seems to have it cached).
The command: Python: Restart Language Server resets everything and it works for a little while before having the same issue.
I have not been able to play with settings, extensions, or attempt detailed logging to see if the issue can be narrowed down. I have a non-trivial codebase this is failing on and since I have a workaround of regularly restarting the language server, have not been able to attempt to create a minimal reproducible code sample.
I do not believe I saw this behavior until a recent VS Code update (not sure if it was this most recent one for the version I am currently running, or maybe one prior to this),, and was using Pylance a decent amount, although sporadically, over the past few months. I have a fairly new employee who recently got all setup with the same setup as myself (in general, with small differences), and I saw this issue on his PC a few days ago, so it does not appear to be something specific to my PC. This occurs on more than one of my team's Python projects/repos for myself as well.
I generally have multiple VS Code instances open to multiple projects.
Logs
I added the trace logging property to the .vscode/settings.json in the root of my git repo for one of the projects I'm experiencing this on, but I get this error:

However, I was able to add it to the workspace file.

I don't see much going on in the log beyond stuff like this:
Background analysis message: resumeAnalysis
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\conftest.py ...
[BG(1)]   parsing: d:\code\mca_tools\tool_apl_idl\conftest.py [fs read 0ms] (0ms)
[BG(1)]   binding: d:\code\mca_tools\tool_apl_idl\conftest.py (0ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\conftest.py ...
[BG(1)]     parsing: d:\code\mca_tools\tool_apl_idl\.venv\Lib\site-packages\_pytest\config\__init__.py [fs read 0ms] (16ms)
[BG(1)]     binding: d:\code\mca_tools\tool_apl_idl\.venv\Lib\site-packages\_pytest\config\__init__.py (6ms)
[BG(1)]     parsing: d:\code\mca_tools\tool_apl_idl\.venv\Lib\site-packages\pluggy\__init__.py [fs read 1ms] (1ms)
[BG(1)]     binding: d:\code\mca_tools\tool_apl_idl\.venv\Lib\site-packages\pluggy\__init__.py (0ms)
[BG(1)]     parsing: d:\code\mca_tools\tool_apl_idl\.venv\Lib\site-packages\pluggy\_hooks.py [fs read 1ms] (3ms)
[BG(1)]     binding: d:\code\mca_tools\tool_apl_idl\.venv\Lib\site-packages\pluggy\_hooks.py (1ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\conftest.py (31ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\conftest.py (31ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\run_me.py ...
[BG(1)]   parsing: d:\code\mca_tools\tool_apl_idl\run_me.py [fs read 1ms] (2ms)
[BG(1)]   binding: d:\code\mca_tools\tool_apl_idl\run_me.py (0ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\run_me.py ...
[BG(1)]     parsing: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\this.pyi [fs read 1ms] (1ms)
[BG(1)]     binding: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\this.pyi (0ms)
[BG(1)]     parsing: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\antigravity.pyi [fs read 0ms] (0ms)
[BG(1)]     binding: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\antigravity.pyi (1ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\run_me.py (3ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\run_me.py (5ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\idl\TLog.py ...
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\src\idl\TLog.py ...
[BG(1)]     parsing: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\datetime.pyi [fs read 0ms] (3ms)
[BG(1)]     binding: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\datetime.pyi (2ms)
[BG(1)]     parsing: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\shutil.pyi [fs read 0ms] (2ms)
[BG(1)]     binding: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\shutil.pyi (0ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\src\idl\TLog.py (18ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\idl\TLog.py (18ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\idl\__init__.py ...
[BG(1)]   parsing: d:\code\mca_tools\tool_apl_idl\src\idl\__init__.py [fs read 1ms] (1ms)
[BG(1)]   binding: d:\code\mca_tools\tool_apl_idl\src\idl\__init__.py (0ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\src\idl\__init__.py (0ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\idl\__init__.py (1ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\kvconfig\__init__.py ...
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\src\kvconfig\__init__.py (0ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\kvconfig\__init__.py (0ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\kvconfig\tests\__init__.py ...
[BG(1)]   parsing: d:\code\mca_tools\tool_apl_idl\src\kvconfig\tests\__init__.py [fs read 0ms] (0ms)
[BG(1)]   binding: d:\code\mca_tools\tool_apl_idl\src\kvconfig\tests\__init__.py (0ms)
[BG(1)]   checking: d:\code\mca_tools\tool_apl_idl\src\kvconfig\tests\__init__.py (0ms)
[BG(1)] analyzing: d:\code\mca_tools\tool_apl_idl\src\kvconfig\tests\__init__.py (0ms)
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
[FG] parsing: d:\code\mca_tools\tool_apl_idl\src\kvconfig\KvConfig.py (19ms)
[FG] parsing: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 1ms] (16ms)
[FG] binding: c:\Users\Mike.Leiter\.vscode\extensions\ms-python.vscode-pylance-2021.11.1\dist\typeshed-fallback\stdlib\builtins.pyi (12ms)
[FG] binding: d:\code\mca_tools\tool_apl_idl\src\kvconfig\KvConfig.py (0ms)

Code Snippet / Additional information
Some things that may be relevant:
Example 1 - Intellisense impact
Note the weird mis-coloration that occurs in this code screenshot after the issue presents, below. This includes words switching between different colors in the middle of words multiple times, with multiple different color types. It's possible we are seeing the coloration shifted by a few rows (the colors are right for a different row), given that the ghost errors/warnings appear to also be shifted (see example 2):

Example 2 - Ghost Linting errors
I added a new (fake) function - this error especially seems to generate linting errors when a new function or method is added to code after it happens. All the syntax is correct from an indentation perspective, etc., and all this code was linting with no errors right before this:

And the corresponding errors in the "Problems" window:

In the above, the final warning ("csv_encoding" is not defined), is actually legitimate, but the yellow squiggly is not in the right place!  It's reporting position [48, 34], but the actual error is on line 46 ([46, 34]).
After restarting the Language Server, the mistaken errors go away, and the legit warning has moved to the correct location, and all code coloration/formatting issues are resolved.

Example Requirements.txt
Note that I have installed the local project with pip in editable mode (i.e. py -m pip install -U -e . from the root of the repo), and am using a src folder, with 2 package underneath it. I have pytest running with auto-discovery, but defaulting to no coverage, and run with coverage manually from the command line.
There don't appear to be any issues with the code executing or debugging. Everything seems to be okay other than the Pylance linting and the code language formatting as shown in the examples above.
-e git+ssh://git@xxx/xxx.git@xxxHASHxxx#egg=xxx
astroid==2.8.5
atomicwrites==1.4.0
attrs==21.2.0
colorama==0.4.4
coverage==6.1.2
importlib-metadata==4.8.2
iniconfig==1.1.1
isort==5.10.1
lazy-object-proxy==1.6.0
mccabe==0.6.1
packaging==21.2
platformdirs==2.4.0
pluggy==1.0.0
py==1.11.0
pylint==2.11.1
pyparsing==2.4.7
pytest==6.2.5
pytest-cov==3.0.0
pytest-runner==5.3.1
toml==0.10.2
tomli==1.2.2
typed-ast==1.5.0
typing-extensions==<IP_ADDRESS>
wrapt==1.13.3
yapf==0.31.0
zipp==3.6.0

XXX

@LightCC, thanks for the report. This is likely a bug in our parser. Would you be able to send us the code file so that we can reproduce it here? Thanks

This doesn't sound like a parser bug. It's too non-deterministic (works some times, not others). Smells more like a data corruption between the client and the language server. It's odd that this is the first report that we've heard of it though.
@LightCC, under what circumstance does it occur? When first opening your project? After editing the code in a file? After pasting code? After performing a code action or "replace all" operation? If you can discern any pattern, that would be a helpful clue.
Also, if you close and reopen an affected file, does the problem go away? If no, does modifying the file (e.g. adding and then deleting a single character) make the problem go away?
What file encoding does VS Code think the file is using? Is it "UTF-8" or something less common? (The encoding is displayed in the lower right part of the VS Code window.)

I think best way to figure out is if we can get from users the text file that shows the issue so that we can get live repro. we time to time got bug report on text and parse tree got out of sync. so far we fixed some but there might be more.

Hey guys, can't give up the code files in complete format as they are work files. I'm happy to try anything or run logs, etc. or try something. I did this partly in case others were seeing the same thing or if there was anything I could do with logs or to capture it, and partly hoping there was a workaround, etc.
I do have word wrap on and I've seen some bugs with that in the past - just an idea, since the error seems to misplace where the code is, rather than operating on old code, it might just be shifted due to word wrap. I remember something about changes to word wrap a few versions ago.
@erictraut Mostly utf-8 encoding, though some of the CSVs are utf-8-sig. I constantly have a ton of files open with multiple workspaces, have my caches turned up a bit and Code was using 5-6 Gb of memory last I checked, but it's all pretty bland, as far as I can tell.
I was struggling a few weeks back with importing as I have multiple packages in this project, but got that all straightened out and have the project properly installed as a Python package now (py -m pip install -e .' in the root) using setup.py, installed to a venv. It's a mix of package, relative and init.py` imports.
I haven't been able to discern any patterns, beyond what I described above. It happens when it happens, but seems to be kicked off by editing a file, especially in a way that adds new lines, functions and indentation blocks. It does NOT appear to ever reset itself, and I tried a lot of things - i.e. you can't just ctrl-z or manually reverse the edit to get it back, so it seems to be going into a bad state when this happens. Until I searched and found out about the Language Server Restart command I was tearing my hair out and tried quite a few ways of potentially forcing it (close/reopen files, close/reopen vs code).
Today on primarily some other files in the same repo compared to yesterday it's mostly gone away, but last night it seemed like it was showing up very regularly (enough that I mapped Restart Language Server to a key combo).

@LightCC any update?

please re-open if this is still an issue.

I also got this Ghost Linting problem. after the pylance extension update and reload, initially the linter seems work fine and after a dozen of seconds, I start to get non-sense lint errors, e.g. indentation error  for the first few lines of import. It seems that these errors are for a complete different file. I tried reload the window from F1 drop down menu, but it didn't help. The linter didn't regains sanity, until I choose restart language server like the OP suggested in this issue.

FYI @zcattacz I experienced the same issue, appears to be introduced in version 2022.11.30, fixed in (Pre-Release) 2022.11.41 (at least it is working for me)

@sleeplessghost  Thanks, good to know. But since it has been happening for quite some time and I found a quicker fix than reload, It no longer bothers me that much : rename the file to arbitrary name wait 1 second, and rename it back, the linter will immediately regain sanity.


pmohal CAC Update [Initial] / MB-8626

Description
This is part of the procedure to do end to end testing which requires uploading a file generated with the help of a CAC [see https://github.com/transcom/mymove/wiki/use-mtls-with-cac ]
Please reference Jira: https://dp3.atlassian.net/browse/MB-8626

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against 95d39ab6fd639779a932696e00b778e0db9195db

One other nit ... can you change the title of the issue to be more descriptive?  The titles end up in our deploy notes.  If there's a JIRA ticket for this, include that in the title as well.

@reggieriser Done and Done, let me know if you need me to do anything else


[core] Task exceeded time limit

The latest unstable build will report 'Task exceeded time limit', but the latest release version not.
Already have '-DJADX_CLS_PROCESS_LIMIT=6000'

Why do you close your issue https://github.com/skylot/jadx/issues/1267 and now open a new one?
I think you misunderstood me. I was talking about an environment variable not a Java system property (this is what you have done).

@google-mirror can you share apk sample? Timeout is quite big, so looks like something goes wrong during decompilation and without sample it is hard to say that exactly.

@google-mirror can you share apk sample? Timeout is quite big, so looks like something goes wrong during decompilation and without sample it is hard to say what exactly.

https://dldir1.qq.com/wework/work_weixin/WeCom_android_<IP_ADDRESS>81_100001.apk
Only 'defpackage' and 'com.tencent.wework.**' included.

Wow that app is really a monster, 110MB of dex code. Decompiling the full app requires quite some time (40 minutes or more) even on a modern CPU with 6 threads and enough RAM.
@skylot I have not checked it but my guess is that the number of classes to decompile only covers the classes selected by the include filter. But I assume that recursively also the classes used by those classes have to be decompiled, too. So the number of classes to decompile my be incorrect in this case and in the end causing the timeout.

@jpstotz yeah! This is a great sample for performance testing. Thank you @google-mirror!
Right now I am profiling it and found/fixed several issues:

some errors not caught and killing processing threads
too many synchronizations at class index, as it executed with class decompilation, which cause processing threads stuck
many more to come 🤣

I have not checked it but my guess is that the number of classes to decompile only covers the classes selected by the include filter.

I will check this, thanks!

I commit several fixes to speed up decompilation and indexing, redone usage info collection. Please check 👍🏽

@skylot Maybe it's time for a faster dom tree algorithm?

@asLody thanks for reminder! Sure, it is always nice to use linear algorithm instead of quadratic (wiki)

I commit several fixes to speed up decompilation and indexing, redone usage info collection. Please check 👍🏽

It doesn't work.
ERROR: Decompile task execution failed
java.util.concurrent.ExecutionException: jadx.core.utils.exceptions.JadxRuntimeException: JavaClass not found by ClassNode: com.tencent.wework.R$raw
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at java.desktop/javax.swing.SwingWorker.get(SwingWorker.java:613)
	at jadx.gui.ui.MainWindow.waitDecompileTask(MainWindow.java:550)
	at jadx.gui.ui.dialog.CommonSearchDialog$LoadTask.doInBackground(CommonSearchDialog.java:537)
	at jadx.gui.ui.dialog.CommonSearchDialog$LoadTask.doInBackground(CommonSearchDialog.java:527)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: jadx.core.utils.exceptions.JadxRuntimeException: JavaClass not found by ClassNode: com.tencent.wework.R$raw
	at jadx.api.JadxDecompiler.getJavaClassByNode(JadxDecompiler.java:462)
	at jadx.core.utils.Utils.collectionMapNoNull(Utils.java:216)
	at jadx.core.utils.DecompilerScheduler.lambda$buildBatches$0(DecompilerScheduler.java:37)
	at jadx.core.utils.Utils.collectionMap(Utils.java:205)
	at jadx.core.utils.DecompilerScheduler.buildBatches(DecompilerScheduler.java:37)
	at jadx.gui.JadxWrapper.buildDecompileBatches(JadxWrapper.java:92)
	at jadx.gui.jobs.DecompileTask.scheduleJobs(DecompileTask.java:58)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.runJobs(BackgroundExecutor.java:103)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.doInBackground(BackgroundExecutor.java:98)
	at jadx.gui.jobs.BackgroundExecutor$TaskWorker.doInBackground(BackgroundExecutor.java:77)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at jadx.gui.jobs.BackgroundExecutor.lambda$execute$0(BackgroundExecutor.java:44)
	... 3 common frames omitted

@google-mirror I commit a fix for that issue. Sorry for regression 😢


[composer] when object is created remotely during a session it is not added to the graph

Repro:

two peers in a space
create a document on one
document should show up in the second peers tree but it doesn't

Notes:

specific to the space root folder

doesn't happen in subfolders

folder logic is separate

peers see new folders but not new documents

Resolved by #5669


luci-app-firewall/luci-base/luci-mod-admin-full: some fixes and improvements

luci-app-firewall: only show SNAT template if there are more then one zone
luci-app-firewall: on forward rule change pre-selection to wan
luci-app-firewall: only show portforward template if there are more then one zone
luci-mod-admin-full: add hint on backup restore that there will not be a factory reset before
luci-base: fix browser.htm template icon view if "cbi_init()" is not called no icon will be found

Thanks for merging


max number of scenes should be 10

aipresenter should have a "check" of the script to make sure it does not violate the rules. One of them being 10 scene max

Instead we are going to do #71


What does not work with Synapse right now

COPY INTO sometable FROM 'https://somecontainer.dfs.core.windows.net/somefolder/'
says:
Configuration property pgsdwh.dfs.core.windows.net not found.
And yes, my endpoint is ADLv2. Curently, coty into works with blob.core.windows.net

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e651729e-3c87-4b5f-22c6-4db750f0eed9
Version Independent ID: 60781c40-06b8-3b7f-e55b-d02011d7f89e
Content: COPY INTO (Transact-SQL) (preview) - (SQL Data Warehouse) - SQL Server
Content Source: docs/t-sql/statements/copy-into-transact-sql.md
Product: sql
Technology: t-sql
GitHub Login: @kevinvngo
Microsoft Alias: kevin

@kartas39 -- Sergei, thank you for your feedback.  Please see these resources:

DBA Stack Exchange
Stack Overflow

You are welcome to post your issue here:

SQL Server Feedback

I am closing this issue now.  You are welcome to @ mention me for any followup.
We hope to hear from you again.
cc @kevinvngo for awareness

@WilliamAntonRohm  William, i believe this is not an issue, but just such functionality, described in documentation, does not exists in Synapse. In general, COPY does not work with ADL2 endpoints at the moment.

@kartas39 -- Sergei, thank you for clarifying.  This appears to be a product issue.  Please file it here:

SQL Server Feedback

COPY Into works fine with ADLS Gen 2 Endpoints , unfortunately you did not provide sufficient information to review why it was failing. The path which you used was incorrect this 'https://somecontainer.dfs.core.windows.net/somefolder/' should  be
'https://storageaccount.dfs.core.windows.net/container/folder'.

I see the same error and i do use the format that you suggest (although  might also be a specific blob name such as / – at least I suppose that is allowed since it works with the ".blob." endpoint).

please follow this blogpost on how to use COPY Into,
https://techcommunity.microsoft.com/t5/azure-synapse-analytics/how-to-use-copy-into/ba-p/1298242


pull request for e2e test at 01/31/2017 05:26:13

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Update ammonite to 2.5.9

About this PR
📦 Updates com.lihaoyi:ammonite from 2.5.8 to 2.5.9
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

🔍 Files still referring to the old version number
The following files still refer to the old version number (2.5.8).
You might want to review and update them manually.
website/docs/reference/cli-options.md
website/docs/release_notes.md

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.lihaoyi", artifactId = "ammonite" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.lihaoyi", artifactId = "ammonite" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

Suspended by #2223


Proposal for feature store (feast) connector

Draft proposal for the feature store connector discussed in #4.
@rcrowe-google @hanneshapke @michalbrys Would appreciate your feedback

@googlebot I signed it!

Feature stores are often capable of feature engineering also, but I'd suggest that for this project we specifically do not support feature engineering in the feature store.  My concern is that I don't want to introduce the potential for training-serving skew.  If we can create some sort of abstracted specification for portable feature engineering that can be applied with features stores, then I'd suggest that should be a follow-on project.

A general ping on this one.  Do we feel that the proposal is sufficiently solid to implement?

A general ping on this one. Do we feel that the proposal is sufficiently solid to implement?

I think it's clear enough to start with a first cut. Some questions from my side (which shouldn't be blocking)

Does it make more sense to start with a FeastConnector and generalize to a FeatureStoreConnector afterwards? I don't feel too strongly.
We have to_xxxx() methods in Feast on our response objects like RetrievalJob and OnlineResponse. We (Feast contributors) can also implement conversion to native Tensorflow types on our side.

We have to_xxxx() methods in Feast on our response objects like RetrievalJob and OnlineResponse. We (Feast contributors) can also implement conversion to native Tensorflow types on our side.

Something like online_response.to_tf_example_list()?  If so, then yes that seems like it would be useful in general, and not just for this component.  Or maybe online_response.to_tf_dataset()?

We have to_xxxx() methods in Feast on our response objects like RetrievalJob and OnlineResponse. We (Feast contributors) can also implement conversion to native Tensorflow types on our side.

Something like online_response.to_tf_example_list()? If so, then yes that seems like it would be useful in general, and not just for this component. Or maybe online_response.to_tf_dataset()?

Happy to follow your lead here. Which would you prefer for the connector?

We have to_xxxx() methods in Feast on our response objects like RetrievalJob and OnlineResponse. We (Feast contributors) can also implement conversion to native Tensorflow types on our side.

Something like online_response.to_tf_example_list()? If so, then yes that seems like it would be useful in general, and not just for this component. Or maybe online_response.to_tf_dataset()?

Happy to follow your lead here. Which would you prefer for the connector?

I think online_response.to_tf_dataset() is more generic and I would prefer that unless you have a strong preference for online_response.to_tf_dataset() @rcrowe-google @woop.

We have to_xxxx() methods in Feast on our response objects like RetrievalJob and OnlineResponse. We (Feast contributors) can also implement conversion to native Tensorflow types on our side.

Something like online_response.to_tf_example_list()? If so, then yes that seems like it would be useful in general, and not just for this component. Or maybe online_response.to_tf_dataset()?

Happy to follow your lead here. Which would you prefer for the connector?

I think online_response.to_tf_dataset() is more generic and I would prefer that unless you have a strong preference for online_response.to_tf_dataset() @rcrowe-google @woop.

You seem to have online_response.to_tf_dataset() twice?

We have to_xxxx() methods in Feast on our response objects like RetrievalJob and OnlineResponse. We (Feast contributors) can also implement conversion to native Tensorflow types on our side.

Something like online_response.to_tf_example_list()? If so, then yes that seems like it would be useful in general, and not just for this component. Or maybe online_response.to_tf_dataset()?

Happy to follow your lead here. Which would you prefer for the connector?

I think online_response.to_tf_dataset() is more generic and I would prefer that unless you have a strong preference for online_response.to_tf_dataset() @rcrowe-google @woop.

You seem to have online_response.to_tf_dataset() twice?

My bad, I meant let's go with online_response.to_tf_dataset() unless you have a strong preference for online_response.to_tf_example_list().

Note that although I closed this (!) we can still use this thread for discussion.

Hi, id really like to help out on this. My email is<EMAIL_ADDRESS>

@rcrowe-google can we merge this?

Reopening the PR


update english

Update english up to date with develop from COZ.

Better ?

It's better now.

I created a pull on your branch with a suggested rewording, which is just a suggestion.
The English still isn't quite right, for example:
During the first 2 million blocks (approximately one year)...
the second year (blocks from 2 to 4 million)
The item in brackets should be consistent - so both the number of years, or both the number of blocks, should be inside the brackets.

good to go.

@canesin please merge this pr when you think that it's ready.


Allow wildcards in whitelist attributes

Support validation on data attributes.

I thought this was going to be about wildcards, but it is framed around a new method for data- / custom "data" attributes. attributes. On review of the issue I still lean towards a wildcard match. Like 'data-' or 'ng-' etc. I think that will be more flexible and simpler to use than another method. It can be implemented in the isSafeAttribute method, doesn't need another data structure to retain them,

Fixed.

Sorry for the late reply. I'm not moving ahead with this, as I think the implementation can be simpler, as I suggested in my earlier comment.


[BUG]: The Subworkflow Completes Even Though The Latest Task Is Scheduled When Optional Task Included

Describe the bug
Hello, I have had opened a issue earlier #2297 and we got recommendation to update conductor version from v2.30 to the 3.x since some fixes related to problem are entered.
We have updated the our conductor and encountered same issue again.
To desribe issue again, the subworkflow which has two retryable(one is optional) task and also fired under fork-join is getting marked as completed even though the latest task is still scheduled.
We have programmed both task to be never completed, so there is no chance that tasks are indeed completed but they marked as scheduled. The occurance seems random and happens even under light load of conductor. For example, I have started 4 exact workflow and bug occured.
It's seems that Conductor V3.x resolved the most of the issue but there is edge case with optional tasks.
Details
You can reproduce the bug by using these definitions. You may need to run couple of workflow and wait at least 2-3 hours until bug occurs. Please note that it may take more time.
Conductor version: Conductor v3.06. We are using it with Elastic Search and Redis
Workflow definition:
conductor_bug_test_main.txt
conductor_bug_subworkflow_test.txt
Task definition:
conductor_bug_tasks.txt
Expected behavior
The latest task should have continued to be executed.
Screenshots
Main workflow:

Subworkflow:

Task Detail page of Subworkflow:

+1

+1

Add a Join task, according to the docs it might allow the tasks to finish.

@CHURLZ Can you explain to put the join task where? I guess you are saying to put a join task to end of decision case but I couldn't find any info about that in conductor doc. By the way, we have already a join task after fork join.

@aravindanr I have now uploaded workflow execution jsons to details section of my main comment; https://github.com/Netflix/conductor/issues/2459#issue-996043851

@aravindanr Hello, is there any update about issue?

@OguzhanUlucay I am not able to reproduce the bug by running the workflow manually several times. I am working on an automation script to run the workflow periodically to check if its reproducible.
While analyzing the code, i see possibilities of workflow state becoming unpredictable if there "locking" is not enabled. Do you have it enabled? If yes, is it zookeeper or redis?

@aravindanr No, We have not enabled the locking. Would you propose to open it? We may open it via redis.


Configure Initial Setup for SAP Mobile Cards

Tutorial URL: https://developers.sap.com/tutorials/cp-starter-digitalexp-mobilecards-1-setup.html
Please specify the step you are referring to Step 3.
Hello I have a Problem to initialize the Service.

Is there a technical Problem?

@marcobuescher Thank you for bringing your issue to our attention.
Do you continue to face the issue? If so, can you please attach a screenshot of your SAP Cloud Platform Mobile Services application list.
Thank you for your support.

@marcobuescher Thank you for your feedback. We haven't heard from you in a while, so we are closing the issue.
If you still have questions, feel free to reopen the issue.


feat(client): ✨ add google analytics

Initialized Google Analytics
Log Sign Up Events for Sign Up Methods
Log Lobby Creation
Log Lobby Joining
Log Games Starting
Log Games Ending
Log Account Deletions

If there is anything you think would be informative to know that I didn't log then let me know. I'll add it.

These all seam like things that could be logged on the server side. Intuitively it seams like logging on the server side would be better because we don't need a bunch of network requests on the user's device.
What do you think?

These all seam like things that could be logged on the server side. Intuitively it seams like logging on the server side would be better because we don't need a bunch of network requests on the user's device.
What do you think?

Yeah, we talked about it on the discord. Also, are there any other events we should log that would be meaningful to us?


Does Fairseq support gradient checkpointing?

Hello,
As far as I understood by reading the Pytorch docs and some other material, the gradient checkpointing seems to be the way to handle very large input strings. Moreover, it seems that Pytorch already supports it natively so no need to do big hacks in order to implement it.
So, may I ask if it is already implemented in Fairseq?
Thank you very much,
Amin

Yes, checkpointing is done entirely in the model, so you just need to follow the pytorch docs and update the model code appropriately. We have done this for for some recent projects, but the checkpointing code can get a bit messy, so we haven't merged it to master.


ConcurrentTimedRotatingFileHandler rotating gzip files fails

in init.py line 842 in doRollover()
https://github.com/Preston-Landers/concurrent-log-handler/blob/874188b4a6a2b2995e99e463fb0533077334d8e8/src/concurrent_log_handler/__init__.py#L840-L842
s is the full filename /path/mylog.log.2023-11-15.1.gz
doing s + gzip_ext makes it /path/mylog.log.2023-11-15.1.gz.gz and throws an error, causing it rotate logs but never delete and never update the .__mylog.log.lock` file so you'll end up with a new file every time a log message is added
Recommended Fix
Change the above to
            for file in self.getFilesToDelete():
                os.remove(file)

Screen shots of debugger

To replicate issue
use a log config like this
        "concurrent_timed_log_handler": {
            "level": "INFO",
            'class': 'concurrent_log_handler.ConcurrentTimedRotatingFileHandler',
            'when': 'midnight',
            'interval': 1,
            'backupCount': 2,
            "use_gzip": True,
            "maxBytes": _bytes_5mb,
            "formatter": "standard_formatter",
            "filename": os.path.join(LOG_DIR, "mylog.log"),
        },

In your log dir, create some log files that are over 2 days old, ex: mylog.log.2023-01-01.1.gz
update the .__mylog.lock to an old timestamp like<PHONE_NUMBER>
Then try logging, it should roll over with every log made

Thanks for the report. Currently investigating / replicating. I'd like to capture it in a unit test before fixing it. I'm glad someone is using the Timed mode, at least.

Thanks @Preston-Landers , try adding this to your TEST_CASES
"use_timed=True, maxBytes=1KiB, interval=2, log_calls=1_000, use_gzip=True": TestOptions(
        use_timed=True,
        log_calls=1_000,
        min_rollovers=5,
        log_opts=TestOptions.default_timed_log_opts(
            {
                "maxBytes": 1024,
                "backupCount": 5,
                "interval": 5,
                "use_gzip": True,
                "debug": True,
            }
        ),
    ),

seeing two errors
[MainThread 3855 Tue Nov 28 14:42:32 2023] Rolling over because of size

FileNotFoundError: [Errno 2] No such file or directory: '/Users/moynihan/projects/concurrent-log-handler/output_tests/stress_test.log.2023-11-28_14-42-32.1.gz.gz'

then it starts rolling over until the filename is too long
[MainThread 3854 Tue Nov 28 14:42:33 2023] Unable to do rollover: [Errno 63] File name too long: '/Users/emoynihan/projects/concurrent-log-handler/output_tests/stress_test.log.2023-11-28_14-42-<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.76.gz'

Ah i see you already pushed an update!  Great!

Yep, sure did. I'll push a new release with this fix sometime this week. Thanks!


Remote operations on dask dataframe

When operating on remote data using dask dataframe, is dask automatically using all the network transfer bandwidth or there is a way to set the transfer rate limit?

Dask is just calling functions like requests.get (for gcs) or boto (for s3)
or arrow (for hdfs) under the hood.  You might want to consult those
projects to see if there is a way to limit bandwidths.
On Mon, Mar 4, 2019 at 1:46 AM R. Patrick Xian<EMAIL_ADDRESS>wrote:

When operating on remote data using dask dataframe, is dask automatically
using all the network transfer bandwidth or there is a way to set the
transfer rate limit?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/4540, or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszB1WOUWQySQD9irWAsKZ_3dxZ_pNks5vTM9PgaJpZM4bbvbl
.

Thanks, Matthew!


Update upstream

Update upstream to include the memory fix that boosts performance for free :)

tried it locally and it looks good to me. I think the trufflehog check is a false positive.


package issue

Package google_api_headers:linux references google_api_headers:linux as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via platforms: linux: default_package: google_api_headers or add an inline implementation to google_api_headers via platforms: linux:
pluginClass or dartPluginClass.

Can you share a bit more details of how you're getting the errors?

Same here after a flutter pub get
Try `flutter pub outdated` for more information.
Package google_api_headers:ios references google_api_headers:ios as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: ios: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: ios:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:linux references google_api_headers:linux as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: linux: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: linux:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:macos references google_api_headers:macos as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: macos: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: macos:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:windows references google_api_headers:windows as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: windows: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: windows:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:ios references google_api_headers:ios as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: ios: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: ios:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:linux references google_api_headers:linux as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: linux: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: linux:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:macos references google_api_headers:macos as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: macos: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: macos:` `pluginClass` or `dartPluginClass`.

Package google_api_headers:windows references google_api_headers:windows as the default plugin, but it does not provide an inline implementation.
Ask the maintainers of google_api_headers to either avoid referencing a default implementation via `platforms: windows: default_package: google_api_headers` or add an inline implementation to google_api_headers via `platforms: windows:` `pluginClass` or `dartPluginClass`.

"I am facing the same error after upgrading to the latest version of Flutter (stable channel). Have you resolve it?"


Add base project files with webpack build

Set up yarn and pull in least required packages to make a build
Create a readme
gitignore
etc.

https://github.com/seanforyou23/patternfly-seed/commit/b9f1ab0ff461ae7a29ded78f7d0d54212378a1d1


Ci/rosetta-cli checks

Description
Currently the only automated tests are covering cardano-rosetta-server. We should be running rosetta-cli check scenarios of limited scope in CI if possible, or at least regularly to provide post-integration coverage. The Docker image build should form part of integration acceptance.
Proposed Solution
ci: smoke test the Docker image

Builds using the available cache from nominated image on Dockerhub, runs the image, then calls the /network/status endpoint.

test: Add rosetta-cli-checks nightly job

Implements build-cardano-rosetta action
syncs a data snapshot from S3, which is passed as a host mount
Installs the version of rosetta-cli using hardcoded commit sha.
Composes configuration at runtime from modules using jq,
establishing 3 scenarios.
Validates configuration files
Runs scenarios: byron, shelley transition, shelley in epoch with
reward payouts
Makes a new remote data snapshot

Looks good!


Cascader: computePresentContent when status change

Please make sure these boxes are checked before submitting your PR, thank you!

[x] Make sure you follow Element's contributing guide (中文 | English | Español | Français).
[x] Make sure you are merging your commits to dev branch.
[x] Add some descriptions and refer relative issues for you PR.
Closes #16209

@yyjjqq94 加一下测试


Algolia Classic Theme does not work with Create React App Webpack configuration

Description
npm run build with the Create React App pipeline results in an apparent syntax error within algolia's autocomplete-theme-classic package:
Hi @Haroenv, I can reproduce this bug with a blank Create-React-App. My organization is running into the same issue with Algolia CSS.

Here is the git repo: https://github.com/hansod1/reproduce-css-bug
Here is the output of npm run build:

➜ npm run build

><EMAIL_ADDRESS>build
> react-scripts build

Creating an optimized production build...
Failed to compile.

<EMAIL_ADDRESS>ParserError: Syntax Error at line: 1, column 26

Reproduction

Clone this repo: https://github.com/hansod1/reproduce-css-bug, which was a completely stock "create react app" install. I added the algolia CSS import to App.JS and I added the Algolia dependencies to package.json.
run npm run build
Observe syntax error printed (see above)

Expected behavior
No errors, I expect the app to build in Create-React-App
Environment

OS: OSX
Browser: No browser
Autocomplete version: 1.2.1

as said in the previous issue (https://github.com/algolia/autocomplete/issues/586), we can't do anything around it, as it's a postcss bug (https://github.com/hansod1/reproduce-css-bug). There's not much to do, as even updating the value parser I still had an error.
As a workaround, you can patch the library to add spaces: https://github.com/hansod1/reproduce-css-bug/pull/1
Note that still, this is a bug in postcss, not in the css of autocomplete, the values are valid.
I guess as a workaround, we could possibly distribute unminified files too (@francoischalifour, what do you think?)?

@Haroenv Yep, that would at least provide an escape hatch. I added it to our backlog.

Update: We'll address this by next week by providing a non-minified CSS build.


Nextflow : fails to query SGE queue

Describe the bug
Using the executable package by nextflow, querying the queue in SGE results in invalid status line: squeue: error: Invalid user: ?.
See also nextflow-io/nextflow#1644.
This occurs on our cluster so I’m not sure it’s reproductible.
Steps To Reproduce
Steps to reproduce the behavior:

Starts an SGE job using nextflow that runs in more than a few seconds.
Look into ~/.nextflow.log
The job is considered by Nextflow as
Process PrepareGenome:BWAIndexGenome (1) terminated for an unknown reason -- Likely it has been terminated by the external system even though it’s still running.

Expected behavior
A normal execution without errors.
Additional context
As a hack, I used the nextflow script instead of the nextflow executable.
url = "https://github.com/nextflow-io/nextflow/releases/download/v${version}/nextflow";
The script does its own install so I’m aware that it is against Nix philosophy.
Another alternative would be to compile from source but nextflow uses gradle.
Notify maintainers
@Etjean
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.18.0-348.el8.0.2.x86_64, Rocky Linux, 8.5 (Green Obsidian), nobuild`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(root): `""`
 - channels(1565): `"nixpkgs"`
 - channels(1629): `"nixpkgs-22.05"`
 - channels(1696): `"nixpkgs"`
 - channels(1038): `"nixpkgs"`
 - nixpkgs: `/Home/Users/apraga/.nix-defexpr/channels/nixpkgs`

Update: I have the same issue with both nextflow executable and nextflow script.  It looks like Java cannot get the user name properly with nix.
A workaround is to enforce the username to nextflow by prefixing nextflow runs with NFX_OPTS=-D"user.name=apraga"

This still occurs with latest nextflow 22.10.6 (patched according to #224242).

Update: the following derivation works when starting nextflow from an interactive node.
  postFixup = ''
    wrapProgram $out/bin/nextflow \
      --prefix PATH : ${lib.makeBinPath nativeBuildInputs} \
      --set JAVA_HOME ${openjdk17.home} \
      --add-flags "-c ${nextflow-config}" \
      --set NXF_OPTS "-D\"user.name=\$USER\""
  '';

However, if nextflow is started from a compute node, the issue is still there.


pkg/ifuzz: another x86 regression test

Add a different decoding failure that occurred with iset.ModeProt16. This one is also fixed by https://github.com/google/syzkaller/pull/4733.

Before sending a pull request, please review Contribution Guidelines:
https://github.com/google/syzkaller/blob/master/docs/contributing.md

How are you finding them? Are there more bugs? Can we make the fuzz test find all differences with XED?

How are you finding them? Are there more bugs? Can we make the fuzz test find all differences with XED?

These two bugs were reported by syzbot on TestGenerate/x86 at the same time with two different seeds<PHONE_NUMBER>281506768 and<PHONE_NUMBER>629800279). I didn't specifically look for them :)
Because initially x86_test.go only supported ModeLong64, I couldn't make the second test case to pass, so I dropped it (thought it might be a different bug). But then I found out I just need to supply a different mode, and decided to include this test as well, because it tests a slightly different configuration.
I am not sure XED will help finding these cases better than TestGenerate, because:

bugs are seed-dependent and might not manifest instantly;
bugs like the one described in https://github.com/google/syzkaller/pull/4733 do not manifest when encoding/decoding a single instruction. They require multiple instruction patterns to match the same prefix of the text, and that is unlikely to happen if those patterns have different length.


Add GLHYMPS porosity and permeability maps to Geoserver

Add 2 maps for catchment metadata to Geoserver: Porosity and Permeability.
Write WPS to extract data through a process

@richardarsenault Do you have a link to these layers or do we need to look into options?

Hi, yes here's a (time-sensitive) link: https://www.dropbox.com/transfer/AAAAAEHGEyuXY6_ogcWoo6wcn_4Jvdf9na4M8xVCiOaSePn6dB6UbVI
Thanks!

@richardarsenault If you can keep that link active until tonight, I can upload the files to our GeoServer in the next day. Collecting it via process will take a bit more time.

Hey Richard, I honestly can't recall. Feel free to resend the link!

Alright, here it is! Be careful, the compression ratio is a bit crazy:
https://www.dropbox.com/t/KRtwjXCP8NAU1Zni
Thanks!


"Mogoș" is listed in the drop-down menu

Description:
"Mogoș" is listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Mogoș" in the search bar.

Expected results
The location is listed in the drop-down menu as "Mogoș, Alba (Mogoș)".


[Rancher2] Provide Windows Server version context for Rancher

Relevant issue: https://github.com/rancher/docs/issues/2991
The motivation for the PR is in the issue above.

Fixed @aiyengar2's nits.


feat: add initial setup for portal and cx-iam

Description
This PR adds the helm charts from portal and cx-iam (versions from 3.2/23.09 release) to the umbrella chart.
It doesn't contain tests in this sense and for a proper proper e2e testing functional tests as wells as more config would need to get added, still, for an initial integration in the umbrella helm chart this PR should be sufficient.
Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review:

[x] DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
[x] Copyright and license header are present on all affected files

PR looks straight forward. Are these failing tests ok, @almadigabor ? I don't get the reason for the failure from error messages for both failed checks.

I don't think the errors are related to the pull request. Kyverno fails as some of the dependencies have not set runAsNonRoot in the pod specification. Helm test fails with the edc dataplane pod if I get it correctly.

@carslen @almadigabor thanks for reviewing! Last weeks I successfully tested the install on my local Minikube.
IMO the issue with ct here, maybe upgrading the helm testing action would be worth try or replacing it entirely with a simple helm install.

I just tried to start this branch locally in my minikube and had following problems:

two pods will keep restarting (processes-worker and e2e-sharedidp)
some pods does not have the given Helm install name as prefix

IMO the issue with ct here, maybe upgrading the helm testing action would be worth a try or replacing it entirely with a simple helm install.

Regarting to your last comment, @evegufy: I do not think we should replace the ct with a helm install - this test framework is quiet common and will print a detailed log.
For example: in the last run of this pr you can see that the runner might be to small to run that heavy load:0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
We might need to use larger runner or try to limit the load.

I just tried to start this branch locally in my minikube and had following problems:

two pods will keep restarting (processes-worker and e2e-sharedidp)
some pods does not have the given Helm install name as prefix

IMO the issue with ct here, maybe upgrading the helm testing action would be worth a try or replacing it entirely with a simple helm install.

Regarting to your last comment, @evegufy: I do not think we should replace the ct with a helm install - this test framework is quiet common and will print a detailed log.
For example: in the last run of this pr you can see that the runner might be to small to run that heavy load:0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
We might need to use larger runner or try to limit the load.

Thanks for testing!
Regarding 'processes-worker': that's cronJob, and it's normal that it restarts every couple of minutes
Regarding 'sharedidp': what's your local OS? I tested on Ubuntu. If you're using Mac with Apple Silicon, the reason for the restarts could be the mismatching cpu arch. The image is only available for amd64, in this case it can help to provide more cpu cores to your Minikube to enable proper emulation of the different cpu arch, but the performance on Mac will even then probably not be great and the pod will need a couple of restarts to start up.
Good hint regarding the insufficient cpu. We probably need to add a config for the KinD action as the default values aren't sufficient.
I'm still not quite convinced if ct is really suitable for testing such a chart:
When having enabled both keycloak instances (sharedidp and centralidp), I encountered for instance this error which I wasn't encountering on my local Minikube.
Also, in another context, I already ran into https://github.com/helm/chart-testing/issues/332 but this could be not an issue if we upgrade helm testing action.
Still, I'm open to try some other things instead of removing ct.


Pre-start testhosts

Description
Pre-start testhosts for non-parallel runs for the next source in row. So we don't run any tests in parallel but we have a testhost ready to pick up the next source and run it. This makes the testhost startup and adapter lookup amortize better.
Related issue
Fixes part of #3647

Not fully finished, but not very complex. And it has no impact on testhost as it is using just the existing messages. There is also no huge gap between the request start and the actual work, as with test sessions. So we don't have to worry about locking files or anything because they would have been locked anyway, because we are in "a run".
Basically we just split up the run task into two parts, DiscoveryInitialize (RunInitialize), and Discovery (Run). And then run them staggered, so we use more threads (people usually have mulitcore cpus these days), but don't run tests in parallel (people usually don't want that).  So we get the benefit of always having a testhost ready, without the complications of parallel test execution, even if isolated at process level:
|----- prepare ----|---run---|
       |----- prepare ----|  |---run---|
       |----- prepare ----|            |---run---|
                             |----- prepare ----||---run---|

/azp run

Code and tests lgtm but I'd like to have some numbers to show perf gain of the feature on small or big projects.

I'll get back to you with some perf measurements.

Also, fyi, I'd recommend you to avoid using interpolated strings when you don't use variables as it has some unneeded perf cost (e.g. $"my string" should be "my string")

For the interpolated strings, that was my bad, forgot to remove them when transforming the Debug.WriteLine into EqtTrace.Verbose. They should be fixed now.

@cvpoienaru are there any numbers please?

/azp run

/azp run

/azp run

Is it possible to disable this feature somehow? It currently breaks test discovery for us with multiple
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)

Looks like it doesn't help with issue. Will wait for #4469 fixed :)


[OSPO Book] Move appendices to own chapter

Chapter 2 currently has two appendices in the middle of its text (Appendix A and Appendix B). This hurts the reading flow a bit. I would suggest to move these appendices to an own chapter about OSPOs in public administrations.

I've included this issue to review in the WG during our first contributors call this February 👍


Uses strip_html parameter

This PR adds strip_html parameter to the conversations and messages API calls.

This is a breaking change, I guess.

Tests are failing because they weren't updated to the new APIs

Tests are happy now :smile:


Add new pulp repo tags

Do we need some container image tags as well?

I've got some package versions bumps for the release notes:
[will@juno stackhpc-release-train-package-versions]$ jq '. | map(select(.name=="kernel")) | sort_by(.version, .release) | last' centos/8-stream/BaseOS/x86_64/os/*/packages.json
{
  "arch": "x86_64",
  "name": "kernel",
  "release": "408.el8",
  "version": "4.18.0"
}
{
  "arch": "x86_64",
  "name": "kernel",
  "release": "448.el8",
  "version": "4.18.0"
}

[will@juno stackhpc-release-train-package-versions]$ jq '. | map(select(.name | test( "^ovn[^-]*$"))) | sort_by(.version, .release) | last' centos/8-stream/nfv/x86_64/openvswitch-2/*/packages.json
{
  "arch": "x86_64",
  "name": "ovn22.06",
  "release": "24.el8s",
  "version": "22.06.0"
}
{
  "arch": "x86_64",
  "name": "ovn22.09",
  "release": "11.el8s",
  "version": "22.09.0"
}

[will@juno stackhpc-release-train-package-versions]$ jq '. | map(select(.name | test( "^openvswitch[^-]*$"))) | sort_by(.version, .release) | last' centos/8-stream/nfv/x86_64/openvswitch-2/*/packages.json
{
  "arch": "x86_64",
  "name": "openvswitch2.17",
  "release": "31.el8s",
  "version": "2.17.0"
}
{
  "arch": "x86_64",
  "name": "openvswitch2.17",
  "release": "71.el8s",
  "version": "2.17.0"
}

libvirt remains the same

Do we need some container image tags as well?

They are building here: https://github.com/stackhpc/stackhpc-kayobe-config/actions/runs/4115798154

Do we need some container image tags as well?

They are building here: https://github.com/stackhpc/stackhpc-kayobe-config/actions/runs/4115798154

Ok. Bear in mind bifrost is built separately, and will end up with a different tag.

Looks good. I will merge into https://github.com/stackhpc/stackhpc-kayobe-config/tree/refresh/multinode-load-balancers. Which is the wallaby/branch merged in multinode-load-balancers (wallaby multinode environment).

Lets add a release note saying packages have being refreshed and summarise versions of these packages:

kernel version
libvirt version
ovs version
ovn version

Lets add a release note saying packages have being refreshed and summarise versions of these packages:

kernel version
libvirt version
ovs version
ovn version

Yes, let's :)

Lets add a release note saying packages have being refreshed and summarise versions of these packages:

kernel version
libvirt version
ovs version
ovn version

Yes, let's :)

I've added the latest releases and versions from this repo. I don't know why there's so many different versions of each listed, so would appreciate if this can be double checked :)


Proposal: Include Selectors

I think selectors are really awesome for separating state from components and preform calculations.
I started on this effort since I saw both a need from the app, and I think it is a good practice to not store calculated values in the store.
That being said, some of the logic in the app counter acts these changes, so I paused the effort to make sure this goes in the right direction. Additionally, these changes may benefit from including redux-thunk (#232).
I wanted it to be an issue, but as a PR it is simpler to compare.

While I agree that selectors are a useful pattern, I don't believe they're universally necessary, so I'm not sold that they make sense in the kit. It's yet another thing to understand and learn when trying this out, so I lean toward omitting.

I understand that and agree. Closing.


[Core] Temporarily revert eslint version to ~2.2.0

@oliviertassinari @mbrookes
This addresses the eslint+estraverse issue (https://github.com/eslint/eslint/issues/5476).
We can bump the version once a solid conclusion has been reached.

Should we also update the example and the doc?

@oliviertassinari Sure, then we can just revert this commit's changes.

@nathanmarks Thanks.


Deserialize field new

Typed partial deserialization of document by field name. Currently only primitive type fields are supported

Typed partial deserialization of document by field name. All types of fields are supported

Hi,
Good for me for now, merging this, probably can be done some optimization on the deserialization of embedded, we discussed that already in private and will land in a not too far future.
Regards


feat: add shiki and @shikijs/monaco syntax highlighting

This fixes https://github.com/aayushai/TranspilerX/issues/34 and fixes https://github.com/aayushai/TranspilerX/issues/22 (NOTE: 22 is old).
Please note this is a redo of the fix in #34.
With the recent additions made to TranspilerX, there is now a light mode and dark mode.
This pull request incorporates that by switching the theme of the Editor components to use vitesse-dark or vitesse-light depending if the user is on dark mode or light mode respectively.
This pull request also uses the proper code formatting as defined in .vscode/settings.json and .prettierrc (which results in a larger diff in the src/CodeConverter.jsx file due to the fix in formatting).

Hello @aayushai, with the new PR, can you let me know if the fix for both the toast message and the Editor components have been applied?

Now it's perfect.

Awesome, thanks for letting me know!


inotify doesn't work on NFS volumes

Need a way to force a refresh as inotify  doesn't work if we're using NFS to store the media
eg
podman volume create --opt type=nfs --opt o=ro --opt device=mediaserver:/mnt/Videos Videos

podman run -d --name minidlna \
  --net=host \
  -v Video:/opt/Videos \
  -v /var/cache/minidlna:/var/cache/minidlna:Z \
  geekduck/minidlna

Is there any way to get a refresh without reloading/rebuilding the DB. I'm running with a persistent Cache and DB. Start/Stop doesn't cause a re-scan
podman stop minidlna
podman start minidlna

You can change entry point with
exec /usr/sbin/minidlnad  -r -S
For force rescan each start docker

Thanks. I've already created a custom version of entrypoint.sh
#!/bin/sh

if [ -e /run/minidlna/minidlna.pid ]; then
    rm /run/minidlna/minidlna.pid
fi

exec /usr/sbin/minidlnad -r -S

And then Run via
podman run -d --name minidlna \
  --net=host \
  -v VideoVol:/opt/Videos \
  -v Audio:/opt/Music \
  -v /root/containers/minidlna/cache:/var/cache/minidlna:Z \
  --mount type=bind,src=/root/containers/minidlna/entrypoint.sh,target=/usr/bin/entrypoint.sh,relabel=shared \
  geekduck/minidlna

To force a rescan podman restart minidlna

Having an API option to force a rescan rather than restarting the pod would be very useful.