Use Steam Linux Runtime (Soldier) container when using Proton 5.13+

According to this comment from Proton developer, Proton 5.13 is supposed to be used inside "Steam Linux Runtime - Soldier" container.
This might be related to issue #152 (See screenshot in https://github.com/lhark/truckersmp-cli/issues/152#issuecomment-780584545) but not 100% sure.
Implementation

Download/update Steam Linux Runtime when Proton is downloaded/updated

Default path is $XDG_DATA_HOME/truckersmp-cli/SteamRuntime, can be overridden by specifying --steamruntimedir option (e.g. --steamruntimedir ~/.steam/steamapps/common/SteamLinuxRuntime_soldier)

When starting with Proton 5.13 (or newer), use run like this: /path/to/SteamLinuxRuntime_soldier/run -- /path/to/proton run ...
The AppId of Steam Linux Runtime is stored in new file steamruntime.json

How to use "run" command
/path/to/SteamLinuxRuntime_soldier/run ([OPTIONS...]) -- [COMMAND] ([ARGUMENTS...])

Example
(Execute "cat /etc/os-release" inside container)
$ ~/.steam/steamapps/common/SteamLinuxRuntime_soldier/run -- cat /etc/os-release
PRETTY_NAME="Steam Runtime 2 (soldier)"
NAME="Steam Runtime"
VERSION_ID="2"
VERSION="2 (soldier)"
VERSION_CODENAME=soldier
ID=steamrt
ID_LIKE=debian
HOME_URL="https://store.steampowered.com/"
SUPPORT_URL="https://help.steampowered.com/"
BUG_REPORT_URL="https://github.com/ValveSoftware/steam-runtime/issues"
BUILD_ID="0.20210126.1"
VARIANT=Platform
VARIANT_ID="com.valvesoftware.steamruntime.platform-amd64_i386-soldier"

(Execute "cat /etc/os-release" without container)
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.10 (Groovy Gorilla)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.10"
VERSION_ID="20.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=groovy
UBUNTU_CODENAME=groovy

I strangely haven't encountered any problem regarding using Proton 5.13 without the steam runtime yet. Do you have any idea why I (and you?) don't see any issues here?

You and I are using recent Linux distributions where required libraries and executables are luckily compatible with Proton 5.13
There may be some "binary incompatible" cases on some users' systems

Pure x86_64 userspace (No x86 32-bit libraries installed)
Library-specific ABI incompatibility (e.g. libfoo.so.4 vs. libfoo.so.5)
Libc ABI incompatibility
C++ ABI incompatibility

I guess that Proton 5.13 is supposed to be used inside the container in order to be more distro-independent.

Ok, sadly it's not working for me. There's just nothing happening at all after the discord bridge startup (doesn't this has to also run in the runtime?).
Without the discord bridge it's crashing directly:
** INFO **  Proton output:
ERROR: ld.so: object '/home/lucki/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
ERROR: ld.so: object '/home/lucki/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
esync: up and running.
Setting breakpad minidump AppID = 227300
Steam_SetMinidumpSteamID:  Caching Steam ID:<PHONE_NUMBER>2794497 [API loaded no]
pid 63210 != 63209, skipping destruction (fork without exec?)

Unfortunately I can't reproduce it on my Ubuntu 20.10.
Discord bridge too uses the container:
    if major >= 6 or (major == 5 and minor >= 13):
        run_in_steamrt = os.path.join(Args.steamruntimedir, "run")
        argv = [run_in_steamrt, "--"]
        wine = argv.copy()

    # ...

    wine_command = os.path.join(Args.protondir, "dist/bin/wine")
    wine.append(wine_command)

    # ...

    if not Args.singleplayer and not Args.without_wine_discord_ipc_bridge:
        ipcbr_proc = start_wine_discord_ipc_bridge(argv, env)

So it's already successfully running on your machine using Steam Linux Runtime.

Does singleplayer work?
Does WINEPREFIX=/path/to/prefix /path/to/SteamLinuxRuntime_soldier/run /path/to/Proton/dist/bin/wine /path/to/truckersmp-cli.exe output the usage (Usage: truckersmp-cli GAMEDIR MODDIR) ?
Try some options for run/pressure-vessel-wrap. If the usage of Steam Linux Runtime is wrong, please correct me.
I often see WriteProcessMemory with argument "[]" failed with error 5: Access denied. error and then it fails to start multiplayer (See #146). I guess this is one of Proton 5.13 specific issues, but after some retrying it starts.

 The script is getting messy…

Does singleplayer work?

No, crashing right away: steam-227300.log

Does WINEPREFIX=/path/to/prefix /path/to/SteamLinuxRuntime_soldier/run /path/to/Proton/dist/bin/wine /path/to/truckersmp-cli.exe output the usage (Usage: truckersmp-cli GAMEDIR MODDIR) ?

Yes.

Try some options for run/pressure-vessel-wrap. If the usage of Steam Linux Runtime is wrong, please correct me.

The usage seems correct. I can see with argv = [run_in_steamrt, "--verbose", "--"] that the command is correct.
pressure-vessel-adverb[188725]: Command-line:
pressure-vessel-adverb[188725]:         '/home/lucki/.local/share/truckersmp-cli/Proton/proton'
pressure-vessel-adverb[188725]:         'run'
pressure-vessel-adverb[188725]:         '/mnt/gamehub/native2proton/Euro Truck Simulator 2/bin/win_x64/eurotrucks2.exe'
pressure-vessel-adverb[188725]:         '-nointro'
pressure-vessel-adverb[188725]:         '-64bit'

but after some retrying it starts.

Tried a few times, nothing.

12328.605:0020:0024:err:steam:run_process Failed to create process L""Z:\mnt\gamehub\native2proton\Euro Truck Simulator 2\bin\win_x64\eurotrucks2.exe" -nointro -64bit": 2

Do other Windows Steam games work with Proton 5.13 + Steam Runtime run command? Do they start via Steam client when Proton 5.13 is used?
About 3-3.5 months ago, my Proton 5.13 and some Linux native Steam games stopped working (they didn't start) but Proton 5.0 was still working. This issue was fixed after updating my Steam client.
I guess that such issues come from a combination of:

Steam client (for Linux) version
Steam Runtime version
Proton version

As of 21st March 2021, I'm using:

Steam client:<PHONE_NUMBER> (Feb 12 2021, Steam API v020)
Steam Runtime:

SteamLinuxRuntime: v0.20210309.0-0-gb38a1fb
pressure-vessel: 0.20210305.0+srt1

scout: 0.20210309.0

soldier: 0.20210309.0

Proton: 5.13-6

Do other Windows Steam games work with Proton 5.13 + Steam Runtime run command? Do they start via Steam client when Proton 5.13 is used?

Yes, although Steam is using wait…run other games are working fine with proton 5.13 including ETS2 launched from steam forced to use Proton.

my Proton 5.13 and some Linux native Steam games stopped working (they didn't start) but Proton 5.0 was still working. This issue was fixed after updating my Steam client.

All games are launching fine from steam, it's just our external launch. I'm on steam beta<PHONE_NUMBER> from the 19.03.21. Runtimes are updated yesterday.

although Steam is using wait…run other games are working fine with proton 5.13 including ETS2 launched from steam forced to use Proton.

Does SteamGameId=227300 SteamAppId=227300 STEAM_COMPAT_DATA_PATH=/path/to/proton_prefix STEAM_COMPAT_CLIENT_INSTALL_PATH=/path/to/Steam/installation /path/to/SteamLinuxRuntime_soldier/_v2-entry-point -- /path/to/Proton\ 5.13/proton waitforexitandrun /path/to/ETS2/bin/win_x64/eurotrucks2.exe work? (--verb=waitforexitandrun option can be omitted because it's the default value.)

Does SteamGameId=227300 SteamAppId=227300 STEAM_COMPAT_DATA_PATH=/path/to/proton_prefix STEAM_COMPAT_CLIENT_INSTALL_PATH=/path/to/Steam/installation /path/to/SteamLinuxRuntime_soldier/_v2-entry-point -- /path/to/Proton\ 5.13/proton waitforexitandrun /path/to/ETS2/bin/win_x64/eurotrucks2.exe work?

No, that's also crashing with the same messages.

12328.605:0020:0024:err:steam:run_process Failed to create process L""Z:\mnt\gamehub\native2proton\Euro Truck Simulator 2\bin\win_x64\eurotrucks2.exe" -nointro -64bit": 2

According to Proton source (steam_helper/steam.cpp), the output says that the returned value from GetLastError() is 2, which means "The system cannot find the file specified.": See https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-
BTW, the run (pressure-vessel-wrap) command has --filesystem option:
$ /path/to/SteamLinuxRuntime_soldier/run -h | grep filesystem
  --filesystem=PATH                     Share filesystem directories with the container. They must currently be given as absolute paths.

(Note: It can be specified multiple times for multiple paths)
I have Wine 6.0 in /opt/wine-devel but the files can not be accessed without --filesystem=/opt/wine-devel option:
$ head -n 1 /opt/wine-devel/share/wine/wine.inf
;; .INF script for the basic Wine configuration

$ LANG=C /path/to/SteamLinuxRuntime_soldier/run -- head -n 1 /opt/wine-devel/share/wine/wine.inf
head: cannot open '/opt/wine-devel/share/wine/wine.inf' for reading: No such file or directory

$ /path/to/SteamLinuxRuntime_soldier/run --filesystem=/opt/wine-devel -- head -n 1 /opt/wine-devel/share/wine/wine.inf
;; .INF script for the basic Wine configuration

So could you try SteamGameId=227300 SteamAppId=227300 STEAM_COMPAT_DATA_PATH=/path/to/proton_prefix STEAM_COMPAT_CLIENT_INSTALL_PATH=/path/to/Steam/installation /path/to/SteamLinuxRuntime_soldier/run --filesystem="/mnt/gamehub/native2proton/Euro Truck Simulator 2" -- /path/to/Proton\ 5.13/proton run "/mnt/gamehub/native2proton/Euro Truck Simulator 2/bin/win_x64/eurotrucks2.exe"? If it fails, try adding --filesystem=/path/to/another/dir option(s).

Yes, that's it.

So with multiplayer I still get 3 out of 5 starts the following error message with nothing happening afterwards. If it got over that point it's working as it should.
WriteProcessMemory with argument "[]" failed with error 5: Access denied.
Also the discord bridge isn't working, I guess because it's started in it's own separated container.

For some reason, it seems that it's not possible to run wine-discord-ipc-bridge and our inject program at the same time when the container is used.

Order
Behavior

1st:winediscordipcbridge.exe 2nd:truckersmp-cli.exe
Multiplayer game doesn't start (WriteProcessMemory() fails)

1st:truckersmp-cli.exe 2nd:winediscordipcbridge.exe
Multiplayer game starts but winediscordipcbridge.exe exits abnormally

When --without-wine-discord-ipc-bridge is specified, game starts even when Steam Runtime is used.
We may need to make Steam Runtime support optional (allow users not to use the container) and add "wine-discord-ipc-bridge is not compatible with Steam Runtime" to known issues.

You're right it works every time without that discord bridge.
Do you know if it's possible to add that process to the container instead of running both parallel?

Unfortunately I'm not sure.

While thinking a bit about this: The bridge socket would probably also be in the container and couldn't get picked up by a external Discord instance. But I haven't verified this. So I think we could safely drop the discord bridge when the container is used as it would work anyway.
The container would also block all possible other stuff like trucksbook. So making it possible to disable the runtime seems necessary.

I tried launching both our inject program and Discord bridge from another (helper) script that was executed by run:
Steam Runtime (run) --> Helper script --+--> Inject program
                                        |
                                        +--> Discord bridge

The result: Even when using Steam Runtime, both programs could be running.
The remaining issue and some notes:

Sharing XDG_RUNTIME_DIR (/run/user/[uid] on my system) issue: When specifying --filesystem option for discord-ipc-[number] socket(s), run fails to start game due to bwrap: Can't create file at /run/user/xxxx/pulse/native: No such device or address or bwrap: Can't create file at /run/user/xxxx/bus: No such device or address

Workaround: When I started Discord (client and bridge) and TruckersMP with XDG_RUNTIME_DIR=/path/to/another/dir (e.g. XDG_RUNTIME_DIR=/home/username/run/) instead of specifying --filesystem=/run/user/[uid], Rich Presence worked even when using Steam Runtime

Users need to start Discord with XDG_RUNTIME_DIR=/path/to/another/dir beforehand
I added --xdg-runtime-dir option that sets the environment variable for Discord bridge and TruckersMP

The runtime dir has some special requirements listed here. Maybe bwrap is trying some invalid stuff there.
I guess when starting discord with a special runtime dir all other integrations won't work unless they're also started with that runtime dir… that doesn't sound convenient but for now it seems it's our best bet on this.
I've also asked in their dev discord some while ago about adding processes into the runtime container but I got no response. Starting them from inside the container seems like a good workaround :)

Starting them from inside the container seems like a good workaround :)

I was considering doing that too. However it seems we can't do that in a simple way :(
If the user is using .deb version of Discord (https://discord.com/api/download?platform=linux&format=deb) we would need to share /usr/share/discord/ but directories in /usr/ can not be shared for some reasons.
Do we need to copy it to a temporary directory? Is it better to install .tar.gz version of Discord (https://discord.com/api/download?platform=linux&format=tar.gz) to $XDG_DATA_HOME/truckersmp-cli/Discord instead (unless --without-wine-discord-ipc-bridge is given) ?

Starting them from inside the container seems like a good workaround :)

I was considering doing that too. However it seems we can't do that in a simple way :(

Maybe I've read wrong but I was thinking you're starting the helper script from inside the container already.

If the user is using .deb version of Discord (https://discord.com/api/download?platform=linux&format=deb) we would need to share /usr/share/discord/ but directories in /usr/ can not be shared for some reasons.
Do we need to copy it to a temporary directory? Is it better to install .tar.gz version of Discord (https://discord.com/api/download?platform=linux&format=tar.gz) to $XDG_DATA_HOME/truckersmp-cli/Discord instead (unless --without-wine-discord-ipc-bridge is given) ?

Honestly, that's too much in every regard. And besides that's another thing we have to care about Discord is probably already actively running on the system if someone wants the rich presence integration which will most likely have some unknown side effects. I tend more towards making without the default and if someone really wants it can activate it and set that runtime directory.

directories in /usr/ can not be shared for some reasons.

BTW, currently the helper script depends on some of our modules and it would not work if truckersmp-cli is installed only in /usr/ (system-wide installation).
To avoid the issue, we need to stop using our modules in the helper script:

setup_logging() in main.py: Stop using logging module and use print() for printing outputs
start_wine_discord_ipc_bridge() in utils.py: Split into 2 parts

Checking and downloading Discord bridge: Call the function in start_with_proton() (before starting the helper script) / start_with_wine()
Starting wine-discord-ipc-bridge: Create subprocess.Popen object in the helper script / start_with_wine()

Regarding splitting start_wine_discord_ipc_bridge(), I'm considering opening another PR as a preparation. The function would be renamed to setup_wine_discord_ipc_bridge().

Seems to work :)
I haven't tested the discord in particular but I guess it'll work.
Instead I've tried with that TrucksBook because that's easy to see as it's spawning a window.
I wasn't getting the connection between the game and Trucksbook but I guess that's not related to this.

There's this option for pressure-vessel:
--pass-fd                             Let the launched process inherit the given fd.

Unix sockets are available as file descriptors, maybe this allows us to redirect the socket without that runtime workaround.
I've tried a bit yesterday but I couldn't make it to pick up the discord socket directly. Pressure-vessel is expecting a single integer but I can't specify the discord process the integer is for.

This would break option compatibility with older versions.

We're still below v1, breakage is to be expected.

--filesystem /full/path/to/discord/socket (not directory path) fixed the sharing issue and I think this PR can be merged soon.
For now I'm not considering removing --without-steam-runtime option because it might be useful for troubleshooting.

Some additional notes about TrucksBook which may be interesting for general executable handling:

I think that our proc_kill at the end breaks the pipe causing the error:esync: up and running.
Setting breakpad minidump AppID = 270880
Steam_SetMinidumpSteamID:  Caching Steam ID:<PHONE_NUMBER>2794497 [API loaded no]
Traceback (most recent call last):
  File "/home/lucki/git_clones/truckersmp-cli/./truckersmp-cli", line 19, in <module>
    main()
  File "/home/lucki/git_clones/truckersmp-cli/truckersmp_cli/main.py", line 176, in main
    start_game()
  File "/home/lucki/git_clones/truckersmp-cli/truckersmp_cli/main.py", line 330, in start_with_proton
    print(line.decode("utf-8"), end="")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 6: invalid start byte

TrucksBook has to be running before starting the game so I had to add a sleep

I think that our proc_kill at the end breaks the pipe causing the error:

Try adding exception handling for UnicodeDecodeError:
                for line in proc.stdout:
                    try:
                        print(line.decode("utf-8"), end="")
                    except UnicodeDecodeError:
                        print("!! NON UNICODE OUTPUT !!", repr(line), sep="  ", end="")

I'm considering adding this change in another PR after merging this PR: start_with_wine() has similar code too (subprocess.check_output() call), and I'd like to change both in the PR.

TrucksBook has to be running before starting the game so I had to add a sleep

The sleep is useless for users who don't use TrucksBook. We would need to add an option (e.g. --sleep-before-start=[seconds]) to steamruntime_helper.py, but I won't add it in this PR because it's currently unused.

I have problems updating proton (5.13 -> 6.3 by setting the default json) (using the default path):
** DEBUG **  Updating Steam Runtime (AppID:1391110)
** INFO **  Running SteamCMD:
  /home/lucki/.local/share/truckersmp-cli/steamcmd/steamcmd.sh
    +login gzlucki
    +force_install_dir /home/lucki/.local/share/truckersmp-cli/SteamRuntime
    +app_update 1391110 validate
    +quit
Redirecting stderr to '/home/lucki/.local/share/Steam/logs/stderr.txt'
[  0%] Suche nach verfügbaren Updates...
[----] Installation wird überprüft...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'gzlucki' to Steam Public ...
Logged in OK
Waiting for user info...OK
src/clientdll/applicationmanager.cpp (7072) : Assertion Failed: eSubFolder != k_EAppDataFolderInstall
src/clientdll/compatmanager.cpp (1738) : Assertion Failed: BFileExists( strSteam2WrapperSrcPath.String() )
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x5) verifying install, progress: 32.23 (73378157 / 227657853)
 Update state (0x5) verifying install, progress: 82.10 (186896925 / 227657853)
Success! App '1391110' fully installed.
** DEBUG **  Updating Proton (AppID:1580130)
** INFO **  Running SteamCMD:
  /home/lucki/.local/share/truckersmp-cli/steamcmd/steamcmd.sh
    +login gzlucki
    +force_install_dir /home/lucki/.local/share/truckersmp-cli/Proton
    +app_update 1580130 validate
    +quit
Redirecting stderr to '/home/lucki/.local/share/Steam/logs/stderr.txt'
[  0%] Suche nach verfügbaren Updates...
[----] Installation wird überprüft...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'gzlucki' to Steam Public ...
Logged in OK
Waiting for user info...OK
src/clientdll/applicationmanager.cpp (7072) : Assertion Failed: eSubFolder != k_EAppDataFolderInstall
src/clientdll/compatmanager.cpp (1738) : Assertion Failed: BFileExists( strSteam2WrapperSrcPath.String() )
ERROR! Failed to install app '1580130' (Invalid install path)
SteamCMD exited abnormally

Manually calling steamcmd worked (with the same strange failing assertion):
/home/lucki/.local/share/truckersmp-cli/steamcmd/steamcmd.sh +login gzlucki +force_install_dir /home/lucki/.local/share/truckersmp-cli/Proton +app_update 1580130 validate +quit
Redirecting stderr to '/home/lucki/.local/share/Steam/logs/stderr.txt'
[  0%] Suche nach verfügbaren Updates...
[----] Installation wird überprüft...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'gzlucki' to Steam Public ...
Logged in OK
Waiting for user info...OK
src/clientdll/applicationmanager.cpp (7072) : Assertion Failed: eSubFolder != k_EAppDataFolderInstall
src/clientdll/compatmanager.cpp (1738) : Assertion Failed: BFileExists( strSteam2WrapperSrcPath.String() )
 Update state (0x5) verifying install, progress: 0.00 (6246 /<PHONE_NUMBER>)
[…]
 Update state (0x5) verifying install, progress: 75.68 (792091134 /<PHONE_NUMBER>)
 Update state (0x61) downloading, progress: 0.01 (127480 /<PHONE_NUMBER>)
[…]
 Update state (0x61) downloading, progress: 99.40<PHONE_NUMBER> /<PHONE_NUMBER>)
Success! App '1580130' fully installed.

After the version was switched it now works again from inside our script:
** DEBUG **  Updating Steam Runtime (AppID:1391110)
** INFO **  Running SteamCMD:
  /home/lucki/.local/share/truckersmp-cli/steamcmd/steamcmd.sh
    +login gzlucki
    +force_install_dir /home/lucki/.local/share/truckersmp-cli/SteamRuntime
    +app_update 1391110 validate
    +quit
Redirecting stderr to '/home/lucki/.local/share/Steam/logs/stderr.txt'
Looks like steam didn't shutdown cleanly, scheduling immediate update check
[  0%] Suche nach verfügbaren Updates...
[----] Installation wird überprüft...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'gzlucki' to Steam Public ...
Logged in OK
Waiting for user info...OK
src/clientdll/applicationmanager.cpp (7072) : Assertion Failed: eSubFolder != k_EAppDataFolderInstall
src/clientdll/compatmanager.cpp (1738) : Assertion Failed: BFileExists( strSteam2WrapperSrcPath.String() )
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x5) verifying install, progress: 95.87 (218261336 / 227657853)
Success! App '1391110' fully installed.
** DEBUG **  Updating Proton (AppID:1580130)
** INFO **  Running SteamCMD:
  /home/lucki/.local/share/truckersmp-cli/steamcmd/steamcmd.sh
    +login gzlucki
    +force_install_dir /home/lucki/.local/share/truckersmp-cli/Proton
    +app_update 1580130 validate
    +quit
Redirecting stderr to '/home/lucki/.local/share/Steam/logs/stderr.txt'
[  0%] Suche nach verfügbaren Updates...
[----] Installation wird überprüft...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'gzlucki' to Steam Public ...
Logged in OK
Waiting for user info...OK
src/clientdll/applicationmanager.cpp (7072) : Assertion Failed: eSubFolder != k_EAppDataFolderInstall
src/clientdll/compatmanager.cpp (1738) : Assertion Failed: BFileExists( strSteam2WrapperSrcPath.String() )
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x5) verifying install, progress: 45.75 (478849032 /<PHONE_NUMBER>)
Success! App '1580130' fully installed.

I can't reproduce this even when the default path is used. I retested after removing $XDG_DATA_HOME/truckersmp-cli/steamcmd/ (= reinstalled SteamCMD) but got the same result.
I didn't change how to use SteamCMD, I only simplified update_game() in this PR.
You might want to retest with master and check whether this PR introduced the issue.
$ cat ~/.local/share/truckersmp-cli/Proton/version
1612904805 proton-5.13-6
$ truckersmp-cli update ats -i 1580130 -v
Running Steam on ubuntu 21.04 64-bit
STEAM_RUNTIME is enabled automatically
Pins up-to-date!
Steam client's requirements are satisfied
WARNING: Using default/fallback debugger launch
/home/username/.steam/ubuntu12_32/steam -nominidumps -nobreakpad -shutdown
** INFO **  Running SteamCMD:
  /home/username/.local/share/truckersmp-cli/steamcmd/steamcmd.sh
    +login account
    +force_install_dir /home/username/.local/share/truckersmp-cli/SteamRuntime
    +app_update 1391110 validate
    +quit
Redirecting stderr to '/home/username/.steam/logs/stderr.txt'
[  0%] 更新を確認中...
[----] インストール状況を確認中...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'account' to Steam Public ...
Logged in OK
Waiting for user info...OK
 Update state (0x5) verifying install, progress: 0.00 (0 / 227657853)
Success! App '1391110' fully installed.
** INFO **  Running SteamCMD:
  /home/username/.local/share/truckersmp-cli/steamcmd/steamcmd.sh
    +login account
    +force_install_dir /home/username/.local/share/truckersmp-cli/Proton
    +app_update 1580130 validate
    +quit
Redirecting stderr to '/home/username/.steam/logs/stderr.txt'
[  0%] 更新を確認中...
[----] インストール状況を確認中...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK.
Logging in user 'account' to Steam Public ...
Logged in OK
Waiting for user info...OK
 Update state (0x5) verifying install, progress: 0.00 (37326 /<PHONE_NUMBER>)
 Update state (0x5) verifying install, progress: 23.76 (248676218 /<PHONE_NUMBER>)
 Update state (0x5) verifying install, progress: 40.04 (419078010 /<PHONE_NUMBER>)
 Update state (0x5) verifying install, progress: 63.08 (660230562 /<PHONE_NUMBER>)
 Update state (0x5) verifying install, progress: 86.83 (908824442 /<PHONE_NUMBER>)
 Update state (0x61) downloading, progress: 5.25 (54939926 /<PHONE_NUMBER>)
 Update state (0x61) downloading, progress: 15.57 (162961750 /<PHONE_NUMBER>)
 Update state (0x61) downloading, progress: 26.53 (277703854 /<PHONE_NUMBER>)
Success! App '1580130' fully installed.
...

Hm, it's also present in master but only when switching versions. Has to be something locally then.

Ok, I think we're good to go. Haven't found anything obvious.

I actually found something: Workshop mods in singleplayer don't work anymore but that's rather low prio because we're mainly focused on multiplayer where mods aren't allowed anyway.

Workshop mods in singleplayer don't work anymore

I can't reproduce this but Steam library directories may need to be shared.
If this happens only when --without-steam-runtime is not specified, try adding your Steam library directories manually:
def start_with_proton():
    # ...
        shared_paths = ["/path/to/steam/library1", "/path/to/steam/library2", Args.gamedir, Args.protondir, Args.prefixdir]

Hm you're right, I had to share one of them which contained the workshop content.
The strange part is that the workshop content is also present in the (--gamedir)/steamapps/workshop/content/appid/ synced from our steamcmd but appearently not used.


Chore.prefixed_queue_names doesn't take --queues or --except-queues into account

Not sure how much of an issue it actually is, but we handle the queue configurations really inconsistently. In some places it's prefixed, in others its not, in some cases it relies on job classes, in others config. We should unify the queue reference handling, as the current setup is difficult to understand.

We've definitely struggled a bit on this, no question.
I had thought the last round of changes we made moved it closer to relying more on the configured job classes, but there is definitely a lot of complexity in here still.
One issue is that while except_queues is a good idea, we simply haven't relied on it much, and thus it hasn't gotten a lot of attention. Truth be told, I had considered pulling the feature out, but figured it was fine as it was for the time being until we got around to a deeper re-architecting of how Chore itself is configured.
If you have any design notes or specific ideas on how you'd like to see it cleaned up, feel free to contribute them to the thread


Differences between Web vs Browser (Cordova)

Discussion about if it is worth to keep both build process (npm run start and npm run start.browser) since the cordova already support a web/browser target!
Should mantain only cordova to keep it simple?
Any pros and cons is welcome!
Things to consider: LiveReload, Different code when using device (hardware) features (even to web).
More info at: https://www.raymondcamden.com/2016/03/22/the-cordova-browser-platform/

One thing is the faster build when npm run start (when no cordova involved).


Deserialize from toml::Value?

I have a toml::Value somewhere and I would like to deserialize it to a structure and use the underlying deserializer, it seems everything is tailored around strings which makes it very difficult (?) to achieve this usecase.
Am I missing something?

Value implements Deserializer and provides a convenience wrapper in the form of Value::try_into.
Is that what you were looking for?

Correct! Thank you!


Shortcuts popover tweaks

Move UA Parser to the webapp so we can choose to show all shortcuts or not.
Slight margin tweak to plus icon

@kyleharper have done this, moved most of the work to the app and simplify the ui component


Docs should advise this.observe("$foo") over this.store.observe("foo") in oncreate handlers

According to sveltejs/svelte#1181, only observers of the former form will be destroyed if the component is destroyed, yet the docs tell people to use the latter form in this section, and don't document the former anywhere. This could direct developers to shoot themselves in the foot and write memory-leaking apps.

Thanks — I went the other way, documenting that the listener should be removed, as listening for $foo has caveats (the component will only receive those changes if $foo is referenced in the template or in a computed property)


Port issue fix for non-windows OS

Executed Build pipeline
https://github.com/OfficeDev/Microsoft-Teams-Samples/actions/runs/6338596369


Bump golang to version 1.21 on release branch

Issue #, if available:
The Govulncheck has been failing on release branch 19 for quite sometime now. We need to bump multiple items on release branch to fix the govulncheck. While the check would still fail until we bump the helm to version 3.14.x, this change should at least mitigate the CVEs associated with go 1.20.x.
Description of changes:
Testing (if applicable):
Documentation added/planned (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Closing as release branch golang version has been bumped to 1.21.


[docs] Add react compatibility in supported platforms

Closes #16829

@pranshuchittora Thanks for taking the time! I have moved the React support section to the bottom of the page, as I believe, it's the least important section of the page. I have also removed any mention to the older version so the content stays simple. As we have explored in the issue, this information would interest people that already have experience with React, they should be able to find the information on their own. Also, I want to incentive our users to be on the latest version.

LGTM :+1:

Thanks


If example contains null the whole resolver breaks and errors out as there is no null check in handleCircularReferences

Q&A (please complete the following information)

OS: [e.g. macOS] Windows
Environment: [e.g. Chrome 59, Node.js v10.0.0] Firefox 104
Method of installation: [e.g. npm, unpkg] npm
OpenAPI Resolver version: [e.g. 3.8.0] latest
OpenAPI version: [e.g. Swagger 2.0, OpenAPI 3.0, OpenAPI 3.1] OpenAPI 3.0

Content & configuration

OpenAPI definition:
# Can't share

OpenAPI Resolver usage:
openApiResolver({
  // Not used
})

Describe the bug you're encountering

To reproduce...
Steps to reproduce the behavior:

Have an example in your OpenAPI docs which contains a null value, in our case it's for the deleted_at column
Try loading it with OpenAPI Explorer, debug the whole day, figure out it's inside this package and find the root cause

Expected behavior

It should just parse
Screenshots

Additional context or thoughts

The error is caused on the following line:
https://github.com/Rhosys/openapi-resolver.js/blob/f16c6e828948f3af44b7956dee7c48bbc5b2202b/src/index.js#L15
For some reason its also parsing the example object, which in our case contains a deleted_at: null - this results in the above line to error out as there is a null being passed in and because Javascript is weird: typeof null === "object"

Can you provide a minimal repo spec, so we can investigate?

The exact error seems to be on the given line: https://github.com/Rhosys/openapi-resolver.js/blob/f16c6e828948f3af44b7956dee7c48bbc5b2202b/src/index.js#L15
As the check above it doesn't check for null itself and null is an object according to JavaScript - I can give a PR - give me a few secs

Got it, i think a !objectWithRef || in the proceeding if statement is likely whe fix

I made the PR - I added it as and extra IF statement so we can return null - for example, in my case it crashed because my example contained deleted_at: null - this I would like to keep as null and not turn into a undefined as that's not what my API will be returning
#<3GithubWorkspaces - made this a really quick edit

Also, you don't need the check for undefined as typeof undefined === "undefined"

That's actually not how undefined works :). undefined means delegate the handling of this back to the caller instead of explicitly handling it.  That's why I made that recommendation since this is more consistent with the expectations of the function handling.
But we can go with this, although it might be confusing with future reviewers.

That's why I switched the PR to the release/4.0 branch. 4.0.30 should already be available.

Does https://github.com/Rhosys/openapi-explorer use 4.0.30 also? I just checked and I am still getting the error

Checking the decompiled code, it doesn't seem to be:

Not unless someone created/merged an update. The easiest way to check is look at the package.json. in general the library isn't pinned to specific versions, but if you are using the the prepackaged version, nothing would he told it to recompile.

The non compiled version now has a flexible package version for this resolver. Hopefully that solves your issue https://github.com/Rhosys/openapi-explorer/pull/80 otherwise, I would appreciate knowing better your need to use the compiled version of the library. Of course if it is really required upgrading the compiled package can be done as well, but I'm assuming this will be sufficient.

Yeah I used the prepackaged version -> node_modules/openapi-explorer/dist/browser/openapi-explorer.min.js (if I remember correctly of the top of my head)

Ah that explains it, they'll need to an update to the explorer to use the new version. I'm actually away from computer this week, so as soon I get back, I can roll the version there (or feel free to make a PR in that repo and I can approve/merge it in)


AB#5710

Proposed changes

Related issues

Checklist

[x] I consider the submitted work as finished
[x] I tested the code for its functionality
[ ] I wrote test cases for the relevant uses case
[ ] I added/update the relevant documentation (either on github or on notion)
[ ] Where necessary I refactored code to improve the overall quality

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

Looks good.  Please resolve the merge conflict and then we can get this merged in


Build plugins in module

Close https://github.com/jhipster/jhipster-lite/issues/2177

MavenCommandHandler clearly need a refactoring but my current ideas are making things worse... depending on this PR lifecycle I'll make a dedicated issue for this refactoring :D

I'm merging this PR first, to avoid other possible conflicts


Fix storpool test & remove powermock

Description
This PR fixes unit tests for plugins/storage/volume/storpool/ and removes powermock.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[x] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

@blueorangutan package

@vishesh92 a [SF] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 6270

@blueorangutan test

@vishesh92 a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Tests are skipped for other storage plugins for some reason. I will enable and push the changes in this PR for them as well.

[SF] Trillian test result (tid-6770)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40479 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7652-t6770-kvm-centos7.zip
Smoke tests completed. 111 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

@slavkap Can you review this?
One minor change in StorPoolUtil.java which checks if /var/log/cloudstack/management/ directory exists or not and depending on that selects the path of the log file. Tests were failing if /var/log/cloudstack/management/ was not present on the filesystem.

Thanks, @vishesh92, the code LGTM

@slavkap Do you need to test this, as StorePoolUtil cahnged?

Merging based on @slavkap 's and others review, build and smoketests.


Not scanning anymore?

Did Niantic already squash it? It doesnt seem to scan anymore

same I just downloaded it

Still works for me

Parasect and hypno just popped up for me so its def still working

Is working fine for me

u have to go tools and refresh or move the Cursor to set a new scanning point.

yea that doesnt do me much good when im out though :/

It would be cool to receive those notifications even if I am not sharing
the link via Ngrok. Just when a Pokemon listed on the notifications list
appear, receive a notification via Pushbullet, Twitter, Telegram or
Whatsapp.
Alejandro José Leiva Valverde
Ingeniero en Producción Industrial
Celular: 8308-9843
2016-08-07 18:34 GMT-06:00 xzeikx<EMAIL_ADDRESS>
yea that doesnt do me much good when im out though :/
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mchristopher/PokemonGo-DesktopMap/issues/722#issuecomment-238119331,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AT7LlfvEA7Mqm5NWYXx_E7v7hS6BRB19ks5qdnmHgaJpZM4JemOh
.


Add an example with CloudMicrophysics.jl coupled to the grid mean

It would be useful to know how the interface looks for a case with cloud microphysics coupled just to the grid mean. I think a good starting point would be to extend one of the existing cases, maybe the bubble?
This will later help us figure out a viable setup for microphysics coupled to either the grid mean or to the sub-grid via TurbulenceConvection.jl.
cc: @jiahe23

This could also help with the design choices on the TurbulenceConvection.jl as mentioned in issue 808


Hello, I would like to ask which type in the file detection/configurations/vssm1/mask_rcnn_vssm_fpn_coco_tiny.py corresponds to in the current vmamba.py file, and I couldn't find this type in FORWARDTYPES

It is "v05_noz" or "v3_noz", I changed the config file of vmamba in classification, but forgot to change the configs in detection and segmentation.


Cannot build with pixi

Hey,
when trying to install we cannot build the libraries, and when concat:libraries gets to pixi, it freaks out about a base64 string it's expecting inside of a path. Windows has a hard time moving around our project folder as well because it seems to think this file really exists and its path is way too long.
Here's the huge text.
Running "concat:libraries" (concat) task
>> Source file "components/springroll-pbskids/dist/modules/google-analytics.js" not found.
Warning: Unable to read "C:\Program Files (x86)\Jenkins\jobs\ourgame\workspace\components\pixi.js\bin\data:application\json;charset:utf-8;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbIm5vZGVfbW9kdWxlcy9icm93c2VyaWZ5L25vZGVfbW9kdWxlcy9icm93c2VyLXBhY2svX3ByZWx1ZGUuanMiLCJub2RlX21vZHVsZXMvYXN5bmMvbGliL2FzeW5jLmpzIiwibm9kZV9tb2R1bGVzL2Jyb3dzZXJpZnkvbm9kZV9tb2R1bGVzL3BhdGgtYnJvd3NlcmlmeS9pbmRleC5qcyIsIm5vZGVfbW9kdWxlcy9icm93c2VyaWZ5L25vZGVfbW9kdWxlcy9wcm9jZXNzL2Jyb3dzZXIuanMiLCJub2RlX21vZHVsZXMvYnJvd3NlcmlmeS9ub2RlX21vZHVsZXMvcHVueWNvZGUvcHVueWNvZGUuanMiLCJub2RlX21vZHVsZXMvYnJvd3NlcmlmeS9ub2RlX21vZHVsZXMvcXVlcnlzdHJpbmctZXMzL2RlY29kZS5qcyIsIm5vZGVfbW9kdWxlcy9icm93c2VyaWZ5L25vZGVfbW9kdW

<omitting like 100k lines of base64 text>

AgfTtcbn1cblxuaWYgKCFnbG9iYWwuY2FuY2VsQW5pbWF0aW9uRnJhbWUpIHtcbiAgICBnbG9iYWwuY2FuY2VsQW5pbWF0aW9uRnJhbWUgPSBmdW5jdGlvbihpZCkge1xuICAgICAgICBjbGVhclRpbWVvdXQoaWQpO1xuICAgIH07XG59XG4iXX0=" file (Error code: ENAMETOOLONG). Use --force to continue.

Looks like the latest version of SpringRoll's Pixi.js fork was built with base64 maps turned on. Workaround in the meantime by changing springroll.json's reference to "pixi.js" and changing to  "pixi.min.js" (minified version doesn't have the source maps).

I reported the issue on pixi.js.

The workaround seems to fine for us to use for now. Thanks for following up on this so quickly! :rainbow:

I've encountered this same issue and found that this is due to a bug in grunt-contrib-concat which is solved by the pull request to grunt-contrib-concat module #130 which was merged and released in version grunt-contrib-concat v0.5.1
The way I've gone around this issue is simply bump the version from 0.5.0 to 0.5.1 in grunt-springroll.

@xoorath - If you're satisfied with either resolution to this issue, please update the status of the bug, and close it. Once you've done that, we can do the same in our internal issue tracking. Thank you!

Closing this issue as a workaround and fix has been provided.


fix: server overrides

Fixes #198

Thanks for the quick update! 🙌
(Also, thank you for building this whole thing. Saved me at least 2 days of wrangling CDK and I owe you at least a coffee for how much it's taught me about writing composable constructs)

@zmillman, thank you!
I'm building on the shoulders of giants: @revmischa, @khuezy


Snakemake installs

Hi,
I love MeSS. It has been exactly what I have needed in the past it worked like a charm. I am trying a fresh install on a new machine (Mac OSX 12.4) and have been running into a recurring issue that I cannot resolve.
Essentially I can follow the conda/mamba install commands but when I run mess to call the snakemake command, I am hit with persistent environment install errors. Specifically, Assembly_finder.yml which cannot install ete3 due to the correct pyqt being unavailable
Could not create conda environment from /Users/btully/miniconda3/envs/mess/lib/python3.8/site-packages/mess/assembly_finder/rules/../envs/Assembly_finder.yml:
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
Could not solve for environment specs
The following packages are incompatible
├─ ete3 is not installable because there are no viable options
│  ├─ ete3 3.1.1 would require
│  │  └─ pyqt >=4.11.4,<4.12.0a0 , which does not exist (perhaps a missing channel);
  └─ ete3 [3.1.1|3.1.2|3.1.3] conflicts with any installable versions previously reported;

and
seqkit.yml
CreateCondaEnvironmentException:
Could not create conda environment from /Users/btully/miniconda3/envs/mess/lib/python3.8/site-packages/mess/scripts/../envs/seqkit.yml:
Could not solve for environment specs
The following package could not be installed
└─ seqkit does not exist (perhaps a typo or a missing channel).

Troublingly, it seems like seqkit may have been removed from bioconda recently:
conda search -c bioconda seqkit                                                                                !10842
Loading channels: done
No match found for: seqkit. Search: *seqkit*
# Name                       Version           Build  Channel
biopet-seattleseqkit             0.1               0  bioconda
biopet-seattleseqkit             0.1      hdfd78af_1  bioconda
biopet-seattleseqkit             0.2               0  bioconda
biopet-seattleseqkit             0.2      hdfd78af_1  bioconda

But similarly, pyqt only seems to be available in conda-forge as >5.15.2
I think this may not be a MeSS issue - but a larger conda package issue. I wanted to bring it to your attention and see if there is an obvious solve I am missing or to open it to others who might a figured it out.
Thanks!

Turns out, all it takes is to make a comment like this to figure out that it was a Mac OSX. Switched to a Linux set-up, no issues.


fix: exit with non-zero if anything fails

if something fails CLI should not exit with 0 (older NodeJS versions do not fail on unhandled rejections autpmatically)
CLI does not need to export anything
CLI integration tests should be done in shell env, not in unit test env
homemade promisify thrown away as well as await of stage steps because because they all are synchronous, and promisify is actually misleading because there should be only one, plus TS will not allow dangling promises in case any of the steps returns a promise
asyncForEach thrown away because for await...of
invoke should inherit stdio to provide seamless output

Hi @qwelias, CC @PatrykMilewski
I've added async flow to let invocations to subscribe on data event in favour of #10, but your proposal is even much better. Thanks a lot for the contribution.
We now have an extra call for build (as a part of travis.yml and test:integration), and a slight drop in coverage. So I'll fix this a bit later.


Non-Validating peer is not able to send a successful DISC_HELLO with security enabled

I have observed some anomalies while attempting to incorporate a single non-validating peer node into a network with four validating peer nodes.
The non-validating peer's validator.enabled flag is set to false and security is enabled (all other security options are turned off).  The consensus module is first set to obcpbft and then noops.  The results are identical for both scenarios using either the obcpbft or noops consensus module.
The first scenario involves 4 validating nodes and 1 non-validating node.  Security is enabled, but all of other security options are disabled, e.g., TLS.  All 4 validating peers are started first.  As soon as I observed that all validating peers could see each other, I started up the non-validating peer.  The following messages were seen in the obc-peer logs for the root node and non-validating peer:
obc-peer log from the non-validating peer:
23:02:31.689 [peer] SendMessage -> DEBU 036 Sending message to stream of type: DISC_HELLO

obc-peer log from the root node:
23:02:31.691 [peer] beforeHello -> DEBU 381 Received DISC_HELLO, parsing out Peer identification 23:02:31.691 [peer] beforeHello -> DEBU 382 Received DISC_HELLO from endpoint=peerEndpoint:<ID:<name:"non-validator01" > address:"<IP_ADDRESS>:30303" type:NON_VALIDATOR pkiID:"|\216\034\336T\033lP\206\201\013(\234\037\014j+\257\023H\331\214\202a\275\363\347\367\304\271\2774\302\244WN\370\311\023\2139D\\\255\243&Z\231" > 23:02:31.691 [peer] handleChat -> ERRO 383 Error handling message: Peer FSM failed while handling message (DISC_HELLO): current state: created, error: transition canceled with error: Error Verifying signature for received HelloMessage: Invalid signature. It is empty.
The second scenario involves 4 validating nodes and 1 non-validating node.  Security is disabled for all nodes, both validating and non-validating.  All 4 validating peers are started first.  As soon as I observed that all validating peers could see each other, I started up the non-validating peer.  When security is disabled, the non-validating peer is able to insert itself into the peer network as seen in the log messages below:
Peers of the root node show that the non-validating peer is part of the network of peers:
curl http://localhost:5000/network/peers {"peers":[{"ID":{"name":"vp1"},"address":"<IP_ADDRESS>:30303","type":1},{"ID":{"name":"vp2"},"address":"<IP_ADDRESS>:30303","type":1},{"ID":{"name":"vp3"},"address":"<IP_ADDRESS>:30303","type":1},{"ID":{"name":"non-validator01"},"address":"<IP_ADDRESS>:30303","type":2}]}
However, when security is disabled and a chaincode is deployed to one of the validating peers, the following errors occur in the obc-peer log of the non-validating peer.  The deployment of the chaincode to all validating peers occurred without error.
obc-peer log for non-validating peer:
 22:46:24.178 [peer] handleChat -> ERRO 451 Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (96) while in state: established 22:46:24.394 [peer] HandleMessage -> DEBU 452 Handling OpenchainMessage of type: CONSENSUS 22:46:24.394 [peer] handleChat -> ERRO 453 Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (96) while in state: established 22:46:24.395 [peer] HandleMessage -> DEBU 454 Handling OpenchainMessage of type: CONSENSUS 22:46:24.395 [peer] handleChat -> ERRO 455 Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (96) while in state: established 22:46:24.396 [peer] HandleMessage -> DEBU 456 Handling OpenchainMessage of type: CONSENSUS 22:46:24.396 [peer] handleChat -> ERRO 457 Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (96) while in state: established 22:46:24.508 [peer] HandleMessage -> DEBU 458 Handling OpenchainMessage of type: CONSENSUS 22:46:24.508 [peer] handleChat -> ERRO 459 Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (13543359) while in state: established 22:46:24.577 [peer] HandleMessage -> DEBU 45a Handling OpenchainMessage of type: CONSENSUS 22:46:24.577 [peer] handleChat -> ERRO 45b Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (94) while in state: established 22:46:24.600 [peer] HandleMessage -> DEBU 45c Handling OpenchainMessage of type: CONSENSUS 22:46:24.600 [peer] handleChat -> ERRO 45d Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (96) while in state: established 22:46:24.600 [peer] HandleMessage -> DEBU 45e Handling OpenchainMessage of type: CONSENSUS 22:46:24.600 [peer] handleChat -> ERRO 45f Error handling message: Peer FSM cannot handle message (CONSENSUS) with payload size (96) while in state: established
Since the non-validating peer is not involved in consensus, it was strange to see the above messages in the non-validating peer's log.  Should we be concenred with these error messages?  Perhaps this is working as designed- I'm not sure, so I wanted to bring it to the community's attention.

Closed by #1220

Hello Harrijk. I'm doing the test of 4 vp and 1 nvp network test on v0.6 but it not work. Would you please share yml for my reference? Many thanks


KFP pipeline run link does not work if the KFP API endpoint ends with "/"

Describe the issue
I believe this is a regression
To Reproduce
Steps to reproduce the behavior:

Create A KFP RTC, specifying a KFP endpoint that ends in "/", e.g. http://cloning99.fyre.ibm.com:12345/pipeline/
Create a notebook
Create a pipeline
add the notebook
run pipeline
click the "Run details" link in the submission confirmation dialog

Screenshots or log output
If applicable, add screenshots or log output to help explain your problem.
Log Output

Paste the log output here.

Expected behavior
A clear and concise description of what you expected to happen.
Should link to
http://cloning99.fyre.ibm.com:12345/pipeline/#/runs/details/ae3e8a95-22e6-4542-81ea-4efe4668269e

instead of
http://cloning99.fyre.ibm.com:12345/pipeline//#/runs/details/ae3e8a95-22e6-4542-81ea-4efe4668269e

Deployment information
Describe what you've deployed and how:

Elyra version: 2.2.2+
Installation source: from source
Deployment type: local installation

I believe this is a regression

Looks like this was only explicitly done for git integration: https://github.com/elyra-ai/elyra/pull/1363

We realized in the sync-up today, that this could be a regression that had previously been fixed in the frontend and re-introduced with the pipeline 2.0 changes.  We'll be sure to address this in the server via this time.
Question: Should any url-valued properties have a trailing slash?  If not, this could probably be handled generically when the metadata is persisted.

Question: Should any url-valued properties have a trailing slash? If not, this could probably be handled generically when the metadata is persisted.

Not sure there is one answer that is correct 100% of the time.


Add 5-second delay before attempting Flatpak updates

Hello everyone,
I've submitted this pull request to resolve https://github.com/gmodena/nix-flatpak/issues/45, where flatpak-managed-install.service attempts Flatpak updates before NetworkManager can fully establish a network connection, leading to the service failing.
I've introduced a brief 5-second delay upon initial login after a reboot to address this. This minor adjustment ensures NetworkManager has adequate time to provide an IP address to our wireless network interface, preventing update failures due to premature connection attempts.
I welcome your insights and feedback on this approach to improve the reliability of the service update process.
If you have any questions, a troubleshooting discussion on the linked issue is available.
Thank!

Hmm no I think about it, if you like this approach I believe we should bump the wait a bit.
If I recall correctly, the folks at https://universal-blue.org wait a few minutes after the user has logged in before triggering the system updates, similarly using a systemd timer—my 2 cents.

Hey @cig0 thanks for this!
"I would prefer not to adopt this workaround yet; at least till I have a better understanding of the failing scenario (still trying to repro).
One drawback I see, is that this change would introduce a 5s busy wait on every boot. This would introduce delay on systems that don't have the same networkd / nix-flatpak failure you reported.
Might we end up adopting this workaround, I would like the timeout to be user configurable via a services.flatpak option. Also, just FYI, the home-manager unit would need to be adjusted too.

Hey @cig0 thanks for this!
"I would prefer not to adopt this workaround yet; at least till I have a better understanding of the failing scenario (still trying to repro).
One drawback I see, is that this change would introduce a 5s busy wait on every boot. This would introduce delay on systems that don't have the same networkd / nix-flatpak failure you reported.
Might we end up adopting this workaround, I would like the timeout to be user configurable via a services.flatpak option. Also, just FYI, the home-manager unit would need to be adjusted too.


Fix base href resolving

Created another issue and fixed this here. Your solution only handled the HapHyperlinkParser. I made a similar change to the base class so all impls could benefit from it. Also added some unit tests to prove it. Thanks for pointing this out.


Update to work with re-factored global indicators workflow, and support 1000 cities challenge

The global_indicators repo had some large re-workings recently to allow the methods used for our 25-city study to more easily be generalised for new cities (eg for the 1000 cities challenge).   To work with the new methods, including removal of between-city walkability comparison (as per https://github.com/global-healthy-liveable-cities/global-indicators/pull/143) and streamlined project configuration (https://github.com/global-healthy-liveable-cities/global-indicators/issues/144) the global_scorecards methods also need updating.

Just digging into some of the details of this, its a bit complicated.   The previous approach was done for a processed set of cities.   Now, we want it to be done on demand for a single city, which is quite a different paradigm.  A lot of code will need to changed to reflect this different approach.
The functions generate resources and generate scorecards could be run as part of a process workflow…
References (eg comparison values) should be self-contained --- not to other cities, except perhaps as fixed-for-now placeholders related to reference values from 25-city study.  Previously we loaded up the data and empirically determined comparisons (e.g. 25th, 50th and 75th percentiles for percentage of population with access to amenities).
We can define details about the set of policies in a policy.yml file (in process/configuration/templates), and presumably for a single city there would be some policy rating CSV file which would then be parsed to derive presence and quality scores and checklist ratings.
Its necessary to re-approach, simplify and then generalise the approach we took for the 25 cities -- Its do-able, but its a bit hard go-ing.  I'm leaving this comment as a placeholder to feel some sense of achievement once this has been overcome!

The global_scorecards process has been updated, and now works for a single configured and processed city in the context of the current updated 'global indicators' code (at time of writing, on 'enhancements' branch: https://github.com/global-healthy-liveable-cities/global-indicators/commit/ddb6369b4f89e2b4dad27b09e355a62db528bc24),
The process now outputs folders for reports and figures in the study region's own data folder, e.g.

and

This is different to the previous working code for this repository (tagged 0.1), which was designed specifically for a 25-city study (creating outputs at https://doi.org/10.25439/rmt.c.6012649).   The idea is, that now creating a report will be part of the regular workflow for each city.
As such, I envisage that this repository will be archived and the main scripts incorporated as an additional step into the global indicators 'process' workflow.
But for now, this issue has been resolved and will be closed.


Fix decoding to use websafe base64

Since the base64 encoding we're getting back is URL safe, this uses Google's classes for decoding the base64 encoded data.
This PR also removes sdk-integration-specs.

@diogeneshamilton @dasmer any news on merging this? I'd like to be using a more up to date version of the library and I need this fix.

@dasmer @diogeneshamilton any update on this? Still using a custom fork of the library because I need this and #98.


Integrating Passport.js

I've figured out a way to integrate Passport.js as an inline node; I'll post it to my node-red-nodes fork when it's done, just wondering if this is something that anyone else has tried or has an interest in seeing work a particular way.

This was a long time ago :) I couldn't find it in your node-red-nodes fork. Really looking for a way to use passport as part of a flow rather than using it as part of express middleware.


dc_tasks: set current, function, output

Here are a few useful tasks for DC sources. It is tedious to change the function mode, but that is the desired result to make the user is aware of the consequences: the source will suddenly turn off.
A question I had was: should the protection from changing the output and function be at the level of the driver or at the level of the task. As of now it is at the level of the driver, which means that you can never change output with this driver without setting the source to 0. The good thing is that future tasks will have to comply by this. The bad thing is that future drivers may not implement these conditions... I am open to comments.

I missed it but you also broke a test...

I will merge as is but it may be a good idea (and exercise) to add tests in the future for the new tasks.


ADD in Dockerfile leads to 'unexpected EOF' in Vagrant under Win

I have a serious error that occurs not in every execution: Inside a vagrant-Ubuntu-machine deployed under Windows 7 I build a Dockerfile containing an ADD url statement. Sometimes the download starts slowly, becomes even slower and finally fails with following output: ERROR: unexpected EOF.
Here is the related vagrant file...
Vagrant.configure("2") do | config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.provision :shell, path: "bootstrap.sh"
  config.vm.provider "virtualbox" do |vb|
    vb.customize ["modifyvm", :id, "--memory", "2048"]
  end
end

... here the bootstrap.sh for the vagrant file...
#!/usr/bin/env bash
if [ ! -e /usr/lib/apt/methods/https ]; then
	apt-get update
	apt-get install -y apt-transport-https
fi
echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
apt-get update
apt-get install -y lxc-docker

... and finally the Dockerfile I tried to build inside the VM. The Dockerfile is pruned after the ADD line, to show just the relevant part:
FROM java:latest
MAINTAINER Thomas Steinbach

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y unzip
ADD http://downloads.sourceforge.net/project/lportal/Liferay%20Portal/6.2.1%20GA2/liferay-portal-tomcat-6.2-ce-ga2-20140319114139101.zip /tmp/

However, sometimes it works, some times not. To reproduce, install Vagrant under Windows, put the first two codeblocks in the appropriate Files (Vagrantfile and bootstrap.sh) and run following command:
vagrant up

SSH to the VM, creat the Dockerfile with the content shown above and build it. Maybe you have to retry those steps several times, till the error occurs.

i cannot reproduce with the latest docker version, if you try and can, please give us the steps to reproduce


Can not login to SINGLE_USER_MODE enabled instances.

Some instances use SINGLE_USER_MODE option that redirects the top page to user profile page.
It would be nice if it opens up '/auth/sign_in' path directly.

Thank you for reporting this! I actually didn't know the single user mode. I'll try to change a page shown before login.

This issue should be addressed in 9c3f0fe. Please wait for next release.

I'll close this issue because it was addressed in v0.2.2. Please try the latest. If you find there is still a bug, please let me know by commenting this thread or making a new issue.


Surface errors sooner if file is not summarized for long time

ODSP (PUSH v1) used to have a limit of 5/10K un-summarized ops. When client reached this limit, it would nack main client and only allow summarizer to proceed.
This was a good signal that something goes wrong, even though we run into many bugs in runtime where we were not able to summarize in that state.
With PUSH v2, that logic is no longer possible (or rather easy to implement).
And in general, we want client to be more in control here, as client knows what is expected and what is not (i.e. documents may have vastly different thresholds and expectations from games).
But we can't allow infinite number of ops for most of our scenarios. An example of that latest Chapter 2 document that took over 5 minutes to catch up.
I think what we need to do is

Have one threshold at which every client would generate telemetry error. I.e. if there are over 5K unsummarized ops, client produced telemetry error event.
Hight threshold (likely 2x of above) there we disallow main client to send any ops, and force summary, likely by any client (though maybe that's not right). Note that this might be a gap stop solutions for some other problems, like stale client in quorum.

Thresholds are best when controlled by isntantiateRuntime, i.e. particular implementation, but we should have reasonable defaults.
One thing to be careful - client may load from stale (cached) summary which might be behind. I think the logic for # 1 should be that client resets it's counter every time it sees a summaryAck, and once we have 5K ops over it without any summary it logs telemetry error (and maybe resets counter, to keep generating error every 5K ops, logging total number of ops without summary from start - would be very useful metric). And once it reaches the tip of the file (likely - runtime switched to "connected write" mode, though we can improve here), with over 10K un-summarized ops, it does not send any ops by starts summarizer and waits
for successful summary.
This feels rather important for our ability to "see" bad cases (the first part - issuing logging errors). So it might makes sense to fork this issue and do some things sooner than others?

I believe we mostly have the bits and pieces implemented, including Summarizer:Behind event and PUSH blocking clients at 10K ops. So closing


Support some math symbols needed in algebra

Description
Hello, I am using typst to write some formulas related to algebraic logic, and find that I can't enter the symbol $\not \models$.
There is a related symbol called models $\models$, but putting not directly in front of it doesn't output the symbol I want.
If such a symbol were to be added, I think it could be named models.not .

And another thing, in typst, I can enter AA BB CC DD to input $\mathbb{ABCD}$ (mathbb in latex), whitch is really useful in algebraic logic.
But there doesn't  seem to be a way to input $\mathcal{ABCD}$ (mathcal in latex), whitch is also useful in algebraic logic.
Use Case
The symbol and font are useful in algebraic logic formulas.

Regarding the first request: cancel(models)  (you might need to add some space around it for it to look good for now, while #1023 isn't merged)
Regarding the second: you're looking for $ cal(A) cal(B) cal(C) cal(D) $

Thank you for your reply.

You can always input unicode characters with an escape sequence, even if they don't have a name
In this case, use \u{22ad}, provided the font you're using has it (Stix Two Math does at least)
$\u{22ad}$


🐞 Directory upload is slow

What is the issue?
client.container().from().withDirectory("/src", client.host().directory("."), { exclude: [ <lots, like node_modules> ] }.withWorkdir("/src").withExec(["/bin/bash", "-ce", "du -sch ; tree -a"]).stdout()
After adding tons of directories to the exclusion list, this command results in:

inside the container: 1.7 M total disk usage, 78 directories, 178 files
two upload steps: one that takes 58.53s and one that takes 14.46s, in serial

Running the commands directly on the host (which has run pnpm install etc.): 3.0G total disk usage, 20728 directories, 72993 files
In my opinion, this is far too long to wait for every run, since changes to the host source files will almost never be cached.
Log output
No response
Steps to reproduce
Unsure how to reproduce.
Original repository is a small Typescript/React/Node monorepo (a side project of mine) with 9 packages.
SDK version
Node v0.8.8
OS version
macOS 14.0 on M1/Apple Silicon

@ari-becker Did you cached the installation process?
I tried to reproduce this with our own NodeSDK, it seems a bit bigger than your typescript repository but it tooks only 10 seconds to upload.
Here's the code I made
package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	ctx := context.Background()

	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		panic(err)
	}

	source := client.Host().Directory("./sdk/nodejs")

	out, err := client.
		Container().
		From("alpine").
		WithDirectory("/src", source, dagger.ContainerWithDirectoryOpts{Exclude: []string{"node_modules"}}).
		WithWorkdir("/src").
		WithExec([]string{"du", "-sch"}).
		WithExec([]string{"tree", "-a"}).
		Stdout(ctx)
	if err != nil {
		panic(err)
	}

	fmt.Println(out)
}

The step that upload the folder takes only 10 seconds (constantely)
▽ │ host.directory ./sdk/nodejs
█ │ [10.12s] upload ./sdk/nodejs
┣ │ [9.92s] transferring eyJvd25lcl9jbGllbnRfaWQiOiJtN2hybWQxamQxenZkdHhmZ3JodjdsN2JnIiwicGF0aCI6InNkay9ub2RlanMiLCJpbmNsdWRlX3BhdHRlcm5zIjpudWxsLCJleGNsdWRlX3BhdHRlcm5zIjpudWxsLCJmb2xsb3dfcGF0aHMiOm51bGwsInJlYWRfc2luZ2xlX2ZpbGVfb25seSI6ZmFsc2UsIm1heF9maWxlX3NpemUiOjB9:

I'll also check if it doesn't come from the node version but it shouldn't be the case since upload operation are done on the engine side (which is independant from the SDK used)
If possible, could you share your repository so I can try on it?

Same test with the nodeSDK and it's still really fast: less than 7seconds to upload.
import { connect } from<EMAIL_ADDRESS>
connect(
	async (client) => {
		const source = client.host().directory('./sdk/nodejs', { exclude: [ 'node_modules' ] });

		const out = await client
			.container()
			.from('alpine')
			.withDirectory('/src', source)
			.withWorkdir("/src")
			.withExec(["du", "-sch"])
			.withExec(["tree", "-a"])
			.stdout()

		console.log(out)

	},
	{ LogOutput: process.stderr });

█ │ [6.44s] upload ./sdk/nodejs
┣ │ [6.39s] transferring eyJvd25lcl9jbGllbnRfaWQiOiJsaWw3cWFhNGUzNzUyMDA2N3R5dzUwMXo2IiwicGF0aCI6InNkay9ub2RlanMiLCJpbmNsdWRlX3BhdHRlcm5zIjpudWxsLCJleGNsdWRlX3BhdHRlcm5zIjpbIm5vZGVfbW9kdWxlcyJdLCJmb2xsb3dfcGF0aHMiOm51bGwsInJlYWRfc2luZ2xlX2ZpbGVfb25seSI6ZmFsc2UsIm1heF9maWxlX3NpemUiOjB9:

Same test with the nodeSDK and it's still really fast: less than 7seconds to upload.
import { connect } from<EMAIL_ADDRESS>
connect(
	async (client) => {
		const source = client.host().directory('./sdk/nodejs', { exclude: [ 'node_modules' ] });

		const out = await client
			.container()
			.from('alpine')
			.withDirectory('/src', source)
			.withWorkdir("/src")
			.withExec(["du", "-sch"])
			.withExec(["tree", "-a"])
			.stdout()

		console.log(out)

	},
	{ LogOutput: process.stderr });

█ │ [6.44s] upload ./sdk/nodejs
┣ │ [6.39s] transferring eyJvd25lcl9jbGllbnRfaWQiOiJsaWw3cWFhNGUzNzUyMDA2N3R5dzUwMXo2IiwicGF0aCI6InNkay9ub2RlanMiLCJpbmNsdWRlX3BhdHRlcm5zIjpudWxsLCJleGNsdWRlX3BhdHRlcm5zIjpbIm5vZGVfbW9kdWxlcyJdLCJmb2xsb3dfcGF0aHMiOm51bGwsInJlYWRfc2luZ2xlX2ZpbGVfb25seSI6ZmFsc2UsIm1heF9maWxlX3NpemUiOjB9:

@TomChv I can confirm that this runs even quicker on my machine, for the dagger repo. With that said, sdk/nodejs has a du -sch of only 116M; my argument is that (a) setting exclusions should reduce upload times since those directories do not need to be uploaded, (b) in a large (multi-GB) repository with many exclusions, the time savings from those exclusions are not being realized.
I'll try to repro later on one of the larger public monorepos: https://github.com/korfuri/awesome-monorepo#notable-public-monorepos

Maybe there's two sections of code, one that transfers the host directory to buildkit, and one that actually writes files into the container, and the exclude list is improperly handled in the first section but is properly handled in the second section?

I think it's because your exclude is made after copying the whole directory? Can you try again but instead of doing:
client
  .container()
  .from()
  .withDirectory("/src", client.host().directory("."), { exclude: [ <lots, like node_modules> ]})
  ...

You try to do
client
  .container()
  .from()
  .withDirectory("/src", client.host().directory(".", , { exclude: [ <lots, like node_modules> ]}))
  ...

I think you should see a difference in your output

oh wow, yup that did it, now it's done within 170 ms, much better.
@TomChv IMO this is a big footgun in the SDK, it's much too easy to add exclude to container.withDirectory instead of host.directory; TypeScript will accept either usage as valid, and Dagger will silently accept either usage, but with large consequences in large host directories.
Up to you if you think this issue should be closed (because I fixed my problem) or kept open (i.e. as an issue to have an API with less of a footgun?).

also, the quickstart unintentionally directs users to use this footgun, which is maybe how I shot myself in the foot in the first place 😅

@TomChv IMO this is a big footgun in the SDK, it's much too easy to add exclude to container.withDirectory instead of host.directory; TypeScript will accept either usage as valid, and Dagger will silently accept either usage, but with large consequences in large host directories.

It actually depends of your use case, if you work from your host filesystem, I totally agree that it can be pretty confusing if you should use exclude on host.Directory() or withDirectory(). However, if you pull a repository or copy a directory from another image's container, then it makes sense to use withDirectory(). This is why we actually support both exclusion.
We will work on the documentation to make that clearer.

Up to you if you think this issue should be closed (because I fixed my problem) or kept open (i.e. as an issue to have an API with less of a footgun?).
also, the quickstart unintentionally directs users to use this footgun, which is maybe how I shot myself in the foot in the first place 😅

https://github.com/dagger/dagger/pull/5854 is opened to fix that issue. :rocket:
I'm closing this issue, feel free to reopen it if you have another issue with directory upload being slow.


Python: Separate set and setitem strings

Motivation and Context
Separate set and setitem doc strings

Description
Separate set and setitem doc strings

Contribution Checklist

[X] The code builds clean without any errors or warnings
[X] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
[X] All unit tests pass, and I have added new tests where possible
[X] I didn't break anyone :smile:

Python 3.10 Test Coverage Report • FileStmtsMissCoverMissingsemantic_kernel/functions   kernel_plugin.py187299%474, 477TOTAL6232103083%
Python 3.10 Unit Test Overview

Tests
Skipped
Failures
Errors
Time

1341
1 :zzz:
0 :x:
0 :fire:
13.087s :stopwatch:


[FEATURE] : Design new Schemas for Cart and Order

Is your feature request related to a problem? Please describe.
As this is an e-commerce application and has items to be sold so users can buy them, database schema designs like Order and Cart are required in the backend.
Describe the solution you'd like
Design a new mongoose schema for Order and Cart.
Describe alternatives you've considered
This is the solution I have planned to implement; other alternative solutions would not fit the scenario.
Additional context
I will be designing the schemas for Order and Cart according to the requirements of the other schemas.

Is this issue still active could you assign this issue to me?

@chirantanbanik , is this issue still open could you assign it to me

@shivansh84ya You can start working on this issue.

Hi @chirantanbanik
Thanks, Will raise PR once ready

Hi @chirantanbanik ,
I've successfully raised a pull request for the new Cart and Order schemas (issue #119). Please review the changes and let me know if any adjustments are needed.
Thanks!


add process env for max buffer

not sure what branch to push
#854

Thanks @pelim  , Can you use the develop branch as the base. We will update the build instructions

Codecov Report

Merging #857 (aa3ccf9) into develop (96f3421) will decrease coverage by 0.81%.
The diff coverage is 78.94%.

@@             Coverage Diff             @@
##           develop     #857      +/-   ##
===========================================
- Coverage    66.22%   65.41%   -0.82%
===========================================
  Files           46       47       +1
  Lines         1593     1573      -20
  Branches       342      342
===========================================
- Hits          1055     1029      -26
- Misses         538      544       +6

Impacted Files
Coverage Δ

packages/core/src/utils/defaultProcessOptions.ts
33.33% <33.33%> (ø)

...core/src/command/commandExecutor/ExecuteCommand.ts
12.90% <100.00%> (+2.90%)
:arrow_up:

packages/core/src/package/SFPPackage.ts
85.13% <100.00%> (+0.20%)
:arrow_up:

...es/core/src/package/packageQuery/PackageFetcher.ts
100.00% <100.00%> (ø)

packages/core/src/command/SFDXCommand.ts
52.38% <0.00%> (-42.86%)
:arrow_down:

packages/core/src/queryHelper/QueryHelper.ts
100.00% <0.00%> (+11.11%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3cebd8...aa3ccf9. Read the comment docs.

will close this PR and open a new one - the git rebase from release -> develop did not work expected  :-)


support windows \r encoding

support windows benchmarking

Motivation
(Write your motivation for proposed changes here.)
Have you read the Contributing Guidelines on pull requests?
(Write your answer here.)
Test Plan
(Share your test plan here. If you changed code, please provide us with clear instructions for verifying that your changes work.)
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/diem/diem/tree/main/developers.diem.com, and link to your PR here.)
If targeting a release branch, please fill the below out as well

Justification and breaking nature (who does it affect? validators, full nodes, tooling, operators, AOS, etc.)
Comprehensive test results that demonstrate the fix working and not breaking existing workflows.
Why we must have it for V1 launch.
What workarounds and alternative we have if we do not push the PR.

@wrwg transfered

After discussing it, we will look at a PR for unicode characters in comments, but will not be accepting anything at this time that allows \r, sorry

Just curious: if only the specific Windows sequence \r\n would be accepted, but not a standalone \r, wouldn't that solve the security problem? It appears pretty restrictive to not allow Move on native Windows.

T

Just curious: if only the specific Windows sequence \r\n would be accepted, but not a standalone \r, wouldn't that solve the security problem? It appears pretty restrictive to not allow Move on native Windows.

That solution sounds reasonable to me, but I don't have the right context on the parser to understand how easy it is to do this.
@LemonHX : would you mind de-scoping this PR to the parser change only (i.e., move the benchmarking code to a separate PR?)

@sblackshear I will do that

@wrwg so we could add to our documentation that if you are windows user please config your git before generating any move file

@wrwg \r is used in native mac...

@wrwg \r is used in native mac...

... and we will not allow it if it is used. We only allow regular ascii files, where lines can be separated either with \n (mac) or \r\n (windows). I think this is very easy to add to the lexer.

Superseded by #90


Fix package name in installation instructions

Installing the coffee npm package is a common mistake when trying to install the Coffeescript coffee REPL. The "coffee" package is an unrelated test library.
coffeescript is the package to install if you want the coffee REPL.

@kytrinyx Rights please, I will take care.

@ferhatelmas Thanks! Done.

@kytrinyx Thanks :smile:

@daveyarwood Thanks a lot for fixing this long standing problem.


When I was calculating a 15MW wind turbine, I wanted to explore the effect of ocean currents on the turbine, but the test results were less than satisfactory

In the case of the 15MW floating wind turbine, the platform response does not seem to be obvious after I introduce the sea current compared with the environment with no wind and no waves. Under the rated wind speed of 10.59m/s condition, I set the wave height of 4.9m, and after the introduction of the sea current, despite the obvious increase in the surge, the calculation result is almost exactly the same regardless of the setting of the speed of the sea current, and regardless of the angle of the sea current, may I ask if this is a normal situation?
In addition, I didn't understand the three modes of currents in HydroDyn in the user's manual. Are these three modes independent of each other, and should I choose one of them to set the velocity and direction of the currents in the calculation, or are all three current models used in the FAST calculation at the same time?

Dear @lin990316,
Are you using the IEA Wind 15-MW reference wind turbine model atop the UMaine semisumbersible from the following repository: https://github.com/IEAWindTask37/IEA-15-240-RWT/tree/master/OpenFAST/IEA-15-240-RWT-UMaineSemi?
The sea current functionality of HydroDyn works in conjunction with the viscous drag calculation of the strip-theory members.  However, the aforementioned OpenFAST model does not have any strip-theory members; rather, viscous effects are captured through a simplistic approach, lumping the contribution of viscous effects into an AddBQuad matrix.  Unfortunately with this approach of capturing viscous effects, enabling sea current in HydroDyn will not influence the floater response (which is not correct).  In general, it is better to capture viscous effects through strip-theory members.
I'm not sure I fully understand your second question, but the current modeling options of HydroDyn are documented here: https://openfast.readthedocs.io/en/main/source/user/hydrodyn/input_files.html#current.
Best regards,

Dear @jjonkman
Thank you Dr for your answer, I am using exactly the model you mentioned,
“ it is better to capture viscous effects through strip-theory members. ”
Regarding the method you are talking about I don't understand it very well, do I need to implement it by means of modifying the source code, i.e. I can't study the effect of currents on the response of the turbine if I just use the existing FAST software and model?
thanks again
Best regards,

Dear @lin990316,
A source code change is not needed to fix this.
Instead, the HydroDyn input file for this OpenFAST model should be changed so that viscous effects are handled through the strip-theory solution rather than through AddBQuad.  My understanding is that such an OpenFAST model already exists for this reference system, but I'm not sure if that model is hosted publicly.  This is probably a better question to ask in the IEA Wind 15-MW reference wind turbine repository: https://github.com/IEAWindTask37/IEA-15-240-RWT/issues.
Best regards,


Added initial support for Obsidian daily notes to Tana Calendar notes

Added additional parameter to specify the daily note pattern used for daily notes. It uses the same format as Obsidian using moment.
The translation is based on the assumption that calendar notes are of type date and the uid is the same as the name (a date) in MM-DD-YYYY format. Like the followiing:
{
      "uid": "09-07-2022",
      "name": "09-07-2022",
      "children": [
        {
          "uid": "ZWWrfuNpM",
          "name": "Today I will write about [[59QkLn2qT]]",
          "children": [],
          "createdAt":<PHONE_NUMBER>148,
          "editedAt":<PHONE_NUMBER>648,
          "refs": [
            "59QkLn2qT"
          ],
          "type": "node"
        }
      ],
      "createdAt":<PHONE_NUMBER>798,
      "editedAt":<PHONE_NUMBER>798,
      "refs": [],
      "type": "date"
    },

Added also new tests to verify new code. fileConverter.test fails with an error also present in the existing test. I had no time to figure it out:
  - Expected  - 0
    + Received  + 6

    @@ -94,6 +94,12 @@
          "fields": 0,
          "leafNodes": 9,
          "topLevelNodes": 1,
          "totalNodes": 10,
        },
    +   Array [
    +     "heading",
    +     "Some",
    +     "Link",
    +     "Link2",
    +   ],
      ]

I have limited knowledge of TS so feel free to point all the obvious stuff that I may have made.
Thanks!

hey, very cool that you did this! Totally forgot about Daily Page conersion until now.
However, unfortunatley I shuffled quite a lot of stuff around to enable folder-structure based conversion and some other features.
On the positive side, I think most of your changes should be transferable to the new structure because the central functions are mostly similar, so its best if you re-fork and add your changes in the new structure. If you want, I can add them too - but then you wouldnt show up as contributor.

Hi @LuccaHellriegel, no Issues, I started solving conflicts with the first commits you did but I were not able to get the time to finish them before you pushed the next set of changes 😅 . No Issues at all I was planning to adapt to your changes since you know the core structure of the converter much more than me.
As said the first set of conflicts are resolved already I will try to resolve your new changes today.
As per the date format, I think its a great idea to read the config directly from obsidian! However I think is still valuable to have an optional parameter, since I know people that have an obsidian vault but no obsidian installed. What do you think?

Ah, also now the daily note pattern is taken from obsidian config files, not taken by cli argument

Hi @vicrdguez , thanks for the great work getting the merge conflicts fixed and reading from the obsidian folder.
Just three more little things:

could you remove the fs-call from the vault context? The filesystem calls will be replaced with an adapter sooner or later which supports local + browser environements and I think it does not need to be in the constructor, for simplicities sake you could just set the public dailyNoteFormat prop after the call
please check the vault converter test, I am sure with your changes it will fail, so the full.tif.json needs to be adapted
I also see some unformatted code, could you re-format everything with prettier please?


Changed below to above in docs/api/README.md

Not quite sure that this is correct but it do make more sense to me.
Changed below to above in Every function described below is a top-level export. You can import any of them like this:

:+1:


Changing Model Param after Initialization VLLM Model

Feature request
Introduce a more flexible method to specify model parameters directly within the LLMChain or VLLM method, allowing users to modify these parameters without creating a new model instance each time.
Motivation
Currently, when utilizing the Langchain library's VLLM class, modifying model parameters (such as temperature, top_k, top_p, etc.) requires creating a new instance of the VLLM class. This process becomes cumbersome, especially when frequent adjustments to model parameters are necessary.
Current Situation:
`
from langchain.llms import VLLM
llm = VLLM(
model="mosaicml/mpt-7b",
trust_remote_code=True,  # mandatory for hf models
max_new_tokens=128,
top_k=10,
top_p=0.95,
temperature=0.8,
)
template = """Question: {question}
Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "Who was the US president in the year the first Pokemon game was released?"
print(llm_chain.run(question))
`
It is very tedious to keep creating new model when i want to change model params like (temperature and etc) .
In naive VLLM, this can be done so easily:
llm = LLM(model="qwen/Qwen-7B-Chat", revision="v1.1.8", trust_remote_code=True)
outputs = llm.generate(prompts, model_params)
outputs = llm.generate(prompts, model_params)
Your contribution
NA

Other LLM integrations may differ, but from my experience using Ollama integration, you can pass parameters to the generate function using bind.
For example:
model = Ollama(model='mixtral')
chain1 = prompt1 | model.bind(model='llama2') | StrOutputParser()

So instead of the chain using the 'mixtral' model, it will use 'llama2' model. This is because bind will pass any parameters to the generate function via kwargs parameter. It will then use that to format the post request payload:
https://github.com/langchain-ai/langchain/blob/v0.1.0/libs/community/langchain_community/llms/ollama.py#L176
Can you give more information on which LLM integration you are working with?


Consolidate decimal cast code into DecimalConversions

I wonder if those casts in SPI shouldn't have tests in SPI, but I guess coverage in presto-main should be enough..


Investigate parsing into user structures directly

E.g.
std::vector<X> v;
std::string_view json = /*...*/;

boost::json::parse_into( v, json );

where X is something that Boost.Json recognizes, such as a struct with value_to defined, or (given #626) a described struct.
As an example of why this is useful, on one popular benchmark
https://github.com/kostya/benchmarks#json
https://github.com/kostya/benchmarks/blob/master/json/test_boost_json.cpp
having the ability to parse directly into either std::vector<coordinate> or a custom container that accumulates in its push_back member function instead of storing the elements could potentially gain a lot of performance.

I made a prototype, https://gist.github.com/pdimov/bcdaa4fef606fdcd0f21195a79bf8648
Unfortunately it's slower. But still cooler. :-)
1.json: 74181420 bytes
boost::json::parse: 456 ms
  x: -5.00063e-30, y: 5.00037e+30, z: 0.500451: 115 ms
parse_into: 702 ms
  x: -5.00063e-30, y: 5.00037e+30, z: 0.500451: 2 ms

The manual loop, parsing via the JSON DOM is not performing error-checking on the types in the DOM. It'll exhibit UB if the JSON is valid but not correctly representing the co-ordinate array, so I think it's getting a better score than it should.

It's copied from the benchmark code.
This needs to be refactored to contain the subhandlers inline in the composite handlers, rather than allocating them with new each time. ("Structured concurrency" :-) ) (Also, needs a proper error enum. And a handler for tuples. And perform range checking on the numbers. I ran out of night.)

Also needs a handler for json::value, so that json files that have random unstructured parts can still fit into a structure. I had to remove the opts part of the original json generated by https://github.com/kostya/benchmarks/blob/ef8dbdec90bbcb3b242da8211ff7db5396e23979/json/generate_json.rb because it's too chaotic.

This needs to be refactored to contain the subhandlers inline in the composite handlers, rather than allocating them with new each time. ("Structured concurrency" :-) ) (Also, needs a proper error enum. And a handler for tuples. And perform range checking on the numbers. I ran out of night.)

https://gist.github.com/pdimov/c74627565d35cdff97d7f4d0ae7a7b00
This one is faster.
1.json: 74181420 bytes
boost::json::parse: 431 ms
  x: -5.00063e-30, y: 5.00037e+30, z: 0.500451: 131 ms
parse_into coordinates: 234 ms
  x: -5.00063e-30, y: 5.00037e+30, z: 0.500451: 2 ms
parse_into coordinates2: 182 ms
  x: 0.500451, y: 5.00037e+30, z: 0.500451: 0 ms

It needs its own suite of parsing functions, e.g.
parse_object_into
parse_array_into
parse_string_into
parse_number_into
parse_bool_into
parse_null_into

Using pull-parsing with this signature
template< typename V >
bool
parse_string_into(
  char const*& begin,
  char const* end,
  V& v,
  error_code& ec );

template<class V> using parser_for = basic_parser<detail::handler<V>>;

there should probably be is_bool trait for consistency.


DQL using INSTANCE OF with parameters return no results

If I defined the following DQL and it returns the expected results.
$dql = <<<DQL
SELECT o FROM \MyProj\AbstractMyObject AS o
WHERE o INSTANCE OF \MyProj\AlphaObject
DQL;
$q = $this->getEntityManager()->createQuery($dql);
$q->getResult(); // 1 row returns

However, when I supply the instance name as a parameter I get no results.
$dql = <<<DQL
SELECT o FROM \MyProj\AbstractMyObject AS o
WHERE o INSTANCE OF :type
DQL;
$q = $this->getEntityManager()->createQuery($dql);
$q->setParameter(
    'type',
    '\MyProj\AlphaObject'
);
$q->getResult(); // 0 row returns

What could I be doing wrong? Or how can I can I debug this?  I know to get the query with $q->getSQL(), but I can't see how to confirm exactly what value it resolves for use with the parameterised query.
I have also tried using \MyProj\AlphaObject::class and combinations beginning without the slash.  Ultimately I'm looking to supply an array types, which also does not work, so I'm focusing on getting one parameter to work first.
These are the versions of the various doctrine packages I am using.
doctrine/annotations               v1.2.7  Docblock Annotations Parser
doctrine/cache                     v1.6.0  Caching library offering an object-oriented API for many cache ba...
doctrine/collections               v1.3.0  Collections Abstraction library
doctrine/common                    v2.6.1  Common Library for Doctrine projects
doctrine/dbal                      v2.5.5  Database Abstraction Layer
doctrine/inflector                 v1.1.0  Common String Manipulations with regard to casing and singular/pl...
doctrine/instantiator              1.0.5   A small, lightweight utility to instantiate objects in PHP withou...
doctrine/lexer                     v1.0.1  Base library for a lexer that can be used in Top-Down, Recursive ...
doctrine/orm                       v2.5.6  Object-Relational-Mapper for PHP```

Try binding $em->getClassMetadata(AlphaObject::class);
On 2 Jun 2017 05:28, "courtney-miles"<EMAIL_ADDRESS>wrote:
If I defined the following DQL, it returns the expected results.
$dql = <<<DQLSELECT o FROM \MyProj\AbstractMyObject AS o WHERE o
INSTANCE OF \MyProj\AlphaObjectDQL;$q =
$this->getEntityManager()->createQuery($dql);$q->getResult(); // 1 row
returns
However, when I supply the instance name as a parameter I get no results.
$dql = <<<DQLSELECT o FROM \MyProj\AbstractMyObject AS o WHERE o
INSTANCE OF :typeDQL;$q =
$this->getEntityManager()->createQuery($dql);$q->setParameter(
'type',    '\MyProj\AlphaObject');$q->getResult(); // 0 row returns
What could I be doing wrong? Or how can I can I debug this? I know to get
the query with $q->getSQL(), but I can't see how to confirm exactly what
value it resolves for use with the parameterised query.
I have also tried using \MyProj\AlphaObject::class and combinations
beginning without the slash. Ultimately I'm looking to supply an array
types, which also does not work, so I'm focusing on getting one parameter
to work first.
This are the versions of the various doctrine packages.
doctrine/annotations               v1.2.7  Docblock Annotations Parser
doctrine/cache                     v1.6.0  Caching library offering an
object-oriented API for many cache ba...
doctrine/collections               v1.3.0  Collections Abstraction library
doctrine/common                    v2.6.1  Common Library for Doctrine projects
doctrine/dbal                      v2.5.5  Database Abstraction Layer
doctrine/inflector                 v1.1.0  Common String Manipulations
with regard to casing and singular/pl...
doctrine/instantiator              1.0.5   A small, lightweight
utility to instantiate objects in PHP withou...
doctrine/lexer                     v1.0.1  Base library for a lexer
that can be used in Top-Down, Recursive ...
doctrine/orm                       v2.5.5  Object-Relational-Mapper for PHP
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/doctrine/doctrine2/issues/6483, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJakA5nQpaWPX3E5V99uA2Dp_JMkosPks5r_4FVgaJpZM4NtyfU
.

Try binding $em->getClassMetadata(AlphaObject::class);

I have given that a go, but it does not appear to have helped.
This is my resulting code.
$dql = <<<DQL
SELECT o FROM \MyProj\AbstractMyObject AS o
WHERE o INSTANCE OF :type
DQL;
$this->getEntityManager()->getClassMetadata(AlphaObject::class);
$q = $this->getEntityManager()->createQuery($dql);
$q->setParameter(
    'type',
    '\MyProj\AlphaObject'
);
$q->getResult(); // 0 row returns

In case it matters, this is triggered from within an Entity Repository method.

$dql = <<<DQL
SELECT o FROM \MyProj\AbstractMyObject AS o
WHERE o INSTANCE OF :type
DQL;
$q = $this->getEntityManager()->createQuery($dql);
$q->setParameter(
    'type',
    $this->getEntityManager()->getClassMetadata(AlphaObject::class);
);
$q->getResult(); // 0 row returns

Thank you @Ocramius.  This works. Apologies for my lack of understanding on how getClassMetadata() is used.
So the last thing I can't figure out it how to get this to work with 2 or more instances.
So this is what I'm trying.
$dql = <<<DQL
SELECT o FROM \MyProj\AbstractMyObject AS o
WHERE o INSTANCE OF (:types)
DQL;
$q = $this->getEntityManager()->createQuery($dql);
$q->setParameter(
    'types',
    [
        $this->getEntityManager()->getClassMetadata(AlphaObject::class),
        $this->getEntityManager()->getClassMetadata(BetaObject::class)
    ]
);
$q->getResult();

I was given the impression somewhere supplying an array would work, but can't find where now.
Thanks again for your help so far.

I've found that trying to use INSTANCE OF with a parameter doesn't work. The parameter is bound with the values given (i.e. the actual class name) instead of the discriminator type value.
So, this:
$dql = <<<DQL
SELECT o FROM \MyProj\AbstractMyObject AS o
WHERE o INSTANCE OF (:types)
DQL;
$q = $this->getEntityManager()->createQuery($dql);
$q->setParameter(
    'types',
    [
        $this->getEntityManager()->getClassMetadata(AlphaObject::class),
        $this->getEntityManager()->getClassMetadata(BetaObject::class)
    ]
);

Will result in this kind of SQL query:
SELECT <fields> FROM my_object o WHERE o.<discriminator_column> IN (:types)

And the types parameter will be the below instead of the discriminator type values:
[
    'MyProj\AlphaObject',
    'MyProj\BetaObject',
]

@courtney-miles INSTANCE OF accepts a single value: you tried to use it like an IN() expression

Closing here, as all questions were fully answered. If you think that the documentation could be improved, please send a patch there.


Add documentation for the parse -> options parameter

Not sure if this will be covered by #21

This should be covered by #21, will work on it :)

Closed with #22


Feature/issue 34 routing infrastructure

@tnightingale @bensinjin @apollomuses @trisapeace PR is up, please have a look. I'll fix the broken unit test tomorrow.

@trisapeace  Files that contain JSX code are named .tsx, files that are just straight TypeScript with no JSX in them are named .ts

This is looking good to me.

lgtm 👍🏻


Change Creditor Name

Change type of page only field is Company name instead of full person name.

Changed to person or business option and can input single company name if chosen