Using the plugin on a Windows machine, one is forced to use backslashes for path separation

Hi,
I don't know if this is intended or not, but to make the plugin run on Windows I had to set the pom.xml like this:
  <plugin>
	<groupId>org.panteleyev</groupId>
	<artifactId>jpackage-maven-plugin</artifactId>
	<version>1.3.0</version>

	<configuration>
	  <name>xxxxxx</name>
	  <appVersion>${project.version}</appVersion>
	  <vendor>xxxxxxx</vendor>
	  <input>target\jmods</input>
	  <destination>target\dist</destination>
	  <mainJar>cli-1.0-SNAPSHOT.jar</mainJar>
	  <javaOptions>
		<option>--enable-preview</option>
		<option>-Dfile.encoding=UTF-8</option>
	  </javaOptions>
	</configuration>

	<executions>
	  <execution>
		<id>win</id>
		<configuration>
		  <icon>src\icons\xxxxxx.ico</icon>
		  <winMenu>true</winMenu>
		  <appVersion>0.0.1</appVersion>
		</configuration>
		<goals>
		  <goal>jpackage</goal>
		</goals>
		<phase>package</phase>
	  </execution>
	</executions>
  </plugin>

...while I expected I could use / as file separator, as it's possible with other plugins, if I'm not wrong.

Hi, this is definitely not the case, Check my other project for example:
https://github.com/petr-panteleyev/password-manager
Works on Windows 10 and 11.


Improve the sniffer design.

I've redesigned sniffer class. Currently the socket transport isn't a part of sniffer. Changes also should allow to add any other transport to the sniffer.

Current coverage is 72.40% (diff: 100%)

Merging #1073 into master will decrease coverage by 2.36%

@@             master      #1073   diff @@
==========================================
  Files           127        126     -1
  Lines         17328      18325   +997
  Methods        2431       2555   +124
  Messages          0          0
  Branches       2120       2240   +120
==========================================
+ Hits          12957      13269   +312
- Misses         3727       4364   +637
- Partials        644        692    +48

Powered by Codecov. Last update e3daefa...5961fcd


added custom core keeper docker image which uses nss wrapper

Description
added custom docker image for core keeper game as a requirement to use the libparty.so plugin https://github.com/PlayFab/PlayFabParty. this plugin is used for cross play on core keeper servers. The following plugin on recent release was causing segfault errors. after investigation I found that the cause was a failure on function calls in c lib getuid(); and getpwuid(uid); which relies on access to the /etc/passwd file. due to the nature of the docker containers when using wings, this directory is not available so the command fails. Another game valheim experienced similar issues. Until the latest version of wings is released, which contains fixes for mounting this directory as pointed here https://github.com/pterodactyl/wings/pull/197, this is the workaround.
All Submissions:

[X] Have you ensured there aren't other open Pull Requests for the same update or change?
[X] Have you created a new branch for your changes and PR from that branch and not from your master branch?

New Image Submissions:

[X] Have you added your image to the Github workflows?
yes

[X] Have you updated the README list to contain your new image?
yes

@parkervcp

@parkervcp

no need to ping him, there other people here that can do the same.
I get that this is an issue but pterodactyl and pelican wings have now a build in fix, just that ptero has to release a newer version before anyone actualy will use it.
for me this is fine but a bit redundand for something that will fix itself in the future.
So I will aprove this but parker himself has the final message if he want to merge this or not.


The prefix-border should disappear after the prefix-timeout expires

Currently, the prefix-border persists after the prefix-timeout has expired, which is visually confusing.

Related to #120 - https://github.com/koekeishiya/kwm/issues/120#issuecomment-174723467


Cannot connect to remote machine if winrm has not been configured

I'm getting the following error when trying to install a Boxstarter package into a Hyper-V VM.
$p is a reference to a custom gist.
PS C:\Windows\system32> Enable-BoxstarterVM -VMName $vmname -Credential $cred -CheckPointName BareOS | Install-Boxstarte
rPackage -PackageName $p
Boxstarter: Configuring local PowerShell Remoting settings...
Boxstarter: Testing remoting access on DESKTOP-2SL8M1N...
Boxstarter: Testing WSMAN...
Boxstarter: Testing WMI...
Boxstarter: Stopping myvm
Boxstarter: Mounted C:\VMs\myvm\myvm.vhdx with system volume to Drive F
Boxstarter: Enabled LocalAccountTokenFilterPolicy with result: The operation completed successfully.
Boxstarter: VHD Dismounted
Boxstarter: Started myvm. Waiting for Heartbeat...
Boxstarter: Creating Checkpoint BareOS
Boxstarter Version 2.10.3
(c) 2017 Matt Wrock. http://boxstarter.org

Boxstarter: Configuring local PowerShell Remoting settings...
Boxstarter: Configuring CredSSP settings...
Boxstarter: Testing remoting access on desktop-2sl8m1n...
Install-BoxstarterPackageOnComputer : Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
At C:\ProgramData\boxstarter\Boxstarter.Chocolatey\Install-BoxstarterPackage.ps1:353 char:21
+ ...             Install-BoxstarterPackageOnComputer $_.ConnectionURI.Host ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Install-BoxstarterPackageOnComputer

Errors       : {Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)), Access is denied. (Exception
               from HRESULT: 0x80070005 (E_ACCESSDENIED))}
ComputerName : desktop-2sl8m1n
Completed    : False
FinishTime   : 18.02.18 11:15:46
StartTime    : 18.02.18 11:15:23

Any suggestion?

Is that user an administrator on the remote machine? Try to see if you can run Enter-PSSession targeting that machine with the same credentials.

I have the same issue and Enter-PSSession is working for me.

Input:
$creds=Get-Credential DESKTOP-T4F13M1\justauser
Enable-BoxstarterVM -VMName $vmName -Credential $creds | Install-BoxstarterPackage -Package HelloWorld -Credential $creds

Output:
Boxstarter: Configuring local PowerShell Remoting settings...
Boxstarter: Testing remoting access on DESKTOP-T4F13M1...
Boxstarter: Testing WSMAN...
Boxstarter: Testing WMI...
Boxstarter: Stopping 17679
Boxstarter: Mounted C:\VMs\drive_images\17679.vhdx with system volume to Drive D
Boxstarter: Enabled LocalAccountTokenFilterPolicy with result: The operation completed successfully.
Boxstarter: VHD Dismounted
Boxstarter: Started 17679. Waiting for Heartbeat...
Boxstarter Version 2.10.3
(c) 2017 Matt Wrock. http://boxstarter.org

Boxstarter: Configuring local PowerShell Remoting settings...
Boxstarter: Configuring CredSSP settings...
Boxstarter: Testing remoting access on desktop-t4f13m1...
Install-BoxstarterPackageOnComputer : Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
At C:\ProgramData\Boxstarter\Boxstarter.Chocolatey\Install-BoxstarterPackage.ps1:353 char:21
+ ...             Install-BoxstarterPackageOnComputer $_.ConnectionURI.Host ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Install-BoxstarterPackageOnComputer

Errors       : {Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)), Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))}
ComputerName : desktop-t4f13m1
Completed    : False
FinishTime   : 5/25/2018 11:16:51 PM
StartTime    : 5/25/2018 11:16:22 PM

Boxstarter: Rolling back remoting settings changed by Boxstarter...
Boxstarter: Unable to disable CredSSP locally

I did not run:
Enable-PSRemoting -Force
I was hoping to eliminate all the manual steps and thought that is what Enable-BoxstarterVM would take care of.
My script did work after logging in and running the 'Enable-PSRemoting -Force' command.
Anyone know of a way to set this automatically?  Perhaps through Autounattend.xml?

I also removed the -Credential $creds in the second piped to Install-BoxstarterPackage.
Working Input:
$creds=Get-Credential DESKTOP-T4F13M1\justauser
Enable-BoxstarterVM -VMName $vmName -Credential $creds | Install-BoxstarterPackage -Package HelloWorld

@zaphodtx This is not an issue in Boxtstarter from what I can understand? To allow a remote connection winrm needs to be configured but you can't connect to do that without it being enabled. Chicken and egg.

This has been stale for some time now so closing. We can reopen this in the future if needed.


[Central_v2] For any user profile information (or similar in Central_v2), we need to show first initial last name for teachers.

If they select none.

This is a small change that affects a few flows, but will affect many components going forward.
In any case where the teacher name has to be shown, the teacher name will be shown in this combo:
[honorific][first initial][period/fullstop][last name]
If no honorific is chosen, none is shown.
Example with honorific: Mr. R. Hinn!
Example without honorific: R. Hinn
An image of one with an honorific is below for the user profile flow:
Image


Another script for another version

Python2 and Python3 are too different. There should be 2 different scripts for both versions.

Ok. Let's prefer 3 and implement just for 3.

Ok. Let's prefer 3 and implement just for 3.

Indeed, let's not care about Python 2 compatibility when there are less than 3 month left until Python 2's official end-of-life.
All new Python software should be written for Python 3 (and is allowed to be Python-3-only, without any support for Python 2).


Live Minecraft Zero Day security issue! Mitigation possible via startup flag

According to many sources there is a currently active zero day vulnerability. See more here:
https://arstechnica.com/information-technology/2021/12/minecraft-and-other-apps-face-serious-threat-from-new-code-execution-bug/
Attackers are already scanning the net for vulnerable servers.
The issue leads to remote code execution on the server!
According to many sources the  JVM flag -Dlog4j2.formatMsgNoLookups=true mitigates this issue.
I suggest quickly adding this flag as default for all  itzg /docker-minecraft-server versions and installations.
Edit:
Apparently Fabric released loader version  0.12.9 with a fix.
Edit2:
CVE: CVE-2021-44228
https://www.randori.com/blog/cve-2021-44228/

More context: https://mobile.twitter.com/slicedlime/status/1469150993527017483

apparently, this fix only works for 1.17 servers if I understand this correctly:
https://www.minecraft.net/en-us/article/important-message--security-vulnerability-java-edition
Im not sure this is ready to close @itzg

I didn't mean to close it. It got auto-closed with the PR.

...however, beyond the mitigation included in #1178 I need advice on what all I can do.

I think it might make sense to also pin this issue to make sure people find the correct information.

FWIW, the change has been merged and published to all the variants and tagged at the respective 2021.23.0 for each variant.

I'll just drop some information here for people who might be interested:
Description of the exploit:
https://www.lunasec.io/docs/blog/log4j-zero-day/
Github Advisory with more links:
https://github.com/advisories/GHSA-jfh8-c2jp-5v3q
In the meantime, I would encourage everyone with publicly accessible servers to take them offline or make sure they are not accessible from the internet. It has apparently been confirmed that bots are already scanning for vulnerable servers.

According to https://www.minecraft.net/en-us/article/important-message--security-vulnerability-java-edition there are workaround for v1.7-1.11.2 & v1.12-1.16.5 too. Could we get those added? Thanks!

@sheepa Thank you for sharing that. Yes, I'll work on getting that added.

Additionally Paper requires special xml files, for example paper 1.11.2 wont boot with the vanilla xml files.
I've pasted the discord message below, I don't know if you want to wait for a more official release but thought I'd let you know.
1. Download the following file (for your version) and place it in the main server directory.
2. Run server with -Dlog4j.configurationFile=log4j2.xml flag (before -jar)

1.8-1.9: https://cdn.discordapp.com/attachments/722785635881975910/918816545705189386/log4j2.xml
1.10: https://cdn.discordapp.com/attachments/722785635881975910/918816283414384670/log4j2.xml
1.11: https://cdn.discordapp.com/attachments/722785635881975910/918816627636731914/log4j2.xml
1.12: https://cdn.discordapp.com/attachments/722785635881975910/918816730061606912/log4j2.xml
1.13: https://cdn.discordapp.com/attachments/722785635881975910/918816806624440380/log4j2.xml
1.14: https://cdn.discordapp.com/attachments/722785635881975910/918816971800334396/log4j2.xml
1.15: https://cdn.discordapp.com/attachments/722785635881975910/918817030512205874/log4j2.xml

Thanks @sheepa . Yeah, I'm thinking I would prefer waiting for an official file location; however, I will adjust these changes I'm doing now to avoid disruption of Paper. Do you know if the vanilla configs are fine for Forge and non-Paper variants in the Bukkit family?

@itzg Unrelated but fabric 1.17-1.18 is fixed as long as loader version 1.12.9+ is used.
https://github.com/FabricMC/fabric-loader/commit/7f8086f2bfe8e8f75c9a319ccd7f2be40be1265e

I personally suspect Mojang will release fixed server jar files soon as the command line arguments are only not really long-term solutions.

Additionally, i've seen some people say that

This exploit does not work on Java 8 and above by default because of this change: https://www.oracle.com/java/technolog... (see "Improved protection for JNDI remote class loading"). This means you are most likely safe from code being executed on your machine.

which would ofc be great if it is true.

Yeah that would be awesome and pretty much make this whole thing a non-issue since Java 8+ has been the norm (other than the Java 7 variant I provide).
In any case, updated images will be ready soon: https://github.com/itzg/docker-minecraft-server/pull/1179

Thanks @sheepa . Yeah, I'm thinking I would prefer waiting for an official file location; however, I will adjust these changes I'm doing now to avoid disruption of Paper. Do you know if the vanilla configs are fine for Forge and non-Paper variants in the Bukkit family?

I have not tested all but for forge the vanilla ones at least let you start the server, unsure about protection. Spigot doesnt work but have released updated jars for buildtools.

Additionally, i've seen some people say that

This exploit does not work on Java 8 and above by default because of this change: https://www.oracle.com/java/technolog... (see "Improved protection for JNDI remote class loading"). This means you are most likely safe from code being executed on your machine.

which would ofc be great if it is true.

That would be awesome indeed, just odd that there is not mention of that on the official minecraft.net article. Is this confirmed?

Merges, build, and tag at 2021.23.1 are on the way.

The proposed fixes by mojang have been implemented in #1179.
Vanilla should be fine now. Things might be different for non-vanilla servers so take care!
Help wanted though: If we missed something please let us know!

Latest images are now published to include Mojang's published fix. They have also been tagged with their respective tag at 2021.23.1

Do the proxy images need updating too? https://github.com/itzg/docker-bungeecord

Is there any specific guidance from bungee and the others? As mentioned in Discord, any software updates on their part can be picked up by restarting the container.

This might be a stupidly simple question, for which I apologise in advance, but I'm the sort of docker user that can do simple stuff like docker run and exec, and following "how-to"s for complicated things, but doesn't really know the intricacies of how it all works ...  so... how do I upgrade an existing installation so as the fixes are installed?   Thanks!

@irrelevantdotcom This guide should help you.
https://phoenixnap.com/kb/update-docker-image-container
Ofc, if you use some GUI to manage your images/containers (e.g. your NAS's web interface, portainer, etc..) then you should do it through gui.

Thank you for the reply, but that guide appears to boil down to "docker rm" your existing container and "docker run" a new one (using a latest pull) with the same name ... this sounds dangerous to the data within....

Thank you for the reply, but that guide appears to boil down to "docker rm" your existing container and "docker run" a new one (using a latest pull) with the same name ... this sounds dangerous to the data within....

Oh, that's fine. That's why and persistent data is saved on mapped paths. These are not part of the container
That means, even if you delete the Minecraft server container, the data is still on your drive because it was never saved inside the container. You can then simply start a new container with the dame /data mapping and it will be like nothing ever changed.
If you're still scared, you can always make a backup of the data from your /data mount before removing the container. :)

And if you didn't use mapped volumes in the first place then that's what this section is for
https://github.com/itzg/docker-minecraft-server#converting-anonymous-data-volume-to-named-volume

Thank you both; I'll do backups, and try that later... (servers are shut down for now.)

@itzg thank you for your prompt attention regarding this. I can confirm, by using the tool created by Huntress
We’ve created a tool to help you test whether your applications are vulnerable to CVE-2021-44228.
You can access the tool here: https://log4shell.huntress.com/

that pulling itzg/minecraft-server:latest appears to have patched my system. I ran a test before upgrading and was found to be vulnerable. After pulling the latest image and restarting Minecraft Launcher the test did not log any further entries.

Additionally, i've seen some people say that

This exploit does not work on Java 8 and above by default because of this change: https://www.oracle.com/java/technolog... (see "Improved protection for JNDI remote class loading"). This means you are most likely safe from code being executed on your machine.

which would ofc be great if it is true.

We mustn't assume this is the case, unfortunately. Though some reputable sources were saying only early Java 8 versions are susceptible to RCE - and it is true that a vulnerability was mitigated then - alternate paths very likely exist and will be used.

Additionally, i've seen some people say that

This exploit does not work on Java 8 and above by default because of this change: https://www.oracle.com/java/technolog... (see "Improved protection for JNDI remote class loading"). This means you are most likely safe from code being executed on your machine.

which would ofc be great if it is true.

We mustn't assume this is the case, unfortunately. Though some reputable sources were saying only early Java 8 versions are susceptible to RCE - and it is true that a vulnerability was mitigated then - alternate paths very likely exist and will be used.

I agree. A tester would be nice? Do anyone know of a tester? If not I could try and create one.

Thanks @ebell451 !
@sheepa perhaps not as automated as you're thinking, but the posted tool https://log4shell.huntress.com/ might be part of the tooling solution.

Will this be coming to the multiarch-latest tag? I tried to pull the latest image but it keeps saying it's on the newest one (which is dated 2 months ago)

Will this be coming to the multiarch-latest tag? I tried to pull the latest image but it keeps saying it's on the newest one (which is dated 2 months ago)

That tag is deprecated. Please switch to one of the active tags
https://github.com/itzg/docker-minecraft-server#deprecated-image-tags

Edit: Apparently Fabric released loader version 0.12.9 with a fix.

How do I force the fabric loader to be 1.12.9?
My workflow:

Downloaded the newest image: docker pull itzg/minecraft-server
Set the loader to be version FABRIC_LOADER_VERSION=0.12.9 in my docker-compose.yml
Restarted my server: docker-compose up -d

In my log I still see<EMAIL_ADDRESS>minecraft_1  | [15:11:58] [main/INFO]: [FabricLoader] Loading 4 mods:
minecraft_1  | 	-<EMAIL_ADDRESS>minecraft_1  | 	- java@17
minecraft_1  | 	-<EMAIL_ADDRESS>minecraft_1  | 	-<EMAIL_ADDRESS>
Any hint will be appreciated.

docker-compose up -d won't recreate the container, you'll need to:
docker-compose stop _servicename_
docker-compose rm _servicename_
 docker-compose up -d _servicename_
Any data which is not in a volume is lost on docker-compose rm

How do I force the fabric loader to be 1.12.9?

I've encountered this before too. It's a bit confusing.
The container only updates fabric and the fabric loader on the first start of a Minecraft version install. Specifically only if fabric-server-${VANILLA_VERSION}.jar is not present.
This is due to
https://github.com/itzg/docker-minecraft-server/blob/eccc989887bc4757d9c3f7083230609bbf7453df/scripts/start-deployFabric#L9
and
https://github.com/itzg/docker-minecraft-server/blob/eccc989887bc4757d9c3f7083230609bbf7453df/scripts/start-deployFabric#L13
which is checked before new loader/installer versions are downloaded. (I'm not sure if this is intended behaviour @itzg but whatever)
Solution: Just delete the server jar and the fabric jar's from the data directory of your server. Starting the docker after that installs the newest fabric loader version.

@StoneLabs  and @itzg , can you explain this?

In the meantime, I would encourage everyone to do the following: 1.18.1: You are safe.

Hm…? Have a look at my latest itzg/minecraft-server container with 1.18.1 - and still an unsafe log4j-core-2.14.1.jar 😢
🇩🇪  ✔ ~/bin [master|✔]
23:54 # ll /var/lib/docker/volumes/137f5aa71947c350ec198ff1e9f4b8ebb27bb4d98e950ea6bab40d36a41319ef/_data/
.rw-rw-r-- 1000 1000   2 B  2 days ago  banned-ips.json
.rw-rw-r-- 1000 1000   2 B  2 days ago  banned-players.json
.rw-rw-r-- 1000 1000  68 B  2 days ago  eula.txt
drwxrwxr-x 1000 1000 4.0 KB 2 days ago  libraries
drwxrwxr-x 1000 1000 4.0 KB 2 days ago  logs
.rw-rw-r-- 1000 1000  44 MB 2 days ago  minecraft_server.1.18.1.jar
.rw-rw-r-- 1000 1000   2 B  2 days ago  ops.json
.rw-r--r-- 1000 1000 1.2 KB 2 days ago  server.properties
.rw-rw-r-- 1000 1000   2 B  2 days ago  usercache.json
drwxrwxr-x 1000 1000 4.0 KB 2 days ago  versions
.rw-rw-r-- 1000 1000   2 B  2 days ago  whitelist.json
drwxrwxr-x 1000 1000 4.0 KB a day ago   world

🇩🇪  ✔ ~/bin [master|✔]
23:54 # ll /var/lib/docker/volumes/137f5aa71947c350ec198ff1e9f4b8ebb27bb4d98e950ea6bab40d36a41319ef/_data/libraries/org/apache/logging/log4j/log4j-core/2.14.1/log4j-core-2.14.1.jar
.rw-rw-r-- 1000 1000 1.7 MB 2 days ago  /var/lib/docker/volumes/137f5aa71947c350ec198ff1e9f4b8ebb27bb4d98e950ea6bab40d36a41319ef/_data/libraries/org/apache/logging/log4j/log4j-core/2.14.1/log4j-core-2.14.1.jar

🇩🇪  ✔ ~/bin [master|✔]
23:54 # docker images | grep itzg
itzg/minecraft-server                    latest       c6b43dc544b1   2 days ago     874MB

I also downloaded latest 1.18.1 Minecraft server jar directly from https://feedback.minecraft.net/hc/en-us/articles/4416161161101-Minecraft-Java-Edition-1-18-1-
And still found this unsafe log4j-core-2.14.1.jar 😢
🍏 ✔ [63%] ▅ 🎄 Thomas.Merz@dm-C02CGH01MD6M:~/Downloads [ttys006/2824]
00:09 $ zip -sf server.jar | grep log4j-core
  META-INF/libraries/org/apache/logging/log4j/log4j-core/
  META-INF/libraries/org/apache/logging/log4j/log4j-core/2.14.1/
  META-INF/libraries/org/apache/logging/log4j/log4j-core/2.14.1/log4j-core-2.14.1.jar

I opened https://bugs.mojang.com/browse/MC-245526 for this issue in 1.18.1

This is an interesting point. I assume that, despite the JAR having the unsafe version in its filename, either the JAR is patched (e.g. by removing or stubbing JdniLookup.class) or mitigations are applied at runtime.

I'm not sure how to make use of the fix. I did a docker-compose pull && docker-compose up -d and then I attached to my container to see what things looked like. I did a px faux | grep java to see what the java pid was and got 526. Then I did sudo -u minecraft cat /proc/526/environ which gave me:
root@9b4cbb397fe5:/data# sudo -u minecraft cat /proc/526/environ
MOTD=Endeavouring to EndeavourHOSTNAME=9b4cbb397fe5LANGUAGE=en_US:enLEVEL=ftb-endeavourJAVA_HOME=/opt/java/openjdkRCON_PASSWORD=minecraftORIGINAL_TYPE=FTBAAUTOPAUSE_PERIOD=10RCON_PORT=25575EULA=trueTYPE=FTBAPWD=/dataFTB_MODPACK_ID=80ONLINE_MODE=trueJVM_DD_OPTS=fml.readTimeout:240 fml.loginTimeout:900FAMILY=FORGEHOME=/dataLANG=en_US.UTF-8FTB_MODPACK_VERSION_ID=2094MODE=survivalWHITELIST=<REDACTED>PVP=falseALLOW_NETHER=trueALLOW_FLIGHT=trueTERM=xtermAUTOPAUSE_TIMEOUT_INIT=600ENABLE_RCON=trueGID=1000AUTOPAUSE_TIMEOUT_KN=120VERSIONS_JSON=https://launchermeta.mojang.com/mc/game/version_manifest.jsonSHLVL=0INIT_MEMORY=6GSERVER=forge-1.16.5-36.2.9.jarDIFFICULTY=easyAUTOPAUSE_KNOCK_INTERFACE=eth0LC_ALL=en_US.UTF-8PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binVERSION=LATESTMEMORY=1GVANILLA_VERSION=1.18.1UID=1000ENABLE_AUTOPAUSE=falseOLDPWD=/dataJAVA_VERSION=jdk8u312-b07MAX_MEMORY=10GAUTOPAUSE_TIMEOUT_EST=3600

There is no JVM_OPTS in the environment variables. Also the only log4j xml file I see on the system is /tmp/log4j2.xml
This is an FTB server (FAMILY=FORGE according to the latest patches). Any advice?

… either the JAR is patched (e.g. by removing or stubbing JdniLookup.class) or mitigations are applied at runtime.

Nope: JdniLookup.class is definitely NOT removed as my query showed already and once again:
for fs in $(mount|grep -E "ext4|overlay|xfs"|awk '{print $3}'); do
  echo $fs:
  find $fs/ -xdev -iname "*.jar" -or -iname "*.war" | while read line; do
    if [ $( zip -sf "${line}" | grep org/apache/logging/log4j/core/lookup/**JndiLookup.class** ) ]; then
       echo "${line}" | tee -a /var/log/CVE-2021-44228.log
    fi
  done
done

This shows /var/lib/docker/volumes/137f5aa71947c350ec198ff1e9f4b8ebb27bb4d98e950ea6bab40d36a41319ef/_data/libraries/org/apache/logging/log4j/log4j-core/2.14.1/log4j-core-2.14.1.jar which is my latest MC server docker container from @itzg 🤷🏻♂
And I don't see any mitigation, too, for my running MC server:
🇩🇪  ✔ ~ [master|✔]
08:54 # pgrep -fla minecraft_server.1.18.1.jar
15711 mc-server-runner --stop-duration 60s --named-pipe /tmp/minecraft-console-in java -Xmx1G -Xms1G -jar minecraft_server.1.18.1.jar
16417 java -Xmx1G -Xms1G -jar minecraft_server.1.18.1.jar
🇩🇪  ✔ ~ [master|✔]
08:54 # for pid in $(pgrep -fla minecraft_server.1.18.1.jar|awk '{print $1}'); do cat /proc/$pid/environ; done
HOSTNAME=763340d15a15LANGUAGE=en_US:enJAVA_HOME=/opt/java/openjdkRCON_PASSWORD=secretORIGINAL_TYPE=VANILLAAUTOPAUSE_PERIOD=10RCON_PORT=12345EULA=TRUETYPE=VANILLAPWD=/dataFAMILY=VANILLAHOME=/dataLANG=en_US.UTF-8AUTOPAUSE_TIMEOUT_INIT=600ENABLE_RCON=trueGID=1000AUTOPAUSE_TIMEOUT_KN=120VERSIONS_JSON=https://launchermeta.mojang.com/mc/game/version_manifest.jsonSHLVL=0MCRAM=1GSERVER=minecraft_server.1.18.1.jarAUTOPAUSE_KNOCK_INTERFACE=eth0LC_ALL=en_US.UTF-8PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binVERSION=LATESTMEMORY=1GVANILLA_VERSION=1.18.1UID=1000ENABLE_AUTOPAUSE=falseOLDPWD=/dataJAVA_VERSION=jdk-17.0.1+12AUTOPAUSE_TIMEOUT_EST=3600HOSTNAME=763340d15a15LANGUAGE=en_US:enJAVA_HOME=/opt/java/openjdkRCON_PASSWORD=secretORIGINAL_TYPE=VANILLAAUTOPAUSE_PERIOD=10RCON_PORT=12345EULA=TRUETYPE=VANILLAPWD=/dataFAMILY=VANILLAHOME=/dataLANG=en_US.UTF-8AUTOPAUSE_TIMEOUT_INIT=600ENABLE_RCON=trueGID=1000AUTOPAUSE_TIMEOUT_KN=120VERSIONS_JSON=https://launchermeta.mojang.com/mc/game/version_manifest.jsonSHLVL=0MCRAM=1GSERVER=minecraft_server.1.18.1.jarAUTOPAUSE_KNOCK_INTERFACE=eth0LC_ALL=en_US.UTF-8PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binVERSION=LATESTMEMORY=1GVANILLA_VERSION=1.18.1UID=1000ENABLE_AUTOPAUSE=falseOLDPWD=/dataJAVA_VERSION=jdk-17.0.1+12AUTOPAUSE_TIMEOUT_EST=3600🇩🇪  ✔ ~ [master|✔]
08:55 #

I also downloaded latest 1.18.1 Minecraft server jar directly from https://feedback.minecraft.net/hc/en-us/articles/4416161161101-Minecraft-Java-Edition-1-18-1-
And still found this unsafe log4j-core-2.14.1.jar 😢

Just because the jar is present, does not mean that the server itself is vulnerable to CVE-2021-44228. There are other methods beyond removing JndiLookup.class that mitigate the exploit. These can be applied at runtime.
Until a statement from Mojang proves us wrong, we should assume such mitigations were taken as explicitly stated in the blog post.
You can further verify this by simply attempting the exploit on an 1.18.1 vanilla server as seen below. Note here, that the server did at no time attempt to establish a connection to the dummy server. I've additionally verified this using packet analyzer tools.

On the topic of CVE-2021-4504:
It is, to my knowledge not known whether Minecraft servers are affected or not. However, as stated by apache:

the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations.

we should therefore assume it is not affected until Mojang says otherwise. Additionally, CVE-2021-4504 does not allow for RCE, only DoS.

I'm not sure how to make use of the fix.

Tip for next time: Use strings /proc/.../environ instead of cat /proc/.../environ. The output is much easier to read :).
But if I'm seeing this correctly, you are running mc version 1.18.1? If so, you are good. There is no need for arguments. If not, please open a new issue, so we can solve this and keep things a little more organized.

Thank you @StoneLabs
If there are further concerns about the mitigations please contact Mojang or Apache.
I am considering locking this issue from further comments, so please respect what is in the power of this image to address and what is not.

Since I didn't see it listed as a potential mitigation in this thread, I'd like to propose one of the other mitigations which would be effective against CVE-2021-45046 (even if it is just a DoS) as well as the original 44228.
zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class
(This was an example, likely would have to be tweaked with a path)
Removing the entire affected class would ofc cause errors should it be used, but in all reality Minecraft, and indeed most software products, shouldn't be using it at all.
Alternatively, the new Log4j 2.16 has blocked use of JNDI entirely.

@itzg Appreciate the work you do here. I've tested all vanilla/snapshot/forge/spigot/bukkit/spigot/paper/etc jars and if you want I can share the logic for making most versions working securely, snapshots for example does work for the official xml files. forge has special xml files.

@itzg Appreciate the work you do here. I've tested all vanilla/snapshot/forge/spigot/bukkit/spigot/paper/etc jars and if you want I can share the logic for making most versions working securely, snapshots for example does work for the official xml files. forge has special xml files.

@sheepa I would prefer to rely on upstream fixes and not get in the business of custom mitigations; however, I would be interested in seeing a PR if it's not too much effort.

Has there been any triage attempted regarding the latest vulnerability found in log4j v2.16 that is patched in v2.17?

Has there been any triage attempted regarding the latest vulnerability found in log4j v2.16 that is patched in v2.17? https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45105

As far as my understanding goes, Minecraft servers are not affected by CVE-2021-45105.


Http filter dynamic forward proxy: bug fix

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

can we get a test demonstrating the buggy behavior and regression testing the fix? thanks! /wait

There is envoy issue https://github.com/envoyproxy/envoy/issues/29087 reported by Boris from my team.
in this issue, it describes the detailed issue and reproduce procedure and issue owner(@yanjunxiang-google) agree to change code.
but for now, Boris and I don't know how to change test code. also, waiting for further comment from owner.
thanks
Cliff

/wait-any

@mattklein123 @yanjunxiang-google @alyssawilk
First thanks very much for your support and give comments.
Next, could you help check if the change can be approved and let's move forward. thanks.

I'm both waiting on @yanjunxiang-google for review, and unlikely to approve until there are tests.  I'm sorry you're unsure how to add tests but if we make code changes without regression testing there's nothings to show this fixes anything, or that it won't get broken again.
/wait

It's a good idea to test the fix.  I am not sure whether TSAN can help here.
@yanavlasov

@yiyibaoguo Thanks for your contribution.  Regarding test, could you please do below:

Changing the Envoy code by adding a 5s sleep here but without your fix. This 5s sleep will increase the race window:
https://github.com/envoyproxy/envoy/blob/7479b6cbdde02186b2fdbad3137965f628d3f42f/source/extensions/common/dynamic_forward_proxy/dns_cache_impl.cc#L402
Reproduce the issue with this test Envoy binary by sending concurrent requests as you described in https://github.com/envoyproxy/envoy/issues/29087 and check the issue is there.
Then adding your fix also with this 5s sleep, and build the test Envoy binary, then sending the same concurrent requests and verify the issue does not exist any more.

Please attach the Envoy trace logs for both 2) and 3) to this PR.

@yanjunxiang-google

@yiyibaoguo Regarding test, could you please do below:

Changing the Envoy code by adding a 5s sleep here but without your fix. This 5s sleep will increase the race window:
https://github.com/envoyproxy/envoy/blob/7479b6cbdde02186b2fdbad3137965f628d3f42f/source/extensions/common/dynamic_forward_proxy/dns_cache_impl.cc#L402
Reproduce the issue with this test Envoy binary by sending concurrent requests as you described in Bug found in dns_cache_impl for dynamic_forward_proxy #29087 and check the issue is there.
Then adding your fix also with this 5s sleep, and build the test Envoy binary, then sending the same concurrent requests and verify the issue does not exist any more.

Please attach the Envoy trace logs for both 2) and 3) to this PR.

I need talking with @coolg92003 for this test further. I had tested 1) & 2) before raising the ticket. (the only difference is sleep time is longer than 5s) However, i did not save log at that time.

@yanjunxiang-google

@yiyibaoguo Regarding test, could you please do below:

Changing the Envoy code by adding a 5s sleep here but without your fix. This 5s sleep will increase the race window:
https://github.com/envoyproxy/envoy/blob/7479b6cbdde02186b2fdbad3137965f628d3f42f/source/extensions/common/dynamic_forward_proxy/dns_cache_impl.cc#L402
Reproduce the issue with this test Envoy binary by sending concurrent requests as you described in Bug found in dns_cache_impl for dynamic_forward_proxy #29087 and check the issue is there.
Then adding your fix also with this 5s sleep, and build the test Envoy binary, then sending the same concurrent requests and verify the issue does not exist any more.

Please attach the Envoy trace logs for both 2) and 3) to this PR.

I need talking with @coolg92003 for this test further. I had tested 1) & 2) before raising the ticket. (the only difference is sleep time is longer than 5s) However, i did not save log at that time.

Cool, glad to hear you had done this test. Sleep longer than 5s is okay. Please attach the logs once you have it.

@yanjunxiang-google
### Issue reproduce: based on original code adding below change: (sleep 10s)
Code Change:
 primary_host_info->host_info_->setAddresses(new_address, std::move(address_list));

const std::chrono::milliseconds DELAY_DURATION{10000};
std::this_thread::sleep_for(DELAY_DURATION);
runAddUpdateCallbacks(host, primary_host_info->host_info_);

Test Result:
First request with 200 ok:
Second request with 503 response:
Log Snapshot:
[2023-10-13 06:56:36.916][3503][debug][forward_proxy] [source/extensions/common/dynamic_forward_proxy/dns_cache_impl.cc:112] cache hit for host '<IP_ADDRESS>:9922'
[2023-10-13 06:56:36.916][3503][debug][forward_proxy] [source/extensions/filters/http/dynamic_forward_proxy/proxy_filter.cc:264] [Tags: "ConnectionId":"1","StreamId":"4270318989915127933"] DNS cache entry already loaded, continuing
[2023-10-13 06:56:36.917][3503][trace][http] [source/common/http/filter_manager.cc:571] [Tags: "ConnectionId":"1","StreamId":"4270318989915127933"] decode headers called: filter=envoy.filters.http.dynamic_forward_proxy status=0
[2023-10-13 06:56:36.917][3503][debug][router] [source/common/router/router.cc:519] [Tags: "ConnectionId":"1","StreamId":"4270318989915127933"] cluster 'service1' match for URL '/trace'
[2023-10-13 06:56:36.917][3503][debug][upstream] [source/extensions/clusters/dynamic_forward_proxy/cluster.cc:396] host <IP_ADDRESS>:9922 not found
[2023-10-13 06:56:36.917][3503][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1942] no healthy host for HTTP connection pool
[2023-10-13 06:56:36.917][3503][debug][http] [source/common/http/filter_manager.cc:985] [Tags: "ConnectionId":"1","StreamId":"4270318989915127933
### Issue fix verification: based on fix code adding below change: (sleep 10s)
 primary_host_info->host_info_->setAddresses(new_address, std::move(address_list));

const std::chrono::milliseconds DELAY_DURATION{10000};
std::this_thread::sleep_for(DELAY_DURATION);
runAddUpdateCallbacks(host, primary_host_info->host_info_);

Test Result:
First request with 200 ok:
Second request with 200 ok:
Log Snapshot:

cache miss for first request
[2023-10-13 07:10:34.555][3862][debug][forward_proxy] [source/extensions/common/dynamic_forward_proxy/dns_cache_impl.cc:119] cache miss for host '<IP_ADDRESS>:9922', posting to main thread
[2023-10-13 07:10:34.555][3862][debug][forward_proxy] [source/extensions/filters/http/dynamic_forward_proxy/proxy_filter.cc:278] [Tags: "ConnectionId":"0","StreamId":"125683892423546047

cache miss for second request:
[2023-10-13 07:10:37.391][3863][debug][forward_proxy] [source/extensions/common/dynamic_forward_proxy/dns_cache_impl.cc:119] cache miss for host '<IP_ADDRESS>:9922', posting to main thread
[2023-10-13 07:10:37.391][3863][dbug][forward_proxy] [source/extensions/filters/http/dynamic_forward_proxy/proxy_filter.cc:278] [Tags: "ConnectionId":"1","StreamId":"8115899584894407690"] waiting to load DNS cache entry

after 10s first request resolve done
[2023-10-13 07:10:44.555][3854][debug][upstream] [source/extensions/clusters/dynamic_forward_proxy/cluster.cc:300] Adding host info for <IP_ADDRESS>:9922
[2023-10-13 07:10:44.555][3854][debug][upstream] [source/extensions/clusters/dynamic_forward_proxy/cluster.cc:279] adding new dfproxy cluster host '<IP_ADDRESS>:9922'
[2023-10-13 07:10:44.555][3854][debug][upstream] [source/common/upstream/upstream_impl.cc:425] transport socket match, socket default selected for host with address <IP_ADDRESS>:9922

200 ok for two request at time [2023-10-13 07:10:44]

Log Detail:
issue reproduce.txt
issue fix verification.txt

Thanks for sharing the test log.

LGTM

@mattklein123 @alyssawilk
Could you help check if the change can be approved? thanks a lot for support.

I still don't see a test and I generally don't think it's a good idea to change code without regression testing /wait

@yanjunxiang-google
Could you help comment above reply from @alyssawilk?

@yiyibaoguo
Could you please add a unit test? The idea is to mock onDnsHostAddOrUpdate(), and check the first_resolve_complete_ is false with your change.  You can try add the test in this file: https://github.com/envoyproxy/envoy/blob/fbac4c7615dcfdf89ab1f846e68a881986c0968d/test/extensions/common/dynamic_forward_proxy/dns_cache_impl_test.cc#L164C7-L164C7.
Also, could you please add a runtime flag to guard your change just to be safe?

@yiyibaoguo
Could you please add a unit test? The idea is to mock onDnsHostAddOrUpdate(), and check the first_resolve_complete_ is false with your change. You can try add the test in this file: https://github.com/envoyproxy/envoy/blob/fbac4c7615dcfdf89ab1f846e68a881986c0968d/test/extensions/common/dynamic_forward_proxy/dns_cache_impl_test.cc#L164C7-L164C7.
Also, could you please add a runtime flag to guard your change just to be safe?

@alyssawilk @yanjunxiang-google
We do not have more knowledge about this part after taking about 1 day to study it. Not sure if you can do the change directly
from your view? thanks

For adding unit test to mock onDnsHostAddOrUpdate(),  you can take a look below draft PR:
https://github.com/envoyproxy/envoy/pull/30506/

Hi @yanjunxiang-google
I see the change including test and fix. Could you help submit this change? We can abandon the current one. thanks!

@yiyibaoguo This PR has a lot of good logs and comments. It probably is slightly easier to track if the fix is committed within the PR.   However, sure, I can help on submit if you prefer.

@yanjunxiang-google
Thanks a lot,  we prefer that if you can help sumbit the fix.

close it because https://github.com/envoyproxy/envoy/pull/30506 is merged


Update r128.h to include latest fixes for mingw-64 32 bit builds.

Credit to @bruvzg for submitting the fixes upstream.

Please update the commit reference in thirdparty/README.md too.

Updated thirdparty/README.md.

Thanks!


HUD broken with Firefox 74.0

HUD is not shown on the first page visited (in this case a local bodgeit).
It appears when the next page is viewed, but HUD tools fail - nothing happens when you click on them.
Lots of Vue errors in the console: "SyntaxError: An invalid or illegal string was specified".
Initially reported vi the User Group: https://groups.google.com/d/msg/zaproxy-users/m-qH7lSao6E/KI2Q460kAgAJ
Chrome 80 appears to still work.

It looks like the referrer value is not being set correctly, which could be due to either https://bugzilla.mozilla.org/show_bug.cgi?id=1601496 or https://bugzilla.mozilla.org/show_bug.cgi?id=1601743 :/

Update - various of the HUD files use document.referrer:

drawer.js
display.js
panel.js
management.js

My current plan is to replace these with a constant and to inject the referrer taken from the header via ZAP.
Initial attempt got some of the HUD working (alerts) but not all of it - getting errors that the target origin provided doesnt match the recipient windows origin, so looks like I may have replaced too much?

Interesting - I just tried a very simple test in FX 74 and the document.referrer element is correct when linking between pages on one site. It also seems to work on https://whatismyreferer.com/
That implies theres something else going on I dont yet understand...

I've been reminded that FX 74 introduced Enhanced Tracking Protection, which removed the referrer when tracking cookies are detected. This sounds possible, but I've just tried disabling ETP with the HUD but it still breaks :(

Without having looked to deep into it, the idea is that one of our hacks of passing some info to the HUD from ZAP via the referer header is failing because FF is removing referer info?

Yep, I think thats it :/

I wonder if we can just pass it through via a GET or POST parameter and carve it out in the ServiceWorker's proxy capability...

In the meantime, is there a last known version of FF that works ?  Nevermind, I just installed version 73 and the HUD is working.

The HUD stopped working when this change was introduced: https://bugzilla.mozilla.org/show_bug.cgi?id=1583700
(Nightly<PHONE_NUMBER>4816, changeset: https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=ad7a152bc66c0d411a6fb0b210d675abed9693c7 )
Setting the preference browser.tabs.documentchannel to false restores the previous behaviour (i.e. referrer is set) and the HUD works again.

A new version of the selenium add-on has now been released which fixes this.

A new version of the selenium add-on has now been released which fixes this.

I updated the selenium add-on and confirmed that it worked.

FYI I've raise this bug on Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1626616

I still can't get it to work in FF74. I've reverted to 73.01 and the HUD works perfectly if I launch it via ZAP, but launch FF via the system and manually setup proxy and certs, the HUD works mostly but it bugs out and the intercept button stops working and I can't delete items side bars.

Oh yeah, if you launch Firefox manually then you'll needs to set the browser.tabs.documentchannel pref to false manually. The fix can only do that if you launch Firefox from ZAP.

I have browser.tabs.documentchannel set to false. Without it the HUD doesn't AT ALL.  But even WITH that setting set false the hud still acts a little funny. Like the break icon doesn't turn red when clicked even though the request does get intercepted sometimes.  It seemed to be working well for a couple days then it started acting up. hmm.   Is there a way to get the zap launch to use a persistent profile so that I can have a few saved passwords in there ?


Wordwrap breaks links and mermaid charts when using use_pymdownx

??? note "{{texts.description.title()}}"
    {{operation.description | wordwrap(76) | indent(4)}}
{% endif %}

The wordwrap here breaks long links, and mermaid superfences code blocks.

Hi @RobertoPrevato can you comment if you expect this to be expected behaviour?


Not in scope: type constructor or class ‘D.Client’

I tried to reproduce the example from the README on my XMonad config, but got the error:
xmonad.hs:61:14: error:
    Not in scope: type constructor or class ‘D.Client’
    Module ‘XMonad.DBus’ does not export ‘Client’.
   |
61 | myLogHook :: D.Client -> PP
   |              ^^^^^^^^

It seems ghc is right :smile:
In any case, the following helped me:
diff --git a/README.md b/README.md
index 8a8ce4f..63962dc 100644
--- a/README.md
+++ b/README.md
@@ -26,9 +26,10 @@ To send status information from XMonad you need to add xmonad-dbus as dependency
     import XMonad
     import XMonad.Hooks.DynamicLog
     import qualified XMonad.DBus as D
+    import qualified DBus.Client as DC

     -- Override the PP values as you would like (see XMonad.Hooks.DynamicLog documentation)
-    myLogHook :: D.Client -> PP
+    myLogHook :: DC.Client -> PP
     myLogHook dbus = def { ppOutput = D.send dbus }

     main :: IO ()

I would create a PR, but maybe there is a better solution.
Thank you for xmonad-dbus!

ahh, thx I wrote example without clarifying it actually, will fix this


Large values in data are not passed properly to Stan

Describe the bug
When putting very large numbers (roughly around 1e9) into data, wrong walues may get passed to stan.
To Reproduce
m_data <- cmdstan_model(write_stan_file("
data {
  real x;
}

model {
  print(x);
}
                                   "))

m_data$sample(chains = 1, iter_warmup = 1, iter_sampling = 0, fixed_param = TRUE,
         data = list(x = 3e8))
# Prints 3e8

m_data$sample(chains = 1, iter_warmup = 1, iter_sampling = 0, fixed_param = TRUE,
         data = list(x = 3e9))
# Prints  -1.29497e+09

Expected behavior
Print 3e9 in the second case
Operating system
Windows 10
CmdStanR version number
<IP_ADDRESS>00 (just installed from GitHub)

I just transferred this issue from CmdStanR to CmdStan because I verified that the same problem happens when using CmdStan directly without CmdStanR. But I'm not sure if this is a CmdStan bug or if it should be moved to stan-dev/stan.

perhaps it's an I/O error.  is the input data for CmdStan in rdump or JSON format?
JSON doesn't distinguish between int and real, even so, 10^9 is should be w/in signed int range; OTOH, Windows might be using very short ints.

perhaps it's an I/O error. is the input data for CmdStan in rdump or JSON format?

The example provided by Martin uses CmdStanR so it was using JSON (and I verified the JSON file was written correctly and also that the same problem occurs without CmdStanR just using CmdStan).

OTOH, Windows might be using very short ints.

This happens on my Mac and Martin's Windows computer so it doesn't seem to be related to the OS.

OK then,  will try to investigate.

Thanks for looking into this so quickly and for the additional verification!

The problem is that 3e9 is written in JSON or R dump as<PHONE_NUMBER> and that is parsed as an integer. If I place
x <-<PHONE_NUMBER>

in a .data.R I get an error that its beyond int range.
In JSON however this:
{
  "x" :<PHONE_NUMBER>
}

gets parsed as an integer and then seems to get converted to a double in some bizarre way.
This however works fine
{
  "x" : 3e9
}

gets parsed as an integer and then seems to get converted to a double in some bizarre way.

Any idea why it's a problem with 3e9 but doesn't seem to be a problem with 3e8?

I am guessing it has to do with the fact that 3e8 can be represented as an integer, but 3e9 can not be.

Are the values as read by the JSON parser already incorrect?

It looks like RapidJSON supports JSON schema.
Maybe it's possible to generate SchemaDocument DOM for the Stan model data block and provide the Schema to the JSON parser?

Are the values as read by the JSON parser already incorrect?

Yeah, the logic is supposed to be that the parser first attempts parsing a number without a decimal point as an integer and once it recognizes the number is too big it parses its as a double. That seems to not be working as expected. I havent had the time to look into that. Will do ASAP.

Maybe it's possible to generate SchemaDocument DOM for the Stan model data block and provide the Schema to the JSON parser?

That is doable as we can get info on types on data variables from stanc3. I would avoid this for now but we can start to think about it if there is no solution for the above.

I've looked into the values that generate problems in my case.
For example
{"qData": [2245935000,<PHONE_NUMBER>] }

But it looks like at least rapidjson master parses them well (as Uint on my machine). But then in cmdstan I get qData[sym1__] is -2.04903e+09 for the first value. And qData[sym1__] is -1.87667e+09 for the second one.

the logic is supposed to be that the parser first attempts parsing a number without a decimal point as an integer

adding a check that the sign is as expected would catch this problem, no?

For me it looks like the cause of the error is at json_data_handler.hpp#L228 when unsigned is converted to signed. For the numbers from the example above this leads to the flip of the sign.

Integers in Stan are always signed. I guess we should not let rapidjson parse them to uint as uint is useless for the Stan use case.

Integers in Stan are always signed. I guess we should not let rapidjson parse them to uint as uint is useless for the Stan use case.

IIUC the rapidjson code, it's not possible to change which handler would be called.
Unsigned number would be always processed by Uint[64] handlers. But the logic of the cmdstan handler could be changed.
Probably -- as it's already done for the reals -- the array has to be switched to values_r if the overflow would occur when converting unsigned to int.


[improve][broker] Emit the namespace bundle listener event on extensible load manager

PIP: https://github.com/apache/pulsar/issues/16691
Motivation
Some pulsar plugins like KoP require using the namespace bundle listener to be aware of the bundle load or unload event. In KoP it will use to immigrate or emigrate the group/txn coordinator data.
But in the extensible load manager, we do not trigger the event.
Modifications

Trigger the unload/load/split event when using the extensible load manager.
Add unit test to verify this feature.

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Codecov Report

Merging #20525 (654dab1) into master (c39f7bb) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master   #20525      +/-   ##
============================================
+ Coverage     72.91%   72.94%   +0.03%
- Complexity    31916    31929      +13
============================================
  Files          1867     1867
  Lines        138558   138561       +3
  Branches      15217    15217
============================================
+ Hits         101026   101076      +50
+ Misses        29501    29461      -40
+ Partials       8031     8024       -7

Flag
Coverage Δ

inttests
24.19% <0.00%> (+0.01%)
:arrow_up:

systests
25.05% <0.00%> (+0.10%)
:arrow_up:

unittests
72.22% <100.00%> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ache/pulsar/broker/namespace/NamespaceService.java
70.47% <ø> (+0.12%)
:arrow_up:

...xtensions/channel/ServiceUnitStateChannelImpl.java
84.43% <100.00%> (+0.05%)
:arrow_up:

... and 61 files with indirect coverage changes


workbench.action.terminal.newWithCwd fails with paths with spaces

Type: Bug
Set up keybinding for the workbench.action.terminal.newWithCwd action in VSCode Version: 1.95.1 like below and use it on the keybindings.json file (on macOS).
The terminal can not be created with the error 'The terminal process failed to launch: Starting directory (cwd) "~/Library/Application Support/Code/User" does not exist.'
Most likely due to the spaces in the fileDirname.
  Note the single ticks around '${fileDirname}'.

Sorry, for the late reply.
My escaping game is not up to snuff, so I am not sure if I am doing something wrong and I tried all possible combinations...but it seems not to help (see first video).
On further inspection, the path with spaces does survive as shown in the notification, so it might not be the escaping of the path. It also works on other paths that contain lots of spaces (see second video) but it also does not work with the settings.json as well. Maybe something is special with that directory? I thought it might be the home ~/ expansion nut that seems to work on other paths.
If I can do something else, please let me know! This is rather low priority for me, however, I am stumped :)
Thanks
https://github.com/user-attachments/assets/c25867cc-a599-4094-91b8-e7386de2fd7c

https://github.com/user-attachments/assets/e9b6a940-2c0e-4f9d-b9b5-d4003bb979bf

Thanks for the info, closing this as I don't think I'll get to it in reality any time soon due to the volume of other bugs I am responsible for.


Support for 2.4 ghz wi-fi phones.

i have the tata nexon 2023 model.
I tried the "Wireless Android  Auto" with Raspberry Pi Zero W 1.1  and the USB cable came with the mobile box (old).
When I tried using my phone (Asus Zenfone Max Pro M1) using Android PIE OS, it had only 2.4 GHz Wi-Fi. It doesn't show any issues or connection success messages or connect to the phone, but it shows no device response sometimes and sometimes. I have connected manually to Bluetooth and Wi-Fi, but no use. When I opened Android Auto and toggled on the wireless connection, it showed that it didn't support 5 GHz. Wi-fi band.
After that I tried with Samsung f55 it was connected with Bluetooth. And working fine.
@nisargjhaveri  would you please resolve to connect to the 2.4 GHz band?

If you're using RPi zero. It only has 2.4Ghz wifi and that's what we're using.
On the phone side, Android Auto officially is supported only with 5GHz wifi. It has been working okay with the 2.4 GHz wifi on most cases, but I haven't tried with a device that does not have 5 GHz wifi at all.
Does going to Android Auto developer mdoe and enabling Wireless Android Auto help? You can enable devoper mode by tapping in the version in the Android Auto app a few times (5 or 7 time, not sure). Once enabled, "Developer settings" will be available in the triple dot menu on the top.


docs Update README

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx
Problem Summary: Update README about v3.X release
Check List (For Committer)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No colde files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Release note

None

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

run buildall


How to use with marp-cli ?

Thanks for these beautiful styles.  I cloned your repo and am able to see the styled slides perfectly with the vs code marp extension.
However, I have not been able to make these work from the command line.  Perhaps this is a question best directed at the marp-cli repo (on the use of themes), but I thought I'd check to see if perhaps you might know since it's an important use case.
The relevant CSS files do not seem to be found. For example from the root of the marpstyle folder I try:

marp --server ./examples  --theme-set ./style

This correctly gives me a choice to open any of the slide demos from the examples folder. But when I open any demo such as example-kant.md the rendered version in the slider presenter looks like this:

Do you have any experience with using these styles from the command line, or any suggestions?  THanks!
update: I did post the same issue to the marp-cli repo, because thinking of this led to a few other questions: https://github.com/marp-team/marp-cli/issues/440

Problem solved as explained at https://github.com/marp-team/marp-cli/issues/440, I needed to add the --html option at the end to make it work.
marp --server ./examples --theme-set ./style --html


Window does not stay on top when "Always on top" is enabled

Windows Terminal version
1.19.11213.0
Windows build number
Microsoft Windows NT 10.0.22631.0
Other Software
No response
Steps to reproduce

Toggle "Always on top" on in settings and save
Keep using terminal
Eventually it stops staying on top

The only way to resume the "Always on top" behaviour is to disable and re-enable the settings with a save in between.
Expected Behavior
No response
Actual Behavior
Terminal window does not stay on top of other windows.

Okay, all the context:

This was originally fixed in #8125
But then I think I broke it in 1.18

Someone else once reported this in #16650

I bet this is a real regression. Should be easy enough to figure out how I messed this up in the process model refactor.

@zadjii-msft is there some debug info I could grab from the application when the issue is happening?


Build issues with Python2

Hi,
This package builds fine on my Fedora 26 box, but I'm having issues on RHEL 6, RHEL 7 and Fedora 25.
-bash-4.2$ python setup.py bdist running bdist running bdist_dumb running build installing to build/bdist.linux-x86_64/dumb running install running install_egg_info running egg_info writing requirements to UNKNOWN.egg-info/requires.txt writing UNKNOWN.egg-info/PKG-INFO writing top-level names to UNKNOWN.egg-info/top_level.txt writing dependency_links to UNKNOWN.egg-info/dependency_links.txt reading manifest file 'UNKNOWN.egg-info/SOURCES.txt' writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
...and this UNKNOWN word leads to other issues as you can guess.
BTW, this command also does not build ldap2pg binary, too.
Can you please let me know how to build ldap2pg on RHEL 7?
Regards, Devrim

Hi @devrimgunduz
You need a recent setuptools that supports setup.cfg metadata.

@devrimgunduz Here is how i build a rpm : https://github.com/dalibo/ldap2pg/blob/master/packaging/build_rpm.sh . Does this help you ?

Hi @devrimgunduz , do you have some update on this ? I can make setup.py compatible with setuptools shipped with RHEL 7.

Hi @bersace . Apologie for the late response, I missed your previous comment. That does not help, because RPM builds are not allowed to install anything external, like pip
FWIW, I build similar packages which use setup.cfg, on RHEL 6 and RHEL 7, like psycopg. What is different in ldapfdw?

Ok, I have to make it compatible.
ldap2pg puts metadata in setup.cfg, following the effort of pypa to be able to build python package without boilerplate (only a setup.cfg). That's just a nice to have. See http://setuptools.readthedocs.io/en/latest/history.html#v30-3-0
setuptools is quite active. But let's support RHEL 7 setuptools.

Hi @devrimgunduz I uploaded ldap2pg 2.1a1 with #90 merged. Can you test it ?

@devrimgunduz version 3.0 is shipped with the fix. Please reopen if you still encounter error ! Regards.

Hi,
Thank you! I built and pushed the RPMs. They will be available in next 40 mins.
Regards, Devrim


Indexed parts show the wrong type

I generated a single rbxm for workspace using remodel, then I added it to my project json file like this:
    "Workspace": {
      "$path": "src/Workspace.rbxm"
    }

When I hover over the variable, I expect to see its type as BasePart. Instead it just shows the name of the instance.

This is on purpose, it makes the type the same as the instance name, the underlying type is a BasePart though.
Do you think it would make more sense to use the class name as the type instead? How does Studio show the type

It does not look like the underlying types are the same. I also get type errors when I try to swap BaseParts between variables.

Roblox Studio shows the variable's type as its class name.

The type error when swapping seems to be related to #12
Even if we changed the name, that error would probably still be there, not entirely sure what's causing it yet.
I think I probably will change the name to the class name though, it makes it better to reason about (and you can already see the name in the script!)


Wasm es6 modules fail to load with Jest, but work in Node

🐛 Bug Report

I'm able to load an es6 module with Node, but I'm unable to load the same module when running with Jest. When I try loading the module with Jest, I get the following error:

node --experimental-modules --experimental-wasm-modules --experimental-vm-modules node_modules/.bin/jest

(node:72954) ExperimentalWarning: VM Modules is an experimental feature. This feature could change at any time
(Use node --trace-warnings ... to show where the warning was created)
FAIL  tests/test_jest.test.mjs
● Test suite failed to run
/Users/me/projects/bin_packing/jest-wasm/node_modules/bin_packer_3d/bin_packer_3d_wasm_bg.wasm:1
({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){

SyntaxError: Invalid or unexpected token

  at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1350:14)

To Reproduce
Steps to reproduce the behavior:

Clone this repo: https://github.com/modulitos/jest-wasm-esm-example
run npm run test-jest

Note that running npm run test-node loads the module successfully in Node. So why does it fail with Jest?
Expected behavior

The module loads when  running npm run test-jest.
Link to repl or repo (highly encouraged)

https://github.com/modulitos/jest-wasm-esm-example
envinfo

❯ npx envinfo --preset jest
npx: installed 1 in 1.358s
System:
OS: macOS 10.15.7
CPU: (4) x64 Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
Binaries:
Node: 14.15.4 - ~/.asdf/installs/nodejs/14.15.4/bin/node
npm: 6.14.10 - ~/.asdf/installs/nodejs/14.15.4/bin/npm
npmPackages:
jest: ^26.6.3 => 26.6.3

The error is about Jest module resolution tries to resolve bin_packer_3d as bin_packer_3d_wasm_bg.wasm  while NodeJS module resolution resolves as bin_packer_3d_wasm_bg.js. Adjust moduleNameMapper to map bin_packer_3d to bin_packer_3d_wasm_bg.js should fix the error.
However, after correcting Jest module resolution, I ran into another error
 RUNS  __tests__/test_jest.test.mjs
Exception in PromiseRejectCallback:
/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:586
    return module;

RangeError: Maximum call stack size exceeded
    at loadEsmModule (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:586:19)
    at linkModules (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:616:19)
    at /Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:572:18
    at node:internal/vm/module:320:30
    at SourceTextModule.<computed> (node:internal/vm/module:319:36)
    at link (node:internal/vm/module:197:22)
    at loadEsmModule (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:571:12)
    at linkModules (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:616:19)
    at /Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:572:18

which Idk how to solve.
If simply solving your original error, moduleNameMapper is enough.

The error is about Jest module resolution tries to resolve bin_packer_3d as bin_packer_3d_wasm_bg.wasm  while NodeJS module resolution resolves as bin_packer_3d_wasm_bg.js. Adjust moduleNameMapper to map bin_packer_3d to bin_packer_3d_wasm_bg.js should fix the error.
However, after correcting Jest module resolution, I ran into another error
 RUNS  __tests__/test_jest.test.mjs
Exception in PromiseRejectCallback:
/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:586
    return module;

RangeError: Maximum call stack size exceeded
    at loadEsmModule (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:586:19)
    at linkModules (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:616:19)
    at /Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:572:18
    at node:internal/vm/module:320:30
    at SourceTextModule.<computed> (node:internal/vm/module:319:36)
    at link (node:internal/vm/module:197:22)
    at loadEsmModule (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:571:12)
    at linkModules (/Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:616:19)
    at /Users/ahn/jest-wasm-esm-example/node_modules/jest-runtime/build/index.js:572:18

which Idk how to solve.
If simply solving your original error, moduleNameMapper is enough.

Thank you for the tip! I updated the repo with the correct moduleNameMapper config, and can confirm that I'm getting the same RangeError: Maximum call stack size exceeded problem.
Any suggestions for how I can troubleshoot this issue is appreciated 🙏

Just now I noticed that it’s a combination of wasm + ESM. Maybe you should try jest@next first and if not working then go to the ESM tracking issue to mention this issue there.

Ah, thanks again for the insight. Can you link me to the "ESM tracking issue"? I'm not sure what you're referring to, but happy to mention this issue there.
I updated to jest@next (updated the repo linked above as well), but now I'm getting this error:

TypeError: wasm.binpacker_packing_algorithm is not a function

It's a different error, but it seems like we're getting closer! But I'm still not sure how to debug this 🤔

https://github.com/facebook/jest/issues/9430

Jest does not support WASM at this time. I'm not sure how support would be added. Is it enough to do WebAssembly.compile on the source code (sorta like the example here)?
I have zero idea how WASM works, so I can't really do much here. Are there any JS globals available in WASM? If not I guess just executing the file and returning the result should be enough?

I guess reading https://github.com/WebAssembly/esm-integration/tree/master/proposals/esm-integration is a start

Thanks everyone for the clarification! This makes much more sense now.
I didn't realize that WASM isn't supported in Jest - I was a little naive in thinking that if I can load a module in Node, then I should be able to load it in Jest.
I appreciate the help 🙏

I was a little naive in thinking that if I can load a module in Node, then I should be able to load it in Jest.

That's how it's supposed to work, but node has added a bunch of stuff for ESM (including WASM) in the last couple of years we haven't been able to follow (especially since most is still behind flags on the Node side)

Are there any JS globals available in WASM?

No, JavaScript global variables are not accessible from WebAssembly by default. The earliest versions of WebAssembly did not have global variables at all. Modern WebAssembly does have global variables, and they can be shared between JavaScript and WebAssembly.

Is anyone currently working on this?

I managed to tweak jest-runtime and this allowed me to import *.wasm from native ESM modules. This is obviously not a production-grade solution, but I hope it can helpful to someone.

Demo repo: kachkaev/wasm-ts-esm-in-node-jest-and-nexjs
The patch: jest-runtime-npm-29.1.2-567b60ffd5.patch

My knowledge of WebAssembly and SyntheticModule APIs is extremely limited, so I’m not sure if my solution is good enough even for an MVP. It’d be great if someone a bit more experienced could sense-check the patch first.
My _importWasmModule function maps WebAssembly.Module imports and exports to an instance of SyntheticModule quite naively, based on just a couple of wasm file examples. It’s possible that I’m not covering some common cases, but I can’t tell that because I don’t know where to find more wasm files 😅

Fair enough. It looks like you pretty much just mapped https://github.com/nodejs/node/blob/7cdf745fddade8b7fc644f44121fbba041d3ac4c/lib/internal/modules/esm/translators.js#L322-L348 to SyntheticModule, so seems fine to me. Let's try and if people hit cases that don't work adjust?

Interesting! I did not see this translator so was crafting the patch in the ‘blind watchmaker’ mode 😅
Not sure I’ll have capacity to create a PR in my work hours, so unfortunately I can start it only around 22-23 Oct (the earliest). I’d be happy if someone else makes an attempt before me – feel free to recycle the demo repo.

@SimenB I started https://github.com/facebook/jest/pull/13505, which essentially copies my experimental patch over to jest-runtime. I have to wrap up for today, but happy to continue the PR later this week if things go as planned. Any feedback iwelcome!

Nice! Just tested 29.3.0 against the same demo project and it worked without a local patch! 🎉

I'm still having a failure with virgil-crypto on node 16, which needs access to a hardware randomness (I assume... it complains about an "entropy source" failing). I'm not sure how it goes about accessing it, but it works when jest isn't involved.

There is a bug in the initial implementation, to do with imports into the Wasm module. Details here: https://github.com/facebook/jest/pull/13505#discussion_r1018498749
If you see
TypeError: modulePath.endsWith is not a function
  at isWasm (../../../node_modules/jest-runtime/build/index.js:210:41)

you can use this patch for now:
diff --git a/node_modules/jest-runtime/build/index.js b/node_modules/jest-runtime/build/index.js
index 9906023..d8458dd 100644
--- a/node_modules/jest-runtime/build/index.js
+++ b/node_modules/jest-runtime/build/index.js
@@ -1509,7 +1509,7 @@ class Runtime {
     const moduleLookup = {};
     for (const {module} of imports) {
       if (moduleLookup[module] === undefined) {
-        moduleLookup[module] = await this.loadEsmModule(
+        moduleLookup[module] = await this.linkAndEvaluateModule(
           await this.resolveModule(module, identifier, context)
         );
       }

@knightcode can you open up a new issue?


⚠ Secret Website 3 has degraded performance

In 0f97dd5, Secret Website 3 ($SECRET_WEBSITE_3) experienced degraded performance:

HTTP code: 200
Response time: 2320 ms

Resolved: Secret Website 3 performance has improved in 229a2be.


Android support

I had write android native support but still need some review 'cause a bit different with me and the author.
I unable to install directly from master to test but I can make in my personal playground project.

Thanks for the PR, working on merging this on a separate branch as I don't want to pull in all of the changes that are included here. I'll update you soon!

@brentvatne Sure, Thank you so much.

merged in https://github.com/expo/react-native-appearance/commit/ea710d3e114d06f1c4ed55678060ef8201a0e78f - thank you!


Extract aya-build for building eBPF crates

Please ignore all but the last commit.

This change is

needs a rebase and a bless


Cannot read properties of undefined (reading 'from') at node_modules/safe-buffer/index.js

Uncaught TypeError: Cannot read properties of undefined (reading 'from')
at node_modules/safe-buffer/index.js (index.js:12:12)
at __require (chunk-UXIASGQL.js?v=7ea3fefc:8:50)
at node_modules/jws/lib/sign-stream.js (sign-stream.js:2:14)
at __require (chunk-UXIASGQL.js?v=7ea3fefc:8:50)
at node_modules/jws/index.js (index.js:2:18)
at __require (chunk-UXIASGQL.js?v=7ea3fefc:8:50)
at node_modules/jsonwebtoken/decode.js (decode.js:1:11)
at __require (chunk-UXIASGQL.js?v=7ea3fefc:8:50)
at node_modules/jsonwebtoken/index.js (index.js:2:11)
at __require (chunk-UXIASGQL.js?v=7ea3fefc:8:50)

I can't fix that -.-

i have the same issue!

same question, node version 16.20.0, vite


Add LICENSE file

I just noticed that this repo does not state licensing information yet. I would add one, but I'm unsure which one is most appropriate in this context. Thoughts?

I'd say BSD 3-Clause, in line with our other projects of similar nature.


ISPN-13039 Upgrade commons-io to 2.8.0

https://issues.redhat.com/browse/ISPN-13039

Integrated into master, thanks @pruivo !


fix for SA for zen

Signed-off-by: aobrzut<EMAIL_ADDRESS>
Description of your changes

Fixes #
I have:

[x] Read and followed Crossplane's contribution process.
[x] Run make reviewable to ensure this PR is ready for review.
[x] Added backport release-x.y labels to auto-backport this PR if necessary.

if the new changes require, also:

[ ] Extended the tests in BedrockShim repo or created an issue for it.
These tests should cover for example checking new expected resources or their content.

How has this code been tested

/lgtm

Pls open tracking issue for a more elegant fix. This one is very ugly


Error in acceptance tests when calling for flash

I'm working on an application, using ember-cli-flash. It's working well, however while writing acceptance tests I'm running into the error
Error: Uncaught Error: Assertion Failed: You have turned on testing mode, which disabled the run-loop's autorun. You will need to wrap any code with asynchronous side-effects in a run (http://localhost:7357/assets/vendor.js:15330)
    at global.onerror (http://localhost:7357/assets/test-support.js:9602:10)

at the line that calls the flash.
Ember.get(this, 'flashMessages').success(data.meta.success);

I tried various things to wrap it in a run loop but it didn't work - notably when I wrapped the line in the run loop, I get an error that the timeout has been exceeded.
Here's a link to the repo: https://github.com/JakeDluhy/ember-boilerplate
and the file: https://github.com/JakeDluhy/ember-boilerplate/blob/master/app/components/user-contact.js
I'd appreciate any insight that you have for me, thanks!

Is the test helper installed and present? You should have something like this as your app/tests/test-helper.js
import resolver from './helpers/resolver';
import './helpers/flash-message';

import {
  setResolver
} from 'ember-qunit';

setResolver(resolver);

I didn't... But I added it so that now my test-helper.js file looks like
import resolver from './helpers/resolver';
import './helpers/flash-message';

import { setResolver } from 'ember-mocha';

setResolver(resolver);

And it didn't fix it. Is it a problem that I'm using mocha?

What version of ember-cli-flash have you installed? You should update to the latest release 1.3.8 if you're not already on it.

I'm on 1.3.8

Apparently, I was wrapping my entire ajax request in Ember.run, when I need to wrap the 'setters' in the promise .then (http://stackoverflow.com/questions/31467547/ember-error-while-testing-you-will-need-to-wrap-any-code-with-asynchronous-side).
I was doing
Ember.run(function() {
  Ember.$.ajax({
    ...
  })
  .done((data) => {
    flashMessages.success(data.meta.success);
  })
  .fail((xhr) => {
    flashMessages.danger(response.errors.error);
  });
});

where I just needed to do
Ember.$.ajax({
  ...
})
.done((data) => {
  Ember.run(function() {
    flashMessages.success(data.meta.success);
  });
})
.fail((xhr) => {
  Ember.run(function() {
    flashMessages.danger(response.errors.error);
  });
});

Additionally, after I did that I was still having timeout problems, which is related to the duration of the flash. To make my tests run faster, I set the default timeout to a lower number. (I do some extra stuff in the first 500 seconds, hence the 501)
if (environment === 'test') {
  ...

  ENV.flashMessageDefaults = {
    timeout: 501
  }
}


v0.9.2 - Update versions

Coverage remained the same at 99.074% when pulling 09856553ed92f90bc6282a43925020f4ad70236e on develop into 28451f8213941b94bf72a838f81b69597d096059 on master.

Coverage remained the same at 99.074% when pulling 09856553ed92f90bc6282a43925020f4ad70236e on develop into 28451f8213941b94bf72a838f81b69597d096059 on master.


Steps towards a NeuroLibre pre-print publication

We plan to submit the manuscript as a preprint to NeuroLibre. In this issue we discuss what it takes, who is doing what, etc.

I told Nikola I'd give a crack at it for some time I have available this week to get an idea of how quick that might get done; I'll start a new repo for it just because I need to mess with some github settings to get it working with Jupyter Book

Repo -> https://github.com/shimming-toolbox/rf-shimming-7t-neurolibre

@mathieuboudreau we aim at submitting the manuscript in ~1w from now. Do you have an ETA for the neurolibre?

@jcohenadad A few update points and general points before answering:

Annoyingly, had difficulty getting datalad setup (and working)  in a Binder-generated container; had to resort to download a snapshot of the data directly and host it as a Google Drive link (for my other NeuroLibre submissions, we ended up having to do this too because NeuroLibre requires that data be fetched and archived using repo2data, and in addition to my troubles mentionned aboved, repo2data doesn't support fetching the derivatives/ folder when a datalad link is provided.
Annoyingly x2, running the notebook (well, the registration step) requires more RAM than I can provide Docker with my 3 yr old laptop (6GB out of my system's 8GB). MyBinder also could not handle this, they've got a very low RAM allowance (2GB). I reached out to Agah to see if I could the NeuroLibre test server to test the capability of running the notebook there, but the test server is not currently operational. The NeuroLibre production server (post-publication) has a ~7.8 GB RAM limit which I think would make this notebook compatible (Colab has 12GB), but I wouldn't be able to know for sure prior to publication. I asked Agah if it would be can to raise this limit for a specific publication, and he said possibly.
Due to the issue above, I couln't ensure that the notebook is reproducible in an environment outside Colab.
Due to the issue above x2, I resorted to running the notebook in Colab, zipping the data+output folder, and host it on Drive. I set this link as the data that repo2data will/would archive in the NeuroLibre submission.
Following this, I've setup a slightly pretty-fied notebook draft that's compatible with a NeuroLibre submission.

Link: https://shimming-toolbox.github.io/rf-shimming-7t-neurolibre/
As for getting it published in NeuroLibre before you click submit:

I think it's possible; @agahkarakuzu and I were able to get my T1 manuscript resubmission through the NeuroLibre steps quite quickly. At the minimum, a DOI is generated & available much earlier than the final accept step from NeuroLibre
[ ] The "Jupyter Book" as is is just a copy of your notebook at the moment; I wasn't given any instructions on going beyond that. I was thinking about adding plotly figures next now that I've got the data "archived" and a much easier to work with setup, but for a 1w deadline maybe that could only be done for your lineplots and not images.
[ ] I had considered adding some sample images of the outputs for some of your steps to fill in the document a bit, but can probably skip this.
[ ] If you want & give me access to the manuscript, I could add some (not all) of that content to the notebook to give a bit more context (the Colab notebook is all the info I had about this work, so I can't really write it out myself)
[ ] I'd need information about the manuscript submission (eg summary; could be abstract. acknowledgements), possibly a new paragraph or two (eg statement of need).
[ ] I'd need a list of authors, their ORCID, their affiliations

Hope this is all clear - let me know which directions you'd like me to go in for some of the points above. Maybe I could push @agahkarakuzu a bit more to get access to the production server during the review stage to ensure that, after publication, the notebook would run on their servers without crashing during the registration step and without timing out.

@mathieuboudreau to clarify a few points:

At the minimum, a DOI is generated & available much earlier than the final accept step from NeuroLibre

We can generate a DOI-formatted interactive preprint URL, but it will not be minted an official DOI before publishing it. Also to achieve this, we need to have the repo submitted and the REVIEW started (so that we have the corresponding issue ID).

Is there a way to not have to duplicate the repos into a new repos? One scenario I anticipate is that, in 6 months, we find a bug on the original repos, we fix it on the original repos and we forget to fix it on the 2nd repos.

Is there a way to not have to duplicate the repos into a new repos? One scenario I anticipate is that, in 6 months, we find a bug on the original repos, we fix it on the original repos and we forget to fix it on the 2nd repos.

Yes I think this is possible, I can either update this current repo to be neurolibre compatible (and move any colab-specific files/notebooks, though there may be a way to merge them together somehow), or simply make a specific branch and point neurolibre to that one. I was just using the new repo to do dev so that you could mute it if you got annoyed by the frequent commits and such (but wanted to keep an eye on this current one).

@jcohenadad I've done some upates + converted the plots to a plotly figure (https://shimming-toolbox.github.io/rf-shimming-7t-neurolibre/), there's a few paths that I can take from here depending on your preference(s)
Path 1
Keep the overall structure of your Colab notebook (eg text is flow of technical info about the analysis), i.e. as it is now.
Path 2
Convert the NeuroLibre submission to a full "preprint" version of the manuscript, i.e. all the text in it but with the code for the figures enbedded as hidden cells in the HTML, i.e. like we did for our T1 mapping challenge manusript and cNeuromod manusript.
The disadvantage with taking this path, is that I'd have to wait for all the co-authors are done their changes to the manuscript before then adding the text/references to the NeuroLibre notebook & formatting the text (takes about a day). This path means a bit more of a delay before submitting to NeuroLibre (and to MRM), as you gave the co-authors until Friday to give their feedback.
Now for both paths, there's a few other decisions/limitations to consider:
Structure of repo notebook(s)
You mentioned that you'd like the notebook to live in this current repo, in case you want to update it/them in the future. Note that, the NeuroLibre publication is essentially an archive of the notebook/data/environment; even if you make changes here, the HTML and accompanying Jupyter Notebook that people would view will never get updated, regardless of what you change here.
So that brings my question, would you want to have two separate notebooks in this repo (one just for NeuroLibre that would likely not be changed later on, and one mostly for a Colab link here that you would always do changes to), or one that would be compatible with both. If the latter case, just a forwarning that, to make the notebook compatible and work smoothly with both the NeuroLibre submission pipeline and NeuroLibre Binderhub (ie., not have it execute the actual pipeline by default; only download the processed data and plot it. An optional flag to run the pipeline in Binderhub would be set), the notebook would not be as clean as your current Colab one (i.e. most cells would have flags that would change the behaviour/what is run depending on if it's in a plot-only mode, running in Binder, or running in Colab).
Note that, overall, Colab provides more resources, so it may be nice to have a notebook compatible with it "as a backup" in case the notebook hits a limit in Binder (though I'd really like it to run completely there; @agahkarakuzu have you got an ETA/idea of how I could test that during the submission?)
If you'd like to have a quick chat tomorrow to touch base on some of these questions let me know; tl;dr I can either do some final touches and submit NeuroLibre close to what it is right now likely by end-of-day tomorrow, or wait for the manuscript and make it look like a full preprint of the manuscript.

Thank you so much for all your efforts @mathieuboudreau, the notebook as is now looks great. I think "path 1" makes more sense because:

I don't see the point in duplicating the text of the published article (which will be open access)
Less work for you
No need to wait for co-authors to finish their edits

But if we go with path 1, one can also wonder: what is the point of a neurolibre book if it's essentially a 'more cosmetic' version of the google colab? Some arguments

results are already produced (ie: no need to wait for the notebook to run)

counterargument: we can save the outputs from a notebook

possible countercounterargument: can the saved outputs be interactive?

neurolibre is a publishable object

Tagging @nstikov @pbellec because this discussion is at the core of the user-who-wants-to-get-their-notebook experience.

Thanks @jcohenadad !
Here are a few more listed benefits of using NeuroLibre:

It generates (and hosts) a docker container image that stores a snapshot of the project + the dependencies that were installed during submission. This image will live «« forever » on the servers, as opposed to images generated by MyBinder (purged regularly), and no worry of complications resulting from reinstallation of the dependencies on Colab in 1, 2, 5, 10 years (eg see your current requirements file; no fixed versions).
NeuroLibre provides a technical evaluation of the submission (ie the notebook runs, and does what it says the authors does). They also will point out some potential flaws in your environment/script setup.
A snapshot of the data is archived on the NeuroLibre servers (ie no downloads will occur when in a Docker session on their servers)
Access to the NeuroLibre Binderhub (more memory, longer computation time limit vs MyBinder)
Web hosting of the Jupyter Book that’s generated
As you mentionned, a DOI associated to the notebook that you can cite

There might be more, @agahkarakuzu knows the backend in and out, so he can comment best more likely.

in brief: neurolibre tests the submission and archives everything needed to reproduce the work as proper academic records, for the long run. Collab does not offer any of that.

Debugging an issue now related to a folder permissions error during the SCT installation when in a docker container. It worked for me everyday last week, but today started to fail.
Opened an issue on repo2docker: https://github.com/jupyterhub/repo2docker/issues/1334
And DM'd @joshuacwnewton (for now, will post to forum later if he deems relevant to SCT specifically) with the following log:

mathieuboudreau@b363f05df303:~$ mkdir content
mathieuboudreau@b363f05df303:~$ cd content
mathieuboudreau@b363f05df303:~/content$ cd ..
mathieuboudreau@b363f05df303:~$ git clone https://github.com/spinalcordtoolbox/spinalcordtoolbox ~/content/sct

cd ~/content/sct
yes | ./install_sct
Cloning into '/home/mathieuboudreau/content/sct'...
remote: Enumerating objects: 60992, done.
remote: Counting objects: 100% (2211/2211), done.
remote: Compressing objects: 100% (1317/1317), done.
remote: Total 60992 (delta 1462), reused 1474 (delta 880), pack-reused 58781
Receiving objects: 100% (60992/60992), 119.15 MiB | 16.17 MiB/s, done.
Resolving deltas: 100% (35117/35117), done.

*******************************
* Welcome to SCT installation *
*******************************

Checking OS type and version...

Linux b363f05df303 6.6.12-linuxkit #1 SMP Fri Jan 19 08:53:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Checking requirements...

OK!

SCT version ......... 6.2.dev0
Installation type ... in-place
Operating system .... linux (unknown)
Shell config ........ /home/mathieuboudreau/.bashrc

SCT will be installed here: [/home/mathieuboudreau/content/sct]

Do you agree? [y]es/[n]o:
Skipping copy of source files (source and destination folders are the same)

Do you want to add the sct_* scripts to your PATH environment? [y]es/[n]o:
Downloading Miniconda...

wget -nv -O /tmp/tmp.j49W08kc15/miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

2024-02-07 19:15:28 URL:https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh [141613749/141613749] -> "/tmp/tmp.j49W08kc15/miniconda.sh" [1]

Installing Miniconda...

bash /tmp/tmp.j49W08kc15/miniconda.sh -p /home/mathieuboudreau/content/sct/python -b -f

PREFIX=/home/mathieuboudreau/content/sct/python
Unpacking payload ...

Installing base environment...

Downloading and Extracting Packages: ...working... done

Downloading and Extracting Packages: ...working... done
Preparing transaction: ...working... done
Executing transaction: ...working... done
installation finished.

Creating conda environment...

CondaError: Error encountered while attempting to create cache directory.
  Directory: /home/mathieuboudreau/.cache/conda/notices
  Exception: [Errno 13] Permission denied: '/home/mathieuboudreau/.cache/conda'

Installation failed!

Please find the file "/home/mathieuboudreau/content/sct/install_sct_log.txt",
then upload it as a .txt attachment in a new topic on SCT's forum:
--> http://forum.spinalcordmri.org/c/sct

mathieuboudreau@b363f05df303:~/content/sct$

in brief: neurolibre tests the submission and archives everything needed to reproduce the work as proper academic records, for the long run. Collab does not offer any of that.

Trying to refine the argument. TL,DR google collab notebooks break relatively fast.
There is a very fast decay of google collab notebooks. Check this example from a tutorial with cneuromod data which I believe was set two years ago: https://colab.research.google.com/drive/10aKI0NcSqWbwxOgvBrcv6xk-LXhKNM2u?usp=sharing#scrollTo=iL3KlwjxgOOq
It won't run because it uses data hosted on google drive, and the data is no longer available.
As Mathieu already pointed out, this decay will also happen with dependencies. I would be very surprised that a google collab environment runs after 3-5 years (or much less than that really). A google search for "broken dependencies google collab" gives over 100k hits (sample).
Fun fact: ubuntu does retire their channels. Controlling versions in a docker build file does not mean this environment will build in a few years. The only way to make a work reproducible is to save the binary environment along with the code.
I would not be surprised if neurolibre preprints still run in a couple decades, provided we manage to maintain the platform. I am saying this because we archive binary container images, and that given the very large number of container binaries out there it is almost certain that archivists will develop solutions for long term support of these images.

Debugging an issue now related to a folder permissions error during the SCT installation when in a docker container. It worked for me everyday last week, but today started to fail. This would impact a NeuroLibre build, as they use repo2docker as well I believe.

Issue is now fixed, see this comment: https://github.com/jupyterhub/repo2docker/issues/1334#issuecomment-1933233055

@mathieuboudreau just checking what is the timeline for the NeuroLibre submission? Thanks

@jcohenadad I mad a PR last week (#91) and tagged/pinged you for review. I’m waiting for the these changes to be merged before submitting to NeuroLibre, as you requested in this thread that the submission be hosted in this repo and not in the separate repo I had made. So the ETA is as soon as you approve those changed/content and it gets merged master, I’ll submit (which takes just minutes).

Just currently doing a check the the script/setup reproduces on Colab now as a last-minute sanity check (already found and fixed one minor bug), and that the repo2docker setup also works. Will submit to NeuroLibre ASAP after this check

Submitted to NeuroLibre,

There is not a public link yet, this is from my dashboard when I'm logged in. Once @agahkarakuzu or @pbellec accept it for review, there will be a GitHub issue opened on the NeuroLibre GitHub and I'll post the link here.

amazing! is that ok to submit to MRM at this point?

If we can wait a short amount of time so that they can trigger the start of the process, then I believe it would generate a DOI that could add to the manuscript (even though the neurolibre review process hasn’t been done/completed).
@agahkarakuzu or @pbellec, could you do this ASAP?

@mathieuboudreau the DOI link (after publication) will be:
https://doi.org/10.55458/neurolibre.00025
Reproducible preprint will be served at:
https://preprint.neurolibre.org/10.55458/neurolibre.00025
I've seen your notes on the submission form regarding two options, (2hr run vs quick run), we'll test and see how it goes.

@mathieuboudreau if you give me write access to the repo, I can quickly push fixes needed, or I can send PRs, whichever is more convenient for you.

Done.

amazing! is that ok to submit to MRM at this point?

@jcohenadad the submission has passed the pre-review stage (https://github.com/neurolibre/neurolibre-reviews/issues/24) and is currently under review (https://github.com/neurolibre/neurolibre-reviews/issues/25).
I've updated the details Agah shared (doi and link) in the Data Availability Statement; they aren't currently active but I've indicated in the manuscript that it has been submitted and is under review along with those links.

@jcohenadad now that you've decided to keep the old MPL-based figure in the manuscript instead of the Plotly one, would you like me to re-integrate the MPL code in the notebook that generates that image? The lines of code that were removed are in https://github.com/shimming-toolbox/rf-shimming-7t/commit/0d0516096365e6c2f97ae8afd82d11cbed301648 as your comment https://github.com/shimming-toolbox/rf-shimming-7t/pull/91#pullrequestreview-1878413109 suggested to switch to the plotly one for the manuscript (and thus, the MPL code wasn't needed anywhere anymore).

i think there has been some misunderstanding https://github.com/shimming-toolbox/rf-shimming-7t/commit/0d0516096365e6c2f97ae8afd82d11cbed301648#commitcomment-138731916


Apply ignore first, and ignore -flto flag.

Some standard headers are not found by clang-tidy if -flto flag is
given.

I ran it on an open source project and I didn't see any regression.


Entra Application Proxy - Increase "transactions per second"

From the article, it mentions that there has limits in the Application Proxy. Is this a soft-limit that we can increase it via support ticket? What is the higher value we can increase to? Or, is it a hard-limit? It is because we want to publish on-premises web applications that may be demanding.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#azure-active-directory-limits
A maximum of 500 transactions per second per Application Proxy application.
A maximum of 750 transactions per second for the Azure AD organization.*
*A transaction is defined as a single HTTP request and response for a unique resource. When clients are throttled, they'll receive a 429 response (too many requests). Transaction metrics are collected on each connector and can be monitored using performance counters under the object name Microsoft AAD App Proxy Connector.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 36f49a06-90b1-cbe1-3853-2c38277b79f8
Version Independent ID: 02ff6f6c-3ef6-b48f-060d-b290419e0aba
Content: Azure subscription limits and quotas - Azure Resource Manager
Content Source: articles/azure-resource-manager/management/azure-subscription-service-limits.md
Service: azure-resource-manager
Sub-service: management
GitHub Login: @tfitzmac
Microsoft Alias: tomfitz

@simonkw2000
Thanks for your feedback! We will investigate and update as appropriate.

@simonkw2000, thank you for the question. As mentioned in the document, (under note section of "Managing limits") -
When a service doesn't have adjustable limits, the following tables use the header Limit without any additional information about adjusting the limit. In those cases, the default and the maximum limits are the same.
For Application Proxy, it's very rare that the limits will be exceeded (except in case of software bugs in published app, hardware config issue, DDOS attack etc.). Therefore, the option to update that limit is not mentioned in the doc. This can be only done by the product group and only after verification that the limits are exceeding or the transaction per seconds are close to the limits.
You may open a support ticket to initiate the process of validation of limit exhaustion and then the request will be forwarded to the product group for further consideration of the request.


change spelling, fix minor mistakes

closes #771
I tried to match a list of British / American difference pairs with each OEO-module, changed the spelling of American terms and added alternative terms if needed. I also fixed some spelling mistakes I found.
Here is the list of terms I found:
british-american-terms-oeo-by-module.txt

The AE spelling for the lables as has exact synonym are still missing.

As I understood it, the consensus in the meeting was to use alternative term, which makes sense in connection to #619