Add Windows container support

This adds Windows image with SSH support. I can add tests if desired.

I built the image and pushed it to a local registry (stefanscherer/registry-windows). After that I configured jenkins with SSH key injection.
Remote file system root is set to "c:\Users\jenkins".
User is set to "jenkins".
Anything else I need to know regarding configuration in jenkins?
Because of short test results in
"Could not find the file /root in container ..."

While testing it locally I didn't see that issue. Can you post a full log? Do you have any env variables set in Jenkins global configuration?

Here is the current log. Its jenkins LTS with the current docker plugin 1.1.7 and SSH slaves plugin 1.30.1 running as master itself in a docker container. The master requests to for the win-ssh-agent (based on your files) image via docker daemon on a windows 10 pc. The windows 10 pc contains docker desktop (CE) edition with windows containers.
Aug 13, 2019 12:59:25 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Asked to provision 1 slave(s) for: win-ssh-agent
Aug 13, 2019 12:59:25 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud canAddProvisionedSlave
Provisioning 'localhost:5000/win-ssh-agent' number 1 (of 1) on 'windows docker cloud'; Total containers: 0 (of 4)
Aug 13, 2019 12:59:25 PM INFO com.nirima.jenkins.plugins.docker.DockerCloud provision
Will provision 'localhost:5000/win-ssh-agent', for label: 'win-ssh-agent', in cloud: 'windows docker cloud'
Aug 13, 2019 12:59:25 PM INFO hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
Started provisioning Image of localhost:5000/win-ssh-agent from windows docker cloud with 1 executors. Remaining excess workload: 0
Aug 13, 2019 12:59:25 PM INFO com.nirima.jenkins.plugins.docker.DockerTemplate pullImage
Pulling image 'localhost:5000/win-ssh-agent:latest'. This may take awhile...
Aug 13, 2019 12:59:25 PM INFO io.jenkins.docker.client.DockerAPI getOrMakeClient
Cached connection io.jenkins.docker.client.DockerAPI$SharableDockerClient@41f0787 to DockerClientParameters{dockerUri=tcp://lp13007:2375, credentialsId=null, readTimeoutInMsOrNull=300000, connectTimeoutInMsOrNull=60000}
Aug 13, 2019 12:59:26 PM INFO com.nirima.jenkins.plugins.docker.DockerTemplate pullImage
Finished pulling image 'localhost:5000/win-ssh-agent:latest', took 987 ms
Aug 13, 2019 12:59:26 PM INFO com.nirima.jenkins.plugins.docker.DockerTemplate doProvisionNode
Trying to run container for localhost:5000/win-ssh-agent
Aug 13, 2019 12:59:26 PM INFO com.nirima.jenkins.plugins.docker.DockerTemplate doProvisionNode
Trying to run container for node win-ssh-agent-0000vofz0qwso from image: localhost:5000/win-ssh-agent
Aug 13, 2019 12:59:26 PM INFO com.nirima.jenkins.plugins.docker.DockerTemplate doProvisionNode
Started container ID 2919d3af76f178f5e7b27920d83bd846617d78dc9309cb356833ecc5b83d59bc for node win-ssh-agent-0000vofz0qwso from image: localhost:5000/win-ssh-agent
Aug 13, 2019 12:59:26 PM SEVERE com.github.dockerjava.core.async.ResultCallbackTemplate onError
Error during callback
com.github.dockerjava.api.exception.NotFoundException: {"message":"Could not find the file /root in container 2919d3af76f178f5e7b27920d83bd846617d78dc9309cb356833ecc5b83d59bc"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:103)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)

Aug 13, 2019 12:59:26 PM SEVERE com.nirima.jenkins.plugins.docker.DockerCloud$1 run
Error in provisioning; template='DockerTemplate{configVersion=2, labelString='win-ssh-agent', connector=io.jenkins.docker.connector.DockerComputerSSHConnector@7f369995, remoteFs='C:\Users\jenkins', instanceCap=1, mode=NORMAL, retentionStrategy=com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy@2b65f1cf, dockerTemplateBase=DockerTemplateBase{image='localhost:5000/win-ssh-agent', pullCredentialsId='', registry=DockerRegistryEndpoint[null;credentialsId=null], dockerCommand='', hostname='', dnsHosts=[], network='', volumes=[], volumesFrom2=[], environment=[], bindPorts='', bindAllPorts=false, memoryLimit=null, memorySwap=null, cpuShares=null, shmSize=null, privileged=false, tty=false, macAddress='null', extraHosts=[]}, removeVolumes=false, pullStrategy=PULL_ALWAYS, nodeProperties=[], disabled=BySystem,0 ms,4 min 59 sec,Template provisioning failed.}' for cloud='windows docker cloud'
com.github.dockerjava.api.exception.NotFoundException: {"message":"Could not find the file /root in container 2919d3af76f178f5e7b27920d83bd846617d78dc9309cb356833ecc5b83d59bc"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:103)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)

What might be interesting too is, that starting the container with docker run -d win-ssh-agent, it starts and quits after some seconds (I guess it shouldn't).
docker logs shows
WARNING: Waiting for service 'OpenSSH SSH Server (sshd)' to start...
PS C:\Users\jenkins>

Jenkins docker plugins don't support Windows containers for agents. You'd need to spin up the agent container manually until that changes. Also, when running manually, if you use -it it won't exit. It's something I have been trying to figure out since sshd is running as a service.

Jenkins docker plugins don't support Windows containers for agents. You'd need to spin up the agent container manually until that changes.

I feared something like that. Hmm ... have to think about. Thanks Alex for your help.

@slide Any good hint/example how to do it manually without docker-plugin?

I just ran the container manually on my server. That is not ideal obviously. I don't have a better solution at this point.

@BlueAndi I'll see if I can get around to upstreaming our fix to docker plugin that adds support for Windows sadly it kinda depends on Git client v4 which is still in beta

@BlueAndi I'll see if I can get around to upstreaming our fix to docker plugin that adds support for Windows sadly it kinda depends on Git client plugin v3 which is still in beta.

@casz That would be great! You are everywhere, from JCasC to here ... cool ... don't forget to sleep between. ;-)

https://github.com/jenkinsci/git-client-plugin/pull/371 has been released in git client plugin 2.8.1

I have 0 experience with Windows cointainers so can't offer a significant advise on this PR.

I will it a try in the next days.

@casz Did you upstream the fix to the docker plugin with windows support?

There is PR open but I haven't been able to get around to it: https://github.com/jenkinsci/docker-workflow-plugin/pull/184
You should be able to test it with this: https://ci.jenkins.io/job/Plugins/job/docker-workflow-plugin/job/PR-184/1/artifact/org/jenkins-ci/plugins/docker-workflow/1.20-rc400.6982db7f7860/docker-workflow-1.20-rc400.6982db7f7860.hpi

@casz As a first feedback, it looks good and I got it running. This means the container is remotely created, started and stopped again. I will do some more tests. The windows container base on this PR, but not with the latest modifications.

@slide After the container is started, I try to connect via ssh jenkins@localhost for test. But I get permission denied. A look into the containers sshd logs shows:
Bad owner on C:/Users/jenkins/.ssh/authorized_keys

Any hint?

@BlueAndi Not sure why you would use localhost unless you are passing port 22 through?

I am on the same machine, where the docker container is running. Its just for the test. I get the same problem, connecting from a different machine. The sshd in the container aborts with the bad ownership log. Today I took a look to the openssh documentation and there they said only SYSTEM, Administrators and the owner shall have access. Not sure about the additional SID, see above.

The SID you see should be the jenkins user account SID. I'll try and look at this later today.

This is the patch I did for git client plugin for it to work with Openssh: https://github.com/jenkinsci/git-client-plugin/pull/447

@BlueAndi I am unable to reproduce your issue. I am running the container as follows:
 docker run -p 2022:22 jenkins4eval/ssh-agent:test-windows "MYPUBLICKEYHERE"

Then I am ssh'ing with the following:
ssh -i path\to\private\key jenkins@localhost -p 2022

This is working for me just fine.

@slide Strange ... do you have the log about the bad ownership in your sshd log? Maybe I am pushed into the wrong way by this log?! I will rebuild the whole image tomorrow again and check again.

What log are you looking at?

If I look at C:\ProgramData\ssh\logs\sshd.log I do NOT have any errors in that file.

I cleaned today everything and build the image again, but unfortuantely no success.

Do you have a user name Jenkins on the system you are building the image on?

No, I have not.
BTW I checked the SID in the container:
PS C:\Users\jenkins> wmic useraccount get name,sid
Name                SID
Administrator       S-1-5-21-246356162-514904233-2839682062-500
DefaultAccount      S-1-5-21-246356162-514904233-2839682062-503
Guest               S-1-5-21-246356162-514904233-2839682062-501
jenkins             S-1-5-21-246356162-514904233-2839682062-1000
WDAGUtilityAccount  S-1-5-21-246356162-514904233-2839682062-504

but does not match with
PS C:\Users\jenkins\.ssh> icacls.exe .\authorized_keys
.\authorized_keys NT AUTHORITY\SYSTEM:(I)(F)
                  BUILTIN\Administrators:(I)(F)
                  S-1-5-83-1-3924327273-1158986629-2692956073-3342425729:(I)(F)
                  BUILTIN\Users:(I)(RX)

How are you building the image?

I build it on a windows 10 pc, with docker for desktop ce v<IP_ADDRESS>.
Docker ist configured for windows containers, of course.
The Dockerfile, the CreateProfile.psm1 and the setup-sshd.ps1 scripts are in the same directory, where I call docker build -t lp13007:5000/win-ssh-agent .

Did you pass -f Dockerfile-windows?
docker build -f Dockerfile-windows -t <yourtag> .

No, I didn't. Is this a must, because I configured to use windows containers at all?

Yes, you need to build with the correct Dockerfile. Docker doesn't
automatically use Dockerfile-windows, you have to specify or else it will
be using the file named Dockerfile.
On Wed, Sep 18, 2019 at 8:26 AM Andreas Merkle<EMAIL_ADDRESS>wrote:

No, I didn't. Is this a must, because I configured to use windows
containers at all?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jenkinsci/docker-ssh-slave/pull/27?email_source=notifications&email_token=AAANUP2TKKZOZI6VTBYUWVDQKJCC7A5CNFSM4IKCCIHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7AOYGA#issuecomment-532737048,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAANUP3M73XGVLZS4V3HAKLQKJCC7ANCNFSM4IKCCIHA
.

--
Website: http://earl-of-code.com

My dockerfile name is "Dockerfile" ;-)

Where are you pulling your base image from? Are you building this PR and then do you have a FROM?

I downloaded only the files from this PR and build it. The base image is the one, of your Dockerfile: openjdk:8-jdk-windowsservercore-1809

Ok, so you downloaded the Dockerfile-windows and renamed it Dockerfile? That should work fine.

Yes and thats why I am wondering what's wrong.

Can you try running the jenkins4eval/ssh-agent:latest-windows image I just pushed?

A colleague did that for me and unfortunately, the same problem appears:
2620 2019-09-18 21:41:48.033 debug3: Bad owner on C:/Users/jenkins/.ssh/authorized_keys
2620 2019-09-18 21:41:48.033 Authentication refused.

So we tested it on a different windows 10 pc, with docker desktop ce.
:-(
How did you create the keys? Thats the only thing, we didn't check right now.
And you are running on Win10 too?

I have only run on Windows Server 2019. I don't have a Windows 10 PC with docker enabled. To create the key I used puttygen and then just copied the public key portion to pass as the argument to the container for the docker run. The ssh-setup.ps1 is what sets the permissions on the key file, you could try adding some logging to that file to dump permissions and user info and so forth.

Windows Server 2019 may be the interesting difference. We will investigate further.
BTW thanks for your help, much appreciated!

What version of Docker are you using?

I tried running on a Windows 10 system. I had to use a different tag to build the image, then I was able to SSH into the system using the key just fine. No error about bad ownership on the key. Can you provide all the steps you are using to try and get this working?

The steps with your image:

Open a windows console.

docker pull jenkins4eval/ssh-agent:latest-windows

ssh-keygen -t rsa, entering no passphrase and storing it in the user directory (%USERPROFILE%\.ssh)

docker run -p 2022:22 jenkins4eval/ssh-agent:latest-windows "ssh-rsa XXXXXXXXXXXXX" using the public key from %USERPROFILE%\.ssh\id_rsa.pub

Open a 2nd windows console.

del %USERPROFILE%\.ssh\known_hosts to ensure a new fingerprint will be created.

ssh -i %USERPROFILE%\.ssh\id_rsa jenkins@localhost -p 2022

Confirm with "yes" to continue connecting, because of missing fingerprint.

Result: Permission denied (publickey). and in the container sshd log, the Bad owner ...

Which tag did you have to use?

My Windows 10 version required me to use 1709 for the WINDOWS_DOCKER_TAG arg. You have to match the Windows version of the container to the Windows version of the host machine. If you are using Docker Desktop, I had to use the NAT address of the container instead of using localhost. I am not sure why that is the case, you can get the NAT IP address using the following.
docker inspect --format '{{ .NetworkSettings.Networks.nat.IPAddress }}' <container>

We use a 1903 ... funny is, that the 1809 runs on a 1903, but only with the latest updates from Microsoft. openJDK provides right now no image with 1903, so I have to build first a openJDK image with 1903. But that could be the key.
Regarding NAT, it was no problem. We didn't have to change something.
But lets try with 1903 and I will send feedback about the result.

Yeah, I removed the NAT portion of my comment because I realized my network was setup weird on that system. Once I tried with a normal network setup I was able to use localhost no problem.

In the meanwhile I built a openJDK image with
FROM mcr.microsoft.com/windows/servercore:1903
and used this as base for. But ... without success ... same problem. argh
Lets stop here now. I will continue the investigation next week.
Anyway again thank you very much!!!! Without your help and suggestions it would be much harder.

@ndeloof Are you planning on continuing to maintain this repo? If not, would it be ok if I added myself to the committer list via the ircbot?

FYI
I searched in the openSSH sourcecode for "Bad owner on ..." to find out whats the problem in detail.
The code checks the SID:
	if (!IsWellKnownSid(owner_sid, WinBuiltinAdministratorsSid) &&
		!IsWellKnownSid(owner_sid, WinLocalSystemSid) &&
		!EqualSid(owner_sid, user_sid)) {
		debug3("Bad owner on %S", path_utf16);
		ret = -1;
		goto cleanup;
	}

I would assume its failing on the following:
!EqualSid(owner_sid, user_sid))

It's working now, but I am not sure whether I understand it.

First I created a local user "jenkins" on the host system.
In the Dockerfile I removed VOLUME "${JENKINS_AGENT_HOME}" "C:\Users\${user}\AppData\Local\Temp".
Rebuilt the image.
Now inside the container the ./.ssid directory and the authorized_keys file have the correct permissions without any strange SID. And I can successful connect with ssh.

Do you have a local "jenkins" user on your host system?
And I found the notes especially for windows container volumes: https://docs.docker.com/v17.09/engine/reference/builder/#volume

Volumes on Windows-based containers: When using Windows-based containers, the destination of a volume inside the container must be one of:
a non-existing or empty directory
a drive other than C:

No, I do not have a local jenkins user on the host system.

If I use VOLUME "${JENKINS_AGENT_HOME}" "C:\Users\${user}\AppData\Local\Temp", the problem raises up again, independed of a existing jenkins user on the host system. Try to do tomorrow some more tests.

We can definitely change it to another location inside the image. Maybe this is a difference between Docker CE and Docker EE?

According to this, the VOLUME stuff is no longer an issue on Windows Server 2019. https://blog.sixeyed.com/what-you-can-do-with-docker-in-windows-server-2019-that-you-couldnt-do-in-windows-server-2016/. I'm not sure if Windows 10 would be considered to be more like Windows Server 2016 or Windows Server 2019, but that could be the root of the issues you are seeing.

But you tested it on windows 10 as well and without an existing jenkins user. Did you use there docker EE?

No, I did not use EE.

I pushed a version of the image based on this PR to jenkins4eval/ssh-agent:test-windows. Please try it out and let me know if it works.

Same problem. Can you please create additional the user "jenkins" in the host system, before you build the image? Ensure that the user directory exists.
See again the Hyper-V related SID:
PS C:\Users\jenkins\.ssh> icacls.exe .\authorized_keys
.\authorized_keys NT AUTHORITY\SYSTEM:(I)(F)
                  BUILTIN\Administrators:(I)(F)
                  S-1-5-83-1-2115873755-1246499530-1890222465-4150819093:(I)(F)
                  BUILTIN\Users:(I)(RX)

That is a problem because there won't necessarily be a enkins user on the systems when this is built in automated builds.

That is true. BTW the VOLUME "${JENKINS_AGENT_HOME}" "C:\Users\${user}\AppData\Local\Temp" in the Dockerfile assumes there is a jenkins user on the host system, where the container is running.

I believe that line is creating two volume mount points.
Dockerfile's VOLUME specify one or more volumes given container-side paths. But it does not allow the image author to specify a host path. On the host-side, the volumes are created with a very long ID-like name inside the Docker root.

Is there any additional feedback on this, or can it get merged? I don't have access to merge in this repo.

Nothing from my side.


Add positive feedback sound to play at the same time as eating sound when getting correct answer

User Story
As a UX designer,
I want to play this positive SFX at the same time as the eating sound when I get a correct answer,
So that we are adding positive sound effects that help to increase engagement.
Note: I can't find the actual sound from FTM Unity that plays so I have substituted it with another SFX I found while browsing.
Acceptance Criteria
Given that I am playing FTM JS,
When I drag the correct stone to the monster,
Then I hear this new positive SFX and the eating SFX at the same time before the spoken feedback audio.

Hi @burrage,
I trust you're doing well. I'm seeking clarification regarding the word puzzle feature. With multiple stones for a single word, should the positive SFX  play after placing all correct stones, as we're triggering feedback audio when all stones are correct? Or should it play after each successful stone drop? Clarifying this will help ensure we maintain a cohesive user experience.
Best regards,
Nikhil

10 out of 10 question @Nikhil77sutara !
Looking back at the Unity version, on word spelling levels, a very slightly different sound effect plays after each correct stone followed by the SFX we linked on the very last stone.
Since it is such a subtle difference in sound that might not make a huge difference, let's go ahead and try this same sound effect linked in the task card for each correct stone and see how it sounds and we can decide from there.

10 out of 10 question @Nikhil77sutara !
Looking back at the Unity version, on word spelling levels, a very slightly different sound effect plays after each correct stone followed by the SFX we linked on the very last stone.
Since it is such a subtle difference in sound that might not make a huge difference, let's go ahead and try this same sound effect linked in the task card for each correct stone and see how it sounds and we can decide from there.

Sure ,
Thanks Ben!


ghc-mod doesn't respond to changes in staged files

I received the error below when trying to commit changes in a project that used vogue. Upon responding to the first issue (line 30, redundant import) by deleting the offending line, I re-staged it and attempted to commit. The same error was returned once again.
Is this a problem with ghc-mod itself, or the way that git-vogue uses it?
cabal succeeded with:
Checking project in: .
  [+] Checked cabal file

ghc-mod failed with:
Checking project in: .
  [-] examples/todo/ToDo.hs:30:1:Warning: The import of ‘Servant.Common.Text’ is redundant  except perhaps to import instances from ‘Servant.Common.Text’To import instances alone, use: import Servant.Common.Text()
      examples/todo/ToDo.hs:214:1:Warning: Top-level binding with no type signature: jsGL :: AjaxReq
      examples/todo/ToDo.hs:214:1:Warning: Top-level binding with no type signature: jsAI :: AjaxReq
      examples/todo/ToDo.hs:214:1:Warning: Top-level binding with no type signature: jsGI :: AjaxReq
      examples/todo/ToDo.hs:214:1:Warning: Top-level binding with no type signature: jsUI :: AjaxReq
      examples/todo/ToDo.hs:214:1:Warning: Top-level binding with no type signature: jsDI :: AjaxReq
      examples/todo/ToDo.hs:230:9:Warning: This binding for ‘mod’ shadows the existing binding  imported from ‘Prelude’ at examples/todo/ToDo.hs:8:8-11  (and originally defined in ‘GHC.Real’)
      examples/todo/ToDo.hs:39:10:Warning: Orphan instance: instance ToJSON UUID
      examples/todo/ToDo.hs:42:10:Warning: Orphan instance: instance FromJSON UUID
      examples/todo/ToDo.hs:46:10:Warning: Orphan instance: instance FromText UUID
      examples/todo/ToDo.hs:49:10:Warning: Orphan instance: instance ToText UUID

hlint succeeded with:
Checking project in: .
  [+] Checked 3 file(s)

stylish succeeded with:
  [+] Checked 3 file(s)

There's no magic happening. If git-vogue says it's broken, ghc-mod standalone should say the same thing.

Can we have a way to reproduce this?


replace screenshot with code block

replacing screenshot with code block to improve accessibility and ux.
also removed image file

thanks for the approval @joellabes ! i think i understand it a bit more 🙏


Fix build warnings in unit tests

Fix build warnings in FancyZones unit tests

Fix available in 0.15.0 https://github.com/microsoft/PowerToys/releases


Getting the total duration of the album

Hi, great work. I am finding it really useful.
Is there an easy way to get the total duration for an album without having to add up all the individual track durations?

Hey, sorry for the super late reply.
I've added this in dbc858c896c3fc89b6d534951c0e7c7c54cec28a.


Bump isort in pre-commit

Codecov Report

Merging #101 (021dec2) into main (a7ab9be) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #101   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           15        15
  Lines          816       816
=========================================
  Hits           816       816

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7ab9be...021dec2. Read the comment docs.


docker installation recipe returning undefined method `trace' for nil:NilClass

Description
Encountered an error running a recipe to install docker.
Running handlers: [2021-12-14T16:38:17+00:00] ERROR: Running exception handlers
Running handlers complete [2021-12-14T16:38:17+00:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 35 seconds
[2021-12-14T16:38:17+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2021-12-14T16:38:17+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2021-12-14T16:38:17+00:00] FATAL: NoMethodError: yum_package[docker] (testcookbook::docker line 17) had an error: NoMethodError: undefined method trace' for nil:NilClass`
Upon further investigation, /opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb has:
      def initialize(new_resource, run_context)
        super
        @candidate_version = nil
      end
I assume super invoking initialize() in /opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider.rb as shown below. However, it does NOT seem to. Hence, the logger variable is NEVER instantiated.
    def initialize(new_resource, run_context)
      @new_resource = new_resource
      @action = action
      @current_resource = nil
      @run_context = run_context
      @converge_actions = nil
      @logger = if run_context
                  run_context.logger.with_child({ resource: new_resource.name, cookbook: cookbook_name, recipe: recipe_name })
                else
                  Chef::Log.with_child({ resource: new_resource.name, cookbook: cookbook_name, recipe: recipe_name })
                end
      @recipe_name = nil
      @cookbook_name = nil
      self.class.include_resource_dsl_module(new_resource)
    end
Chef Version
14.10.9 (server is 12.19.26)
Platform Version
RHEL 7
Replication Case
test.rb
package 'docker' do   action [ :install, :upgrade ] end
Client Output
chef-client -o testcookbook::test  -l info
Starting Chef Client, version 14.10.9
[2021-12-14T17:58:13+00:00] INFO: *** Chef 14.10.9 ***
[2021-12-14T17:58:13+00:00] INFO: Platform: x86_64-linux
[2021-12-14T17:58:13+00:00] INFO: Chef-client pid: 9585
[2021-12-14T17:58:15+00:00] WARN: Run List override has been provided.
[2021-12-14T17:58:15+00:00] WARN: Original Run List: []
[2021-12-14T17:58:15+00:00] WARN: Overridden Run List: [recipe[testcookbook::test]]
[2021-12-14T17:58:15+00:00] INFO: Run List is [recipe[testcookbook::test]]
[2021-12-14T17:58:15+00:00] INFO: Run List expands to [testcookbook::test]
[2021-12-14T17:58:15+00:00] INFO: Starting Chef Run for testcookbook08
[2021-12-14T17:58:15+00:00] INFO: Running start handlers
[2021-12-14T17:58:15+00:00] INFO: Start handlers complete.
[2021-12-14T17:58:15+00:00] INFO: Error while reporting run start to Data Collector. URL:  https://<IP_ADDRESS>/organizations/test/data-collector Exception: 404 -- 404 "Not Found"  (This is normal if you do not have Chef Automate)
resolving cookbooks for run list: ["testcookbook::test"]
[2021-12-14T17:58:15+00:00] INFO: Loading cookbooks<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>[2021-12-14T17:58:15+00:00] INFO: Skipping removal of obsoleted cookbooks from the cache
Synchronizing Cookbooks:
  - testcookbook (0.0.1)
  - bds (0.2.50)
  - docker (4.9.2)
  - sysctl (0.0.0)
  - commonUtils (11.3.2)
  - line (0.5.1)
  - test_platform (0.1.0)
  - cert_and_key (0.1.0)
Installing Cookbook Gems:
[2021-12-14T17:58:37+00:00] INFO: Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.
Fetching gem metadata from https://www.rubygems.org/..........
Resolving dependencies...
Using bundler 1.16.1
Using excon 0.89.0
Using multi_json 1.15.0
Using docker-api 1.34.2
Bundle complete! 1 Gemfile dependency, 4 gems now installed.
Use bundle info [gemname] to see where a bundled gem is installed.
Compiling Cookbooks...
Converging 1 resources
Recipe: testcookbook::test
  * yum_package[docker] action install[2021-12-14T17:58:37+00:00] INFO: Processing yum_package[docker] action install (testcookbook::test line 15)
    NoMethodError
    ---------------------
   undefined methodtrace' for nil:NilClass`
    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/testcookbook/recipes/test.rb
     15: package 'docker' do
     16:   action [ :install, :upgrade ]
     17: end
     18:
    Compiled Resource:
    ------------------
   # Declared in /var/chef/cache/cookbooks/testcookbook/recipes/test.rb:15:infrom_file'`
    yum_package("docker") do
      package_name "docker"
      action [:install, :upgrade]
      default_guard_interpreter :default
      declared_type :package
      cookbook_name "testcookbook"
      recipe_name "test"
    end
    System Info:
    ------------
    chef_version=14.10.9
    platform=redhat
    platform_version=7.9
    ruby=ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
    program_name=/bin/chef-client
    executable=/opt/chef/bin/chef-client
[2021-12-14T17:58:39+00:00] INFO: Running queued delayed notifications before re-raising exception
Running handlers:
[2021-12-14T17:58:39+00:00] ERROR: Running exception handlers
Running handlers complete
[2021-12-14T17:58:39+00:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 26 seconds
[2021-12-14T17:58:39+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2021-12-14T17:58:39+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2021-12-14T17:58:39+00:00] FATAL: NoMethodError: yum_package[docker] (testcookbook::test line 15) had an error: NoMethodError: undefined method trace' for nil:NilClass`

Stacktrace
NoMethodError: yum_package[docker] (testcookbook::test line 15) had an error: NoMethodError: undefined method 'trace' for nil:NilClass
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:523:in block in target_version_array'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:595:in 'block in each_package'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:591:in 'each'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:591:in 'each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:591:in 'each_package'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:483:in 'target_version_array'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:86:in 'block in <class:Package>' (eval):2:in 'block in action_install'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider.rb:236:in 'instance_eval'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider.rb:236:in 'compile_and_converge_action' (eval):2:in 'action_install'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider.rb:182:in 'run_action'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource.rb:578:in 'run_action'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/runner.rb:70:in 'run_action'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/runner.rb:98:in 'block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/runner.rb:98:in 'each'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/runner.rb:98:in 'block in converge'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource_collection/resource_list.rb:94:in 'block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource_collection/stepable_iterator.rb:114:in 'call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource_collection/stepable_iterator.rb:85:in 'step'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource_collection/stepable_iterator.rb:103:in 'iterate'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource_collection/stepable_iterator.rb:55:in 'each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/resource_collection/resource_list.rb:92:in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/runner.rb:97:in 'converge'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/client.rb:720:in 'block in converge'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/client.rb:715:in 'catch'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/client.rb:715:in 'converge'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/client.rb:754:in 'converge_and_save'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/client.rb:286:in 'run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/application.rb:303:in 'run_with_graceful_exit_option'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/application.rb:279:in 'block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/local_mode.rb:44:in 'with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/application.rb:261:in 'run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/application/client.rb:444:in 'run_application'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/application.rb:66:in 'run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/bin/chef-client:25:in '<top (required)>' /bin/chef-client:74:in 'load'
/bin/chef-client:74:in '<main>'
>>>> Caused by NoMethodError: undefined method 'trace' for nil:NilClass
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.10.9/lib/chef/provider/package.rb:523:in 'block in target_version_array'

It looks like you're using Chef 14 which has been EOL for several years now. First step should be to update to a supported version of Chef Infra Client to confirm this is still an issue. We've entirely rewritten how yum packages are handled in more recent version of Chef so there's a solid chance any issue with yum packages in Chef has been resolved.


Add filename to Content-Disposition header (for Renditions component)

The Details - Renditions component uses the HTML5 download= attribute to specify the rendition download filename. This does not work in all browsers (ie FF).
Investigate setting the Content-Disposition header on these requests in AEM (which all browsers respect).
By way of @justinedelson

This is resolved in ASC 2.0.0+


Update NPM versions

Currently the version released on NPM is 0.2.0 and on Github repo is 0.3.1.

fixed, thank you


Investigate WebAssembly.instantiateStreaming

Currently using WebAssembly.instantiate, but WebAssembly.instantiateStreaming might be a better option.

No need to do response.arrayBuffer() if we use WebAssembly.instantiateStreaming.
async function loadWasm() {
  const wasm = await WebAssembly.instantiateStreaming(
    await fetch("algovivo.wasm")
  );
  return wasm.instance;
}

#14


Border Clipping

What does the pull request do?

What is the current behavior?

What is the updated/expected behavior with this PR?

How was the solution implemented (if it's not obvious)?

Checklist

[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Avaloniaui.net with user documentation

Breaking changes

Fixed issues

Is a geometry clip really that slow? Is there some data that justifies all of this specialization? Also why don't we use a geometry implementation that wraps the rounded rect drawing call?
This could be entirely written in the platform code.

Using geometry clip would conflict with the existing Clip property, so we can't reuse it.
Using specialized SKRRect overload is faster than SKPath. That's exactly why Skia has specialized clipping for SRRect.

Can't we introduce a new property public IGeometryImpl VisualClip { get; protected set; }?
Clip has priority if it is defined.
By default, this returns just a lightweight RectangleImpl that uses optimized code path

This might be a good time to align to WPF instead of UWP. UWP unnecessarily sacrificed some power here by not allowing rectangular clipping.
UIElement.Clip
Both WPF and UWP have UIElement.Clip properties. However, WPF can take any Geometry allowing for non-rectangular clipping. UWP can only use a RectangleGeometry for clipping.

WPF: public Geometry UIElement.Clip
UWP: public RectangleGeometry UIElement.Clip

UIElement.ClipToBounds
In WPF it's possible to clip child contents to the parents bounds by setting ClipToBounds to True. UWP doesn't have this property at all. The work-around is to use UIElement.Clip which can only do rectangular clipping.

@robloo we already have the clip property for all elements i think. idk about the cliptobounds, iirc we have that property but it doesnt work...

@jmacato Ok, apologies for misunderstanding. I see that you are already following WPF which is great! I also see this is related to rendering specifically so I clearly need to read things through more before commenting.
http://reference.avaloniaui.net/api/Avalonia.Controls/Control/#Properties

The difference is that we have the Clip property on Visual rather than on StyledElement(UIElement)


Bugs about headers and the new namespace switching feature

Hello, I have tested the new version and it is much better, but I found some new problems when making picker:

When I modify the name or save or seve as the title, the modified version is always The first title,My selected title has not been modified， I think this is a bug;
When I try the new namespace switching function, there will be a pop-up window for me to pop up the window again to let me modify the namespace, which should be a place to be improved

I'm not sure, i can't reproduce on my side but i fixed something like this in previous version then i think it is ok now.
This is now fixed in new commit :)

Thanks for your precious time testing our tool !


Tabs in File Explorer

I would like to see Tabs in File Explorer so that we can't get lost in the crap.
The tabs can be switched with a mouse-click and a shortcut that is used in Browsers.
i.e., "Ctrl+1" can be used to switch to the first tab while "Ctrl+T" would be used to open a new tab.
On the New Tab, there would be Quick Access or PC whatever the settings are set.
Also, the option will be there on the Folder Dropdown Menu like "Open this Folder in a New Tab."
Plz implement this option in the future Updates.
Here is a image:

👍👍👍

Check out https://thewincentral.com/microsoft-planning-to-bring-windows-10-sets-feature-back-from-grave/.
@crutkas Is this something we can promote in a way, considering the amount of issues we've already seen with this request?

Great feedback, the request is tracked in issue #10


pypi.org: HTTP access no longer an option

It used to be the case that http://pypi.python.org/simple was accessible without HTTPS. This was important in legacy environments that don't support the certificates offered:
curl -L http://pypi.python.org/simple
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

pip has a --trusted-host option to support this use case, and has the appropriate caveats in the docs.
This doesn't seem to be accessible any more for pypi.org. Could this please be restored?

Based on the error, you are receiving a redirect from HTTP to HTTPS. This has been the behavior from pypi.python.org for as long as I can remember.
$ curl -I http://pypi.python.org/simple/
HTTP/1.1 301 Moved Permanently
Server: Varnish
Retry-After: 0
Location: https://pypi.org/simple/
...

The root of the issue you have opened is that your client is unable to negotiate a TLSv1.2 connection to PyPI. Until April 10th, 2018 this would have succeeded.
PyPI is no longer accessible to clients which do not support TLSv1.2 due to our CDN provider sunsetting support. We cannot reenable access for outdated TLS protocols.
If you provide more information on your environment someone may be able to assist you in upgrading to support modern TLS protocols.

@ewdurbin it was previously possible to access /simple without HTTPS as recently as February 2018. I have in my bash logs:
 7485  01-02-18 10:52  pip install --trusted-host pypi.python.org -i http://pypi.python.org $pkg

@Wilfred You were not communicating with the index over HTTP, in the request log you have supplied above, pip would have received a HTTP 301 with Location: https://pypi.python.org/simple and proceeded to follow the redirect into HTTPS.
The root issue here is that your client does not support TLSv1.2, and PyPI only supports TLSv1.2 or higher.
Again. If you are able to provide more information about your environment (OS version, TLS implementation and version, Python version, pip version) someone may be able to help you.


XOR LINKED LIST IMPLEMENTATION IN C

🚀 Feature
XOR LINKED LIST IMPLEMENTATION IN C
Have you read the Contributing Guidelines on Pull Requests?
Yes.
Motivation
(Please outline the motivation for the proposal.)
Pitch
(Please explain why this feature should be implemented and how it would be used.)

/assign


Deploying Upgradable Contracts Failed half way due to rising gas prices

Hi there, our contract deployment failed on mainnet half way through the process due to rising gas prices. Any idea if I can resume where it left off when the prices lower again / are more stable?

Hi @miketeix,
I am sorry to hear that you are being impacted by network congestion.
When an upgradeable project is first deployed, three contracts are deployed.

The logic contract
The ProxyAdmin
The proxy contract pointing to the logic contract along with calling the initialization logic.

Which transactions have been successful?
You can manually modify mainnet.json in the .openzeppelin directory to record what contracts have been deployed and then rerun the deploy command.
I recommend putting your Configuration Files in Version Control before doing any modifications to your <network_name>.json file.
Please be aware that the network is still congested (https://www.ethgasstation.info/) so you will need to adjust your gas prices.

Thanks a lot that was very helpful.

Indeed, thanks for the reply!

@miketeix @loki-sama Can you please clarify if your question was about the OpenZeppelin CLI or about the OpenZeppelin Upgrades Plugins?
I think @abcoathup's reply was about the CLI.
As for the plugins, you can rerun an operation and it will resume where it left off.


Is it possible to extract uniform buffer default value?

I use Microsoft directx shader compiler to generate spirv code from hlsl and process it using spirv-cross library.
Inside hlsl code I define a constant buffer:
cbuffer XX
{
   float x;
   float2 param[2]={float2(0,1), float2(2,3)};
}

Is it possible to obtain the default values for 'param'?
This can be done in d3d reflection api (when compiling to dxil byte code instead of spirv), but I haven't seen an option to do it in spirv cross.
I'm using the spirv c shared api.

I don't think this can be expressed in SPIR-V to begin with, so no.

Thank you for your reply.
That's unfortunate, it means I would have to compile the shaders 2 times, first for hlsl to extract values with d3d reflection, and then to spirv to be able to convert to glsl.
If possible I'd like to make this a feature request, to spirv and spirv-cross.
Do you know a good place where I could report this feature request for spirv?
Thank you very much

I think https://github.com/KhronosGroup/SPIRV-Headers is the place for SPIR-V spec related issues.

Many thanks!
Just posted a feature request on the link that you've mentioned.

Nothing to do here for now.


Fix typos in resumable.mdx file.

What is it?

[ ] Feature / enhancement
[ ] Bug
[x] Docs / tests

Description
Fixed typos in resumable.mdx file.
Checklist:

[x] My code follows the developer guidelines of this project
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[ ] Added new tests to cover the fix / functionality

thanks!


Improve tests

Changed log

Since the Travis CI about default dist settings are changed, defining the matrix PHP versions and different dist for tests.
The PSR-0 autoloading is deprecated and using the PSR-4 autoloading is instead.

According to the Travis CI build board, I think this repository should be active and the PR will do tests via Travis CI building hook.

Thanks!


Default values for p-value and FDR

Description of feature
Hi everyone,
I am trying to find the default values for p-value and FDR when the pipeline does peak calling with MACS2. In the latest version, they are both set to "null", but I think there should be also somewhere reported default values. Anyone has any idea about this?
Thank you in advance!

They don't have any default value, if either macs_fdr or macs_pvalue are not provided by the user then these arguments are not passed to the macs2 command, see here.

I will close this issue now for lack of activity, if you have any further questions please feel free to re-open it.


Trigger staging bundle patch release for v0.19.8:

Issue #, if available:
Description of changes:
Trigger staging bundle patch release for v0.19.8
Testing (if applicable):
Documentation added/planned (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

/lgtm
/approve


Disable builds of packages not applicable on s390x

Checklist before submitting

[x] Did you read the contributor guide?
[ ] Did you update any affected documentation?
[ ] Did you write any tests to validate this change?

Description
Disabled builds of packages that we don't build on s390x architecture, this will enable to use opence-env.yaml for s390x builds too.
Review process to land

All tests and other checks must succeed.
At least one maintainer must review and approve.
If any  maintainer requests changes, they must be addressed.

@ravigumm FYI.


Fix logger crash on app exit

Fixes https://github.com/iTwin/imodel-native/issues/706

extract-api is probably switching the order of things when run locally vs CI...clean, build, extract-api should fix. EDIT: uh except you don't have any TypeScript changes...

yes, this one's interesting - I contacted @DanRod1999 regarding this

looking at some other runs, to see if they also fail. Don't really understand what's going on, but I would assume that if this problem keeps happening master branch in itwinjs-core should also be broken, but based on its run history it seems fine. Might just have to wait for the outcome of this current run.

/azp run imodel-native

/azp run imodel-native

@DanRod1999 this is still failing

/azp run imodel-native


Roll engine 02e6a130e7dd..2d2cfc0884ff (3 commits)

https://github.com/flutter/engine/compare/02e6a130e7dd...2d2cfc0884ff
git log 02e6a130e7dd23cd70ea0b2d947f3e1a0269190b..2d2cfc0884ff58e3c4379adea1f5356893d475d3 --no-merges --oneline
2d2cfc088 Expose a hasRenderedFirstFrame() method in FlutterView (#34275). (flutter/engine#9285)
804011742 Fix TextInputPlugin NPE caused by PlatformViewsController ref in new embedding (#34283). (flutter/engine#9283)
c7b25f1c4 Roll src/third_party/skia df586b7d568d..f39124b0764d (11 commits) (flutter/engine#9284)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

Auto-roller completed checks. About to merge.


MPO implementation missing

Hi, are you guys planning to upload how you implement MPO in this task to the repo?

Hello,
We will not be uploading any implementations of the agent, if you are interested in deepmind implementation of agents please visit https://github.com/deepmind/acme/ and for the agent implementation of MPO please see https://github.com/deepmind/acme/tree/master/acme/agents/tf/mpo.


[MPLUGIN-397] Upgrade Parent to 36

fix javadocs
remove l10n-maven-plugin from reports
use plugins version from parent

Ha, dependabot makes this much easier. It brainlessly changes just one line. :-D


'shared' is unavailable in application extensions for iOS

I'm trying to implement BottomSheet with Xcode 13 beta 3 and iOS 15 beta 3 but I get this compiler error:
'shared' is unavailable in application extensions for iOS: Use view controller based solutions where appropriate instead.
Perhaps this has changed in a recent iOS 15 beta?

What is odd is that I only get this error when I import BottomSheet as a package. If I download the source and use it directly, it works.

Here's a sample project to reproduce the issue (Xcode 13 beta 3 13A5192i):
BottomTest.zip

I also have this issue on the same Xcode version.

Related to https://twitter.com/twannl/status/1415643862761152513

I came across that too. If you inject the UIApplication into the class it works. I can create a PR for this.

This issue should now be fixed in release 0.2


Feature request - Display servings and/or add servings calculation

First of all, this is a pretty neat project!
I'd like to suggest adding support for calculating different amounts of servings or at least showing the original amount of servings that the recipe will produce. This would improve usability quite a bit imo :)

Thanks!
I agree, and at least the servings are already available in the data, so this could be added easily. For calculating a different amount, I am not quite sure as it may introduce errors for some ingredients, but we can give it a try :)
It'll take me a few weeks until I can look at it, but feel free to make a contribution if you want to get yourself involved and see this feature earlier.

I added the number of servings with b8c0390f0f65c9e18e8e9763c88971a220c87b89. If you still want to see the option to calculate the amount of ingredients for a different number of servings, please open a new issue on Codeberg 🙂


RHIROS-457 Fix system counts of is_configured & systems api and refactor

@apuntamb @kgaikwad  I've tested it locally. For a system with multiple PP, we're getting single count for that. Also, the count is now showing number of systems with PP associated with a particular account.


ANC Progress Report - Lab Section Change [3h]

Issue
After feedback from the team, we are altering the lab section of the ANC progress report. The lab section will have a button that will take the user to a separate "pop out" screen that shows the lab results overview (how we have it currently just in a new screen alone). From there, the user can click on the blue arrow button to access the history of a particular lab (as it works now).
Here  is a quick video overview.
Figma files are here.

Main Progress Report Section
Lab Results Overview
Particular Lab Overview

To Do:

Style per design

// @adamhstewart FYI

@LudmilaGoichman
All points fixed. Please test on Nutrition env.

@jeanette-pelizzon

Nutrition env
tablet

Questions/Found Issues:

Photos still appear in the middle of the report instead of at the bottom as they do in the image in the issue description. It is correct?

in the app (nurse)
in the app (CHW)
in the design

The CHW Activity section is not displayed in the report as it 's displayed in the design. Should this be fixed in the current issue?

CHW report (all activities completed and synced)
Nurse report in the first encounter after CHW encounter
Design

Checked and Approved:

The lab section has the "See Lab Results" button that takes the user to a separate "pop out" screen that shows the lab results overview

Nurse
CHW

The separate "pop out" screen that shows the lab results overview after clicking the "See Lab Results" button:

Nurse:

CHW:

In this, separate "pop out" screen, the user can click on the blue arrow button to access the history of a particular lab:

the Back arrow on the Lab Results and Lab History screens works as expected - returns to the previous screen

@LudmilaGoichman

Photos still appear in the middle of the report instead of at the bottom as they do in the image in the issue description. It is correct?

This is part of #91, and not deployed yet.

The CHW Activity section is not displayed in the report as it 's displayed in the design. Should this be fixed in the current issue?

@Diama1 is working on it, and it's still in progress.


Clear out pact directory during setup?

Usually I end up wrapping the test script with something that clears out the pacts folder - it would be great if we could do this automatically.

that sounds like a good idea, we could use something like rimraf.  (https://github.com/isaacs/rimraf)
do you store each pact in a pacts folder, as a root file, or in subfolders? i store consumer/provider pairs in pacts folder, non nested
If we delete the entire folder, we would need to consider the use of multiple test scripts running in parallel, in a single repo, as we couldn't delete the whole folder, as we may lose pacts generated by other tests.

Thinking about this, as we know the users pact directory, we could add a flag to say clearPactDirBeforeTest?

You have to do this at test setup time, which is a different time to the time all the current jest-pact code runs (because you can have pactWith in multiple spec files).
You can do it with a global setup module exported by jest-pact - see here https://jestjs.io/docs/configuration#globalsetup-string . We could do jest-pact/clearPactFolder or similar.
However, to implement it, you need to know what the test directory is. The setup is currently passed to pactWith, which would not be readable during the setup time.
The plan I was loosely thinking was to implement pactrc config, which would obviate the need for the "sensible defaults" used in jest-pact (because we could implement / move that implementation there instead).

Hmmm, all very good points. Been a while since I've dived into Jest's innard, but there were some interesting things that came out of the pact-watch discussion that might allow us more configurability
I love the idea of a pact config file, and inspired by some of the experiences of CLI tools, so have been experimenting with some projects to scaffold a repo or setup pact in a project.
I think there is so much pain felt from things like env vars (different shells/os's/Makefiles), that reading sensible defaults from a config file, would be a nice feature.
Wonder if we might want a generic json one, and then language specific ones, as I'm sure each language has its own convention for setup files?

Project-wide, we might want a general pact config. I was thinking we'd implement something that works for pact-js, and then see if other projects wanted to pick it up (although I was also thinking that anything language specific would go in its own key in the JSON).

I don't know why I'm saying we. I'm not planning to do any of this work now that I'm no longer a maintainer. Sorry.

It's okay @TimothyJones I know it is the collective we ❤
Thanks for all your many contributions, and still caring about beautiful developer experiences.

What about not clearing the pact directory on setup, but instead write the pact to temporary files, only updating the official pact files when/if the test run is successful? Seems pretty destructive to nuke the pact directory even if the tests fail...or am I missing something?

That just makes the test suite more complicated, because you need to tell pact the merge / replace the file when it's done. Technically the tests can be run in parallel which is why it's necessary to remove it in advance, to ensure no stale/outdated interactions remain there.
There's nothing important about the files so nuking them in advance shouldn't be a problem.
It's probably only a problem if you publish from your dev machine (which is not recommended) because you're unlikely to have stale interactions on CI

Yes, true. It would complicate things with little benefit...

It's probably only a problem if you publish from your dev machine (which is not recommended)

It's also a problem if you commit your pact files (which is also not recommended, although I don't recall it being explicit in the documentation). I've definitely seen installs where this is the case.


How To Access iFrame when name and Id changes every time dynamically

<iframe id="ipp-activity-panel-14207" name="ipp-activity-panel-14207" html5viewid="/bpm/portal/Ext/activityPanel/oid=14207" anchorid="ippActivityPanelAnchor"  ></iframe>
how to access this type of iframe which have dynamic name ??  @Aussiroth

I find it.

frame ipp-activity-panel-14285
I want to use a variable instead of 14285
panel_number = 14285
frame ipp-activity-panel-panel_number
frame ipp-activity-panel-"'panel_number'"
frame ipp-activity-panel-'panel_number'
frame ipp-activity-panel-"panel-number"
frame ipp-activity-panel-"'panel-number'"
frame ipp-activity-panel-{panel-number}

None of the above lines work for me.

Please suggest what should I do to achieve this ??? @Aussiroth

Hi, I believe right now using dynamic variables will not work for the frame step. I will check with Ken and see if we can update the frame step as well to allow for this use case. We had a quick pass through and thought that such a use case with dynamic frame name would be very unlikely, so we left it as is.
@kensoh Hey Ken, looks like this use case happened earlier than expected, do you think you can update this, else I can try to spare some time next week to go through and figure out the frame step.

Hey Alvin, I had a look, the adding of casper.then() in frame_intent() in tagui_parse.php is easy, but the intelligent closing of that function enclosure after a frame ends is difficult to implement while ensuring backward compatibility for other } enclosures.
I'll suggest consider dropping this as not supported, since dynamic frame names was not meant to be supported in the first place. Even if you try to implement, will have to test exhaustively with CK's use cases to validate backward compatibility. Because frame websites are generally being phased out and hard to find test cases.

https://github.com/kelaberetiv/TagUI/issues/313#issuecomment-438928312
please give your advice on this. I am Waiting @Aussiroth


Remove unused adoption job

Remove data-plane adoption job that deploys standalone in a nested vm,
since we have a new job that supercedes it.
As a pull request owner and reviewers, I checked that:

[x] Appropriate testing is done and actually running

recheck

/approve


Support auto-indent/automatically closing elements?

Does atom-typescript support any kind of real-time formatting for .tsx files? In JSX files, I get indentation, new-lines, and and closing tags automatically, but it seems like none of that is working with tsx files.
Admittedly, I can force Atom to use a different grammar for tsx files and all of the automatic features come back, but then you lose any typescript-linting. Is there a way to have both?

and all of the automatic features come back

Which grammar. If we copy paste their settings file https://github.com/TypeStrong/atom-typescript/blob/master/settings/language-tsx.cson we might get the same result :rose:

I've had the best luck with using the grammar from Atom's language-babel package:
https://atom.io/packages/language-babel

Indeed they are different https://github.com/gandm/language-babel/blob/master/settings/language-babel.json vs. https://github.com/TypeStrong/atom-typescript/blob/master/settings/language-tsx.cson
Bit hesitant. Would prefer it was something from the atom team / atom team maintained

FWIW, it's one of the most-downloaded packages out there, so I don't imagine it's going anywhere any time soon. 😉


[WIP] Add gateway and VirtualServices for istio services

See  issue #4822 and issue #5535
pilot pod restart would cause pilot's endpoint IPs to be changed. Therefore, using pilot's endpoint IPs in the remote cluster to access to pilot is not appropriate for a sound/production deployment of istio in multiple cluster environment, and should be avoided. This is a deployment issue, and there are many ways to address the problem depending on the environment and load balancer used. For example, with k8s, using a load balancer type for the pilot service and expose the load balancer IP to the remote cluster will work just fine.
Since a remote sidecar requires access to multiple istio services (pilot, policy, telemetry, stats), each of the service would have to be exposed with a load balancer IP, which results in a few balancer IPs that are needed. Using LB IPs might come with a cost or be restricted by quota in certain environments. Through discussions in the istio-dev alias, it seems to be desirable to use a single LB IP to access all the istio services required. One way to achieve that is to use a istio Gateway, which seems to be favored by the community.
This PR gives the initial attempt to come up with a gateway manifest. Due to a bug, it will not work in 0.8.0 yet. It'll work with master branch due to the fix (PR #6042).
A few caveats:

the ingress gateway should expose the same ports as the istio services that's being proxied by the gateway. This is important so that remote sidecars use the same configuration.
Initially, I just added a couple of pilot's ports (15010 for grpc, 15007 for http-discovery, which doesn't seem to be needed).
some istio services use UDP (such as stats). UDP proxy is not supported yet. Therefore, it may not be a complete solution until UDP proxy is added in the istio gateway resource.
mTLS support is not added
I placed the file in install/kubernetes. whether or not it should be/can be integrated in the helm install needs to be discussed.

The manifest can be applied on the primary cluster as
 istioctl apply -f istio-gateway.yaml
if a load balancer ip is available for the ingress gateway in your environment, then use the load balancer IP as pilot's endpoint IP. Otherwise, if pod-to-pod network exists, the endpoint IP of the ingress gateway may be used for a quick run.
Note that this is a work in progress PR. We need to figure out/discuss what services/ports should be exposed by the Gateway, and have this manifest fully populated. So please comment.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: baodongli
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: ijsnellf
Assign the PR to them by writing /assign @ijsnellf in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@baodongli: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
aa150f686bbc07302c3eb43bc70b09b53daccde2
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

"mTLS support is not added"  - is your plan to add mTLS support before merging?

@john-a-joyce yes, will add mTLS

@costinm will try to add into helm. Just not sure about ordering of helm/tiller rendering the gateway/virtualservice resources. But will give it a try and see.
Please comment on the UDP proxy since stats is using UDP.

@rshriram has this been closed in favour of some other pull request?
From reading https://github.com/istio/istio.github.io/pull/1667 and other discussion, the only way to securely deploy the Istio control plane for multi cluster is through Gateway resources (due to Google ILBs being regional, thus I cannot access Istio system components from a cluster in another region).
I'm keen to get this configuration up and running if so - I'm unsure whether this approach (in this PR) is no longer recommended?


setup_nfs_provisioner not fully automated

Here are a few issues with the this utils/setup_nfs_provisioner.sh script:
a) Does not install unzip with is required.
b) Does not pull a version of yq for ppc64le
c) yq fails to parse deployment.yaml
./setup_nfs_provisioner.sh: line 82: unzip: command not found
./setup_nfs_provisioner.sh: line 84: cd: external-storage-master/nfs-client/: No such file or directory
INFO: generating deployment.yaml for ppc64le...
./setup_nfs_provisioner.sh: line 107: deploy/deployment-ppc64le.yaml: No such file or directory
ERROR: failed to update deployment.yaml -- aborting...

After installing unzip and yq:
[root@rdr-rhop-f5d9a8-mon01-bastion-0 utils]# ./setup_nfs_provisioner.sh  /home/nfsshare  <IP_ADDRESS>
./setup_nfs_provisioner.sh: line 29: [: argument expected
INFO: proceeding with making this node as NFS server...
Last metadata expiration check: 1 day, 1:09:33 ago on Thu 30 Sep 2021 01:19:56 PM UTC.
Package nfs-utils-1:2.3.3-41.el8_4.2.ppc64le is already installed.
Dependencies resolved.
Nothing to do.
Complete!
Warning: ALREADY_ENABLED: nfs
success
Warning: ALREADY_ENABLED: rpc-bind
success
Warning: ALREADY_ENABLED: mountd
success
success
INFO: downloading and extracting external-storage zip...
INFO: generating deployment.yaml for ppc64le...
Error: unknown command "write" for "yq"
Run 'yq --help' for usage.
ERROR: failed to update deployment.yaml -- aborting...

Thanks @manojnkumar for reporting this. The script was first written with yq v3, and it has since released v4 which is incompatible with v3. Time to revisit and make necessary fixes that'd work with yq v4.


Port scanner.cc to C

Resolves #84

Ok, this is getting close, got a handful of failures on the markdown parser.

All passing now.

Hm, CI is still getting some errors.

The inline parser has some errors, but those errors where also on head so not sure what's going on. I'll look into it.
Can you adjust permissions so my changes trigger ci without approval?

LGTM, just needs updates to some config files so bindings don't break: https://github.com/MDeiml/tree-sitter-markdown/search?q=scanner.cc
(Sorry I didn't notice that earlier :sweat_smile:)

Great, thank you.

New release? ;)


Update active-directory-aadconnect-health-agent-install.md

Add PowerShell 4.0 to requirements

@MikeBazMSFT,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@MikeBazMSFT : Thanks for your contribution to the Azure documentation! The author, @karavar, has been notified to review your proposed change.

#sign-off
Thanks Mike


Possible Blaze Client Performance Regression in 0.21.23

Seeing some anecdotal issues with performance regressions between 0.21.22 and 0.21.23. In particular, it appears to be the blaze-client (and possibly not the server) which is having problems. The symptoms are slightly higher mean response times, and much higher tail response times (even the p70s climb, it it's most stark in p95, p99 and p9999s). The regression appears to be on the order of magnitude of 2x in the p99s, and the instability is quite significant.
Working to see if I can get some narrower data on this or even ideally a reproduction.

Can confirm that the performance regression is real. The general outcome is that the p9999 latency is just about doubled. p50 seems mostly unaffected. We're bisecting snapshots and running production performance stress tests between 0.21.22 and 0.21.23 right now to see if we can find the commit that introduced the regression.

Current bisection status: the regression was likely introduced in either #4810 or #4812.

After applying some educated dichotomy between 0.21.22 and 0.21.23 and running performance tests against an actual service, I can confirm that https://github.com/http4s/http4s/pull/4810 introduced the regression.

Additionally, @Baccata was able to confirm that #4815 appears to amplify the regression, bringing the p9999 degradation from 2x to around 3x. This pattern is seen across multiple services which are using http4s-blaze-client, so I don't believe it is in any way usage-dependent.
My recommendation would be that these semantics be made opt-in and configurable. @rossabaker @RafalSumislawski

In absolute terms, how high are the response times? When you say p9999 increased 3x, is it an increase from 10ms to 30ms, or 100ms to 300ms, or something even bigger? Is the system running under 100% CPU load in the stress tests?
#4810 added starting and IO operation (read), and #4815 added waiting for an IO operation (write) to complete on the execution path before the point in which the result of the request is available. I think we could try moving them to background, in the sense that the response would be made available early, and then on a different thread the connection would enter some intermediate state in which it would be prepared to be recycled back into the connection pool.
Also, in many cases we may be able to reduce the delay introduced by the waiting for the write to complete, by skipping a write if the content is an empty array.

In absolute terms, the 3x is from ~35ms to ~100ms. The service is not running under 100% CPU load (I signed off work so can't check for actual numbers but I can tell you that much)

Thanks. I'll see if I can reproduce something similar locally.

I think I finally found something. In most tests I observe comparable latencies with 0.21.22 and 0.21.23. And in these tests the p9999 is basically a garbage collector benchmark. With the right parameters I managed to create a test scenario in which 0.21.23 pushes the GC to the wall leading to p9999=660ms, while in the same scenario 0.21.22 has p9999=57ms. I can't be sure if that's what is happening in your case, but I think it's worth a shot.
Trying to explain this difference I found a temporary memory leak in blaze-client 0.21.23. It leaks ~2.5kB per request, for the duration of idleTimeout. Through this leak 0.21.23 can put additional pressure on the GC leading to increased latency.
I've prepared a patch https://github.com/RafalSumislawski/http4s/commit/74c2fda81456ea10f9c00a86ce2566a838707d0e which prevents the mem-leak and improves latency in the aforementioned test. It's not a proper solution to the problem, but it should be enough to verify if what I found is the same thing that's troubling your application.
@djspiewak / @Baccata Could you try using http4s built from that commit and see if it solves the problem in your case?

@RafalSumislawski thanks, I'll have a go this week !

@RafalSumislawski so we tested your patch this morning on two different services.

On one of the services, stats are similar enough to what we had in 0.21.22 that we'd be comfortable upgrading to a version containing your patch.
The other service has stricter expectations with regards to latency, and tests accordingly. We did definitely notice an improvement compared to 0.21.23, but it's not back to what it was in 0.21.22.

Here are some graphs showing the differences in a gatling simulation :
0.21.22

your patch

@Baccata would you be able to collect some garbage collector statistics from 0.21.22 and the version with the patch? I'm trying to understand if the difference is still caused by GC or if I should look elsewhere. It would be perfect if you could provide GC logs from both runs. But if that's not possible, some simple statistics like:

number of young/old garbage collections
an example (or better an average) young/old GC time
the size of the heap (if you're letting the JVM adjust the size)
and utilisation of the heap (or better utilisation of each heap region)
would also be useful.

By the way which JVM version and which GC are you running?

@RafalSumislawski sorry for the delay ! Here's a few answers :

the service in question is using zulu-openjdk 11, with default GC
the jvm opts are as follows, on a docker container that has 4GB memory reserved,

    -XX:+AlwaysPreTouch
    -XX:MetaspaceSize=256M
   -XX:MinRAMPercentage=50
   -XX:MaxRAMPercentage=80
```

More importantly, we've realised that the particular simulation we've used to produce the graphs in my previous message may be impacted by external noise that would skew the numbers. We managed to get a good run, then a couple bad ones, without changing the setup.

We're gonna run the simulation a few more times later (hopefully) today, see if we get more consistent successes (or failures), and will compare against 0.21.22. If we get consistent successes with 0.21.22, and failures with your patch in place, we'll share some GC/heap stats.

In the meantime, the first service against which I tested your patch definitely demonstrated improvements over 0.21.23. So that's positive 👍

This regression caused some concern at my new job, but I haven't been there long enough to produce any kinds of numbers.  Our RCs are getting old, and there's a lot of momentum at $WORK and in the community to do the final releases.
Would everyone be comfortable doing a round of releases with this patch, including final for 0.22 and 0.23?

And also, thank you both for pursuing and testing this.  I'm delighted to see the progress and collaboration!

Would everyone be comfortable doing a round of releases with this patch, including final for 0.22 and 0.23?

IMO this is the correct approach. 0.21 has a stable release which contains this regression, so 0.22 and 0.23 will certainly be no worse.

Would everyone be comfortable doing a round of releases with this patch ...

I think so

... including final for 0.22 and 0.23?

I'm sorry to say I cannot confidently say "go ahead with finals" at this time, because the perf environment tied to the particular service that exhibits weird behaviour is being used for another investigation at the moment, and we haven't been able to confirm (nor invalidate) whether the weird behaviour in question is tied to http4s at all. We should be able to resume tomorrow.
But since Daniel's already answered, I'm sure he'll happily take the blame if things go south 😄

But since Daniel's already answered, I'm sure he'll happily take the blame if things go south 😄

I mean, that would have happened anyway… :-P

I also feel fairly confident in our ability to patch it binary-compatibly. Let's move forward.

I hope you all do realize that the current "patch" (RafalSumislawski@74c2fda) is more of an experiment designed to verify my hypothesis, rather than a proper fix of the issue. That being said a reserved some time today to work on a fix, so stay tuned.
I have no strong opinion on whether or not to do the stable release.


Keep eventual root $type property in Json indentation

If the message sent to the service bus used Newtonsoft with TypeHandling defined, when indenting the string the $type property is dismissed by ServiceBusExplorer in Message Text components.
As we cannot use TypeHandling (ServiceBusExplorer does not know about user classes) I propose here to search and replace $type keyword during deserialization / serialization process.
Fix tested and approved on Service Bus Queues

:white_check_mark: Build ServiceBusExplorer 4.0.88-PullRequest0110.2.build.198 completed (commit https://github.com/paolosalvatori/ServiceBusExplorer/commit/3a97556fa9 by @sescandell)

Thanks a lot Stephane! @sescandell  Did yoy verify that this change has no effect when the message format is non-JSON (XML, text, binary, etc.)?

@paolosalvatori I just added some simple tests.
Don't know how to add Binary or other ones but, the thing is, I juste changed the JsonSerializerHelper::Indent method that receives a string. Is there any reason to test anything else than XML, JSON and raw text?
Let me know if you want me to add something else.

:white_check_mark: Build ServiceBusExplorer 4.0.88-PullRequest0110.3.build.199 completed (commit https://github.com/paolosalvatori/ServiceBusExplorer/commit/5b688f9c51 by @sescandell)

Ok... I'll merge the update. Thanks Stephane!


chore(cargo): Update dependency

What this PR does 📖

Updates dependency

Which issue(s) this PR fixes 🔨

N/A

Special notes for reviewers 🗒
Its recommended to create a new account with this update and test out photos and banners.
Additional comments 🎤

Wanted to leave some notes, was just a draft but Darius wanted me to check this out and see how it performed. Was going good but I did realize PFP would not load after 60 seconds or restarting the app so Darius is going to take a look into that


Spike: Use raft consensus for networking

Why
We created a new test suite about resilience of our network stack in #1532 (see also #1106, #1436, #1505). With this in place, we can now explore various means to reach our goal of a crash-tolerant network layer.
This fairly old research paper explored various consensus protocols used in blockchain space and reminds us of the correspondence between consensus and broadcasts:

the form of consensus relevant for blockchain is technically known as atomic
broadcast

Furthermore, it listed at least one of these early, permissioned blockchains that achieved crash-tolerance of $t < n/2$ by simply using etcd with its Raft consensus algorithm.
What

Run fault injection tests with a hydra-node network connected through etcd
Create a PR with results of the spike, but do not merge it

How

Create a Hydra.Network.Etcd network component that implements broadcast using the replicated log of etcd

Move authentication into application
Re-use ~--peer~ from command line
Fork ~etcd~ when starting
Message type + sender as key
Use watch and revisions to be notified of messages while offline

Maybe: Compact messages (this will upper bound reliance to faults?) or use leases

After directly identifying a required change in semantics of our NetworkComponentss #1624, I started implementation yesterday and achieved so far:

Initialization of a etcd cluster by re-using existing --peer (and other) command line arguments
Sending and receiving of messages: very basic, hex-encoded, using etcdctl as "client" and by polling a single key
First manual testing and some end-to-end tests / benchmarks working -> fragile (likely because of process calls and polling)

Notes so far:

First step: Starting a etcd in the background of network component.
When making etcd available to hydra-node (repl), I stumbled over cached paths (somewhere in dist-newstyle)
Separated --host and --port is annoying, should just parse a full Host.
etcd has a few command line arguments to get right, this guide was helpful: https://etcd.io/docs/v3.5/op-guide/clustering/
Etcd client libraries are not in the best state in haskell land, resorted to just invoking etcdctl for now (this will be have horrible performance)

Need to marshal binary encoded messages to etcdtcl put through stdin using hex
Can use etcdctl get -w json to get a JSON encoded result with base64 encoded values (of hex encoded bytes)
Should really use a grpc or at least http/json client

Who should be signing and verifying messages? Re-use authentication layer or build it into etcd network component (and re-use keys for transport-level security)?

Decided to re-use withAuthentication and do some plumbing
Expand list of valid senders to include "us" (because we do not do short-cuts anymore)

Stopping the last etcd instance does not work properly. It seems like its not gracefully handling the SIGTERM while reconnecting to other nodes.
Surprisingly, the etcd network component even works if we just poll a single key in a busy loop and re-deliver this message over and over. At least in a manual, interactive test using the hydra-tui.

Continued and got the networked offline mode nodes and 3-node benchmarks to run today:
Writing results to: /tmp/bench-3-nodes.json-701cf5c6814573b8/results.csv
Confirmed txs/Total expected txs: 9000/9000 (100.00 %)
Average confirmation time (ms): 53.090586988
P99: 65.27539939ms
P95: 62.45322745ms
P50: 53.0196145ms
Invalid txs: 0
Benchmark bench-e2e: FINISH

Its likely slow because the current state of the branch is still invoking etcdctl put for sending and parsing stdout of etcdctl watch.
Next steps

Figure out how to leverage / handle unreliable network

Submission while not connected to the majority seems to be just swallowed
How can a minority catch up after reconnecting? From which revision on to watch?

Run fault injection tests against this
Summarize findings

Notes from today are in the logbook: https://github.com/cardano-scaling/hydra/wiki/Logbook-2024-H1#sn-on-using-etcd-for-networking

Persisted some of the original thoughts about Network component properties in #1649.
Also, this updated architecture diagram also mentions the same properties and displays how etcd is embedded in the hydra-node:

Got the etcd-based network now running in our fault injection test suite (locally, CI jobs are annoying still)
Notes from making it work:

Benchmarks using docker containers failed because the etcd is sensitive to inconsistencies between host=/=port and peer command line arguments.

Tested with various package loss percentages. 10, 15, 20, 25 all still seem fine!

With 40 percent package loss I see the bench driver fail: Something went wrong while waiting for all confirmations: at bench/Bench/EndToEnd.hs:482:33 in main:Bench.EndToEnd: waitNext: ConnectionClosed and later time out.

As the inlined call to pumba makes it clear, this is only testing outbound traffic from alice to bob and carol!
nix run github:noonio/pumba/noon/add-flake -- -l debug netem \
  --duration 20m \
  --target <IP_ADDRESS> \
  --target <IP_ADDRESS> \
  loss --percent ${percent} \
  "re2:hydra-node-1" &

This pumba invocation should result in 10 percent package loss to alice: nix run github:noonio/pumba/noon/add-flake -- -l debug netem --duration 20m --target <IP_ADDRESS> loss --percent 10 "re2:hydra-node-"

To run the package drop tests locally, we just need to use the instructions from fault tolerance github action (also updated on this branch). Also persisted here for easy reproducibility.
Fault injection
Build the pumba-compatible docker image:
nix build .#docker-hydra-node-for-netem
./result | docker load

Set up a local devnet like in https://hydra.family/head-protocol/docs/getting-started/, but using the updated local hydra-node from the .#demo dev shell:
cd demo
./prepare-devnet.sh
docker compose up -d cardano-node
sudo chmod a+w devnet/node.socket
export CARDANO_NODE_SOCKET_PATH=devnet/node.socket
./seed-devnet.sh $(which cardano-cli) $(which hydra-node)

Start the pumba compatible hydra nodes:
docker compose -f docker-compose.yaml -f docker-compose-netem.yaml up -d hydra-node-{1,2,3}

Now we can run the workload with:
mkdir -p benchmarks
nix run .#hydra-cluster.components.benchmarks.bench-e2e -- demo \
  --output-directory=benchmarks \
  --scaling-factor=100 \
  --timeout=1000s \
  --testnet-magic 42 \
  --node-socket=devnet/node.socket \
  --hydra-client=localhost:4001 \
  --hydra-client=localhost:4002 \
  --hydra-client=localhost:4003

While this runs, we can start dropping packets from alice with, e.g. 10 percent likelihood:
nix run github:noonio/pumba/noon/add-flake -- -l debug netem \
  --duration 20m \
  --target <IP_ADDRESS> \
  --target <IP_ADDRESS> \
  loss --percent 10 \
  "re2:hydra-node-1"

And/or drop packets to alice with, e.g. 10 percent likelihood:
nix run github:noonio/pumba/noon/add-flake -- -l debug netem \
  --duration 20m |
  --target <IP_ADDRESS> \
  loss --percent 10 \
  "re2:hydra-node-"

Solved the two scenarios:

Broadcast while on a minority cluster -> persist outbound messages and retry
Deliver messages when (re-)connecting to majority cluster -> keep a last known revision and watch starting with it

Kept notes in the logbook and will summarize findings tomorrow: https://github.com/cardano-scaling/hydra/wiki/Logbook-2024-H1#2024-09-26

Summary of Raft-based network using etcd

Seems very robust, but "overkill"?
Requires etcd as runtime dependency
Similar user experience

Full, static topology with listing everone as --peer
Simpler configuration via peer discovery possible

Introspectable network / cluster state

could improve debugging experience

etcd has a few features we have not thought of yet

use TLS to secure peer connections
separate advertised and binding addresses

Unclear yet whether this (mis-)use of etcd would work in the long run

Originally designed for key-value store
Only request and storage size limits: https://etcd.io/docs/v3.5/dev-guide/limit/
Website claims "Benchmarked at 1000s of writes/s"

Approach

Broadcast implemented as put to the etcd cluster works well

Using indidivual keys per peer is not needed, but handy to debug

Need an outbound buffer to handle failure to put

unsuccessful when disconnected / connected to only a minority cluster
this highlights possibility that broadcast can fail -> discuss with researchers about this abstraction!

Relying on the overall revision to deliver on recovery works well

This also de-duplicates any past messages
Maybe not needed as our protocol seems resilient against duplication

Cluster configuration would be less fragile if we know everyones --node-id
Not tested compaction feature, but could use to prune revision history and bound memory/disk usage
PeerConnected and PeerDisconnected would need to change

we might get some information from etcd
more likely though, our connectivity is interesting
distinguish between connected (to majority) or disconnected (only on minority)

Protocol versioning would need to change

could be trivial by using versioned keys or values

Re-used Authenticate network component to provide message signing

Alternative: move message signing / verification into application logic

Performance

Unclear if faster or slower?
Slower when comparing average confirmation time benchmarks with Github action runs:

single node: 4ms -> 40ms
three nodes: 20ms -> 100ms

Faster when comparing average confirmation time benchmarks on my machine:

single node: 26ms -> 26ms
three nodes: 100ms -> 50ms

Definitely some potential as implementation spawns etcdctl processes on every broadcast
Also persisting of last known revision is a synchronous write (could be asynchronous, best-effort)
Is etcdctl ensuring durability of their write-ahead-log?


Comments in global options get marked as duplicate

When there's multiple comments in a global options block, the # gets read as the global option name, and then gets marked as a duplicate. Comments here should take precedence over that rule.

Also, some global options can be repeated, actually, so I'm not sure that's a correct rule to have. See log which can be repeated with different logger names https://caddyserver.com/docs/caddyfile/options

Should I just remove the duplicate global options check?  I wrote it more as a PoC with the language server rather than a proper validation rule.  I can easily fix the comments, but checking if a directive is allowed to be duplicated is more challenging.
Technically the log directive itself cannot be duplicated, but if you specify a unique argument it can.
{
  log test { }
  log test2 { }
}

is valid, while
{
  log test { }
  log test { }
}

should trigger a warning.

I don't think it's a particularly useful check. Caddy will complain when adapting if it matters.
But I am an "expert" and I'm less likely to make that mistake anyways, so my opinion isn't from a perspective of a beginner.

https://github.com/matthewpi/vscode-caddyfile-support/commit/ee91c9af970d6fea298cfd4bcf877413656f1714

Thanks!
Do comments count as comments now (i.e. # gets lexed before it being an "option")? That commit doesn't seem to suggest you adjusted that.

Ah, I just noticed that as well.  I'll see what I can do before cutting a release later.

Do comments count as comments now (i.e. # gets lexed before it being an "option")? That commit doesn't seem to suggest you adjusted that.

Fixed by https://github.com/matthewpi/vscode-caddyfile-support/commit/d2dbe7637142124f3f3f865ee562cf8467c516ca


STATUS_OBJECT_NAME_NOT_FOUND(The object name is not found.)

Any thoughts on why by running the python zzz_exploit.py x.x.x.x becomes like this ?
[] Target OS: Windows 5.1
[+] Found pipe 'netlogon'
[+] Using named pipe: netlogon
Groom packets
attempt controlling next transaction on x86
success controlling one transaction
modify parameter count to 0xffffffff to be able to write backward
leak next transaction
CONNECTION: 0x8171e930
SESSION: 0xe11493f0
FLINK: 0x7bd48
InData: 0x7ae28
MID: 0xa
TRANS1: 0x78b50
TRANS2: 0x7ac90
modify transaction struct for arbitrary read/write
[] make this SMB session to be SYSTEM
[+] current TOKEN addr: 0xe1d8c030
Bad TOKEN_USER_GROUP offsets detected while parsing tokenData!
RestrictedSids: 0xe1d60c30
RestrictedSidCount: 0x1f4
userAndGroupCount: 0x4c
userAndGroupsAddr: 0xe1d8c0b8
Attempting WINXP SP0/SP1 x86 TOKEN_USER_GROUP workaround
userAndGroupCount: 0x3
userAndGroupsAddr: 0xe1d8c0b8
[] overwriting token UserAndGroups
[] have fun with the system smb session!
[-] got exception
CRITICAL:root:SMB SessionError: STATUS_OBJECT_NAME_NOT_FOUND(The object name is not found.)
Done

I am having the same error here.
Any steps forward?
If using eternalblue_exploit7.py instead, I get:
shellcode size: 1013
numGroomConn: 13
Target OS: Windows 5.1
This exploit does not support this target

Hey everyone, sorry for the delayed reply. The reason the  CRITICAL:root:SMB SessionError: STATUS_OBJECT_NAME_NOT_FOUND(The object name is not found.) error comes into play can be a variety of things:

Most commonly - you do not have appropriate access to a given share for SMB client to connect to. Usually the exploit targets C$ and rightly so because you should be admin. It is possible other restrictions might exists...

OR

There is AV of some sort stopping exploitation. Might be as simple as that.

Seeing as this is field tested daily by myself and others, there doesn't seem to be any bug in the exploit code itself. That said, if you would like to provide further data about your target's configuration @nighymighty then I might be able to determine what the root cause is in this case. That will allow me to diagnose if this is a problem with the exploit code that is new, or if it is simply that your target is correctly protected.
I'll leave this issue open for a bit to give time for your feedback, but otherwise I'll close it if it becomes inactive. Thank you!
As for @michele-dedonno observation. This is correct and expected. The other exploits to not support versions older than Windows 7. Please read each exploit code script carefully to see what targets are supported/tested. The zzz_exploit.py script uses the named pipe techniques from the eternal romance/champion/etc... variants and is NOT eternal blue, the rest are eternal blue. As such the zzz_exploit.py script supports almost every version given you have access to a named pipe, where the others attempt to exploit eternal blue on compatible systems (of which there are less). Because of this, windows XP (5.1) is only supported by the zzz_exploit.py script. Hope that helps!


More than a page of menu item

When there is more than a page of menu item i can't tap on the lasts because they are "hidden"

@GZaccaroni This issue is the same as issue #57. There's a fix for this issue in release 0.4.
Don't hesitate to reopen if it's still an issue for you. Thanks!


Any plan to release a C++ version implementation?

Hi, the work is excellent.
Any plan to release a C++ version implementation?

Sorry, we've been too busy lately, even for the next few months. As a result, I'm afraid we can't implement the C++ version at the moment.


fix work with guard 2.13.0

update gem
add notification on build or test
add go-app folder

I can't accept this PR as it is now, it's adding a lot of noise with the code style changes, quoting, etc. Makes it very difficult to review.
Please don't add all that noise at this point and let's just compare the changes to make it work with 2.13