Supporting HW accelerated WebGL content or native apps (OpenGL)

This issue has been originally filed by @talent-less as the issue #5 in the Generation 1 repository accetto/ubuntu-vnc-xfce-firefox. I've moved the complete conversation here, because the Generation 1 repo is approaching its retirement and in will be probably removed in the future. Also the subject fits better this repository.

@talent-less wrote on Apr 16, 2021:
The docker image works great, but it would be even better if it can run HW accelerated WebGL content or native  apps (OpenGL) , like blender, etc.
FYI: https://medium.com/@pigiuz/hw-accelerated-gui-apps-on-docker-7fd424fe813e

@accetto wrote on Apr 25, 2021:
Hello @talent-less ,
I'm working on it and expect that I'll publish some experimental images soon. At the moment it seems like an image with VirtualGL and Blender would be sufficient for testing, would it?
However, I'll not be able to test it to the end, because I currently do not own any computer with Nvidia GPU. Would you agree to help with testing?
If yes, then what would be your prefered scenario and what Nvidia drivers should I pre-install for you in the test images?
Regards, accetto

@talent-less wrote on Apr 26, 2021
@accetto Sure! Blender should be sufficient.
I have a Intel integrated-graphics and a NVIDIA GTX1050. Maybe I can also test it on other machines with more powerful graphics like 2080Ti. For the driver, I would try sudo apt install nvidia-driver-460

@acceto wrote on Apr 26, 2021:
Thanks, that will help. We'll probably need a few iterations, because it's not exactly my playground, but hopefully we'll reach something. :)
Do you maybe know, what exactly the nvidia-headless-460 drivers do? Is it something exclusively for the cloud (maybe) or is it a kind of superset/subset to the nvidia-driver-460 drivers?

I guess nvidia-headless-460 is for Ubuntu without the X server. Maybe I can try both on aws. But I think that shouldn't be very problematic, as the driver is only required on the host machine.
For gl libs in the docker you can maybe refer to
https://github.com/nytimes/rd-blender-docker/blob/master/dist/2.80-gpu-ubuntu18.04/Dockerfile
https://gitlab.com/nvidia/container-images/opengl/-/blob/ubuntu20.04/glvnd/runtime/Dockerfile

@accetto wrote on Apr 27, 2021:
Thanks for the hints. I've already a few alternatives developed that seem to work. However, it would help, if you could describe the first use case, you would like to test. How do you plan to use the container? Just locally or with a cloud? Do you plan to use it with VNC or do you want to share the X11 socket? Would you prefer TurboVNC or would be TigerVNC OK?
BTW: There will be an unexpected delay, because there is an ad-hoc problem with the TigerVNC download hosting (s. this issue). I have to configure a mirror and update all my containers, because it will be gone this weekend.

@talent-less wrote on Apr 28, 2021:
I will probably use it on a cloud platform as I don't have a NVIDIA GPU on my laptop. I think it will be very useful for people who want to perform long-running tasks like blender rendering while remaining a basic level of interactivity with the GUI instead of having the full thing running in headless mode.
It would be great if everything can be streamed via noVNC, just like in your firefox project. Can it be a layer on top of the firefox project? Then I can run both firefox and other OpenGL software using that image as starting point. If that is not possible, connecting with a VNC client via TCP should also be good.
I will not use the domain socket way, and I think it is already done by other containers

@accetto wrote on Apr 28, 2021:
Thanks, let's start with this use case. I plan to prepare a few versions, with Firefox and also with Chromium. They will include noVNC and also TigerVNC, because they go together in my containers. Then I'll include Mesa3D utilities and VirtualGL. If I've understood you correctly, I should not install NVidia drivers inside the containers.
BTW, I always forget to mention it. You've filed the issue by the image accetto/ubuntu-vnc-xfce-firefox, which is a Generation 1 image based on Ubuntu 18.04 LTS. However, I plan to release Generation 3 images, that are based on Ubuntu 20.04 LTS. They are generally slimmer and faster. I'll probably put it all into the project accetto/headless-drawing-g3. Would it be OK? Is there no show stopper for you?

@talent-less wrote on Apr 30, 2021:
Sounds great!
accetto/headless-drawing-g3 should be a better place for software like blender :)

@accetto wrote on May 1, 2021:
@talent-less , FYI
For the case that you've noticed that I've already started to publish Blender images into the repository accetto/ubuntu-vnc-xfce-blender-g3 on Docker Hub. These are not yet the images we've discussed here.
Those Blender images do not include any support for OpenGL/WebGl/VirtualGL yet. You can use them later to compare the performance, for example.
However, because I have no experience with Blender, I would appreciate, if you could test if something is missing there. Generally I try to keep the images as slim as possible, so I try to include only the really required packages at first.
But, for example, I've noticed that Blender uses Python and it seems, that it also installs it itself. However, it's still possible, that something is missing there.
Btw, you can always install additional packages, because sudo is included. Do not forget to execute sudo apt-get update first. The default sudo password is headless.

@accetto wrote on May 3, 2021
I've just published the images with Blender that include Mesa3D utilities and VirtualGL. Please go to the Docker Hub repository accetto/ubuntu-vnc-xfce-blender-g3 and look for the tags starting with 'vgl-'. Note that these images are not mentioned on the Readme page, because they are still experimental.
Unfortunatelly I cannot test it myself, because I currently do not have access to any Nvidia GPUs  and I'm also missing experience with Blender and VirtualGL.
Few additional remarks still:

Mesa3D utilities should support OpenGL and WebGL
Welknown testing apps glxgears (for OpenGL) and es2gears and es3tri (for WebGL) are also included
WebGL can be tested by visiting the WebGL website

I've found that VirtualGL can be used inside a container like:
vglrun -d :1 glxgears
vglrun -d :1 blender

I'm not sure if it's all what is needed, but it seems that the display argument is required. However, I hope that you already know what is needed.
The resources for the images are on GitHub in the experimental branch exp-vgl.
Please let me know if it is working and/or something is still to improve.

@latent-less wrote on May 5, 2021:
@accetto  Hi, I tired sudo docker run --name blender -p 6901:6901  --gpus all --device=/dev/dri/card0:/dev/dri/card0  accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc-novnc-firefox-plus
https://get.webgl.org/ says the browser supports WebGL, and blender can run with or without vglrun -d :1 blender over novnc. However, I didn't feel any speed up with vglrun in the blender GUI.
apt install glmark2 and vglrun -d :1 glmakr2 crashed with
=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     Mesa/X.org
    GL_RENDERER:   llvmpipe (LLVM 11.0.0, 256 bits)
    GL_VERSION:    3.1 Mesa 20.2.6
=======================================================
[build] use-vbo=false:X Error of failed request:  BadMatch (invalid parameter attributes)
  Major opcode of failed request:  131 (MIT-SHM)
  Minor opcode of failed request:  3 (X_ShmPutImage)
  Serial number of failed request:  44
  Current serial number in output stream:  45

glmakr2 without vglrun  works, but it seems to use software rendering instead of the GPU.
glxinfo | grep vendor says
server glx vendor string: SGI
client glx vendor string: Mesa Project and SGI
OpenGL vendor string: Mesa/X.org

vglrun -d :1 glxinfo | grep vendor says
server glx vendor string: VirtualGL
client glx vendor string: VirtualGL
OpenGL vendor string: Mesa/X.org

but glxinfo crashed.
Then I tried sudo docker run --name blender -p 6901:6901  accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc-novnc-firefox-plus without GPU, blender works fine, and the browser still says that I have WebGL.
apt install glmark2  and glamrk2 works .
=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     Mesa/X.org
    GL_RENDERER:   llvmpipe (LLVM 11.0.0, 256 bits)
    GL_VERSION:    3.1 Mesa 20.2.6
=======================================================

It seems that there is a software OpenGL (SGL) somewhere trying to simulate the GPU.
I will try installing nvidia-driver-450-server in the container (same as on the host machine) and see if it helps.

@talent-less wrote on May 5, 2021:
It's a bit strange that even before I install any driver in the container, nvidia-smi is in the /usr/bin and it can detect the graphics card in the container without any problem. I assume the docker plugin mounts it for me. I also tried some Nvidia CUDA programs and they can run without problem.
It's just opengl that still falls back to the software rendering.
Do I need install X on the host and share it into the container with -v /tmp/.X11-unix/X0:/tmp/.X11-unix/X0:rw like in<EMAIL_ADDRESS>

@talent-less wrote on May 5, 2021:
I also tired sudo docker run --name blender  --gpus all --device=/dev/dri/card0:/dev/dri/card0 -p 6901:6901 -v /tmp/.X11-unix/:/tmp/.X11-unix/:rw -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=graphics,utility,compute,video  -d  accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc-novnc-firefox-plus but the docker image exit(0) immediately without producing any logs.

@accetto wrote on May 5, 2021:
Thanks for the feedback.
I'm still not quite sure about the configuration you use. Do you run the container on your lokal host or in a cloud? Do you try to use an NVidia card included in your host computer or do you try to use a GPU on a cloud? I've got an impression, that you've got a host with an Nvidia card and you're trying to share local devices with the container, is that right? And one more important thing - are you on Linux or on Windows?
I'm unable to repeat your scenarios, but this is what I've found when I've been experimenting on my Linux computer. Maybe it'll help.
The scenario is about running on local Linux computer and sharing X11 between the host computer and the container. A few remarks first:

Because we're sharing X11, we have to share also the display, so everything will run on the display :0 (not on :1).
Because we're sharing the display of the host, we should not start the VNC server inside the container, so we have use the starting option --skip-vnc. (However, if the VNC server is not running, then also the noVNC will not be unavailable.)
I've found, that Blender requires also the sound device, if started with vglrun.
We have also allow access to the local X server, of course.

This is what has worked for me during testing. The best approach is to use two separate terminal windows.
Start a new container in the first terminal window:
# allow access to the local X server
xhost +local:$(whoami)

# start the container
docker run -it --rm \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-e DISPLAY=$DISPLAY \
--device /dev/snd:/dev/snd:rw \
--group-add audio \
--device /dev/dri/card0 \
--name devrun \
--hostname devrun \
accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc --skip-vnc

Then in the second terminal window connect to the running container and do what you need. For example:
# connect to the container
docker exec -it devrun bash

# install glmark2
sudo apt-get update
sudo apt-get install -y glmark2

# start glmark2 without vglrun
glmark2

# or with vglrun (should not crash this time)
vglrun glmark

# or blender
blender

# however, I've got crashes with blender with vglrun (maybe som more options are required)
vglrun blender

However, it can also be, that you simply need to install nvidia-docker on your host computer. Or maybe we're still missing something in the image.
Tomorrow I'll check the link you've send above.
Btw, sometimes it helps to start the containers with the --debug option, your could get some hints about the problem.

@talen-less wrote on May 6, 2021:
I was testing on a clean Ubuntu 20.10 installation with latest docker and nvida-docker plugin on a physical machine at home without any virtualization.
I tired sharing the local X into the container with novnc enabled, I guess that's why it crashed.
I don't think sharing local X with novnc disabled is what I would prefer, as  I would like to share the container to a remote user.

@talent-less wrote on May 6, 2021:
@accetto were you able to get opengl working with novnc?
I guess glmark should be showing the correct vendor (Intel, for example) instead of Mesa

@accetto wrote on May 6, 2021:

I guess glmark should be showing the correct vendor (Intel, for example) instead of Mesa

I'm not quite sure about this. I'm afraid that the nvidia-docker would recognize and work only with Nvidia hardware (see the Note to the reader in this article).

I was testing on a clean Ubuntu 20.10 installation with latest docker and nvida-docker plugin on a physical machine at home without any virtualization.

We would need at least one system with Nvidia GPU to be able to test it all. Do you have an Nvidia GPU in your host? Alternatively, do you have an access to a cloud GPU (as descibed in this article)?

I don't think sharing local X with novnc disabled is what I would prefer, as I would like to share the container to a remote user.

I think that it would help if you could draw a diagram describing the configuration you would like to use. Then we could concentrate on that particular scenario. If you do not have a diagramming software and don't want to use an online one, then you can use the container accetto/ubuntu-vnc-xfce-drawio-g3.

@talent-less wrote on May 7, 2021:

We would need at least one system with Nvidia GPU to be able to test it all.

Sure. The Ubuntu machine has a 1050Ti Nvidia GPU.

I also tried some Nvidia CUDA programs and they can run without problem.

And I think I have got nvidia-docker plugin working, because I can run GPU program (CUDA programs) in your docker image. These program doesn't use opengl related libraries though.
I also have access to a cloud with 1080Ti GPUs, but I have to pay for using it. So I would like to test the image first on the local ubuntu machine.
However I would test as if the local machine were in the cloud, i.e. no physical display connected and display output is sent via novnc to my laptop.

@talent-less wrote on May 7, 2021:

I'm afraid that the nvidia-docker would recognize and work only with Nvidia hardware (see the Note to the reader in this article).

Sure, I know.

@accetto were you able to get opengl working with novnc? I guess glmark should be showing the correct vendor (Intel, for example) instead of Mesa

I asked this because I suspect that novnc is overriding the graphics hardware (nvidia in my case and intel in your case ) with software opengl from mesa.

@accetto wrote on May 7, 2021:

I also have access to a cloud with 1080Ti GPUs, but I have to pay for using it. So I would like to test the image first on the local ubuntu machine.

Exactly my view. :)

And I think I have got nvidia-docker plugin working, because I can run GPU program (CUDA programs) in your docker image. These program doesn't use opengl related libraries though.

That's a really good news! What has been needed to get it running? As I've already mentioned, OpenGL/Blender is not exactly my playground, unfortunatelly. And not having any Nvidia hardware at the time makes me kind of one eye blind. :) However, I'll be glad to co-develop a useful image.
Do you have already some ideas what should/could we change in the image? I'll try to get some hints from the articles we've found, but it'll take some time.
Should I include some test apps into the images, like glmark2, so you would not need to install them each time as new?

@talent-less wrote on may 7, 2021:

I am not sure if glvnd should be installed together with Mesa. Mesa seems to ship a software OpenGL and blender/firefox seems to use it instead of the real HW at the moment.

Oh that is out-dated.

In the process of migrating to Mesa 18.0, Canonical has updated their out-of-tree Mir patches for Mesa. They are also now enabling the OpenGL Vendor Neutral Dispatch Library "GLVND" that allows multiple OpenGL drivers to happily co-exist on the same system.

https://phoronix.com/scan.php?page=news_item&px=Ubuntu-18.04-Getting-Mesa-18.0
So maybe install libglvnd0  will make it work automagically.

@accetto  wrote on May 7, 2021:
Thanks, I'll check it. I'll also include glmark2. Let me know, if also some other test apps or utilities would be useful. We can remove them later if the images will become too big.
BTW, should I continue building all the tags I've published? For example, I'm not sure if you use by testing the images with chromium and the ones without noVNC.

@talent-less wrote on May 7, 2021:
I will only use the ones with Firefox and novnc. Maybe you can disable other ones for now and reenable them when the opengl issue is resolved.

@accetto wrote on May 7, 2021:
All the libraries you've listed above I've already included. You can check it by executing the following:
sudo apt-get update && \
apt search libglvnd0 ; \
apt search libgl1 ; \
apt search libglx0 ; \
apt search libegl1 ; \
apt search libgles2

Only the json file is not there because I didn't know about it. You can easily add it yourself and test it again. I'll also try to find out if something else could be missing.
Remark: Not all libraries are explicitely visible in the Dockerfile because many of them are installed as dependencies. In this case they are mostly part of the mesa stuff.

@accetto wrote on May 8, 2021:
I've just published an updated image tag for you and removed the other ones. I can build them any time if you'll need them for testing.
You can download the image by:
docker pull accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc-novnc-firefox-plus
I've added the glmark2 test app and also the json file you've pointed to. However, I've noticed, that there already has been the file 50_mesa.json with exactly the same content. Unfortunatelly adding the file didn't solve the problem of crashing glmark2 if started with vglrun.
The json file points to the module libEGL_nvidia.so.0, which is not installed in the image. I have tried to add it by installing the package libnvidia-gl-440, but it didn't help with the crashing problem. I assume that nvidia-docker and Nvidia hardware are still required.
One more tip for experimenting.
If you want to force the container's VNC to use the display #0, you can do it by overriding the VNC paramater in runtime, as it is described here. You can override also other VNC parameters and simplify you docker run commands. Just be careful to bind a single file only, not a folder. The file content could look something like this, including the empty VNC password to avoid tipping it each time :)
export DISPLAY=:0
export VNC_PW=
# export DISPLAY=:2
# export VNC_COL_DEPTH=32
# export VNC_VIEW_ONLY=true
# export VNC_RESOLUTION=1024x768
# export VNC_PORT=5902
# export NO_VNC_PORT=6902

Then you can use, for example, just vglrun glxgears in the container.
However, using the display #0 still doesn't help with the crashing vglrun glmark2, at least not by me. Using the debug option like vglrun glmark2 -d provides some output, but I still don't see the reason.

@accetto wrote on May 8, 2021:
I've played a little bit with the image accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc-novnc-firefox-plus and I may have something for you.

This is how it looks like if you start glmark2 inside the container without vglrun using just glmark2 command:

If you use vglrun glmark2, then it looks already better:

Notice that there is no -d option used with the vglrun. It means, that glmark2 still runs on the display :0, leaving the display :1 for the VNC/noVNC access and the vglrun passes the GLX commands through.
You can check it if you don't allow access to your local display :0 by leaving out the command xhost +local:$(whoami). Then you will get the following error:

Sharing the audio device is required for Blender. You can test it the same way: vglrun blender.
Please let me know if it worked.

@accetto wrote on May 10, 2021:
Hello @talent-less , does it already work?

@talent-less wrote on May 13, 2021:
@accetto
Sorry for the late reply.
It doesn't seem to work on my side. The container exit immediately after running your command.
I think the local Ubuntu gnome session is doing something strange on the X domain socket file so the container crashed right after docker run.  The container didn't put anything into stdout or stderr. Is there any way to get some logs?
However, I can start the container via SSH from my laptop when the ubuntu gnome session is logged out locally. But there I couldn't run xhost +local:$(whoami) as xhost said "couldn't find display". I can ignore that and start container, but it ended up like in your last screenshot.
Do I need to somehow start the X server first?

@accetto wrote on May 13, 2021:
No, your X server is already running, if you are on Linux and if you already running a GUI desktop session.
Check it like this (on your host, not inside the container):
sudo ls /tmp/.X11*

### you should get something like this
X0

### check also the DISPLAY variable
echo $DISPLAY

### you should get something like this
:0.0

If xhost +local:$(whoami) doesn't work, then try xhost +, which grants access to everybody. Be sure that you run it on your host, not in the container. If it still doesn't work, then it could be a permission problem. Try to use sudo then.

@accetto wrote on May 13, 2021:
Maybe some more tips about what you can do during troubleshooting.
You should probably check the readme file for startup options help. You can also display it by starting a container with the option -h and also --help-usage. You will probably find the options --debug, --verbose, --tail-vnc, --skip-vnc and --skip startup most helpful.
You can bind the dockerstartup directory as an external volume and you'll get the logs directly there. You can make modifications in the startup scripts, e.g. by adding some debug reporting, and  then start the container with the --skip-startup option and then execute the startup script from the second terminal. Something like this:
### start a container in the first terminal skipping the startup skript
docker run -it -P --name mytest <other-options> <image> --skip-startup bash

### connect from the second terminal ...
docker exec -it mytest bash

### ... and execute the startup script manually, possibly passing some startup options
headless@mytest:~$ /dockerstartup/./startup.sh --verbose

@talent-less wrote on May 13, 2021:
@accetto  Thank you so much for the work and hints! I am able to get vglrun glmark2 and vglrun  blender working on a NVIDIA GPU !
======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     NVIDIA Corporation
    GL_RENDERER:   GeForce GTX 1050 Ti/PCIe/SSE2
    GL_VERSION:    4.6.0 NVIDIA 450.119.03
=======================================================

Here is the configuration for NVIDIA GPUs.
sudo docker run -it --rm --gpus all --name blender -p 6901:6901 -P   -v /tmp/.X11-unix/X1:/tmp/.X11-unix/X0:rw   --device /dev/dri/card0   --device /dev/snd:/dev/snd:rw  -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=graphics,utility,compute,video  --group-add audio accetto/ubuntu-vnc-xfce-blender-g3:vgl-vnc-novnc-firefox-plus

I didn't configure libglvnd so I guess it takes the default graphics card from the  X server on the host.
My default DISPLAY was on X1 and it crashed the container previously. -v /tmp/.X11-unix/X1:/tmp/.X11-unix/X0:rw fixs that.
So the last missing piece would be figuring out how to run the container in the cloud.
@accetto  Do you know how to run xserver and xhost from ssh on a headless Ubuntu Server instead of Ubuntu Desktop?

@accetto wrote on May 13, 2021:

I am able to get vglrun glmark2 and vglrun blender working on a NVIDIA GPU

That's a really great news! :+1:

Do you know how to run xserver and xhost from ssh on a headless Ubuntu Server instead of Ubuntu Desktop?

Not really, because I did not need it yet. Basically it should be similar to the desktop case, just without starting the Xfce4 stuff. Actually, the container is a headless server, so it should be possible to use it for testing. Check for example this article. You can always start a container skipping the startup script as described above and modify anything inside the container. The included nano editor does nor require GUI. However, ssh is not included, so you need to install it yourself.
You can save intermediate stages as helper images using docker commit.
You can also try PuTTY, which provides X11 forwarding.

@talent-less wrote on May 14, 2021:
I am a little bit confused now. Do I really need the X server on the host?

VirtualGL redirects an application's OpenGL/GLX commands to a separate X server (that has access to a 3D graphics card), captures the rendered images, and then streams them to the X server that actually handles the application.

On a headless sever, by default I have no X server talking to the  GPU on the host. Can I somehow configure the X server in the container to use the GPU?

@accetto wrote on May 15, 2021:

Do I really need the X server on the host?

I would say so, but I'm actually also not 100% sure at the moment. I would recommend to start drawing some diagrams, to clarify what is running where and what is talking to what and how.
I think, that the use case we've just tested goes like this:
The Blender application is running inside the container and it draws on the display 1. The container has no GPU and it's actually a kind of a remote host. The Nvidia GPU is available only on the local host and it can work only with display 0, as I've undestood. So we need VirtualGL inside the container (= remote host) to pass the graphical commands and the rendered content between the displays 1 and 0. In other words, Blender thinks that it's drawing on the display 1 and the GPU thinks that it's drawing on the display 0. The VirtualGL makes them to believe that they are both right. :)
From that description I would derive, that both hosts (remote and local) need some kind of X server, because they both want to draw on a graphical display. There is no need for GPU on a non-graphical terminals.
Well, maybe I've misunderstood something, but that's actually my point. I would draw some diagrams for the use case you need.
BTW, I would like to move this conversation to the project accetto/headless-drawing-g3, because it actually belongs there and this Generation 1 repository is slowly approaching its retirement. Another alternative would be the GitHub Discussions by the sibling repository accetto/ubuntu-vnc-xfce-g3. What do you think?

Hi, I have been using virtualGL for a while but I the performance was still not ideal compared to directly running on the host. glmark2 suggest a 15% loss on my machine.
Also , the configuration requires x server to be installed and configured (xhost) on the host , which might be a headache for some cloud platforms (especially container-only ones) that do not provide you access to the host system at all.
Luckily, I found https://github.com/ehfd/docker-nvidia-egl-desktop which uses glvnd instead of virtualGL and therefore doesn't require x server on the host.
@accetto Are you interested in continuing improving of the container?

Hi, I have been using virtualGL for a while but I the performance was still not ideal compared to running programs directly on the host. glmark2 suggest a 15% loss on my machine.

This is no surprise, because it's generally true for any kind of virtualization. However, I'm not sure, if you really need to use VirtualGL at all. I saw it always as an option. The example I've provided before was actually only a quick experiment, when I was testing if I can get my Intel GPU reported inside the container. Because I do not have NVidia, I'm not able to test the exact use case you're interested in, unfortunatelly. I also don't have any cloud subscription with GPU resources, so I'm not able to test such use cases either. I've hoped that you'll find out the best way to use it. :)

Also , the configuration requires x server to be installed and configured (xhost) on the host , which might be a headache for some cloud platforms

Actually I'm not sure if you need to share the X Server socket and to use xhost at all. I've provided such an example only because you've said that you want to test the GPU provided by your host. I've supposed that you'll not do the same on the cloud. This is why you wanted noVNC after all, isn't it?
At the most beginning you've provided this link. In that article the author describes two main scenarios - sharing the X11 socket and installing the X server stuff inside the the container. The author prefers the first scenario, but my containers actually implement the second one, which is also more suitable for the cloud, I would assume.
Have you already tried to use my container the same way as the one you've mentioned above?
I haven't tried or analyzed it yet, but on the first look there should be no difference. The author self describes it as

container supporting GLX/Vulkan for NVIDIA GPUs by spawning its own X Server and noVNC WebSocket interface instead of using the host X server.

and this actually exactly the same what my containers do. The glnvd is also available as part of the package libglvnd0, which is already included. Only the noVNC port is 6901 and I the NVidia drivers are not installed yet. But if only those are missing, then it would be no problem. Can you try to use my container exactly the same way? However, install the NVidia drivers first.

Are you interested in continuing improving of the container?

Sure I'm. I'll be glad to make the containers more useful also with applications that could benefit from using GPU, even if I don't have such use cases at the moment. My main use cases are more about encapsulated working environments for experimenting, testing and development. Because of not having an access to any NVidia GPU locally or on the cloud, I would welcome any help by testing such use cases, that I cannot do myself. Also I see Blender as a generally useful application, but I don't have any experience with it.

Thanks for your reply. I tried installing nvidia driver inside the container but that didn't help and glxinfo still showed Mesa instead of my GPU.
I noticed that in https://github.com/ehfd/docker-nvidia-glx-desktop/blob/cec9907cf2ad826aac53946e40bb9226fc4ea5b1/bootstrap.sh#L11, the open source nvidia driver is used instead of the proprietary one. I will have a try today.

I assume that the answer from above wasn't really ment for me, was it? :)

Have you already tried to use my container the same way as the one you've mentioned above?

I was trying to manually perform what docker-nvidia-glx-desktop does in your container, but I failed to get it to work.

I see. :)
I didn't have time yet to test or analyze the other image, but I've actually ment to use the same or similar kind of docker run command line, without X11 socket sharing etc. I'm not sure if it would work right away, of course.
It can be, that there is something important inside the base image nvidia/opengl:1.2-glvnd-devel-ubuntu20.04. I'll try to look at it also. Maybe the best trick would be to use that image as the base. However, I would not be able to really test it at the moment.
As a tip, you can install stuff like NVidia drivers in the run-time and then export the running container as a new image and then use that image for the next experiment and so on. Something like this:
Start a new container docker run --name mycont ...
Install the NVidia drivers inside the container:
### update the apt cache first
sudo apt-get update

### install the NVidia drivers
sudo apt install nvidia-driver-460

Then export the running container as a new image (from outside the container):
docker commit mycont my/image:nvidia

Then you can create new containers from the image my/image:nvidia, install more stuff in the run-time, export important milestones as new images and so on.
It really doesn't help that I'm missing any NVidia hardware. I'm trying to find some quick and affordable solution, but no success yet.

I did some testing and I want to share the results.
I've used the glmark2 benchmark application and an image containing mesa-utils and VirtualGL.
First I've run glmark2 directly on my host and I've got the following result:
=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     Intel
    GL_RENDERER:   Mesa Intel(R) UHD Graphics 620 (KBL GT2)
    GL_VERSION:    4.6 (Compatibility Profile) Mesa 20.2.6
=======================================================
...
=======================================================
                                  glmark2 Score: 3936
=======================================================

Then I've run it inside the container, but using the display #0 of the host. The VNC must not be started in this case.
xhost +local:$(whoami)

docker run -it -P --rm \
    -e DISPLAY=${DISPLAY} \
    --device /dev/dri/card0 \
    -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
    -- name devrun \
    <image-with-mesa-and-virtualgl> --skip-vnc

xhost -local:$(whoami)

From the second terminal:
docker exec -it devrun glmark2

The result has been:
=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     Intel
    GL_RENDERER:   Mesa Intel(R) UHD Graphics 620 (KBL GT2)
    GL_VERSION:    4.6 (Compatibility Profile) Mesa 20.2.6
=======================================================
...
=======================================================
                                  glmark2 Score: 4074
=======================================================

It's interesting that the result was even slightly better than in the first case, but it's probably just a usual deviation.
Then I've tested it inside the container, not using the VisualGL first. Software rendering was used in this case. I've accessed the container with TigerVNC Viewer. The VNC was running, of course.
xhost +local:$(whoami)

docker run -it -P --rm \
    --device /dev/dri/card0 \
    -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
    --name devrun \
    <image-with-mesa-and-virtualgl>

xhost -local:$(whoami)

The result has been significantly worse:
headless@devrun:~$ glmark2
** GLX does not support GLX_EXT_swap_control or GLX_MESA_swap_control!
** Failed to set swap interval. Results may be bounded above by refresh rate.
=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     Mesa/X.org
    GL_RENDERER:   llvmpipe (LLVM 11.0.0, 256 bits)
    GL_VERSION:    3.1 Mesa 20.2.6
=======================================================
...
=======================================================
                                  glmark2 Score: 378
=======================================================

It's just about 10% of the first result directly on the host! I've got the score of just 306 by accessing the container via noVNC. That was another test run, of course. The slightly lower value is expectable, because there is an additional processing by websockify.
At last I've run the test inside the container using vglrun glmark2. The container has been started the same way as the one above. I've got the following result:
headless@devrun:~$ vglrun glmark2
=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     Intel
    GL_RENDERER:   Mesa Intel(R) UHD Graphics 620 (KBL GT2)
    GL_VERSION:    4.6 (Compatibility Profile) Mesa 20.2.6
=======================================================
...
=======================================================
                                  glmark2 Score: 549
=======================================================

It can be seen, that Intel UHD Graphics 620 has been used for rendering and that the result is about 45% better then the previous one. So it seem that VirtaulGL is really working. By accessing the container via noVNC I've got the score of 447.
I would make the following conclusions from these tests:

The best performance can be achieved by sharing the display of the host and not running the VNC server inside the container.
It still makes sense to install 3D applications inside containers. They can be nicely encapsulated and it seems that they could run without a performace penalty if sharing the display with the host.
It seems that the bottleneck is actually the VNC server running inside the container. According the results from above it costs about 90% of the performace. However, it can be different with NVidia hardware and NVIDIA Docker Toolkit installed.
If you still want/need to run the VNC server inside he container, then it's better to use the vglrun command from the VirtualGL toolkit, because the performance gain seems to be about 45%. It can be probably much more with NVidia hardware and drivers.
I would say, that the best approach for the cloud would be to try to come as close to the scenario (1) as possible.

@talent-less FYI
I've included the Mesa3D and VirtualGL support into the master branch and therefore I've removed the experimental branch exp-vgl. I've also re-published the images with Blender and they all include the Mesa3D libraries and the VirtualGL toolkit now. The glvnd is also included, of course. Please use those new images for experimenting.
I've also started a new discussion Supporting OpenGL/WebGL and using HW acceleration.


MRG: DOC: Explain better how baselining is performed

Try to explain in more detail how baseline-correction is actually being performed. I have to admit I regularly have to check the code because I forget. It's embarrassing and inconvenient having to dig through the code. With the help of this PR, the inconvenience shall be reduced, while the embarrassment remains.

thx @hoechenberger


Headers files with TOML content allowed in CI but fail in CLI

Context: https://github.com/netlify/cli/pull/754#issuecomment-603792668
- Do you want to request a feature or report a bug?
bug
- What is the current behavior?
Netlify CI allows _headers files to have TOML content, but the CLI doesn't mirror this behavior.
- If the current behavior is a bug, please provide the steps to reproduce.

Be sure to run the latest netlify-cli locally from source
Clone the Gatsby Netlify CMS starter: https://github.com/netlify-templates/gatsby-starter-netlify-cms
Run netlify dev in the cloned repo

- What is the expected behavior?
The CLI should not fail to parse a headers file that Netlify CI parses successfully.
- Local Environment Information

This issue is occurring on netlify/cli master, unreleased, as of this commit:
https://github.com/netlify/cli/commit/fa86a6179419bf38022f4bcc36643b758d059736

@rybit can you confirm whether we're allowing _headers files to have TOML content intentionally, or loop in someone who can?

I'm pretty sure that we do not parse _header files with any toml parser, however, deploys don't fail when there is a parsing error, we only log the errors and move on. That repository has probably just worked, but never actually injected any header.

I just tested and can confirm that TOML styles rules in _headers files don't work on production. I think it's a good idea to let people know that they have invalid syntax when they run netlify dev. This will help save some frustration about why the rules are not being applied.

New to Gatsby and Netlify, I've started with the recommended https://github.com/netlify-templates/gatsby-starter-netlify-cms starter, installed netlify-cli, ran netlify dev and got the Error: path should come before headers message. Took some time to find the culprit, downgrading to v2.40.0 helped. I think the template should be updated.

I opened a PR in the template repo to fix the syntax https://github.com/netlify-templates/gatsby-starter-netlify-cms/pull/546

Hi there, I still have this issue event updated all my packages to the latest version.
After I saw this error, I saw the fix has been merged, so I thought by updating all packages to the latest version, it could solve this problem , so I ran
npx npm-check-updates -u
but the issue is still here.
please help

Hi @franva, do you mind opening a new issue with a reproduction?


Select available channels for RTV

<select> and <input> sync with each other. If no RTV is provided the default cookie opt-in is used, so a link to change it is provided.

@dvoytenko


Update getting started

Related to https://github.com/http4s/http4s/issues/3260

Also, the test failures are unrelated.  We might need to turn on the CI skip on a few more of these, as this is happening more lately.

So should then I change something or is this Okay? I do not have clear what is going on 😄

It is good. :smile:


Configure Azure Pipelines

This pipeline configuration builds the application and produces a build for the release configuration.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: KevinBechard sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


fix incorrect learning rate warm-up after restarting from ckpt

A simple fix to #181

Could you explain how this works? Where is resume_last_lr_step defined? When is self.last_lr_step ever assigned to anything except -1?

Oh, I forgot the definition of resume_last_lr_step(). It's added in the 69a8a28 commit.

One more thing: this assumes that the resume_from_ckpt is a ZERO checkpoint. You should add a quick check that the resume_from_ckpt argument is a directory and not a file.

The non deepspeed ckpt file is supported now.


iptm and ptm directories

Hi, could you let me know where in the output or tfold-release directory I can find the iptm and ptm scores for my models? I'm trying to present their validity in accuracy and would like to collect them. Thank you so much in advance!

Hi, by default, the algorithm uses AF2 model_1 which does not compute pTM/pAE. (It does compute and output pLDDT though.) You can try switching to model_1_ptm in tfold_patch/tfold_config.py/af_params['model_names'] and see if you get pTM/pAE in the pickled file that AF2 outputs. I haven't tried that.

Okay, thanks! I’ll make sure to try that


crash when trying to convert image manually

./acceleration-service/accelctl  task create anonymous/hub.docker.com/library/nginx
2024-01-15T14:17:52.105500000Z time="2024-01-15T14:17:52.103805662Z" level=info msg="POST /api/v1/conversions 200 9.432762ms 587>5bytes <IP_ADDRESS>" module=api
2024-01-15T14:17:52.105670000Z time="2024-01-15T14:17:52.104344145Z" level=info msg="pulling image anonymous/hub.docker.com/library/nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08" module=converter
2024-01-15T14:17:52.139351000Z time="2024-01-15T14:17:52.139156808Z" level=info msg="received webhook request from <IP_ADDRESS>:39324" module=api
2024-01-15T14:17:52.168598000Z time="2024-01-15T14:17:52.168251079Z" level=info msg="POST /api/v1/conversions 200 29.081074ms 587>5bytes <IP_ADDRESS>" module=api
2024-01-15T14:17:52.208599000Z time="2024-01-15T14:17:52.208307144Z" level=info msg="received webhook request from <IP_ADDRESS>:39336" module=api
2024-01-15T14:17:52.245770000Z time="2024-01-15T14:17:52.245372522Z" level=info msg="POST /api/v1/conversions 200 37.047493ms 587>5bytes <IP_ADDRESS>" module=api
2024-01-15T14:17:52.276090000Z time="2024-01-15T14:17:52.275572672Z" level=info msg="received webhook request from <IP_ADDRESS>:39340" module=api
2024-01-15T14:17:52.288114000Z time="2024-01-15T14:17:52.287599582Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:6a977dec78ebd58b02ad596d5b14c68ace7c3989c944a63ea601082399ff215b" mediatype=application/vnd.in-toto+json size=30079
2024-01-15T14:17:52.288294000Z time="2024-01-15T14:17:52.287710912Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:b5c71c53363f4930c0c81a63d5e5e4aa4681e7f3c9b5f24771065f4aef5dbe24" mediatype=application/vnd.in-toto+json size=2552722
2024-01-15T14:17:52.293492000Z time="2024-01-15T14:17:52.292947803Z" level=info msg="POST /api/v1/conversions 200 17.353913ms 587>5bytes <IP_ADDRESS>" module=api
2024-01-15T14:17:52.300763000Z time="2024-01-15T14:17:52.300587106Z" level=info msg="GET /api/v1/conversions 200 1.047785ms 0>3366bytes <IP_ADDRESS>" module=api
2024-01-15T14:17:52.313416000Z time="2024-01-15T14:17:52.313054485Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:b4667fb61a2a6ef3fed1bcdfd9dbffa022ed66416bf918fbb01f276e17ff96f5" mediatype=application/vnd.in-toto+json size=30166
2024-01-15T14:17:52.313651000Z time="2024-01-15T14:17:52.313259913Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:0aee1be3dc624ed7f544d15327ad2e15b3f261028f95ba6e813182a2f90295bb" mediatype=application/vnd.in-toto+json size=2563209
2024-01-15T14:17:52.438790000Z time="2024-01-15T14:17:52.438478099Z" level=warning msg="reference for unknown type: application/vnd.in-toto+json" digest="sha256:f240738a2112072c68bee74bbebb78259b42bdf846b9d9e367e304f2107adde6" mediatype=application/vnd.in-toto+json size=29954
2024-01-15T14:17:52.552566000Z time="2024-01-15T14:17:52.551780936Z" level=info msg="pulled image anonymous/hub.docker.com/library/nginx@sha256:3bd66d9f2dde2fef40a4e8957a06fb42976137eb337e7496cdc3957220ca2c5a , elapse 505.835195ms" module=converter
2024-01-15T14:17:52.552715000Z time="2024-01-15T14:17:52.551823497Z" level=info msg="converting image anonymous/hub.docker.com/library/nginx@sha256:3bd66d9f2dde2fef40a4e8957a06fb42976137eb337e7496cdc3957220ca2c5a" module=converter
2024-01-15T14:17:52.565620000Z time="2024-01-15T14:17:52.565151773Z" level=info msg="pulled image anonymous/hub.docker.com/library/nginx@sha256:ef7877a6114bccf081ac21439438b699779fbe111940fb1352cabee5b5e92b38 , elapse 483.398754ms" module=converter
2024-01-15T14:17:52.565767000Z time="2024-01-15T14:17:52.565203265Z" level=info msg="converting image anonymous/hub.docker.com/library/nginx@sha256:ef7877a6114bccf081ac21439438b699779fbe111940fb1352cabee5b5e92b38" module=converter
2024-01-15T14:17:52.621251000Z time="2024-01-15T14:17:52.620945645Z" level=info msg="pulled image anonymous/hub.docker.com/library/nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08 , elapse 477.061265ms" module=converter
2024-01-15T14:17:52.621424000Z time="2024-01-15T14:17:52.620993021Z" level=info msg="converting image anonymous/hub.docker.com/library/nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08" module=converter
2024-01-15T14:17:52.638219000Z panic: runtime error: invalid memory address or nil pointer dereference
2024-01-15T14:17:52.638558000Z
2024-01-15T14:17:52.638653000Z runtime/debug.Stack()
2024-01-15T14:17:52.638711000Z 	/usr/lib/golang/src/runtime/debug/stack.go:24 +0x7a
2024-01-15T14:17:52.638785000Z golang.org/x/sync/singleflight.newPanicError({0x2044c60, 0x2aadcd0})
2024-01-15T14:17:52.638857000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:44 +0x45
2024-01-15T14:17:52.638943000Z golang.org/x/sync/singleflight.(*Group).doCall.func2.1()
2024-01-15T14:17:52.639099000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:193 +0x5b
2024-01-15T14:17:52.639187000Z panic({0x2044c60, 0x2aadcd0})
2024-01-15T14:17:52.639252000Z 	/usr/lib/golang/src/runtime/panic.go:890 +0x262
2024-01-15T14:17:52.639311000Z github.com/goharbor/acceleration-service/pkg/converter.(*Metric).imageSize(0xc00053b3b0, {0x2313be8, 0xc000657d10}, {0x2318aa0, 0xc000100460}, 0x0, {0x2310cb0, 0x2b7aee0})
2024-01-15T14:17:52.639417000Z 	/root/acceleration-service/pkg/converter/metric.go:61 +0x92
2024-01-15T14:17:52.639515000Z github.com/goharbor/acceleration-service/pkg/converter.(*Metric).SetTargetImageSize(0xc00053b3b0, {0x2313be8, 0xc000657d10}, 0xc000135740, 0x0)
2024-01-15T14:17:52.639570000Z 	/root/acceleration-service/pkg/converter/metric.go:44 +0xd8
2024-01-15T14:17:52.639777000Z github.com/goharbor/acceleration-service/pkg/converter.(*Converter).Convert(0xc000135740, {0x2313be8, 0xc000657d10}, {0xc002724c80, 0x7f}, {0xc00031b0e0, 0x43}, {0x0, 0x0})
2024-01-15T14:17:52.639884000Z 	/root/acceleration-service/pkg/converter/converter.go:163 +0x15aa
2024-01-15T14:17:52.639962000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Convert(0xc0003ad2f0, {0x2313be8, 0xc000657d10}, {0xc00085e580, 0x7f})
2024-01-15T14:17:52.640045000Z 	/root/acceleration-service/pkg/adapter/adapter.go:130 +0x69a
2024-01-15T14:17:52.640119000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2.1.1()
2024-01-15T14:17:52.640227000Z 	/root/acceleration-service/pkg/adapter/adapter.go:164 +0xbc
2024-01-15T14:17:52.640360000Z github.com/goharbor/acceleration-service/pkg/metrics.(*OpWrapper).OpWrap(0xc0001250b0, 0xc00230db88, {0xc000725b50, 0x1, 0x1})
2024-01-15T14:17:52.640423000Z 	/root/acceleration-service/pkg/metrics/metrics.go:82 +0x9c
2024-01-15T14:17:52.640475000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2.1()
2024-01-15T14:17:52.640517000Z 	/root/acceleration-service/pkg/adapter/adapter.go:163 +0x133
2024-01-15T14:17:52.640569000Z golang.org/x/sync/singleflight.(*Group).doCall.func2(0xc000725c97, 0xc000101040, 0xc00230df08)
2024-01-15T14:17:52.640616000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:198 +0xc8
2024-01-15T14:17:52.640690000Z golang.org/x/sync/singleflight.(*Group).doCall(0x2b3e430, 0xc000101040, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.640849000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:200 +0x15c
2024-01-15T14:17:52.640900000Z golang.org/x/sync/singleflight.(*Group).Do(0x2b3e430, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.640971000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:113 +0x385
2024-01-15T14:17:52.641087000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2()
2024-01-15T14:17:52.641154000Z 	/root/acceleration-service/pkg/adapter/adapter.go:162 +0xe5
2024-01-15T14:17:52.641207000Z github.com/goharbor/acceleration-service/pkg/adapter.NewWorker.func1()
2024-01-15T14:17:52.641356000Z 	/root/acceleration-service/pkg/adapter/worker.go:44 +0x5d
2024-01-15T14:17:52.641414000Z created by github.com/goharbor/acceleration-service/pkg/adapter.NewWorker
2024-01-15T14:17:52.641502000Z 	/root/acceleration-service/pkg/adapter/worker.go:41 +0x13e
2024-01-15T14:17:52.641574000Z
2024-01-15T14:17:52.641618000Z
2024-01-15T14:17:52.641666000Z goroutine 30 [running]:
2024-01-15T14:17:52.641714000Z golang.org/x/sync/singleflight.(*Group).doCall.func1()
2024-01-15T14:17:52.641797000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:170 +0x4b5
2024-01-15T14:17:52.641861000Z golang.org/x/sync/singleflight.(*Group).doCall(0x2b3e430, 0xc000101040, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.641910000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:205 +0x173
2024-01-15T14:17:52.641957000Z golang.org/x/sync/singleflight.(*Group).Do(0x2b3e430, {0xc00085e580, 0x7f}, 0xc00230df08)
2024-01-15T14:17:52.642013000Z 	/root/go/pkg/mod/golang.org/x/sync@v0.5.0/singleflight/singleflight.go:113 +0x385
2024-01-15T14:17:52.642079000Z github.com/goharbor/acceleration-service/pkg/adapter.(*LocalAdapter).Dispatch.func2()
2024-01-15T14:17:52.642225000Z 	/root/acceleration-service/pkg/adapter/adapter.go:162 +0xe5
2024-01-15T14:17:52.642276000Z github.com/goharbor/acceleration-service/pkg/adapter.NewWorker.func1()
2024-01-15T14:17:52.642321000Z 	/root/acceleration-service/pkg/adapter/worker.go:44 +0x5d
2024-01-15T14:17:52.642378000Z created by github.com/goharbor/acceleration-service/pkg/adapter.NewWorker
2024-01-15T14:17:52.642425000Z 	/root/acceleration-service/pkg/adapter/worker.go:41 +0x13e
2024-01-15T14:17:53.917939000Z time="2024-01-15T14:17:53.917486234Z" level=info msg="Version: v0.0.10-36-gfb1bf94 fb1bf94ee1ea0f48f5310e8e0fece66ae72ae394.20240115.1201\n"
2024-01-15T14:17:54.043654000Z time="2024-01-15T14:17:54.043445176Z" level=info msg="[API] HTTP server started on <IP_ADDRESS>:2077"

It looks like nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08 is not a standard OCI V1 image: reference for unknown type: application/vnd.in-toto+json. So get imageSize failed:
https://github.com/goharbor/acceleration-service/blob/b82ccee1e73845741b033f8b04ab418fad5b84ef/pkg/converter/metric.go#L61-L86

I try to pull nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08 failed by nerdctl pull nginx@sha256:35a707bebd92e639a71fec1d5b344a4eb00f113123e38c54a5b84962928f5d08.
WARN[0007] reference for unknown type: application/vnd.in-toto+json
FATA[0007] failed to unpack image on snapshotter overlayfs: mismatched image rootfs and manifest layers

@Desiki-high Maybe also fix the panic.


Remove Yoshubs from the Credits

time and time again, this has been topic of discussion, her code was already done and redone several times, not to mention, the original cache implementation wasn't even made by her, it was made by haya, which she only added into, she doesn't deserve the credit she's getting anymore does she?

so true sayof

meerged internally


fix: Reduce the amount of HTTP requests we make

What happened?
There are certain areas where we make too many HTTP requests, some times unnecessarily repeated HTTP requests, sometimes requests that respond with data that we no longer need (this one is more tech debt), and sometimes we request data too early (i.e. at application bootstrap rather than when we visit the page that needs the data).
We also poll 2 endpoints every 10 seconds for data throughout the lifetime of the application. Some of this data is no longer needed and most of this data should be synced from other HTTP calls rather than relying on polling. We should look to remove the polling when only used for local data syncing.
Possibly related to https://github.com/kumahq/kuma-gui/pull/1122 which is awaiting potential approval. At which point this approach could be rolled out to the rest of the application, probably beginning with finishing up the policies area (this is where we use data gained at app bootup), and then rolling it out across all the collection pages first, followed by the detail pages.
Once complete we should preferably no longer be making duplicated HTTP requests and only be requesting data when we need it. Additionally we should try to remove polling to ensure data stays in sync locally, especially polling whilst on pages that don't require the data that we are polling for.

A lot of things have improved here. We should check this once all the datasource stuff has landed

We looked at this. The only ones left is getting rid of the bootstrap and this is covered somewhere else.


[FEAT] purge returns deleted messages

Problem Description
In V4 channel.purge returns a list of deleted messages.
It would be nice to have this in V5 too.
Proposed Solution
channel.purge(...) -> list[Message]
Alternatives Considered
No response
Additional Information
No response
Code of Conduct

[X] I agree to follow the contribution requirements.

Think it's a good idea - it shouldn't be too hard anyhow.

...turns out this would be breaking. Still not a bad idea IMO, but it'll have to be discussed.

Reesolved in #1396 without breaking


same material conflict

if I have a polygon with Color '#1890ff', and hide it. then I create anthor polygon use same material color '#1890ff', the hidden one will show at the same time。
I used Cesium version1.50

Can you please provide a Sandcastle example reproducing this issue @crazymad-tools ?
I'm closing because there's not enough information to determine that this is a bug. My guess is you need to create a separate material for the other polygon, since materials with the same properties are cached for performance.


Support Guest Registration

register to updates to use V2 API Path
add register_as_guest method to MatrixClient

Other than that, LGTM

What is missing for this to get merged? Username/password only registration no longer works since https://github.com/matrix-org/matrix-python-sdk/commit/a7715c3b82c87b83e46cb8b609177bf950ef74d2

LGTM :+1:


Add package to deb.torproject.org

I ran into #260 after setting up deb.torproject.org, and discovered my version of torbrowser-launcher was out-of-date (0.2.6-1 on Ubuntu 16.10). Would it be possible to add torbrowser-launcher to the deb.torproject.org so that there is a single, trusted source for everything TBB related?
(This was mentioned in #29 and #31 , so was this previously planned and cancelled?)

Same on Ubuntu 16.04, outdated version that didn't work.

@theel0ja In case you missed it, the fix in #263 works for the package in Ubuntu 16.04, though I find that keyserver.ubuntu.com is usually a bit more reliable than pgp.mit.edu if you're still having problems.
I'd definitely support the idea of integrating into the TorProject deb repo though. Especially since torbrowser-launcher is the only reason I need to keep contrib enabled on my Debian systems, and this would let me avoid that.

Just updated to latest version from PPA, works properly.

FWIW: There is no build of TorBrowser made by the TorProject, hence it is not available as a package on deb.torproject.org.
torbrowser-launcher is an independent wrapper for downloading the TorBrowser sources.
The package available in Debian and Ubuntu is made by Debian people, not Tor people.
Complicated entanglements! :)

I assumed that torbrowser-launcher downloaded the build available at https://www.torproject.org/download/download-easy.html.en -- i.e. bypassing reliance on any distro's (potentially out-of-date) build?
So currently I need to:

Add deb.torproject.org as a source to get the latest Tor router
Add ppa:micahflee/ppa as a source to get the latest torbrowser-launcher
Use torbrowser-launcher to get the latest TBB
Rely on sources and torbrowser-launcher for further updates

My suggestion (if feasible) is to include torbrowser-launcher in deb.torproject.org and thereby skip step 2. (Step 2 is also tricky on non-Ubuntu Debian-based distros.)

@kwill As far as I know, micah isn't a direct contributor to torproject, so he has no control over what's hosted on their site.

Created https://trac.torproject.org/projects/tor/ticket/22965 and cross-linked the tickets.


How to cache const elements in JSX

Description
I wrote vue in TSX (not with single  vue component file), e.g., a modified HelloWorld component. In this file, lots of elements are constant and won't be changed. In React, they can be extracted by babel plugin. But with vue's jsx babel plugin, they won't be extracted. So can I just simplely create some constant vnodes (just like what react's plugin does), and reuse them across different component instances?

Please open your issue in the vuejs/jsx repo


ScrollIntoView in lists not working correctly

Describe the bug
When I want to scroll a list to a specific item using ScrollIntoView its not working as desired.
My expectation is that the the item or index which is provided to ScrollIntoView will be on top of the list. Of course only when there are man other following items in the list.
It looks like I can achieve the desired behaviour only when I scroll to the end of the list, and then back to the desired item.
Also when an item gets selected in the list then ScrollIntoView also does not work as desired.
To Reproduce

Check out the sample from this branch: https://github.com/agnauck/AvaloniaListBoxBug3/tree/scroll-to-item-bug
Click "scroll to start" to scroll to the first item
Click "Add Top 1"

the sample should mimic an infinite scroll up behaviour. When you click "Add Top 1" or "Add Top 2" it adds 10 more items to the beginning of the list, and scrolls back to the item that was on top before the items were inserted.
Add Top1 does:

save last item
insert 10 new items at the beginning
scroll back to the saved item

Add Top 2 does:

save last item
insert 10 new items at the beginning
scroll to the end of the list
scroll back to the saved item

Make sure you don't select an item. When items get selected then ScrollIntoView also does not work as expected.
Screenshots
https://github.com/AvaloniaUI/Avalonia/assets/1344950/3363f55c-ab13-40b9-8e59-e7ad6783d7f4
Environment

OS:  Windows
Avalonia-Version: 10.0.7

additional info
it applies to non-virtualized and virtualized lists

I haven't look deep into the issue, but normally ScrollIntoView is needed to be executed after desired element was materialized and actually layouted, as otherwise it would have 0,0 boundaries. Something like, Dispatcher.UIThread.Post usually enough.
But that's only if you have this issue with newly added item, that wasn't yet materialized.

I also tested with Dispatcher or also some simple command buttons. It does not make difference.
But I also don't know what the current expected behaviour should be.
When I call ScrollToEnd followed by ScrollIntoView then the desired item is on top of the list.
When not calling ScrollToEnd before the Item ends up randomly on top, on bottom or somewhere in between.
With infinite scrolling the positioning needs to be exact and reproducible


Output tour isn't a tour?

I'm having trouble understanding the output of this solver, from the example input in the README.md.
array([ 0, 48, 31, 44, 18, 40,  7,  8,  9, 42, 32, 50, 10, 51, 13, 12, 46,
       25, 26, 27, 11, 24,  3,  5, 14,  4, 23, 47, 37, 36, 39, 38, 35, 34,
       33, 43, 45, 15, 28, 49, 19, 22, 29,  1,  6, 41, 20, 16,  2, 17, 30,
       21], dtype=int32)

How is this a tour if it doesn't start and end on the same vertex/index?

I'm guessing the last node is omitted on purpose since it's assumed to be 0, but I just wanted to check.

@Carpetfizz  That is correct, the last node (equal to the starting node) is omitted.


Update autoconf

Update autoconf so it work for all directories; 2d, 2dp, 3d, 3dp

Use ax_ext.m4 instead of having so many entries for SIMD detection in configure.ac

At the moment it is adding sse, avx, and avx512 flags all at the same time if they are available and is causing warnings. Please fix this so that it only selects the last option.

resolved by ac20871


Support application/problem+json

Currently api/throw-error yields the following json:
{
  "message": "Error",
  "details": {...}
}

It would be cool if this could be changed to the application/problem+json RFC. The function signature can easily be the same.

@sebastianpoeplau and @harti2006, WDYT about @prayerslayer's suggestion?

Sounds like a good idea to me.

My impression is that the parser component shouldn't be responsible for generating HTTP responses. The parser should return the result of parsing which can be handled by an error handler down the chain. This way a user of s1st can optionally add a middleware down the chain to handle errors according to their specification.


(iOS) Multiple target support.

Describe what happened
Beforehand, please let me know if I missed something from the source or if this is a expected behavior!
While configuring the project and running the wizard, I've noticed that the buildPhase "Upload dSYMs to Datadog" was only created for one of the targets.
In our app, we have the targets "staging" and "production". In our Podfile, these targets are also targets of our "main" abstract target.
I came about this issue out of luck: while reviewing the diffs in the .pbxproj, I noticed that the buildPhase was only delcared in one of the targets (staging).
Steps to reproduce the issue:
📝 Have a React Native project with 2 iOS targets and run the wizard.
Expected behaviour:
📝 All targets should be affected by the wizard.
Actual behaviour:
📝 Only one target was affected by the wizard.
Additional context
I've done some tinkering and copied the build phase from one target into the other. I'll add a comment if it works as expected.

Node version v16.20.0 (NVM)

react-native version 0.67.5

@datadog/react-native-wizard version 0.4.0

@datadog/mobile-react-native version ^2.3.2

an explanation of what might cause the bug and/or how it can be fixed
This probably happens due to the following snippet from "src/commands/setup/add-xcode-dsyms-build-phase/inject-dsyms-build-phase.ts"

   xcodeProject.addBuildPhase(
        [],
        "PBXShellScriptBuildPhase",
        "Upload dSYMs to Datadog",
        // ================
        // THIS GUY:
        null /* target */,
        // ================
        {
          shellScript: `set -e\\n../node_modules/.bin/datadog-ci dsyms upload $DWARF_DSYM_FOLDER_PATH`,
          shellPath: "/bin/sh",
        }
      );

We could have an array of targets and resolve a promise for each of those targets. Something like:
 const addBuilPhaseToPbxprojPromisse = targets.map(target => ...);
      await Promise.all(addBuilPhaseToPbxprojPromiesse);

Hey @Volksfeld, sorry for the long wait and thank you for your feedback.
You are correct, we should grant support for multiple targets in our wizard.
Did you manage to fix your problem by using the workaround you described?
I will keep you posted here as soon as we have a PR!


Prometheus integration with MicrometerReactiveLogger

Hi, could you shed some light on how to use this class? I defined this bean in my config class in a Spring Boot project, but I cannot get it to work. I defined the name but at last I don't see the name in the http://localhost:8080/_system/check/prometheus page.
I have a sample project here in https://github.com/WesleyBlancoYuan/spring-boot-reactive-webflux-template. Just have a look.

NVM, I found the problem: those metrics will only show after actually calling those feign clients methods.


Add snowflake proto

Signed-off-by: Kevin Su<EMAIL_ADDRESS>TL;DR
Add snowflake in FLyteidl for the snowflake plugin to use.
We will get "query statement"  from the target instead of custom, so I didn't define "statement" in his proto.
Related to https://github.com/flyteorg/flyteidl/pull/214
Type

[ ] Bug Fix
[ ] Feature
[x] Plugin

Are all requirements met?

[x] Code completed
[ ] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
https://github.com/flyteorg/flyte/issues/704
https://github.com/flyteorg/flyteplugins/pull/202
Follow-up issue
NA

Closing in favor of: https://github.com/flyteorg/flyteidl/pull/214


[TODO] Add feature to serialize glTF to memory

Implement writing glTF data to memory(std::string)

#178 implemented a feature to serialize glTF to stream.


bug(revisions): Disabling feature flag while revision is in progress leads to unexpected behavior

Describe the bug
If you disable a feature flag when a revision is in progress you'll see both the revision and the published collection in the my-collections page
Is this acceptable behavior?

This reads like an edge case to me.

Can we close as this is not something we are likely going to address in the near future?

Agree with @maniarathi. Closing.


Add mobile support

Add mobile support for drag and drop functionality.

Draggable and Droppable Actions:

Add touch event listeners (touchstart, touchmove, touchend) in src/lib/actions/draggable.ts and src/lib/actions/droppable.ts.
Update handleDragStart and handleDragEnd in draggable.ts to handle touch events.
Update handleDragEnter, handleDragLeave, handleDragOver, and handleDrop in droppable.ts to handle touch events.
Optimize touch event handling in droppable.ts by debouncing touch events.

Styles:

Add visual feedback styles for touch interactions in src/lib/styles/dnd.css.
Add media queries for responsive design in src/lib/styles/dnd.css.
Update src/app.css to include mobile-specific styles and media queries.

Svelte Components:

Update drag and drop logic to handle touch events in src/routes/+page.svelte, src/routes/grid-sort/+page.svelte, src/routes/horizontal-scroll/+page.svelte, src/routes/nested/+page.svelte, and src/routes/simple-list/+page.svelte.
Add visual feedback for touch interactions in the above Svelte components.

For more details, open the Copilot Workspace session.

@rezapex Thanks for the PR. LGTM! Merging it.

No problem!


What does CENTER(SIZE)_VARIANCE mean in defaults.py?

I don`t know these configure in default.py:
Hard negative mining
_C.MODEL.CENTER_VARIANCE = 0.1
_C.MODEL.SIZE_VARIANCE = 0.2
↑
Theyr used in /ssd/util/box_utils.py when boxes invert into locations or locations invert into boxes.But I dont know why.
change MAX_PER_CLASS to 400 as official caffe code will slightly increase mAP(0.8025=>0.8063, 0.7783=>0.7798)
_C.TEST.MAX_PER_CLASS = 200
_C.TEST.MAX_PER_IMAGE = -1
↑
I dont know these either and I cant find where they`r used in project.
Can anyone help?Thanks all the time.

Variance is used to encode/decode prior bboxes .
https://github.com/weiliu89/caffe/blob/4817bf8b4200b35ada8ed0dc378dceaf38c539e4/examples/ssd/ssd_pascal.py#L322
MAX_PER_CLASS and MAX_PER_IMAGE are used in post_processor.py


Fix Metrics Cache Size Settings Widget

May need to build new type of component to contain the Metrics Cache Size information and reset button. As it is, the widget is stretched and difficult to read.

Closing as the solution to this is a duplicate of #602, which I'll bump up to a priority 1 bug to reflect the fact that it's currently broken.


IMP: add optional to File and FileCollection.

No tests yet!
Also adds better error message for unimplemented pathmaker as well.

Looking good! I would add one more test to demonstrate that leaving optional out entirely has the default behavior of False.

Ok, I am going to add one more test that checks that it raises if you give it a file that it does not know how to store.


adding kuberhealthy

Signed-off-by: Eric Greer<EMAIL_ADDRESS>[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md

/ok-to-test

/lgtm

/hold

/retest

/hold cancel

/retest

/retest

It was good talking with you today @viglesiasce.  I actually want to try a pixelbook now.
Looks like the most recent build error involves the CRD already existing in the test cluster?
I1213 03:56:24.977] Error: release kuberhealthy-1q9tq4e294 failed: customresourcedefinitions.apiextensions.k8s.io "khstates.comcast.github.io" already exists

Perhaps I will wait a few and give it a retest.

I think it's ready for a LGTM?!  @viglesiasce

/lgtm


Email notifications point to localhost

I have my notifications set for all of the settings, but have only been able to test volume low, and new session.
When these notifications come all of the links point to http://localhost. This causes the links to go nowhere unless logged onto the server.
When emails are sent to add users or reset passwords the link uses my dynamic dns hostname.  The links will properly direct the user to the kegbot server from the internet.

How do you get to URL settings


Multichain js test robustness

 Increase test robustness of Multichain.js

looks like the error handling in the source file changes behavior. look in branch multichainfeedback for a proposed fix; that solution might be abstracted into a wrapper function instead of copypaste the handling code everywhere, but I couldn't get it to work in 5 min.
I've added some additional tasks to the card in Trello; they should be split, this branch reviewed, and we'll prioritize the other tasks in the backlog.


Added the test_suite thingy to setup.py

Hey,
I figured someone would maybe perhaps want to use the 'python setup.py test' so I added the test_suite field in setup.py. :)

Command seems to work without this.  Thanks for pointing it out though.


Add keyboard controls to playback controller

As discussed in #108, more keyboard controls are added to the playback controller.
For now, this includes:

Space for pause/resume
Right/Left Arrow for next/previous song
+/- for increase/decrease volume

One todo for this change (in a new PR?) would be reading those keys from the config file to make the shortcuts changeable via the settings. But I'd need some input on on how this should be implemented.

Thanks! I've re-based on top of master, as it's needed for the focus mechanism to work, and added some code to request focus again after i.e. the search input resigns it. Seems to work!

Seems like the shortcuts are only recognised when a song is selected. It would be perhaps useful to have play shortcut default to the top song on the top playlist of the home section.
Also macOS "play" shortcut key is ignored in the above circumstance and so itunes loads
Maybe this is the wrong place for this comment

Maybe the macOS play shortcut (I guess you are referring to the one on the touch bar?) is not recognized as a media shortcut by druid? I will look into this in the next days. Thank's for bringing this up!

Yes seems all mac keyboards have a play/pause and back/forward track buttons.
Loving the app, daily use. Thanks!


build: Update to header 1.2.184

Update known-good
Generate source

Change-Id: I6f8219e31038b0efe1e141147292d041655dd640

CI Vulkan-Tools build queued with queue ID 2663.

CI Vulkan-Tools build # 487 running.

CI Vulkan-Tools build # 487 passed.


Anyres compatible fine-tuning of llava-1.6 mistral 7b and 34b

Lowrank fine-tuning with anyres for the LLaVA Next models :)

that is a good pr.. i am finetune this pr ..thanks .
@arielnlee

Ofc, glad you found it useful! I'm sure the author's version is far superior (<3 llava), but wanted to leave this here for others to use until we get the real magic :)
@awzhgw

@arielnlee  I encountered an issue during the training process. I am using the LoRA fine-tuning method, and my data consists of two parts:
Image-question-answering dialogues.
Pure question-answering dialogues.
During training, I found that the training speed for the second part of the dataset is very slow, and it is as slow as the first part. After investigation, I found that the reason is:
在train.py的LazySupervisedDataset 类的__getitem__ 方法：
        if 'image' in self.list_data_dict[i]:
            data_dict['image'] = image
            data_dict['image_size'] = image_size
        elif self.data_args.is_multimodal:
            # image does not exist in the data, but the model is multimodal
            crop_size = self.data_args.image_processor.crop_size
            data_dict['image'] = torch.zeros(3, crop_size['height'], crop_size['width'])
            data_dict['image_size'] = crop_size
        return data_dict

when i delete this code 👍
elif self.data_args.is_multimodal:
# image does not exist in the data, but the model is multimodal
crop_size = self.data_args.image_processor.crop_size
data_dict['image'] = torch.zeros(3, crop_size['height'], crop_size['width'])
data_dict['image_size'] = crop_size
return data_dict
the train process error:
Traceback (most recent call last):
  File "/export/App/training_platform/PinoModel/LLaVA/llava/train/train_mem.py", line 9, in <module>
    train(attn_implementation="flash_attention_2")
  File "/export/App/training_platform/PinoModel/LLaVA/llava/train/train.py", line 1092, in train
    trainer.train()
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1537, in train
    return inner_training_loop(
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1854, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2744, in training_step
    self.accelerator.backward(loss)
  File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 1958, in backward
    self.deepspeed_engine_wrapped.backward(loss, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/deepspeed.py", line 167, in backward
    self.engine.backward(loss, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
    ret_val = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", line 1964, in backward
    self.optimizer.backward(loss, retain_graph=retain_graph)
  File "/usr/local/lib/python3.10/dist-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
    ret_val = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/zero/stage3.py", line 2152, in backward
    self.loss_scaler.backward(loss.float(), retain_graph=retain_graph)
  File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/fp16/loss_scaler.py", line 63, in backward
    scaled_loss.backward(retain_graph=retain_graph)
  File "/usr/local/lib/python3.10/dist-packages/torch/_tensor.py", line 491, in backward
    torch.autograd.backward(
  File "/usr/local/lib/python3.10/dist-packages/torch/autograd/__init__.py", line 251, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

and then perform fine-tuning for the pure text stage?
Can I do this? ? ? Finally trained a good llava model

i got adapter_model.safetensor instead of adapter_model.bin after LORA FInetuning of 1.6-mistral and getting error as
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:03<00:00,  1.30it/s]
Traceback (most recent call last):
File "/home/rohith/LLaVA-1.6-ft/scripts/merge_lora_weights.py", line 22, in
merge_lora(args)
File "/home/rohith/LLaVA-1.6-ft/scripts/merge_lora_weights.py", line 8, in merge_lora
tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, device_map='cpu')
File "/home/rohith/LLaVA-1.6-ft/llava/model/builder.py", line 112, in load_pretrained_model
mm_projector_weights = torch.load(os.path.join(model_path, 'mm_projector.bin'), map_location='cpu')
File "/home/rohith/miniconda3/envs/llava/lib/python3.10/site-packages/torch/serialization.py", line 986, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/rohith/miniconda3/envs/llava/lib/python3.10/site-packages/torch/serialization.py", line 435, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/rohith/miniconda3/envs/llava/lib/python3.10/site-packages/torch/serialization.py", line 416, in init
super().init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/home/rohith/Documents/mistral-llava/mm_projector.bin'
when trying to merge model

@rohithbojja  may be mode_path is error 。。 please give me :model_path, mode_base args

@rohithbojja
nohup python scripts/merge_lora_weights.py --model-path=../checkpoints/llava-v1.6-34b-xxx-lora-5000 --model-base=../checkpoints/llava-v1.6-34b --save-model-path=../checkpoints/llava-v1.6-34b-xxx-5000 &

@rohithbojja
nohup python scripts/merge_lora_weights.py --model-path=../checkpoints/llava-v1.6-34b-xxx-lora-5000 --model-base=../checkpoints/llava-v1.6-34b --save-model-path=../checkpoints/llava-v1.6-34b-xxx-5000 &

ive fixed my adding "lora" to model-path

Can you please provide some example of your training data?
system="""<|im_start|>system\nAnswer the questions.""",
roles=("<|im_start|>user\n", "<|im_start|>assistant\n"),

I was wondering why you chose to add a new conversation format. I was trying to tune based on your PR and my existing data made for LLaVA 1.5 fine-tuning which uses 'V1' version, but currently running into issues where the tokenizer length is mismatching

Checkout my Wandb logs.
https://wandb.ai/21b81a66a5/huggingface/runs/4pslu1px/overview?nw=nwuser21b81a66a5

#!/bin/bash
deepspeed llava/train/train_mem.py
--lora_enable True --lora_r 16 --lora_alpha 32 --mm_projector_lr 2e-5
--deepspeed ./scripts/zero2.json
--model_name_or_path /home/rohith/llava-v1.6-mistral-7b-bnb-4bit/
--version mistral_instruct
--data_path /home/rohith/Desktop/vqa/vqa/images/filtered_dataset.json
--image_folder /home/rohith/Desktop/vqa/vqa/images/
--vision_tower openai/clip-vit-large-patch14-336
--mm_projector_type mlp2x_gelu
--mm_vision_select_layer -2
--mm_use_im_start_end False
--mm_use_im_patch_token False
--mm_patch_merge_type spatial_unpad
--image_aspect_ratio anyres
--group_by_modality_length False
--bf16 False
--fp16 True
--output_dir /home/rohith/LLaVA-1.6-ft/llava_lora_mistral_med/
--num_train_epochs 1
--per_device_train_batch_size 1
--per_device_eval_batch_size 1
--gradient_accumulation_steps 4
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 500
--save_total_limit 5
--learning_rate 2e-5
--weight_decay 0.
--warmup_ratio 0.05
--lr_scheduler_type "cosine"
--logging_steps 1
--tf32 True
--model_max_length 4096
--gradient_checkpointing True
--dataloader_num_workers 4
--lazy_preprocess True
--report_to wandb \
using this script gives me error

ValueError: .to is not supported for 4-bit or 8-bit bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct dtype.
and
using original model doesnt give any error,
used
panoyo9829/llava-v1.6-mistral-7b-bnb-4bit model

Can anyone share the filtered_dataset json for the 34b training?
Yours,
Alex
On Apr 24, 2024 at 4:42 AM -0700, Rohith Bojja @.***>, wrote:

#!/bin/bash
deepspeed llava/train/train_mem.py
--lora_enable True --lora_r 16 --lora_alpha 32 --mm_projector_lr 2e-5
--deepspeed ./scripts/zero2.json
--model_name_or_path /home/rohith/llava-v1.6-mistral-7b-bnb-4bit/
--version mistral_instruct
--data_path /home/rohith/Desktop/vqa/vqa/images/filtered_dataset.json
--image_folder /home/rohith/Desktop/vqa/vqa/images/
--vision_tower openai/clip-vit-large-patch14-336
--mm_projector_type mlp2x_gelu
--mm_vision_select_layer -2
--mm_use_im_start_end False
--mm_use_im_patch_token False
--mm_patch_merge_type spatial_unpad
--image_aspect_ratio anyres
--group_by_modality_length False
--bf16 False
--fp16 True
--output_dir /home/rohith/LLaVA-1.6-ft/llava_lora_mistral_med/
--num_train_epochs 1
--per_device_train_batch_size 1
--per_device_eval_batch_size 1
--gradient_accumulation_steps 4
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 500
--save_total_limit 5
--learning_rate 2e-5
--weight_decay 0.
--warmup_ratio 0.05
--lr_scheduler_type "cosine"
--logging_steps 1
--tf32 True
--model_max_length 4096
--gradient_checkpointing True
--dataloader_num_workers 4
--lazy_preprocess True
--report_to wandb
using this script gives me error
ValueError: .to is not supported for 4-bit or 8-bit bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct dtype.
and
using original model doesnt give any error,
used
panoyo9829/llava-v1.6-mistral-7b-bnb-4bit model
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

Can anyone share the filtered_dataset json for the 34b training?
Yours,
Alex
On Apr 24, 2024 at 4:42 AM -0700, Rohith Bojja @.***>, wrote:

#!/bin/bash

deepspeed llava/train/train_mem.py

--lora_enable True --lora_r 16 --lora_alpha 32 --mm_projector_lr 2e-5

--deepspeed ./scripts/zero2.json

--model_name_or_path /home/rohith/llava-v1.6-mistral-7b-bnb-4bit/

--version mistral_instruct

--data_path /home/rohith/Desktop/vqa/vqa/images/filtered_dataset.json

--image_folder /home/rohith/Desktop/vqa/vqa/images/

--vision_tower openai/clip-vit-large-patch14-336

--mm_projector_type mlp2x_gelu

--mm_vision_select_layer -2

--mm_use_im_start_end False

--mm_use_im_patch_token False

--mm_patch_merge_type spatial_unpad

--image_aspect_ratio anyres

--group_by_modality_length False

--bf16 False

--fp16 True

--output_dir /home/rohith/LLaVA-1.6-ft/llava_lora_mistral_med/

--num_train_epochs 1

--per_device_train_batch_size 1

--per_device_eval_batch_size 1

--gradient_accumulation_steps 4

--evaluation_strategy "no"

--save_strategy "steps"

--save_steps 500

--save_total_limit 5

--learning_rate 2e-5

--weight_decay 0.

--warmup_ratio 0.05

--lr_scheduler_type "cosine"

--logging_steps 1

--tf32 True

--model_max_length 4096

--gradient_checkpointing True

--dataloader_num_workers 4

--lazy_preprocess True

--report_to wandb \

using this script gives me error

ValueError: .to is not supported for 4-bit or 8-bit bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct dtype.

and

using original model doesnt give any error,

used

panoyo9829/llava-v1.6-mistral-7b-bnb-4bit model

—

Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you commented.Message ID: @.***>

https://drive.google.com/file/d/1gYLOFaz7Mn-E2u9ksT0R2BOai7MnmNcm/view?usp=drivesdk
It has following structure
VQA-
1.images
|_img1
|_img2
2.train
|_filtered_dataset.json
1 image is truncated,
Remove it,
Use this to detect
from PIL import Image
import os
trunk_ = 0
def is_truncated(image_path):
try:
# Open the image file
img = Image.open(image_path)
# Check if the image is truncated by trying to load it
img.load()
return False  # Image is not truncated
except Exception as e:
print(f"Error loading image {image_path}: {e}")
return True  # Image is truncated or corrupt
def check_for_truncated_images(directory,trunk_):
# Iterate through all files in the directory
for filename in os.listdir(directory):
# Check if the file is an image
if filename.endswith(('.jpg', '.jpeg', '.png', '.gif', '.bmp')):
image_path = os.path.join(directory, filename)
if is_truncated(image_path):
print(f"The image {filename} in directory {directory} is truncated.")
trunk_ = 1
else:
trunk_=0
print(trunk_)
Provide the directory path containing images
directory_path = '/workspace/vqa/images'
check_for_truncated_images(directory_path,0)
Also remove the entry in json.
Otherwise you'll end up failing at 30% or so
Good luck

i run this python -m llava.serve.model_worker --host <IP_ADDRESS> --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path /hy-tmp/result --model-base /hy-tmp/Mistral-7B-Instruct-v0.2, this error has occurred, as shown in the following image:

/hy-tmp/result is lora finetune llava 1.6-mistral result and this warning appears after training :/usr/local/miniconda3/envs/llava-wenwu/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /hy-tmp/llava-v1.6-mistral-7b - will assume that the vocabulary was not modified.

Hi! Thanks for sharing!
However, when I execute your training script, it also trains the vision encoder (adapter_model.safetensors contains vision encoder weights)
Is there a way to disable the gradient backprop into these weights as done for the original llava fine-tuning?

Can you please provide some example of your training data?
system="""<|im_start|>system\nAnswer the questions.""",
roles=("<|im_start|>user\n", "<|im_start|>assistant\n"),

I was wondering why you chose to add a new conversation format. I was trying to tune based on your PR and my existing data made for LLaVA 1.5 fine-tuning which uses 'V1' version, but currently running into issues where the tokenizer length is mismatching

@findalexli Hi there, did you find out why? I used the new template and ran into tokenization mismatch errors so i'm going to try with v1 now. Let me know if you managed to finetune a llava1.6! :)


FIX: Not looping leading to TypeError

A flag like --loop in this case is typically used as a "true-if-supplied" flag. However, it was instead (confusingly counter-intuitively) used as a "false-if-supplied" flag.
This is now fixed. python -m pyxdf.examples.playback_lsl.py --loop will loop, whereas not supplying --loop will not loop.

Actually, there is more wrong. Not looping is not possible currently:
  File "C:\Users\stefan\AppData\Local\miniforge3\envs\naf\Lib\site-packages\pyxdf\examples\playback_lsl.py", line 229, in <module>
    main(
  File "C:\Users\stefan\AppData\Local\miniforge3\envs\naf\Lib\site-packages\pyxdf\examples\playback_lsl.py", line 192, in main
    timer.update()
  File "C:\Users\stefan\AppData\Local\miniforge3\envs\naf\Lib\site-packages\pyxdf\examples\playback_lsl.py", line 103, in update
    overrun = self._n_loop * self._boundary
              ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

@cbrnr I'd be happy to receive a review on this!

playback_lsl is mine, but I'm happy to see others use it and improve it.
The previous default behaviour was to loop, and now the default behaviour is to not loop. I prefer if the default is to loop, but then how do we fix the argument name in argparse? Do we only offer "--no-loop"?
I'm not great with argparse. I tend to use typer more but I don't think we need to add that as a dependency unless we really plan to expand the CLI offerings. Anyway, in typer, bool args get exposed both in the affirmative and negative. e.g., this would give use --loop and --no-loop; both would be exposed regardless of what we set the default to.

I can also be convinced that not-looping is the better default behaviour.

Hey @cboulay -- thanks for programming and sharing this script, it has been very helpful for me!

Do we only offer "--no-loop"?

yes, that would be a sensible alternative solution.

I can also be convinced that not-looping is the better default behaviour.

I think from a user experience perspective it's more natural to expect a playback to NOT loop by default (think of music that you play).
Please let me know if you want me to change the default back to looping, and adjust the flag to be --no-loop.
Other than that, it would be ready for merge from my side :)

Just my two cents (and you said you can be convinced), I'd also think that not looping is a better default. But I leave this up to you, so feel free to merge whenever you're happy @cboulay!

@sappelhoff can you rebase please? Afterwards I think we can merge.

@cbrnr sure -- done!

Thanks @sappelhoff!


Fix hg branches

This is a series of fix to make the mercurial plugin behave like the git one.

Mercurial default branch is "default"
Only follow first parent on merges changesets

I added a test with merges that work for both mercurial & git backends.

Coverage increased (+0.005%) to 70.081% when pulling 29ad80dd1e8f7f1c0af2c129c51a8b00782c04d5 on philpep:fix-hg-branches into 5b46e9510ec6aa7e4b2d93e990f04b5f2e736a43 on spacetelescope:master.

Coverage increased (+0.005%) to 70.081% when pulling 91bd7d51d4307aa05bb9045b9069e4de244fdd14 on philpep:fix-hg-branches into 5b46e9510ec6aa7e4b2d93e990f04b5f2e736a43 on spacetelescope:master.

Coverage decreased (-0.1%) to 69.977% when pulling 50af5ac3cd293196ae41bb752b7bc5f90964e92c on philpep:fix-hg-branches into 5b46e9510ec6aa7e4b2d93e990f04b5f2e736a43 on spacetelescope:master.

thanks, merged


[fractal-server] AttributeError: module 'bcrypt' has no attribute 'about'

fractal-server  | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
fractal-server  | INFO  [alembic.runtime.migration] Will assume transactional DDL.
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade  -> 50a13d6138fd, Initial schema
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade 50a13d6138fd -> 4c308bcaea2b, Add Task.args_schema and Task.args_schema_version
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade 4c308bcaea2b -> f384e1c0cf5d, Drop Task.default_args columns
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade f384e1c0cf5d -> 70e77f1c38b0, Add ApplyWorkflow.first_task_index and ApplyWorkflow.last_task_index
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade 70e77f1c38b0 -> a7f4d6137b53, Add ApplyWorkflow.workflow_dump
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade a7f4d6137b53 -> 8f79bd162e35, Add Task.docs_info and Task.docs_link
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade 8f79bd162e35 -> 99ea79d9e5d2, Add Dataset.history
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade 99ea79d9e5d2 -> 84bf0fffde30, Add dumps to ApplyWorkflow
fractal-server  | INFO  [alembic.runtime.migration] Running upgrade 84bf0fffde30 -> e75cac726012, Make ApplyWorkflow.start_timestamp not nullable
fractal-server  | Run alembic.config, with argv=['-c', '/usr/local/lib/python3.10/site-packages/fractal_server/alembic.ini', 'upgrade', 'head']
fractal-server  | fractal_server.app.db
fractal-server  | INFO:     Started server process [10]
fractal-server  | INFO:     Waiting for application startup.
fractal-server  | WARNING:passlib.handlers.bcrypt:(trapped) error reading bcrypt version
fractal-server  | Traceback (most recent call last):
fractal-server  |   File "/usr/local/lib/python3.10/site-packages/passlib/handlers/bcrypt.py", line 620, in _load_backend_mixin
fractal-server  |     version = _bcrypt.__about__.__version__
fractal-server  | AttributeError: module 'bcrypt' has no attribute '__about__'

Ref https://github.com/fractal-analytics-platform/fractal-server/issues/1041


Configure github actions

travis-ci is outta here

thanks, just one minor comment but lgtm otherwise!
will add enhancements to upload the test results in the future.

Thank you!


Visual Studio version '15.0' not found for VSTS Hosted Build

I upgraded my project to VS2017 and now my build is broken - I modified the code to use some new C# constructs (which seemed like a good idea at the time!)
It looks like, even though I modified my build agent task to use 'VS 2017', it is still using MSBuild 14.0.
Is there something I can do to get working, without resorting to a custom build agent (or reverting my shiny new C# code)?

This is a very big issue for me, as we are currently starting a new project and want to target netstandard, using the new project templates. Currently we can not implement any CI builds through hosted agents on VSTS. We want to stay away from private agents, as we do not have the resource to maintain them for one, and two, this really should have been implemented and pushed out at the same time VS2017 was launched.
The build task is quite misleading, as you can select VS2017 as the VS build environment, but this is still RC version not RTM.....
Any news on when this will be pushed out to hosted agents, would really like/need to get VS2017 RTM solutions building through CI in VSTS.

It seems that the build uses the csc.exe from the Microsoft.Net.Compilers nuget package (have just bumped to 2.0.1) to actually build the projects, so all the C#7 features work fine. Not sure if having the wrong VS version matters too much here. I am however getting a lot of tests failing due to missing methods, sadly 2017 isn't an option for tests yet.

is there any update on this?

Dupe of #3311
See latest comment in that issue

The hosted builds are still failing for me with the same error. The linked duplicate issue seems to indicate that the problem was addressed in the build hosts or am I missing something?

Also trying MSBuild I see a similar error.

Unable to find MSBuild: Version = 15.0, Architecture = x86. Looking for the latest version.

@jpierson - you have to pick the VS2017 queue on your definition.  That will get you to a machine that has VS 2017 (msbuild 15.0)

Ok for others running into the same issue it appears that the trick was to also change the Agent setting under Options for the build definition itself and not just the MSBuild/VSBuild version.

Thanks @bryanmacfarlane, I just saw your comment as I was pasting in my findings.

Yeah, we are aware of that problem.  Having to "match up" multiple settings is a big problem.  We are discussing various options.

You also have to do the same thing for your unit tests in the VS Test step(s).

Agents are not available under Options anymore?

ah, moved to 'Process'


Avoid wan gossip by configuring consul with advertise_addr_wan localhost

We do not need wan nodes talk each other, but it seems wan gossip, causing the
nodes in different datacenters talk to each other, so setting advertise_addr_wan
to <IP_ADDRESS> in the environments which doesn't need connections between multiple
datacenters.

please test this

please test this

please test this

please test this

please test this

This is not going to fix it as the nodes are joining to lan gossip and not to wan :( ,  so closing


link to our contribution guidelines

This links to our contribution guidelines. We should set this up for a CoC and Security policy as well. Thanks!

@k-nasa the main appeal for including this is that it'll show in the sidebar whenever someone creates a new issue or PR. Same goes for CoC and security policy.

Thanks.
I understand


Software Update 2<IP_ADDRESS>

This pushes a variety of updates:

updates to latest hm-config version
updates to latest hm-diag version
bumps miner to latest GA 063020210

Confirmed that everything appears to be working on Testnet. Merging.


Hi, I'm using the "time.php" example, in order to send a message to all clientes from a control panel.
My problem is that when I sent the message, the clients continue receiving it again and  again.
I deduce that this is because the "$loop" continues infinitely, and on each loop the message is sended again.
Which is the correct way to front this problem?
Thank you.

Solved:
PHP Code:

require_once("vendor/autoload.php");

// Constants
define('LOG_FILE', "log.txt");
define('WS_IP', 'ws://190.210.xxx.xxx:8084');

// Log:
$logger = new \Zend\Log\Logger();
$writer = new Zend\Log\Writer\Stream(LOG_FILE);
$logger->addWriter($writer);
$logger->notice("... init...");

use Devristo\Phpws\Server\WebSocketServer;

$loop = \React\EventLoop\Factory::create();

$server = new WebSocketServer(WS_IP, $loop, $logger);

$i = 0;

$loop->addPeriodicTimer(3, function(React\EventLoop\Timer\Timer $timer) use(&$i, $server, $logger){

    $logger->notice(++$i);

    if ($i >= 2) {
        $loop->cancelTimer($timer);
    }

	foreach($server->getConnections() as $client) {

		$client->sendString("Hello world: " . $i);

	}

});

$server->bind();

$loop->run();