Vulkan Backend

What does the pull request do?
Adds vulkan backend for skia
What is the updated/expected behavior with this PR?
The backend can be enabled with UseVulkan property in Win32PlatformOption or X11PlatformOptions.
How was the solution implemented (if it's not obvious)?
The backend uses Silk.Net as the vulkan bindings
Checklist

[x] Add render control
[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Documentation with user documentation

Breaking changes
None
Obsoletions / Deprecations

Fixed issues
Fixes #4274

You can test this PR using the following package version. 0.10.999-cibuild0019019-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019019-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019021-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019021-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019022-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019022-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

@s5bug You can follow the guide from https://github.com/AvaloniaUI/Avalonia/wiki/Using-nightly-build-feed and instead of nightly feed url use the one provided by the PR bot above. And then simply update avalonia package version to the one from the bot as well.

The bot doesn't post a message every commit, is that still fine?

You can test this PR using the following package version. 0.10.999-cibuild0019159-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019192-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Ok, using https://nuget.avaloniaui.net/repository/avalonia-all/index.json didn't work, but https://pkgs.dev.azure.com/AvaloniaUI/aa84306f-2981-47b9-8206-edb3bed6250d/_packaging/64388c94-6632-79aa-93af-ab6362308af7/nuget/v3/index.json did. Unfortunately I don't currently have the time to actually write and test Vulkan code.

You can test this PR using the following package version. 0.10.999-cibuild0019270-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Open for review

You can test this PR using the following package version. 0.10.999-cibuild0019274-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019275-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

imo we need to at least cover those types with our own. Never a good idea to expose 3rd party types on the core library's API.

I'm seeing both Avalonia.Skia and Avalonia.Vulkan having public API members exposing raw Silk.NET types.

I understand that replacing Silk.NET for Avalonia.Vulkan might be problematic, and I won't be strictly against of it. Especially if it's only an optional package as an opengl replacement.
But can we avoid it in the Avalonia.Skia?

Silk.Net objects can simply be replaced by only exposing their handles, which are just ulong fields. Those can be used by the user on their own vulkan binding, or if using Silk.Net, just passing it to the respective constructors.

Silk.Net objects can simply be replaced by only exposing their handles, which are just ulong fields

this is acceptable for me :)

@emmauss Perhaps you can add original shader code there with a readme about how to compile it to spirv. In case somebody wants to modify it later.

I've added the glsl files

You can test this PR using the following package version. 0.10.999-cibuild0019290-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019297-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019341-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019354-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Blocked by #7841

You can test this PR using the following package version. 0.10.999-cibuild0019437-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019489-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019495-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019535-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019569-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Hello, I tried it against ControlCatalog.NetCore, but it yielded an empty transparent window. Using Intel 10Gen UHD Graphics + NVIDIA GTX 2060 both with latest drivers.

Am I missing something?

You can test this PR using the following package version. 0.10.999-cibuild0019579-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0019612-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Hello, I tried it against ControlCatalog.NetCore, but it yielded an empty transparent window. Using Intel 10Gen UHD Graphics + NVIDIA RTX 2060 both with latest drivers.

~Am I missing something?~
Never mind, it works after I set UseWgl = true.

did you run the catalog with --vulkan? Setting UseWgl to true will prevent vulkan from using the window surface

Can (kinda) reproduce: Resizing froze my entire desktop for a good few seconds. No crash though.

You can test this PR using the following package version. 0.10.999-cibuild0019725-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

did you run the catalog with --vulkan?

I didn't notice the --vulkan option before. Now it's mostly working as expected, but resizing the window is crashing the app.

Could you post the exception details?

Can (kinda) reproduce with https://github.com/s5bug/EmuFish/tree/95d49a9508b9adc823a8f68cc841524d2ab09d74: Resizing froze my entire desktop for a good few seconds. No crash though.

Can you try setting PresentMode to Immediate in the VulkanOptions?

Issue still occurs with https://github.com/s5bug/EmuFish/tree/250b528cf146b00400d586b93eb28d5b839ab6ba
Immediate present mode is selected: https://github.com/s5bug/EmuFish/blob/250b528cf146b00400d586b93eb28d5b839ab6ba/EmuFish/Program.fs#L59

Issue still occurs with https://github.com/s5bug/EmuFish/tree/250b528cf146b00400d586b93eb28d5b839ab6ba
Immediate present mode is selected: https://github.com/s5bug/EmuFish/blob/250b528cf146b00400d586b93eb28d5b839ab6ba/EmuFish/Program.fs#L59

Running your project on Windows 11, I do not experience your issues. But I have experienced the same thing on linux with gnome while working on the feature. What OS and gpu do you use?

You can test this PR using the following package version. 0.10.999-cibuild0019734-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Edition         Windows 10 Home
Version         21H2
Installed on    2022-01-11
OS build        19044.1586
Experience      Windows Feature Experience Pack 120.2212.4170.0

GPU is a Radeon RX 560.

Could you post the exception details?

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Avalonia.Vulkan.VulkanDisplay.CreateSwapchain(Avalonia.Vulkan.VulkanInstance, Avalonia.Vulkan.VulkanDevice, Avalonia.Vulkan.VulkanPhysicalDevice, Avalonia.Vulkan.VulkanSurface, Silk.NET.Vulkan.Extent2D ByRef, Avalonia.Vulkan.VulkanDisplay)
   at Avalonia.Vulkan.VulkanDisplay.Recreate()
   at Avalonia.Vulkan.VulkanDisplay.EnsureSwapchainAvailable()
   at Avalonia.Vulkan.VulkanSurfaceRenderingSession.Begin()
   at Avalonia.Vulkan.VulkanSurfaceRenderingSession..ctor(Avalonia.Vulkan.VulkanDisplay, Avalonia.Vulkan.VulkanDevice, Avalonia.Vulkan.Surfaces.VulkanSurfaceRenderTarget, Single)
   at Avalonia.Vulkan.Surfaces.VulkanSurfaceRenderTarget.BeginDraw(Single)
   at Avalonia.Vulkan.Skia.VulkanRenderTarget.BeginRenderingSession()
   at Avalonia.Skia.SkiaGpuRenderTarget.CreateDrawingContext(Avalonia.Rendering.IVisualBrushRenderer)
   at Avalonia.Rendering.DeferredRenderer.EnsureDrawingContext(Avalonia.Platform.IDrawingContextImpl ByRef)
   at Avalonia.Rendering.DeferredRenderer.UpdateRenderLayersAndConsumeSceneIfNeeded(Avalonia.Platform.IDrawingContextImpl ByRef, Boolean)
   at Avalonia.Rendering.DeferredRenderer.Render(Boolean)
   at Avalonia.Rendering.DeferredRenderer.Paint(Avalonia.Rect)
   at Avalonia.Controls.TopLevel.HandlePaint(Avalonia.Rect)
   at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
   at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
   at Avalonia.Win32.Win32Platform.RunLoop(System.Threading.CancellationToken)
   at Avalonia.Threading.Dispatcher.MainLoop(System.Threading.CancellationToken)
   at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(System.String[])
   at ControlCatalog.NetCore.Program.Main(System.String[])

Could you post the exception details?

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Silk.NET.Vulkan.Extensions.KHR.KhrSwapchain.CreateSwapchain(Silk.NET.Vulkan.Device, Silk.NET.Vulkan.SwapchainCreateInfoKHR ByRef, Silk.NET.Vulkan.AllocationCallbacks*, Silk.NET.Vulkan.SwapchainKHR ByRef)
   at Avalonia.Vulkan.VulkanDisplay.CreateSwapchain(Avalonia.Vulkan.VulkanInstance, Avalonia.Vulkan.VulkanDevice, Avalonia.Vulkan.VulkanPhysicalDevice, Avalonia.Vulkan.VulkanSurface, Silk.NET.Vulkan.Extent2D ByRef, Avalonia.Vulkan.VulkanDisplay)
   at Avalonia.Vulkan.VulkanDisplay.Recreate()
   at Avalonia.Vulkan.VulkanDisplay.EnsureSwapchainAvailable()
   at Avalonia.Vulkan.VulkanSurfaceRenderingSession.Begin()
   at Avalonia.Vulkan.VulkanSurfaceRenderingSession..ctor(Avalonia.Vulkan.VulkanDisplay, Avalonia.Vulkan.VulkanDevice, Avalonia.Vulkan.Surfaces.VulkanSurfaceRenderTarget, Single)
   at Avalonia.Vulkan.Surfaces.VulkanSurfaceRenderTarget.BeginDraw(Single)
   at Avalonia.Vulkan.Skia.VulkanRenderTarget.BeginRenderingSession()
   at Avalonia.Skia.SkiaGpuRenderTarget.CreateDrawingContext(Avalonia.Rendering.IVisualBrushRenderer)
   at Avalonia.Rendering.DeferredRenderer.EnsureDrawingContext(Avalonia.Platform.IDrawingContextImpl ByRef)
   at Avalonia.Rendering.DeferredRenderer.UpdateRenderLayersAndConsumeSceneIfNeeded(Avalonia.Platform.IDrawingContextImpl ByRef, Boolean)
   at Avalonia.Rendering.DeferredRenderer.Render(Boolean)
   at Avalonia.Rendering.DeferredRenderer.Paint(Avalonia.Rect)
   at Avalonia.Controls.TopLevel.HandlePaint(Avalonia.Rect)
   at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.WindowImpl.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
   at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
   at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
   at Avalonia.Win32.Win32Platform.RunLoop(System.Threading.CancellationToken)
   at Avalonia.Threading.Dispatcher.MainLoop(System.Threading.CancellationToken)
   at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(System.String[])
   at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime[[System.__Canon, System.Private.CoreLib, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.String[], Avalonia.Controls.ShutdownMode)
   at ControlCatalog.NetCore.Program.Main(System.String[])

My system environment:
Edition		Windows 11 Pro for Workstations
Version		22H2
Installed on	2022/04/07
OS Build	22593.1
Experience	Windows Feature Experience Pack 1000.22593.1.0

GPU?

@s5bug @hez2010 could you test again with the current version.

You can test this PR using the following package version. 0.10.999-cibuild0019774-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Resizings are significantly smoother :+1: On the same level as native applications like Windows Explorer.

@hez2010 could you test again with the current version.

It now works without any issue.

You can test this PR using the following package version. 0.10.999-cibuild0019822-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Amazing work, just wondering, not sure if it's possible but do you plan to make it work with WinUIComposition?
Seems like the WinUICompositorConnection expects an OpenGL platform interface to create a D3D device, so it might require a bit of refactoring.

You can test this PR using the following package version. 0.10.999-cibuild0019897-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Amazing work, just wondering, not sure if it's possible but do you plan to make it work with WinUIComposition? Seems like the WinUICompositorConnection expects an OpenGL platform interface to create a D3D device, so it might require a bit of refactoring.

I currently don't have plans to support WinUIComposition.

You can test this PR using the following package version. 0.10.999-cibuild0020031-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020089-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020151-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020279-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020299-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020353-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020433-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Any plans to merge this PR?

You can test this PR using the following package version. 0.10.999-cibuild0020501-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020680-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0021026-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0021607-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

Is this unblocked now?

You can test this PR using the following package version. 0.10.999-cibuild0022173-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0022179-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0022183-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

I am happy to test this as well.
Question: thiss seems pretty far already. Is there a plan to integrate this for version 0.11?

We should not add several megabytes of extra dlls as a mandatory dependency. So Vulkan support should be completely optional, without any references to Silk.NET from either Avalonia.Skia or platform support packages
Will make a thorough review of the PR itself later.

Would it be an option not to use Silk.net at all an just do your own Vulkan bindings? Yes, that is something to maintain as Vulkan goes on but it would give you full control over things with minimal impact (and extra megabytes of DLL's). We have created our own bindings using our own code generator. We may be able to contribute such a thing.

Would it be an option not to use Silk.net at all an just do your own Vulkan bindings?

After https://github.com/AvaloniaUI/Avalonia/pull/8580 Avalonia.OpenGL.dll is only 71.4KB, so the budget for Vulkan glue code would be 100-150KB at most.

Would it be an option not to use Silk.net at all an just do your own Vulkan bindings?

After #8580 Avalonia.OpenGL.dll is only 71.4KB, so the budget for Vulkan glue code would be 100-150KB at most.

Ok, I saw what you did there. Basically, you commented unused elements. Would you then prefer to have the vk bindings as an integrated element in the Avalonia.Vulkan code such that we could reduce it to what we need? Positive: we may be able to trimm that down to a very small portion of what is in Silk.net. Negative: we need to maintain it ourselves which (I guess) is not so hard given the (small) size of the subset we need.

For OpenGL we only use a handful of functions required to set up the context and to pass it to Skia. I'd expect the Vulkan backend to do the same. Some platform abstraction to get VKInstance and the address of vkGetInstanceProcAddr, then just get a handful of function pointers via vkGetInstanceProcAddr. I won't expect that to take more than 50-100kb.
Right now it needs System.Text.Json,  Microsoft.CSharp,  System.Dynamic.Runtime, Microsoft.Extensions.DependencyModel, System.Text.Encodings.Web and other meaty dependencies.
Right now it assumes that the user would want to utilize Silk.NET instead of VulkanSharp or melinoe/vk.

For OpenGL we only use a handful of functions required to set up the context and to pass it to Skia. I'd expect the Vulkan backend to do the same. Some platform abstraction to get VKInstance and the address of vkGetInstanceProcAddr, then just get a handful of function pointers via vkGetInstanceProcAddr. I won't expect that to take more than 50-100kb.
Right now it needs System.Text.Json, Microsoft.CSharp, System.Dynamic.Runtime, Microsoft.Extensions.DependencyModel, System.Text.Encodings.Web and other meaty dependencies.
Right now it assumes that the user would want to utilize Silk.NET instead of VulkanSharp or melinoe/vk.

Understood. The problem with these 3rd party dependencies is that you quickly run out of support just like with SharpDX. For our old DX 3D renderer we therefore hosted the SharpDX software ourselves and made the according bugfixes. As a result of that we decided for our own implementation of Vk bindings (using our own code generator script) including abstraction. Since you have done that for OpenGL it would make a lot of sense to do the same for the Ava.Vulkan backend. It could be included in the very same assembly.
Again, since I am very much interested in an Avalonia vulkan backend I may be able to contribute the Vk bindings and abstraction, but I assume that @emmauss would have such a thing anyways and also understands the details much better.
Any way we can make decisions here to finalize the integration of this great work?
@kekekeks : anything else that prevents us from integrating this PR? Docu, ...?

anything else that prevents us from integrating this PR? Docu, ...?

It's not compatible with WinUI/DirectComposition output and uses HWND directly. That triggers various window resize issues since we can't tell Windows that we don't need GDI drawing and WM_PAINT.
Also, some info posted on gitter:

it also would need to be refactored to not use ISkiaGpu
the whole ISkiaGpu is designed for "3rd party addons"
it means "I'd like to ignore your GPU code completely, here is GrContext and SKCanvas, use it"
which is completely OK, but not something we want to be used from the main code base
the backend should not know about Skia
instead skia backend would have to know about some vulkan basics, just like it knows about OpenGL
vulkan support would have to register its own render targets instead of using HWND/XID
etc
as it is right now, it doesn't really fit into our project architecture
at the current point PR is usable as is, but in its current state it's essentially a 3rd party addon
which is completely OK while rendering system changes are still blocked
so we need to introduce required extension points
the problem is, I would probably have to do said refactoring myself
since it would require some non-trivial changes in non-vulkan code
so probably not before preview2

Removing 3rd party dependencies would allow us to integrate Vulkan directly to the platform code instead of adding extension points, so it would be less work to get vulkan support merged

Also, some info posted on gitter:

it also would need to be refactored to not use ISkiaGpu
the whole ISkiaGpu is designed for "3rd party addons"
it means "I'd like to ignore your GPU code completely, here is GrContext and SKCanvas, use it"
which is completely OK, but not something we want to be used from the main code base
the backend should not know about Skia
instead skia backend would have to know about some vulkan basics, just like it knows about OpenGL
vulkan support would have to register its own render targets instead of using HWND/XID
etc
as it is right now, it doesn't really fit into our project architecture
at the current point PR is usable as is, but in its current state it's essentially a 3rd party addon
which is completely OK while rendering system changes are still blocked
so we need to introduce required extension points
the problem is, I would probably have to do said refactoring myself
since it would require some non-trivial changes in non-vulkan code
so probably not before preview2

I agree. Though, I am very happy about @emmauss  having put together such an addon. It is certainly designed to be non-intrinsic into Avalonia core. It is also a great template to do your own addon. Kudos to you!
I am not the Avalonia architecture expert but I agree that any new rendering backend that shall be part of Avalonia as such, should mirror the way OpenGL is integrated, so both backends would go hand in hand and would be (if necessary) refactored in the same fashion. I would actually love to see this happening asap since this is the only true way to integrate custom Vulkan engines.

https://github.com/AvaloniaUI/Avalonia/pull/8589 introduces more generic infrastructure for obtaining function pointers. E. g. for
    unsafe partial class Test
    {
        [GetProcAddress("myFoo")]
        public unsafe partial bool Foo(int x, float y);

        public Test(Func<string, IntPtr> getProcAddress)
        {
            Initialize(getProcAddress);
        }
    }

It generates
unsafe partial class Test
{
    delegate* unmanaged[Stdcall]<int,float,bool>_addr_Foo;
    public unsafe partial bool Foo(int @x, float @y)
    {
        return _addr_Foo(@x, @y);
    }
    void Initialize(Func<string, IntPtr> getProcAddress)
    {
        var addr = IntPtr.Zero;
        // Initializing Foo
        addr = IntPtr.Zero;
        addr = getProcAddress("myFoo");
        if (addr == IntPtr.Zero) throw new System.EntryPointNotFoundException("_addr_Foo");
        _addr_Foo = (delegate* unmanaged[Stdcall]<int,float,bool>)addr;
}

IMO that should be more than enough for custom Vulkan bindings that only need 20-30 of Vulkan functions


Range outside

Enable the user to have maxValue < minValue, highlighting the space around instead of the space between.
Added a new option to support this: showOutRange

Hi Thanks for contributing to this project.
Could you provide a JSFiddle showing what does this PR please?

Hi,
I put a little demo here: https://jsfiddle.net/ekLbqcwr/8/
there is one also in the demo folder.

Hi ValentinH,
I improved a couple of things more for this case but I need to introduce a breaking change.
I push on my fork, I close this and open a new pull request?

I'm sorry but I don't understand the point of this feature... Could you explain a bit more why you need this?

I'm actually using the slider to show a range of time during the day.
You can have for example 9->18 or 18->9.

OK, now I get it. What would be the breaking change?

It's for who is using the custom template, they'll need to add another rz-bar-wrapper in the same position as here (line 6,7,8):
https://github.com/alexpireddu/angularjs-slider/blob/be655f10dbb945caaae33640d0116c7636d44140/tests/specs/custom-tpl.html

To be honest, I don't want to add this to the library because it's adding quite a bit of complexity just for a really specific use case. I would prefer that you keep this in a fork.

Yeah probably it suits good only for the time case.

Indeed and also there are many edge cases to handle with all the options that you probably don't need.. Thus, you can create you custom version with only your requirements.


operator gitlab-operator-kubernetes (0.18.0)

Thanks for submitting your Operator. Please check the below list before you create your Pull Request.
New Submissions

[ ] Are you familiar with our contribution guidelines?
[ ] Have you packaged and deployed your Operator for Operator Framework?

[ ] Have you tested your Operator in all supported installation modes?
[ ] Have you considered whether you want to use semantic versioning order?
[ ] Is your submission signed?
[ ] Is operator icon set?

Updates to existing Operators

[ ] Did you create a ci.yaml file according to the update instructions?
[ ] Is your new CSV pointing to the previous version with the replaces property if you chose replaces-mode via the updateGraph property in ci.yaml?
[ ] Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?
[ ] Have you tested an update to your Operator when deployed via OLM?
[ ] Is your submission signed?

Your submission should not

[ ] Modify more than one operator
[ ] Modify an Operator you don't own
[ ] Rename an operator - please remove and add with a different name instead
[ ] Modify any files outside the above mentioned folders
[ ] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[ ] Description of the managed Application and where to find more information
[ ] Features and capabilities of your Operator and how to use it
[ ] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[ ] Human readable name and 1-liner description about your Operator
[ ] Valid category name1
[ ] One of the pre-defined capability levels2
[ ] Links to the maintainer, source code and documentation
[ ] Example templates for all Custom Resource Definitions intended to be used
[ ] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file an issue against this repo and explain your need
2 For more information see here

/merge possible

msg=Error: Error launching source instance: InsufficientInstanceCapacity: We currently do not have sufficient m6a.xlarge capacity in the Availability Zone you requested (us-east-1d). Our system will be working on provisioning additional capacity. You can currently get m6a.xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-1a, us-east-1b, us-east-1c, us-east-1f.

/retest

/merge possible

/merge possible


[Feature] Share location

Share location between people (and show it on an integrated map maybe)

it is planned feature but not yet implemented.

No need to integrate a map. You can link to a location as geo:48.85828,2.29449. Works with Riot as-is, and will open in any good map app you have installed. (OsmAnd, Google Maps, others...)
Example:
https://matrix.to/#/#riot-android:matrix.org/%2414950160261511516pRlnj:matrix.org

one year plus later, any update on this feature?

This feature may be consider as a meta regarding the Riot universe, and maybe the Matrix universe.

@albuic Ah, I misunderstood the issue. I guess someone might be able to write an integration that does this. (Live map)
Anyway, as bmarty said, this isn't specific to the Android client. If you write an integration the proper way, it should work on all matrix clients that support integrations.

Cool would we also if people could leave location aware comments, such as "saw a rare animal here" or "road needs fixing here" etc.

There's already a PR for riot-web, Location-Events by @Half-Shot . He went for Open Street Map for map service.
Now for android . Google maps service (which require Google Play Services) seems best but there are various android devices (like with Lineage OS) which don't use Google Play Services . And Riot runs on variety of android devices .
So best solution seems to be

Integrate Both API (Google Maps and OSM)
If (Google Api is available) Use it

Please do not use anything google related if possible.
I would like to suggest Hauk - https://github.com/bilde2910/Hauk
I installed it on my server and it is used a lot already. It is good and is using OpenStreetMap and it is really easy to set up.

Hi, developer of Hauk here. If there is interest in using my project for location sharing, I'd be more than happy to help and will do my best to assist in incorporating the needed functionality on my end :-)

Hi, developer of Hemppa (a bot) here. It can translate location updates to human-readable (and clickable) links. It can also generate them from location names (via OSM api).
There is still no way to send a real GPS position from a mobile device.
https://github.com/vranki/hemppa

I do not know anything about the technical background here, but
at least I would like to give an advice (before someone implements
something terribly useless):
The XMPP client "Conversations" for Android has a "share location"
implementation that -- although minimalistic -- already covers most
of the use cases, IMHO:

You click on "Send location" ("Insert location" in the middle of
the text you were typing would be even nicer).

A map gets shown, centered on your current location (with a green
icon in the center).

You can now either just tap the "Share" button (or "Cancel") or you
can zoom and pan the map to have the green Icon centered over
the location you want to share. So if you are currently visiting
Berlin/Germany, but want to show your chat partners where you
live, you pan the map to London/England, the exact spot where your
house is located, and then press "Share", which will send the location
of your house in England. If after panning the map you want to re-center
it to your current location, just hit the green round crosshair icon in the
lower right hovering over the map.

After clicking on "Share", you'll get back to your message entry and
can now just hit the "Send" icon on the right. For both you and your
chat partner, the message will consist of a button labeled
"SHOW LOCATION".

If you you click on that button, a map is shown, centered on that very
location. It has two icons in the upper right corner for:

sharing (which lets you select things like "Copy to clipboard",
"Save as..." (from the Amaze file manager), "Conversations",
"Messages", "K-9 Mail" etc.)
Copy to clipboard (which will then contain the text
"geo:51.513624917984316,-0.10244404062649437")

Additionaly, there is a round green icon in the lower right hovering
over the map that is labeled with the typical "directions" arrow.
After clicking it, you can select a navigation app ("ZANavi, OsmAnd~,
Navit, Map, ...) which will then calcuate a route from your current
location to that location you just shared.

I like that design @mjk-gh . We could just copy that.
Another feature I would like to see is "location streaming" - tracking users movements continuously instead of sending a single location update.
For example Alice is driving a car and wants to share location with Bob:

Alice presses "Stream location" in a room with Bob
Alice's Element sends location updates automatically to room between configurable intervals. Default 1 minute?
User can end location streaming when not needed anymore

On Bob's side

Notification "Alice has started location streaming [Map button]"
Map button could be next to Alice's username in user list.
Pressing the button would open up a map with Alice's latest location shown.

Multiple users could stream their location in a room and their locations could be checked from per-user map button or room-wide button that would show all users who are sharing location. For example search and rescue situation could benefit from this.
Some things that need more thinking (perhaps more spec than Element specific stuff):

We  need a way to distinguish between single location event and a location stream.
How would this work if a user has multiple devices sending location stream? Should the streams be per device or per user (previous stream stops if new one begins?)
Should one user be able to send multiple streams? For example a bot that could stream locations of multiple vehicles.


ViewPagerAndroid undefined is not an object (evaluating '_this.props')

Hi Everyone!
Issue Description
I'm new in react-native and i'm just gonna make pager in my app.
Steps to Reproduce / Code Snippets
So I'm doing it as shown in documentation:
render() {
        return (
            <ViewPagerAndroid initialPage={0}>
                <View style={styles.pageStyle}>
                    <Text>First page</Text>
                </View>

                <View style={styles.pageStyle}>
                    <Text>Second page</Text>
                </View>
            </ViewPagerAndroid>
        );
    }

and I get this:

Maybe i'm doing something wrong!
Thank you in advance,and sorry for my english!!
Additional Information

React Native version: 0.34.1
Platform: Android
Operating System: macOS

Perhaps the same issue as #9966

Faced same issue in React Native0.35
Updated to React Native 0.36 and now working fine

Closing this issue because it has been inactive for a while. If you think it should still be opened let us know why.


[ESP32-S3] usb_serial_jtag printing skips bytes (IDFGH-9870)

Original Issue: https://github.com/espressif/esp-idf/issues/9318
Making a new issue to get more eyes on this and to summarize the issue.
IDF version.
v4.4.4
Development Kit.
ESP32-S3 Dev Kit C
What is the expected behavior?
printf should log all the characters, even when using usb_serial_jtag.
What is the actual behavior?
characters are skipped when using USB Serial JTAG, sometimes
Steps to reproduce.
void app_main()
{
    printf("\nabcdefghijk");
    int k = 0;
    // repeatedly print numbers 0 to 10240 (and value of k)
    while (true) {
        for(int i = 0; i < 1024 * 10; i++) {printf("%i,%i\n", k, i);}
        vTaskDelay(1);
        k++;
    }
}

Expected output:
> 0,0
> 0,1
> 0,2
> .....
> 0,10238
> 0,10239
> 1,0
> 1,1
> 1,2
> ...
Actual output:
> ....
> 0,2372
> 0,2373
> 0,6250
> 0,6251
More Information.
I've likely figured it out.
The skipping character problem only repros with printf & putc. It does not repro with uart_tx_char directly, or esp_rom_printf.
printf & putc are both buffered. They write to an internal buffer, and then flush when the buffer is full, or a newline is hit.
My guess is that _fflush_r is buggy. It's not waiting long enough to actually flush the data, and therefor we are printing faster than we actually can, and skipping bytes.
fflush is implemented in ROM only. PROVIDE( _fflush_r = 0x4000150c ); I don't have the source code for it.  I think fflush just needs to wait on some registers in both UART and USB Serial JTAG peripherals before returning.

@chipweinberger This is actually expected, since USB_SERIAL_JTAG driver will drop characters if the host is not picking them up fast enough. Could you try increasing TX_FLUSH_TIMEOUT_US in components/vfs/vfs_usb_serial_jtag.c and see if that makes any difference in your test?
(_fflush_r simply calls usb_serial_jtag_write, which may or may not write all the requested bytes to the output.)

My understanding is there is a subtly. If the USB host "cannot keep up" then the usb device will send bytes slower. We will only drop bytes if the host picks up zero bytes for +50ms.

perhaps we should keep a counter of when we drop bytes to make this condition easier to debug (i.e. and log the state of this counter after usb transfers resume). This is otherwise tricky to notice it is happening, or why it happens.

Could you try increasing TX_FLUSH_TIMEOUT_US

Yes Ill give this a try.

(_fflush_r simply calls usb_serial_jtag_write, which may or may not write all the requested bytes to the output.)

Can you elaborate? or share the code?

Can you elaborate? or share the code?

Here is a backtrace I see if i set a breakpoint in usb_serial_jtag_write after hitting a breakpoint in _fflush_r:
(gdb) bt
#0  usb_serial_jtag_write (fd=0, data=0x3fc8cac8, size=13)
    at /Users/ivan/e/esp-idf/components/vfs/vfs_usb_serial_jtag.c:141
#1  0x420056d2 in console_write (fd=<optimized out>, data=0x3fc8cac8, size=13)
    at /Users/ivan/e/esp-idf/components/vfs/vfs_console.c:73
#2  0x42005246 in esp_vfs_write (r=0x3fc8ee34, fd=1, data=0x3fc8cac8, size=13)
    at /Users/ivan/e/esp-idf/components/vfs/vfs.c:436
#3  0x42008f70 in __sflush_r (ptr=ptr@entry=0x3fc8ee34, fp=fp@entry=0x3fc8c884)
    at /Users/brnomac003/.gitlab-runner/builds/qR2TxTby/0/idf/crosstool-NG/.build/riscv32-esp-elf/src/newlib/newlib/libc/stdio/fflush.c:224
#4  0x42008ffa in _fflush_r (ptr=ptr@entry=0x3fc8ee34, fp=fp@entry=0x3fc8c884)
    at /Users/brnomac003/.gitlab-runner/builds/qR2TxTby/0/idf/crosstool-NG/.build/riscv32-esp-elf/src/newlib/newlib/libc/stdio/fflush.c:278
#5  0x420098ce in __sfvwrite_r (ptr=ptr@entry=0x3fc8ee34,
    fp=fp@entry=0x3fc8c884, uio=uio@entry=0x3fc8ed34)
    at /Users/brnomac003/.gitlab-runner/builds/qR2TxTby/0/idf/crosstool-NG/.build/riscv32-esp-elf/src/newlib/newlib/libc/stdio/fvwrite.c:251
#6  0x42009e18 in _puts_r (ptr=0x3fc8ee34, s=s@entry=0x1 "")
    at /Users/brnomac003/.gitlab-runner/builds/qR2TxTby/0/idf/crosstool-NG/.build/riscv32-esp-elf/src/newlib/newlib/libc/stdio/puts.c:91
#7  0x42009e8e in puts (s=0x1 "", s@entry=0x3c025478 "Hello world!")
    at /Users/brnomac003/.gitlab-runner/builds/qR2TxTby/0/idf/crosstool-NG/.build/riscv32-esp-elf/src/newlib/newlib/libc/stdio/puts.c:129
#8  0x42005dca in app_main ()
    at /Users/ivan/e/esp-idf/examples/get-started/hello_world/main/hello_world_main.c:17
#9  0x42012676 in main_task (
    args=<error reading variable: value has been optimized out>)
    at /Users/ivan/e/esp-idf/components/freertos/app_startup.c:208
#10 0x403854d8 in vPortTaskWrapper (pxCode=<optimized out>,
    pvParameters=<optimized out>)
    at /Users/ivan/e/esp-idf/components/freertos/FreeRTOS-Kernel/portable/riscv/port.c:202

_fflush_r and __sflush_r are part of Newlib, you can find the code here. These functions don't deal with timeouts, the just call write syscall (aliased to esp_vfs_write) which in turn calls the write function of the specific VFS driver. If anyone drops the bytes, that would be the VFS driver (in this case, usb_serial_jtag_write function).

The USB Host polling rate is well below this afaik (like ~15ms), so this should only happen when USB is malfunctioning of unplugged.

I think in practice the serial port driver on the host also doesn't have an infinite buffer. If the application (monitor) doesn't fetch data from the driver fast enough, i think the driver will just not empty the USB endpoint buffer.

Perhaps we should keep a counter of when we drop bytes to make this condition easier to debug (and log the state of this counter after usb transfers resume). This is otherwise tricky to notice it is happening, or why it happens.

diff --git a/components/vfs/vfs_usb_serial_jtag.c b/components/vfs/vfs_usb_serial_jtag.c
index d17a2a4d6c..264035abec 100644
--- a/components/vfs/vfs_usb_serial_jtag.c
+++ b/components/vfs/vfs_usb_serial_jtag.c
@@ -101,6 +101,8 @@ static vfs_usb_serial_jtag_context_t s_ctx = {
     .rx_func = usb_serial_jtag_rx_char
 };

+static int usb_serial_jtag_bytes_dropped = 0;
+
 static int usb_serial_jtag_open(const char * path, int flags, int mode)
 {
     s_ctx.non_blocking = ((flags & O_NONBLOCK) == O_NONBLOCK);
@@ -121,10 +123,10 @@ static void usb_serial_jtag_tx_char(int fd, int c)
         if (usb_serial_jtag_ll_txfifo_writable()) {
             usb_serial_jtag_ll_write_txfifo(&cc, 1);
             s_ctx.last_tx_ts = esp_timer_get_time();
-            break;
+            return;
         }
     } while ((esp_timer_get_time() - s_ctx.last_tx_ts) < TX_FLUSH_TIMEOUT_US);
-
+    usb_serial_jtag_bytes_dropped++;
 }

 static int usb_serial_jtag_rx_char(int fd)

Adding a counter, and running the example you have attached, I definitely see that usb_serial_jtag_bytes_dropped counter is getting incremented. (I set a watchpoint on it using watch usb_serial_jtag_bytes_dropped in GDB.)

Thanks for your debugging effort, Ivan! So, perhaps you are right all along that I just need to increase TX_FLUSH_TIMEOUT_US.

Actually, my previous test was on C3. After running your example on S3 I also can't reproduce the original issue anymore (with printf). I used Ctrl-T Ctrl-L in idf.py monitor to enable logging to file, and used this script to check the result.
Unfortunately I don't have time now to try again on C3 and find the difference. If you have further findings in the meantime, please leave a comment, I'll take a look at this issue again later.
Just to check, are you testing with CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y or CONFIG_ESP_CONSOLE_SECONDARY_USB_SERIAL_JTAG=y.

I'm using CONFIG_ESP_CONSOLE_SECONDARY_USB_SERIAL_JTAG=y

I think I need to ask you for more detailed instructions to reproduce (including maybe the software you are using on the host to receive the serial output, your OS, etc.) I am not seeing this issue with ESP32-S3 even with CONFIG_ESP_CONSOLE_SECONDARY_USB_SERIAL_JTAG=y.

I'll add usb_serial_jtag_bytes_dropped++; to my code and retest later today.

So I have some bad news. I am not able to repro this issue on my current configuration:

ESP32-S3
macOS 13.3.1
esp-idf v5
using python serial

Python:
    # this calls get_security_info using esptool.py
    jflashReboot.reboot(st)

    console = serial.Serial()
    console.baudrate = 460800
    console.port = "/dev/cu.usbmodem101"

    i = 0
    while True:
        c = console.readline().decode('ascii', 'ignore')
        print("	  | " + c, end="")
        c = c.replace("\r\n","")
        if len(c) == 0:
            continue
        if "abcdefghijk" in c:
            continue
        parts = c.split(",")
        val = int(parts[0])
        if val != i:
            print("fail " + c)
            break
        i = (i+1) % 1024

C:
void app_main(void)
{
    vTaskDelay(100 * 3);
    printf("\nabcdefghijk\n");
    // repeatedly print numbers 0 to 1024
    while (true) {
        for(int i = 0; i < 1024; i++) {printf("%i,%i\n",i,usb_serial_jtag_bytes_dropped);}
        vTaskDelay(1);
    }
}

Python logs:
          |
          | abcdefghijk
          | 0,97
          | 1,97
          | 2,97
          | 3,97
          | 4,97
          | 5,97
          | 6,97
          | 7,97

As expected, we do drop some characters in the beginning before python connects. But we never drop anymore (neither the usb_serial_jtag_bytes_dropped  variable, nor are numbers skipped in the log).
I still would not feel comfortable causing this issue gone. I repro'd it recently on the config I mentioned earlier (maybe it was a macOS 12 bug?).
But as for me, I cant be of much further assistance it seems until I can repro it again.

From b8902f24f9a2a5593a9fdaffe596b79936ceb794 Mon Sep 17 00:00:00 2001
From: michel<EMAIL_ADDRESS>Date: Wed, 3 May 2023 18:39:41 +0200
Subject: [PATCH 1/2] usb_serial_jtag: Fix tx so that bytes are not lost when
 the TX FIFO is busy

---
 .../driver/usb_serial_jtag/usb_serial_jtag.c  | 86 ++++++++++++++-----
 components/vfs/vfs_usb_serial_jtag.c          |  4 +
 2 files changed, 69 insertions(+), 21 deletions(-)

diff --git a/components/driver/usb_serial_jtag/usb_serial_jtag.c b/components/driver/usb_serial_jtag/usb_serial_jtag.c
index 75e80e2cd4..e461e04bc9 100644
--- a/components/driver/usb_serial_jtag/usb_serial_jtag.c
+++ b/components/driver/usb_serial_jtag/usb_serial_jtag.c
@@ -6,6 +6,7 @@

 #include <string.h>
 #include <stdbool.h>
+#include <sys/param.h>
 #include "esp_log.h"
 #include "hal/usb_serial_jtag_ll.h"
 #include "hal/usb_phy_ll.h"
@@ -22,26 +23,29 @@
 #define USB_SER_JTAG_RX_MAX_SIZE        (64)

 typedef struct{
-    intr_handle_t intr_handle;          /*!< USB-SERIAL-JTAG interrupt handler */
+    intr_handle_t intr_handle;  /*!< USB-SERIAL-JTAG interrupt handler */

     // RX parameters
-    RingbufHandle_t rx_ring_buf;        /*!< RX ring buffer handler */
-    uint32_t rx_buf_size;               /*!< TX buffer size */
-    uint8_t rx_data_buf[64];            /*!< Data buffer to stash FIFO data */
+    RingbufHandle_t rx_ring_buf;                    /*!< RX ring buffer handler */
+    uint32_t rx_buf_size;                           /*!< RX buffer size */
+    uint8_t rx_data_buf[USB_SER_JTAG_RX_MAX_SIZE];  /*!< Data buffer to stash RX FIFO data */

     // TX parameters
-    uint32_t tx_buf_size;               /*!< TX buffer size */
-    RingbufHandle_t tx_ring_buf;        /*!< TX ring buffer handler */
+    uint32_t tx_buf_size;                         /*!< TX buffer size */
+    RingbufHandle_t tx_ring_buf;                  /*!< TX ring buffer handler */
+    uint8_t tx_data_buf[USB_SER_JTAG_ENDP_SIZE];  /*!< Data buffer to stash TX FIFO data */
+    size_t tx_stash_cnt;                          /*!< Number of stashed TX FIFO bytes */
 } usb_serial_jtag_obj_t;

 static usb_serial_jtag_obj_t *p_usb_serial_jtag_obj = NULL;

 static const char* USB_SERIAL_JTAG_TAG = "usb_serial_jtag";

-static void usb_serial_jtag_write_and_flush(const uint8_t *buf, uint32_t wr_len)
+static int usb_serial_jtag_write_and_flush(const uint8_t *buf, uint32_t wr_len)
 {
-    usb_serial_jtag_ll_write_txfifo(buf, wr_len);
+    int size = usb_serial_jtag_ll_write_txfifo(buf, wr_len);
     usb_serial_jtag_ll_txfifo_flush();
+    return size;
 }

 static void usb_serial_jtag_isr_handler_default(void *arg) {
@@ -52,17 +56,48 @@ static void usb_serial_jtag_isr_handler_default(void *arg) {
     if (usbjtag_intr_status & USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY) {
         // Interrupt tells us the host picked up the data we sent. If we have more data, we can put it in the buffer and the host will pick that up next.
         // Send data in isr.
+
+        // If the hardware fifo is avaliable, write in it. Otherwise, do nothing.
         if (usb_serial_jtag_ll_txfifo_writable() == 1) {
             // We disable the interrupt here so that the interrupt won't be triggered if there is no data to send.
             usb_serial_jtag_ll_disable_intr_mask(USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY);
             size_t queued_size;
-            uint8_t *queued_buff = (uint8_t *)xRingbufferReceiveUpToFromISR(p_usb_serial_jtag_obj->tx_ring_buf, &queued_size, 64);
-            // If the hardware fifo is avaliable, write in it. Otherwise, do nothing.
-            if (queued_buff != NULL) {  //Although tx_queued_bytes may be larger than 0. We may have interrupt before xRingbufferSend() was called.
-                //Copy the queued buffer into the TX FIFO
-                usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY);
-                usb_serial_jtag_write_and_flush(queued_buff, queued_size);
-                vRingbufferReturnItemFromISR(p_usb_serial_jtag_obj->tx_ring_buf, queued_buff, &xTaskWoken);
+            uint8_t *queued_buff = NULL;
+            bool is_stashed_data = false;
+            if (p_usb_serial_jtag_obj->tx_stash_cnt != 0) {
+               // Send stashed tx bytes before reading bytes from ring buffer
+               queued_buff = p_usb_serial_jtag_obj->tx_data_buf;
+               queued_size = p_usb_serial_jtag_obj->tx_stash_cnt;
+               is_stashed_data = true;
+            }
+            else {
+               queued_buff = (uint8_t *)xRingbufferReceiveUpToFromISR(p_usb_serial_jtag_obj->tx_ring_buf, &queued_size, USB_SER_JTAG_ENDP_SIZE); // Max 64 data payload size in a single EndPoint
+            }
+
+            usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY);
+              if (queued_buff != NULL) {  // Although tx_queued_bytes may be larger than 0. We may have interrupt before xRingbufferSend() was called.
+                // Copy the queued buffer into the TX FIFO
+
+                // On ringbuffer wrap-around the size can be 0 even though the buffer returned is not NULL
+                if (queued_size > 0) {
+                    uint32_t sent_size = usb_serial_jtag_write_and_flush(queued_buff, queued_size);
+
+                    if (sent_size < queued_size) {
+                        // Not all bytes could be sent at once, stash the unwritten bytes in a tx buffer
+                        size_t stash_size = MIN(USB_SER_JTAG_ENDP_SIZE, queued_size - sent_size);
+
+                        // Copy the missed bytes to tx stash buffer. May copy from stash buffer to itself
+                        memcpy(p_usb_serial_jtag_obj->tx_data_buf, &queued_buff[sent_size], stash_size);
+                        p_usb_serial_jtag_obj->tx_stash_cnt = stash_size;
+                    }
+                    else {
+                        p_usb_serial_jtag_obj->tx_stash_cnt = 0;
+                    }
+                }
+                if (is_stashed_data == false) {
+                    vRingbufferReturnItemFromISR(p_usb_serial_jtag_obj->tx_ring_buf, queued_buff, &xTaskWoken);
+                }
+
                 usb_serial_jtag_ll_ena_intr_mask(USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY);
             }
         } else {
@@ -91,13 +126,14 @@ esp_err_t usb_serial_jtag_driver_install(usb_serial_jtag_driver_config_t *usb_se
     ESP_RETURN_ON_FALSE((usb_serial_jtag_config->rx_buffer_size > USB_SER_JTAG_RX_MAX_SIZE), ESP_ERR_INVALID_ARG, USB_SERIAL_JTAG_TAG, "RX buffer prepared is so small, should larger than 64");
     ESP_RETURN_ON_FALSE((usb_serial_jtag_config->tx_buffer_size > 0), ESP_ERR_INVALID_ARG, USB_SERIAL_JTAG_TAG, "TX buffer is not prepared");
     p_usb_serial_jtag_obj = (usb_serial_jtag_obj_t*) heap_caps_calloc(1, sizeof(usb_serial_jtag_obj_t), MALLOC_CAP_INTERNAL|MALLOC_CAP_8BIT);
-    p_usb_serial_jtag_obj->rx_buf_size = usb_serial_jtag_config->rx_buffer_size;
-    p_usb_serial_jtag_obj->tx_buf_size = usb_serial_jtag_config->tx_buffer_size;
     if (p_usb_serial_jtag_obj == NULL) {
         ESP_LOGE(USB_SERIAL_JTAG_TAG, "memory allocate error");
         err = ESP_ERR_NO_MEM;
         goto _exit;
     }
+    p_usb_serial_jtag_obj->rx_buf_size = usb_serial_jtag_config->rx_buffer_size;
+    p_usb_serial_jtag_obj->tx_buf_size = usb_serial_jtag_config->tx_buffer_size;
+    p_usb_serial_jtag_obj->tx_stash_cnt = 0;

     p_usb_serial_jtag_obj->rx_ring_buf = xRingbufferCreate(p_usb_serial_jtag_obj->rx_buf_size, RINGBUF_TYPE_BYTEBUF);
     if (p_usb_serial_jtag_obj->rx_ring_buf == NULL) {
@@ -161,12 +197,20 @@ int usb_serial_jtag_write_bytes(const void* src, size_t size, TickType_t ticks_t
     ESP_RETURN_ON_FALSE(src != NULL, ESP_ERR_INVALID_ARG, USB_SERIAL_JTAG_TAG, "Invalid buffer pointer.");
     ESP_RETURN_ON_FALSE(p_usb_serial_jtag_obj != NULL, ESP_ERR_INVALID_ARG, USB_SERIAL_JTAG_TAG, "The driver hasn't been initialized");

+    int ret_size = 0;
+
     const uint8_t *buff = (const uint8_t *)src;
     // Blocking method, Sending data to ringbuffer, and handle the data in ISR.
-    xRingbufferSend(p_usb_serial_jtag_obj->tx_ring_buf, (void*) (buff), size, ticks_to_wait);
-    // Now trigger the ISR to read data from the ring buffer.
-   usb_serial_jtag_ll_ena_intr_mask(USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY);
-   return size;
+    BaseType_t data_sent = xRingbufferSend(p_usb_serial_jtag_obj->tx_ring_buf, (void*) (buff), size, ticks_to_wait);
+    if (data_sent == pdTRUE) {
+        // Now trigger the ISR to read data from the ring buffer.
+        usb_serial_jtag_ll_ena_intr_mask(USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY);
+        ret_size = size;
+    }
+    else {
+        ret_size = 0;
+    }
+    return ret_size;
 }

 esp_err_t usb_serial_jtag_driver_uninstall(void)
diff --git a/components/vfs/vfs_usb_serial_jtag.c b/components/vfs/vfs_usb_serial_jtag.c
index 51a089dd05..0f45f56b0f 100644
--- a/components/vfs/vfs_usb_serial_jtag.c
+++ b/components/vfs/vfs_usb_serial_jtag.c
@@ -407,11 +407,15 @@ static void usbjtag_tx_char_via_driver(int fd, int c)
 {
     char ch = (char) c;
     TickType_t ticks = (TX_FLUSH_TIMEOUT_US / 1000) / portTICK_PERIOD_MS;
+
+    // Will return immediately if the driver is blocked because there is no USB connection
     if (usb_serial_jtag_write_bytes(&ch, 1, 0) != 0) {
         s_ctx.tx_tried_blocking = false;
         return;
     }

+    // If the last attempt with blocking write failed do not block again until
+    // a non-blocking write suceeds
     if (s_ctx.tx_tried_blocking == false) {
         if (usb_serial_jtag_write_bytes(&ch, 1, ticks) != 0) {
             return;
--
2.34.1

I've put your changes into a PR: https://github.com/espressif/esp-idf/pull/11344/files

May I leave some comments here?

The patch perhaps can't totally solve your issue, because that code only work when console is registered. (If the patch works, that is really interesting)
https://github.com/espressif/esp-idf/issues/9318 For this issue, I can't reproduce on my linux laptop, I don't know whether this related to monitor, perhaps I need to have a try on MAC.

because that code only work when console is registered.

Why is that?

Because usb_serial_jtag.c is a file that we call it driver part, it's only used in blocking mode, in idf, it's console. If you can see the code, who calls usb_serial_jtag_driver_install is only esp_console_new_repl_usb_serial_jtag
non-blocking mode is handled in vfs_usb_serial_jtag.c, I'm figuring out what happens.

BTW, may I know what computer(os) you're using? @chipweinberger

I use a Macbook pro, M1,  MacOS 13.3.1 (22E261)

Because usb_serial_jtag.c is a file that we call it driver part, it's only used in blocking mode, in idf, it's console. If you can see the code, who calls usb_serial_jtag_driver_install is only esp_console_new_repl_usb_serial_jtag

That is a good insight. Yes, I was able to reproduce these issues without the console registered.


MM-15431 Removing apple cert configmaps and replacing them with k8s secrets

Jira Ticket -> https://mattermost.atlassian.net/browse/MM-15431

The current helm chart is using configmaps to create the pem files for Apple certs.
The update replaces configmaps with K8s secrets.

After the PR merge the new push proxy helm chart version will be pushed to the production k8s cluster and will be used by Mattermost community.

@cpanato It is tested but not yet deployed in prod. I was waiting for the PR review. I will deploy it later today and if all good, then I will merge this ;)

@stylianosrigas but did you test the upgrade?

@cpanato I created a new cluster where I deployed the old push-proxy helm chart and then I upgraded using my new chart. It upgraded smoothly with no interruption in the notifications.


Deserializing runtime types

Sorry if this is not the right place to ask, but I've been stuck with this problem for a while and I'm not sure if it's possible to achieve this using Serde.
I have what is effectively a dynamic (runtime) representation of a struct. I want to use this representation to deserialize some data. To my understanding, the typical solution would be to treat it as a map and use deserialize_map alongside a custom Visitor, however I cannot do that as the serialization format I'm using has length-prefixed maps. The data I'm interested in deserializing does not treat these types as maps, and so there's no length prefix. Although serialize_struct is pretty much what I want, I can't use that either as the names of the members are not &'static str.
More concretely, here's a condensed example:
enum Kind {
    Struct,
    Number,
}

// Using this representation of the struct, I know the types and
// names of each member.
struct MyStructRepr {
    name: String,
    members: Vec<(String, Kind)>,
}

// The actual data into which I want to deserialize the input.
enum Data {
    Struct {
        members: Vec<Data>,
    },
    Number(usize),
    String(String),
}

Is this doable with Serde?

You can try to do it in two steps. If I understood the problem correctly, you can try to Implement Deserialize for MyStructRepr, which closely matches original data. Then implement Deserialize for Data, which calls MyStructRepr::deserialize(deserializer)?, then map intermediate representation to wanted one. Or even maybe without two Deserialize impls, your custom Visitor can return type which is just extracts data which you can map later to expected result.

@Rulexec Unfortunately that doesn't address my issue.
MyStructRepr is just a way to represent the definition of a type. For example, consider the following type:
struct Foo {
    my_member: i32,
}

The same type could instead be represented as:
let foo = MyStructRepr {
   name: "Foo".to_string(),
   members: vec![("my_member".to_string(), Kind::Number)]
};

The point here is that the definition of the type is not available at compile time, as these are dynamic types created at runtime. MyStructRepr should not implement Deserialize; it should only be used as a way to drive the deserializer, i.e. tell the deserializer what comes next. This leads me back to my initial problem: driving a deserializer based on runtime information.
I cannot use deserialize_struct, as it requires that the names of all member fields are &'static str (something they are not in my case), and I cannot use deserialize_map as the serialization format I am using encodes maps differently from structs. I can make deserialize_struct work by leaking the strings, but I'd rather not do that.

MyStructRepr should only be used as a way to drive the deserializer,

This looks like MyStructRepr is itself should be (part of) deserializer, but I under impression that you want to deserialize that type from another deserializer.
The deserialize_* methods called by the deserializable types, so if MyStructRepr is a deserializer, then what you want should be possible.

@Mingun

under impression that you want to deserialize that type from another deserializer.

Yes, that is what I'm trying to achieve. MyStructRepr is not a deserializer itself, it just contains information about the type that should be used to drive another deserializer. To make things more tricky, it has to work with different deserializers, including e.g. serde_json.

I'm trying to implement a similar workflow for postcard and am also running into &'static str limitations: https://github.com/serde-rs/serde/issues/2218#issuecomment-2523876811

Unlike #2218, removing the &'statics from Deserializer methods seems more tractable since it doesn't need to affect any return types (those are determined by the Visitor, which does not receive &'statics). The naive transformation from &'static _ -> &_ in Deserializer::deserialize_*() methods seems to "just work" with minor tweaks to Deserializer implementations. If serde 2.0 were to ever happen, this seems like it could be changed with relatively little upgrade burden.

If serde 2.0 were to ever happen, this seems like it could be changed with relatively little upgrade burden.

That's not something that can be done without a huge amount of churn. We have no plans to do so


Block multiple members to make changes to the same file at the time to avoid merge conflict at max.

We need a way/feature (like in MS excel) to show in IDE itself if multiple members has got the same file open at the same time and at which line they are working. Also block members to make changes to the same file at the same time which would definitely reduce the chances of merge conflict a lot. The IDE should be able to do this with help of the online code repository (gitlab, github etc).
This will be a very very useful feature. Kindly do not ignore this and this would save a huge amount of time of developers. Also VS Code code be more user friendly and preferred by devs.

Hello @lszomoru, could you please have a look into this feature request. : )

@VSCodeTriageBot it seems that the issue is not being look into. Is there a possibility to get it reviewed at the earliest possible.

Thank you very much for filing this issue. What you are describing aligns with the principles of a centralised source control system where files can be locked. This feature request is out of scope for VS Code but it is something that could be explored as a potential VS Code extension.


Add generic IR symbol deduplication (vm.rodata, flow.executable, etc)

During all of our compilation phases there are symbols generated that may have duplicates. On the simpler end for example several vm.rodata symbols may have the same exact byte contents (coming from the same UTF8-encoded string, etc). Moving higher in the stack there is duplication that is trickier to determine that would be nice to handle in a generic way. Several flow.executables may contain the same exact IR snippets (and is likely if the input IR was generated by library calls/etc).
A DuplicateSymbolFolding pass that ran on a region (module, etc) that did IR diffing could handle all of these cases.

I'm not sure that a "generic" pass for all of this stuff is feasible/advisable, but passes that dedup flow.executable ops and vm.rodata ops individually may help in some cases. I don't, however, see a lot of big resnet low hanging fruit here:

Of the 53 convolutions, once partitioned, I find 2 that are redundant.
Depending on how partitioned, there are maybe ~20 regions (~20% of the whole) of mostly duplicated elementwise/broadcast ops. I haven't analyzed completely, but without further work, few of these are trivial dups; however, ~all of them could be dups if they were de-specialized, replacing static dims with dynamic dims. We'd need a cost model for that, but it would probably be safe to do this in most cases (?)
Once compiled down to the vm.rodata level, there are no duplicates. A manual inspection revealed a lot of near dups, differening by two words. I verified a couple of these and they come from the elementwise regions mentioned above.

It seems that if optimizing on this axis, some level of deduping will bear fruit, although I am dubious as to how much it helps resnet specifically. Surely, though, if we don't have anything to do this deduping, we'll lack the motivation to do other optimizatation (mostly de-specialization) that ultimately yield benefits.
Given that, there are three levels where we could do this:

flow.executable
hal.executable
vm.rodata

Number 1 is annoying, because at this level, the executable still contains arbitrary IR, requiring a full recursive equivalence check. Possible, but annoying. Number 2 can be easier because all of the non-trivial parts have been compiled at this point, and all of that just becomes an attribute compare. In both cases, there is some internal symbol variability that, if we made it canonical, we could just directly compare the IR. Right now, there would be several special cases due to non-load-bearing differences.
Number 3 is probably too late, but valuable as a final fallback (likely useful for any global constants that should be deduped): code has already been emitted to initialize each executable, so simply deduping the symbols might save some binary size but would not help with initialization time/overhead because they would just alias the backing data, not undo the initialization code.
So leaving out priority of doing this, I would probably implement:

Some canonicalization cleanups to executable symbols so that they do not trivially diff, making matching easier.
A hal.executable or flow.executable deduper (tbd which one is better).
An rodata deduper (which is pretty trivial) to catch any final things that are easily aliases together.

Also, since size/startup overhead is not the primary thing we are losing on right now, I'd be tempted to back burner this until later.

yeah rodata is pretty much classic link-time deduping -- too late to change things but helps with binary size.
flow.executable is the easiest to modify/despecialize as we have a good deal of flexibility in shapes and constants (it's trivial to pull them out). A fancy cost function would be nice, but in a lot of cases we can make some simple decisions at the HLO level that we know are safe (making outer dimensions used by convs dynamic, etc). Basically, anything that would end up just changing the invocation count (workgroup count) of the dispatch should be free. If our convs are all of different shapes but similar kernel sizes that should gain much more deduplication.
One nice thing about running deduplication at the various stages is that in some cases what isn't easily identifiable as a duplicate early on may become a duplicate later - for example, if one has a shape of x4 and another x16 but the backend always pads to x16. That avoids the need for super involved higher level despecialization as just ensuring good canonicalization while lowering will help.
Something you can check is ensuring the rodata doesn't contain names. For example, I know that the function name is based on the original outlined dispatch region name, which then gets exported as the entry point. That'll cause the rodatas to differ as they then contain the unique name of the source function. Making every flow.executable entry point name "main" or something would be a good sanity check on that.

Another thing to try is disabling RematerializeDispatchConstantsPass - that is what currently takes a dispatch region with no constant values (only ops acting on input args) and inlines some of the constants - deduping prior to that may yield a better hit rate.

The startup time is most pronounced on Vulkan (where we need to compile each shader with the driver) - 120ms on desktop for me for resnet and probably much more on mobile. We'll want to multithread that and add caching, but caching isn't guaranteed and it's still a cold start issue. #1587 & co that link executables together will help with LLVM AOT/VMLA (as all will end up in a single shared object/VM module) but in SPIR-V land the shaders will stay separate. Not a high priority vs other things as you say, but will eventually be important and could make use of this if we are getting the higher level parts in place anyway.

Ok, I'll go ahead and implement this, along with some utility functions that may prove useful. I think that as we get sophisticated in other ways, having these passes in there will mean that we see the results, which creates the right ramps to optimize further.

Looking at this dump of resnet: https://gist.githubusercontent.com/benvanik/280d94acaecbfb577984f16b391aedc7/raw/bcccb340afbe4810d15cf6285fa4ea4b22654873/resnet%2520regions
I went through and did a simple textual comparison of regions (select inner contents, ctrl-f);

etc etc
this is with the old identify dispatch region code - I'm not sure if you were looking at it with your new identify dispatch region 2 algorithm (that may pessimize things), but there's definitely a lot of duplication an IR diff dragnet should be able to catch.
Note that this is prior to running the RematerializeDispatchConstantsPass as mentioned above - after that runs there's a much higher chance that the region bodies will differ by small constant values, and a pass that ran after dedupe to find all dispatches and compare common constants (or only do RematerializeDispatchConstantsPass on flow.executables after outlining) would fix that.

Some thoughts on discord as to additional reasons to generalize deduplication (in the short term too):
https://discordapp.com/channels/689900678990135345/760577505840463893/777429896270970890
https://discordapp.com/channels/689900678990135345/760577505840463893/777447747091431434


[question] EIP1559 adoption on useDApp

What do we need to start using metamask with EIP1559? Maybe we can attack this new feature available also in many testnets.

With the latest version of ethers it should be now fully supported.
Closing for now, feel free to reopen if you think anything is lacking.


Brenle mailboxplan updates

Learn Build status updates of commit 1fb7e2e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-MailboxPlan.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 74e0243:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: CannotMergeCommit] Cannot merge commit 74e02435fa04f1dcb3a3f992d5d42783e106880a in branch brenle-mailboxplan-updates of repository https://github.com/brenle/office-docs-powershell into branch main (commit adae34e5f53489905621be1a48b5c0ba4bf788a3). Please follow this documentation: https://help.github.com/articles/resolving-a-merge-conflict-using-the-command-line/ to use git.exe to resolve you content conflicts locally and then push to remote.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Learn Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit ddbacac:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: CannotMergeCommit] Cannot merge commit ddbacac57d31d4826009a4d42eb091b89cc52b69 in branch brenle-mailboxplan-updates of repository https://github.com/brenle/office-docs-powershell into branch main (commit adae34e5f53489905621be1a48b5c0ba4bf788a3). Please follow this documentation: https://help.github.com/articles/resolving-a-merge-conflict-using-the-command-line/ to use git.exe to resolve you content conflicts locally and then push to remote.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Learn Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit f2b0764:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: CannotMergeCommit] Cannot merge commit f2b07644f2da8874e4056e6d563612454b3dbbfc in branch brenle-mailboxplan-updates of repository https://github.com/brenle/office-docs-powershell into branch main (commit adae34e5f53489905621be1a48b5c0ba4bf788a3). Please follow this documentation: https://help.github.com/articles/resolving-a-merge-conflict-using-the-command-line/ to use git.exe to resolve you content conflicts locally and then push to remote.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Learn Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit ee72d10:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-MailboxPlan.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit facc705:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-MailboxPlan.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit eeab1d6:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-MailboxPlan.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/Set-RetentionPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


TypeScript NPM Package?

I see that the Tools/cdm-types/cdm-types.ts file exists, but including it in a project that requires stricter linting rules leads to compilation issues due to the loose use of "any".
Is there an NPM package published with the type definitions? If so, can you provide the package name here and somewhere prevalent in the documents?
If not, can you publish it up ASAP to help drive adoption?

npm package was published recently. thanks for the nudge :)

Thanks @JeffBern


The transition does not work with dynamic routes

It works on my side, but only in development mode, after I restart development server. But for some reason when I push it to production on Vercel, it does not work properly - the SSG pages are not cached via edge CDN, but are server via serverless functions, and the view transition effect is not working at all.

Same issue

Same ...

I had the same issue. My problem was the layout.tsx (with  element) was in the same folder as the page.tsx . I needed to create a layout 1 level up and move the ViewTransitions there.


didnt work with express and ejs

idk wtf is wrong with the code so that it wont show correctly on my bot dashboard?
it works fine when i open it with  live server

Looks like the stylesheet isn't being loaded, can you check your browser console for any errors? Also check if the stylesheet is loaded by having a peek into the DevTool's network tab.

TLDR; You don't have styling because the CSS isn't loaded, as mentioned by HexM7. You should see an error in the console about that.
I see your path is /dashboard/:guildID/embed. On (or near) line 16 in index.html, there's a HTML link like this
<link rel="stylesheet" href="./assets/css/index.css" />

Because your path is /dashboard/:guildID/embed, this link actually points to /dashboard/:guildID/embed/assets/css/index.css. It fails because the path /dashboard/:guildID/embed/assets/css/index.css does noo exist on your server.
You could fix this by either manually editing that line to add path to your CSS, or in Express, you can make a route for /assets/css/index.css that redirects (res.redirect) to your actual CSS file


Adding " Tamil Translation"

Is your feature request related to a problem? Please describe.
#enhancement
I checked lang folder there is no translation in Tamil language.
Describe the solution you'd like
I would like to add translation in tamil language
Describe alternatives you've considered
No response
Additional context
No response

@Sanjaybhattwebkul i would like to add tamil translations

@Shrey-Nahar
You can create a Pull Request for Tamil Translation, it would be appreciated.

@Sanjaybhattwebkul
what all should be updated apart from

auth.php
pagination.php
passwords.php
validation.php

what other files should I update after doing changes in the above files.

Hi there,
Please add the translation for the following packages.
https://github.com/bagisto/bagisto/tree/master/packages/Webkul/Admin/src/Resources/lang/
https://github.com/bagisto/bagisto/tree/master/packages/Webkul/Core/src/Resources/lang/
https://github.com/bagisto/bagisto/tree/master/packages/Webkul/Shop/src/Resources/lang/
https://github.com/bagisto/bagisto/tree/master/packages/Webkul/Velocity/src/Resources/lang/

working on it ,do add "in progress" label.

Hi @Shrey-Nahar
Any update?


Billing modal issue.

Tries to fix the billing modal issue.

@elisalimli did this work for you? Or you weren't able to reproduce?

Still I could not reproduce the error in my local. But should fix the issue. If I'm not mistaken, you were also experiencing the issue in your local too. Please check if it's fixed.


Cannot find module @prisma/client in Vercel deployment

I tried deploying a dummy application to vercel using graphql-yoga and nexus-prisma and i keep getting this runtime error on vercel.
Cannot find module '/vercel/path0/node_modules/@prisma/client'
Require stack:
- /var/task/apps/graphql/node_modules/.nexus-prisma/index.js
- /var/task/apps/graphql/node_modules/nexus-prisma/dist-cjs/entrypoints/main.js
- /var/task/apps/graphql/src/types/user.js
- /var/task/apps/graphql/src/queries.js
- /var/task/apps/graphql/src/utils/schema.js
- /var/task/apps/graphql/src/utils/yoga.js
- /var/task/apps/graphql/src/index.js
Did you forget to add it to "dependencies" in `package.json`?
RequestId: 9a9af12a-0496-468a-925d-d70e72c7710a Error: Runtime exited with error: exit status 1
Runtime.ExitError

My types/user.ts file is just a copy from the documentation and looks like this:
import { objectType } from 'nexus'
import { User } from 'nexus-prisma'

const UserType = objectType({
  name: User.$name,
  definition(t) {
    t.field(User.id)
    t.field(User.email)
    t.field(User.name)
    t.field(User.createdAt)
  },
})

export default UserType

Some info in package.json that might be relevant:
{
  "name": "graphql",
  "version": "1.0.0",
  "main": "src/index.ts",
  "scripts": {
    "build": "sucrase ./src -d ./dist --transforms typescript,imports",
    "start": "node dist/index.js",
    "postinstall": "prisma generate"
  },
  "dependencies": {
    "@prisma/client": "^4.13.0",
    "graphql": "^16.6.0",
    "graphql-yoga": "^3.9.1",
    "nexus": "^1.3.0",
    "nexus-prisma": "^1.0.5",
    "prisma": "^4.13.0",
  },
}

note: i'm using yarn since i found previous issues similar to this with pnpm but it's not really my issue here

@skittlesaur for some reason prisma generator returns location of @prisma/client in /vercel/path0/node_modules/@prisma/client instead of /var/task/apps/graphql/node_modules/@prisma/client.  Can you share the whole setup?

This is the concerning logic https://github.com/graphql-nexus/nexus-prisma/blob/6af78592ca3727c05455cb11a6a90a3724299548/src/cli/nexus-prisma.ts#L123

@skittlesaur for some reason prisma generator returns location of @prisma/client in /vercel/path0/node_modules/@prisma/client instead of /var/task/apps/graphql/node_modules/@prisma/client. Can you share the whole setup?

sure, it's just a template im setting up for my future projects so it barely has any code. here's the link to the repo
https://github.com/skittlesaur/my-turbo-template

@skittlesaur for some reason prisma generator returns location of @prisma/client in /vercel/path0/node_modules/@prisma/client instead of /var/task/apps/graphql/node_modules/@prisma/client. Can you share the whole setup?

sure, it's just a template im setting up for my future projects so it barely has any code. here's the link to the repo https://github.com/skittlesaur/my-turbo-template
the graphql project im trying to deploy is located in apps/graphql

Can you somehow get the information where is the real location of  @prisma/client once deployed to vercel ?

I'm not really sure how i can do that. however, in the build logs has the following lines of prisma client generation  and nexus if that's helpful
✔ Generated Prisma Client (4.13.0 \| library) to ./../../node_modules/@prisma/client in 50ms
19:43:57.941

✔ Generated Nexus Prisma to ./node_modules/.nexus-prisma in 10ms
19:43:57.942

Setting it to prismaClientImportId: '@prisma/client' works now :D
I'm quite curious what was actually happening and how did that fix it because i been looking for an answer to this for a couple of days now and delaying my uni projects
And thank you so much for the help! :D

I'm not familiar with the Vercel internals in relation to your setup. But the problem is that runtime @prisma/client is not in the same place when generate was called (build phase) as in runtime. It could be related to your mono repo with your build tooling.


router mode always be hash in ie9

Whatever the router mode set in nuxt.config.js , it's always be hash in ie9, is it nuxt.js error or vue router?

IE 9 don't support html 5 history mode
it's change to hash mode automatic .


fix(behavior_path_planner): keep shift points in the lane

Signed-off-by: beyza<EMAIL_ADDRESS>Description
Fixes: #2208
This PR rechecks if the shift points are inside the drivable area.

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@beyzanurkaya Thank you for the PR. Would you make the description more concrete?

This PR rechecks if the shift points are inside the drivable area.

Actually, the drivable area is generated by the same module behavior_path_planner. The drivable area is extended in the avoidance scenario so that the vehicle can avoid an obstacle across a lane boundary.
Your implementation seems "clamp the shift length not to over the lane", but it restricts to avoid an obstacle over lanes.

@mehmetdogru will share a scenario to reproduce the bug related to this issue & PR.

@xmfcx
Unfortunately, I couldn't reproduce the problem we have on this specific turn using planning simulator. I guess we will need to record some bag files to present the issue.

@mehmetdogru @beyzanurkaya in original issue, @maxime-clem mentioned about another PR that might fix the issue, which is already merged now.
https://github.com/autowarefoundation/autoware.universe/issues/2208#issuecomment-1302850731
If it is already solved, do you think we want to close this PR&Issue, or would like to run more tests to make sure that it is solved completely?

@mitsudome-r
Since I can't reproduce the error with planning_simulator I really can't say if it is solved or not. We will record some rosbags to explain and understand the issue if it still exists.

@mehmetdogru Thanks we will wait for the rosbag then.


Added 21-Merge-Two-Sorted-Lists.kt

File(s) Modified: 21-Merge-Two-Sorted-Lists.kt
Language(s) Used: kotlin
Submission URL: _https://leetcode.com/submissions/detail/768376496/_

Thanks, @Raven1233!


LE read remote features

I think we can share BR/EDR and LE behavior in the same function. However, BR/EDR type is a little bit complicated(3+1 pages), so returning them might not be so useful.

BTW, why we need this feature is because sending LE data without performing feature/version exchange is very easy to fail (on the first packet).


Create comparison page

Checklist

[X] I have searched the issues.
[X] I have searched the docs.

❓ Questions and Help
Create documentation page where we compare etna library with other open source libraries out there.

Will work on it in community track


Testing/gpu

Before submitting
Please complete the following checklist when submitting a PR:

[x] All new features must include a unit test.
If you've fixed a bug or added code that should be tested, add a test to the
tests directory!

[x] All new functions and code must be clearly commented and documented.
If you do make documentation changes, make sure that the docs build and
render correctly by running make docs.

[x] Ensure that the test suite passes, by running make test.

[x] Add a new entry to the .github/CHANGELOG.md file, summarizing the
change, and including a link back to the PR.

[x] Ensure that code is properly formatted by running make format.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.

Context:
L-Kokkos supports GPU execution with CUDA-12 and L-GPU is on the verge of being merged into pennylane-lightning. There are no GPU tests to back these backends however.
Description of the Change:

Add GPU test workflow.
Combine test and codecov builds for efficiency.
Introduce a few keywords to enhance robustness in the CI (e.g. fail-on-cache-miss).
Remove deprecated ENABLE_KOKKOS option from doc.

Benefits:
Possible Drawbacks:
Combining test and codecov builds might have issues due to AVX kernels (to be monitored).
Related GitHub Issues:

Awesome work @vincentmr Does this mean we can use CUDA 12 with LKokkos now?

Yes. In fact, CUDA-12 is now required to compile L-Kokkos as it uses C++20 features (not just a CUDA-lib API like L-GPU).


a hint should be added for the user to disable Yggdrasil before connecting with mycelium

Description
https://github.com/threefoldtech/tfgrid-sdk-ts/issues/2186
through the test run found out before connecting to mycelium vm user should disable yggdrasil
using
systemctl stop yggdrasil on linux
'launchctl kill yggdrasil' on mac
may a clear msg help the user and save time to connect easily ( and it may be added to mycelium docs as well)

may be added here

no need to do this as they can work together


⚠ Bofrost.de has degraded performance

In 3bfd0f9, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 887 ms

Resolved: Bofrost.de performance has improved in 31c63a9 after 9 minutes.


Update import_map.json to reference new sdk version

Update import_map.json to reference new sdk version

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


colsample_bytree vs feature_fraction

It seems like feature_fraction and colsample_bytree refer to the same hyperparameter, but when using the python API with 2.0.10, colsample_bytree is ignored (or perhaps overridden):
parameters = {
    'application': 'binary',
    'objective': 'binary',
    'metric': 'auc',
    'is_unbalance': False,
    'boosting': 'gbdt',
    'verbose': 1
}

param_grid = {
    'learning_rate': [0.05,],
    'n_estimators': [200],
    'max_bin': [200,],
    'max_depth': [-1],
    'num_leaves': [20],
    'min_data_in_leaf': [10,],
    'feature_fraction': [0.2, 0.5, 1.0],
    'colsample_bytree': [0.2, 0.5, 1.0],
    'bagging_fraction': [0.8],
    'bagging_freq': [20,],
}

K = 5
skf = sklearn.model_selection.StratifiedKFold(n_splits=K, random_state=3228, shuffle=True)
skf.get_n_splits(x_train, id_train)

estimator = lgb.LGBMClassifier(n_jobs=1, **parameters)
gbm = sklearn.model_selection.GridSearchCV(estimator, param_grid, n_jobs=5, cv=skf, verbose=2, scoring=gini_score)

gbm.fit(x_train, y_train, categorical_feature=[cn for cn in x_train if cn.endswith('_cat')])

for p, score in zip(gbm.cv_results_['params'], gbm.cv_results_['mean_test_score']):
    print p['colsample_bytree'], p['feature_fraction'], score

Results in:
0.2 0.2 0.275032985584
0.2 0.5 0.276042409395
0.2 1.0 0.264998474875
0.5 0.2 0.275032985584
0.5 0.5 0.276042409395
0.5 1.0 0.264998474875
1.0 0.2 0.275032985584
1.0 0.5 0.276042409395
1.0 1.0 0.264998474875

It might be helpful to clarify the behavior (which takes precedence, etc).

@synapticarbors
it is because you use both feature_fraction and colsample_bytree. They are refer to the same config, and LightGBM will ignore the one with longer name.
If you only only one of them, both of them can work.

But then... why have two at all?!

they are the same thing, just alias


Auto Register stores dir in all layers

Currently if user did not specify custom storesDir then the expected to auto import stores from stores dir but this is not working inside layers

Added automatic registration of the stores directory across all Nuxt layers.
Updated the documentation to clarify configuration handling with Nuxt layers.

will this still register the stores dirs defined in layers if the main app defines it's own storesDirs? It seems like that null check would make it only apply the layers stores when the parent app sets it to null.

The options are resolved and merged from both the main app (layer) and all other layers. So If the user has defined storeDirs in either the main layer or any additional layer, there is no need to fallback to defaults, as this was the old behavior also

This looks very good; thanks for writing the patch. I was just hitting this layer issue.
Why change the falsey check into a null check? If you want to keep it so, perhaps use ===.
(I wonder why the second if-statement isn't an else. Might want to clean that up while at it.)

I made this wrapper module to work around this issue: https://gist.github.com/tommie/1e505f3f7a281698bdb9fdc7a303e6c0

Thank you

Why change the falsey check into a null check? If you want to keep it so, perhaps use ===.
I thought that !options..storesDirs is not clear enough and since the need for the check for undefined and null i did ==


Update Blazor package versions to match the .NET Core 3.0 versions

We still need to figure out something to not push 3.0.0 rtm versions to nuget, but that can wait.


Validation and Schema

As pointed out by @daussho in #177 it is a good idea to implement a Schema feature to make validation with features like "unique" possible.
That schema could include many conditions to validate new documents before inserting them:

required
unique
int
float
min
minLength
maxLength
array
string
email
numeric
AND MUCH MORE...

Interesting libraries are:
https://github.com/laravel/framework/tree/8.x/src/Illuminate/Validation
Documentation:
https://laravel.com/docs/8.x/validation
AND:
https://github.com/rakit/validation

@Timu57
I'm considering trying SleekDB as well as https://github.com/nette/schema.
It could be a good team, right?

Hi @Timu57
Sure, SleekDB does nothing schema related, so if you want to use another library to enforce a schema, you are free to do so :relaxed:
SleekDB doesn't restrict you in any way :grin:

@Timu57 Yeah!
It's just I thought it was the perfect libraries to get inspired for what you wanted to do ;)

Hi. I’m pretty new to this.
If this is “discussion” then I’d like to add I really dislike DB (nerds) way of thinking about “types”.
Laravel way of validation is very bad as well.
It’s all very nice and dandy to validate if a color is a string or an int but it’s useless.
And it’s all cool and great to store an age in an int or in a bigInt but next year I’ll be a year older. My birthday will stay the same forever, as the color of my eyes, but my hair will turn grey, and my address has a country, city, zipcode, street, house number in some countries, and others don’t. But I can move. Some house numbers have letters. Some addresses are a description on how to get there. With 2 large numbers any point on earth can be defined.
Far to often the DB validation of values is based on the way it should store it, and not about what the value represents in real life.
How fluid is the value? How is it changeable towards time? How should the front end render it?
If you’re going to store everything as a string I don’t see any reason to validate the values towards 1980’s types (int/long/double/string).
You will still be able to store 42 as a string in the variable “color”… and you will still have no idea how to render it, or if will be 42 forever or just today. Or is it even a color or an index to a color?
Just my 2 cents.
I’ve been writing about 7 years now on software that collects data from websites and haven’t found any software that understands values.
And all solutions keep asking the developer to help the software on how it should store the data, it’s 2021 for heaven sake.


New Project Flow using AbstractNewProjectStep

Enables the user to create a new project on clicking the "New Project" button through the Welcome Screen.
Fixes #24

This new commit updates most of the files that were pushed earlier, following the flow similar to that of Pycharm. This allows us to have more control over the New Project Wizard's UI as well as it's internal settings. The earlier flow restricted us to add "Project Name" in the creation window as well as assumed "empty settings" when it was tested using Breakpoints. This commit fixes all those.
I have tried to use Kotlin when creating new classes, but also used Java at some places (like Pycharm) to ensure code consistency especially where the base classes are in Java.
Some #TODOs have been included  in the  areas where I anticipate we'll update the flow in the future.
This is how the new UI looks -
@emrekultursay apologies for such a large PR. I tried to reduce it as much as possible, but LMK if it becomes difficult to understand. PTAL at your convenience.