sdl_glimp: detect broken OpenGL context returned by SDL

sdl_glimp: detect broken OpenGL context returned by SDL
This is meant to detect bugs like:

https://github.com/DaemonEngine/Daemon/issues/945

When this happens, SDL2 returns a glContext pointer that is not equal to nullptr but doing things like glGetString( GL_VERSION ) returns a nullptr…
Unvanquished 0.54.0 Linux amd64 Nov 30 2023
cmdline: -pakpath /home/illwieckz/.local/share/unvanquished/base/pkg -set logs.suppression.enabled off
[New Thread 0x7ffff115a6c0 (LWP 1386737)]
[FS] Lib path: Unvanquished/daemon/build
[FS] Home path: /home/illwieckz/.local/share/unvanquished
[FS] Pak search path: /home/illwieckz/.local/share/unvanquished/base/pkg
[FS] Pak search path: /home/illwieckz/.local/share/unvanquished/pkg
][New Thread 0x7ffff01ff6c0 (LWP 1386738)]
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/unvanquished_0.54.0.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/tex-common_2.3.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/res-players_0.54.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/res-legacy_0.54.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/res-weapons_0.54-dirty.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/res-buildables_0.54.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/res-voices_0.54.dpk'...
[FS] Loading pak '/home/illwieckz/.local/share/unvanquished/base/pkg/res-soundtrack_0.54.dpk'...
execing 'default.cfg'
execing 'presets/input/wasd.cfg'
execing 'autogen.cfg'
execing 'keybindings.cfg'
----- Client Initialization -----
Loading RSA keys from pubkey
Daemon RSA public-key found.
----- Client Initialization Complete -----
IP: <IP_ADDRESS>
IP: <IP_ADDRESS>
IP: <IP_ADDRESS>
IP6: ::1
IP6: fe80::1ac0:4dff:fe8e:948%enp197s0
IP6: fe80::c03:daff:fe05:57f0%br0
Opening IP6 socket: [::]:*
Opening IP socket: <IP_ADDRESS>:*
--- Common Initialization Complete ---
Calling GetRefAPI…
SDL_Init( SDL_INIT_VIDEO )...
Using SDL version 2.26.5
SDL using driver "x11"
Initializing OpenGL display
Using GLEW version 2.2.0
Display aspect: 1.778
...setting mode -1: 1920×1080
[New Thread 0x7fffca5cb6c0 (LWP 1386742)]
[New Thread 0x7fffc9cca6c0 (LWP 1386743)]
[Thread 0x7fffca5cb6c0 (LWP 1386742) exited]
[New Thread 0x7fffca5cb6c0 (LWP 1386744)]
[New Thread 0x7fffcaef46c0 (LWP 1386747)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 3.2 core
][Thread 0x7fffcaef46c0 (LWP 1386747) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386748)]
[Thread 0x7fffcaef46c0 (LWP 1386748) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386749)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 2.1 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386749) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386750)]
[Thread 0x7fffcaef46c0 (LWP 1386750) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386751)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 2.0 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386751) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386752)]
[Thread 0x7fffcaef46c0 (LWP 1386752) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386753)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 1.5 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386753) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386754)]
[Thread 0x7fffcaef46c0 (LWP 1386754) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386755)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 1.4 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386755) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386756)]
[Thread 0x7fffcaef46c0 (LWP 1386756) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386757)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 1.3 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386757) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386758)]
[Thread 0x7fffcaef46c0 (LWP 1386758) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386759)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 1.2 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386759) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386760)]
[Thread 0x7fffcaef46c0 (LWP 1386760) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386761)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 1.1 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386761) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386762)]
[Thread 0x7fffcaef46c0 (LWP 1386762) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386763)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 24-bit OpenGL 1.0 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386763) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386764)]
[Thread 0x7fffcaef46c0 (LWP 1386764) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386765)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 3.2 core
][Thread 0x7fffcaef46c0 (LWP 1386765) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386766)]
[Thread 0x7fffcaef46c0 (LWP 1386766) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386810)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 2.1 compatibility
][Thread 0x7fffcaef46c0 (LWP 1386810) exited]
[New Thread 0x7fffcaef46c0 (LWP 1386833)]
[Thread 0x7fffcaef46c0 (LWP 1386833) exited]
[New Thread 0x7fffcaef46c0 (LWP 1387415)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 2.0 compatibility
][Thread 0x7fffcaef46c0 (LWP 1387415) exited]
[New Thread 0x7fffcaef46c0 (LWP 1387436)]
[Thread 0x7fffcaef46c0 (LWP 1387436) exited]
[New Thread 0x7fffcaef46c0 (LWP 1387994)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 1.5 compatibility
][Thread 0x7fffcaef46c0 (LWP 1387994) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388009)]
[Thread 0x7fffcaef46c0 (LWP 1388009) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388010)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 1.4 compatibility
][Thread 0x7fffcaef46c0 (LWP 1388010) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388011)]
[Thread 0x7fffcaef46c0 (LWP 1388011) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388012)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 1.3 compatibility
][Thread 0x7fffcaef46c0 (LWP 1388012) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388013)]
[Thread 0x7fffcaef46c0 (LWP 1388013) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388014)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 1.2 compatibility
][Thread 0x7fffcaef46c0 (LWP 1388014) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388015)]
[Thread 0x7fffcaef46c0 (LWP 1388015) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388021)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 1.1 compatibility
][Thread 0x7fffcaef46c0 (LWP 1388021) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388022)]
[Thread 0x7fffcaef46c0 (LWP 1388022) exited]
[New Thread 0x7fffcaef46c0 (LWP 1388026)]
Xlib:  extension "AMDGPU" missing on display ":1".
Xlib:  extension "GLX_ARB_context_flush_control" missing on display ":1".
Warn: Broken context: 16-bit OpenGL 1.0 compatibility
][Thread 0x7fffcaef46c0 (LWP 1388026) exited]
Warn: SDL returned a broken OpenGL context.

Please report the bug and tell us what is your operating system,
OpenGL driver, graphic card, and if you built the game yourself.

Actually we may call Sys::Error as soon as we detect a broken glContext, as I doubt there are situations where SDL2 can return a broken context but the next tested context would be valid…
Let me know what you think about it.

Actually we may call Sys::Error as soon as we detect a broken glContext, as I doubt there are situations where SDL2 can return a broken context but the next tested context would be valid…
Let me know what you think about it.

Yeah I'd rather keep it simple and Sys::Error immediately

Actually any context seems to be non-null but broken even if I request something that doesn't exist:
Warn: Broken context: 24-bit OpenGL 99.55 core

This means I got a non-null (but broken) context pointer for OpenGL 99.55. 🤷♀

Actually we may call Sys::Error as soon as we detect a broken glContext, as I doubt there are situations where SDL2 can return a broken context but the next tested context would be valid…
Let me know what you think about it.

I added a commit that add some logging:

Explicitly print detected OpenGL ABI at CMake configuration time:

-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
-- Found OpenGL ABI: GLVND

Print detected OpenGL ABI at engine run time:

Using SDL version 2.26.5
SDL using driver "x11"
Using OpenGL ABI "GLVND"
Initializing OpenGL display
Using GLEW version 2.2.0

Tell the user to rebuild with LEGACY if GLVND is detected when issue #945 happens, and link to #945:

Warn: SDL returned a broken OpenGL context.

Please report the bug and tell us what is your operating system,
OpenGL driver, graphic card, and if you built the game yourself.

GLVND OpenGL ABI detected, try reconfigure your build with:

  cmake -DOpenGL_GL_PREFERENCE=LEGACY

See: https://github.com/DaemonEngine/Daemon/issues/945

I added a commit:

Set Linux OpenGL ABI to LEGACY if OpenGL_GL_PREFERENCE is not set.

The backward-incompatible GLVND OpenGL ABI should only be used
by developers who have a good reason to require it.
See:

https://github.com/DaemonEngine/Daemon/issues/945#issuecomment-1835091256

Use CMake default OpenGL ABI for Linux if OpenGL_GL_PREFERENCE is set to empty string.

I noticed a logging bug in CMake, it always report that it finds the GLVND libOpenGL.so library even when it will link against libGL.so. That's also why the extra logging I added is useful.
CMake with LEGACY libGL.so linkage:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
-- Found OpenGL ABI: LEGACY

Linkage:
$ ldd daemon | grep GL
	libGL.so.1 => /opt/amdgpu-pro/lib/x86_64-linux-gnu/libGL.so.1 (0x00007f4e2cefa000)
	libGLEW.so.2.2 => Unvanquished/daemon/external_deps/build-linux-amd64-default_9/prefix/lib/libGLEW.so.2.2 (0x00007f4e2b782000)

CMake with GLVND libOpenGL.so linkage:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
-- Found OpenGL ABI: GLVND

Linkage:
$ ldd daemon | grep GL
	libOpenGL.so.0 => /lib/x86_64-linux-gnu/libOpenGL.so.0 (0x00007faa0318f000)
	libGLEW.so.2.2 => Unvanquished/daemon/external_deps/build-linux-amd64-default_9/prefix/lib/libGLEW.so.2.2 (0x00007faa01a3c000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007faa0150a000)
	libGL.so.1 => /opt/amdgpu-pro/lib/x86_64-linux-gnu/libGL.so.1 (0x00007faa0114b000)

This is a bug on CMake side on this way and those patches are working anyway.

New commit adds more logging.
GLVND available but LEGACY used:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
-- Found OpenGL ABI: LEGACY
-- Found OpenGL library: /usr/lib/x86_64-linux-gnu/libGL.so

GLVND available and GLVND used:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
CMake Warning at CMakeLists.txt:632 (message):
  The GLVND OpenGL ABI is backward incompatible with legacy drivers.

  See https://github.com/DaemonEngine/Daemon/issues/945

-- Found OpenGL ABI: GLVND
-- Found OpenGL library: /usr/lib/x86_64-linux-gnu/libOpenGL.so

GLVND not available:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libGL.so
-- Found OpenGL ABI: LEGACY
-- Found OpenGL library: /usr/lib/x86_64-linux-gnu/libGL.so

Building with LEGACY but only libOpenGL.so is found:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so
CMake Warning at CMakeLists.txt:643 (message):
  LEGACY OpenGL ABI not available but GLVND is, try
  OpenGL_GL_PREFERENCE=GLVND

CMake Error at CMakeLists.txt:645 (message):
  Could NOT find LEGACY libGL.so

Building with GLVND but only libGL.so is found:
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libGL.so
CMake Warning at CMakeLists.txt:632 (message):
  The GLVND OpenGL ABI is backward incompatible with legacy drivers.

  See https://github.com/DaemonEngine/Daemon/issues/945

CMake Warning at CMakeLists.txt:635 (message):
  GLVND OpenGL ABI not available but LEGACY is, try
  OpenGL_GL_PREFERENCE=LEGACY

CMake Error at CMakeLists.txt:637 (message):
  Could NOT find GLVND libOpenGL.so

So I deeply rewritten the code to detect and report the used OpenGL ABI.

If OpenGL_GL_PREFERENCE is not set, the LEGACY ABI is used.
If OpenGL_GL_PREFERENCE is set to empty string, we let CMake decides (GLVND if available, LEGACY otherwise).
If GLVND is used a warning is emitted.
If GLVND is requested but not available, the configuration errors out (it doesn't use LEGACY in the back of the user).
If LEGACY is requested but not available, the configuration errors out (it doesn't use GLVND in the back of the user).
When a requested ABI is not available but another one is, the error message suggests the other ABI.

If you want to have a cmake option that forces the GL link target rather than acting as a "preference", I think it would be better to do this by using the specific link targets. If I understand the documentation correctly ${OPENGL_gl_LIBRARY} can only be the legacy one and OpenGL::OpenGL+OpenGL::GLX is GLVND. This lets you avoid using OpenGL_GL_PREFERENCE at all. So you could have something like
if (FORCE_OPENGL STREQUAL LEGACY)
    set(DAEMON_OPENGL_LIBRARIES ${OPENGL_gl_LIBRARY})
elsif(FORCE_OPENGL STREQUAL GLVND)
    set(DAEMON_OPENGL_LIBRARIES OpenGL::OpenGL OpenGL::GLX)

Although I don't see why forcing instead of preference is really needed. For releasing purposes, validate-release warns if the list of dynamically linked libraries has changed, so we won't let regressions slip through.

I believe forcing is needed because the only reason I can imagine why someone would want to build a GLVND build of Dæmon is to test underlying software. In other words, the only ones needing to build Dæmon with GLVND ABI are GLVND developers, Graphics driver developers, Wayland developers, etc. And then they need to be sure to test what they want to test because they would not test Dæmon, but they would use Dæmon-with-GLVND to test their software.

I have mixed feeling about abusing OpenGL_GL_PREFERENCE but on the other hand I expect people to use this option to change the ABI, so even if our behaviour would be a bit different, using it would provide the named ABI as expected, and anyway all the extra logging would make sure the user knows what is happening, even if the behaviour is a bit different.


BUG: The Chatbot is Only Present in Home Page

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
Hey @Anushkabh and @himanshu-03
In this website the chatbot is only present in Home page

I would like to add the chatbot across all the pages.
Please assign this issue to me under GSSOC24 ?
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@himanshu-03 Assign this issue


Power supply by port 'power 1', then USB port can't used?

I wan't download the log from USB port use mavros, the pixhawk is powered by port 'power 1'. But I find the USB can't used? Does this can be solved ?

This should be possible. What is the error that you're seeing?
Can you try to power by USB first?

This could be the same problem as #12201.

@keguo1119 , if you have a chance to test with PR #12428, it might fix your issue.  Please let us know if it does.

@mcsauder ， I have merge the mcsauder:mavlink_config_stream_bugfix_master into current master branch, and test the firmware on pixhawk on fmu-v3. It works fine.
And now my new question is  do we just need merge the mavlink_main.h mavlink_main.cpp file into the old version 1.8.2, or some other files also need to be change to fix the bug on the old version(like 1.8.2)?

@mcsauder,  could you point out how to fix the bug,  base on version 1.8.2? It's easy to merge the master branch into our current version, but the testing will spend a lot of time.

@keguo1119 , It appears you can backport the same fix to v1.8.2.  I haven't bench tested it, but here is your example: https://github.com/mcsauder/Firmware/tree/v1.8.2_mavlink_config .  If this works for you let us know and we can close out this issue when #12428 gets merged.

@mcsauder , I have modify our own version base on the example, it works fine. Thank you very much.
Otherwise , the key changes is the two functions below, is that right?

That is correct. Just cut/paste code to create those two methods and allow them to be called from task_main() rather than at the constructor. Glad to hear that worked for you!

@mkschreder

It says set CBRK_USB_CHK to 197848  has some risk. what's the potential risk if I disable the  the USB connected checks in the commander. (If not the UAV can't take off)

@keguo1119 , the risk is if the USB link is disconnected during flight it will reboot the autopilot.
@dagar , we can close out this issue once #12428 is merged.

Can we solve this risk in software aspect? I will try to solve this by supply the USB power from the power port!!

Hi @keguo1119 , as luck would have it, #9664 was just merged in, so I think everything should be working to meet your needs.  Can we close out this PR now that #12428 is merged as well?

Yes，this issue can be closed!


Fixed auto-detected typos

I discovered jj a few days ago, and it seems pretty neat!
While I familiarize myself with the codebase (there will be an actual PR at some point, I promise), here's a few typos typos was screaming at me for.

Grant of Copyright License. Subject to the terms and conditions of this Agreement, You hereby grant to Google and to recipients of software distributed by Google a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works.

WTF?
So I'm granting google specifically the right to do whatever the f**k it wants with my contributions irrespective of the project license now and forever? Uhh... No? I'd much rather start my own fork than this.
You're claiming that this is a community project, even though google is involved.

See https://github.com/jj-vcs/jj/discussions/4849 for additional discussion.
I am not a lawyer, but I'm not sure that the specific text you're quoting is as unusual as you're remarking.

Google probably wants the right to distribute the project code for fairly mundane reasons. As an example, GitHub includes similar terms here, although much more readable than the Google CLA. (I personally recall being shocked by seeing similar terms in the Facebook terms until somebody explained to me that they need the right to reproduce and redistribute my photos for free so that my friends can actually see them.)
I'm not sure that the text actually grants Google rights regardless of the future license of the project (in that I think 'sublicense' doesn't let you change the license arbitrarily, only add restrictions, and you would expect a consumer to be able to use the original project code and license as an alternative), but I can't say authoritatively.
There's commentary in the above discussion noting that copyright is still held by the original authors.

@arxanas Yeah, I just finished diving into the CLA. You (people at google) should really have a word with your legal department. IMHO this is all very unnecessary (See discord).
FYI, I ended up signing the CLA. Granting google an apache license plus a DCO seems reasonable.


Setup linting

Need to setup linters to check the SV package code.
This might be done using either dedicated linters or simply compiling and elaborating the package using any simulator (+ all warnings has to be promoted to errors).

Added simulators with strict compilation as linters. No other linting is expected yet.


Add cost to bubble

Summary

Instead of requiring humans to do the math calculating the length of time for each bubble, store the cost in the data set, so that it can be displayed in the bubble. This should help with adding any new data points.
Display cost at the bottom of the bubbles, using Intl.NumberFormat

https://user-images.githubusercontent.com/1160238/114592800-f401af80-9c50-11eb-9b3a-4f5ae7cdbfc0.mov

Add a few additional data points and sources

Testing

Pulled down the repo and confirmed in index.html file. See attachment video.

Love the awesome website - I thought this would be a nice addition!

I like this idea.
I will need to update the values so they are a bit more accurate (I rounded them a little for simplicity)

Ooops. I forgot to test on Safari and found there was a bug. It seems that, only in Safari, if you use maximumFractionDigits, you need to specify minimumFractionDigits or it throws an error (and the bubbles didn't show up). Learned something new.


Add Pointer Lock  to the list

Add Pointer Lock https://dvcs.w3.org/hg/pointerlock/raw-file/default/index.html to the list

This issue was imported from Google Moderator
Moderator votes: +12
Issue added by wuz on 2014-04-08
To vote this issue up or down, simply include +1 or -1 in your comment.

same as #983

same as #983

Duplicate of #983 so adding half the votes there


Fix tolerations for virt-operator-strategy-dumper job

What this PR does / why we need it:
This PR specifies:
tolerations:
- operator: "Exists"
affinity:
  podAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: kubevirt.io
          operator: In
          values:
          - virt-operator
      topologyKey: "kubernetes.io/hostname"

for virt-operator-strategy-dumper job
Which issue(s) this PR fixes
There is a problem when all nodes in cluster have taints, KubeVirt can't run virt-operator-strategy-dumper job.
The provided fix will always run the job in same place where virt-operator runs
Special notes for your reviewer:
Release note:
Schedule virt-operator-strategy-dumper job into the same node with virt-operator

/test pull-kubevirt-unit-test

I migrated fork from kvaps/kubevirt to deckhouse/3p-kubevirt, now it will be supported by deckhouse project
fyi
/remove-lifecycle rotten
/remove-lifecycle stale


Check that generated docs and mocks are up-to-date

Description
Our generated markdown files and mockery mocks are sometimes outdated.
This PR adds a CI step to check if those files were updated.
In addition it:

removes an old check for the docs that did not work
update the make doc scripts to actually work in the CI
add an issue-845-fix: True annotation to the mockery config as the edge case anyway does not apply to our codebase
update all outdated files

Jira

How can this be tested?

Check if CI ran through
Change one md file and run act -j generated-files locally => it should fail

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 64.51%. Comparing base (4215cbe) to head (692b1c2).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4172   +/-   ##
=======================================
  Coverage   64.51%   64.51%
=======================================
  Files         399      399
  Lines       26790    26790
=======================================
  Hits        17284    17284
  Misses       8145     8145
  Partials     1361     1361

Flag
Coverage Δ

unittests
64.51% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

left small renaming suggestions, otherwise LGTM!


Setup XCode Cloud for Builds

These builds should also be moved to a greater level of optimization (O3)

Xcode builds are setup


ciao: Change default value for gopath

Using $GOPATH as the value for gopath is not
so easily achievable because the GOPATH value
for the deployment machine does not get copied
automatically to other nodes.
Also setting a fact on one node does not set it
on other nodes.
The easiest way to share this value across the
different ciao roles is to set it as a variable
that can be overriden in groups_vars/all file or
via the cmdline with --extra-vars "gopath=$GOPATH"
Fixes: #57
Signed-off-by: Alberto Murillo<EMAIL_ADDRESS>

lgtm

LGTM


Add keynote recording links

I wasn't sure if we wanted to add it to the bio page of the speaker or add a link to the schedule (or both).
Here are the links to the recordings:
Day 1 - Angie - http://videocast.nih.gov/summary.asp?Live=16473&bhcp=1
Day 2 - Panel - http://videocast.nih.gov/summary.asp?Live=16475&bhcp=1
Day 3 - 3D Printing - http://videocast.nih.gov/summary.asp?Live=16477&bhcp=1

Done, but the pictures that @jonraedeke had included on the keynote OS panel are no longer showing for me. Not sure what's up with that page. Perhaps the image source is no longer valid?


Strain is an unknown rank

"Strain" appears as a rank identifier throughout the input data. This is however not a valid rank and should be filtered out when processing the data. This PR proposes a fix for issue #13.

I was wrong, "strain" is a valid rank that was recently added to the NCBI taxonomy. This PR will be reverted.

Taxonomically speaking, strain is not a rank. Also, for Unipept there’s no use to aggregate at the strain level. A strain is a single individual (clone).


Generic Config files subdirectory

Feature Request
Feature Information:
I'd like to propose a new method to help clean up and handle config files. This would clear up clutter, allow for config files to match names between games (Game.ini is in multiple games), and avoid update steps/stores the configs locally. This also would prevent mismatched config files as you would only receive the latest templates when you get the latest configs.
Currently, they are all placed in the main directory on GH. I suggest users should create sub-folders that match the game's kvp name (conan-exiles.kvp folder would be conan-exiles). Only the kvp file will be in the main directory. The json and any base configs/template files would all go into the sub-folder. All the folders would be downloaded to a base AMP ADS install. A new setting would be created that ADS would use to grab all the files within the sub-folder and places them into the root directory of newly created instances using a format like: app.ConfigDirectory=./conan-exiles.
I confirm:

[x] that I have searched for an existing feature request matching the description.

I'm closing this since it would be more preferable to not have multiple configs at all and do it all from one json file instead.


Uncrustified SMP

Description

Test Steps

Related Issue

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

This PR is included in #53. Close this PR first.


@autorun/reaction() decorators on component methods

I know this is a duplicate of #122, but I'd like to make a real proposal instead of just describing how to achieve the behaviour using "vanilla" MobX autorun/reaction.
Currently, to create custom reactions on a component, we have to do something like this:
@observer
class Test extends React.Component {
    reaction = autorun(() => console.log(foo.prop);

    componentWillUnmount() {
         this.reaction();
    }
}

This works fine, but it's not exactly convenient and as easy to use as it could be. Moreover, the use of MobX with React allow us to drastically reduce our usage of lifecycle methods, especially annoying stuff like componentWillReceiveProps/componentWillUpdate which are in most cases best handled by MobX (post-render events will forever remain since they are our only point of access to the DOM, which is stricly React's business). So being forced to keep a componentWillUnmount around as boilerplate code, especially when MobX's main advantage over something like Redux is that it requires none, is simply disappointing.
@mweststrate speaks a lot about how using componentWillMount is an anti-pattern and that we should react to state changes instead, and he is exactly right. The problem is that using componentWillMount requires no boilerplate or does not require that reactions occur outside of the component, and it achieves most of the time the same behaviour. And is what people know and use. Another strong point of MobX is that it's unopinionated, so it shouldn't indirectly enforce some application structure over another. For instance, injecting stores into components is the better approach to using them directly, but I can still do very easily the latter if I believe setting up dependency injection isn't worth it.
We also have a weird componentWillReact lifecycle hook for which I never found a use (the exemple on the readme simply calls console.log), and is basically a worse componentWillUpdate.It does not have access to the old values and simply triggers any time an observable used in the render method is updated, which is not very precise and all in all very alien to how MobX reactions work.
My proposal is simple: I'd like to have some kind of decorator, which probably won't be called @autorun or @reaction for name-clashing reasons, that I can put on any method inside an @observer component, that will add the reaction boilerplate for me. It should have both autorun and reaction signatures, maybe @classReaction for autorun behaviour and @classReaction(selector, fireImmediately?) for reaction behaviour (I don't like that name, we'll need a better one).
That way, I can replace
async componentWillMount() {
      this.data = await server.load(this.props.id);
}

by
@classReaction
async load() {
     this.data = await server.load(this.props.id);
}

which is in all respects a better solution, and one that you'll have no problem convincing people to use.
I used the componentWillMount replacement exemple because it's the simplest use case for it, but I have a lot of more advanced uses of reactions that would definitely benefit from something like this too.
With this, I think we would be able to definitely relegate lifecycle methods to pure React-uses only (DOM events/manipulation is the big thing I see), which would help consolidate the roles of each framework in a React-MobX application.

@JabX thanks for the suggestion, I see more requests for this. Not sure but it might fit in the following package as well: https://github.com/farwayer/mobx-decorators
It might be interesting to add and @autorun decorator to either that package or the mobx package. Using that decorator one or multiple times on an instance should then also create a disposer method that cleans all disposers for that instance. Either instance.mobxDisposeReactions() or disposeInstanceReactions(instance)`  or something similar.
If that instance method present, mobx-react could easily invoke it, if it is on the component instance that is being unmounted.

@mweststrate
I think it belongs more into mobx-react because it will be tied to React itself (relying on componentWillUnmount) and because I suppose it would reach a broader audience it it were packaged. Not to mention the implemention would probably be simpler since it would fit right in the observer enhancements instead of in an additional layer that would probably require an extra decorator on the class.

I just tried to use @autorun decorator simply assuming it would be there, since @observable and @computed is there.  I want to maintain an @observable backed crossfilter, and I need to explicitly dispose my dimensions and groups and recreate them when the data changes.

Follow up question, how should these decorators be called? i think it will be very confusing if both mobx and mobx-react expose reaction and autorun
Another solution could be providing an utility to easily setup disposers, so that one could write:
@observer
class Test extends React.Component {
   componentWillMount() {
      this.addDisposer(reaction(() => stuff, () => { stuff(); }))
   }
}

I do like
import {reaction, observer} from "mobx-react"

@observer
class Test extends React.Component {
   @reaction(() => this.props.id)
   fetchData(id) {

   }
}

better indeed, but I fear import reaction from mobx-react might be very confusing....

See #561: Mobx-react 5.3.0 introduced the disposeOnUnmount decorator: https://github.com/mobxjs/mobx-react#disposeonunmountcomponentinstance-propertykey--function--function


move from buildjet to github runners

fixes: https://github.com/pulumi/home/issues/2732
Move off of buildjet and onto gh runners. Assuming this passes and the preview builds correctly, should be good to go.

preview: http://www-pulumi-docs-origin-pr-9495-037a74ab.s3-website.us-west-2.amazonaws.com/


Feature request: Improve admin screen

When viewing the admin screen, django-constance shows a list containing all the options and a single save button for all options.
Would it be able to have the list be readonly? When user clicks on a single option, to have it open a page for 1 specific config with a save button to change only one config?
This would avoid accidentally updating other config unexpectedly and is more consistent with other Django admin screens (such as user admin).
Thoughts?

You can contribute anytime. Seems like a great feature to me.


[bug] editor try to save existed file as unexisted file

Describe the bug
when implement #185, introduces this bug
To Reproduce
Steps to reproduce the behavior:

Open any existed file
modify and save

Expected behavior
The existed file should be saved to the current file
Desktop (please complete the following information):

OS: macOS
Sauron version: d20816d732980456e8370beb0f9c0a3b3ec989ad
Racket version: 8.3

NOTE: this fix no need to run into changelog, #185 is unreleased


docker machine ux

consider way to make it work nicer with docker machine created envs. may be ask for docker machine env name and look at fs? also maybe detect unix socket automatically?

merged mutual tls stuff and made some improvements.


'ImageableRect` Property description missing words

If you look at the property description for ImageableRect it is missing some information. The text reads like so:

Content outside of the , whether less or greater than the page size...

Note the missing word(s) before the comma
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb1d7a5a-767e-bb30-e287-caeef5f44bab
Version Independent ID: 85b0d795-5a00-3f14-bcab-3ffa73840a95
Content: PrintPageDescription Struct (Windows.Graphics.Printing) - UWP app developer
Content Source: https://cpubwin.visualstudio.com/DefaultCollection/windows-uwp/_git/winrt-api-build?path=%2fwinrt-api-build%2fwindows.graphics.printing.printpagedescription.yml&version=GBlive&_a=contents
Product: windows
Git Login: wdg-dev-content
Microsoft Alias: wdg-dev-content

bump just noticed this issue has still not been resolved and the issue is still present.

Thank you for reporting this typo in the documentation. It has been fixed.


your apis an api.gov.uk

Problem with 'Department for Education' (https://www.api.gov.uk/dfe/)

redirect problem


Partial builds are slow in Webpack 3.2.0

Do you want to request a feature or report a bug?
Issue
What is the current behavior?
Partial builds are slow in Webpack 3 compared to Webpack 2
If the current behavior is a bug, please provide the steps to reproduce.
I just upgraded webpack in my project (based on vue-webpack template) from v2.6.1 to the latest v3.2.0. The partial build time became much slower, from previous ~1.7 sec it's now ~5.5 sec. Initial builds are same ~55 sec.
Also tried to upgrade webpack-dev-middleware from ^1.10.0 to ^1.11.0 and webpack-hot-middleware from ^2.18.0 to ^2.18.2. Didn't help.
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
Node 8.0.0, yarn 0.24.5, macOS Sierra, package.json, no SSR enabled

Thanks for your report. It would be great if you reduce your issue to a small reproducible example. Best put this example into a github repository together with instructions how to get to the problem.

Hey @sokra. I knew you will ask this. I wish I could show you some reproducible example but it's a private project I'm working on. I thought that this issue may be there for every setup, not sure though. So maybe it can be reproduced by just trying to downgrade/upgrade Webpack on any existing setup and see if there is a difference

I unfortunately don't have a reproducible example either, however I'm also seeing increases in rebuilds in our production application. I have a 2.4 -> 3.2 upgrade staged and I'm seeing the following change in times:

Activity
2.4
3.2

Prod build (e.g. full optimizations, minify)
111.47s
96.75s

Local dev builds (e.g. watch mode, hot module, no minify)
88.28s
66.84s

Partial rebuild (comment out random import, then uncomment)
5.16s
8.32s

Commenting this to add additional reports of this behavior. (I have the latest 3.2.0 that was released yesterday)

It's difficult to find the issue without a reproductable repo, but I'm happy to guide you through finding it.
Could you isolate the exact version at which the build got slower? Try to isolate the dependency first. If you change webpack back to 2.4.1 does it still occur? If yes, what happens with 3.0.0. And so on.

Same issue with my project. Rebuild on watch raised from 0.6 s to 2 s.

I think I found the issue.

❤

@dcousineau @ilearnio Could you measure webpack 3.3.0 and check if this is fixed?

Keeping old numbers, but upgrading to 3.3.0 and re-running builds:

Activity
2.4
3.2
3.3

Prod build (e.g. full optimizations, minify, gzip)
111.47s
96.75s
100.04s

Local dev builds (e.g. watch mode, hot module, no minify)
88.28s
66.84s
78.40s

Partial rebuild (comment out random import, then uncomment)
5.16s
8.32s
1.34s

I noticed the very first partial rebuild was about 5s (on par with 2.4) but a subsequent re-builds hovered around the 1-1.5s range. Since rebuild time for development mode is our biggest thing, this is huge win. Thanks @sokra!
(I'm itching to use the next version of Babel with their parser speed improvements and see how much more it drops build times, as well as I'm starting to investigate DLLs to increase build speeds given how aggressive our vendor bundle is)

@sokra I just tried to install the fresh 3.3.0 version. It's still much slower as for me. The partial rebuilds are taking 3.5 - 7.5 sec compared to 1.8 sec when using Webpack v2.6.1

@ilearnio Heave you tried to update the extract-text-webpack-plugin to the version 3.x? I see you use 2.x in your package.json, and it's not fully compatible with Webpack 3.x`

@sokra It my tests commenting out a function went down from ~1600ms to 600ms after upgrading from Webpack 3.2.0 to 3.3.0. Awesome! Thank you!

the same for me, i have update every package. but it's mucher slower than v2.6.1

@githoniel Is this with the first rebuild only? I'm on the 3.5.5 and haven't seen this issue since 3.3.0. Do you have the same set of plug-ins?

@the-spyke
extract-text-webpack-plugin 3.0
webpack 3.4.0
not only the first build is slow and rebuild always do a full rebuild.
i have to downgrade to use webpack 2.7

same issue here. I found that webpack >3 and extract-text plugin are relatived to this issue. After downgrade to webpack@2 and extract text plugin@2. Partial build is much fast. In my project. It takes about 1 sec to partial build at webpack 2, but when using the webpack 3, it takes about 16sec


[WIP]MM-T563 New message bar - Reply posted while scrolled up in same channel

Summary

MM-T563 New message bar - Reply posted while scrolled up in same channel
Ticket Link

https://automation-test-cases.vercel.app/test/MM-T563

@openmohan Let us know status of this PR, thanks!

I will work on this today @saturninoabril.
Sorry, Got stuck with few other tasks.


Update es.json

For localization of destiny tracker

Please resubmit all of these PRs as a single PR.
Thanks


Add Romza repos to the GitHub feed

I'm submitting a ...

[x] feature request

Figure out a way to get the feed from Romza into the website. Either add a new section or ideally merge the two feeds together.

Resolved in https://github.com/RomzaLabs/simplehome/pull/17


Module renaming

RenderPhoto and PhotoContainer and AllPhotos modules do not have clear names. Naming should be clearly reflective of what they do, links and references in other files should be changed and then the READ.ME should be updated.
Affected files are:

./components/RenderPhoto
./components/PhotoContainer

./components/AllPhotos

Hey, I think I can work on this. Can you please assign this to me?

Hey, I think I can work on this. Can you please assign this to me?

Hi Emad, sure. Thank you. which one you are thinking to start first. this one or Code Clean up?

I think  I should go with Code Clean Up.

I'm going to do this first and the Code Clean up

Actually naming a file or function is really hard but I have got renaming the most hardest in the world to do. I prefer to play chess with death angel :) by the way any idea about he component naming? I just really got stuck and nothing has came out. I actually have tried to track component actions and figured out the functionality of each but there is still nothing fancy, or not fancy just normal but logical.

Actually naming a file or function is really hard but I have got renaming the most hardest in the world to do. I prefer to play chess with death angel :) by the way any idea about the this issue? I just really got stuck. I actually have tried to figure out the functionality of each but there is still nothing fancy, or not fancy just normal but logical.

TBH I was double checking this and realized this is not a good first issue lol It is very complicated, but thanks for taking time to look in to it. I think let's leave this one for now. If and whenever you have some free time I appreciate if you can help on one of the following issues?

This one is easier: https://github.com/functionland/photos/issues/33
This one might be harder: https://github.com/functionland/photos/issues/45

That's alright. I'm going to give a try on these you have mentioned and inform you as soon as I can. Tnx :)

I close this for now as module name change is trivial


[BUG][WEBJS] - Session doesn't start. Always in FAILED state.

Session doesn't start. It always ends up in FAILED state
Whenever a new session is requested, the state becomes SCAN_QR_CODE. After receiving the code, when we scan the code, the session doesn't start. It goes to failed state.
However, the session in the mobile phone gets established.
This is the error in the logs.
[Nest] 34  - 04/17/2024, 7:44:43 AM   ERROR [WhatsappSession - 8BBE9AAFD225465CB175FEAAC3E3A876] Error: Evaluation failed: TypeError: Cannot read properties of undefined (reading 'default')
at puppeteer_evaluation_script:5:95
Version
 {
  version: '2023.11.1',
  engine: 'WEBJS',
  tier: 'PLUS',
  browser: '/usr/bin/chromium'
}

Could you try to update to the latest version?
We have an important update for WEBJS a month ago
https://www.patreon.com/posts/waha-plus-webjs-100765474

It should show you 2024.4.3

Hi,
I got the latest version, and it is working now. Thanks.
{
"version": "2024.4.3",
"engine": "WEBJS",
"tier": "PLUS",
"browser": "/usr/bin/chromium"
}

Hi,
I got the latest version, and it is working now. Thanks.
{
"version": "2024.4.3",
"engine": "WEBJS",
"tier": "PLUS",
"browser": "/usr/bin/chromium"
}


Terrorist and Soldier AI Update

📝 Description
Improves the Terrorist and Soldier AI, making them more fun to fight
🎯 Goals

Remove the zillion explicit casts to EntityCreature cause it was confusing me
Improve the AI in combat.

❌ Non Goals

Fix bugs about Grenades
New behavior (Later, firerate could be controlled by whatever gun they are using)

🚦 Testing
I have not tested this yet
⏮ Backwards Compatibility
1.12.2
📚 Related Issues & Documents
#428
🖼 Screenshots/Recordings
n/a (yet)
📖 Added to documentation?

[ ] 📜 README.md
[ ] 📑 Documentation
[ ] 📓 Javadoc
[ ] 🍕 Comments
[x] 🙅 No documentation needed

The changes needed are just a few important things I noted.
But in general, please note that any changes, especially things like this, should be accompanied by reasoning, just changing things without explaining why isn't great for me to be able to merge it.
Also as indicated on the Trello (for once Trello is up to date lol) a complete rework of the NPC's is coming soon-ish, (although you already know that from the issue lol).
My advice here is probably to wait until the rework is done, then test things and play a bit with the rework, then come back to this.

Cool, Thank you


[Driver] Clarify unsupported option message

Previously, with the tool and option both included in the suggestion, it
was not immediately clear which of the two was being suggested as a fix.

Resolves SR-5677.

@swift-ci please smoke test

@swift-ci Please smoke test

@swift-ci Please smoke test

Hm, no response. @shahmishal?

Debugging:
@swift-ci please smoke test

Debugging:
@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@jrose-apple Found the issue, will resolve it soon.

@swift-ci please smoke test Linux

Merged. Thanks again, Deyton! Feel free to mark the JIRA as Resolved.

Thanks for the quick turnaround!
Getting up and running was mostly smooth, once I realized which version of Xcode to use. This was made worse because Xcode 9 Beta 4 was no longer available! The availability problem is mostly just bad timing (and AVFoundation bugs in Beta 5), but in general, I think it would be helpful to verify that the correct version of Xcode is active at the beginning of the build script. As it is now, it takes a long time to get to the failure point.
Another thing that would have helped me personally is some specific guidance about what goes into getting your first code change merged. There's lots of high-level guidance at swift.org/contributing, but it still left me unsure about the specific steps I would need to take. Something like this would be very helpful:

How to Make Your First Swift Contribution
Fix a Bug

Find a starter bug on bugs.swift.org.
Assign it to yourself.
Read the docs (maybe labels or components map to files in the /docs directory?)
Commit your changes (following guidelines)
Open a PR.
Update the JIRA bug.
Receive feedback.
Etc...

I like to have consistent processes on my projects, so I tend to worry about doing the wrong thing when contributing elsewhere. It seems like the Swift project is loose about some of this stuff, so maybe it makes more sense as a third-party blog post rather than a part of the official documentation.

Thanks for the feedback! These are all good notes.


While downgrade the flutter SDK this problem is occured

Thank you for taking the time to file an issue!
In order to route, prioritize, and act on this, please include:

Dart SDK Version (dart --version)
Whether you are using Windows, MacOSX, or Linux (if applicable)
Whether you are using Chrome, Safari, Firefox, Edge (if applicable)

Missing some or all of the above might make the issue take longer or be
impossible to act on.

Is it really an issue? For general questions consider starting with Stack
Overflow:
https://stackoverflow.com/questions/tagged/dart
Also consider our Gitter channel for light-weight/quick discussions:
https://gitter.im/dart-lang/sdk

Please include what exactly has occurred.

Please open a new issue with the requested information.  There isn't enough to go on here.


Save theme customizer settings

Ability to have the theme you set save.

Realizing that I converted this to the wrong repo. Won't let me change, my bad!

Placing this option within the settings page of a user account


Metal support?

From here: https://developer.apple.com/ios/whats-new/ it seems that iOS 12 has deprecated OpenGL (look for "Deprecation of OpenGL ES")
From what I understand here: https://flutter.io/technical-overview/ flutter relies on Skia as the rendering engine.
Reading (very quickly) the Skia documentation, I understand that is based on OpenGL / Vulkan ( https://skia.org/user ).
Question is: what is the state of Flutter support for Metal?
Does it depend on Skia? For Skia I could find only mentions of Metal in the roadmap: https://docs.google.com/document/d/1LSdO3I-IdZVZCtEnW9pu3k_32gfPIcB_YeSn-OJBGZI/edit

"Metal support has long been considered, this announcement just accelerates that work"
https://twitter.com/danardelean/status/1003737735134351362

Ohh missed that tweet! Thanks @nackermann !

Linking to the tracking issue for this (https://github.com/flutter/flutter/issues/18208) for those who come to this bug organically through Google.

It depends on skia.
See: https://docs.google.com/document/d/1LSdO3I-IdZVZCtEnW9pu3k_32gfPIcB_YeSn-OJBGZI


Add pg_tables and pg_views to catalog

Fixes: https://github.com/kyleconroy/sqlc/issues/1747
Problem - I was unable to write a query against pg_timezone_names.
Actual failing query
select
  name,
  abbrev,
  utc_offset,
  to_char((current_timestamp at time zone name), 'HH:MI am') as current_time
from pg_catalog.pg_timezone_names
order by utc_offset;

This because sqlc did not know about these built-in tables / views.
Warning
I tried to make sqlc-pg-gen and make regen, but I was getting slightly different output, so in this commit I only committed the changes that affected the tables, not the existing functions (which I do not know why they were changed, perhaps a different version of postgres).

@kyleconroy I might need your help regenerating the files.  I am having a heck-uva time.
➜  sqlc_skabbes git:(catalog_tables_views) ✗ make regen
go build -o ~/bin/sqlc-dev ./cmd/sqlc/
go run ./scripts/regenerate/
2022/07/16 11:35:01 internal/endtoend/testdata/wasm_plugin_sqlc_gen_greeter: sqlc-dev generate failed
# package greeter
error generating code: call: Exited with i32 exit status 1
wasm backtrace:
    0: 0x1cf49 - <unknown>!__wasi_proc_exit
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable to may show more debugging information

exit status 1
make: *** [regen] Error 1

Very strange error message. What os / arch are you on?

I'm on an M1 Mac, arm64 I suppose.

I added the env var, and get this next error message.
➜  wasm_plugin_sqlc_gen_greeter git:(catalog_tables_views) ✗ WASMTIME_BACKTRACE_DETAILS=1 sqlc-dev generate
# package greeter
error generating code: loadModule: Module was compiled without WebAssembly backtrace support but it is enabled for the host

@kyleconroy given this commit fixed the build, I think it has something to do with the pure size of the data or number of tables in pg_catalog.
I believe since all of a sudden there are tables in the pg_catalog schema, it is exporting them - but probably that's not what you want / wanted with your plugin design.

I'm not sure what limit we're running into, but for the meantime let's drop the pg_catalog schema before sending it to the WASM plugin.
diff --git a/internal/ext/wasm/wasm.go b/internal/ext/wasm/wasm.go
index a55d8551..937ece21 100644
--- a/internal/ext/wasm/wasm.go
+++ b/internal/ext/wasm/wasm.go
@@ -175,7 +175,23 @@ func (r *Runner) loadWASM(ctx context.Context, cache string, expected string) ([
        return wmod, nil
 }

+// Due to some memory or file size limit, we must drop the pg_catalog
+// schema
+func removePGCatalog(req *plugin.CodeGenRequest) {
+       if req.Catalog != nil && req.Catalog.Schemas != nil {
+               schemas := req.Catalog.Schemas
+               for i, _ := range schemas {
+                       if schemas[i].Name == "pg_catalog" {
+                               req.Catalog.Schemas = append(schemas[:i], schemas[i+1:]...)
+                               return
+                       }
+               }
+       }
+}
+
 func (r *Runner) Generate(ctx context.Context, req *plugin.CodeGenRequest) (*plugin.CodeGenResponse, error) {
+       removePGCatalog(req)

@skabbes Something is wrong with the updated pg-gen. We should be generating two functions for COUNT. Here's the current function signatures on main.
		{
			Name: "count",
			Args: []*catalog.Argument{
				{
					Type: &ast.TypeName{Name: "any"},
				},
			},
			ReturnType: &ast.TypeName{Name: "bigint"},
		},
		{
			Name:       "count",
			Args:       []*catalog.Argument{},
			ReturnType: &ast.TypeName{Name: "bigint"},
		},

And here's the new definition:
		{
			Name:       "count",
			Args:       []*catalog.Argument{},
			ReturnType: &ast.TypeName{Name: "bigint"},
		},
		{
			Name:       "count",
			Args:       []*catalog.Argument{},
			ReturnType: &ast.TypeName{Name: "bigint"},
		},

We're missing the any parameter. Maybe it's something to do with the cross join?

No, sorry I didn't explain well.  routines in Postgres can be regular functions, procedures, aggregates, and window functions.  https://www.postgresql.org/docs/current/catalog-pg-proc.html
If it is an aggregate, there are 2 kinds of arguments, the "direct" arguments, and the "aggregate" arguments.  Previously, sqlc did not differentiate between them, and classified all of them as "direct" arguments.
Most aggregate functions that are used don't have any "direct" arguments at all, so it all kinda worked out just fine.  However, percentile_cont and friends have both direct and aggregate arguments that sqlc was not equipped to handle.
You can see that you manually overrode the pg_catalog just to make that work in this commit.
My commit did the work to differentiate between these direct and aggregate arguments to routines.
So when I said:

I think there needs to be more work in the compiler to treat / lookup these functions differently based on the context in the query. So not sure where to go from here.

What I meant was we need to make "looking up a function" (actually, a routine) smarter in the sqlc internals, before this can work.  Either that, or re-apply the manual workaround you did from September 2021.
The reality is that the function  percentile_disc actually takes 2 parameters, the 0.5 (direct arguments) and the within group (order by authors.name) (the aggregate argument).
select percentile_disc(0.5) within group (order by authors.name)

@kyleconroy  I think looking forward for sqlc, it needs to differentiate between them.  I'd be happy to put some work into making that a reality, but just wanted to check-in and make sure you agree that's the right path.

You can see that you manually overrode the pg_catalog just to make that work in this commit.

I honestly forgot about the manual changes I made to pg_catalog.go. While it was the right thing to do to fix the bug, it should have been fixed in sqlc-pg-gen. I'll take the // Code generated by sqlc-pg-gen. DO NOT EDIT. seriously in the future.

I'd be happy to put some work into making that a reality, but just wanted to check-in and make sure you agree that's the right path.

Yep, I think we're on the right path. That said, we're now fixing something unrelated to the original intent of this PR (aggregate functions vs PostgreSQL internal tables / views). I'd be happy to split this up if you'd like to get the tables and views in first, but it's up to you.

I think this:
https://github.com/kyleconroy/sqlc/pull/1809
Is the first step in actually making progress here.

I islolated the pg_* fixes in this PR:
https://github.com/kyleconroy/sqlc/pull/1811
I think the "aggreagated function args vs direct args" can / should be addressed independently.

Agreed. Going to close this PR.


Recuperate zoom and rendered position changed

There are these two functions who got lost from version 0.1 to 0.2 of ipycytoscape.
If you download this release file: https://github.com/QuantStack/ipycytoscape/releases/tag/0.1.4
You'll see it, but basically, we need to readd them to the code:
  this.cytoscape_obj.on('zoom', () => {
    this.model.set('zoom', {'level': this.cytoscape_obj.zoom()});
    this.model.save_changes();
  });

  this.cytoscape_obj.on('rendered_position', () => {
    this.model.set('rendered_position', {'renderedPosition': this.cytoscape_obj.rendered_position()});
    this.model.save_changes();
  })

  zoom_change() {
    this.cytoscape_obj.zoom(this.model.get('zoom'));
  }

  rendered_position_change() {
    this.cytoscape_obj.rendered_position(this.model.get('rendered_position'));
  }

This should lie inside https://github.com/QuantStack/ipycytoscape/blob/1cba2f411d6e323d83025d52dd314817f064ad94/src/widget.ts#L334
We also have to make sure it works, a calling of these functions in one of the example notebooks will be enough :)

this.model.set('zoom', {'level': this.cytoscape_obj.zoom()});

needs to be this.model.set('zoom',  this.cytoscape_obj.zoom());
looks like the cause of https://github.com/QuantStack/ipycytoscape/issues/26

Also, what are these for? On the python side they don't seem to do much? https://github.com/QuantStack/ipycytoscape/blob/8769ec2d36a9ae5ec8420ffc1c96fbb0f062d284/ipycytoscape/cytoscape.py#L306-L307
Are they in case someone wants to modify those things from python? like:
for z in [.25, 1, 2]:
    cytoscapeobj.zoom =z
    sleep(1000)

Ah, one consequence of this is that if you have multiple views of the same object they all are forced to have the same zoom level. Is this intended? My preference would be for different views to be able to have different zoom levels and rendered positions so i could look at multiple parts of the same graph at once using different cells.

So this code is for the frontend part, actually.
The idea of these functions is doing something like, if the zoom is > 90% then, show tips or something like that. And the rendered position was actually requested by an user, it can be used to create an analyses graph, where you want to preset the locations of the nodes.

My preference would be for different views to be able to have different zoom levels and rendered positions so i could look at multiple parts of the same graph at once using different cells.

That'd be cool. I don't really know how to do this, because I think the API for this is bounded to the whole graph object but don't know much about it, might be doable!

So this code is for the frontend part, actually.

Huh, I think I'm misunderstanding something then. To me it looks like what these functions do is make sure that zoom and rendered_position are synced back to python.

And the rendered position was actually requested by an user, it can be used to create an analyses graph, where you want to preset the locations of the nodes.

I think I'm confused here as well. Looking at that code (https://github.com/QuantStack/ipycytoscape/blob/b39d99fc66cc416d5f6e32fa776896257f2d44f6/src/widget.ts) it seems that rendered_position neither affects anything on the frontend nor is affected by changes to the cytoscape graph.

That'd be cool. I don't really know how to do this, because I think the API for this is bounded to the whole graph object but don't know much about it, might be doable!

This was possible until I tried re-adding the above functions. My understanding of how widgets work is that there is one underlying python model, but every time you create a new output (i.e. cytoscapeobj as the last line in a cell) then a new javascript object is created to display there. So as long as the zoom in cytoscape doesn't get synced back to the python model then the different views can't communicate it to eachother. Is it bad practice to have the sync only go one way? So the zoom for all views can be set from python, but zooming on any given given doesn't result in the zoom value is aware of changing?
Here's the behavior without readding those lines:

and with them readded:

I think they diverge in position in such a crazy way because only the zoom attribute is being synced between them. Though, I would have thought that they would start with the same rendered position. Maybe there is some randomness used in the choosing the layout that affects this.


Change upvalue size from 48 to 32 bytes

Motivation
Currently the size of the UpVal struct will be 48 bytes on 64 bit systems. This is caused by the two doubly linked lists an open upvalue is inserted into. This needs 32 byte for the pointers and can be improved to 16 bytes by making some changes.
Description
To reduce the size from 32 byte to 16 the open upvalues are removed from the global open upvalue list. This list is only used in the atomic phase to mark all the values from gray upvalues. However, this can also be achieved by traversing all threads and for every thread through all the upvalues. This is done in a optimized way by this implementation and details will follow.
But first we take a look at the second linked list an open upvalue is in. This thread local list contains all open upvalues for the thread sorted from the top of the stack. However, this does not really need to be a doubly linked list. A single linked list is sufficient as upvalues are only inserted from the top and the topmost one is the first to be closed. Currently this is a doubly linked list to allow an upvalue to be removed from the list when it is deleted. However, this does not need to happen in the destructor.
Instead of the doubly linked list this implementation use a single linked list and a pointer to the thread the open upvalue belongs to. Furthermore, open upvalues can now be either white or black, never gray. This actually violates the GC invariant. However, we ensure that the invariant holds in the end.
Now, when an open upvalue is marked it will put the thread in a white-thread-with-black-upvalue-list if the thread is white. When the thread is later found and marked it is removed from this list. After the atomic phase all threads in this list are iterated over and black upvalues are closed. This allows to delete all upvalues with a trivial destructor as they are never in a live list. The value such open upvalues refer to are guaranteed black since they are marked through the upvalue and upvalue store barriers and in case the thread mutates the value the thread is actually not white.
Since in the end all values from a living thread (aka threads not in the white-thread-with-black-upvalue-list) are marked it is not an issue when an open upvalue refers to a white value in the stack written by the mutator.
Finally, to ensure that no of the upvalues of a live thread need to be removed from the linked list all are marked black when the thread is marked.
Advantages

Decreases the size of UpVal on 64 bit systems from 48 bytes to 32 bytes.
Dead open upvalues on dead thread do not need to be visited which is currently the case.
Trivial destructor for upvalues and removal of the closing of upvalues from the lua_State destructor. (Trivial destructors are preferable as they allow to remove an object without touching the dead object. This can be useful when putting a bitmap into the page header with bits for new & marked as it allows to only touch those bitmasks and not the objects.)

Disadvantages

All upvalues of a live thread are marked as live by the thread. Currently there might be dead upvalues which are then deleted. However, this is of questionable usefulness.
All threads in the white-thread-with-black-upvalue-list are visited in the atomic phase. However, this should have the same cost as iterating though all open upvalues in the current implementation.

Benchmarks
Following are some benchmarks with the current vs suggested implementation.
Note that these benchmarks did show large variations on different runs on my setup!

Benchmarks
Output of python3 bench/bench.py --vm luau-base --compare luau --extra-loops 1 --folder bench/gc:

Test
Min
Average
StdDev%
Driver
Speedup
Significance
P(T<=t)

BinaryTree
30.216ms
34.107ms
1.625%
luau-base

BinaryTree
31.341ms
36.295ms
4.949%
luau
-6.027%
likely worse
2%

GC: Boehm tree
1984.451ms
2164.497ms
2.697%
luau-base

GC: Boehm tree
2007.325ms
2128.197ms
2.742%
luau
1.706%
likely same
34%

GC: tree pruning (eager fill)
80.885ms
90.690ms
2.269%
luau-base

GC: tree pruning (eager fill)
77.140ms
87.441ms
2.319%
luau
3.716%
likely better
3%

GC: tree pruning (eager fill, gen)
84.967ms
102.744ms
2.702%
luau-base

GC: tree pruning (eager fill, gen)
83.761ms
97.934ms
2.367%
luau
4.912%
likely better
1%

GC: tree pruning (lazy fill)
79.194ms
91.138ms
2.099%
luau-base

GC: tree pruning (lazy fill)
82.538ms
91.860ms
2.465%
luau
-0.785%
likely same
62%

GC: hashtable keys and values
57.859ms
63.467ms
1.599%
luau-base

GC: hashtable keys and values
52.005ms
63.042ms
1.914%
luau
0.674%
likely same
59%

Upvalues: dead
59.051ms
64.824ms
1.690%
luau-base

Upvalues: dead
53.701ms
59.201ms
1.410%
luau
9.498%
likely better
0%

Upvalues: gray
33.224ms
36.692ms
1.220%
luau-base

Upvalues: gray
30.136ms
33.440ms
1.812%
luau
9.725%
likely better
0%

mandel
79.056ms
91.937ms
2.588%
luau-base

mandel
82.554ms
96.405ms
2.728%
luau
-4.634%
likely worse
1%

LargeTableCtor: array
7.018ms
7.776ms
1.479%
luau-base

LargeTableCtor: array
7.181ms
7.909ms
1.705%
luau
-1.671%
likely same
14%

LargeTableCtor: hash
11.037ms
13.365ms
3.763%
luau-base

LargeTableCtor: hash
12.555ms
13.185ms
1.209%
luau
1.366%
likely same
49%

Pcall: pcall yield
40.021ms
46.948ms
2.037%
luau-base

Pcall: pcall yield
41.319ms
45.848ms
1.374%
luau
2.400%
likely same
6%

3d-raytrace
13.373ms
14.914ms
2.309%
luau-base

3d-raytrace
12.989ms
14.547ms
1.854%
luau
2.520%
likely same
9%

TableCreate: nil
29.400ms
32.672ms
1.763%
luau-base

TableCreate: nil
28.664ms
31.532ms
1.846%
luau
3.613%
likely better
1%

TableCreate: number
35.743ms
38.502ms
1.346%
luau-base

TableCreate: number
33.039ms
38.049ms
1.786%
luau
1.192%
likely same
29%

TableCreate: zerofill
124.501ms
134.746ms
1.299%
luau-base

TableCreate: zerofill
120.485ms
132.865ms
1.269%
luau
1.416%
likely same
12%

TableMarshal: {n=select,...}
19.525ms
22.587ms
1.616%
luau-base

TableMarshal: {n=select,...}
18.772ms
21.633ms
1.486%
luau
4.409%
likely better
0%

TableMarshal: table.pack
15.978ms
18.128ms
2.068%
luau-base

TableMarshal: table.pack
15.249ms
17.807ms
2.084%
luau
1.803%
likely same
22%

TableMarshal: {...}
11.406ms
13.120ms
1.429%
luau-base

TableMarshal: {...}
11.588ms
13.409ms
1.768%
luau
-2.157%
likely same
6%

Total
2796.903ms
3082.856ms
---
luau-base

Total
2802.343ms
3030.599ms
---
luau
1.724%

'luau' change is 1.699% positive on average
Output of python3 bench/bench.py --vm luau-base --compare luau --extra-loops 10:

Test
Min
Average
StdDev%
Driver
Speedup
Significance
P(T<=t)

base64
28.618ms
34.330ms
0.871%
luau-base

base64
29.097ms
33.792ms
0.833%
luau
1.593%
likely better
1%

chess
130.647ms
147.318ms
0.702%
luau-base

chess
125.681ms
144.201ms
0.771%
luau
2.162%
likely better
0%

life
97.753ms
109.337ms
0.610%
luau-base

life
87.368ms
100.052ms
0.690%
luau
9.280%
likely better
0%

qsort
82.126ms
97.804ms
1.053%
luau-base

qsort
76.813ms
95.627ms
1.849%
luau
2.276%
likely better
4%

sha256
25.308ms
29.036ms
0.610%
luau-base

sha256
23.985ms
28.140ms
0.935%
luau
3.182%
likely better
0%

ack
53.159ms
61.760ms
0.654%
luau-base

ack
53.791ms
60.395ms
0.637%
luau
2.260%
likely better
0%

binary-trees
30.521ms
35.391ms
0.641%
luau-base

binary-trees
30.257ms
34.551ms
0.638%
luau
2.431%
likely better
0%

fannkuchen-redux
11.654ms
13.746ms
0.902%
luau-base

fannkuchen-redux
11.396ms
13.676ms
0.654%
luau
0.513%
likely same
37%

fixpoint-fact
55.719ms
65.007ms
0.588%
luau-base

fixpoint-fact
57.571ms
64.589ms
0.490%
luau
0.646%
likely same
10%

heapsort
22.390ms
26.041ms
0.600%
luau-base

heapsort
22.289ms
26.163ms
0.643%
luau
-0.463%
likely same
30%

mandel
73.551ms
82.656ms
0.480%
luau-base

mandel
75.405ms
84.108ms
0.545%
luau
-1.727%
likely worse
0%

n-body
27.983ms
32.354ms
0.596%
luau-base

n-body
26.898ms
32.463ms
0.698%
luau
-0.334%
likely same
47%

qt
62.664ms
74.106ms
1.057%
luau-base

qt
63.549ms
72.662ms
0.682%
luau
1.986%
likely better
0%

queen
1.622ms
1.962ms
0.883%
luau-base

queen
1.630ms
1.965ms
0.909%
luau
-0.174%
likely same
79%

scimark
74.973ms
83.245ms
0.530%
luau-base

scimark
75.792ms
85.079ms
0.460%
luau
-2.156%
likely worse
0%

spectral-norm
14.519ms
17.339ms
0.715%
luau-base

spectral-norm
14.416ms
17.063ms
0.697%
luau
1.617%
likely better
0%

sieve
97.572ms
112.653ms
0.932%
luau-base

sieve
100.176ms
113.292ms
1.138%
luau
-0.564%
likely same
45%

3d-cube
7.465ms
9.050ms
0.645%
luau-base

3d-cube
7.325ms
8.811ms
0.611%
luau
2.714%
likely better
0%

3d-morph
8.291ms
9.838ms
0.569%
luau-base

3d-morph
8.369ms
9.854ms
0.416%
luau
-0.159%
likely same
66%

3d-raytrace
9.137ms
10.639ms
0.425%
luau-base

3d-raytrace
8.884ms
10.699ms
0.717%
luau
-0.559%
likely same
19%

controlflow-recursive
4.360ms
5.113ms
0.752%
luau-base

controlflow-recursive
4.275ms
5.060ms
0.666%
luau
1.049%
likely better
4%

crypto-aes
14.489ms
16.835ms
0.642%
luau-base

crypto-aes
14.440ms
17.190ms
0.451%
luau
-2.064%
likely worse
0%

fannkuch
22.369ms
25.767ms
0.553%
luau-base

fannkuch
22.084ms
25.398ms
0.588%
luau
1.452%
likely better
0%

math-cordic
14.646ms
17.092ms
0.719%
luau-base

math-cordic
14.922ms
17.312ms
0.610%
luau
-1.269%
likely worse
1%

math-partial-sums
4.517ms
5.221ms
0.551%
luau-base

math-partial-sums
4.577ms
5.260ms
0.559%
luau
-0.741%
likely same
6%

n-body-oop
47.531ms
54.330ms
0.552%
luau-base

n-body-oop
48.432ms
55.078ms
0.568%
luau
-1.358%
likely worse
0%

tictactoe
130.463ms
145.940ms
0.465%
luau-base

tictactoe
133.632ms
146.080ms
0.465%
luau
-0.095%
likely same
77%

trig
21.122ms
25.126ms
0.660%
luau-base

trig
21.231ms
25.043ms
0.550%
luau
0.331%
likely same
45%

Total
1175.167ms
1349.035ms
---
luau-base

Total
1164.283ms
1333.603ms
---
luau
1.157%

'luau' change is 0.756% positive on average

It's worth noting that unlike Lua/LuaJIT, we try to not allocate UpVal objects if at all possible. As such, sizeof(UpVal) is mostly inconsequential - simplicity/robustness as well as atomic pause times are more important. Off the top of my head I'm hesitant to make the existing tracking structures more complicated for this reason, however we did discuss maintaining a list of threads with open upvalues (similar to latest upstream twups I think?) at some point, so I'd need to look at the discussions we had at the time. If there are important reasons that make a change like this pay off in the future, or if there's a version of this change that is much simpler but only removes one pointer of the two, that would feel worthwhile.
@vegorov-rbx feel free to chime in as well.
This obviously depends on the specific application and it's possible to write code that's very heavy on upvalues, but here's a snapshot of a moderately sized heap from Roblox Studio:
table      35552848 bytes 164563 objects
proto      15299564 bytes 30622 objects
string      5705443 bytes 28432 objects
function    4122944 bytes 71785 objects
userdata     965604 bytes 32818 objects
thread       394592 bytes   244 objects
upvalue      158064 bytes  3293 objects

atomic pause times are more important

This should make the atomic phase shorter as only the upvalues on threads in the white-thread-with-black-upvalue-list need to be visited and not all open upvalues like it is currently the case, but I did not actually check that it is faster.

If there are important reasons that make a change like this pay off in the future

As noted in the advantages this allows the destructor of the upvalue to be the default one which allows to remove the object without actually calling the destructor. This could speed up sweeping as currently all objects are iterated over which includes dead objects with can be costly. With moving the mark bit out of objects, sweeping could be done on this bitmap without loading all objects into memory.

if there's a version of this change that is much simpler but only removes one pointer of the two

You need at least the thread local pair to find upvalues for closure creation. In case all threads with upvalues are in a list this could be sufficient but would likely not have a trivial destructor.

FWIW I don't think the trivial destructor is realistically attainable for most objects at the moment - it would require a pretty dramatic rework of the entire memory management scheme, as it requires GC to be made aware of all internal pointers. So I would focus on the simplicity of upvalue tracking - iow, it's valuable to not have to remove the upvalue from the list but only if this falls out of simpler traversal / tracking. If the new structures make it more difficult to reason about the upvalue state / lifetime then it would be hard to justify them because we just don't have many upvalue objects normally for openupval traversal or UpVal size to matter.

I don't think the trivial destructor is realistically attainable for most objects at the moment - it would require a pretty dramatic rework of the entire memory management scheme, as it requires GC to be made aware of all internal pointers.

I am actually trying to do this to see the speedup this could give. Upvalues were the only problematic thing and this was a byproduct.

If that's the goal I'm wondering if there's an easier way to get there. The reason why the upvalue destructor needs to unlink the upvalue from the list is that the upvalue might be in the list but might be dead. However, we already traverse the open list during atomic phase to mark the values that weren't marked by the write barrier - can we traverse that list again after all remarks are done, and unlink upvalues that are dead instead? This would mean that during sweeping we know that any dead upvalues are never in any open list. We might need to remove the resurrection logic from upvalue finding but I don't think that's very useful either.

P.S. If the above works, I think this will also allow us to remove the back pointer (making openupval singly-linked) similarly to this PR, because it would effectively make sure that we only unlink the upvalue when traversing the entire open list, instead of having to do it during sweeping in random order.

P.S. If the above works, I think this will also allow us to remove the back pointer (making openupval singly-linked)

This would make closing an upvalue with the LOP_CLOSEUPVALS instruction slow as it would need to travers the full openupval link to find the element.

Yeah, you're right. This can be mitigated by deferring the removal from the global list until atomic stage (so that we clear dead or closed upvalues from the list) but that's probably not very compelling.

This can be mitigated by deferring the removal from the global list until atomic stage (so that we clear dead or closed upvalues from the list) but that's probably not very compelling.

When closing the value the linked list pointers are overwritten by the value since this is a union.
And I think my implementation also has a bug since it assumes that currently running threads are never white.

When closing the value the linked list pointers are overwritten by the value since this is a union.

Yeah, for this to work we would need to move the global next pointer out of the union (which would by default keep the size the same, as the "open" half of the union is much larger than necessary atm).

Thinking about this a little more, deferring removal from global list is not a great idea because it effectively increases the atomic pauses - most open upvalues would probably not stay open for very long.
There are some other ways to structure UpVal to reduce the size but they seem to come at an increased cost of access.
I'd say the best course of action here is to ignore the size of UpVal and focus on removing unlinking by traversing the open upvalue list explicitly when atomic ends.
If we wanted to target size of UpVal after that, the better option seems to be to use twups from upstream - that allows us to have just a single doubly-linked list in UpVal for similar complexity of traversal; maintaining twups is not a huge deal because it doesn't require reasoning about the GC invariants etc. That said, per above I don't think we should go there until there's a demonstrated evidence that the size of UpVal is a problem.

Okay so after some experiments it looks like twups from Lua isn't perfect either - it actually has somewhat similar complexity to this change; by itself it increases upvalue-related work during pauses significantly on our internal tests, and while that can be mitigated by making twups doubly-linked, that is not at all easier to maintain vs what we do right now.
However, I've implemented a simpler version (somewhat different approach) that seems to achieve the desired benefits - more clarity about upvalue state, no need to run custom code during either thread or upvalue sweeping, and simpler management of pretty much everything. It only saves one pointer from the UpVal structure rather than two, but since UpVal size isn't a priority it seems like a better compromise. We'll do further testing next week and hopefully this can land soon.

The aforementioned change has been merged under LuauSimplerUpval flag; once that flag is removed, we'll get a 40-byte UpVal struct and trivial destructors for upvalues and threads (threads do need to destroy stack/callinfo arrays still, naturally).


Package Blocks Keyboard Events

In Windows (all I've tested) when the package is in the project then HardwareKeyboard no longer works.
https://api.flutter.dev/flutter/services/HardwareKeyboard-class.html

在Windows中（我测试过的所有）中，当软件包在项目中时，HardwareKeyboard就不再工作了。> https://api.flutter.dev/flutter/services/HardwareKeyboard-class.html
你是怎么操作它的

I  have the same issue.
Any solution ?

Possibly related to #29


Instructions Exception "Couldn't find the edges for..."

We throw an exception when we cannot find the other edges. It happens for some routes that these cannot be found.
We should make the code more robust and maybe just return the calculated sign?

This depends. Is this a bug then an exception is appropriated or is this just a warning then we could return an unknown instruction - something that we would also need for our clients to support and fallback if a new instruction is unknown

This depends. Is this a bug?

In a way, yes, but it is weird from a user perspective.Logging with an error and simply returning the sign might be ok in this situation. On the other hand throwing an exception would help us identifying issues.

On the other hand throwing an exception would help us identifying issues.

I'm always for this strict behavior (if it is a bug) as it helps us to improve GH in the long term.

Is this error only happening with at least one via point or did it happen also for just one start and one destination?

Is this error only happening with at least one via point or did it happen also for just one start and one destination?

I happened only close to via points. But I think this was related to #1013, so we might be able to simply close this once #1013 is fixed.

Not sure if this is something I should post here, but I think it could be related. I had the "Couldn't find the edges for..." exception popping up in a map I had already downloaded entirely (in this case, the Hamburg map) for a route that is working on the GraphHopper hosted map but it's not working locally.
https://graphhopper.com/maps/?point=53.555691%2C9.95036&point=53.556287%2C9.950358&point=53.543104%2C10.002367&point=53.541534%2C10.002655&locale=en-US&vehicle=car&weighting=fastest&elevation=true&use_miles=false&layer=Omniscale
The route is
Start:    53.555691,9.95036
Point 1: 53.556287,9.950358
Point 2: 53.543104,10.002367
End:      53.541534,10.002655
It's happening with other routes as well, but I am not sure if this is something we should account for as not working all the time or if it can be something that I am not doing correctly. Can anyone shed some light?

@brunoban we are currently working on this issue. This was created with the improved instructions recently. If you are looking for a fast solution, have a look at this PR. I expect it to get merged soon.


Photon Integration

https://github.com/Eeshwar-Krishnan/PhotonFTC

[ ] Add Photon
[ ] Test Photon
[ ] Evaluate improvement

I would like to request that we do not add photon yet, as I want to make sure it won’t conflict with virtual hardware first.

Sounds good, but based on how photon is enabled, and my understanding about how our virtual hardware is being implemented it should be pretty easy to only enable it while on real hardware, but I can hold off


How to set sname and enable remsh

Is there a way to invoke elixir with sname set, so I can remsh into it?
When booting erlang these are set as boot parameters, but I can see no 'correct' place to set sname
node() returns  :nonode@nohost, Node.isalive? returns false.
Yet the hostname is specified in grisp.ini.mustache, and doesn't appear to be picked up by Node

Addressed in https://github.com/grisp/mix_grisp/pull/16


Error on edit markdown

Vim bootstrap version(number in first line): No version given
Operating system: MacOS Catalina
Vim/Nvim version: VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Sep 21 2020 04:03:24)
Languages: markdown

Description
I writing a new Markdown file and when I add a new code block with the sequence of characters "```", the editor auto-remove the third "`" caracter. I tried to remove markdown as an available language on vim-polyglot and remove conceal thinking that it, but it not solve the problem.
The lines that I add to vimrc file was:
let g:vim_markdown_conceal = 0
let g:vim_markdown_conceal_code_blocks = 0

Please can you help me?

I tested here using let g:vim_markdown_conceal_code_blocks = 0 and it works!
You can add these two lines in .vimrc.locals and it will be executed before other configurations.


Add SWMR functionality to filestore for AreaDetector HDF5

This is a WIP (DO NOT MERGE) to add SWMR functionality.

Attn @danielballan, @ambarb, @cmazzoli
This is now working for counts where there are multiple events:
See this notebook for an example of reading data while the scan is running

Looks good. Let's generate SWMR data with a new SPEC, AD_HDF5_SMWR. I guess it's fine to use one general-purpose handler that opens either AD_HDF5 or AD_HDF5_SWMR, but it might be useful to distinguish which is which at some point in the future (without opening the file to look).

This seems reasonable to me.

@danielballan I think that makes a lot of sense. Given that SWMR is an option at creation time of the data then I think is is a really good idea to write the entry with the correct SPEC. The bluesky could even be intelligent enough to see if SWMR is on and encode the spec with AD_HDF5_SWMR or otherwise.

Makes sense to me.

@danielballan @tacaswell
I think this is now good to go, I switched up a bit the HDF5 classes and added the SWMR options. I also added the AreaDetector timestamp handlers as these are generic to AD and not CSX special
Also, unit tests are written, but tests for SWMR are disabled until updated h5py and libhdf5 are done. How do you want to handle it?
So... comments or are we good to go?

That looks better to me.

Looks good to me. I agree the tests can wait until we are deploying latest stable hdf5 lib everywhere. It's worth the effort to wrangle that right now. Merging.

@danielballan @tacaswell thanks!


(rational-editing) Add delete-selection- and subword-mode

I think having selected text be overwritten is expected behaviour for most people.
Being able to M-f and M-b through camelCasedWords also seems a reasonable default.
I am quite new to contributing on GitHub (or anywhere for that matter), so if these should rather be Issues that PRs or I miss anything else here, kindly let me know. :)

Looks good to me! I have the same in my config.
I also have:
  ;; backspace deletes one character instead of one column
  (global-set-key (kbd "DEL") 'backward-delete-char)

but I'm not sure that is general enough for inclusion.

I think that the global-subword-mode should not be used in text-mode or other computer languages.  As you also has global-superword-mode, which suits other languages then Java better.
Better to add subword-mode or superword-mode to hooks.

I think we should leave out global-subword-mode and suggest it perhaps in a documentation file for this project.  I like the delete-selection-mode though!