Cross compilation to WebAssembly / WASI

This PR aims to support to enable cross-compilation to other architectures, and more namely WebAssembly and WASI.
We've detailed a bit more in the announcement blogpost (which should be released soon: https://wasmer.io/posts/py2wasm-a-python-to-wasm-compiler )
Ideally, we would like most of the changes to be in Nuitka, and py2wasm being a very thin layer on top of that.
More patches will be needed but this PR more or less what enables the bulk work (I do not want to overwhelm the maintainer with too many changes at once).
To summarize, this PR should achieve the following:

Being able to cross-compile / serialize&desserialize to 32 bit architecture (Wasm32, WASI)
Added support for some hacks required to compile to WASI

So, cross compilation has not been supported for a long time, however, this looks like a good case for getting it started, esp. if you are willing to help. I invite you to reach out to me in private email if you can allocate a bit of resources to it, and we can split to work.
In my plan, it takes a slave Python process, that is capable of answering questions. Things like os.name and so on, that are compile time evaluated, are currently all hidden behind "hard imports" that e.g. indicate it to be  constant value, and then there are modules, allow listed to be used with getattr that turn the result into compile time values.
There are also functions like
def getComputationResult(node, computation, description, user_provided):
    """With a computation function, execute it and return constant result or
    exception node.

    """

that are used by the core of Nuitka optimization a bunch. Other places that you found, would e.g. have to cover encoding a value for the target platform with struct.pack and the like. Many places are already allowing for proper abstraction of things, and the challenge would be to find ways to not loose a bunch of compile time performance for the normal case.
The plan would be to make an interface, that gets asked questions, and responds with either a slave Python used, which in your case would be a WASI one, where computations are actually executed. Communication to this slave Python (in my goal set) is like is in the transport level SSH and/or subprocess input/output, and cached. This interface can then be chosen. You can lie in that interface all you want, or hard code things if you feel the need for it, but I would prefer caching to be used over hard coding, if we discover this to be slow. For the standard case, said interface would be trivial, but it would provide the hooks needed, to know what WASI gives for os.name without us knowing what it does.
This interface could then be implemented by something that implements more or less things. For Nuitka, I am also interested in cross version compilation. Like being able to compile Python 3.7 as a target Python on 3.13 with no-GIL threads and being able to move the Nuitka core from 2.6 / 3.x dual lingo, to more modern Python.
Another side of the interface would be to also plug Scons C compiler selection and configuration.
I would envision that Nuitka plugins end up doing these things, with the limitation probably that only one plugin can handle the slave Python interface. The plugins interface of Nuitka however, should probably provide the infrastructure for communication between the two Python interpreters. The pickle might be a good start, but I seem to recall that it is incapable of doing everything, we might have to do something, that allows for the host Python to speak of values and types, that only the slave Python actually knows for real, but this can be expanded over time. I am assuming, that you can live with matching the Python version, and I can later figure out how to unlock current code to produce 3.13 programs when Nuitka is running on 2.6 and vice versa.

Awesome @kayhayen . I'll send you an email soon!

dlopen() is always going to fail, so I'd suggest adding something like:
--- a/nuitka/build/static_src/MetaPathBasedLoader.c
+++ b/nuitka/build/static_src/MetaPathBasedLoader.c
@@ -1181,6 +1181,7 @@ static PyObject *_nuitka_loader_load_module(PyObject *self, PyObject *args, PyOb
     PyThreadState *tstate = PyThreadState_GET();

 #ifndef _NUITKA_STANDALONE
+# ifndef __wasi__
     if (installed_extension_modules != NULL) {
         PyObject *extension_module_filename = DICT_GET_ITEM0(tstate, installed_extension_modules, module_name);

@@ -1189,6 +1190,7 @@ static PyObject *_nuitka_loader_load_module(PyObject *self, PyObject *args, PyOb
             return callIntoInstalledExtensionModule(tstate, module_name, extension_module_filename);
         }
     }
+# endif
 #endif

     return IMPORT_EMBEDDED_MODULE(tstate, name);
@@ -1598,6 +1600,7 @@ static PyObject *_nuitka_loader_exec_module(PyObject *self, PyObject *args, PyOb
     // During spec creation, we have populated the dictionary with a filename to load from
     // for extension modules that were found installed in the system and below our package.
 #ifndef _NUITKA_STANDALONE
+# ifndef __wasi__
     if (installed_extension_modules != NULL) {
         PyObject *extension_module_filename = DICT_GET_ITEM0(tstate, installed_extension_modules, module_name);

@@ -1613,6 +1616,7 @@ static PyObject *_nuitka_loader_exec_module(PyObject *self, PyObject *args, PyOb
             return callIntoInstalledExtensionModule(tstate, module_name, extension_module_filename);
         }
     }
+# endif
 #endif

     return EXECUTE_EMBEDDED_MODULE(tstate, module);

This allows zig cc to be used as the wasm compiler.

For zig cc we did a bunch of work.  For 3.11 or higher we got a workaround that uses their C++ compiler or what we call non C11 mode

very interested in this

It seems there is no update and comment recently. May I ask about the merging plan and blocking points about this feature? It will be very helpful for Wasm world to have python as a new guest language. We are very looking forward to it.

I am currently in a crunch for 2.3 release, I will get in touch with the PR authors once I have the time for it after release. Until then 3.12 support was more important.

No worries @kayhayen , just ping me once things are ready to go!

Awesome work! Very exciting to see this coming through 💪

I scheduled a conference, let me rebase the branch for you to 2.3, if I can, I do rebase for bisection at release times

Seems, I didn't get the right to change the PR branch, so what I am going to do, is to cherry pick the parts that conflicted into factory, so that you can just drop those C parts with wasm defines. The OpenBSD changes overtook you there.

So, my merge results are on the factory branch. When you git rebase to the factory branch, simply git rebase --skip, and you should be good. I would find it ideal for the telco to see this already applied. For the C changes that I merge, I will ask you some questions, to add comments or clarify error messages. I didn't merge your Python parts, for which we will need to do some abstraction.

any update?

We are working on it, parts were merged for last release, making many of the original changes unnecessary, and there are other PRs to make this more viable. It's a long term thing though, but the current result is available from them via PyPI even.

Is there a command line option(of nuitka) to enable cross-compilation? At the moment, it seems like the only way I can do this is by modifying the py2wasm repository directly

This is great work! Thx a lot. Looking forward to using it!


[WIP] (require|disallow)YodaConditions: add support for listing specific operators

Fixes https://github.com/jscs-dev/node-jscs/issues/1026

Coverage increased (+0.0%) to 99.0% when pulling ee117d391f2009e1288be46f3af4b2e5b8fb9ed2 on Slayer95:yoda-conditions-array into 0feb327f1551ededd70489d2e2466f776ebe95e4 on jscs-dev:master.

looks good!

Coverage increased (+0.0%) to 99.0% when pulling 3633159e45f4b2014c1e589b6cc1d06b5752135a on Slayer95:yoda-conditions-array into 0feb327f1551ededd70489d2e2466f776ebe95e4 on jscs-dev:master.

Coverage increased (+0.0%) to 99.0% when pulling 6425ebdbca4f45f2c79676d84f47d711910787ef on Slayer95:yoda-conditions-array into 0feb327f1551ededd70489d2e2466f776ebe95e4 on jscs-dev:master.

I thought about it. Look like we lost a reason of denying yoda conditions.
Afaik it was a bad code style because there is a difference between a = 5 and a == 5. And mistype is not a syntax error. But it's impossible to mistype a < 5. Do we really need this fix or we should just rethink this rule?

I agree that yoda conditions are only useful to guard against that typo (i.e. requireYodaConditions can be limited to it. However, that's more easily dealt with other sort of rules, like JSHint's "Expected a conditional expression and instead saw an assignment".
Regarding disallowYodaConditions, an argument based on aesthetics and backwards compatibility can be made to keep supporting the current behavior, but I wouldn't mind its getting restricted to ==.
Interestingly, ESLint provides the option to forbid Yoda Conditions except for ranges, which is my use case.
requireYodaConditions only needs to distinguish between = and ==

Is this PR mergeable (once rebased)?
cc @zxqfox @hzoo

@mrjoelkemp I think it's too complicated and can be simplified. I'm not sure that 5 <= value should be interpreted as yoda condition.

Coverage increased (+0.0%) to 99.09% when pulling ef4b343f792756e78b33f63bac3fa65b2f268e10 on Slayer95:yoda-conditions-array into 02bf7ac13d7082ea349ac5b9b77d41380dfdd110 on jscs-dev:master.

Is that still WIP?

Waiting for feedback / replies to @zxqof

ping @zxqfox

Sorry for delay, somehow missed this PR. ;-(

Coverage increased (+0.0%) to 99.17% when pulling 7034ae9d4222f3e87e9b234512139aa4abc669f8 on Slayer95:yoda-conditions-array into f963cd496d4e5d89bb8ffee9abc1c8114b69f41f on jscs-dev:master.

Updated tests as requested.

@Slayer95 Thanks!
LGTM

LGTM! Good job!

Just a remainder, if pr has two "LGTM" you can merge it, @mdevils, @zxqfox could one of you do that?

@markelog Yes, sure :)

Merged into 2.0. Thank you!


[Feature] Update gbdk-2020

Is your feature request related to a problem? Please describe.
No.
Describe the solution you'd like
Updating gbdk-2020. There have been several pull requests that would seemingly speed things up, such as faster sprite handling.
Describe alternatives you've considered
N/A
Additional context
I have compiled the latest gbdk-2020 for Ubuntu and get the following output when trying to build a game in GB Studio 2 Beta:
Preparing variables...
Preparing strings...
Preparing images...
Preparing ui...
Preparing sprites...
Preparing avatars...
Preparing music...
Preparing scenes...
Preparation complete
Compiling events...
Unlink _gbsbuild
Copy default engine
Looking for local engine in assets/engine
Local engine not found, using default engine
Generate header: data_ptrs.h
Generate data file: data_ptrs.c
Generate header: banks.h
Generate data file: bank_6.c
Generate data file: bank_7.c
Generate object file: bank_6.o
Generate object file: bank_7.o
music_track_101_ approx size in bytes: 6056
music_track_102_ approx size in bytes: 263
Approximate Music bank sizes: 1
Music bank for each track: 8, 8
data_ptrs.c rewritten with new song banks

lcc -Wa-l -Iinclude -c -o obj/main.o src/main.c
lcc -Wa-l -Iinclude -c -o obj/asm.macros.o src/core/asm.macros.s
lcc -Wa-l -Iinclude -c -o obj/Scroll_a.o src/core/Scroll_a.s
lcc -Wa-l -Iinclude -c -o obj/Camera_a.o src/core/Camera_a.s
lcc -Wa-l -Iinclude -c -o obj/gbt_player_bank1.o src/core/gbt_player_bank1.s
lcc -Wa-l -Iinclude -c -o obj/Actor_a.o src/core/Actor_a.s
lcc -Wa-l -Iinclude -c -o obj/gbt_player.o src/core/gbt_player.s
lcc -Wa-l -Iinclude -c -o obj/Math.o src/core/Math.c
lcc -Wa-l -Iinclude -c -o obj/Scroll.o src/core/Scroll.c
lcc -Wa-l -Iinclude -c -o obj/Sprite.o src/core/Sprite.c
lcc -Wa-l -Iinclude -c -o obj/Scroll_b.o src/core/Scroll_b.c
lcc -Wa-l -Iinclude -c -o obj/Sprite_b.o src/core/Sprite_b.c
lcc -Wa-l -Iinclude -c -o obj/Camera.o src/core/Camera.c
lcc -Wa-l -Iinclude -c -o obj/Core_Main.o src/core/Core_Main.c
lcc -Wa-l -Iinclude -c -o obj/FadeManager.o src/core/FadeManager.c
lcc -Wa-l -Iinclude -c -o obj/FadeManager_b.o src/core/FadeManager_b.c
lcc -Wa-l -Iinclude -c -o obj/Projectiles.o src/core/Projectiles.c
lcc -Wa-l -Iinclude -c -o obj/Projectiles_b.o src/core/Projectiles_b.c
lcc -Wa-l -Iinclude -c -o obj/UI_b.o src/core/UI_b.c
lcc -Wa-l -Iinclude -c -o obj/MusicManager.o src/core/MusicManager.c
lcc -Wa-l -Iinclude -c -o obj/Actor.o src/core/Actor.c
lcc -Wa-l -Iinclude -c -o obj/ScriptRunner.o src/core/ScriptRunner.c
lcc -Wa-l -Iinclude -c -o obj/Actor_b.o src/core/Actor_b.c
lcc -Wa-l -Iinclude -c -o obj/BankManager.o src/core/BankManager.c
lcc -Wa-l -Iinclude -c -o obj/ScriptRunner_b.o src/core/ScriptRunner_b.c
lcc -Wa-l -Iinclude -c -o obj/Stack.o src/core/Stack.c
lcc -Wa-l -Iinclude -c -o obj/DataManager.o src/core/DataManager.c
lcc -Wa-l -Iinclude -c -o obj/Collision.o src/core/Collision.c
lcc -Wa-l -Iinclude -c -o obj/Trigger.o src/core/Trigger.c
lcc -Wa-l -Iinclude -c -o obj/Trigger_b.o src/core/Trigger_b.c
lcc -Wa-l -Iinclude -c -o obj/Input.o src/core/Input.c
lcc -Wa-l -Iinclude -c -o obj/UI.o src/core/UI.c
lcc -Wa-l -Iinclude -c -o obj/BankData.o src/core/BankData.c
lcc -Wa-l -Iinclude -c -o obj/Palette.o src/core/Palette.c
lcc -Wa-l -Iinclude -c -o obj/Platform.o src/states/Platform.c
lcc -Wa-l -Iinclude -c -o obj/PointNClick.o src/states/PointNClick.c
lcc -Wa-l -Iinclude -c -o obj/Shmup.o src/states/Shmup.c
lcc -Wa-l -Iinclude -c -o obj/TopDown.o src/states/TopDown.c
lcc -Wa-l -Iinclude -c -o obj/Adventure.o src/states/Adventure.c
lcc -Wa-l -Iinclude -c -o obj/data_ptrs.o src/data/data_ptrs.c
mkdir -p build/rom
lcc -Wl-yo16 -Wa-l -Wl-g_shadow_OAM=0xC000 -Wl-m -Wl-j -Wl-yt27 -Wl-ya4 -Iinclude -o build/rom/game.gb obj/main.o obj/asm.macros.o obj/Scroll_a.o obj/Camera_a.o obj/gbt_player_bank1.o obj/Actor_a.o obj/gbt_player.o obj/Math.o obj/Scroll.o obj/Sprite.o obj/Scroll_b.o obj/Sprite_b.o obj/Camera.o obj/Core_Main.o obj/FadeManager.o obj/FadeManager_b.o obj/Projectiles.o obj/Projectiles_b.o obj/UI_b.o obj/MusicManager.o obj/Actor.o obj/ScriptRunner.o obj/Actor_b.o obj/BankManager.o obj/ScriptRunner_b.o obj/Stack.o obj/DataManager.o obj/Collision.o obj/Trigger.o obj/Trigger_b.o obj/Input.o obj/UI.o obj/BankData.o obj/Palette.o obj/Platform.o obj/PointNClick.o obj/Shmup.o obj/TopDown.o obj/Adventure.o obj/bank_6.o obj/data_ptrs.o obj/bank_7.o obj/music_bank_8.o
?ASlink-Warning-Undefined Global _NR13_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR14_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR50_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR41_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _set_sprite_prop referenced by module
?ASlink-Warning-Undefined Global _NR51_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR42_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR52_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR43_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _strcpy referenced by module _tmp_lcc353391
?ASlink-Warning-Undefined Global _rand referenced by module _tmp_lcc353091
?ASlink-Warning-Undefined Global _NR44_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _LYC_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _SCX_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _SCY_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _move_sprite referenced by module
?ASlink-Warning-Undefined Global _BGB_profiler_message referenced by module _tmp_lcc352251
?ASlink-Warning-Undefined Global __moduchar
referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global __moduchar referenced by module _tmp_lcc353091
?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc352811
?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353091
?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353551
?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353591
?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353631
?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353671
?ASlink-Warning-Undefined Global _LCDC_REG referenced by module _tmp_lcc352451
?ASlink-Warning-Undefined Global _LCDC_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _LCDC_REG referenced by module _tmp_lcc353091
?ASlink-Warning-Undefined Global __divuchar referenced by module _tmp_lcc353091
?ASlink-Warning-Undefined Global __mulint referenced by module _tmp_lcc352451
?ASlink-Warning-Undefined Global __mulint referenced by module _tmp_lcc353231
?ASlink-Warning-Undefined Global _set_win_tiles referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global _set_win_tiles referenced by module _tmp_lcc353391
?ASlink-Warning-Undefined Global _set_sprite_data referenced by module _tmp_lcc353181
?ASlink-Warning-Undefined Global _set_sprite_data referenced by module _tmp_lcc353431
?ASlink-Warning-Undefined Global _add_LCD referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global __muluchar referenced by module _tmp_lcc353091
?ASlink-Warning-Undefined Global _joypad referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global ___sdcc_call_hl referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global ___sdcc_call_hl referenced by module _tmp_lcc352971
?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc352971
?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc353181
?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc353391
?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc353431
?ASlink-Warning-Undefined Global _WX_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _WX_REG referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global _WY_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _WY_REG referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global _OBP0_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _OBP0_REG referenced by module _tmp_lcc352731
?ASlink-Warning-Undefined Global _OBP1_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _set_sprite_tile referenced by module
?ASlink-Warning-Undefined Global _strcat referenced by module _tmp_lcc353391
?ASlink-Warning-Undefined Global _add_TIM referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _TAC_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _BGP_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _BGP_REG referenced by module _tmp_lcc352731
?ASlink-Warning-Undefined Global _strlen referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global _set_bkg_data referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global _set_bkg_data referenced by module _tmp_lcc353181
?ASlink-Warning-Undefined Global _set_bkg_data referenced by module _tmp_lcc353431
?ASlink-Warning-Undefined Global _STAT_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _initrand referenced by module _tmp_lcc352411
?ASlink-Warning-Undefined Global __modsint referenced by module _tmp_lcc352811
?ASlink-Warning-Undefined Global __modsint referenced by module _tmp_lcc352851
?ASlink-Warning-Undefined Global _NR10_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _NR11_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global _TMA_REG referenced by module _tmp_lcc352651
?ASlink-Warning-Undefined Global _NR12_REG referenced by module _tmp_lcc352891
?ASlink-Warning-Undefined Global __divsint referenced by module _tmp_lcc352851
make: *** [Makefile:53: build/rom/game.gb] Error 1
2

Warnings:
- ?ASlink-Warning-Undefined Global _NR13_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR14_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR50_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR41_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _set_sprite_prop referenced by module
- ?ASlink-Warning-Undefined Global _NR51_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR42_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR52_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR43_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _strcpy referenced by module _tmp_lcc353391
- ?ASlink-Warning-Undefined Global _rand referenced by module _tmp_lcc353091
- ?ASlink-Warning-Undefined Global _NR44_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _LYC_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _SCX_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _SCY_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _move_sprite referenced by module
- ?ASlink-Warning-Undefined Global _BGB_profiler_message referenced by module _tmp_lcc352251
- ?ASlink-Warning-Undefined Global __moduchar
- referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global __moduchar referenced by module _tmp_lcc353091
- ?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc352811
- ?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353091
- ?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353551
- ?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353591
- ?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353631
- ?ASlink-Warning-Undefined Global __muluschar referenced by module _tmp_lcc353671
- ?ASlink-Warning-Undefined Global _LCDC_REG referenced by module _tmp_lcc352451
- ?ASlink-Warning-Undefined Global _LCDC_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _LCDC_REG referenced by module _tmp_lcc353091
- ?ASlink-Warning-Undefined Global __divuchar referenced by module _tmp_lcc353091
- ?ASlink-Warning-Undefined Global __mulint referenced by module _tmp_lcc352451
- ?ASlink-Warning-Undefined Global __mulint referenced by module _tmp_lcc353231
- ?ASlink-Warning-Undefined Global _set_win_tiles referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global _set_win_tiles referenced by module _tmp_lcc353391
- ?ASlink-Warning-Undefined Global _set_sprite_data referenced by module _tmp_lcc353181
- ?ASlink-Warning-Undefined Global _set_sprite_data referenced by module _tmp_lcc353431
- ?ASlink-Warning-Undefined Global _add_LCD referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global __muluchar referenced by module _tmp_lcc353091
- ?ASlink-Warning-Undefined Global _joypad referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global ___sdcc_call_hl referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global ___sdcc_call_hl referenced by module _tmp_lcc352971
- ?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc352971
- ?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc353181
- ?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc353391
- ?ASlink-Warning-Undefined Global _memcpy referenced by module _tmp_lcc353431
- ?ASlink-Warning-Undefined Global _WX_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _WX_REG referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global _WY_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _WY_REG referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global _OBP0_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _OBP0_REG referenced by module _tmp_lcc352731
- ?ASlink-Warning-Undefined Global _OBP1_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _set_sprite_tile referenced by module
- ?ASlink-Warning-Undefined Global _strcat referenced by module _tmp_lcc353391
- ?ASlink-Warning-Undefined Global _add_TIM referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _TAC_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _BGP_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _BGP_REG referenced by module _tmp_lcc352731
- ?ASlink-Warning-Undefined Global _strlen referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global _set_bkg_data referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global _set_bkg_data referenced by module _tmp_lcc353181
- ?ASlink-Warning-Undefined Global _set_bkg_data referenced by module _tmp_lcc353431
- ?ASlink-Warning-Undefined Global _STAT_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _initrand referenced by module _tmp_lcc352411
- ?ASlink-Warning-Undefined Global __modsint referenced by module _tmp_lcc352811
- ?ASlink-Warning-Undefined Global __modsint referenced by module _tmp_lcc352851
- ?ASlink-Warning-Undefined Global _NR10_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _NR11_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global _TMA_REG referenced by module _tmp_lcc352651
- ?ASlink-Warning-Undefined Global _NR12_REG referenced by module _tmp_lcc352891
- ?ASlink-Warning-Undefined Global __divsint referenced by module _tmp_lcc352851
- make: *** [Makefile:53: build/rom/game.gb] Error 1
- 2

Additionally, the pull request I linked above states:

.OAM name changed to _shadow_OAM - must pass that into the linker through the commandline: -g _shadow_OAM=0xC000

I am not sure if I did this correctly by modifying the LFLAGS line in gb-studio/appData/src/gb/Makefile to read:
LFLAGS = $(LFLAGS_NBANKS) -Wa-l -Wl-g_shadow_OAM=0xC000 -Wl-m -Wl-j -Wl-yt27 -Wl-ya4
but it did remove one warning for _shadow_OAM being undefined.

I already made a pull request for this: https://github.com/chrismaltby/gb-studio/issues/424

I'm not sure I see a pull request. However, the v2beta branch already switched over from gbdk to gbdk-2020. To my understanding, there have been further optimisations made to gbdk-2020 since it was incorporated in the beta, which would be great to have.

Sorry, I meant issue. 😅

Update:
Using the latest commit of gbdk-2020, the linker warnings have gone down drastically. "-Wl-g_shadow_OAM=0xC000" is no longer necessary, as gbdk-2020 now sets this. Building the same project now only gives the following warnings:
?ASlink-Warning-Undefined Global '_set_sprite_prop' referenced by module ''
?ASlink-Warning-Undefined Global '_move_sprite' referenced by module ''
?ASlink-Warning-Undefined Global '_set_sprite_tile' referenced by module ''
make: *** [Makefile:53: build/rom/game.gb] Error 1
2

"_set_sprite_prop", "_move_sprite", and "_set_sprite_tile" were removed in the same commit and are only referenced in appData/src/gb/src/core/Actor_a.s
I'm not sure what it would take to fix this. I tried renaming "_set_sprite_prop" to ".set_sprite_prop", "_set_sprite_tile" to ".set_sprite_tile", and "_move_sprite" to "move_sprite" and now get this error:
error: size of the buffer is too small.
make: *** [Makefile:53: build/rom/game.gb] Error 1
2

I imagine it isn't quite as simple as renaming things, so I have to leave this to the pros while I learn more, but this is what I've discovered so far.

Might wait for a master release of gbdk2020, but thanks for testing with the development branch!
Looks like @untoxa has found a faster way of doing these functions for c, but we'll need to redo how it's called in asm, you're probably on the right track swaping _ for . But we might need to pass data differently, seems all the indexing math is added with the inline c compile instead of the assembly function.

most of GBDK library is designed as follows: you have a function that begins with a dot, say .SOME , that contains "payload" and C "wrapper", that beginst with underline _SOME and the purpose of that wrapper is to make .SOME be compatible with smallc calling convention. It fetches parameters from stack, put them into registers that function expects, preserves BC register pair, returns result in HLDE if expected.
if you writing something in ASM, it is couter-productive to put function parameters on stack to make them being fetched and so on. you should call .SOME instead, passing parameters into it, the way it expects.
new GBDK uses efficient inline functions for *_sprite() functions. that means that there is no code in the library that is called from C now, the code of those functions is inlined with the code of the "game", that's why wrappers are no longer needed. it turns out, that inlining of those functions emits smaller code than for pushing parameters and calling the wrapper! :)
as for ASM, you should either access _shadow_OAM array directly (making the similar inlining), or call those LEGACY .SOME functions. of course you should pass parameters into them differently. that will reduce code size and increase speed twice or, maybe, more.

ps: btw, mentioned above, UpdateActors() does not preserve BC.

check please: https://github.com/untoxa/gb-studio/commit/e2acf078fc5fe50eccaf39cfeb5f1302c083359f

that because of the problems with LCC. they are fixed by @zal0 today, i believe. just try the 4.0 release.

No luck, I'm afraid. Still getting the following using the released version:
?ASlink-Error-<cannot open> : "build/rom/game.ihx"
make: *** [Makefile:53: build/rom/game.gb] Error 1
2

On Manjaro, your build of GB Studio V2, official released version of gbdk-2020 v4.0 for Linux.

what is the command it issues? with that line 53. i mean what that line of makefile
$(CC)` $(LFLAGS) -Iinclude -o $@ $^
is resolved into.

With the unmodified makefile, it's resolved to
lcc -Wl-yo16 -Wa-l -Wl-m -Wl-j -Wl-yt27 -Wl-ya4 -Wl-yp0x143=0x80 -Iinclude -o build/rom/game.gb obj/main.o obj/Actor_a.o obj/asm.macros.o obj/Camera_a.o obj/gbt_player_bank1.o obj/gbt_player.o obj/Scroll_a.o obj/Actor_b.o obj/Actor.o obj/BankData.o obj/BankManager.o obj/Camera.o obj/Collision.o obj/Core_Main.o obj/DataManager.o obj/FadeManager_b.o obj/FadeManager.o obj/Input.o obj/Math.o obj/MusicManager.o obj/Palette.o obj/Projectiles_b.o obj/Projectiles.o obj/ScriptRunner_b.o obj/ScriptRunner.o obj/Scroll_b.o obj/Scroll.o obj/Sprite_b.o obj/Sprite.o obj/Stack.o obj/Trigger_b.o obj/Trigger.o obj/UI_b.o obj/UI.o obj/Adventure.o obj/Platform.o obj/PointNClick.o obj/Shmup.o obj/TopDown.o obj/bank_10.o obj/bank_6.o obj/bank_7.o obj/bank_8.o obj/bank_9.o obj/data_ptrs.o obj/music_bank_11.o

Try replacing -Wl-yp0x143=0x80 with -Wm-c

direct patching of roms with -Wl-yp= is not supported anymore. use gameboy specific options of makebin.

No luck. With a DMG (non-Color) game, results in:
lcc -Wl-yo16 -Wa-l -Wl-m -Wl-j -Wl-yt27 -Wl-ya4 -Iinclude -o build/rom/game.gb obj/main.o obj/Actor_a.o obj/asm.macros.o obj/Camera_a.o obj/gbt_player_bank1.o obj/gbt_player.o obj/Scroll_a.o obj/Actor_b.o obj/Actor.o obj/BankData.o obj/BankManager.o obj/Camera.o obj/Collision.o obj/Core_Main.o obj/DataManager.o obj/FadeManager_b.o obj/FadeManager.o obj/Input.o obj/Math.o obj/MusicManager.o obj/Palette.o obj/Projectiles_b.o obj/Projectiles.o obj/ScriptRunner_b.o obj/ScriptRunner.o obj/Scroll_b.o obj/Scroll.o obj/Sprite_b.o obj/Sprite.o obj/Stack.o obj/Trigger_b.o obj/Trigger.o obj/UI_b.o obj/UI.o obj/Adventure.o obj/Platform.o obj/PointNClick.o obj/Shmup.o obj/TopDown.o obj/bank_6.o obj/bank_7.o obj/bank_8.o obj/data_ptrs.o
error: size of the buffer is too small.
make: *** [Makefile:53: build/rom/game.gb] Error 1
2

and with a Color game:
lcc -Wl-yo16 -Wa-l -Wl-m -Wl-j -Wl-yt27 -Wl-ya4 -Wm-yc -Iinclude -o build/rom/game.gb obj/main.o obj/Actor_a.o obj/asm.macros.o obj/Camera_a.o obj/gbt_player_bank1.o obj/gbt_player.o obj/Scroll_a.o obj/Actor_b.o obj/Actor.o obj/BankData.o obj/BankManager.o obj/Camera.o obj/Collision.o obj/Core_Main.o obj/DataManager.o obj/FadeManager_b.o obj/FadeManager.o obj/Input.o obj/Math.o obj/MusicManager.o obj/Palette.o obj/Projectiles_b.o obj/Projectiles.o obj/ScriptRunner_b.o obj/ScriptRunner.o obj/Scroll_b.o obj/Scroll.o obj/Sprite_b.o obj/Sprite.o obj/Stack.o obj/Trigger_b.o obj/Trigger.o obj/UI_b.o obj/UI.o obj/Adventure.o obj/Platform.o obj/PointNClick.o obj/Shmup.o obj/TopDown.o obj/bank_10.o obj/bank_6.o obj/bank_7.o obj/bank_8.o obj/bank_9.o obj/data_ptrs.o obj/music_bank_11.o
error: can't open : No such file or directory
make: *** [Makefile:53: build/rom/game.gb] Error 1
2

did you compile GBDK-2020 yourself or took a binary release?

Both, same output.
sdcc -v gives me:
SDCC : mcs51/z80/z180/r2k/r3ka/gbz80/tlcs90/ez80_z80/ds390/pic16/pic14/TININative/ds400/hc08/s08/stm8/pdk13/pdk14/pdk15 4.0.0 #11528 (Linux)
published under GNU General Public License (GPL)

SDCC on this machine was from the latest binary release. On my arm64-based PinePhone, had to compile SDCC myself. It's currently out of battery, so can't run the command atm, but I get the same output there.

no. you should use at least 4.0.3 #11875 from here: http://sdcc.sourceforge.net/snap.php

Oops, yeah I goofed that up, my apologies.
SDCC : mcs51/z80/z180/r2k/r3ka/gbz80/tlcs90/ez80_z80/z80n/ds390/pic16/pic14/TININative/ds400/hc08/s08/stm8/pdk13/pdk14/pdk15 4.0.3 #11875 (Linux)
published under GNU General Public License (GPL)

Unfortunately, no change afterwards for Color mode. DMG mode now builds the game, but the screen is entirely blank.

if you built the gbdk library yourself with #11528, you must rebuild it again with #11875. or just take a precompiled release.

That's what I've done, rebuild with #11875 and tried the precompiled release. Same result. I'll try Windows and see if there's any difference.

is it color or dmg? long build because of the optimizer that tries to allocate registers better.

This is DMG. Color doesn't build at all.
Is the optimiser only enabled on the Windows build?

no. should be identical on all targets. i'm not sure what's the difference in gbc vs dmg here. build process only differs with that -Wm-yc switch that can't be the problem. can you put your rom somewhere?

Strange. I'm not sure why an old laptop could build it from scratch in just over a minute (4GB RAM, integrated graphics, dual-core CPU @1.2GHz) whereas my desktop took five minutes (32GB RAM, GTX 970, six-core/12-thread CPU @3.8GHz). That seems a wee off to me. Mind you, I just wiped the laptop and installed Manjaro a couple days ago, and the Windows install on my desktop is a couple years old. I'll wipe it when I get the chance and test that part again hopefully soon. It's possible it could just be my Windows install, though the older gbdk took less than a minute on that system.
Added -Wl-w to the end of LFLAGS on line 6 of the makefile. The DMG game is a custom game I've been working on, but I did use the generic example for the Color game, which doesn't build.
game.zip
That is the output of the DMG game.

do you have a working copy built with the previous gbdk-2020? i want to compare and also be sure that the problem is in the update, not your game.
LoadScene() loads garbage.

Aye, here it is:
game.zip
Built from my Linux machine, latest commit of the v2beta branch

that is your old rom: https://d.radikal.ru/d18/2010/e2/213f93276ec6.png
that is your new rom: https://b.radikal.ru/b04/2010/c9/6ec8b832ca67.png
as you can see, some data is missing. probably, we found a bug in makebin?

ps: you might have a problem with your old rom too: bank6 seem to overflow.

hm... i just modified standard banks.c example to occupy banks 4-7, and i have this picture:

so, the problem seems to be in GBStudio or your project, not GBDK.

Great find! The project I created just uses the standard features of GB Studio; there's no custom code by ejecting the engine. Also, the sample project from GB Studio V1 (a DMG game) does the same thing, so I have to imagine this is something to do with GB Studio.
As a side note: I did make a couple bastardised versions of gbdk-2020 two weeks ago. I only have one of the two versions and deleted the other. The first, I tried taking the version of gbdk-2020 that is in GB Studio V2 now and manually modified the files, adding most commits up to this one with the exception of the changes causing the massive amount of warnings as listed in the OP. DMG and Color games built fine, but the sprites were beyond corrupted. Mind you, this was when I simply changed "_set_sprite_tile" to ".set_sprite_tile" and so on in GB Studio. Unfortunately, that was the one I deleted. The other version I manually merged fewer commits, some of the itoa and sprintf ones, which I do have, but doesn't have the sprite and screen optimisations. This one works as expected. I'll see if I can replicate the first bastardised version to remember what I omitted. Either the things I omitted or something from after the commit above can hopefully give us a clue what is going on.

Take a 100% working branch of gb studio 2.0. Fork it. Compile it as is, make sure everything is working (example project).
Put binaries from gbdk-2020 v4.0 into that working fork (take everything from the binary release, don't compile yourself).
Take my two commits only from that pull request: actor_a.s and makefile. Update the core with them.
Compile example project once more (make a full cleanup before).
Check.

You described what I've already done several times.

Latest commit of GB Studio 2.0 works well. Example project compiles.
Deleted the gbdk folder from GB Studio 2.0 and replaced with precompiled version.
Edited the files actor_a.s and makefile appropriately.
Cleared /tmp and the build cache. Hit the build button.
No good.

i took GBStudio public beta 3, compiled version from itch.io
extracted sources of demo project with menu item
edited make.bat: corrected paths to lcc and files, added assets that were missing (!!!):
gbdk\bin\lcc -Wa-l -Iinclude -DCGB -c -o obj/bank_9.o        src/data/bank_9.c        gbdk\bin\lcc -Wa-l -Iinclude -DCGB -c -o obj/bank_8.o        src/data/bank_8.c        gbdk\bin\lcc -Wa-l -Iinclude -DCGB -c -o obj/bank_7.o        src/data/bank_7.c        gbdk\bin\lcc -Wa-l -Iinclude -DCGB -c -o obj/bank_6.o        src/data/bank_6.c        gbdk\bin\lcc -Wa-l -Iinclude -DCGB -c -o obj/bank_10.o       src/data/bank_10.c       gbdk\bin\lcc -Wa-l -Iinclude -DCGB -c -o obj/music_bank_11.o src/music/music_bank_11.c
put updated actor_a.s
ran make.bat
i got a GBC rom with everything in place with the original one

unfortunately, it is not running, something else is wrong. i debugged that a bit - at least VRAM is loaded with the correct tiles, something breaks when it tries to run the script.  this is what i get:

at least, compiling process is smooth (except pt. 3 above), rom layout is correct. it's a GBC rom too.

Thanks @untoxa I've tried out your branch with the GBDK 2020 v4 builds and these are my findings.
Looks like the Mac release on https://github.com/Zal0/gbdk-2020/releases/tag/v4.0 might have an old SDCC version, the included binary is this one
SDCC : mcs51/z80/z180/r2k/r3ka/gbz80/tlcs90/ez80_z80/z80n/ds390/pic16/pic14/TININative/ds400/hc08/s08/stm8/pdk13/pdk14/pdk15 4.0.3 #11868 (Mac OS X x86_64) (11868 rather than 11875)
I've compiled it from source with the right SDCC nightly but even then I'm getting a similar issue to @patrickmollohan
?ASlink-Error-<cannot open> : "build/rom/game.ihx"
make: *** [build/rom/game.gb] Error 1

This is my SDCC now
SDCC : mcs51/z80/z180/r2k/r3ka/gbz80/tlcs90/ez80_z80/z80n/ds390/pic16/pic14/TININative/ds400/hc08/s08/stm8/pdk13/pdk14/pdk15 4.0.3 #11875 (Mac OS X x86_64)
published under GNU General Public License (GPL)

This is the game I'm trying to build
game-gbdk2020v4-test.zip
and I'm just running the following in a Bash prompt to compile
export GBDKDIR="/Users/cmaltby/Desktop/gbdk2020/"
export PATH="/Users/cmaltby/Desktop/gbdk2020/bin:$PATH"
make clean
make rom

I've added -v to  CFLAGS to see all the individual commands that lcc is calling and put the full log up here if that helps:
https://gist.github.com/chrismaltby/2e3987a1d684408a1377e516e6d365c9
I also tried the Windows release and have managed to get a little bit further, it generates a ROM that's just a blank screen! Can probably debug from there though, my first thought would be to strip everything out to just get a single sprite to appear and bring each file back in one-by-one until it stops working.
The problem is though the Windows version is running very slow for me, the project above takes around 7 minutes on a machine with 2.4 GHz 8-Core Intel Core i9 / 64 GB 2667 MHz DDR4!
I'll keep trying things but this is what I've worked out so far.
I'm wondering if the best option is to try to get the game from that attached zip working first (forgetting GB Studio for the moment) and then we can use the working version to make any updates needed to the app.
Thanks everyone for looking into this, definitely excited to get this working!

ah! i commented yout gist, maybe here that info is more useful:
lcc -Wl-yo -Wa-l
must be:
-yo<HOW MUCH>
define the $(CART_SIZE)
i also tried another daily build of SDCC (#11885) it builds much faster in windows. but the problem with non-working rom exists.

crashes on updateFuncs[scene_type]();

scene_type is invalid (0x11h) after LoadScene()

I suggest using far_ptr.h instead of those constructions with pushbank/popbank.

Oh sorry my bad, yeah I've just set CART_SIZE and it builds to the same blank screen we're seeing on Windows, progress!

I actually got the BankManager from ZGB https://github.com/Zal0/ZGB/commit/5b200d37831e52c7cf125a978eaf8b6b76b326d2#diff-0febabe82127c9b9438610d6496b2a57 looks like theres been an update to move it inline asm in recent releases https://github.com/Zal0/ZGB/blob/master/common/src/BankManager.c
Hadn't seen far_ptr.h before, was that new in GBDK2020, just looking at the examples now. So am I right in thinking you use the TO_FAR_PTR macro (or to_far_ptr function to do it in runtime) giving it a function point and a bank number to make a far pointer and you can call it with FAR_CALL and it just handles the bank switching for you?

Definitely. Yes. Call stub is not reentrant, because it uses a global var for the short time, but that is not a problem, unless you are using my posix-like threading library for game boy (see examples among my repos).

Cool, thanks I'll try that out. I've just made a new repo just for handling this issue.
https://github.com/chrismaltby/gbdk2020v4-testgame
This is just a single scene taken from the latest sample project, I've included the sample files and the working rom made by the current GB Studio for comparison.

I'm just going to try branches off that to see what works and what doesn't

So I've had a go at fixing that project, starting by removing almost everything and using the galaxy.c example and bringing parts in bit by bit and got to the minimal changes to make a working version of doing the following
https://github.com/chrismaltby/gbdk2020v4-testgame/pull/2/files
adding a BGB log message around LoadScene fixed it!?! Well, kind of fixed it

all the sprites are using the first sprite tile which I think might be an issue with the new Actor_a.s file, not had chance to look into that yet. Other than that it seems to work as expected.
After working out a single log line could fix the engine I tried another fix which is a single line setting the value of bank to a global that's unneeded and gets overwritten a few lines later fixes it too...
https://github.com/chrismaltby/gbdk2020v4-testgame/pull/3/files
I don't know how the new compiler works but looking at the code around it and what happens it almost feels like the compiler is deferring fetching the value of bank until after the POP_BANK so when it tries to use it on 179 the bank has already switched and it's fetching the wrong value. Could be very wrong about that though.
https://github.com/chrismaltby/gbdk2020v4-testgame/blob/41677d575766c4c11dd0099f028a5342a1ca96f0/src/core/DataManager.c#L166-L179
So getting closer, though it would be good to know why this line has an issue because potentially there's other similar bugs that have been introduced that might be harder to spot.
The other thing I noticed is that if I compile with the environment variable set to COLOR=1 to make a GBC rom it fails to build, might be some flags are wrong again there, this is the full output I'm getting
https://gist.github.com/chrismaltby/cd07264c6d765d9303423c18c5ea6bea
Also I had a go at switching out some of the functions to use far_ptr.h but I didn't want to change too much since I managed to get it working without them. Will definitely switch them out once we've narrowed down this issue though.

i suggest
  static UBYTE bank;
  static UBYTE* data_ptr;

Actor_a.s -- my bad:
.set_spr_prop:                                  ;; c - sprite no, e - value, preserves bc, spoils hl
        push bc
        ld hl, #_shadow_OAM+3
        sla c
        sla c
        ld b, #0
        add hl, bc
        ld (hl), e
        pop bc
        ret

.set_spr_tile:                                  ;; c - sprite no, b - value, preserves bc, spoils hl
        push bc
        ld hl, #_shadow_OAM+2
        sla c
        sla c
        ld a, b
        ld b, #0
        add hl, bc
        ld (hl), a
        pop bc
        ret

i noticed that save name is garbaged, and also platformer crashes. overall performance is better.

@chrismaltby can you please help me try to cut out piece of code with that weird SDCC behaviour to make a minimal example of faulty code generation? that is important for reporting the bug to SDCC team. declaring bank as static https://github.com/chrismaltby/gb-studio/issues/543#issuecomment-703300587 fixes the issue reliably, but it is important to fix the compiler too! that flaw seems to happen because some variables become register-allocated (that is where speed gain comes) and saving/juggling is faulty or missing in some particular case.

Hi @untoxa sure thing, I'll see if I can make a minimal reproducible project from that issue.
I've managed to get color working on Mac by the way, I had to compile from the latest develop branch on gbdk-2020 rather than using the sources from the zip on the v4 release. Not sure what changed but I was all set to figure it and submit a PR when trying the dev branch just fixed it!
Using your latest changes I've actually now got a Mac build of GB Studio that from what I can tell is fully working, I'll push my changes to your PR later.

@chrismaltby yes, LCC changed: https://github.com/Zal0/gbdk-2020/commit/8b3e5492980fdc818333999d80938224cb7db590 processing of that -Wm-yC and similar. also i suggest you to use vs.code & emulicious to debug the core - that helps much (update the emulator). if you need to pass some long parameters into sdcc frontend, also with spaces, then use that kind of construction: -Wf'--max-allocs-per-node 50000' (this one makes register allocation better, for example)

does killing of enemies in platformer (one with space ship) works? i'm playing with your game from here https://github.com/chrismaltby/gb-studio/issues/543#issuecomment-703116622 , not from the git repo.
ps: also, what do you say about performance?

Hi @untoxa I've cut down as much as I could while still keeping the weird SDCC behaviour in this branch of my test project
https://github.com/chrismaltby/gbdk2020v4-testgame/tree/minimalbug
When compiling that project I get a ROM that just displays the following:

but if I change only the bank variable to be static then I get a working ROM
void LoadScene(UINT16 index) {
  static UBYTE bank;
  // UBYTE bank;
  UBYTE* data_ptr;

that should just display the following

Here's the two ROMs with the only difference being that one variable being static or not.
working_rom.gb.zip
broken_rom.gb.zip
Not had chance to try out the performance on the working version just yet as I've been putting this together but I'll try to have a look at that tomorrow

@chrismaltby unfortunately, roms are not of much help here, we need to report it like this: https://sourceforge.net/p/sdcc/bugs/3097/ look into test4.c attached to that issue.

@untoxa Ah I see, needs to be a much smaller reproduction of the issue, I'll have another go at it.
By the way I've pushed up the changes need to GB Studio to work with GBDK 2020v4 into this branch forked from your PR https://github.com/chrismaltby/gb-studio/tree/features/gbdk2020v4 I've only included the Mac binaries so far.
In testing though I am getting issues you mentioned when shooting the enemies on the space level where the graphics become corrupted and the game crashes.

@chrismaltby definitely. but i'm not familliar with the engine enough yet to figure out where that code lies. :) definitely it's not shmup.c, it's something inside the core. please, answer me on discord.

YAY! https://github.com/chrismaltby/gb-studio/pull/588/commits/15912592c8465a6216e2221439ce515095ae677b

unfortunately, that issue with crashing while hit the enemies in shmup stage still exists, even after updating. i thought i fixed it, but i was wrong, it was workarounded with --nolospre switch. but it is necessary to find the source of the problem, not just blindly workaround it. if anyone has a clue what function is faulty - i'll be much obliged.

ah!!! i did that. :)
first abouth the metod, that might be useful for such hopeless cases. i was trying to find the problem for some days, but at last i realized, that i had enough of fishing with the fishing rod and now it's time for dynamite.
inputs: we have a working engine if compiled with --nolospre and engine that breaks when you hit enemies if compiled without that switch.
solution:

i compiled a working rom, and took out all asm-files from the object directory, that were the result of compilation.
i compiled non-working rom and did the same
i took asm files from the sources and added them too, so i got an asm-only gb-studio engine.
then i made a makefile that builds the engine from asm sources only - that's easy
i compared the files in "bad" and "ok" folders - some of them has different size. not that many.
then i re-compiled the engine from "bad" state replacing files one by one from "ok" and checked the result.
bingo! the problem was in Projectiles_b.asm

here is the project: 2.zip
so, the problem was, as i expected, with some corner case of the optimizer, just as it was above: https://github.com/chrismaltby/gb-studio/issues/543#issuecomment-704497615 but, unfortunately, another one.
anyway, the workaround is very simple:
  static UBYTE hit;
  UBYTE i, k, j, frame, flip, fo;
//  UBYTE i, k, j, hit, frame, flip, fo;

and the source of the problem is that hit gets optimized, but something goes wrong and when execution gets to the ScriptStartBg(&actors[hit].hit_1_ptr, hit); garbage is executed, so we observe marvelous pictures as above.
##i still need some help with making a minimal example of invalid code generation for reporting it to SDCC devs! @chrismaltby please, help me.
thank you for reading this.

You legend! Thanks @untoxa!! I'll see if I can get some time over the next few days to get you another minimal reproduction of this issue. Looks like we're almost there! 😁

maybe Philipp @spth can also have a look onto this: https://github.com/chrismaltby/gb-studio/issues/543#issuecomment-707750833

Hi @untoxa After a LOT of struggling I think I've got about a minimal example of the second issue on this branch:
https://github.com/chrismaltby/gbdk2020v4-testgame/tree/minimalbug-projectiles
#include <gb/gb.h>
#include <stdio.h>

#define MAX_PROJECTILES 5
#define MAX_ACTIVE_ACTORS 11
#define MAX_ACTORS 31

typedef struct _BankPtr
{
  unsigned char bank;
  unsigned int offset;
} BankPtr;

typedef struct _PROJECTILE {
  UBYTE col_group;
} Projectile;

typedef struct {
  BankPtr hit_1_ptr;
  BankPtr hit_2_ptr;
} Actor;

Projectile projectiles[MAX_PROJECTILES];
UBYTE actors_active[MAX_ACTIVE_ACTORS];
Actor actors[MAX_ACTORS];

void TestFn2(BankPtr* events_ptr) {
  if (events_ptr->bank == 5 && events_ptr->offset == 0xFC00) {
    printf("OK\n");
  }
  else
  {
    printf("FAIL\n");
  }
}

void TestFn() {
  UBYTE hit;
  UBYTE i;

  for (i = 0; i != 1; i++) {
      hit = actors_active[0];
      if (hit != 0xFF) {
        if (projectiles[i].col_group == 2) {
            TestFn2(&actors[hit].hit_1_ptr);
        } else if (projectiles[i].col_group == 4) {
            TestFn2(&actors[hit].hit_2_ptr);
        }
      }
  }
}

int main()
{
  LCDC_REG = 0x67;
  BGP_REG = OBP0_REG = OBP1_REG = 0xE4U;
  WX_REG = MAXWNDPOSX;
  WY_REG = MAXWNDPOSY;

  projectiles[0].col_group = 2;
  actors_active[0] = 0;
  actors[0].hit_1_ptr.bank = 5;
  actors[0].hit_1_ptr.offset = 0xFC00;

  TestFn();

  return 0;
}

Same as in the GB Studio engine switching UBYTE hit; for static UBYTE hit; fixes the issue, interestingly removing the loop (that in this example doesn't even really do anything but is needed in the engine) also fixes the issue, maybe hit and i are using the same registers?

@chrismaltby not exactly. here is my view on the problem: https://sourceforge.net/p/sdcc/bugs/3135/

@chrismaltby https://sourceforge.net/p/sdcc/bugs/3135/#3d9e/eb33

@chrismaltby SDCC r11925 works solves the issue: http://sourceforge.net/projects/sdcc/files/snapshot_builds/x86_64-w64-mingw32/sdcc-snapshot-x86_64-w64-mingw32-20201027-11925.zip/download

@chrismaltby, here is the latest GBDK-2020 v4.0.1rc:  gbdk-2020-v401.zip with the latest SDCC r11925; GB Studio core compiles nicely and i don't see any problems with it.
to make that all a bit more difficult :) i rewrote initialization code of runtime library, so now it does not fill all WRAM with 0 on start. that allows you to catch errors in code when you use non-initialized variables. you can do that with BGB, by setting it to raise an exception, when non-initialized variable is read:

enabling that, raised loads of exceptions when i tried to run GB Studio game. i fixed them all, please have a look: 4.zip
please note, that, say, ScriptContext script_ctxs[MAX_SCRIPT_CONTEXTS] = {{0}}; does not initialize all array of structs with zeros, look at the generated code:
	.globl	C$ScriptRunner.c$23$1_0$156
;src/core/ScriptRunner.c:23: ScriptContext script_ctxs[MAX_SCRIPT_CONTEXTS] = {{0}};
	ld	hl, #_script_ctxs
	xor	a, a
	ld	(hl+), a
	ld	(hl), a

everything is working, ready to merge, gbdk-2020 v4.0.1 is released: https://github.com/chrismaltby/gb-studio/pull/588


Generate the configuration.md file automatically on changes to the configuration code.

I think it would be nicest if we could have the patch automatically pushed to main after a commit with changes to the config is merged into main.
If we're afraid of automatically pushing to main, we could maybe push into a PR's branch instead, once it's created and has changes to the config crate.
The weakest option would be to just regenerate that file and require it not to be different, like we do with the json schema, forcing contributors to regenerate that file manually when needed.
Depending on what strategy we choose, this might be combined with https://github.com/metalbear-co/mirrord.dev/issues/207.

pushing to main doesnt sound good, id rather think of doing a check on each pr if mirrord/config/ has been changed and the job should fail. this weak check imo is more flexible... and perhaps having a pre commit hook makes sense here?


Favicon

Addressing #58
Added a 32x32 PNG image which should handle most cases.

splendid, thanks! I will look at this later, but thanks!

@all-contributors please add @olimart for maintenance


MacOS: "Cancel and Unlink" closes Maestral window but does not exit Maestral

Describe the bug
Selecting "Close and Unlink" from the folder onboarding screen will close Maestral's active window, but not terminate the application, leaving it invisible.
To Reproduce
Open Maestral. Link Dropbox account. Click "Close and Unlink".
Expected behaviour
Application closes fully.
System:

Maestral version: 1.4.4
OS: [e.g. Ubuntu]: MacOS Big Sur 11.3.1

Again, a manifestation of the same issue as in #403.


switch to propulsor configuration, remove unused add-ons

I switched to ConfigUtils.loadStandardConfigProperties(), which includes environment variables
under the 'env.' prefix. This is new in Commonjava/propulsor#37.
Switched over to using propulsor configuration so we can continue to consolidate onto that as
a platform for our services (for as long as that makes sense). This is the bulk of the changes
here, but I'll mark the things that aren't related when I create the PR.
Changed logging of 404's in ResponseUtils to use logger.debug, but other errors are still
logged as error.
Removed autoprox, setback, and relate, since these are all associated with use cases that
no longer exist for Indy and they were just cluttering up the codebase / complicating the
logic.

retest this please

retest this please


Remove references to LOCAL_DOCKER_REGISTRY

This semantic is not needed as built images can be re-tagged from a built tag. For example:
$ docker build . -t indeedoss/rabbitmq-operator:<version>
$ docker tag indeedoss/rabbitmq-operator:<version> <tld>/rabbitmq-operator:<version>

Removing this semantic should help make building this repo easier.

I'd also include the registry.local.tld/ in this change.


DSD-1677: Adding interaction test examples

Fixes JIRA ticket DSD-1677
This PR does the following:

Adds @storybook/addon-interactions to Storybook for interaction tests in the "With Controls" Story.
Interaction tests should be added in the WithControls example in [component].stories.tsx files only.
The syntax resembles React Testing Library.
More tickets will be created from this.

How has this been tested?

Review https://nypl-design-system-git-dsd-1677-storybook-tests-nypl.vercel.app/?path=/story/components-form-elements-textinput--with-controls
Click on the "rerun" icon to run the tests.

Accessibility concerns or updates

N/A

Checklist:

[ ] I have updated the Storybook documentation accordingly.
[ ] I have added relevant accessibility documentation for this pull request.
[ ] All new and existing tests passed.

Front End Review:

[ ] Review the Vercel preview deployment once it is ready.

This is great.  Some thoughts...

Can you add a link to the PR description pointing to the official Storybook documentation about interaction tests?
I feel we should define a basic battery of tests that should be added to all components.
The list of interactions essentially shows straight code.  Most non-coders will not understand what is shown.  Does that matter?

Can you add a link to the PR description pointing to the official Storybook documentation about interaction tests?
Done
I feel we should define a basic battery of tests that should be added to all components.
Yes, we can try this.
The list of interactions essentially shows straight code. Most non-coders will not understand what is shown. Does that matter?
It's generally meant for devs to see the tests. I was hoping it would display more English-like text but haven't seen many examples with plain English.


Abstract base classes high max_length value

I noticed the CharFields in our abstract base classes provide unusually high max_length values of 4096 when I seemed to remember 255 to be the default. Example:
https://github.com/acdh-oeaw/apis-core-rdf/blob/10c524ef36da57924b738cc92cc230cf5a28a3ec/apis_core/apis_entities/abc.py#L13-L16
According to the Django docs on CharField and Character fields, the value is fine for Postgres, which we use, but could become a problem for MySQL users. (The MySQL docs suggest VARCHAR values exceeding the limit are truncated.)
AFAIK we don't caution against or rule out using MySQL, so if we want our base classes to be more DB agnostic/inclusive of MySQL, we should probably reduce the max_lengths to 255. Alternatively, a general recommendation for Postgres over other databases + stating that we specifically develop with Postgres in mind (for which we could cite the base classes as example) would be helpful.

which default was 255?

Could you please specify which part of the documentation you refer to in this sentence:

According to the Django docs on CharField and Character fields, the value is fine for Postgres, which we use, but could become a problem for MySQL users.

especially in regards to which parts lead you to the conclusion that this "could become a problem for MySQL users"?

which default was 255?

Sorry, imprecise wording. Substitute "typically used".
255 is the number I had memorised as (max) safe value for max_length for CharField. The use of 4096 in our classes confused me so I checked if I'd assumed incorrectly for ages or if 255 was outdated info. A search for the number in our repo just now turns up results which match the image I had in my head.
Re: could become a problem:
Frst sentence from the "Character fields" section I linked earlier:

Any fields that are stored with VARCHAR column types may have their max_length restricted to 255 characters if you are using unique=True for the field.

We are not setting the fields unique=True in Core, but I wouldn't assume that people who inherit from our classes will leave everything as-is. E.g. we want to modify the fields to add verbose_names for TBF; someone else may need to add a unique constraint (assuming this can be done just as easily?) but not realise they might then also have to modify the max_length value. Setting this to 255 would be a precaution/us acting in a more user-friendly way.

E.g. we want to https://github.com/acdh-oeaw/apis-instance-tbf/issues/69 for TBF; someone else may need to add a unique constraint

Interesting... how do you change a field you inherited?

Interesting... how do you change a field you inherited?

I have done this by redefining (overriding?) the inherited fields in my models.
Fields inherited from abstract base classes can be overridden with another field or value, or be removed with None.

Yeah, but isn't this then a new field? It doesn't inherit the attributes of the field you redefined/overwrote (i.e. the max_length value)?

Hmm, I thought a model's __init__ method could be overridden to change field attributes. Though I've admittedly only tried this with verbose_names so far, for which I don't care about what happens on the DB level but only the UI side.

If this is not a thing that should be done and therefore not something we want to support, then the 2nd part of my initial suggestion is the only revelant thing left to discuss.

@koeaw I don't get it, in the MySQL documentation you linked (https://dev.mysql.com/doc/refman/8.4/en/char.html) I can't find any information about truncation after 255 characters. I see a confusing limit of "maximum row size" in MySQL but that's not our problem, MySQL users should know that). What am I missing?

My statements were primarily based on the linked sections in the Django docs. It's possible they are wrong/ambiguous/not reflective of the MySQL docs (or that I misinterpreted them).

Maybe you could give us an example code that leads to the problems you see. I tried using the __init__ method, but that did not have any effect on the datamodel

Maybe you could give us an example code that leads to the problems you see.

I'm not using MySQL (I don't think any of us are?), would def. have included more info if I was experiencing problems because of this myself!
I opened the issue based on what I read in the docs; this earlier comment has the chain of events:

255 is the number I had memorised as (max) safe value for max_length for CharField. The use of 4096 in our classes confused me so I checked if I'd assumed incorrectly for ages or if 255 was outdated info. A search for the number in our repo just now turns up results which match the image I had in my head.

I tried using the __init__ method, but that did not have any effect on the datamodel

Ah, interesting. So this more of a "hack", a way to manipulate the UI side of things only, hmm. Thank you.

Oke, thanks! Given that we don't even know if there is a problem, I'm closing this


[fabioh5] Support UB matrix

Here is the first implementation.

A sample group is create only if expected and contains few informations from UB_pos/mne and sample_pos/mne.
It only reach data from the first header (i do not check hole the frames for efficiency)
Nexus request float for this datasets. numpy.array used by it-self float64. I think it's fine.
I add interpretation=scalar for this 3 datasets. Like this we display it as raw data.
I use medipix.edf to create a unittest. I think it is a very old data, then it would be nice to test the result with an up-to-date file.

Please, change the name orientation_matrix to ub.
People from NeXus have realized they made a mistake and the orientation_matrix is just U and not the product UB.
Sorry for the inconvenience.

There is not problem, it is a small fix.

BTW what about ub_matrix instead of just ub?

I am fine with ub_matrix too. I have given the NeXus people until our release to decide.

This PR can way before merging.

It looks OK to me. If the PR does not interfere with any other one, we can validate it when NeXus people have taken a decision on the name (ub or ub_matrix).

This needs to be merged in order to simplify testing that spech5 and fabioh5 provide the same information and in the same form. At this moment it does not seem to be the case.


.gitignore getting written every time we run a build

Hi,
Each time we run a build, the .gitignore file is checked to see if it has every entry necessary.
However, since it is using os.EOL to split each line and we are using mixed OS (Win/Mac) and our file is in LF mode, for some it works as expected and for others it keeps getting written.
Any better solution or any approach we should take to mitigate this problem?
https://github.com/rdkcentral/Lightning-CLI/blob/727ffafb568763b710f68d2fe66760bff95cc699/src/helpers/build.js#L255-L277

Hi @ricardo-a-alves-alb, I understand that must be annoying!
The difficulty is indeed the differences is EOL per platform. Let me see if we can figure something out to for this :)

hi @ricardo-a-alves-alb we have a fix for this issue (https://github.com/rdkcentral/Lightning-CLI/pull/212). We've tested on Windows, Mac OSx and Linux (Ubuntu) platforms and haven't encountered any problems.
Would you mind checking if this fix solves the issue for your team as well? Then we'll make sure to include this in the upcoming release (which will be January '23)


fix: correct URI of remote odrl-json-ld context

The EDC currently returns HTTP 400 when given the non-expanded example from this repository. This is because it includes https://www.w3.org/ns/odrl.jsonld as the URI for the ODRL-context. However, it is specfied with http and not https [1]. This is hardcoded in the EDC as it doesn't resolve the context remotely but locally [2]. For other json-ld parsers, this may not matter as the http URL is forwarded to https.
This bug may have proliferated to other kits and internal documentation. In this repo, the example is the only instance I could find.
[1] https://www.w3.org/TR/odrl-vocab/#json-ld
[2] https://github.com/eclipse-edc/Connector/blob/9efa8adf191c471425ac7d1920c1d970ba277aeb/extensions/common/json-ld/src/main/java/org/eclipse/edc/jsonld/JsonLdExtension.java#L94

This is because it includes https://www.w3.org/ns/odrl.jsonld as the URI for the ODRL-context. However, it is specfied with http and not https [1].

The spec does NOT require a specific URL here. The part in the spec you are referring to also says:

"or any other @context (file) including the content of http://www.w3.org/ns/odrl.jsonld "

For other json-ld parsers, this may not matter as the http URL is forwarded to https.

It sounds a bit like it was wrong here in this repo. But the location of the remote context is clearly hosted under the https and not http.
The reason to still accept it here is because of an EDC workaround and most people are using this implementation right now.

Please update the commit message accordingly and add your organization / name to
https://github.com/catenax-eV/cx-odrl-profile/blob/main/NOTICE
in a separate commit (same PR)

I believe this PR can be closed - if correct, please go ahead with thanks!

I'd rather like to see this merged instead of closed.
If there's feedback, please leave another review.

I've created a PR to @arnoweiss  repository with the proposed updated commit message.

@arnoweiss you need to rebase and force push, not merge. thanks,
Matthias

We took this offline. Change on main now.


Includes?

I am not looking to do a full podcast, but I may want to attach an audio file, photograph, PDF, or other to a post at some point. Does this plugin support includes?
Thanks,
Matt

No, this plugin does not support includes.
Aside from podcasts (which would really benefit from more features than what this plugin currently supports), what is the use-case for includes? What feed readers support them? What feeds publish them? Why not just use a link in the post content?


Prefix domain implementation

WIP

Closed this PR since it should merge in my fork and not in the main one.
PR can be followed here:
https://github.com/BlueRyse/lisa/pull/1


DMXChannel without Offset breaking parser

Hi, I'm setting up your library to parse GDTF profiles, however it failed with the following error:

File "C:\Users\Aboud\AppData\Roaming\Blender Foundation\Blender\2.90\scripts\addons\dmx\pygdtf_init_.py", line 592, in _read_xml
self.offset = [int(i) for i in xml_node.attrib.get('Offset')]
TypeError: 'NoneType' object is not iterable

I was testing this FixtureType. I found that the first DMXChannel of the file has no Offset parameter, since it's zero.
I'm not sure what the standard says about it, so I didn't make a pull request. However, it's an easy fix.
Please let me know what you think it's the best strategy here: reporting the file problem to GDTF-Share, including the fix or both.
offset = xml_node.attrib.get('Offset')
if (offset != None):
    self.offset = [int(i) for i in offset.split(',')]
else:
    self.offset = [0]

Also, thank you for the work on this library! It's gonna make life so much easier!

Hi, sorry if I'm a bit late with this one - I have just pushed a change along these lines
According to the standard, the default value for Offset is "None", so I have added this as the fallback value if a value isn't provided. I've tested this with the fixture you've linked. Note I have used the string "None" rather than builtin None to keep in line with the documentation they provide
Cheers

Hi, sorry if I'm a bit late with this one - I have just pushed a change along these lines
According to the standard, the default value for Offset is "None", so I have added this as the fallback value if a value isn't provided. I've tested this with the fixture you've linked. Note I have used the string "None" rather than builtin None to keep in line with the documentation they provide
Cheers


Hypergraph creation

Hi authors, awesome work and impressive result, which is current sota and beat the previous approach by a large margin. Could you give the script to create the hypergraph (which is only available now via downloading) from the local and global descriptors? And what exactly are Match_region and Match_loc in your graph.zip. As I read your paper and code, Match_region seems to be the region to calculate the intra edge of the graph, and Match_loc is the matching local between two images. If I'm right, is the shape 9x2 in the Match_loc  the index of matching local descriptors between two images. Thank you.

Hi, thank you for your interest in our work! Actually, the hypergraph is computed in real-time after the query is given and the code is already included in hypergraph_propagation.py. Graph file contains the ordinary knn graph (Neighbors) and the pre-computed matching information. You can understand Match_region and Match_Loc (matched points' locations) as the precomputed homograph between two images using RANSAC. We do not save the homograph matrix itself but store the matched points' regions and locations. More specifically, for the key 'imlist_1_imlist_2', Match_region and Match_loc are the region and locations of the matched points in imlist_2. There is also a key 'imlist_2_imlist_1' to store the region and location of the matched points in image1.  They are calculated offline using RANSAC with the same setting with Cao's DELG paper . We will give the off-line RANSAC code to calculate the Match_region and Match_Loc soon. I hope this answer will help you.

Thank you for your answer. Looking for your RANSAC code to calculate the Match_region and Match_loc.

Hi, we just uploaded the offline RANSAC code. Please check it and contact us if you get any questions. We also welcome more comments about our paper.


CI workflow jobs

Test with Vite (React, React SWC)
Test with Next
Test for SSR

Closing, see #37


sysl mod init

Fixes #537
Changes proposed in this pull request:

Added sysl mod init command
Behavior: leverages go mod init into the CWD. Assumes that the current folder is not already a go module or sysl module.

@anz-bank/sysl-developers

I've created another PR for this PR: https://github.com/nydrani/sysl/pull/1


Add trail

Oh sorry lol, I haven't been watching this repo. (: /
Yeah I'll be busy this week (dum anoiying examz lol, they've been delayed for like 3 times now) too.
Well anyways, is this for the gravity game?

lol I am also busy with exams, cant do much for now... This PR just adds a trail for now, nothing much

Finally! I'm done with my exams!
I'll be working on the trail now.

Nice to see you back :)

Sorry for the delay, I forgot a bit of Python after that break lol. :P
Here's the code (I only needed to edit planet.py):
import pygame as pg
from constants import *

class Planet:
    def __init__(self, mass, radius, pos=(WIDTH / 2, HEIGHT / 2), vel=(0, 0)):
        self.mass = mass
        self.radius = radius
        self.pos = pg.Vector2(pos)
        self.vel = pg.Vector2(vel)
        self.prevPositions = []

    def calcUpdate(self, planets):
        self.field = pg.Vector2(0, 0)
        for planet in planets:
            r = planet.pos - self.pos
            mag = r.magnitude()
            if mag != 0:
                self.field += G * planet.mass * r * mag ** -3

    def applyUpdate(self, dt):
        self.prevPositions.append((self.pos.x, self.pos.y))
        if len(self.prevPositions) > 100:
            self.prevPositions.pop(0)

        self.vel += self.field * dt
        self.pos += self.vel * dt

    def render(self, surf):
        for i in range(len(self.prevPositions)):
            pg.draw.circle(surf, WHITE, self.prevPositions[len(self.prevPositions) - i - 1], self.radius - i/self.radius)
        pg.draw.circle(surf, WHITE, self.pos, self.radius)


Channels not fetched

Any idea why the plug-in can’t fetch the channels anymore? Should have fetched over 300 channels for dvbt
09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching all channels for tv:dvbt
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching 1. list of channels
[09/11/2020, 20:41:52] [BraviaTVOS] TV does not support fetching list for tv:dvbt, skip..
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching all channels for tv:dvbc
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching 1. list of channels
[09/11/2020, 20:41:52] [BraviaTVOS] TV does not support fetching list for tv:dvbc, skip..
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching all channels for tv:dvbs
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching 1. list of channels
[09/11/2020, 20:41:52] [BraviaTVOS] TV does not support fetching list for tv:dvbs, skip..
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching all channels for tv:analog
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching 1. list of channels
[09/11/2020, 20:41:52] [BraviaTVOS] TV does not support fetching list for tv:analog, skip..
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Fetching all commands
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: All inputs fetched! Stored to /var/lib/homebridge/bravia/Fernseher-inputs.json
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: "refreshInputs" setted to false!
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Adding new input source
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: {"name":"HDMI 1","identifier":23,"type":"hdmi1","inputType":3}
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Adding new input source
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: {"name":"HDMI 2/MHL","identifier":24,"type":"hdmi2/mhl","inputType":3}
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Adding new input source
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: {"name":"HDMI 3","identifier":25,"type":"hdmi3","inputType":3}
[09/11/2020, 20:41:52] [BraviaTVOS] Fernseher: Adding new input source

@Sammy990
Did it work with v4.0.2 ?

It did with 3.X. V4 is a fresh install from today

@Sammy990
Thx, i will check this

Thanks mate

@Sammy990
v4.0.4 is online. Should fix the issue


printMessage API documentation

Then printMessage requires two arguments, message and package json, but document is missing pkg argument.
Another is printMessage returns the string, it sounds more like formatMessage instead of actually printing. Maybe considering a rename or adding a new API formatMessage and then deprecate printMessage in next big release

You're right, I forgot to document about the second param. The rename makes sense too, it was initially exposed from an internal util. We could probably do the first part for now to fix this though.

Thanks for updating the docs, is there any plan to rename the API in the short future as well? Do you want me to file another issue to track that?

Yup, would appreciate if you can open a new issue for it so we can track it!


Set shortcuts to any key combo

Instead of requiring Win + Shift, users should be able to set any combination of keys as a shortcut for different modes.

https://github.com/microsoft/PowerToys/tree/main/src/settings-ui/Settings.UI/Controls/ShortcutControl


Compatibility: not compatible with Samsung Galaxy S9+?

Android 10 and latest updates installed, also for Google Play Services. App is not offered in the play store for this device.

Hello @jmoberreuter,
I just came across your issue. Does this issue still remain?
Generally, the Corona-Warn app is available in multiple countries of the PlayStore. For an overview, please check out this FAQ article: https://www.coronawarn.app/en/faq/#international
To be able to use the application, your PlayStore should be set to either of these countries.
Also, it is important that you smartphone runs at least on Android 6, as you can see here: https://www.coronawarn.app/en/faq/#minimum_requirements
Please make sure your Google Play Services are up to date: https://www.coronawarn.app/en/faq/#googleplay, https://www.coronawarn.app/en/faq/#cause_3

Thanks! Changing the country of the store has solved the problem. I did find the error message "not compatible with your device" confusing, though. Thanks for your help and concern.

Why do I have to change the country to install the app? Shouldn't people visiting Germany be able to use this?
I agree with @jmoberreuter that the error message is very unintuitive.


Page navigation error with sidepanel

If side panel is enabled page navigation give me this error
mycode:
Index.html contain :
<div class="side-panel side-panel-right" data-expose-aside="right" data-disable="left" data-page="dashboard,profile,sandogh,charge" id="sideMenu">     </div>
Error at line
settings.element.removeChild(backdrop);
Uncaught DOMException: Failed to execute 'removeChild' on 'Node': The node to be removed is not a child of this node.
at HTMLDivElement.e

Hi @cinohee,
Thank you for opening this issue.
I can't reproduce your bug.. Can you please give me more information?
Such as the OS, browser and what you did to get this error.

Hi @Qathom
This is my Index file
rename to html file
I load data by Ractive to menu.
Is there any problem in my file?
index.txt

Hello
I have the same problem.  The Exception is raised when navigating to a new page.
In the function removeBackdrop the settings.element points to the new page but
the backdrop is not a child of that page.
In  removeBackdrop I changed the line
settings.element.removeChild(backdrop);
to
backdrop.parentElement.removeChild(backdrop);
which solved the problem for me.

Hi all,
This issue should be fixed with v1.5.1.
Hence I'm closing this ticket. Feel free to reopen this one if v1.5.1 doesn't fix your error.


fix: do not bundle sub exports of windicss

currently only Processor is imported from node_modules but all sub exports are bundled.
this adds all windicss exports to tsup's internal rollup config
related: https://github.com/windicss/windicss/issues/199

ah you were faster with a solution ;)

This actually looks a bug of tsup :P I should find something to fix it


test_gpu fails on v100 but passes on k80

On v100, test_cpu passed, but test_gpu failed.
v100 has the same configuration on gcc & cuda & cudnn with k80, but k80's test_gpu passed.
Error message:
Running GPU tests
terminate called after throwing an instance of 'std::runtime_error'
  what():  Error: compute_ctc_loss in small_test, stat = execution failed
Aborted (core dumped)

Tried to modify CMakeLists.txt, but it didn't work.
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_GLIBCXX_USE_CXX11_ABI=0")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORCE_INLINES")
#set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_60,code=sm_60")

set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_53,code=sm_53")
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_60,code=sm_60")
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_61,code=sm_61")
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_62,code=sm_62")

I have seen some similar issues on GTX1080 and GTX1060, but there is no solution.
Is it something related to the GPU version？
Help me plz...
cuda: 9.0.176
cudnn: 7.0
gcc & g++: 5.4.0

BTW, I have tried cuda 8.0 7.5 and gcc 4.8 on v100, the same error message.

volta is compute 70, try that.

volta is compute 70, try that.
it works!
Thanks!


Create notes_tie.lua

New small script. You can tell by now that I'm moving, step by step, to eliminate dependency on JW plugins.

This script will fail to correctly create a tie if

There is a v2 launch between a tied-from and tied-to note.
The same note occurs in two different layers in certain configurations.
There is a key change between the tied-from and tied-to notes.

Also, it is dependent on the order that eachentry serves up entries. This mainly could get you in trouble if there are multiple layers, because the measure position resets to zero. You could end up trying to tie to a note that is before the tied-from note.
At the very least, I would like to see V1/V2 support added and only tie together two notes from the same layer. (V1/V2 support will complicate this script a good bit, but it needs to be there.)
I would consider changing this to use FCNoteEntryLayer instead of eachentrysaved. That will allow you to run one layer at a time easily, and by using the FCNoteEntry.Next method, you can guarantee the entry sequence.
Support for key changes in the middle of a tie is considerably more complicated and perhaps not important enough to attempt.
There is a private function tie_span in the tie.lua library. It wrestles with these issues. Unfortunately you can't use it as-is because it is finding the span of an already-existing tie so it is counting on the Tie and TieBackwards properties already being set.

No need to remove it. Just commit more changes until it's ready to merge.

Instead of FCNoteEntryLayer you could pass the 1-based layer number as the second parameter to eachentry or eachentrysaved. That also gives you control over layers. (This is an apparently little known feature of these built in functions. I really need to document them better than perhaps Jari did. There are some that I've never seen used that are really useful.)
However, the two built-in functions serve up entries one cell at a time, whereas FCNoteEntryLayer gets them all at once. This allows the Next method to cross barlines the way you need it to for ties.
Also, I'm not sure if you saw my comment about grace notes. Grace notes do not tie to other grace notes. They tie to the first non-grace-note entry.

I used layer filtering in eachentry on my layer_mute and layer_hide scripts. (Pull request #195 - pending). Will examine the options!

I've been racking my brain about this, and need you to confirm that I was trying to achieve the impossible.
I think I have most of this in place now. But I have a V2 entry that extends to the end of a measure, which I was trying to tie to a matching V2 entry at the beginning of the next measure. But if you set the TIE bit on the first note, it seems as though Finale wants to tie it to the "subsequent" V1 note, occurring earlier in the same measure. Is that true? And so I should just stop trying to get V2 to tie across barlines?
(And yes - FCNoteEntryLayer handles gaps between measures more usefully than eachentry() )

If the edu length of the V2 sequence is not exactly equal to the V1 entry that launched it, Finale does weird things. It could be that it might try to tie a V2 note to a V1 entry before it. But I think you should focus only on the case where the length of the V2 sequence matches (or is no more than) the length of the the V1 entry that launched it. In that case, a tie from the last V2 entry can be tied to either the following V1 entry or to a V2 entry launched from that next V1 entry. And the next V1 entry can be in the next bar.
I strongly encourage you to use the logic in tied_to andequal_note from the tie.lua library in this repo. The only change you need to make is to compare the actual Displacement and RaiseLower rather than the staff position and current state of the tie flags. That logic is the result of tens of hours of experimentation with Finale back when I wrote the Tie Mover plugins (some 20-odd years ago).
Then the pseudocode inside your loop can be like:
next_entry = next_entry:Next()
if not next_entry then
   break
end
for each note in (next_entry) do
   local tied_to_note = tied_to(note)
   if tied_to_note then
      note.Tie = true
      tied_to_note.TieBackwards = true
   end
end

I'm tempted to rewrite the code in tie.lua so that it can be reused by both, but there are some challenges to trying to make the functions generally applicable to all callers.
That pseudocode should get it working in all cases except when a key change happens in the middle of a tie. That's a bridge you can cross separately if you decide to. (Perhaps using the note string function instead of disp/raiselower.)

I thought I'd show you where I'm up to on the whole script, which has become massively simpler when no longer trying to tie V2 across barlines. In fact I've specifically exclude chasing entries "leaving" V2 from tying to anything, as that seems to lead inexorably towards tying (backwards) to previous notes.
I think I'm using the same logic as tied_to(note), though it's distributed differently. I'm using entry:FindPitch(note) as a convenience - will that suffice? I haven't been using note.TieBackwards since I thought that was handled by eachentrysaved(), but is it advisable under FCNoteEntryLayer? Easy enough to implement in my tie_between_entries().
I've attached the sample file I've been using to test V1/V2 behaviour. Fascinating that copying whole measures (with option click) corrupts the distribution between V1 & V2, favouring shorter stretches of V2 interrupted by intervening V1 entries. Easy to see why V2 was so quickly deprecated.
Which leaves key changes across ties, and I figure anyone who wants to do that shouldn't expect a tie automation script to compensate for their cavalier behaviour.
tie_test_v2.musx.zip

FindPitch is an excellent find. It will kind of work with key signatures as well. (It may try to tie F# to Gb across a key sig change, but we can live with that.)
I did not mean to imply that V2 doesn't tie across barlines. V2 absolutely can tie across a barline if it is the last entry in the bar.
You should also set TieBackwards on the tied-to notes.
I find this code overly complicated and suggest you do the way I suggested. My code is the result ofm extensive experimentation with ties. Also, I didn't see anywhere that you are skipping grace notes as candidates for tied-to. Perhaps I missed it.

Tried simplifying back to the precepts of tie.tied_to() and tie.equal_note() but without tracking the transitions between V1 & V2 this inevitably failed to tie two V1 notes where the first is a Voice2Launch, and always tied an "ending" V2 entry to the "next" V1 entry that follows it in the Edit Frame note sequence but precedes it in the measure.
Creating ties has different needs to tracking existing ties (as in tied_to() and equal_note()) but I can't see another way to do this without the complexity of tracking V1/2 transitions, one way or another. If you run this script over the Finale file I attached before it works at everything except tying V2 over barlines, and I still can't get that to happen in Finale using any available tool whatsoever!

Your sample file is too perverse. You will not be able to tie the V2s across the barline in that file even using Finale's UI. (I tried it with Speedy.) Don't worry about cases where the V2 sequence is longer than the V1 entry that launched it. Finale is not designed to do that.
That said, you did uncover a bug in my tied_to function that I will have to correct, even in the Tie Mover plugin. That is that a V1 can never tie to a V2 note in an entry that it launches. Here is a version of your code I wrote that handles (normal) V2 across barline no problem. I also fixed it to skip entries not contained in the selected region (on both ends).

local note_entry = require('library.note_entry')

-- default to "tie" notes for normal operation
untie_notes = untie_notes or false

local function tied_to(note)
    if not note then
        return nil
    end
    local next_entry = note.Entry
    if next_entry then
        if next_entry.Voice2Launch then
            next_entry = note_entry.get_next_same_v(next_entry)
        else
            next_entry = next_entry:Next()
        end
        if next_entry and not next_entry:IsRest() and not next_entry.GraceNote then
            local tied_to_note = next_entry:FindPitch(note)
            if tied_to_note then
                return tied_to_note
            end
            if next_entry.Voice2Launch then
                local next_v2_entry = next_entry:Next()
                tied_to_note = next_v2_entry:FindPitch(note)
                if tied_to_note then
                    return tied_to_note
                end
            end
        end
    end
    return nil
end

function tie_notes_in_selection()
    local region = finenv.Region()

    for staff_number = region.StartStaff, region.EndStaff do
        for layer_number = 0, 3 do  -- run through layers [0-based]
            local saved_v1_entry = nil -- v1 entry held while v2 running
            local v2_is_active = false
            local entry_layer = finale.FCNoteEntryLayer(layer_number, staff_number, region.StartMeasure, region.EndMeasure)
            entry_layer:Load()
            for entry in each(entry_layer) do
                if not entry:IsRest() and region:IsEntryPosWithin(entry) then
                    for note in each(entry) do
                        local tied_to_note = tied_to(note)
                        if tied_to_note and region:IsEntryPosWithin(tied_to_note.Entry) then
                            note.Tie = true
                            tied_to_note.TieBackwards = true
                        end
                    end
                end
            end
            entry_layer:Save()
        end
    end
end

That works nicely, and very neatly, and V2 ties over barlines as you promised if the bar-end-V2 is a fresh V2 entry at that point. note_entry.get_next_same_v() is very nice. The script is now mostly yours, so should I list you as author or make it authorless?
My "perverse" use of V2 is how I always used V2 before Layers were introduced (1992?) .. as if V2 were Layer 2, and I don't recall any restrictions from Coda at the time about the extent of V2 spans. Mind you I probably didn't read that bit of the manual. The current manual is also mute on that point, in the only place I can find that it mentions V2 at all!

Attribute it however you like. At some point I would like to combine the tied_to function back into tie.lua. You certainly deserve credit for thinking of it and for discovering the FindPitch method.

That's kind of you, in which case I'll leave it with the dual accreditation if that's OK with you.

I have one other thought about the tie-removal routine. I don't think you should be removing tie-ends if you are not also removing tie-starts. You can actually delete entirely the untie_notes_in_selection routine and change the two "payload" line from
                            note.Tie = true
                            tied_to_note.TieBackwards = true

to
                            note.Tie = not untie_notes
                            tied_to_note.TieBackwards = not untie_notes

This will force both ends of the tie to be in the selected region.

A beautifully elegant solution Robert.

I merged and then realized that we may want the untie side to remove ties to nothing. It would be easy enough to do, but it will potentially leave orphaned tie-ends without a lot of fussy extra code. I mean to revisit this when I enhance tie.lua to make a single tied_to function. I can think about it then, if you like.

I also thought of unmatched tie ends and artefacts from other actions that one could reasonably expect "untie" to handle indiscriminately. For my own use I'll re-enable the indiscriminate untie routine. Also Nick just pointed out that this script is not in /src. I'll move it back there now.


[📚] Storage Api with signed urls for puts/gets

It wasn't mentioned anywhere in the documentation, about uploading / downloading with signed urls, So I assume it's not supported. I did notice the FirebaseStorage.instance.refFromURL(actualUrl), however looking at the source, it doesnt take into account the actual signature, it just reads the bucket & path.
I'm not sure how to go about this other than directly writing my own puts, however looking at the actual api, I'd love to use the already existing UploadTask & multipart backend firebase already has, Would it be possible to either document a good way to upload from a signed url in flutter, or perhaps it would be simple enough to support signed urls.
Additional Context
I have dynamic documents created by users & shared by users, so they have their own ids with access levels each. Since you cant read documents from storage rules, and the auth custom claims api restricts you to 1kb (firebase uses 600b out of that) I cant use claims either. My final attempt is by using signed urls for temporary upload access granted by a cloud function, however I'm weary on writing my own http post requests for the uploads because I need to support web also (cors & all), that and i'm not too familiar with how gcloud bucket apis work.
Perhaps someone more involved in the gcloud space / firebase apis might know a clean solution for this? Thanks!

/cc @Salakar for further insights.

Just registering my interest since I recently wanted to do this.
Although, I'm using http requests to the REST API to send files directly, as a work-around.

I really need this improvement.
In my project I stored the file-url to share files with another users. Plus, I restricted the storage rules to get the files just for authenticated users, this approach is useful because block unauthenticated users.
In counterpart, I unable to use some plugins like the Network to file image to download the files, because plugins usually use http/https to get/put files (the rules block the requests).
Sinned urls could very useful to allow this type of interaction, between the plugins that uses http/https requests and the Storage with rules that require some type of authentication.

Hey @cyberpwnn, this kind of feature request is relevant for the underlying SDKs that FlutterFire wraps around. You should raise issues on the iOS SDK & Android SDK. Although I do wonder why it's possible on the admin SDK (server side) but not on the client SDKs. There might be a security concern.


Add max_cat_threshold to GPU and handle missing cat values.

related: https://github.com/dmlc/xgboost/pull/8193

Port the max_cat_threshold parameter to GPU.
Use a 2-pass scan on GPU instead of a pseudo-bin, with missing values being handled.

This PR is redoing https://github.com/dmlc/xgboost/pull/8193?

In a slightly different way.

One problem here is that the missing value reduction is still being calculated on the constructor of EvaluateSplitAgent but not used

Yes, that's a problem. I can skip that by spliting up the Agent class.

If we merge this I think you should take the edge case tests from

The 2 approaches are not entirely the same. Using 2-pass scan helps with an edge case where we need to avoid including the last bin into split calculation. We would like to avoid spliting only on missing value.

Splitting only on missing values seems valid.

Copied the tests from https://github.com/dmlc/xgboost/pull/8193 and added some checks for default direction.


Incorrect fake block shown

Look at the screenshots

Never had this issue before, and it only occurs in 2-3 chunks
I am using the default config
Have no clue what is happening... Look at screenshots
This is the only occurrence in the world
(BTW, using carpet mod noclip to see)
In some areas it is a mix of saplings.
In the odd areas where saplings are missing there are random blocks
Underneath/above the areas are normal
Relogging randomises the blocks, but areas of blocks are the same pos/size/type
Using AntiXray 1.3.2 Fabric 1.20.4

Partly want to know what is happening (Might not be able to), but also if it happened with eg, enchanting tables it could be very laggy/unplayable
Some of the random blocks:

Screenshots:

Areas shot

This is most likely a server side issue. It looks like the block palettes are somehow messed up. This is most likely a mod incompatibility.

How reproduce-able is this issue? If it is reproducible please try to find the incompatible mod using binary search! (Make sure to use a backup in case you use content mods)
Are the blocks that are incorrectly shown present in the regular chunk (does the chunk section in the lowest screenshot contain all these sapping variants as blocks)
Do these chunk sections contain any special (custom) blocks?

1 Unfortunately, the restart fixed it
However, I recently added: https://modrinth.com/mod/minitweaks https://modrinth.com/mod/carpet-tis-addition and https://modrinth.com/mod/essentialaddons
2 There are some oak saplings around  in the area, but not the others
The other blocks except for the polished andesite are not in the area
3 No (not modded with custom blocks)