Linux compatibility

NOTES
DEPENDENCIES, RELATED ISSUES

[ ] HaloSPV3/HXE#217
[ ] HaloSPV3/AmaiSosu#21
[ ] Migrate GUI from WPF to Avalon
[ ] Wine/Proton instructions and/or management
[ ] Add SupportedPlatform attribute to Win32 API and Kernel interop. Add Linux equivalents when possible.

RELATED LINKS

WineHQ Application Entry for Halo SPV3
CrossOver

KEY ISSUES
DEPENDENCY: MahApps.Metro
This GUI framework aids in the creation of Metro-styled GUIs. It is dependent on Window Presentation Framework.
This will be especially difficult to resolve.
Solution

Migrate from WPF to Avalonia.

old proposals
A. Try mingling UNO with our dependencies, though that probably won't resolve MahApps.Metro's WPF dependency nor WPF's dependency on Windows runtimes.
B. Completely migrate away from MahApps.Metro and WPF to Avalonia, GTK#, QtSharp, Qml.NET, or UNO.

Windows Registry
See HaloSPV3/HXE#217
Steam

Pseudo-DRM was dropped. No more Steam issues...for now.
Path inference of MCC based on location of Steam.exe
Unix OSes don't use the Win32PE EXE format. As such, there won't be a Steam.exe to locate.
Possible Solutions
A. If System.Environment.OSVersion.Platform returns PlatformID.Unix, prompt the user to locate a 'Steam' assembly without requiring a file extension.
B. Allow the user to input the path to 'halo1.dll'. This would require the least amount of development time, but it forces more work upon the user if they choose this method to validate ownership of "Halo CE". As usual, the frontend would be implemented in SPV3 using the pre-existing backend in HXE.

ROAD PLAN
There are two routes we can take to resolve this issue.
STOPGAP MEASURES
Platform-dependent code paths will be determined at runtime. This implies the following:

Check the value of System.Environment.OSVersion.Platform to determine which OS we're running on. In most cases, we'll see either Win32NT or Unix. All possible values.

MS Docs Example:
// This example demonstrates the PlatformID enumeration.
using System;

class Sample
{
    public static void Main()
    {
    string msg1 = "This is a Windows operating system.";
    string msg2 = "This is a Unix operating system.";
    string msg3 = "ERROR: This platform identifier is invalid.";

// Assume this example is run on a Windows operating system.

    OperatingSystem os = Environment.OSVersion;
    PlatformID     pid = os.Platform;
    switch (pid)
        {
        case PlatformID.Win32NT:
        case PlatformID.Win32S:
        case PlatformID.Win32Windows:
        case PlatformID.WinCE:
            Console.WriteLine(msg1);
            break;
        case PlatformID.Unix:
            Console.WriteLine(msg2);
            break;
        default:
            Console.WriteLine(msg3);
            break;
        }
    }
}
/*
This example produces the following results:

This is a Windows operating system.
*/

BUILD FOR EACH PLATFORM

Compile apps as native Linux apps.
Proton and WINE settings would be automated (or informed to the user) for convenience.
WPF and WinForms would be replaced by Avalon UNO, .NET 6's MAUI, or another GUI framework to assist in maintaining our WPF-reliant, MahApps.Metro-powered GUI.
All Windows Registry interaction would need to be disabled entirely or adapted to work with Proton and WINE. The Windows Registry is required to determine if Retail or Custom Edition was installed legally. How does Proton and WINE handle that?

All Registry interaction will be removed with SPV3.3.1.

Multi-Platform Alternatives to WPF/WinForms

Avalonia
GTK (GTK#)
MAUI main repo does not have GTK bindings for Linux desktop compatibility. Some forks have bindings to GTK. MAUI does not support Windows 7.
Qt (QtSharp or Qml.NET)
UNO

Formerly tracked in HaloSPV3/HCE#249.

I've added a new header to the OP titled "Multi-Platform Alternatives to WPF/WinForms" with our current suggestions and some new ones.
Multi-Platform Alternatives to WPF/WinForms

Avalonia
GTK
MAUI main repo does not have GTK bindings for Linux desktop compatibility. Some forks have the necessary bindings. Should we wait for those to be merged upstream?
Qt (QtSharp or Qml.NET)
UNO

Qt (QtSharp or Qml.NET)

I didn't even know Qt had .NET bindings :open_mouth:

MAUI won't support Win7, so I'll be looking into UNO next.
They're similar in the sense that they're multi-platform wrappers for native UI libraries, except UNO supports Windows 7.

Does Steam's executable on Linux use the ELF file extension or no extension?
Scratch that.
Better question: how can we infer MCC's path via Steam on Linux distros?
The current method...
...locates Steam's root directory via Steam.exe or a .LNK shortcut to it
...searches Steam's directory for LibraryFolders.vdf, the file that lists all of Steam's local library paths
...searches all of the discovered libraries for "./steamapps/common/Halo The Master Chief Collection/halo1/halo1.dll"
(maybe we should be looking for ".../halo1/maps/a10.map" to ensure Campaign ownership)

Everything related to DRM (including MCC CEA detection) will be removed from or skipped in SPV3.3.1.
SPV3 is going completely free.
This means...

NO STEAM, MCC, OR RUNNING PROCESS REQUIREMENTS — SPV3's Activation sequence will be removed. The "No DRM" patch usually reserved for Halo PC or CEA owners will be always applied for everyone. The only "process" requirement will be when the borderless hack is applied to a running haloce process.
NO WINDOWS REGISTRY — The Registry was only needed for the Custom Edition and Retail Product Keys. Halo's built-in Windows Registry checks will always be patched out. As a result, HXE and SPV3 will also not require the Registry.

~Does Steam's executable on Linux use the ELF file extension or no extension?~ Scratch that. Better question: how can we infer MCC's path via Steam on Linux distros? The current method... ...locates Steam's root directory via Steam.exe or a .LNK shortcut to it ...searches Steam's directory for LibraryFolders.vdf, the file that lists all of Steam's local library paths ...searches all of the discovered libraries for "./steamapps/common/Halo The Master Chief Collection/halo1/halo1.dll" (maybe we should be looking for ".../halo1/maps/a10.map" to ensure Campaign ownership)

The game's location (via SteamID) can be found in:
~/.steam/steam/config/libraryfolders.vdf
So, pretty much the same method should work
As for the DRM being dropped, I'm a bit surprised; I hadn't expected that - why drop MCC's detection?
And how does dropping the check for ownership protect SPV3?

Regarding GUI, Avalonia is the most appealing atm. the https://github.com/AvaloniaUI/Live.Avalonia NuGet package provides Live Reloading XAML on Linux distros, but it does not provide in-IDE XAML design.

For the past month or so, I've been rewriting the "Compress Install folder on NTFS" feature to use Kernel32 P/Invoke instead of WMI.
Neither of these will work on WINE or Proton, so I may just disable P/Invoke calls to the Kernel32 and AdvApi during run-time.
We also make some P/Invoke calls for the borderless hack, so that will need to be disabled, too.

For the past month or so, https://github.com/HaloSPV3/HXE/pull/292.
Neither of these will work on WINE or Proton, so I may just disable P/Invoke calls to the Kernel32 and AdvApi during run-time.

Wine can be detected at runtime, for the Win installer, so you can just detect and disable the necessary NTFS; I actually know a project that did this recently, though not due to NTFS specifically:
https://github.com/neatodev/BmLauncher/blob/master/data/WineChecker.cs

Regarding GUI, Avalonia is the most appealing atm. the https://github.com/AvaloniaUI/Live.Avalonia NuGet package provides Live Reloading XAML on Linux distros, but it does not provide in-IDE XAML design.

As far as I'm aware though, there aren't any XAML design options for IDEs outside of Visual Studio (non-Windows exclusive IDEs); it's still nice it has live-reloading but we're not going to find a Linux XAML designer anywhere until some 3rd party is actually bold enough to make one, unfortunately :/

On top of .NET Framework 4.8 Runtime's poor quality on WINE, it doesn't support the C# 9 extensions present in some CsWin32 generated code. Only .NET 5 and up are guaranteed to support these extensions.
For testing purposes, I'm targeting net462 while overriding its C# 7.3 limits with C# 9 targets, but anything can break unexpectedly this way because of the net4x Runtime's limits.
.NET SDK 6.0.300 can compile SingleFile (all-in-one) executables for Windows 7 SP1 which solves the loose, unavailable, or wrong-version dependencies problems, but I'll need to get rid of WPF and WinForms because they are trim-incompatible and add at least 50MiB to a Self Contained deployment. This is the reason why I'm looking into alternative GUI platforms.
However, .NET 5+ Windows executables don't run on WINE. I'll have to build and release separate executables for Linux and MacOS.
This will require cross-platform GUIs, system-agnostic paths, WINE/Proton automation or user instructions, and...what else? Probably DLL-mod management.

Honestly, I wouldn't be too hard-pressed about MacOS support considering supporting OpenGL and Vulkan via anything (including Wine) is a nightmare and a half because of Apple; not going to talk you out of that, of course, just a heads up reminder I'm sure you might have heard of.
That aside, yeah, .NET5+ has had problems for a while with Wine; although, I know for sure native SingleFile executables are also do-able on Linux via .NET6, so that's cool.
Wine instructions should be easy, until automation is implemented; system agnostic paths really just boils down to some if-checks and avoiding backslashes :p
The meat of this will mainly be the cross-platform GUI; the DLL-mod management will just be moving stuff in and out of the game directory, I take it, right?

supporting OpenGL and Vulkan via anything (including Wine) is a nightmare and a half because of Apple;
I had completely forgotten about that. The only Mac user I know of is Masterz, the project lead SPV3 and Legacies. Iirc, he uses a somewhat older MacBook c. 2015. I don't know of any other Mac users.

Wine instructions should be easy, until automation is implemented; system agnostic paths really just boils down to some if-checks and avoiding backslashes :p
Yup. The hard part will be Wine/Proton management.

the DLL-mod management will just be moving stuff in and out of the game directory, I take it, right?
Indeed. The only data needed from a DLL is the version info and other assembly info, filename, extension, dates, and hashes for verification and file integrity.
The actualy loading of the mods will be handled by a mod loader DLL in the game directory (loaded by the game) and the rest in "./mods". Until Chimera 1.0 and OpenSauce incompatiblity is resolved, we'll be Monolith mod loader 0.3.0 which replaces Halo's keystone.dll. Chimera 1.0 has newer versions of Monolith built-in and replaces Halo's strings.dll (normally contains exception strings and more).

Meanwhile, the only cross-platform visual design available for Avalonia is via JetBrains Rider IDE.

Unfortunately, I can't afford a JetBrains Rider subscription :skull_and_crossbones:
Interesting though, that UNO has anything for that at all :thinking: I heard they were working on one but had been told previously it was for VS Code.
As for the VS Code Avalonia extension... That would be great if it weren't incomplete Lol; I can't imagine it would be done anytime soon though :/

https://stackoverflow.com/a/970134/14894786
Found a solution for DLLImport-related exceptions.

@JediMasterChief notified me in Discord that they'll be documenting their steps for Lutris/Proton compatibility this weekend.
Their method uses a fork of Wine specific to either platform: GloriousEggroll/wine-ge-custom, GloriousEggroll/proton-ge-custom

Here are the steps I took to get SPV3 installed and running on Wine. Sorry about the delay;
Use Winetricks, wine-ge-custom and a 64-bit prefix

Install mfc42 through winetricks
Install Halo CE and 1.10 Patch. You can simply double click if using default prefix, use the terminal to specify wine prefix to use before running the files, or run wine explorer via winetricks in the prefix of choice
Through Winetricks, install .Net 4.6.1 or later and the arial font
Create the directory you want to install SPV3 into via your Linux file manager, assuming you want any other location than
default
Run SPV3 installer using the same methods as step 2.
Navigate to the location you want to install to via browse. Avoid clikcing on the text box. Give the installer a couple of
moments to settle on that location before clicking install.
(potetially optional) Once the Open Sauce step is reached, skip it and instead use the latest amaisosu from github.
Install into the normal Halo CE folder or a copy of it to avoid overwriting SPV3's version of Open Sauce.
Open winecfg, add SPV3's haloce.exe to the applications menu, then with that exe selected in that menu, navigate to libraries and add an native, builtin dinput8.dll override, and configure other options as needed.
Follow the normal SPV3 setup from there.

How do we ensure haloce.exe (or any app for that matter) runs on the highest-performance GPU on any given Linux distro?
Would a specific app or library be needed to interact with drivers or does it have a vendor-agnostic settings panel similar to Windows 10 and 11?

@BinToss This is typically handled by the user; in hybrid-GPU scenarios, Optimus (Intel+Nvidia) would be invoked by the user either through Steam client launch options or with Lutris (non-Steam game launcher)
https://wiki.archlinux.org/title/NVIDIA_Optimus
Typically, doing this through steam directly is more difficult when using Proton, specifically, because Steam's UI for invoking environment variables and telling Proton what to do is a bit of a mess; I'd advise launching the executable by using "PRIME render offload" somehow, either by the to-be native launcher doing this itself or with a bash script:
https://wiki.archlinux.org/title/NVIDIA_Optimus#Using_PRIME_render_offload
https://wiki.archlinux.org/title/PRIME#PRIME_render_offload
As for Intel CPU + AMD GPU / AMD APU + AMD GPU combo specifically, I can't say because I'm not personally familiar with it

Which reminds me, now that you ask, users should be encouraged to have gamemode installed, for better performance; we should also detect if this is installed (or include it ourselves) and load this library to keep performance up

Related news:

Miris started a port of our WPF GUIs to Avalon, but found iterating on Linux to be difficult, to say the least.
DXVK 2.2 introduced improved support for D3D9 Partial Presentation, a feature utilized by WPF. Still, we should prefer targeting Linux (as opposed to relying on WINE and other translators) when it's an option.


Curriculum Review: Discrimination in Software

Prior Knowledge:
No prior knowledge needed.
Content:
Awesome stuff! On slide 6, the last point is about using descriptive variable names. It might be helpful to give examples of what 'good' descriptive names are. This may already be covered in an earlier part of the workshop, so can maybe just be referenced.
Learning goals:
All achieved!

Hi @jeaadams, good idea! I updated the slides for some examples. Let me know what you think!


Pmeshram/feat/updates

Overview/Summary
Added Community call page under Resources section on aka.ms/avm page
This PR fixes/adds/changes/removes

Replace me
Replace me
Replace me

Breaking Changes

Replace me
Replace me

As part of this Pull Request I have

[ ] Read the Contribution Guide and ensured this PR is compliant with the guide
[ ] Checked for duplicate Pull Requests
[ ] Associated it with relevant GitHub Issues or ADO Work Items (Internal Only)
[ ] Ensured my code/branch is up-to-date with the latest changes in the main branch
[ ] Ensured PR tests are passing
[ ] Updated relevant and associated documentation (e.g. Contribution Guide, Docs etc.)

I am doing more modifications, closing this PR and will raise a new one.


Typing 0 in a certain spot of a script in my game creates a memory leak

Godot version:
v4.0.dev.calinou.3b3a55ca0
OS/device including version:
$ uname -a
Linux unfa-desktop 5.12.2-1-MANJARO #1 SMP PREEMPT Fri May 7 17:53:15 UTC 2021 x86_64 GNU/Linux

$ inxi
CPU: 8-Core AMD Ryzen 7 1700 (-MT MCP-) speed/min/max: 1958/1550/3000 MHz Kernel: 5.12.2-1-MANJARO x86_64 Up: 25m
Mem: 6070.5/32103.4 MiB (18.9%) Storage: 3.87 TiB (52.3% used) Procs: 436 Shell: Bash inxi: 3.3.04

$ inxi -G
Graphics:  Device-1: Advanced Micro Devices [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] driver: amdgpu
           v: kernel
           Device-2: Logitech HD Pro Webcam C920 type: USB driver: snd-usb-audio,uvcvideo
           Display: x11 server: X.Org 1.20.11 driver: loaded: amdgpu resolution: 1: 1920x1080 2: 1920x1080~60Hz
           OpenGL: renderer: Radeon RX 580 Series (POLARIS10 DRM 3.40.0 5.12.2-1-MANJARO LLVM 11.1.0) v: 4.6 Mesa 21.0.3

Issue description:
Godot freezes and steadily allocates more and more memory after typing 0 before the first character in line 11 in Chat.gd script in my game.
The bug in action:
https://youtu.be/kII1XJX1LII
Steps to reproduce:

Load provided project
Go to Chat.gd file
Insert 0 before the first character in line 11
Watch Godot freeze and eat memory

Reproduction project:
https://github.com/unfa/liblast/tree/a54eb54293aa771a6b4bd642eee7312af7422e96

I believe this to be the same issue as
godotengine/godot#50750

Consolidating into #50865


Listing a directory returns an array of strings

Calling ftp.list("/my/path") returns an array of strings instead of objects. I tried with a different ftp server and it worked as expected.
This issue is more of a question to understand whether it's something that can be fixed by promise-ftp or is this my ftp servers configuration issue?

My fault. This issue is not something promise-ftp can fix, but node-ftp. A duplicate of https://github.com/mscdex/node-ftp/issues/188


Report modal presentation leak on iOS 17 beta

As a developer integrating Pillarbox I want to be able to use AVKit VideoPlayer as a user interface for my app. This standard user interface has a leak on iOS 17 beta 8, though, when using with a modal presentation having the player stored as @State.
Acceptance criteria

The aforementioned issue has been reported to Apple.

Tasks

[ ] Open Apple feedback.

Issue reported as iOS 17 regression: SwiftUI modals leaks state when stored as an instance of a class (FB13126425). Also discussed on the developer forums here and here.


Alerting of inaccurate GPS measurement in upload session

Resolves #44

Done in https://github.com/protect-earth/TreeTracker/pull/89


Downgrade kafka_exporter to v1.4.0

Related issue: #2748
Related PR: #2772
We use version 1.4.0 since 1.4.1 and 1.4.2 contain critical bug (see https://github.com/danielqsj/kafka_exporter/issues/273).

/azp run

It didn't help 😞. Metrics endpoint is still unreachable in some cases.

# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
go_gc_duration_seconds{quantile="0.75"} 0
go_gc_duration_seconds{quantile="1"} 0
go_gc_duration_seconds_sum 0
go_gc_duration_seconds_count 0
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 13
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.17.1"} 1

/azp run

/azp run


File Types Retrieval

The file type object with its belonging properties is stored as a json file (embedded in resources dir).

This change is

src/main/kotlin/org/bibletranslationtools/fetcher/data/FileType.kt, line 6 at r7 (raw file):
Previously, jsarabia (Joe) wrote…
what is this used for?

I recognize that the text of these properties will be localized in a .properties file, so I put these to store the "key" of those message/text in .properties file.
I'd be happy to change if there's a better way of doing this.

src/main/kotlin/org/bibletranslationtools/fetcher/repository/implementations/FileTypeRepository.kt, line 11 at r8 (raw file):
Previously, jsarabia (Joe) wrote…
When I think of a repository, I think of accessing data from a persistent data store of some kind. FileTypes feel more like an abstract concept (and also, really, do we care so much?)

@jsarabia How about "ResourceType" instead?

src/main/kotlin/org/bibletranslationtools/fetcher/data/FileType.kt, line 6 at r7 (raw file):
Previously, AnonymousWalker (Tony) wrote…
I recognize that the text of these properties will be localized in a .properties file, so I put these to store the "key" of those message/text in .properties file.
I'd be happy to change if there's a better way of doing this.

Alternatively, I am thinking of adding another "model class" that serves the front-end instead of directly using this data class.

src/main/kotlin/org/bibletranslationtools/fetcher/repository/implementations/ProductCatalogImpl.kt, line 28 at r10 (raw file):
Previously, jsarabia (Joe) wrote…
Done what? You're still silently suppressing the exception.

Oh I misunderstand that. Will revise it.


boost: Taming the boost / modularize libraries?

Describe the bug
Maybe not a "bug" in the classical sense, but at least a complexity issue hitting nixpkgs maintenance. Small bugfixes to one of boost's many libraries lead to massive rebuilds for nixpkgs, even though it's not necessary to achieve reproducibility. For example, this can be seen here (https://github.com/NixOS/nixpkgs/pull/254578), where @ofborg attests a huge amount of packages requiring a rebuild, even though only boost::phoenix was changed. From boost itself, only boost::log, boost::spirit and boost::msm use that library (boost::numeric, boost::type_erasure and boost::range could be added if we also count examples and tests), though. Similarly, it can be expected that most of the packages where a rebuild is triggered also don't actually use boost::phoenix.
Steps To Reproduce
Steps to reproduce the behavior:

Apply a small patch to an even more seldomly used boost library in pkgs.boost
Trigger a rebuild of all reverse dependencies according to nixpkgs
Watch it build for a possibly long time (depending on available build hardware)

Expected behavior
Only packages that actually use the patched library get rebuilt.
Additional context
boost itself has switched to modular development of the individual libraries some time ago. Many distributions also split boost libraries into separate packages. Modularization of the libraries could also improve the situation for nixpkgs. With separate packages for the libraries, reverse dependencies could import just the libraries they actually use, avoiding having to be rebuilt when other libraries are changed. If the situation described for boost::phoenix also applies to more libraries, it could also make sense to move examples and/or tests into separate nixpkgs packages, further reducing unnecessary interdependencies.
Notify maintainers
@hjones2199
Metadata

system: "x86_64-linux"
host os: Linux 5.15.0-82-generic, Ubuntu, 22.04.3 LTS (Jammy Jellyfish), nobuild
multi-user?: no
sandbox: yes
version: nix-env (Nix) 2.6.0
channels(root): "nixpkgs"
nixpkgs: /root/.nix-defexpr/channels/nixpkgs

I started working on this by splitting boost_python and boost_numpy into a separate derivation. I'll put it up for review once 1.86 is default and the remaining fixes for that are merged.


Wrong terminology - 'limits' vs. 'requests'

I believe it should be 'resource requests' instead of 'resource limits' in the following sentence under the 'Pods' section of this page.
"When you create a pod, you can define resource limits to request a certain amount of CPU or memory resources."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a445f8cc-6581-ad43-1dba-719a94eda81a
Version Independent ID: 31280746-d8de-91b4-d430-91693c2e869d
Content: Concepts - Kubernetes basics for Azure Kubernetes Services (AKS)
Content Source: articles/aks/concepts-clusters-workloads.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: mlearned

Thanks for the feedback! We are currently investigating and will update you shortly.

I have created a PR with the update, changes will be live in few hours.
Thanks for bringing this to our attention. We will now close this issue. If there are further questions regarding this matter, please tag me in a comment. I will reopen it and we will gladly continue the discussion.


When unauthenticated, replying to a comment results in Error creating comment: undefined

Would be helpful if it actually provides helpful error message instead of that.

Fixed the issue where unauthenticated users received an undefined error when trying to comment. Now, an appropriate error message is shown, and users are redirected to the login page.


Adding detailed audit logging for the Merge option for importing datasets

Rationale
Spec: https://docs.google.com/document/d/110fl8qRIk1o9lEgFq8RKFWNW5vo_cYCiWkE1doFq214/
Add detailed audit logging for merging bulk imports for datasets
Changes

Added a DetailedAuditLogDataIterator to the DI queue
Added DatasetSchemaTable.addAuditEvent override methods

Changes were pulled into https://github.com/LabKey/platform/pull/1839

Changes were pulled into https://github.com/LabKey/platform/pull/1839


[v9rc3] On error the debugger shows the wrong lines of code

When an error is thrown the debugger always shows the wrong line of code. If the error is on line x the code displayed always starts at line x+1
See screenshot for an example (error line 47, code displayed is line 48)

https://github.com/concrete5/concrete5/issues/9745

Oups late to the party, sorry :)
Closing then


chore: rearrange some code to prepare for more span pointers

Motivation
We're going to be adding more span pointers tests so paving the way by rearranging some of the code.
Workflow

⚠ Create your PR as draft ⚠
Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
Mark it as ready for review

Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
Framework is modified, or non obvious usage of it -> get a review from R&P team

:rocket: Once your PR is reviewed, you can merge it!
🛟 #apm-shared-testing 🛟
Reviewer checklist

[ ] If PR title starts with [<language>], double-check that only <language> is impacted by the change
[ ] No system-tests internal is modified. Otherwise, I have the approval from R&P team
[ ] CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
[ ] A docker base image is modified?

[ ] the relevant build-XXX-image label is present

[ ] A scenario is added (or removed)?

[ ] Get a review from R&P team

since there has been some flakiness with the other span pointers tests (may be due to some interaction between moto and async fastapi code) we're removing all but the flask span pointers webhost implementations. we're testing the boto3 tracing interactions, not the webserver interactions, so this should be fine. c.c. @cbeauchesne

the remaining failures seem to be unrelated to my changes?


Use num_streams as parallelism when read thread is enabled.

What problem does this PR solve?
Issue Number: close #6343
Problem Summary:
What is changed and how it works?
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-all-tests

/merge

/rebuild

/run-integration-test


New feature : Email / Slack / Telegram notifications

Extra module to allow notifications across messaging platforms
Description
Currently this only supports the 3 mentioned, but should be fairly easy to extend. There's one central method that will push a message to all configured platforms when you call log.notify()
I still need to add some of the hooks, summary every X minutes and when a loan is made over the xdaythreshold.
I'm actually wondering about the best way to do the summary is. I was initially going to use the schedule module, but since I bothered to add all these in pure python I thought I may as well try to stay away from external modules. I've had a quick look at sched, which seems it might work.
I still have a few TODO items, including all the docs and fixing up a few things I left out.
This closes https://github.com/Mikadily/poloniexlendingbot/issues/153
TESTING STAGE
Currently running
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have read CONTRIBUTING.md
[x] I fully understand Github Flow.
[x] My code adheres to the code style of this project.
[x] I have updated the documentation in /docs if I have changed the config, arguments, logic in how the bot works, or anything that understandably needs a documentation change.
[x] I have updated the config file accordingly if my change requires a new configuration setting or changes an existing one.
[x] I have tested the bot with no issues for 24 continuous hours. If issues were experienced, they have been patched and tested again.

Need to update documentation.

Yea, lots of doc updates to do. Hoping to get some time over the next week to close out the final modifications. I sat down to do them and ended up doing that other pull request instead.

I think this is done from my side, pending any modifications.

I don't really agree with all those Codacy issues, but I'll change them if you want to keep it consistent.

@laxdog isn't that related to Python 2 vs 3?

Yea, you don't have to use them in 2, but you can if you want. You have to use them in 3 as it's a function. Which is why I use them, it's backwards compatible.
I only said I'd change it if you want, as the rest of the project doesn't use them.

@Evanito can you re-review?

Just removed the parenthesis check from Codacy. I have been swamped the past weeks, glad to have some free time.

Sorry for the number of commits, I had to use the online editor.

Crazy!! we have notifications!

Really happy to get this in btw, I've been using it a lot. It's really useful for me. The notification on exception and transfer from exchange account are particularly useful.
Thanks for merging!

@laxdog Thanks for for taking the initiative, your help is appreciated.

That is awesome laxdog, the docs need a little more work, or I'm doing something wrong. I'll file a bug if I can't figure it out, but I think the issue is that with telegram, "telegram_bot_id" should be renamed to "telegram_bot_token", and "telegram_chat_ids" should be broken up into "telegram_bot_name" and "telegram_bot_username", but that's at least how I'm trying to use it, but I get an error with each invocation, despite changing the positions of variables:
Welcome to Poloniex Lending Bot
Started WebServer, lendingbot status available at http://<IP_ADDRESS>:8000/lendingbot.html, http://<IP_ADDRESS>:8000/lendingbot.html
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/sched.py", line 117, in run
    action(*argument)
  File "/home/ronald/bots/poloniexlendingbot/modules/Lending.py", line 96, in notify_summary
    log.notify(Data.stringify_total_lended(*Data.get_total_lended()), notify_conf)
  File "/home/ronald/bots/poloniexlendingbot/modules/Logger.py", line 150, in notify
    send_notification(msg, notify_conf)
  File "/home/ronald/bots/poloniexlendingbot/modules/Notify.py", line 83, in send_notification
    post_to_telegram(msg, nc['telegram_chat_ids'], nc['telegram_bot_id'])
  File "/home/ronald/bots/poloniexlendingbot/modules/Notify.py", line 50, in post_to_telegram
    raise NotificationException("{0}\n{1}".format(e, msg))
NotificationException: HTTP Error 403: Forbidden
Your bot id is probably configured incorrectly

OK, sorry. I'd never used telegram before this implementation, so I'm probably off on the terms.
I've a few things to fix next week, so I'll fix this then. Thanks for the feedback, it's really useful.
I also see you have the same address twice in your bot web address. I'll fix that too.

Something I noticed, was what looked like incomplete variables, take a look at my diff below (it doesn't work still, but it did change the error message, and I should probably just file a bug on it):
diff --git a/modules/Notify.py b/modules/Notify.py
index 1acadf2..795f058 100644
--- a/modules/Notify.py
+++ b/modules/Notify.py
@@ -38,10 +38,10 @@ def post_to_slack(msg, channels, token):
         check_urlib_response(response, 'slack')

-def post_to_telegram(msg, chat_ids, bot_id):
-    for chat_id in chat_ids:
+def post_to_telegram(msg, telegram_chat_ids, telegram_bot_id):
+    for chat_id in telegram_chat_ids:
         post_data = {"chat_id": chat_id, "text": msg}
-        url = "https://api.telegram.org/bot" + bot_id + "/sendMessage"
+        url = "https://api.telegram.org/bot" + telegram_bot_id + "/sendMessage"
         try:
             response = urllib2.urlopen(url, urllib.urlencode(post_data))
             check_urlib_response(response, 'telegram')

Hello,
I'm having the same issue here:
Welcome to Poloniex Lending Bot
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/Fabien/poloniexlendingbot/modules/MarketAnalysis.py", line 54, in update_market_loop
    delete_old_data()
  File "/home/Fabien/poloniexlendingbot/modules/MarketAnalysis.py", line 74, in delete_old_data
    if get_day_difference(row[0]) < max_age:
IndexError: list index out of range

WARN: Exception found when analysing markets, if this happens for more than a couple minutes please make a Github issue so we can fix it. Otherwise, you can safely ignore it. Error: list index out of range
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/sched.py", line 117, in run
    action(*argument)
  File "/home/Fabien/poloniexlendingbot/modules/Lending.py", line 97, in notify_summary
    log.notify(Data.stringify_total_lended(*Data.get_total_lended()), notify_conf)
  File "/home/Fabien/poloniexlendingbot/modules/Logger.py", line 150, in notify
    send_notification(msg, notify_conf)
  File "/home/Fabien/poloniexlendingbot/modules/Notify.py", line 83, in send_notification
    post_to_telegram(msg, nc['telegram_chat_ids'], nc['telegram_bot_id'])
  File "/home/Fabien/poloniexlendingbot/modules/Notify.py", line 50, in post_to_telegram
    raise NotificationException("{0}\n{1}".format(e, msg))
NotificationException: HTTP Error 400: Bad Request
Your bot id is probably configured incorrectly

My config setting is similar to @praeluceo
Should we put the name of the channel somwhere as well?

Hi Guys!
I'm new here. Hope this isn't the wrong place to post this error.
Trying to get Telegram notifications to work but getting this error. Bot id is configured correctly as I tested on a browser and it works fine. Can someone explain what the 400 error is?
Error during summary notification: HTTP Error 400: Bad Request
Your bot id is probably configured incorrectly

Apologies again if this ain't the right place to post this error.
Appreciate any assistance. I'm not a coder.

Hey @Fabien98 and @racydex , I did a whole update of the docs, after this issue was reported. It explains everything as a few people were having problems. You can see the Telegram section here: http://poloniexlendingbot.readthedocs.io/en/latest/configuration.html#notifications

@laxdog Thanks for this update.
I saw the new docs and managed to get  the telegram notifications thanks to this easy step-by-step.
Thank you!


Cloudflare cache files which they shouldn't

Expected Behavior
Cf-Cache-Status: REVALIDATED
Actual Behavior
Cf-Cache-Status: HIT
Section that requires update
I'm not sure
Additional information
My Page Rule settings:
Origin Cache Control: enabled
Cache Level: Cache Everything
I tested two Cache Control header from my origin server:
1.Cache Control: no-cache
2.Cache Control: public, s-maxage=0, max-age=0, no-cache
I expect Cloudflare to revalidate my file just as it says in document:
https://developers.cloudflare.com/cache/about/cache-control/
no-cache Caches and always revalidates. Does not serve stale.
https://developers.cloudflare.com/cache/best-practices/customize-cache/
Resources that match a Cache Everything Page Rule are still not cached if the origin web server sends a Cache-Control header of max-age=0 , private , no-cache , or an Expires header with an already expired date.
But following is what happened to Cf-Cache-Status while I keep refreshing my chrome:
EXPIRED → REVALIDATED → HIT → HIT → HIT
and I do receive stale file
I asked in community, but get no-reply.
Any help is appreciated. Thank you so much.

Howdy @UchihaYuki, sorry to hear you're having issues, but the Cloudflare community is probably your best place to get answers on topics like that.
We really only have the expertise to make updates to the documentation for these Issues, not to troubleshoot issues with implementation.
Going to cc: @dcpena as to whether the docs might need updates, but closing this issue out as it doesn't seem like it involves any doc updates.

Hope somebody and reply my community post:
https://community.cloudflare.com/t/cloudflare-cache-for-no-cache/367521
Thank you, guys.
@dcpena @kodster28


Serializing the state of the Starlark program/REPL

Is there a way to serialize the state of a Starlark execution at a certain point of execution, and sometime later load from the serialized state to continue execution?
Context: I am implementing formal methods system that uses a Python'ish language. To simplify the implementation, I am using Starlark (with Go), so execute individual statements. It is a model checker, so whenever there is a choice of two possible transitions, the model checker will explore both the possibilities. For example, randomly choosing between head/tail will be model checked as, the state of the system will be cloned. In one world, the coin would have come out head and we will explore that path. In the other world, the coin would have landed tail and we will explore that path.
To implement this, I see only a few options.

Execute one statement at a time: This is the current implementation, but slow. After each statement, the state is just the global variables. That is, loops, function calls, conditionals etc have to be reimplemented. After each statement execution, the global variables are frozen. So, these cannot be used as it is to execute the next statement. Therefore, all the global variables should be manually cloned recursively. This has two major issues.

This is slow
maintaining the reference structure is harder :(
For the initial launch and small models, this is okay, but the performance issues show up badly as the users model larger systems.

Use ExecREPLChunk: Almost the same as the previous approach, but using REPL related exec methods prevent freezing the variables. This reduces the need to deep clone the state. This also has two issues.

This is slightly better than before, but the API doc says, the API compatibility might be broken in the future release. I understand this comment was added in the source code long long ago, but curious if this is still the case.
Maintaining the reference structure is still hard

I am looking for any alternate options available to clone or serialize/load the state of the execution.
If it is possible to serialize the REPL state and be able to restore, this would simplify the implementation significantly. Is this possible?

I'll share here my responses to another user who contacted me privately asking for a very similar feature. I've paraphrased  their questions and edited my answers a little. To be clear, this feature is complex and invasive and I am not convinced it is worth supporting (or even fully feasible) in this repo; my advice below should be thought of as how to prepare of a fork of the interpreter that supports it.
...

How can I halt execution at a point, save the entire execution state in a persistent manner (on disk or db, for example), and then continue execution at a further point in time?

Does the state in your case involve running threads, or just the state of the heap after all threads have finished? The latter is a strictly simpler problem because it doesn't require you to get into the guts of the interpreter to the same degree; you just need to implement a GC-like marking phase over the heap and, for each object, serialize it. Of course it requires that you know how to serialize every type of object you encounter, so it needs the "closed world" assumption: it can be implemented if you control the entire application, but not as a library linked against unknown new types of starlark.Value.
Given the closed-world assumption, it seems like it should be relatively easy for you to fork starlark-go and add the hooks you need; you shouldn't need to change the original code very much to do this, which should make it easy for you to keep up with patches (which are in any case infrequent).  Do you expose any Go APIs that mention starlark-go? If so, this would of course make the problem harder.

We completely control the types of values available.

You would need to serialize the state of every thread. That means every Starlark frame (the operand stack, and all values reachable from it; the iterator stack, and all iterators) and every Go frame, including the local state of functions like sorted, min, and max, which all make Go->Starlark calls. Min and max make more than one, so you would need to remember the logical program counter too; and you'd need to do this for every built-in you've defined that can make callbacks (and for every future one that you add). If any of them hold locks, you'd need to record that. And then you need to arrange for both Starlark and Go frames to be resumable at a given logical program counter that makes a starlark.Call. And of course you'll also need to make sure that both ends of the channel agree on the Starlark version: you can't suspend in one version and resume in another.

Why do sorted, max, min, etc, have a local state?

If your task is to save the stack of a Starlark thread so that it can be resumed later, then you need to save the state of any Starlark function implemented in Go that happens to be active too. Functions like sorted, min, and max make callbacks, so can appear in the middle of the stack, not just as a leaf. Therefore you will need to redesign those functions so that they can be suspended and resumed. Of course, it's highly unlikely that any of these three particular functions will make a callback that does more than compare two values, far less trigger a thread suspension. But that's the discipline required by the model; and perhaps in future you will need to add more important Go functions that make Starlark callbacks.

what if we restrict ourselves to suspending stacks of pure Starlark functions?

It does make it simpler. You would need the act of executing the suspend operation to cause every active frame on the stack to record its state into a suspension (serializable continuation). You could do that by handling ErrSuspend after each CALL operation at each frame, or you could record the necessary information beforehand, similar to the way we update frame.pc for each program counter increment. Either way, you would need to ensure the operand stack and iter stack were saved in the frame or continuation. That's the serialization part. For deserialization, you would need to change Call and
Function.CallInternal so that, if called on a Thread with an associated continuation to resume, it would rehydrate the operand stack, set the program counter, and enter the loop at the resumed PC ("in the middle"), immediately triggering another Call, which would recursively restore the rest of the old stack. The thread's "resume" continuation would be discarded and execution would continue as normal.

Thanks @adonovan for the detailed answer.
In my case, it must be a lot simpler.

I always run with a single thread. The state needs to be captured only after that thread completes. No need to pause a running thread.
Closed world assumption. The users can define a new type using a custom language, that would be translated to a specific go type that the application manages. (They all instantiate a single go type Role, with different name/fields). It does not expose a go library for users to create or directly manage starlark-go types.

So the solution still is to manually go over all the objects in heap and construct the object graph like GC's Mark phase.

Thanks @adonovan for the detailed answer.
In my case, it must be a lot simpler.

I always run with a single thread. The state needs to be captured only after that thread completes. No need to pause a running thread.
Closed world assumption. The users can define a new type using a custom language, that would be translated to a specific go type that the application manages. (They all instantiate a single go type Role, with different name/fields). It does not expose a go library for users to create or directly manage starlark-go types.

So the solution still is to manually go over all the objects in heap and construct the object graph like GC's Mark phase.

I agree, those two assumptions do make things a great deal simpler as they mean the only types of Value you need to deal with are your Protean "Role" type, plus those defined by the interpreter itself. The main types--string, list, dict, and so on--are all API-complete, so you can make a perfect clone of a dict by making queries on the public API of the original one. But that's not the case for a handful of opaque values such as that returned by the range function: for those, you'll need to create new API to access their representation. I suggest you start with a quick prototype to see how much new API is needed. If it's just the range type, we could make it API-complete too, but I suspect there are others (e.g. string iterators).

Actually, range is immutable, so I don't even need to clone. For now, the main reason I was asking for serialization is it has a higher likelihood of being already present or to be implemented than a clone functionality (like CompiledProgram type to store and load compiled code for repeated execution).
That is, if I could have a way to serialize REPL session and reload from a saved state, I thought I could use it for cloning. And coincidentally,  https://github.com/google/starlark-go/blob/master/repl/repl.go had LoadModule function, that looked similar to Python library that can save/reload the interpreter session https://github.com/uqfoundation/dill/blob/e09908f4ba480ee326f538aa05715c1cc919bdf2/dill/session.py#L328
In any case, I will implement it the way you suggested.


Link to Image Alignment has a typo

The hyperlink from that should redirect to the notebook has a typo for the Computer Vision project on Image Alignment. The slide can be found here
Currently it is set as: https://colab.research.google.com/github/NeuromatchAcademy/course-content-dl/blob/main/projects/ComputerVision/Image_Alignment.ipynb
but it should be (according to the corresponding GitHub repo), hence no capital letters for Image and Alignment: https://colab.research.google.com/github/NeuromatchAcademy/course-content-dl/blob/main/projects/ComputerVision/image_alignment.ipynb

Thank you very much for catching this broken link! We will fix it asap.


[Communication - Phone Numbers] PhoneNumbersClient - get phone number errors if phone number not found failing in nightly runs

Communication - Phone Numbers nightly test runs are failing with:

Error message:
expected undefined to equal 'NotFound'
Stack trace:
AssertionError: expected undefined to equal 'NotFound'
at Context. (/mnt/vss/_work/1/s/sdk/communication/communication-phone-numbers/test/public/get.spec.ts:40:16)
at processTicksAndRejections (internal/process/task_queues.js:97:5)

For more details check here:

https://dev.azure.com/azure-sdk/internal/_build/results?buildId=1167210&view=results

This issue was fixed and is now passing on more recent runs.


Fix Match Stats Tracking Improperly

Current method used by registering the stats from the WinnerDetermined gamestate callback would make modes that relies on rounds to register every round as a whole match played, duplicating and stacking the stats registered several times for the same map.
I currently changed where the hook happens so this ensures the stats will register only once and only when the match really ends, still disconsidering Epilogue state.

@Zanieon already tested this himself, so I'm fine with merging this without further testing.


Unable to export stats?

Hi!
I've downloaded and activated the plugin.
I couldn't see the list of quizzes in the drop down list for exporting of data though.
See image: http://postimg.org/image/6ck3e4n13/
Thanks!

Hi, i have the same exact problem.hope if there could be a solution for this, it would be a very helpful plugin. Many thanks.

Hello summertrove & ingyanees, and thank you for giving my (sub)plugin a try!
I just found the catch and pushed the fix (involving /admin/partials/*.php files): the table prefix was hardcoded in the queries instead of being retrieved dynamically.
Can you please try again with the new files and give me your feedback?
Thank you in advance - and thank you SO much for helping me fixing stupid bugs :)


Change default consistency level to QUORUM

This PR changes the default consistency level from ONE to QUORUM.

@rkarthik007 @m-iancu
Please review.


if city and LK have same name add (LK) to the name

It's not clear which one is the LK or Bundesland or city

we have two different ways of dealing with this:
dropdown:
Augsburg (kreisfreie Stadt) vs Augsburg (Landkreis)
text on the backside:
Augsburg vs. Landkreis Augsburg
Maybe we can use the second version for the headline? I think the first version makes sense for the dropdown for the alphabetical order


multi function for Qemu EPC

Summary
Impact
Testing

[Experimental Bot, please feedback here]
I need the content of the PR to assess if it meets the NuttX requirements. Please provide the following:

The actual PR content: This includes the PR title, description, and any code changes.
Links: If the PR is already open on GitHub, please share the link.

With this information, I can help determine if the PR meets the listed requirements.


Tune gstreamer parameters for each architecture

We should figure out the best encoder (alternative to vp8enc) for each platform. Maybe we can get some hints from how encodebin chooses the encoder.

Some notes that may be helpful to whoever takes this up:

gstreamer has an internal rank for each element, and that's how encodebin chooses them along with the capabilities (caps)
there is this code here that's one of the few examples using encodebin that i've found:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/tree/master/tests/examples/encoding
of interest, while reading the source, is this section here. Those code snippets are how you can get all elements of a minimum rank. The documentation is here for c and here for python.:

If you're unfamiliar with GObject C, g_class_name_method_name(instance, param1, param2) becomes instance.method_name(param1, param2) in python. Since C was not designed with object oriented programming in mind, this is the convention used.
There are prettier languages that wrap this ugliness and write the GObject C for you. There are examples in that documentation there for Vala. Vala is also really easy to bind for python (bindings are "free"). It's only real downside is it only plays well with GObject, but GStreamer is written in GObject, so it's perfect for that. If anybody chooses this admittedly eccentric language, lmk if you need any help (eg. with build files or cryptic error messages out of valac).

More notes on encoders:

Nano/Tx1 do not support vp9 but do support vp8
Tx2/Xavier does support vp9 but Xavier does not support vp8
the plugin guide for Nvidia with some examples is here.
the webm container format only supports vp8/vp9
the mp4 conainer does not support vp8/vp9 but does support avc (h264) / hevc (h265)
a support table of which container support which codecs is here.

Also, I found out that the Googler I know closely worked on the vp8 encoder and he said it's every effecient even in software (and I've seen it run on a potato), so if we have to have a software fallback, it's a good one.


Fasta does not recover after a missing identifier

print(x) {}
main() {
  print(is String);
}

Expected: a single error reported, and a synthetic identifier inserted.
Actual: the whole function is nuked.
library;
import self as self;

static method #main() → dynamic {
  throw "file:///test.dart:3:9: Error: Expected an expression, but got 'is'.\n  print(is String);\n        ^";
}

@bwilkerson @danrubel

Thank you.
There are actually two problems here:

The parser isn't able to recover from this expression. I think @danrubel will get to that.

Fasta doesn't handle parser errors well and completely destroys the program replacing it with a synthetic main method called #main that just throws a string.

@bwilkerson will fix the first problem.

I believe that the first issue should be fixed with https://dart-review.googlesource.com/c/sdk/+/7558.

This issue might be impacted by https://dart-review.googlesource.com/c/sdk/+/12040.

Yes, also no failing tests in Analyzer marked with this issue.


Registration Agency: AR-CENOC

Hi AidStream team,
Please can the following Registration Agency be added to Aidstream. This agency was added by Org ID some time ago but I cannot see that it has been requested to be added to AidStream as well.
Please see Discuss forum post: https://discuss.iatistandard.org/t/added-proposed-addition-to-the-organisation-registration-agency-codelist-argentina/944
And Org ID page: http://org-id.guide/list/AR-CENOC
Code: AR-CENOC
Name: The National Centre of Community Organisations
Country: Argentina
Description: Law Nº 25.855, de Voluntariado Social (Social Volunteering) established in 2004 a role for El Centro Nacional de Organizaciones de la Comunidad to maintain a database of civil society organisations (CSOs).
Thank you

Hi team,
It has also come to my attention that the following Identifier is also unavailable on AidStream. Perhaps this can be added at the same time as the above?
Org ID page: http://org-id.guide/list/JE-FSC
Code: JE-FSC
Name: Jersey Financial Services Commission
Country: Jersey
Description: Companies and Non Profit Organisations operating in Jersey register with the Financial Services Commission (JFSC).

Hi Josh,
Code: JE-FSC & Code: AR-CENOC both have been added to AidStream.
Thank you.
Regards,
Anjana

Hi Anjana,
Great, thanks for adding them. Can you advise when these will become available on the system? I can't currently see them when I select AR or JE on the Country field.
Thanks,
Josh

Hello Josh,
Can you please check again.
Looking forward to your response.
Regards,
Anjana

There now, thanks Anjana!


Case study models

Types of changes

[x] Enhancement (project structure, spelling, grammar, formatting)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

A description of the changes proposed in the Pull Request
Implements Models for PR

@jmcook1186 I rebased. Please do not reverse merge the remote.


refactor(parser): refactoring the annotation processor and adjusting the way operations are created (GitHub #206)

Codecov Report
Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (c6e3922) 89.83% compared to head (cc373f1) 89.82%.

:exclamation: Current head cc373f1 differs from pull request most recent head 094e4b9. Consider uploading reports for the commit 094e4b9 to get more accurate results

Files
Patch %
Lines

...ler/AbstractStandardAssembleAnnotationHandler.java
88.13%
6 Missing and 1 partial :warning:

...er/AbstractStandardOperationAnnotationHandler.java
96.42%
1 Missing and 1 partial :warning:

...core/parser/operation/SimpleAssembleOperation.java
50.00%
0 Missing and 1 partial :warning:

...re/parser/operation/SimpleKeyTriggerOperation.java
75.00%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##                dev     #207      +/-   ##
============================================
- Coverage     89.83%   89.82%   -0.02%
+ Complexity     1528     1518      -10
============================================
  Files           149      150       +1
  Lines          3767     3792      +25
  Branches        370      371       +1
============================================
+ Hits           3384     3406      +22
  Misses          210      210
- Partials        173      176       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


added race symbol for the order of the last

Here's the race symbol for the order of the last.
I don't know your build process so I couldn't check if it works.
I don't expect any issue given the very small size of the change, but you never know...

Looks good, thanks!


Make index pattern always show under library

Description

Issues Resolved

Screenshot

Testing the changes

Check List

[ ] All tests pass

[ ] yarn test:jest
[ ] yarn test:jest_integration
[ ] yarn test:ftr

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[ ] Update CHANGELOG.md
[ ] Commits are signed per the DCO using --signoff

Codecov Report

Merging #115 (5a93927) into workspace (acaf609) will decrease coverage by 23.13%.
The diff coverage is 0.00%.

:exclamation: Current head 5a93927 differs from pull request most recent head d2313f8. Consider uploading reports for the commit d2313f8 to get more accurate results

@@              Coverage Diff               @@
##           workspace     #115       +/-   ##
==============================================
- Coverage      65.77%   42.64%   -23.13%
==============================================
  Files           3336     1597     -1739
  Lines          64492    32325    -32167
  Branches       10264     6058     -4206
==============================================
- Hits           42420    13786    -28634
+ Misses         19496    17497     -1999
+ Partials        2576     1042     -1534

Flag
Coverage Δ

Linux_1
?

Linux_3
42.64% <0.00%> (-0.01%)
:arrow_down:

Windows_1
?

Windows_2
?

Windows_3
?

Windows_4
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed
Coverage Δ

...on/index_patterns/index_patterns/index_patterns.ts
56.33% <0.00%> (ø)

...terns/index_patterns/redirect_no_index_pattern.tsx
6.66% <0.00%> (ø)

...lugins/data/server/saved_objects/index_patterns.ts
25.00% <0.00%> (ø)

... and 2293 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Issue writing A3

https://github.com/ZUGFeRD/ZUV/issues/11 in reality is a Mustangproject error

committed [7875841] as an interim fix, temporarily uploaded a compiled binary on http://mustangproject.org/deploy/mustang-1.5.2-SNAPSHOT.jar , the (new) next verapdf error is now Font names, names of colourants in Separation and DeviceN colour spaces, and structure type names, after expansion of character sequences escaped with	a NUMBER SIGN (23h), if any, shall be valid UTF-8 character sequences.


Point rtk query at actual back-end url when deployed

We probably want the rtk-query baseUrl to vary based on an environment variable. Something like:
baseUrl = process.env.NODE_ENV === "development" ? localhost:3001 : <actual back-end url>
NODE_ENV is managed by create-react-app and we can't override it so we might want to create a custom REACT_APP_* variable for this.
https://create-react-app.dev/docs/adding-custom-environment-variables/

@thiagobardini I think this is complete with your recent change :+1:


JSONRPCError: Can not authorize with current license

Just got my head set and trying the sdk getting this
JSONRPCError: Can not authorize with current license

Did you have a license ? could you please send me your emotiv id. I will help to check.


fix(material/core): remove label element from option group

Replaces the label element in mat-optgroup with a regular div, because label is primarily meant for labelling interactive elements.

No longer necessary after https://github.com/angular/components/pull/21858 was merged in.


[Draft] Opaque types

Resolves #243
Resolves #68

There are a few kinks to work out before we can stabilize it:

[ ] Dart API
[ ] Safety
[ ] Data structure

Dart API
An opaque pointer to e.g. [u8; 2000] would look like this:
class U82000 extends Opaque {
  U8200._(int ptr, int drop) : super(Pointer.fromAddress(ptr), Pointer.fromAddress(drop));

  void dispose(); // implemented by super, safe to call many times
}

I don't know if there's a lint that enforces calling dispose, but it would be really helpful for this situation.
Otherwise, I think this is an acceptable API to use. Then in the future when weak references are implemented, calls to dispose will become no-op and be deprecated.
Safety
The Rust side however has the issue of whether we should allow Rust functions to obtain a mutable reference to T:
impl<T> Opaque<T> {
  ..
  pub unsafe fn as_mut(&mut self) -> Option<&mut T>;
}

then in a function:
pub fn handle_opaque(mut arr: Opaque<[u8; 2000]>) -> Result<()> {
  unsafe {
    if let Some(arr) = arr.as_mut() {
      std::mem::drop_in_place(arr); // instant UB
    }
  }
  Ok(())
}

In line with other Rust API, since we do not have ownership of T we can't really hand out mutable references, but not having this functionality also precludes a lot of potential usage out of opaque pointers. I'm not quite sure how to proceed here.

don't know if there's a lint that enforces calling dispose, but it would be really helpful for this situation.

I guess no, or at least the lint will never be strong enough. We know in Flutter it is common to have a dispose, e.g. for a Stream, a ChangeNotifier, etc. There do exist lints to warn you when you never dispose them, but nobody can ensure we call them in the right place.

Then in the future when weak references are implemented, calls to dispose will become no-op and be deprecated.

I guess it will still be used and never deprecated? b/c from the comments of dart team https://github.com/dart-lang/language/issues/1847#issuecomment-1002860171, fully automatic memory management is strongly discouraged.
Even worse, by doing this way we can never tell Dart VM that this "int ptr" may occupy huge memory indeed. Then Dart VM will even do not bother to cause a GC and dispose objects since it wrongly thinks nobody is using huge memory.

extends Opaque

btw I guess dart:ffi has a class called Opaque so we may rename it, e.g. FrbOpaque

Question: When the dart object is dispose()ed, how can we ensure nobody will ever touch invalid memory?

The bridge may look like (very roughly):
fn convert_dart_wire_object_to_rust_object<T>(wire_ptr: *const isize) {
  // NOTE manuallydrop
  let dart_owned_obj: Box<RwLock<T>> = ManuallyDrop::new(Box::from_raw_ptr(wire_ptr));
  dart_owned_obj
}

Problem: Are we safe? Notice the convert_dart_wire_object_to_rust_object can be called concurrently by multiple calls! So we may be holding the raw pointer as a Box<RwLock<T>> concurrently.
What do you think?

However, the memory may still be used - for example, a long running Rust method may still be using this. This may be acceptable since it is the direct result of reference counting.

The only way I think this can be safe is that our hypothetical FrbOpaque is a newtype over Arc (then users can specify that they want an Opaque<RwLock<T>> instead). Maybe something like a model of shared ownership would help, if not reference counting?

let dart_owned_obj: Box<RwLock<T>> = ManuallyDrop::new(Box::from_raw_ptr(wire_ptr));

I think there may be some issue with that, I've been seeing it as early as trying to send a *mut [u8; _] only for rustc to tell me it is not Send. So yeah, I think Arc is the best compromise for now.

My attempt on data structures
reference: https://github.com/fzyzcjy/flutter_rust_bridge/issues/243
Dart side: int ptr.
Rust side: Box<Arc<RwLock<T>>>.
Rust usage:
fn convert_dart_wire_object_to_rust_object<T>(wire_ptr: *const isize) {
  // NOTE manuallydrop
  let dart_owned_obj: Box<Arc<RwLock<T>>> = ManuallyDrop::new(Box::from_raw_ptr(wire_ptr));
  // NOTE we clone the Arc and return that clone
  dart_owned_obj.clone()
}

Question: Is it safe?
When raw pointer is passed from dart to multiple rust functions concurrently, multiple rust functions will all rehydrate the same Arc object from the raw pointer, so have problems?

The only way I think this can be safe is that our hypothetical FrbOpaque is a newtype over Arc (then users can specify that they want an Opaque<RwLock> instead). Maybe something like a model of shared ownership would help, if not reference counting?

Same idea here ;) (See my #243 root post as well)

let dart_owned_obj: Box<RwLock> = ManuallyDrop::new(Box::from_raw_ptr(wire_ptr));

plz ignore it, I realize it has problems

When raw pointer is passed from dart to multiple rust functions concurrently, multiple rust functions will all rehydrate the same Arc object from the raw pointer, so have problems?

Haven't tested yet, but I think it might go something like this?
// begin, refcount = 1 as owned by Dart
let obj1: Arc = convert_dart_wire_object_to_rust_object(wire_ptr); // clone, refcount = 2
let obj2: Arc = convert_dart_wire_object_to_rust_object(wire_ptr); // clone, refcount = 3
normal_rust_function_that_accepts_an_object(obj1); // does refcount decrease here?
normal_rust_function_that_accepts_an_object(obj2); // or here?
drop(obj1); // it must not, because refcount must be 3 here
drop(obj2); // similarly, refcount = 2
// refcount = 1, safe

The wildcard is the two normal fns in the middle, if they accept a &mut Arc then they can call Arc::decrement_strong_count and invalidate this assumption (which is unsafe anyways). As I was writing this, I realized you can turn an Arc into a Weak reference, which would be quite hard but not impossible to force-decrement the refcount, so would this make FrbObject a Weak instead?

Arc::decrement_strong_count

maybe we can assume users never do that? that is a unsafe function.

The wildcard is the two normal fns in the middle, if they accept a &mut Arc then they

oh, moreover, I realize they should accept Arc not &mut Arc since indeed those functions can run for a long time.
So it is like:
let obj1: Arc = convert_dart_wire_object_to_rust_object(wire_ptr);
let obj2: Arc = convert_dart_wire_object_to_rust_object(wire_ptr);
normal_rust_function_that_accepts_an_object(obj1);
normal_rust_function_that_accepts_an_object(obj2);

fn normal_rust_function_that_accepts_an_object(o: Arc) {...}

Second attempt on data structures
When reading https://doc.rust-lang.org/std/sync/struct.Arc.html I realize they have:

into_raw
as_ptr
from_raw

can we exploit them, then we do not need Box<Arc<...>> but only need Arc<...>

accept Arc not &mut Arc

Oh, that hypothetical is fine because if it takes ownership then you cannot drop them anyway. But the same worry presents if that inner function somehow decreases the refcount more than once.

we can assume users never do that

Maybe not, but there is still an off-chance one of that function's dependencies will, still that is probably a 99.9% hypothetical that might turn out impossible to debug once present in production code.

post above edited, with a full example

related https://stackoverflow.com/questions/70587003/safely-call-arcfrom-raw-multiple-times-for-a-raw-pointer-in-other-words-ho

Maybe not, but there is still an off-chance one of that function's dependencies will, still that is probably a 99.9% hypothetical that might turn out impossible to debug once present in production code.

Well I agree with you: If the refcount decreases to zero accidentally, the raw pointer immediately become a rubbish, and we have NO WAY to detect it!

Then, what about the object pool https://github.com/fzyzcjy/flutter_rust_bridge/issues/241 approach? If anyway we are using dispose manually, the object pool does not cause any extra complexity. Moreover, it is much, much safer, since we almost never touch a unsafe operation with object pool. We just use indices (like pointers) to index into the pool (a HashMap, for example). Then, invalid indices (read: invalid pointers in this PR) just causes the HashMap to panic, instead of terrible UB as in this PR.
I have already had a object pool implemented in my production environment so I can open source it (and then we make some improvements to it).

If we can do automatic memory management (i.e. no manual dispose) then this pr is wonderful; but since the Dart team says "strongly discourage" such, we should manually call dispose forever,

we should manually call dispose forever

I think that might be the case, seeing how dispose is already part of Flutter culture.

https://stackoverflow.com/questions/70587003/safely-call-arcfrom-raw-multiple-times-for-a-raw-pointer-in-other-words-how/70587029?noredirect=1#comment124780745_70587029 gives an improvement:
let x_ptr = Arc::into_raw(Arc::new(some_big_object));

for i in 0..10 {
    thread::spawn(|| {
        unsafe {
            Arc::increment_strong_count(x_ptr);
            let x_recovered = Arc::from_raw(x_ptr);
            make_use_of_it(x_recovered);
        }
    });
}

unsafe {
    Arc::decrement_strong_count(x_ptr);
    // Could be also
    // drop(Arc::from_raw(x_ptr));
}

Changes up until last commit:

Opaque is now FrbOpaque
opaque::Opaque is now a transparent newtype over Option<Arc<T>>
opaque::Opaque::<T>::as_ref(&self) -> Option<&T> for immutable references
Extensions for Opaque<RwLock<T>>

Excellent! A few things to check before merging

[ ] more simple tests (since this can cause very hard-to-debug bugs)
[ ] some concurrent tests
[ ] discuss the question below

Bug: What if a user calls ffi and call dispose before the rust code executes and increases the reference counting? Currently it will be undefined behavior and cause all troubles!
In other words:
// dart
var object = ... an opaque pointer ...;
unawaited(my_rust_function(object.ptr)); // (1)
object.dispose(); // (2)

// rust
fn my_rust_function(object_ptr: isize) {
  let object: Arc<...> = convert_to_object(object_ptr);
  ... use it ...
}
fn convert_to_object(object_ptr: isize) -> Arc<...> {
  Arc::increment_strong_count(object_ptr); // (3)
  return Arc::from_raw(object_ptr); // (4)
}

It can be the case that, the code is executed in the sequence of (1) - (2) - (3) - (4).
Then awful things happen: At (1) things are still good. After executing (2), the object.dispose will decrease refcount of the Arc. So it has refcount of zero, and the data of Arc is dropped! Then (3) and (4) happens - we are trying to access an invalid (deallocated) pointer. That is TERRIBLE and we are touching invalid memory and can cause UB.

posts above are edited.

What if a user calls ffi and call dispose before the rust code executes and increases the reference counting?

Unfortunately this is a real bug right now, I'll get to fixing this.

The bug turned out to be similar, but not exactly the same as you described. Given the following situation:
var opaque = await createOpaque();
var future = handleOpaque(val: opaque); // Future<Opaque>
opaque.dispose() // refcount = 1
opaque = await future; // refcount = 0!!
expect(opaque.isStale(), false); // passes
opaque.dispose(); // segfault

It was essentially a double free, but otherwise it seemed to be able to increase the refcount before Dart can dispose it.
There is no 100% guarantee that this is the case, but I don't not how to write tests that can challenge that assumption.

Well can you remove
opaque = await future;

since by doing this, you are reassigning the variable and it is harder for people to understand what is happening
maybe
var opaque_a = await createOpaque();
var future = handleOpaque(val: opaque_a); // Future<Opaque>
opaque_a.dispose() // refcount = 1
var opaque_b = await future; // refcount = 0!!
expect(opaque_b.isStale(), false); // passes
opaque_b.dispose(); // segfault

Fair point, I've edited the example to clarify that. This is also part of the test suite btw.

There is no 100% guarantee that this is the case,

May depend on your rust code and the auto generated code as well, since you are indeed disposing the returned object instead of considering the input object.
And, my proposed bug is not that easy to reproduce - it does not happen frequently, since with high probability it is (1)-(3)-(4)-(2) instead of (1)-(2)-(3)-(4). But, since we are developing a library, and concurrency bugs are VERY hard to debug for the users, we should tackle it.

Maybe we should separate the example library from actual testing, it is getting kind of cluttered.

@Desdaemon Agree!

Not a solution, but since this is a data race we're trying to solve, the provably correct solution is some kind of synchronization between Dart and Rust, though this does add a fair amount of overhead. I'm doing some testing in GDB and already saw some fascinating results, but it will take me a while to have something substantial.

How can a Weak still be happily held even when Arc is already disposed an no memory exist?

From std documentation:

Since a Weak reference does not count towards ownership, it will not prevent the value stored in the allocation from being dropped, and Weak itself makes no guarantees about the value still being present. Thus it may return None when upgraded. Note however that a Weak reference does prevent the allocation itself (the backing store) from being deallocated.

This is one possible solution, but it's not clear how we can use it yet.

This is one possible solution, but it's not clear how we can use it yet.

My question is: In my naive guess, when Arc has ref counting down to zero, it frees the data, and there is nothing left in memory (is it correct?). However, some rust code may still be holding a Weak to that location in memory. Then, the Weak is literally holding a pointer to a invalid (i.e. freed) memory. Then how can Weak accurately detect "this is invalid memory"?
I guess my thoughts above must have something wrong. Have not dig very deep into src code yet.

What I ended up with is using a parking_lot::Mutex<()> to wrap the critical sections. I think this is the easiest solution for now, unless anyone has an alternative.

What I ended up with is using a parking_lot::Mutex<()> to wrap the critical sections. I think this is the easiest solution for now, unless anyone has an alternative.

I guess you mean a global lock that is shared among everything?
Sounds like a feasible solution. The performance may be reduced, but hope few people is having this as a bottleneck.

Wait a bit... I get confused. How is this bug https://github.com/fzyzcjy/flutter_rust_bridge/pull/293#issuecomment-1006191364 solved with locking? Cannot see how locking prevents this so could you plz give a bit more hints?

On second thought, that doesn't really solve it. I'll get back to this once I have something better.

@Desdaemon Take you time and thanks for your contribution!

Can't fully comment on this, but would everyone agree on this solution? https://docs.rs/semaphore/0.4.0/semaphore/

Some remarks on Weak and Arc:

Note however that a Weak reference does prevent the allocation itself (the backing store) from being deallocated. (by std::rc::Weak, but arc should be same)

Thus this does not inspire anything to us. Originally I wrongly thought Weak has some magic to detect dangling pointer.

If this PR is too challenging to implement, we may consider the "pool" approach, which is even equivalent to this PR but seems to be safer (less strange corner cases).
My current implementation of pool is: https://gist.github.com/fzyzcjy/ff84c4a4cc37950275505a5fdd99039c (directly copied from my private codebase). However, for more general and easy-to-use pools, we may consider the following (slight modification):
There is one giant pool that holds all objects. You can get objects, put objects, etc, into it, and it will use a integer (the "handle") to represent the object. The handle is analogy to the pointer.
The "subpool" in my current implementation can also be taken into account. I use subpools to make a "scoped" thing. In other words, I can call subpool.dispose() and everything in it will be disposed - then if you forget to dispose something, it can safe your life.

The solution was quite simple in retrospect: if dispose is called non-deterministically, then the cloning of Arc must occur as part invoking the wire functions. This ensures that Rust already holds the allocation and a call to dispose won't drop the memory. Also new is the ability to make opaque pointers to trait objects.
I'm thinking about tapping into the Dart C API for attaching a finalizer instead of forcing users to manually call dispose, but it seems like there are problems with that approach too. Any thoughts?

Also new is the ability to make opaque pointers to trait objects (!)

Interesting! Do not forget to mention it in changelog/doc later ;)

I'm thinking about tapping into the Dart C API for attaching a finalizer instead of forcing users to manually call dispose, but it seems like there are problems with that approach too. Any thoughts?

Well recall that "strongly discourage" comment from Dart official team. IMHO applies to it too. What's worse, with dart GC finalizer, we even do not tell Dart VM how huge the external object is. So Dart GC can happily think that there is no memory pressure, while Rust code allocates, say, 10GB memory and really need a GC.
About the solving of the problem: Let me think twice.

The solution was quite simple in retrospect: if dispose is called non-deterministically, then the cloning of Arc must occur as part invoking the wire functions. This ensures that Rust already holds the allocation and a call to dispose won't drop the memory.

Well may still need some care: every single await in Dart code is indeed some asynchronous gap. In other words, e.g.
func1();
func2();
await func3();
func4();

in the example above, we are sure no other code can execute between func1() and func2(). But, a ton of code can execute between func3() and func4().
Thus, we must be very careful about not introducing any async gap. Even one async gap will break the safety guarantee.
What's worse, frb allows users to customize the executors and handlers. For example, you can customize and wrap dart side's executeAsync, as well as Rust side. So we must be very sure this cannot be violated even with customizations.

This is amazing progress 🚀!
Regarding the finalizers: I think the takeaway right now is clearly that Dart cannot and should not manage the native memory. Dart's GC only kicks in when new objects are allocated on the Dart heap, not when the the memory pressure grows from allocating objects on the native heap. Once https://github.com/dart-lang/language/issues/1847 has landed, it should be possible to automatically call .dispose(), but it won't be a full replacement for calling it manually.
Maybe going forward there is a way to implement some additional GC on the Dart side that tracks the allocation on the native heap with weak references, but that's something that I haven't fully investigated yet.

👀

With the release of flutter 3 and dart 2.17 the NativeFinalizer is available as per https://medium.com/dartlang/dart-2-17-b216bfc80c5d

🙏🏻

Just tested it, and currently, DartSafe is not being included in bridge_generated.rs. Is it something to be done or will the users have to import frb_rust to use Opaque types?

I had to add
import 'package:meta/meta.dart';

to the generated code otherwise the @sealed wouldn't work.

Related:

#775

Btw, given that we already have opaque types implemented, shall we close this PR?


Edit .readme file

add steps required to set the repo in localhost

I want to work on it. Can you please assign it to me?
And please add gssoc-ext label

I want to work on it. Can you please assign it to me? And please add gssoc-ext label

Here you go

Here you go

Solved it in #44


How to add event every year like birthday?

Is there any way to add event for every year in short like birthdays and national holidays?

You just need to come up with the cell layout you want and have an underlying model that contains the desired event dates. Then, in the cellForItemAt method, just customize the cell if the date matches the one of your event.

NSMySelf is correct. If you follow the video tutorial HERE, it will show you an example of how events are handled.

Hi - a lot of the links to the video tutorials are dead (like this one ^^). Is there somewhere I could see the older videos?


UCX 1.15 upgrade

Closes #8137
This updates JUCX to 1.15, updating the example and CI dockerfiles.
The Dockerfile.multi is using aarch64 but the rest are using x86_64.
I will test with cuda12 locally.

build

build


Custom hooks with RPIT no longer compile in Rust 2024

Problem
#[hook] generates code that no longer compiles under Lifetime Capture Rules 2024 (RFC 3498).
Steps To Reproduce
# Cargo.toml
[package]
name = "foo"
edition = "2024"

[dependencies]
yew = { version = "0.21.0", features = ["csr"] }

// src/lib.rs
use yew::prelude::*;

#[hook]
pub fn use_foo() -> impl Sized {
    42
}

$ cargo +nightly-2024-12-12 check
error: lifetime may not live long enough
 --> src/lib.rs:3:1
  |
3 | #[hook]
  | -^^^^^-
  | |     |
  | |     return type of closure `impl Sized` contains a lifetime `'2`
  | let's call the lifetime of this reference `'1`
  | returning this value requires that `'1` must outlive `'2`
  |
  = note: this error originates in the attribute macro `hook` (in Nightly builds, run with -Z macro-backtrace for more info)

Expected behavior
Successful compilation as in Rust 2021
Screenshots
If applicable, add screenshots to help explain your problem.
Environment:

Yew version: 0.21.0
Rust version: nightly-2024-12-12
Target, if relevant: N/A
Build tool, if relevant: N/A
OS, if relevant: N/A
Browser and version, if relevant: N/A

Questionnaire

[ ] I'm interested in fixing this myself but don't know where to start
[ ] I would like to fix and I have a solution
[x] I don't have time to fix this right now, but maybe later

A possible work-around is to manually name all type and lifetime parameters in scope by + use<...>. Example:
#[hook]
// pub fn use_foo() -> impl Sized {
pub fn use_foo() -> impl Sized + use<> {
    42
}

#[hook]
// pub fn use_bar(x: &impl Clone) -> impl Clone {
pub fn use_bar<'a, X: Clone>(x: &'a X) -> impl Clone + use<'a, X> {
    42
}


Out of date eslint plugin causes Parsing error: Cannot read property 'map' of undefined

Describe the bug
I've upgraded my project's TypeScript to version 4. It seems this creates an issue because react-scripts @typescript-eslint/parser is out of date. Could we bump these packages?
Currently I am doing this in my package.json:
  "resolutions": {
    "**/@typescript-eslint/parser": "4.0.1",
    "**/@typescript-eslint/eslint-plugin": "4.0.1"
  },

as a workaround.

The resolutions is for Yarn, for NPM uses https://docs.npmjs.com/cli/shrinkwrap
Anway look like CRA is using version 2 of the plugin.. that means 2 major versions of the plugin and 1 major version of eslint we're behind

Yeah it is quite a way behind. What would be the negatives of bumping it?

https://github.com/facebook/create-react-app/pull/9590 Submitted a PR. Wonder when all the builds will start working :S

I've just realized the PR is tagged with v4 :( is there any chance we can still have this in v3?


Styles/appearance different when deploy to Azure

Hi,
After deploy an Hugo app to Azure (Azure Static Web Site), I found different behavior about Styles.
Attached go two image: Localhost and Azure (production) Styles to know exactly what is happening.
Any idea about how I can solve it?

Thanks a lot

Hi again,
Sorry, my problem I need to use "-b" arguments to deploy correctly.
Best regards
Juanlu


add timeout to node stats

Added fix for metric gabs on high load on single node.

With the elasticsearch version 7 and Opensearch version 1 there is a new timeout parameter implement.
Elasticsearch 6 ignores this timeout parameter and this change have no impact.
Tested with elasticsearch 6.8 and opensearch 1.2
Elasticsearch
https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html#cluster-nodes-stats-api-query-params
Opensearch
https://opendistro.github.io/for-elasticsearch-docs/docs/elasticsearch/rest-api-reference/#nodesinfo
resolve #508

this fix this problem in newer elasticsearch version
https://github.com/prometheus-community/elasticsearch_exporter/issues/330


feat!: initial work for version 2.0.0

Added typescript
Added OpenAPI code generation
Added documentation auto-generation via CI

TODO

Add robotoff API (https://github.com/openfoodfacts/robotoff/blob/master/doc/references/api.yml)

Dependencies

[x] Blocked on https://github.com/openfoodfacts/openfoodfacts-nodejs/pull/263
[x] Blocked on https://github.com/openfoodfacts/openfoodfacts-server/pull/8450

I'm having a lot of problems integrating testing with TypeScript, so I guess I'll postpone that to a next PR.


fix: audit seedlot working great

Description
Closes #927
Changelog
New

Migration SQL script fixing wrong columns

Changed

None

Removed

None

How was this tested?

[ ] 🧠 Not needed
[x] 👀 Eyeball
[ ] 🤖 Added tests

What gif/image best describes this PR or how it makes you feel?

Thanks for the PR!
Any successful deployments (not always required) will be available below.
Frontend
Backend
Oracle-API
Once merged, code will be promoted and handed off to following workflow run.
Main Merge Workflow

LGTM! Only one typo to fix! Question: is it ok to leave the fields regarding lat/long/elev out of the audit table?

Great catch @mgaseta adding them now. Thanks 🙏🏻 🚀


The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #238      +/-   ##
============================================
+ Coverage     69.60%   69.65%   +0.04%
- Complexity      765      766       +1
============================================
  Files           124      124
  Lines          2576     2580       +4
  Branches        326      326
============================================
+ Hits           1793     1797       +4
  Misses          681      681
  Partials        102      102

Impacted Files
Coverage Δ

src/main/java/fridgy/model/util/QuantityCalc.java
76.31% <0.00%> (+0.64%)
:arrow_up:

...rc/main/java/fridgy/model/ingredient/Quantity.java
87.50% <0.00%> (+1.78%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 875b9a7...961d0cb. Read the comment docs.

Also I think there's no mention of what red tags mean in recipe. Should probably that missing ingredients are highlighted as red.


Create newApp.eve, fix #489

Just a blank app file, that fixes #489 by having a file already be there to edit.

Realized it did not adequately solve the problem.

Im new to github and software dev so Im learning the ropes. My apologies I
realized later it didnt work too good.
Ive been learning to make a simple workout log app. Not very much success
yet. I have to keep reading the handbook and quickstart. I really like this
language and I have never made a web app before nor have very much
understanding of software development.
Would be nice to meet on freenode #evecoders is open. :)


Replace H2 with Docker based testcontainers-postgres for testing

Instead of using an H2 in-memory database for testing, a real PostgreSQL database is launched on-demand as a Docker container.
Description
This PR allows integration tests to be executed against a real PostgreSQL database, thereby giving higher confidence with regards to the correctness of the code as well as ensuring strong parity of development, testing and production environments. This is done by using the testcontainers-java library, a widely used (in industry and in the OSS community) Java library for instrumenting Docker containers from Java code for test purposes.
While launching a container instead of using an in-memory data has a certain overhead, one might argue the time and resource overhead is negligible when compared with the gains in confidence for integration correctness. Also, this works as a drop-in replacement by using the special Testcontainers JDBC-Driver-Proxy, while at the same time mounting the data partition of the PostgreSQL container into host memory for better performance.
In addition, this allows to drop the need for maintaining a distinct set of flyway script for H2.
Disclaimer
I am one of the core maintainers of the Testcontainers project. If you have any doubts or questions, I'd be happy to discuss them and to help in supporting the incorporation of Testcontainers for other integration testing use cases as well.

I am wondering why the circle-ci build failed since I tested everything with a local Maven build first. However, I will try again after a mvn clean and investigate further.

@kiview because the build runs inside a container already and can't connect to Docker? :)
At least I don't see the CircleCI config changed, and I guess it needs to be:
https://www.testcontainers.org/supported_docker_environment/continuous_integration/circle_ci/

Just the summarize the current status:
As @bsideup noted, the CircleCI config needed to be adjusted to use a VM instead of a Docker executor, so that Testcontainers is able to spin-up Docker containers for the integration tests.
However, there seems to be no simple way to specify the JDK version to be used in the VM (as opposed to the Docker executor approach) and it defaults to using JDK8, which is the reason why it is failing at the moment.

Closing due to inactivity - please re-open when you got a running version.


Run issues after last merge (puzzle init)

After #70 merge, base puzzle cmd is not working for me. Commit 483557d9cb506cd72ff7eba1605c7d479b2b3c03 works.

Missing requirement for sqlalchemy

[VZMBP puzzle]$ puzzle
Traceback (most recent call last):
  File "/usr/local/bin/puzzle", line 9, in <module>
    load_entry_point('puzzle==0.0.1', 'console_scripts', 'puzzle')()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 558, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2682, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2355, in load
    return self.resolve()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2361, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/vaszac/proj/puzzle/puzzle/__main__.py", line 14, in <module>
    from .cli import cli
  File "/Users/vaszac/proj/puzzle/puzzle/cli.py", line 9, in <module>
    from .factory import create_app
  File "/Users/vaszac/proj/puzzle/puzzle/factory.py", line 5, in <module>
    from .settings import BaseConfig
  File "/Users/vaszac/proj/puzzle/puzzle/settings.py", line 5, in <module>
    from .plugins import VcfPlugin
  File "/Users/vaszac/proj/puzzle/puzzle/plugins/__init__.py", line 8, in <module>
    from .sql import Store as SqlStore
  File "/Users/vaszac/proj/puzzle/puzzle/plugins/sql/__init__.py", line 2, in <module>
    from .store import Store
  File "/Users/vaszac/proj/puzzle/puzzle/plugins/sql/store.py", line 9, in <module>
    from sqlalchemy import create_engine
ImportError: No module named sqlalchemy

After installing sqlalchemy separately, new error (on new clone now): Can't find gemini plugin

[VZMBP puzzle-dev]$ puzzle
Traceback (most recent call last):
  File "/usr/local/bin/puzzle", line 9, in <module>
    load_entry_point('puzzle==0.0.1', 'console_scripts', 'puzzle')()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 558, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2682, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2355, in load
    return self.resolve()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2361, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/vaszac/proj/puzzle-dev/puzzle/__main__.py", line 14, in <module>
    from .cli import cli
  File "/Users/vaszac/proj/puzzle-dev/puzzle/cli.py", line 9, in <module>
    from .factory import create_app
  File "/Users/vaszac/proj/puzzle-dev/puzzle/factory.py", line 5, in <module>
    from .settings import BaseConfig
  File "/Users/vaszac/proj/puzzle-dev/puzzle/settings.py", line 5, in <module>
    from .plugins import VcfPlugin
  File "/Users/vaszac/proj/puzzle-dev/puzzle/plugins/__init__.py", line 8, in <module>
    from .sql import Store as SqlStore
  File "/Users/vaszac/proj/puzzle-dev/puzzle/plugins/sql/__init__.py", line 2, in <module>
    from .store import Store
  File "/Users/vaszac/proj/puzzle-dev/puzzle/plugins/sql/store.py", line 17, in <module>
    from puzzle.plugins import VcfPlugin, GeminiPlugin, Plugin
ImportError: cannot import name GeminiPlugin

Thank you Vasilios

Fixed in #77