Blocking issues in .NET

In order to write software that targets both .NET 4.6 and .NET Core, we need .NET 4.6 to fix several oversights.
Visual studio, ASP.NET, deploy tooling, test tooling, and even the .NET runtime make some bad assumptions:

Every .dll is a .NET dll.
Managed .dlls do not depend on native .dlls
No project ever needs to target or deploy to more than one architecture/platform.
Nobody needs to adjust native or managed dll search paths at runtime.

Specifically, what low-level APIs do we need before we can fix dependency loading ourselves?

The ability to capture and override the load path of every
.NET assembly, not just those which are missing (which AssemblyResolve
provides). This will allow us to avoid BadImageFormatExceptions and
locate compatible binaries instead to bring into the default load
context.

The ability to either

A) Represent all versions of unmanaged and non-Any CPU dependencies
within the metadata of a managed assembly, so that tooling and hosts can correctly
deploy and shadow-copy them.
B) Use a standard assembly attribute to define an entry-point which
will called for every movement (such as test runners, deployment, or
ASP.NET dynamic compilation), which can be responsible for the
process.
C) Expose a comprehensive list of still-accessible locations where
we left the original binaries behind. (In many contexts, neither
Location or Codebase are sufficient for discovery, such as for
shadow-copying or nuget references). This does not handle some
scenarios, like deployment.

We need pervasive architecture awareness, or the ability to create it.

A (respected) way to tell hosts (like ASP.NET/IIS) that the world
will end if they use more than 1 AppDomain per process. Many native
interop scenarios can't handle AppDomains, period.

A way to do #1 in the context of an ASP.NET app . ASP.NET currently
loads everything in the /bin folder, then calls PreAppStart on each
one. We can't assume everything in /bin root will load successfully,
since we still have tooling that is impeded when it comes to targets
and managed/native correlation. Hacking this with <remove
assembly="*"/> isn't well-received; a fully qualified assembly
reference to preload would be better.

The ability to control the assembly probing path at runtime - so
that we can support architecture-specific bin subfolders - within the
default load context. We should even be able to exclude
ApplicationBase, so that we can cleanly hand control over to an
external assembly for dependency resolution. This is somewhat a
duplicate of #1, but with better performance for simple scenarios.

Much of the ecosystem (read: nuget packages) around .NET is reliant
upon Windows APIs. This may may come in the form of System.Drawing,
System.Media, or P/Invoke calls, but it is prevalent. Moving to .NET
Core and vNext will entail the creation or porting of dozens of native
libraries with corresponding interop layers. The current interop pain
is too great for most OSS maintainers to overcome simply for the sake
of .NET Core compatibility, so if we're to have to the future we all
want (.NET Core), we need to fix it.

I'm not sure if the feature will ever make it into .NET but the CoreCLR has a feature call an AssemblyLoadContext, that gives control over all loading (native and managed). This is what we hook in the dnx to get control over what to load at runtime based on the current context.
The dnx has knowledge of the NuGet closure (via the lock file) so it can reason about probing paths and setup the runtime in appropriate way so that DllImport just works (in most cases).
The dnx also works on full CLR and it's more challenging there because of what you mention. The only ways around it would be to use SetDllDirectory or by changing the process PATH. Another possible solution would be to wait until an assembly is loaded and they force load all native dependencies (based on whatever NuGet package semantics we come up with).

I'll have to give AssemblyLoadContext a try. I really hope it makes it into .NET; we need it badly.
According to docs, SetDllDirectory still doesn't override the app directory... Messing up PATH seems like it could have side effects, but options are quite limited.
One problem with waiting until a managed dependency to load is that it breaks C++/CLI, which embeds 1 and only 1 architecture variant inside the managed DLL. Or you mean selecting the right bitness of managed dlls first, then once those are loaded, inspect for native dependencies? ECMA CLI defines an assembly load entry point; the main use for this seems to be to handle native resource init/shutdown. We would break that use case, but it's admittedly rare.
(2) I'm not sure whether a NuGet closure file or new metadata is best for dealing with tooling, like test runners. I run my tests in both 32 and 64-bit mode on the same machine, from the same filesystem. @bradwilson, do you have an opinion on this?
Did I overlook something easy for (3)?

One problem with waiting until a managed dependency to load is that it breaks C++/CLI, which embeds 1 and only 1 architecture variant inside the managed DLL. Or you mean selecting the right bitness of managed dlls first, then once those are loaded, inspect for native dependencies? ECMA CLI defines an assembly load entry point; the main use for this seems to be to handle native resource init/shutdown. We would break that use case, but it's admittedly rare.

That's ok because by the time you're running/loading, you should have all of the context required to pick the right binary.

(2) I'm not sure whether a NuGet closure file or new metadata is best for dealing with tooling, like test runners. I run my tests in both 32 and 64-bit mode on the same machine, from the same filesystem.

I don't see why test runners or tooling are special here. Can you enlighten me?

Test runners are no different than any other piece of runtime code.

Any code that is responsible for executing a .NET assembly - but which needs to permit all involved DLL files to be overwritten at any time - has specific responsibilities.  One could argue that this is the responsibility of the build tools and/or deploy tools, but I would disagree (and good luck convincing the VS team of that).
Some deploy tools try to work around this by taking a set of actions when encountering a locked file:

If the temp folder exists in the same filesystem as the deploy folder, they move the in-use file to a temp folder that is marked for deletion at system boot.
If not, they move it to a custom folder and use a custom cleanup approach.
... and many other techniques; Garrett Serack has a pretty exhaustive list.

However, Visual Studio integrates with arbitrary build tools, and I don't think it is even possible to get them all to implement the kind of hacks required to work around this from the file writer's side.
From the execution side, in any test runner scenario, we want to prevent the files from being locked. To do that, we have to (shadow) copy all the native dlls, managed dlls, managed resources, and native resources - to another location before running them. Are you saying that the .NET framework should be responsible for this process, as part of the AppDomain API?

I don't think it's the responsibility of the VS team. I think it's the responsibility of the .NET team.
The Shadow Copy feature belongs to them. That it is half-implemented for native DLL users is really their issue. Pushing it to all the test runners just begs for a half dozen incompatible (and maybe incorrect) implementations.

@davidfowl, has anything changed with .NET 4.6 that would unblock sub-issue 1 or 2 above?


BLE.Client.iOS crashed on my iPhone 10

To help us fix your issue, please provide the information in the below template. If something causes a crash, provide as much information as you can gather.
Just imagine: we do not know what you are doing!

Note: There is often little we can do without a minimal reproducible sample of the issue, so please provide that in a standalone git repository and link it here.

Steps to reproduce

Download latest PLUG.BLE FROM git
Latest VISUAL STUDIO 2019 Pro on Windows 10 and mac (latest OS)

VS > solution > right-click BLE.Client.iOS > Debug > Start

Expected behavior
Tell us what should happen  SHOULD RUN VIRGIN EXAMPLE
Actual behavior
Tell us what happens instead   BRIEFLY SHOWED SPLASH, THEN BACK TO SPRINGBOARD
Crashlog
[HotReload] (2020-09-01 11:49:14.3): WARN: (BLE.Client.iOS) Hot Reload disabled due to invalid linker settings
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Mono.Security.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.Core.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.Numerics.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.Drawing.Common.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Xamarin.iOS.dll [External]
Thread started:  #2
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.Xml.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Xamarin.Forms.Platform.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Xamarin.Forms.Core.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Microsoft.CSharp.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/MvvmCross.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.Net.Http.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.ServiceModel.Internals.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/System.Runtime.Serialization.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Xamarin.Forms.Platform.iOS.dll
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/MvvmCross.Forms.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/BTProgressHUD.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Acr.UserDialogs.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Plugin.BLE.Abstractions.dll
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Plugin.BLE.dll
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Plugin.Permissions.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Xamarin.Forms.Xaml.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Plugin.Settings.Abstractions.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/Plugin.Settings.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/BLE.Client.dll
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/MvvmCross.Plugins.BLE.iOS.dll
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/BLEClientiOS.exe
2020-09-01 11:49:15.603 BLEClientiOS[3358:878268] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Primary start
2020-09-01 11:49:15.603 BLEClientiOS[3358:878268] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: FirstChance start
2020-09-01 11:49:15.603 BLEClientiOS[3358:878268] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: PlatformServices start
Loaded assembly: /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/OpenTK-1.0.dll [External]
2020-09-01 11:49:15.690 BLEClientiOS[3358:878268] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: MvvmCross settings start
2020-09-01 11:49:15.690 BLEClientiOS[3358:878268] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Singleton Cache start
2020-09-01 11:49:15.690 BLEClientiOS[3358:878268] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: ViewDispatcher start
Thread started:  #3
Thread started:  #4
Thread started:  #5
Thread started:  #6
2020-09-01 11:49:15.816 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Bootstrap actions
2020-09-01 11:49:15.822 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: StringToTypeParser start
2020-09-01 11:49:15.826 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: CommandHelper start
2020-09-01 11:49:15.827 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: PluginManagerFramework start
Thread started:  #7
Thread started:  #8
Thread started:  #9
Thread started:  #10
2020-09-01 11:49:15.973 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Logging.MvxLog) Loading bluetooth low energy plugin
2020-09-01 11:49:15.976 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Create App
2020-09-01 11:49:15.977 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: NavigationService
2020-09-01 11:49:15.979 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Load navigation routes
2020-09-01 11:49:15.980 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: App start
2020-09-01 11:49:15.980 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Application Initialize - On background thread
2020-09-01 11:49:15.992 BLEClientiOS[3358:878320] 2020-09-01 11:49:15 [TRACE] (MvvmCross.Core.MvxSetup) Setup: ViewModelTypeFinder start
2020-09-01 11:49:16.002 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: ViewsContainer start
2020-09-01 11:49:16.003 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Views start
2020-09-01 11:49:16.011 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: CommandCollectionBuilder start
2020-09-01 11:49:16.011 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: NavigationSerializer start
2020-09-01 11:49:16.011 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: InpcInterception start
2020-09-01 11:49:16.011 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: InpcInterception start
2020-09-01 11:49:16.012 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: LastChance start
2020-09-01 11:49:16.034 BLEClientiOS[3358:878320] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Core.MvxSetup) Setup: Secondary end
2020-09-01 11:49:16.036 BLEClientiOS[3358:878268] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Logging.MvxLog) AppStart: Application Startup - On UI thread
2020-09-01 11:49:16.052 BLEClientiOS[3358:878268] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Logging.MvxLog) iOSNavigation
2020-09-01 11:49:16.056 BLEClientiOS[3358:878268] 2020-09-01 11:49:16 [TRACE] (MvxForms) Showing of native host View in Forms is not supported.
2020-09-01 11:49:16.209 BLEClientiOS[3358:878268] 2020-09-01 11:49:16 [TRACE] (MvvmCross.Logging.MvxLog) App Start
=================================================================
Native Crash Reporting
Got a abrt while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
Native stacktrace:
0x102b149a0 - /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/BLEClientiOS :
0x102b0b63c - /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/BLEClientiOS :
0x102b13eec - /private/var/containers/Bundle/Application/2B4D0762-477A-49B4-9CA1-8580C6A8DCD1/BLEClientiOS.app/BLEClientiOS :
0x189072894 - /usr/lib/system/libsystem_platform.dylib : <redacted>
0x189161e58 - /usr/lib/system/libsystem_kernel.dylib : <redacted>
0x189161e88 - /usr/lib/system/libsystem_kernel.dylib : <redacted>
0x18f5cec8c - /System/Library/PrivateFrameworks/TCC.framework/

TCC :
0x18f5cebcc - /System/Library/PrivateFrameworks/TCC.framework/TCC :
0x18f5d2b00 - /System/Library/PrivateFrameworks/TCC.framework/TCC :
0x188f3cb1c - /usr/lib/system/libxpc.dylib :
0x188f313e8 - /usr/lib/system/libxpc.dylib :
0x18902f5a4 - /usr/lib/system/libdispatch.dylib :
0x1890060f8 - /usr/lib/system/libdispatch.dylib :
0x1890161c0 - /usr/lib/system/libdispatch.dylib :
0x189080bac - /usr/lib/system/libsystem_pthread.dylib : _pthread_wqthread
0x189083740 - /usr/lib/system/libsystem_pthread.dylib : start_wqthread
=================================================================
Basic Fault Address Reporting
Memory around native instruction pointer (0x18915d7f4):
0x18915d7e4  c0 03 5f d6 c0 03 5f d6 30 41 80 d2 01 10 00 d4  ......0A......
0x18915d7f4  e3 00 00 54 fd 7b bf a9 fd 03 00 91 b4 80 ff 97  ...T.{..........
0x18915d804  bf 03 00 91 fd 7b c1 a8 c0 03 5f d6 c0 03 5f d6  .....{...._...
_.
0x18915d814  d0 03 80 d2 01 10 00 d4 e3 00 00 54 fd 7b bf a9  ...........T.{..
The app has been terminated.
The app has been terminated.
If something causes an exception paste full stack trace + Exception here
Configuration
Version of the Plugin: e.g. 1.0.0 / 1.0.1-alpha3      Plugin.BLE 2.1.1
Platform: e.g. iOS 10.1 / Android 4.4 / ... (including version!!! e.g. Android 5.1 / i0S 10)  IOS 13
Device: e.g. HTC Sensation /i Phone 7 ...   IPHONE 10

Hi @douglaskbell,
I'm the friendly issue checker.
Thanks for using the issue template :star2:
I appreciate it very much. I'm sure, the maintainers of this repository will answer, soon.

I can confirm this. I think the reason is that the BT permission request is missing. Similar things happen on Mac OS (Big Sur), see #498.

Fixed via #499.


Add some extra features

Additional context
I have just gone through your app, and it looks good and fabulous. I want to tell you something I have to share with you first of all you can add some more features like
-> cube root
-> percent
-> square root
-> exponent and
-> parentheses.
These features can save time and improve the sharpening of a student's brain while practicing.
Another thing is to add a dashboard-like button to showcase the person's data like how many tests were completed and how much time he/she took to complete the tests. And can add a feature like badges like (10% to 30%) (30% to 60%) (60% to 100%) give them a badge while achieving based on percentage.
When we are taking the test you can add the time right or left side to test the person how much time he/she is taking per question.
Looking forward to using the app with some other new features.
Thanks for the app,
Regards
Gangadhara Rao Ande

This app is for kids under the age of 11-12, all these things aren't suitable for it. Maybe we can build another app for that age group too. Thanks for your suggestion.


bug: Wrong text for 'Explorer NeoTree (Root Dir/cwd) ...

Did you check docs and existing issues?

[X] I have read all the LazyVim docs
[X] I have searched the existing issues of LazyVim
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
0.10
Operating system/version
Windows 11
Describe the bug
Only wrong text in the descriptions. Change 'Root Dir' and 'cwd'.
Check in file lua/lazyvim/plugins/editor.lua, lines 23 and 24.
      { "<leader>e", "<leader>fe", desc = "Explorer NeoTree (Root Dir)", remap = true },
      { "<leader>E", "<leader>fE", desc = "Explorer NeoTree (cwd)", remap = true },

should be
      { "<leader>e", "<leader>fe", desc = "Explorer NeoTree (cwd)", remap = true },
      { "<leader>E", "<leader>fE", desc = "Explorer NeoTree (Root Dir)", remap = true },

Steps To Reproduce

Open lazyvim
Press <space>
Read key mapping description

Expected Behavior
Show corrext text

No, the current text is correct. LazyVim uses a combination of LSP server root_dir, cwd and .git pattern to return the root. There are cases where the LSP server root_dir will coincide with cwd.


update LTR to 3.22.13

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Is this correct @SrNetoChan ?

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

@gillins we could just push the main repo into this one using the last 3.22 commit. Basically, push everything and accept all conflits from the main repository.

Yep that sounds great @SrNetoChan - go ahead (I have no idea how to do this). Yes the recipe is the same apart from the version and the checksum.

Ok, leave it to me.

Thanks! Would also be good to add libgdal to the dependencies as per #277

@gillins we can close this one, right?


Try very high staticPageGenerationTimeout value

Problem solved
Short description of the bug fixed or feature added

PR-Codex overview
This PR focuses on modifying the configuration for static page generation in a Next.js application and addressing issues related to dynamic rendering for specific pages.
Detailed summary

Added staticPageGenerationTimeout with a value of 500 in next.config.js.
Removed the dynamic export set to "force-dynamic" in apps/dashboard/src/app/(dashboard)/explore/page.tsx.
Removed the dynamic export set to "force-dynamic" in apps/dashboard/src/app/(dashboard)/explore/[category]/page.tsx.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5128  👈
#5097
#5096
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MananTank and the rest of your teammates on  Graphite


Collective reward/penalties are opaque

Issue
I frequently have to explain what is actually happening with the scale factor and collective rewards. I also have to frequently re-figure out what's going on. It's very opaque.
Need to reward the names of things and also probably write a post.

One source of confusion comes from the fact that  last_voter_rescale and last_nonvoter_rescale are variables that are used to debug instead of really required to be there. So a proposed change is to make these variables become constant functions.
diff --git a/casper/contracts/simple_casper.v.py b/casper/contracts/simple_casper.v.py
index 3b94cc9..6b02682 100644
--- a/casper/contracts/simple_casper.v.py
+++ b/casper/contracts/simple_casper.v.py
@@ -72,9 +72,6 @@ main_hash_justified: public(bool)
 # Value used to calculate the per-epoch fee that validators should be charged
 deposit_scale_factor: public(decimal(m)[int128])

-last_nonvoter_rescale: public(decimal)
-last_voter_rescale: public(decimal)
-
 current_epoch: public(int128)
 last_finalized_epoch: public(int128)
 last_justified_epoch: public(int128)
@@ -244,6 +241,7 @@ def insta_finalize():

 # Returns the current collective reward factor, which rewards the dynasty for high-voting levels.
 @private
+@constant
 def collective_reward() -> decimal:
     epoch: int128 = self.current_epoch
     live: bool = self.esf() <= 2
@@ -323,6 +321,15 @@ def total_curdyn_deposits_in_wei() -> wei_value:
 def total_prevdyn_deposits_in_wei() -> wei_value:
     return floor(self.total_prevdyn_deposits * self.deposit_scale_factor[self.current_epoch])

+@public
+@constant
+def last_voter_rescale() -> decimal:
+    return 1 + self.collective_reward()
+
+@public
+@constant
+def last_nonvoter_rescale() -> decimal:
+    return (1 + self.collective_reward()) / (1 + self.reward_factor)

 @public
 # cannot be labeled @constant because of external call
@@ -505,9 +512,8 @@ def initialize_epoch(epoch: int128):

     self.current_epoch = epoch

-    self.last_voter_rescale = 1 + self.collective_reward()
-    self.last_nonvoter_rescale = self.last_voter_rescale / (1 + self.reward_factor)
-    self.deposit_scale_factor[epoch] = self.deposit_scale_factor[epoch - 1] * self.last_nonvoter_rescale
+    self.deposit_scale_factor[epoch] = self.deposit_scale_factor[epoch - 1] * \
+        (1 + self.collective_reward()) / (1 + self.reward_factor)
     self.total_slashed[epoch] = self.total_slashed[epoch - 1]

     if self.deposit_exists():


HDDS-3686. Insert an argument of ozone shell to accept jvm arguments

What changes were proposed in this pull request?
This is a great feature, i believe everyone would like this new "--jvmargs" argument, we can specify some arguments you like, mostly, i put the "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005" into HADOOP_OPTS, now i doesn't have to do like that.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3686
How was this patch tested?
bin/ozone --jvmargs "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005" admin pipeline list
Listening for transport dt_socket at address: 5005

then, use intellij idea to attach it and enjoy debugging, have a nice day.

I think this is a good idea. However we can already pass JVM args like this:
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005"
bash-4.2$ ozone admin pipeline list
Listening for transport dt_socket at address: 5005

I guess the HADOOP_CLIENT_OPTS option is there as the shell commands use the hadoop libraries to help with the CLI.
Looking at the Ozone doc pages, I don't see this mentioned anywhere and a google search for "site:hadoop.apache.org/ozone HADOOP_CLIENT_OPTS" gives no results either.
I wonder if we should expose the HADOOP_CLIENT_OPTS in the documentation, or rename it to OZONE_CLIENT_OPTS instead ...

@sodonnel  Thank you for telling this. In fact i used to use HADOOP_OPTS and HADOOP_CLIENT_OPTS to achieve my debug purpose, but it hard to support me to write the following script.
bin/ozone scm --init
bin/ozone --daemon start scm
sh /Users/mbl/projects/github/hadoop-ozone/.project/hadoop/copyconfig.sh om
bin/ozone om --init
bin/ozone --daemon start om
bin/ozone --daemon start recon
bin/ozone sh vol create myvol;  bin/ozone sh bucket create myvol/mybucket
bin/ozone --daemon start datanode

I want to start the debug agent of each ozone server with different port. If we have this PR, we can

bin/ozone scm --init
bin/ozone --jvmargs " -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5000" --daemon start scm
bin/ozone om --init
bin/ozone --jvmargs " -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5001"  --daemon start om
bin/ozone --jvmargs " -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5002"  --daemon start recon
bin/ozone sh vol create myvol;  bin/ozone sh bucket create myvol/mybucket
bin/ozone --jvmargs " -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5003"  --daemon start datanode

Another way of using the HADOOP_CLIENT_OPTS / HADOOP_OPTS is by not exporting the variable, but including it on the same line as the command, eg:
> HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -agentlib:jdwp..." ozone scm ...

However, this is not as discoverable as an explicit jvmargs flag like you have added here.
With your change, does this command append to existing HADOOP_OPTS or overwrite any existing arguments set by HADOOP_OPTS?
hadoop_add_param HADOOP_OPTS "$1" "$1"

I am wondering how these two features would work together?

After a bit of trial and error I figure out that:
Anything in HADOOP_OPTS comes first.
These new JVM Flags are appended to that.
Anything in HADOOP_CLIENT_OPTS is appended to that, eg:
"$HADOOP_OPTS --jvmflags $HADOOP_CLIENT_OPTS"

@sodonnel  Thank you for your review.

With your change, does this command append to existing HADOOP_OPTS or overwrite any existing arguments set by HADOOP_OPTS?

The function hadoop_add_param <envvar> <checkstring> <appendstring> will append the appendstring if checkstring is not set.
$1 is what we are adding to
$2 is the name of what we want to add (key)
$3 is the key+value of what we're adding
As the jvmargs will bring a bundle of jvm arguments, so i just check the whole arguments string as lazy.

After a bit of trial and error I figure out that:
Anything in HADOOP_OPTS comes first.
These new JVM Flags are appended to that.
Anything in HADOOP_CLIENT_OPTS is appended to that, eg:
"$HADOOP_OPTS --jvmflags $HADOOP_CLIENT_OPTS"

Thank you for trial this PR.

@sodonnel Thank you for your suggestion, i push a new commit to address it, PTAL.

@sodonnel Thank you for your review again, i push a new commit to address your comment. PTAL

@sodonnel Sorry for the mistake,  BTW, i change the "jvm arguments" to "arguments", i think it would be better if there is only one word here, two word maybe make people think it could be two arguments, and from the context, we know the arguments stand for the jvm arguments.

Thanks for the few iterations on this. Looks good now, so I will go ahead and merge it.

@sodonnel Thanks for your review


Add basic Helm test framework

Description
Add a basic helm test framework for testing that helm will render resources correctly depending on which options are passed to it.
Running the tests should be as simple as running make helm-tests RELEASE_STREAM=<version>.
Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

Deploy preview for calico ready!
Built with commit 544f52ffe9aad1c67f338bd5db175ad652935dd7
https://deploy-preview-2504--calico.netlify.com

Could we add just one basic test to prove it works?


apt-transport-https installation fails on a new host.

Hello,
The role fails on a new host (e.g., with a old apt cache), since it fails to find one of the apt-transport-https dependencies (libcurl3-gnutls). I've got this error on Vagrant using a Debian Wheezy 7.8 box.
To fix this, we only have to add the update_cache option and the cache_valid_time so that it updates the apt cache if it's older then cache_valid_time value.
The PR fixes that and also adds YAML syntax to the task file, which makes it really easier to read and diff.
Thank you!

I am going to close this one as it seems like an edge case.
IF your concerned about this happening them best to run something like the following task before running this role.
- name: Apt-get update before installing
apt: update_cache=yes cache_valid_time=3600

Cheers


Centralize code for all models

Model code is not in any centralized place right now. We want to track down all code and put it into one shared place.
Plan:

Upload to brainscore-vision Git branch to get it all in one centralized, tracked place
AirTable (list with isOnGithub, reproducesOnMajojHong (+ others), isConverted, isMerged, reproducesOnAll)

Open Questions

Resolving models with changed identifiers
Get BrainTree models

https://airtable.com/invite/l?inviteId=invNGt9gVRFguJrUQ&inviteToken=95f8cfa37a21e6c1f93e84d8660cfcf40a3ae32ae5f5ffb81c8bf09f3522a1e5&utm_medium=email&utm_source=product_team&utm_content=transactional-alerts
Airtable link above! Will be updating the model statuses as I come across them.

To ask: How many models are private?

Two models are private in the whole repo, so we can have a lot of these in our BS repo. Waiting on Martin to track down which of those it is before proceeding.

21 models on the leaderboard need to be removed from the model pool before we can upload them to a public space.


Agent doesn't assign the context network

If you run tests inside docker container connected to git services like postgre,
the DDAgent doesn't assign network.  (job.container.network from Contexts)
This does not allow to communicate the test container with the DD agent, are not on the same network.
If there is a github action network in the context, it should automatically be assigned to the ddagent if you want.
For example with argument like "use_context_network_if_exists=true/false"
I have already tried to assign the network to ddagent and works.

hey @jgranesa , all test visibility supported languages now can use agentless mode, and that's the way we recommend to use if your workflow runs in github actions: https://docs.datadoghq.com/tests/setup/javascript/?tab=cloudciprovideragentless. We're deprecating this action


Async topic isn't passed to vows in topic-less subcontexts

Here's a test case. Notice that only the marked vow is not honored, the other ones are just there for comparison.
var vows = require('vows'),
    assert = require('assert');

vows.describe('Vows bug').addBatch({
    'Sync 42': {
        topic: 42,
        'equals 42': function (topic) {
            assert.equal(topic, 42);
        },
        'has the property that': {
            'it is equal to 42': function (topic) {
                assert.equal(topic, 42);
            }
        }
    },
    'Async 42': {
        topic: function () {
            var callback = this.callback;
            process.nextTick(function () {
                callback(null, 42);
            });
        },
        'equals 42': function (topic) {
            assert.equal(topic, 42);
        },
        'has the property that': {
            'it is equal to 42': function (topic) { // <-- This vow fails, topic is undefined!?
                assert.equal(topic, 42);
            }
        },
        'plus 1': {
            topic: function (parentTopic) {
                return parentTopic + 1;
            },
            'equals 43': function (topic) {
                assert.equal(topic, 43);
            }
        }
    }
})['export'](module);

after 53001db5, this can be fixed by changing
suite.js:166 ->
lastTopic = topic = ctx.emitter;
The problem is lastTopic is defined before the callback logic.  Once #145 lands the above change and tests can get added.

I think that #77 is the same thing.  In both cases you are using a callback which will update topic after lastTopic is set.

Version 1.x doesn't support topic-less subbatches, so I'm going to close this.


Add amazon EC2 Provider to IaaS

This PR adds support for usage of EC2 from AWS to gofn, simply instantiating a new provider with secret and access keys.
Example to verify execution:
https://github.com/gofn/gofn/tree/improvements/aws_ec2/examples/amazonec2

Codecov Report

Merging #111 into master will decrease coverage by 1.64%.
The diff coverage is 41.89%.

@@            Coverage Diff            @@
##           master    #111      +/-   ##
=========================================
- Coverage   53.55%   51.9%   -1.65%
=========================================
  Files           5       6       +1
  Lines         450     524      +74
=========================================
+ Hits          241     272      +31
- Misses        185     228      +43
  Partials       24      24

Impacted Files
Coverage Δ

iaas/amazonec2/amazonec2.go
41.89% <41.89%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ab1b51...884c80a. Read the comment docs.


rpi-deep-pantilt not working (instantly crashing)

rpi-deep-pantilt version: 1.2.1
Python version: 3.7.3
TensorFlow version: 2.4.0
Operating System: RaspiOS lite (latest version)

Description
Exactly followed along the install instructions (no errors during installation) but when running any command (test pantilt, test camera, detect...) then I get this error message:
(.venv) pi@raspberrypi:~ $ rpi-deep-pantilt test pantilt
Traceback (most recent call last):
  File "/home/pi/.venv/bin/rpi-deep-pantilt", line 5, in <module>
    from rpi_deep_pantilt.cli import main
  File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/cli.py", line 10, in <module>
    from rpi_deep_pantilt.detect.ssd_mobilenet_v3_coco import (
  File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/ssd_mobilenet_v3_coco.py", line 9, in <module>
    from PIL import Image
  File "/home/pi/.venv/lib/python3.7/site-packages/PIL/Image.py", line 114, in <module>
    from . import _imaging as core
ImportError: libopenjp2.so.7: cannot open shared object file: No such file or directory
(.venv) pi@raspberrypi:~ $

What I Did
I tried to run rpi-deep-pantilt without any command line parameters or with --help but it always crashes with the same error message as above.
Your help is much appreciated!
Thanks in advance!
Best regards
careyer

The following dependencies are missing:
sudo apt install libtiff5  libopenjp2-7


Update Dockerfile

Hello! Please, update dockerfile with new versions of mmdet and mmsegmentation, now it does not work with HTC-like models. Thanks

@Ilyabasharov
Thanks for your contribution. We will update the versions of mmdet and mmsegmentation in dockerfile as soon as possible.


writing-to-files python tutorial

Populate file-handling/writing-to-files.py with a tutorial and challenge.

I would like to work on this issue

@raghuchandan1 Please submit the PR for the other issue before I can assign you to this one. thanks.

I have finished reading-files. Can you please check that and assign this to me?

assigned @raghuchandan1


Refactor arangodb ansible playbook

One of the biggest use cases for this is splitting up the readonly user steps from the rest of the playbook, so it can be selectively applied when needed. However, we also want to take a closer look at our current steps against the steps required, and how we can make it more streamlined and error prone.

We had a brief meeting to discuss this, and we decided the following items should be part of this work:

tighten the way certain tasks are done (e.g., prefer restart to a stop followed by a start)
split the tasks up into "sub playbooks" so they can be run independently of each other
include a "master playbook" that runs everything
use ansible-vault to store necessary secrets for deployment (Arango passwords, cert files, etc.)


[Feature] confirmation of deletion on Vehicle sheet ❌

Can you add system ask about confirmation of deletion ship weapon on Vehicle sheet:

I think about something like this:

Why? It is easy to missclick, and now there is no way back

I don't think this is worth it, personally. In the years I've been using this engine, I cannot name anyone I know who has encountered this (except you, I assume)... and it will slow things down for GMs. For now, I'm pulling it from this milestone. If you get some other people to weigh in and mention that they've done or it would like it, I'd be happy to implement it in a future version.


User Story 1.1 - Issue 1: Volunteer Position font issue

see font-family and font-size incorrect - see issue 1

Hey @xJamesScott, I am retesting this on desktop (chrome and safari), still have the same issue. the font family and font size incorrect. you can see the pictures. it is not matching in Zeplin.

this one is from chrome.

it looks fine


Revoke Closet Skeletons' free agent status

Closet Skeletons should be made crew-aligned. Why? Well:

Closet Skeletons have taken a turn for the worse as of recently, since players pick it mostly to abuse the fact that they basically get a green pass to murderbone and cause near-station wide destruction and really, an unfun envinronment for everyone but themselves, as doing so is technically allowed due to them being free agents.

Closet Skeletons can already roll for sleeper syndicate agent, meaning that in the end they're actually more antagonist-aligned than any other mid-round free agent (Besides perhaps Rat King, but that's a topic for another day), this should be an extra incentive to push them into the crew-aligned area since the chance for them to be evil is already there, but rather than it being up-to-interpretation it's as a fixed antagonist.

Closet Skeletons are past crew from the IC explanations, why should they be evil all of a sudden? I understand they're on an entirely different plane of existence, being magical beings, but this also restricts them off of a lot of things RP-wise rooted from the crew's distrust and wariness, as they should be. I think making them established crew members instead of potential murder-machines to be much healthier for player culture.

They aren't allowed to cause stationwide destruction or murderbone, ahelp that.

Skeletons provide a significant amount of chaos to round, as is the point of free agents. If they are not free agents, then they just become something that's immune to revs, can only become sleeper antag, and is immune to spacing. They become too dependable for command to trust.

they're actually more antagonist-aligned than any other mid-round free agent (Besides perhaps Rat King, but that's a topic for another day)

What other free agents are there that are less "antagonist-aligned"

Closet Skeletons are past crew from the IC explanations, why should they be evil all of a sudden?

It sounds like something to RP for becoming evil because they are not evil. I've been good and evil skeletons, they are free agents, not purely antagonists. If you want an IC explanation it could be RP'ed but that's on the skeleton.

 They are about as much free agent as Hamlet now.

I think skeletons contribute some friction to the game, which is needed, as this game is indeed a game about Everything Going Wrong(TM).
It is obvious that there is a problem with people not following WizDen's rule 2.9, stating that you can't cause major damage beyond your objectives.
Skeletons have always been in a grey area because of this and have seen different rulings by the admin team on upstream. To summarize my thoughts:

Skeletons are free to behave without any restrictions other than Rule 2.9.
Rule 2.9, which defines damage beyond your objectives, requires a base objective to compare. Skeletons don't have a defined objective, it's either "live peacefully or cause chaos" which leads to ambiguity and ultimately an excuse to do extremely destructive actions.
Skeletons aren't required to establish themselves as good or bad and stick to this throughout the round, they can switch whenever they want. Because of this, this commonly leads them to being not considered for many roles due to players not trusting them, which is acceptable, and is good friction in my opinion.
Skeletons are very powerful in combat, which pushes their role towards it. They can heal almost all damage via splashing milk, which they can easily get from raiding botany or asking them. I believe they also have slight resistances to most blunt damage types, but I could be wrong. Even if this wasn't the case, not caring about spacing damage is a huge plus, as it allows skeletons to wear better armor compared to hardsuits, while still being able to move quite fast.
Because of the aforementioned issues, the skeleton ghost role is sometimes taken as a metagrudge role to inconvenience the at-fault department as much as possible if they screwed the person over in their past life.

I think the skeleton role is actually interesting and leads to interesting outcomes and conflict, but it generally revolves around people not following that rule, and overblowing their antagonism to the point where it makes the round now fun. There have been some fun antag skeletons gimmicks though. Some.

Skeletons provide a significant amount of chaos to round, as is the point of free agents. If they are not free agents, then they just become something that's immune to revs, can only become sleeper antag, and is immune to spacing. They become too dependable for command to trust.

I'm all in for chaos, but there's bound to be limits.

they're actually more antagonist-aligned than any other mid-round free agent (Besides perhaps Rat King, but that's a topic for another day)

What other free agents are there that are less "antagonist-aligned"

Most critters and syndicate refugees at the top of my head, the former do not pose a problem as they aren't possibly strong enough to become even a hassle, and the latter don't get the boons skeletons possess (space resistance, milk healing, wide array of damage immunities) plus, being "refugees" it'd make sense from an rp standpoint for them to be grateful for NT having taken them in.

Skeletons aren't required to establish themselves as good or bad and stick to this throughout the round, they can switch whenever they want. Because of this, this commonly leads them to being not considered for many roles due to players not trusting them, which is acceptable, and is good friction in my opinion.

I won't argue that it's bad friction, since in the right hands it can lead to some incredibly interesting standoffs, however I feel like people just push it far too often to the point where in a moment they go from reputable secskele to a capital murderer, it's just very odd to have an immediate switch don't you think? I wouldn't mind this if there was more of a "build-up"

Skeletons are very powerful in combat, which pushes their role towards it. They can heal almost all damage via splashing milk, which they can easily get from raiding botany or asking them. I believe they also have slight resistances to most blunt damage types, but I could be wrong. Even if this wasn't the case, not caring about spacing damage is a huge plus, as it allows skeletons to wear better armor compared to hardsuits, while still being able to move quite fast.

Closet Skellies are by far the best brawler species, they possess resistance to most types of DoT as well as instantaneous adjustable healing, if this issue doesn't get through I at least hope that they get slightly nerfed, since having both "you can act as whatever, whenever" and "you are the best combat species" is a recipe for disaster.

I think the skeleton role is actually interesting and leads to interesting outcomes and conflict, but it generally revolves around people not following that rule, and overblowing their antagonism to the point where it makes the round now fun. There have been some fun antag skeletons gimmicks though. Some.

A good skeleton makes shifts extremely fun, however, I've come to notice it happens less and less, sometimes there's skeletons that partially RP at the beginning to ask for a certain job and such, but after a certain duration they just suddenly have antagonistic intent, and then there's skeletons that just toolbox you on the spot, and both of these are equally as justifiable due to their free agent label, by making them crew-aligned still with the existing risk of them turning out to be sleeper agents we basically invalidate any kind of instant hostility from and towards the skeletons, imho
everything not mentioned I agree with, however, especially how often rule 2.9 is abused

I'm all in for chaos, but there's bound to be limits.

That sounds like a rules issue not with the skeleton itself. Either way if they were not free agents command would trust them too much.

Most critters and syndicate refugees

Critters cannot do anything significant. Syndicate refugees are so rare right now that I haven't even seen a group of them in forever, so I can't comment much on that. But what I've seen before that change when they first came out was people self-aging because "we are syndicate, and we hate NT."

it'd make sense from an rp standpoint for them to be grateful for NT having taken them in.

Just based on the amount a person RPs, Skeletons could do the same thing with being grateful and syndicate refugees can murderbone like the skeletons.

I'm all in for chaos, but there's bound to be limits.

That sounds like a rules issue not with the skeleton itself. Either way if they were not free agents command would trust them too much.

How would command trust them too much? wouldn't they be as trusted as any other crew member?

Most critters and syndicate refugees

Critters cannot do anything significant. Syndicate refugees are so rare right now that I haven't even seen a group of them in forever, so I can't comment much on that. But what I've seen before that change when they first came out was people self-aging because "we are syndicate, and we hate NT."

every time I've seen syndicate refugees they've always had the "we're in this together" mentality, so usually they wouldn't disperse unless every single one of them was planning on antagging. To add onto my previous, syndicate refugees are also considerably more controlled (and possibly even more monitored) than skeletons as they have a lot of their stuff taken away from them immediately after docking.

it'd make sense from an rp standpoint for them to be grateful for NT having taken them in.

Just based on the amount a person RPs, Skeletons could do the same thing with being grateful and syndicate refugees can murderbone like the skeletons.

yeah, they can, but should they? imho neither should ever act in a murderboney manner, and what better way to introduce this other than making their labels unambiguous? no, seriously, I'm open to suggestions

How would command trust them too much? wouldn't they be as trusted as any other crew member?

They can't be rev, they only have a 0-2/70 chance when sleeper is rolled(assuming they have a pda), and would not be able to be a ling whenever that gets added. There would be no reason to turn a skeleton down for being a secoff.

yeah, they can, but should they? imho neither should ever act in a murderboney manner, and what better way to introduce this other than making their labels unambiguous? seriously, I'm open to alternative suggestions to address this particular issue

Some kind of rules change I guess. I don't really see the issue though, it sounds like a skill issue of sec to not be able to take out one skeleton.  Again they are meant to kill people, cause uncertainty, and reduce the quality of the shift.

How would command trust them too much? wouldn't they be as trusted as any other crew member?

They can't be rev, they only have a 0-2/70 chance when sleeper is rolled(assuming they have a pda), and would not be able to be a ling whenever that gets added. There would be no reason to turn a skeleton down for being a secoff.

They can be made sleepers even without a PDA, wouldn't that also mean every mindshielded passenger gets the same trust?

yeah, they can, but should they? imho neither should ever act in a murderboney manner, and what better way to introduce this other than making their labels unambiguous? seriously, I'm open to alternative suggestions to address this particular issue

Some kind of rules change I guess. I don't really see the issue though, it sounds like a skill issue of sec to not be able to take out one skeleton.  Again they are meant to kill people, cause uncertainty, and reduce the quality of the shift.

Issues shouldn't just be thrown under the rug just because one side is better than the other, this is a game, games are meant to make it so everyone has a fun time, this role is actively misused by antag-hungry powergamers that just stomp through sec and command since skellies allow them to do so freely and easily.

I don't understand your singuloose point

they are not meant to kill people, they can do it, sure, they can do anything, since skeletons (as it stands) are free agents only intended to be a jack of all trades that you can't really trust, it's just that personally I think skellies should promote a more rp-oriented gameplay loop rather than just being absurdly good combat species that suffer little to no consequences due to their free agent status, also, reducing the quality of the shift is just wrong in a lot of aspects?

Anyway, I've had a couple discord discussions about this free agent topic and the consensus is that the problematic thing is actually just how broad free agent is, so here's a couple proposed solutions:

Remove it and make every role currently on a moral limbo decisively crew-oriented or antagonistic

Tweak it for each individual role to fix inconsistencies

Change the roles themselves so that they can better feel two-sided (Eg. The crew has nothing to gain from Rat Kings, however, if we added some kind of ability or trait that makes them worth keeping it'll fit it's free agent status considerably better. As for the skeleton, maybe just make it less combat-oriented)

The players must play in one of two ways.


Modrinth

:clipboard: Description

Service : https://modrinth.com
This is the kind of information that should be shown by the new badges

Example :

Modrinth
https://api.modrinth.com/api/v1/mod/wq6HaMZG
#5da426

 downloads

:link: Data

https://github.com/modrinth/labrinth/wiki
https://api.modrinth.com/api/v1/mod/wq6HaMZG
:microphone: Motivation

In my case, I want to show off the downloads of my mod on a specific platform, Modrinth.

https://github.com/badges/shields/issues/4498

I can't see any docs explicitly explaining rate limit. https://api.modrinth.com/ gives a documentation link https://modrinth.com/documentation but its a 404.
 Are you able to link to any docs about rate limits? I think if we were going to take on this integration we'd need to understand this constraint in more detail.

I think rate limits were added there:

https://github.com/modrinth/labrinth/pull/90

But the only thing I found was that.
.wrap(Logger::new("%a %{User-Agent}i"))
.wrap(
	RateLimiter::new(MemoryStoreActor::from(store.clone()).start())
		.with_interval(std::time::Duration::from_secs(60))
		.with_max_requests(100),
)

I think it means 100 requests per 60 seconds for each user agents? I'm not sure, since lots of people can have the same user agent.
@Geometrically, do you know where are rate limits documented on Modrinth?

The ratelimit is per second, so it's actually rather liberal (regulated by Cloudflare). We can add information on this to our docs site soon enough, but it's still very work in progress :)
P.S.: There are also some rudimentary unofficial badge generators for Modrinth as well. See https://modrinth-utils.vercel.app

Aah I thought the rate limits may be per hour or minute. 200 reqs per second is no issue :+1:

Is this issue available to pick up? I'd be willing to work on it if no one else is already doing it.

@BrenekH - feel free to pick it up. Our guide for adding new badges is at https://github.com/badges/shields/blob/master/doc/TUTORIAL.md
I think the issue contains all the info needed to work on this. Although when it comes to colours, shields.io uses colours semantically (to convey information) rather than for branding purposes i.e:

etc
so a semantic colour scheme should be used by default rather than #5da426

A few days have passed and no solution was presented so I assumed this issue is abandoned. Therefore I've opened the PR for that - #7132. Please let me know what are your thoughts.

@NatoBoram this has been merged just now. Feel free to use the new badge!


[RFE] Azure KeyVault Support

The idea of the RFE is to see how we can leverage Azure Keyvault as an external KMS for trousseau.
What do we want to build?
ability to use Azure KeyVault as a KMS for Trousseau
Why do we want to build it?
to offer Azure users an integrated solution that would release them from having to deploy another vault solution they will have to manage

We are bringing AWS support in v2 (a pre-release is ready) for the next milestone.
From this work, a complete redesign has been done making it easier to onboard new KMS.
I will create a new milestone and add Azure KeyVault in it.

addressed within #147

PR add support for Azure KeyVault #148
Closing as duplicate of #147 - the issue to follow for final including documentation update


cert manager reporting ingress (name) not found, but none defined in clusterissuer

cert manager 1.6.1 on Kubernetes 1.21. Ingress controller is haproxy.
When re-issuing an existing certificate, Cert manager seems to want an ingress called "haproxy" from an error I am seeing in the logs:
msg="re-queuing item due to error processing"
error="ingress.networking.k8s.io \"haproxy\" not found"

My issuer does not specify an ingress name (per the docs, only a class):
kubectl get clusterissuer -o yaml
...
  spec:
    acme:
      solvers:
      - http01:
          ingress:
            class: haproxy
        selector: {}

yet when I look at the challenge, it seems to want an ingress by name:
kubectl get challenge -o yaml
...
    issuerRef:
      kind: ClusterIssuer
      name: letsencrypt-prod
    solver:
      http01:
        ingress:
          name: haproxy
      selector: {}
...

I have an ingress class called haproxy:
NAME      CONTROLLER                             PARAMETERS   AGE
haproxy   haproxy-ingress.github.io/controller   <none>       80d

any ideas?

edit: I found this annotation on the order acme.cert-manager.io/http01-override-ingress-name: haproxy; editing the order, removing that line and removing the challenge allowed me to issue the certificate. Not sure how that annotation got there, I'll post here if I find out

It seems that this annotation was only attached to one of my 400+ orders. Closing this for now, hopefully it shows up as useful to someone else troubleshooting.


Change benchmark reporting to expose the information from GCS to Markdown

Dependencies

Depends on: #432
Depends on: #434

Acceptance criteria

[ ] Change the script which consolidates the output of our benchmarks to retrieve the values stored in GCS and generate markdown tables

References

Script which converts results into markdown: https://github.com/astronomer/astro-sdk/blob/main/tests/benchmark/analyse.py

Currently run.sh produces results as ndjson files and then, analyze runs and produces markdown. run.sh also copies the results to GCS.
So instead of querying GCS, run.sh can call analyze which produces markdown.


fixed broken link

custom publish for Azure CLI beta version release

Docs Build status updates of commit ba1672b:
:clock10: Full build: the average full build time is 7 min(s) 38 sec(s), this is based on the last 669 build(s) for this repository.

Docs Build status updates of commit ba1672b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/install-azure-cli-windows.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


refactor(minifier): remove allow clippy::unused_self

#6441  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @Boshen and the rest of your teammates on  Graphite

Merge activity

Oct 10, 10:54 PM EDT: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.


The selected event on the select option doesn't emit

The ionSelect (called select prior to #6568) is never emitted on the option for the select component.

I can help out, may I assign myself to this?

@ryandegruyter I don't think issues can be assigned outside of our team, but if you'd like to work on it and submit a PR it's all yours! The difficulty with this issue is the Select uses the Alert & ActionSheet API underneath so it will need to emit the event in each back to the Select.

Ok  thank you, I underestimated and misunderstood this issue. But I can still give it a shot, when exactly should the ionSelect output emit? When the option is selected/clicked but not when the OK button is clicked?

Yeah it should emit when the option is clicked, prior to pressing the OK button, since once you press the OK button it triggers the ionChange event on the select. :smile:


Jinja Templating

Type
new feature / advancement in original
Description
Ability to create reusable templates for our web application. There are lots of elements in the website that are currently common and in future will be common like the header or footer or the navigation bar. So instead of copy pasting the exact same code on every single page of the website we can simply create a common template layout.html which will have all the common elements and then we can inherit that layout.html in other pages using the template engine jinja2 of flask. It also improves the speed and performance as it compiles template source code to Python bytecode when it's first loaded, so the template is only parsed once, resulting in better runtime performance.
Tools
HTML
Python (flask)
Have you read the Contributing Guidelines on Pull Requests?
Yes

Please assign me this issue.

Sounds good. You have 5 days to make the template for the home page. Refer to the approved UI for the design and feel free to comment here or ask on Slack in case of any issues.

Sounds good. You have 5 days to make the template for the home page. Refer to the approved UI for the design and feel free to comment here or ask on Slack in case of any issues.

@sarthakkundra  Actually keep this issue on-hold. I was not talking about making the home page of our web application according to the UI approved.
i was just planning to add jinja templating in the current basic flask setup or you can just close this issue.
After our home page is ready we can open the same issue again to improve the functionality.


Refactor PCR-based policy-OR functionality

This pull request is an attempt to improve the kmyth-seal and kmyth-reseal functionality addressing PCR-based policy-OR criteria:

The existing approach uses a different .ski format when sealing with a PCR-based policy-OR criteria. This pull request standardizes on a 'PCR SELECTIONS' block encoding a PCR_SELECTIONS struct containing a list of 'TPML_PCR_SELECTIONstructs and aPOLICY ORblock encoding aTPML_DIGEST` struct containing a list of policy digests for all scenarios and negates the need for a command-line parameter to specify the format (addresses kmyth issue #173).
This pull request enables automatically including the PCR selection and policy digest data recovered from the input .ski file in a kmyth-reseal result. The approach taken, however, required significant changes to the kmyth API.
This pull requests attempts to allow the user to specify a PCR-based policy-OR criteria in a clearer and more meaningful way. In the approach taken, the user now expresses an "expected policy" criteria as a string specifying a pair of PCR selection and policy digest values.
For the existing approach, only two policy digests can be specified, and both must use the same PCR selections. This pull request makes this more flexible, allowing up to eight policy-OR criteria, each with independent PCR selections.
Changes to the API required changes to a number of tests. Some testing for new or revised functionality was added, but making the testing more complete is probably still a future goal.
Policy-OR criteria in kmyth are currently only PCR based. In the future, functionality could potentially be extended to allow more flexible and generic policies. This code, though, enforces that all policies in a policy-OR criteria are PCR-based. For the current kmyth, there is no value in using a policy-OR criteria if any of the criteria is not PCR-based.

While attempting to address the comments, I stumbled across some other issues that I attempted to correct:

Both an init_policyOR() and an init_policy_or() function were present in the code. As init_policy_or() was unused, I cleaned up this dead code.
The kmyth-reseal main() used init_policyOR() to configure the PCR selections list and policy digest list structs for a policy-OR authorization criteria. The kmyth-seal main(), however, did not. I attempted to update both to use the policy_OR() function in a similar fashion.
As there were no tests for parse_exp_policy_string_pairs() and init_policyOR(), I added tests for this functionality.
I discovered that the parse_exp_policy_string_pairs() function used "," as the string pair delimiter in one of the strtok() calls and ";" in another. The "," case is invalid because it interferes with its use as a PCR list delimiter. In testing, however, I decided that the choice of ";" as a string pair delimiter is also not ideal. It requires encapsulating the string in quotes to avoid it being decoded as a command delimiter on the bash command line. Therefore, I changed the delimiter to be a forward slash ("/").
I corrected an indexing error in the init_policyOR() code populating the policy digest list struct.
I filled in some missing parameter descriptions for init_policyOR() in the tpm2_interface.h header file
I expanded the parse_exp_policy_string_pairs() code to handle the presence of a '0x' prefix for the hexadecimal string representing a policy digest (a hex-string without the prefix also still works).

Addressed a merge conflict in src/main/seal.c.


[ENHANCEMENT] Add install mode build CI release

Is your enhancement related to a problem? Please describe.
This is a follow-up issue of https://github.com/harvester/harvester/issues/2198. It was planned only to release the install qcow image on tag or daily cron build.
Describe the solution you'd like
Enable BUILD_QCOW=true on the daily cron job or tag build.
However, this is blocked by the CI of missing KVM kernel support https://drone-publish.rancher.io/harvester/harvester-installer/1524/1/2
Could not access KVM kernel module: No such file or directory qemu-system-x86_64: failed to initialize kvm: No such file or directory
Describe alternatives you've considered

Additional context

We can see if this can be achieved with github action migration.

Need to check if the new GH action runner has kvm capability.


"no_branding" is ignored

As far as I known, it's not possible to remove the branding on the loading wheel. Branding on the form (next to up&down arrows) is removed though.


interested to contribute to the organisation

Hello Everyone, my name is Yashwanth. I am a third year Undergraduate from IIT Bhubaneswar. I am proficient in C++,Python. I have experience in working  with Machine Learning algorithms. I am planning to apply for GSoc19 with your organisation. I need help to get started .so,please assign me some task/subgoal to start contributing and work along with you.

This ticket system is not meant to be used for GSoC discussions but for feature requests, bug reports, and pull requests. Please see our Hints for Successful Proposals to get more information about how to discuss GSoC topics and to generally get in contact.


feat:Whether the alarm button is blocked

Description
Please explain the changes you've made
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

@kogel-net  Resolve the conflict


ability to get both gens

there is 2 gens for each prompt, but this api seems to only give one of them. Can you add the ability to get both?

it depends on udio, they are recently giving 1 song per generation

Great news! We are finalizing adjustments to the library, including adapting to recent changes Udio has made to the API, such as call parameters and header issues. Additionally, we are working to enable you to download all the generations that Udio returns, currently 2 per generation. The new version 0.0.3 is very likely to be available later today. I'll let you know as soon as you can start using it. Thanks for your patience and for staying tuned for updates!

We have just released version 0.0.3 of the UdioWrapper library, which includes the ability to download all the generations that Udio returns, currently 2 per generation. We have also adapted the library to the recent changes Udio has made to the API, such as call parameters and header issues. The new version is now available, so you can start using it. Thanks for your patience and for staying tuned for the updates!


Run Issue

Hello, I have been trying to run your code and have gotten it to run but it gets to
Training on train:   0%\|                                \| 0/100 [00:00
And never makes any progress. I am running this on my desktop which is large and GPU is enabled. Not sure if there is something wrong or what I need to do. I am using our.sh to run it. Any advice would be greatly appreciated.

Hi Andrew, what's the system of your desktop? We've only tested our code on Ubuntu 14.04. If your OS is different, you may want to try virtual machine.

Yes, I will try out a VM


Add toy OS called vmt

Sorry, I missed this. Merged in 17a23aa1dae6aef0c64df27d16682ed4e1c6dd14

Not at all! Thank you.


HasMany does not work when offline

Hi @frank06
I have some models designed like below:
@JsonSerializable(anyMap: true)
@DataRepository([BaseAdapter, SiteDetailAdapter])
class SiteDetail extends DataModel<SiteDetail> {
  @override
  final int? id;
  String? name;
  final HasMany<Question>? questions;

  SiteDetail(this.id,
      {this.name,
      this.questions,});

  factory SiteDetail.fromJson(Map<String, dynamic> json)  => _$SiteDetailFromJson(json);

@JsonSerializable(anyMap: true)
@DataRepository([RemoteAdapter])
class Question extends DataModel<Question> {
  @override
  final int? id;
  String? prompt;
  String? type;
  String? status;

  Question(this.id, {this.prompt, this.type, this.status});
  factory Question.fromJson(Map<String, dynamic> json) => _$QuestionFromJson(json);
  Map<String, dynamic> toJson() => _$QuestionToJson(this);
}

It works well when online but when offline, all the questions are empty.
I printed the SiteDetail list:
// OFFLINE
[
    "SiteDetail{id: 1012, name: Cat or Dog Pages, slug: cat_pages, domain: null, status: published, isSelected: null, questions: HasMany<Question>()}",
    "SiteDetail{id: 11, name: Survey test, slug: survey_test, domain: null, status: published, isSelected: null, questions: HasMany<Question>()}"
]

// ONLINE
[
    "SiteDetail{id: 1012, name: Cat or Dog Pages, slug: cat_pages, domain: null, status: published, isSelected: null, questions: HasMany<Question>(1022, 1025)}",
    "SiteDetail{id: 11, name: Survey test, slug: survey_test, domain: null, status: published, isSelected: null, questions: HasMany<Question>(21, 19, 20)}"
]

Is Anything wrong with the models I designed?

Have you upgraded to FD 1.2.1?
If not, since your model has multiple words, it's likely it has to do with this bug: https://github.com/flutterdata/flutter_data/commit/dba64d8e6051f40e19c08fc5ac921da683ccfeef

Hi @frank06
I'm still using the old version (0.8.2). I will try the latest version. Thanks.

By the way, if multiple words is the problem, can I rename my model to a single word to fix it?

Can I just use List under my SiteDetail? (Instead of HasMany, because the question list are under SiteDetail response - sometimes it misses questions)

For relationships to work you must use HasMany. On 1.2.1, did you clear the cache and re-ran codegen? Also, try passing clear: true (or uninstall the app from the simulator). Do you get the same result?

Let me try:

clear cache + re-run codegen
passing clear: true

I will update you later today. Thanks again @frank06

Flutter Data is working in similar scenarios on other applications. I am not sure what it's happening, were you able to debug the issue and provide further information?
Probably unrelated: What are you trying to accomplish by using @DataRepository([RemoteAdapter])? Looks very unusual.
Why is one model using a BaseAdapter and the other one not?

I am having the same issue where findAll(remote: false) doesn't bring the HasMany data. It worked while using watchAllNotifier(remote: false). However, I see in the new versions of flutter_data watchAllNotifier isn't present anymore. Is there a reason to remove watchAllNotifier ?

@oarshad you can still access the notifier by ref.read(modelsProvider().notifier).
What do you mean by findAll not bringing HasMany data? The models returned by findAll have broken relationships? Please clarify and provide some examples so I can look into it.

@frank06 I have created a simple demo application to explain the problem:
https://github.com/oarshad/flutter_data_demo
There are two buttons, Remote Data and Offline Data.
Replicate Steps:
When you run this application for the first time and hit Offline Data. It doesn't print anything on the console, as there is no data. After that, if you hit Remote Data, it does fetch the data from remote and prints all entries along with its child relationships.
Once Remote Data is already pressed, you can hit Offline Data many times and it works as expected whereas printing complete entities along with their relationships.
However, if you now re-install the application and then start with Offline Data, it prints only the main entities and doesn't print child relations. I guess child entities never got initialized until called Remote Data again.

I appreciate that @oarshad ! I will soon have a look

that seems to be working fine 👍
Thank You @frank06


UDN Gateway: Ignore EndpointSlices without a service label

Instead of failing ignore EndpointSlices without a service label set.
These custom, user created slices should be ignored.

cc @tssurya


Serve maintenance page styling on server (not cdnjs.cloudflare.com)

cdnjs.cloudflare.com references are current in our CSP because we use Tachyons to style the maintenance page when a site is being tweaked.
We should serve it locally to tighten CSP.

Done.


Use whitelist in .dockerignore

Closes #116
I like how much cleaner this looks. Didn't even know this was possible. Thanks @dol

Happy to help.


Adding Points Layer with out_of_slice_display and shown

🐛 Bug
I can't seem to add a points layer which uses both the shown and out_of_slice_display parameters.
To Reproduce
If I run the following code:
import napari
import numpy as np

viewer = napari.Viewer()
points_zyx = np.array([[0, 1, 5], [1, 8, 9]])
shown = [False, True]
viewer.add_points(points_zyx, size=[3, 2, 2], shown=shown, out_of_slice_display=True)
napari.run()

I get an error:

But it does work if I set shown=[True, True].

Expected behavior
To be able to add the points layer but only showing some of the points, while at the same time having out_of_slice_display=True.
Environment
napari: 0.4.16
Platform: macOS-12.4-arm64-arm-64bit
System: MacOS 12.4
Python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33) [Clang 13.0.1 ]
Qt: 5.15.3
PyQt5: 5.15.4
NumPy: 1.23.0
SciPy: 1.8.1
Dask: 2022.6.1
VisPy: 0.10.0

OpenGL:
- GL version: 2.1 Metal - 76.3
- MAX_TEXTURE_SIZE: 16384

Screens:
- screen 1: resolution 1728x1117, scale 2.0
- screen 2: resolution 1920x1080, scale 1.0
- screen 3: resolution 1920x1080, scale 1.0

Plugins:
- console: 0.0.4
- napari-svg: 0.1.6
- scikit-image: 0.4.16

Thanks for the bug report @jduffield65!

Hello @jduffield65, could you install the current version of the main branch and test it there?
I think I fixed this issue some weeks ago :)
https://github.com/napari/napari/pull/4699

Confirmed that the above code works for me on current main. Can we close @brisvag ?


404 when saving post

- Do you want to request a feature or report a bug?
Bug
- What is the current behavior?
When I try to save a post, I get a warning saying Failed to persist entry: API_ERROR: Not Found
and the github API returns a 404.
- If the current behavior is a bug, please provide the steps to reproduce.
using this basic config with a metalsmith site:
backend:
  name: github
  repo: bequest/external-site
  branch: master

public_folder: "dist/images/uploads"
media_folder: "src/images/uploads"

collections:
  - name: "blog" # Used in routes, e.g. /admin/collections/blog
    label: "Blog"
    folder: "src/_posts/blog" # The path to the folder where the documents are stored
    create: true # Allow users to create new documents in this collection
    slug: "{{year}}-{{month}}-{{day}}-{{slug}}" # Filename template i.e. YYYY-MM-DD-title.md
    fields: # The fields for each document, usually in front matter
      - {label: "Layout", name: "layout", widget: "hidden", default: "blog"}
      - {label: "Title", name: "title", widget: "string"}
      - {label: "Publish Date", name: "date", widget: "datetime"}
      - {label: "Featured Image", name: "thumbnail", widget: "image"}
      - {label: "Body", name: "body", widget: "markdown"}

I am able to access the CMS at the netlify url, but cannot save after editing a post
- What is the expected behavior?
Post saves without error.
- Please mention your node.js, and operating system version.
Using netlify to deploy. No node version is set in package.json

I have similar problem

I was having this problem with editorial_workflow enabled. After disabling it, save button still won't work, without any errors in the console.
I have deleted OAuth Application from GitHub and deleted Github Provider. After resetting these, I was able to see some activity when clicking Save button.
FIrst of all These errors appears without any actions from my side.

When Clicking save button this happens:

@joefatora we never did follow up on this ticket - are you still having issues?
I've helped a number of folks with what you're seeing (including @almeynman), and so far it's always an authentication issue. The following causes have surfaced:

Running locally as <IP_ADDRESS> instead of localhost
Cookie problems (clear all site data for the domain where you're  having issues and reload)
Using an old version of the CMS
Misspelling the repo information in the config
Not having an oauth provider setup (eg. Netlify) when using the CMS on a live site (not running locally)

For anyone seeing this issue, visit us on Gitter for help.

Here's another authentication issue for the list that causes this 404 error.
I created an OAuth application on my GitHub user account but couldn't get netlify CMS to write to the repo until I deleted that and instead created the OAuth application on the settings page for the organisation that contains the content repo. The user logging in to netlify CMS is a Write collaborator on only the one repo that contains the website code and content.
Perhaps this is just a detail of GitHub OAuth that everyone but me already knows but it was a bit confusing and no docs seem to mention this tip for those of us new to this method of authentication within GitHub so I thought it worth mentioning here.


Changes to the doc & django integration

Relates to #934

Remove the Django DB router. I'm not convinced it was a good idea, and it will complicate things.
ProcrastinatePeriodicDefer should be readonly
add PROCRASTINATE_READONLY_MODELS setting to ease testing
Break Django doc into multiple docs
Add a django test doc with unit & integration fixtures snippets (promoting InMemoryConnector)

Successful PR Checklist:

[x] Tests

[ ] (not applicable?)

[x] Documentation

[ ] (not applicable?)

PR label(s):

[ ] https://github.com/procrastinate-org/procrastinate/labels/PR type%3A breaking 💥
[ ] https://github.com/procrastinate-org/procrastinate/labels/PR type%3A feature ⭐
[ ] https://github.com/procrastinate-org/procrastinate/labels/PR type%3A bugfix 🕵
[ ] https://github.com/procrastinate-org/procrastinate/labels/PR type%3A miscellaneous 👾
[ ] https://github.com/procrastinate-org/procrastinate/labels/PR type%3A dependencies 🤖
[x] https://github.com/procrastinate-org/procrastinate/labels/PR type%3A documentation 📚

the django reference doc is still failing


Add lasallebajio.txt

University Domain : https://www.lasallebajio.edu.mx/
IT courses and careers :
https://www.lasallebajio.edu.mx/oferta/oferta1.php?n=1&p=42
https://www.lasallebajio.edu.mx/oferta/oferta5.php?n=6&p=46
https://www.lasallebajio.edu.mx/oferta/oferta5.php?n=6&p=21
https://www.lasallebajio.edu.mx/oferta/oferta1.php?n=1&p=42

@HugoGomezArenas First stage of verification passed, official domain is lasallebajio.edu.mx as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@HugoGomezArenas Thank you for your patience while waiting for a review and processing your request. Pull request merged.

Thank you so much!!


Added concept of download conditions

PR Checklist

[x] I have read and/or are familiar with the contributor guidelines
[x] I have checked to make sure no existing PRs already satisfy the original requirements of this PR
[x] The commit messages in this PR are clear, and any WIP commits have been squashed
[x] The code passes lint checking
[x] The PR contains no spelling or grammatical errors
[x] Any relevant documentation has been updated
[x] I have included tests for my code

What this PR changes
It introduces the concept of download conditions.
This new API works in much the same way as fetch conditions, but every callback that's registered is evaluated once the headers for a resource have been fetched. This means the developer has access to the Content-Type and Content-Length etc.
This also means that we're removing the dreaded fetchWhitelistedMimeTypesBelowMaxDepth option AND in turn changing how the maxDepth option behaves. These changes should hopefully make sense to developers that now have very flexible ways of determining which resources to fetch and how.
Since I've added unit tests for download conditions, I've also added unit tests for fetch conditions (that previously didn't have any unit tests)
Rationale
See #249

So something I've been thinking about regarding this: would it make more sense to call these new download conditions "fetch conditions" instead, and the old fetch conditions something like "add conditions", or "queue conditions"?
There's kind of a conflict in meaning when adding these and already having a "fetch conditions" concept, when those conditions don't really control fetching, but rather queueing.
What do you think, @cgiffard?

I'm wondering whether they should actually be the same thing, but you can specify where in the request lifecycle the condition takes place, somehow.
🤔

That might make sense actually. But there are also the slight nuances. They're both filtering functions, but aside from that they're applied at different points in time, they also receive different arguments. Are there any examples where a project has used that kind of setup - one filtering concept that can be applied at different hooks? Just to see if there's some good inspiration, I mean

My best idea so far would be to register the filtering functions as event listeners, something like:
crawler.on("fetch", function(queueItem) {
    return false;
});

But that feels less intuitive. Although on the other hand, thinking of jQuery and similar libraries, returning false from an event listener means to prevent the default action for that event. But maybe the problem here is that it would only be possible to return false from some event listeners, and it might not be intuitive to tell which ones?

I kinda like that idea, but I agree it's unintuitive — it's not the default behaviour of EventEmitter and wouldn't be so discoverable. Furthermore it'd mean any checks would continue the existing requirement for synchronicity, and I'm wondering whether we need to move to asynchronous conditions.
But it looks nice.
I guess my best idea so far is something like this:
crawler.addCondition("queue", function(queueItem, queueItemReferrer, callback) {
    checkInMyMagicalDatabase(queueItem, function(err, areWeGoForLaunch) {
        callback(err, areWeGoForLaunch); // Broken out for clarity
    });
});

crawler.addCondition("download", function(queueItem, queueItemReferrer, response, callback) {
    if (somethingSomethingSomething(response)) {
        return callback(null, true);
    }

    doSomethingElseAsync(queueItem, function(err, areWeGoForLaunch) {
        callback(err, areWeGoForLaunch); // Broken out for clarity
    });
});

That's roughly equivalent to what we have now, but async. But is it actually understandable? Or just familiar?
The other way might be to make a QueueItem into a class, (ugh-ish) and attach a method to it like abortLoading() or bailOut() — that means, wherever it's at in the lifecycle, any event could instruct the crawler to just stop whatever it's doing with that queueItem (or flag it in such a way that it would be ignored by the next stage.) Something like:
crawler.on("fetchheaders", function(queueItem) {
    // Don't load the board reports
    if (queueItem.stateData.contentType.match(/application\/pdf/i)) {
        queueItem.abortLoading();
    }
});

crawler.on("queueadd", function(queueItem) {
    // We keep getting these junky URLs from our content management system
    if (queueItem.url.match(/\&amp\;/i)) {
        queueItem.abortLoading();
    }
});

Anyway, I'm not settled on anything yet. Happy to discuss or tease apart other proposals!
The main thing on my mind is that we want a relatively general system, which will enable people to easily implement their own logic without us having to have a whole bunch of really complex branching inside the crawler itself — and that's what download conditions are supposed to do — but what event might be next? At the moment, for instance, we don't fetch 404s, because mostly people don't want the data from 404 pages. But maybe somebody does, and we'll need a way to retrieve them. If we implement our logic as some kind of abstract condition — that could be removed with simple and general configuration, that figurative person who wants to retrieve 404 data will be able to do so without having to file an issue or patch the crawler.
Anyway, something for us to chew on.

A couple of additional thoughts:

Async conditions in defined places are easy to handle. But they should time-out — otherwise other people's code could hang the crawler really easily, and we'll get the blame!
A synchronous method that could be called at any time to abort a queue item is really intuitive, but could suffer from race conditions. Assume the user wants to abort loading an item, but their check is async — we can't just wait around for it without clear instruction. They might come back asking the item to be aborted, but it's already too late (maybe this isn't a problem and they can just filter it out later.)

Perhaps this is a problem-for-later(tm) and we can merge download conditions now, and have it on the roadmap to consolidate at some later point.

Interesting ideas! I kind of like the idea of expanding queueItems into a class with an abort method. This doesn't really cover the current fetch condition scenario however - except if the developer hooks into the discoverycomplete event, but at that stage the queue iten isn't really being aborted, it's just not being put in the queue.
And while there might be value in async conditions, this is something that could really slow down a crawl if applied to every incoming resource. I also like the simplicity of the current synchronous fetch condition API, it's rather intuitive.
But yeah, I agree that we should maybe postpone a bigger decision on this. Let's draft some future plans elsewhere and stick to the original plan here!
There's still the naming question though - do we rename the old fetch conditions now that we have these download conditions? As far as making goes, I think "fetch conditions" make more sense for these hooks. The old hooks could be called "queue conditions" maybe. But this also makes upgrading to 1.0 harder, and perhaps that isn't worthwhile just to have a clearer terminology for newcomers. I'm not sure myself..

Just to pool some thoughts on this, @lgraubner @xhmikosr @tmpfs @thecameronjones @hbakhtiyor, do you have an opinion?
Should we give the name "fetch conditions" to this new concept, and rename the old fetch conditions "queue conditions" or something similar?
Or should we keep the name "fetch conditions" for the current fetch conditions and call this new concept "download conditions"?

Nice. This looks similar to the change I had in another PR!

Good work. The names "fetch conditions" and "download conditions" seem fine for me as I can easily imagine what each one is doing.

Nice work! The terms fetch and download conditions work great for me. Honestly either way would be understandable.

Alright then, let's keep the name "fetch conditions" for the old ones, and "download conditions" for the new ones! @cgiffard do you want to do the merging?

I'll have another look at this one today. Just wanna thoroughly explore the possibility of race conditions before going ahead with this — at least from my first look through it seemed good though!

I just tried the downloadConditions feature and noticed that the "fetch404" "fetch410" and "fetchError" events won't be emitted if the download is denied, even though the response status is already known when the downloadConditions are evaluated. Is this the intended behavior?
My use case for download conditions is to make sure that the requests are successful without having to download the bodies of all resources. Right now, I would have to manually inspect the response in a downloadprevented callback to find out whether a request was successful. It seems unexpected to me that the fetch error events aren't emitted if the download is prevented. To my understanding, knowing the response code is a major advantage of a download condition over a fetch condition.

I think it's worth ensuring that those events are properly emitted, but in the interests of expediency I'd probably like to revisit that after merging this rather than before.

Great catch, @micschro! Would be worth to fix before 1.0.0, but I probably won't be able to fix it for a few days (am on vacation). If you can fix it on master, @cgiffard, then we can merge this first

I pushed a fix for the fetch404 etc. events issue. Now download conditions are only evaluated if the resource will actually be downloaded otherwise. Once again, thanks for looking into the changes, @micschro!

@fredrikekelund Thanks for merging! Sorry about not getting onto this sooner. I noticed a function where it appears we might be duplicating the functionality of Array.prototype.indexOf, but for reasons which were unclear to me — unless there's a specific reason why this was added, I might fix it on master?

@fredrikekelund Also wanted to thank you for the enormous amount of work you put in here — looks excellent. :)


Redis Connection issue

Similar to #1 if I use the docker-compose on the readme, I get this error and nothing seems to work
24-10-17 08:27:03 scriberr-1  | Error: connect ECONNREFUSED <IP_ADDRESS>:6379
2024-10-17 08:27:03 scriberr-1  |     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) {
2024-10-17 08:27:03 scriberr-1  |   errno: -111,
2024-10-17 08:27:03 scriberr-1  |   code: 'ECONNREFUSED',
2024-10-17 08:27:03 scriberr-1  |   syscall: 'connect',
2024-10-17 08:27:03 scriberr-1  |   address: '<IP_ADDRESS>',
2024-10-17 08:27:03 scriberr-1  |   port: 6379
2024-10-17 08:27:03 scriberr-1  | }

services:
  scriberr:
    image: ghcr.io/rishikanthc/scriberr:beta-0.2
    ports:
      - "3000:3000"
      - "8080:8080" #Optionally expose DB UI
      - "9243:9243" #Optionally expose JobQueue UI
    environment:
      - OPENAI_API_KEY=[my-key-here]
      -<EMAIL_ADDRESS>      - POCKETBASE_ADMIN_PASSWORD=password
      - REDIS_HOST=<IP_ADDRESS>
      - REDIS_PORT=6379
      - SCRIBO_FILES=/scriberr
    volumes:
      - ./pb_data:/app/db
      - ./scriberr_data:/scriberr

My apologies. Forgot to add redis to docker compose. Append this to your compose and try again
redis:
    image: redis:7-alpine
    volumes:
      - ./.dockerdata/redis:/data

So adding that I had the same error.  I chucked it into chatgpt, and it suggested I use redis instead of <IP_ADDRESS>.   Here's the chat.  That seemed to work for me.

The error message connect ECONNREFUSED <IP_ADDRESS>:6379 indicates that the scriberr service is unable to connect to Redis on the specified host (<IP_ADDRESS>) and port (6379).
In your docker-compose.yml, you're defining two services: scriberr and redis. However, the environment variable REDIS_HOST=<IP_ADDRESS> is pointing the scriberr service to try and connect to Redis on localhost (within the scriberr container), but Redis is running in a separate container.
Containers in a Docker network communicate with each other using their service names, not localhost. To fix this issue, update the REDIS_HOST environment variable in the scriberr service to use the service name of the Redis container (i.e., redis).
Here’s the updated part of your docker-compose.yml:

services:
  scriberr:
    environment:
      - REDIS_HOST=redis
      - REDIS_PORT=6379
This will allow the scriberr service to communicate with the Redis service properly across containers. Docker Compose automatically sets up a network for the services, and the service name (redis) can be used as the hostname within the network.


Use another augmented assignment statement

:eyes: Some source code analysis tools can help to find opportunities for improving software components.
:thought_balloon: I propose to increase the usage of augmented assignment statements accordingly.
diff --git a/src/emcee/autocorr.py b/src/emcee/autocorr.py
index 7962451..bb53645 100644
--- a/src/emcee/autocorr.py
+++ b/src/emcee/autocorr.py
@@ -13,7 +13,7 @@ def next_pow_two(n):
     """Returns the next power of two greater than or equal to `n`"""
     i = 1
     while i < n:
-        i = i << 1
+        i <<= 1
     return i

:thought_balloon: Do any factors hinder the wider application of functionality which became generally available with Python 2?


[SR-9696] Confirm some errors to LocalizedError

closes SR-9696
https://bugs.swift.org/browse/SR-9696
ArgumentParserError and ArgumentConversionError make confirming to LocalizeError.
This PR may help to read error when using these errors as directly.

Thanks!

@swift-ci smoke test


Missing assignment for output?

On the line linked below, I noticed that the output is not assigned to a variable. Should it be assigned to x or a new variable c3? I wonder, because it doesn't look like the operation is inplace, but perhaps I am mistaken.
https://github.com/czbiohub/noise2self/blob/2e351cb69ba9802cfee275be187aeee6bc17fda6/models/babyunet.py#L32

Great catch. Fixed in b306ca7.


Fix incorrect sentinel value in YetiDriver

Describe your changes
The sentinel value used in wait_for_caps() to determine if the for loop reaches the end without breaking before is incorrect. The for loop exits when i reaches 50, not 49.
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] I have made corresponding changes to the documentation
[x] I read the contribution documentation and made sure that my changes meet its requirements

@matteocarnelos Could you please sign-off your commit? Otherwise we cannot merge, this is a LF requirement

@corneliusclaussen Done!


Fixed: PLUGIN-499

JIRA ticket: https://cdap.atlassian.net/browse/PLUGIN-499

Is this also affecting bq sink and bq multi sink? If so, we should fix it for those plugins too
What are the test cases that were performed for these change?

This PR is for ticket PLUGIN-499 BigQuery source does not delete the temporary tables created while exporting data to GCS, so our focus was only for the BigQuery Source.

The test cases that we performed for these changes:

Use BigQuery source without filter and table without partition: No temporary table gets created.
Use BigQuery source with filter: Temporary table gets created and gets deleted at the end.
Use BigQuery source for time partitioned table: Temporary table gets created and gets deleted at the end.
Use BigQuery source for VIEW: Temporary table gets created and gets deleted at the end.
Use BigQuery source for MATERIALIZED_VIEW: Temporary table gets created and gets deleted at the end.

For 1, it applies to bigquery sink as well. Please file a separate jira for it:
https://github.com/data-integrations/google-cloud/blob/develop/src/main/java/io/cdap/plugin/gcp/bigquery/sink/BigQueryOutputFormat.java#L365

/gcbrun

Is this also affecting bq sink and bq multi sink? If so, we should fix it for those plugins too

This PR is for ticket PLUGIN-499 BigQuery source does not delete the temporary tables created while exporting data to GCS, so our focus was only for the BigQuery Source.

Can you please check if this issue is also applicable to BQ sink and multi sink? If yes, we can open JIRAs to track fixing the sinks.

Is this also affecting bq sink and bq multi sink? If so, we should fix it for those plugins too

This PR is for ticket PLUGIN-499 BigQuery source does not delete the temporary tables created while exporting data to GCS, so our focus was only for the BigQuery Source.

Can you please check if this issue is also applicable to BQ sink and multi sink? If yes, we can open JIRAs to track fixing the sinks.

This is the JIRA to track fixing for sink and multi sink: https://cdap.atlassian.net/browse/PLUGIN-525