Created account export feature

There are still two unfinished steps:

Allow anonymous users to export their accounts by providing an email field
Make the navigation sidebar mobile-friendly

Apart from that, more descriptive error messages should be added to the export result.
Fixes #1821

maybe not in scope for this pr, but we should think/worry about attachment size limitations, it could easily happen that we are unable to send the email if the export file is too big for the recipients mailbox/server and thus the export will never reach the user(specified email), i would change this in favor of sending a download link in the email. this link could grant access to a (one time?) download of the export, if its accessible more than once we should at least invalidate/delete the entry after a certain date or if the user has requested a new export.the benefit of this approach would be that we could link the export file from within the owner webapp as well (and possibly deny subsequent exports for some time, as this is probably a very costly operation)

We should definitely worry about this, but solving that problem would mean creating a download manager system like google has it for data exports.


Add LightningDataModule.load_from_checkpoint to load datamodules directly from checkpoint

What does this PR do?

Fixes #12062
Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[ ] Did you write any new necessary tests? (not for typos and docs)
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] Did you list all the breaking changes introduced by this pull request?
[ ] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃
cc @carmocca @awaelchli @borda @ananthsub @ninginthecloud @jjenniferdai @rohitgr7

This is a nice feature addition!
I have one concern question where I believe we should look for a decision before merging the PR. In #10375 I propose to deprecate support for saving and restoring from a singular hparams container passed as input argument. If we decide to do this deprecation in this release, it would be better if the implementation here would already adopt the new restricted (simpler) way of parsing the args. If you think this is less convenient, then I'm also good with this. I just want to avoid us introducing a new feature with a broader scope and immediately deprecate partial functionality.
cc @carmocca

let's do it in this release in a follow-up. I have seen similar issues related to it on slack so would be worth restricting the containers here.


RDP_Nesting Template Issue - Doesn't detect

There appears to be an issue with the RDP nesting template. I rdpd to my server(KayWinVm) using my credentials kaywinvm\kayadmin. From KayWinVM I rdpd to another server (Windows10Pro) using credentials windows10pro\kayadmin. However this did not trigger any alert. The both servers are logging fine to the sentinel workspace.

@kay07949 - thanks for the feedback.  I believe this is because the detection requires that he same account be used.  This detection defines the account as the Domain\UserName, in your case it appears these are different accounts, meaning the domains are different.  Are you able to use the same domain\username for the RDP hop?  If so, can you verify you get the expected result then?

I will give that a try thanks !

@kay07949  - Wondering if you have had a chance to test further?

Hi,
I tested and it works well!
Thanks,
Kay


Drag and drop from operation not working

I trying to drag the "Magic" from operation to recipe but it doens't work. it only work when i refresh the website plus only work once. (Means i can't add more than ONE recipe).
Cyberchef version : 9.21
Browser: Firefox 83.0

Seems to be working ok for me. I'll close this for now since no one else is reporting the same thing but if anyone else is having this issue, comment here and we will reopen it.


BUG: Incorrect unescaping of inlined props in hydrated component

@nickreese
I've found an error when elder can't parse props for the hydrated component.
Let's see the use case when we have a prop which contains a string with escaped double quotes:
{ prop: "This is a string with \"escaped\" quotes" }

In src/partialHydration/inlineSvelteComponent.ts there's a function to escape unwanted characters:
export function escapeHtml(text: string): string {
  return text
    .replace(/&/g, '&')
    .replace(/</g, '<')
    .replace(/>/g, '>')
    .replace(/"/g, '"')
    .replace(/'/g, ''');
}
...
  const replacementAttrs = {
    class: '"ejs-component"',
    'data-ejs-component': `"${name}"`,
    'data-ejs-props': `"${escapeHtml(JSON.stringify(props))}"`,
    'data-ejs-options': `"${escapeHtml(JSON.stringify(hydrationOptions))}"`,
  };

After JSON.stringify() the prop looks like this:
'{"prop":"This is a string with \\"escaped\\" quotes"}'

After applying escapeHtml():
'{"prop":"This is a string with \\"escaped\\" quotes"}'

Then in src/partialHydration/mountComponentsInHtml.ts there's a code to unescape characters and parse the prop back to JSON:
export const replaceSpecialCharacters = (str) =>
  str
    .replace(/\\\\n/gim, '\\n') // <-- ???
    .replace(/"/gim, '"')
    .replace(/</gim, '<')
    .replace(/>/gim, '>')
    .replace(/'/gim, "'")
    .replace(/\\"/gim, '"') // <-- This breaks the unescaping
    .replace(/&/gim, '&');
...
    try {
      hydrateComponentProps = JSON.parse(replaceSpecialCharacters(match[3]));
    } catch (e) {
      throw new Error(`Failed to JSON.parse props for ${hydrateComponentName} ${match[3]}`);
    }

Parsing back to an object with JSON.parse will trigger an error, because replaceSpecialCharacters () incorrectly reverts the prop string:
{"prop":"This is a string with "escaped" quotes"}

BANG! Escaped quotes are gone, JSON.parse throws an exception.
I think both .replace(/\"/gim, '"') and .replace(/\\n/gim, '\n') should be removed. Both escapeHtml() and replaceSpecialCharacters() should be revertable, i.e. convert the same set of charactes to escaped ones and back.

@dmitrysmagin Somehow GitHub didn't notify me of this issue or I missed the notification. 👼
Interestingly I encountered this on ElderGuide.com this past week and on my private branch of Elder.js it is fixed.
I've added robust tests to make sure the input and output are the same of these two functions and pushed in 1.6.14 let me know how you get on with them.


Confused results when multiple sockets are active

Refs are managed per socket but the self callbacks dictionary is keyed only be the ref. This means that if one opens multiple sockets, the self callbacks dictionary can overwrite the handler for one push or join message with the handler for a push or join message on another socket. Much ugliness follows.
The fix for this appears to be to key the self callbacks dictionary by (Ref, Endpoint) instead of just Ref. Pull request coming.

I see the problem....
Not sure you can use a tuple as a dictionary key, so you would probably need to have some nested dicts, i.e. first keyed Endpoint then by Ref.

Tuples of comparable items are comparable and hence are viable keys. It's
probably best to do it as (Ref, Endpoint) as opposed to (Endpoint, Ref)
because the refs are more likely to be different and hence the comparisons
will be ever so slightly faster on average.
Mark
On Fri, Aug 18, 2017 at 1:09 AM, Sascha Timme<EMAIL_ADDRESS>wrote:

I see the problem....
Not sure you can use a tuple as a dictionary key, so you would probably
need to have some nested dicts, i.e. first keyed Endpoint then by Ref.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saschatimme/elm-phoenix/issues/33#issuecomment-323289109,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AP4jHb_ZGKBAkIMVQUKBtEa8wg_xeqzQks5sZUbBgaJpZM4O6sgC
.

@markhamburg I know it was a while ago, but I'd be interested in knowing if this PR (https://github.com/saschatimme/elm-phoenix/pull/40) fixes the problem you were having. I was able to reproduce it locally with a contrived example but it'd be nice to have some confirmation from a real-world use case.

#40 closed that


refactor(connector): make source connector's metrics guarded

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
fix #17529

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[ ] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

I observed a wired curve online as following

It's probably caused by the usage of Guarded metrics in this PR, which is incorrect.
"Guarded" means that when drop() is called, it removes the label from the Prometheus client.
This PR changes with_label_values to with_guarded_label_values, which results in the metrics being reported once and then immediately dropped.
The correct approach is to place GuardedMetrics in a location where its lifecycle matches that of the object associated with the metrics (for example, placing the metrics of a Source in the SourceExecutor or an internal struct within it). This way, when the SourceExecutor is dropped, the metrics are also dropped simultaneously.


Setup to test IAP on Android (currently returning zero products)

Hi all - i'm struggling with how to setup on Android. I've implemented the InAppBilling plugin on Xamarin Forms and have tested it on an iPhone SandBox which it seems to be working OK (retrieve products, purchase product).
However, when testing on Android device, GetProductInfoAsync it is always returning empty. In the Google Play Console, I have setup the products (Subscriptions) with the correct Product ID and the status is showing as Active.
I am using Visual Studio Mac and have deployed the app to my Android device in both Debug and Release modes but in both cases it is still returning zero products.
I must be missing some config or something somewhere but I cannot figure out what it is - any suggestions on what might be the missing step?

Hi, how did you get subscriptions working on iOS, I've been trying to test it but GetProductInfo returns null like product wasn't available. I follow the steps on https://jamesmontemagno.github.io/InAppBillingPlugin/TestingAndTroubleshooting.html but nothing works. Thanks in advance.

Unable to reproduce working fine for me. https://www.revenuecat.com/blog/the-ultimate-guide-to-subscription-testing-on-ios


null at end of text files sometimes

Solution loop through, if last 4 is null remove last 4 chars

#11 solves this


Use immutable support methods

I really appreciated the implementation of Immutable.js support you have done, but that had a little awkward implementation which did not use methods offered by Immutable.js itself and it caused an error. This PR makes use of them instead of original one for fixing errors.

Hi @izumySy! I appreciate the bug report, but the code in your PR will fail for any users that are not using Immutable. Can you give me more details about the error (or better yet, a repro case)? List should have .is() defined on it - is there some other way to use List where that method won’t be defined?

At first, I found that with your implementation, I get this error.

This is because is method is not an member of List class, and it is just an static method accepting two Immutable objects as arguments. I tried to fix the error with a correct usage of it, but now I found that my PR fails. I am now trying to manage it. Sorry for that!

See implementation of Immutable#is. This at first uses triple equality. I think equals method might be replaced with it.
https://github.com/facebook/immutable-js/blob/master/src/is.js

Again, this pulls in the entirety of immutable.js as a dependency, and puts the burden on the consumer to use a tree shaking optimizer to remove it. We don't want to do that.
Please:

Remove the import from immutable
Copy the whole .is function that you linked into react.ts, and rename is to equals
Where the function calls isValueObject, instead call the isImmutable function that I cribbed from Immutable

Alternatively, install immutable-is as a dependency and use that instead of copy+pasting code. Whatever you prefer!

And a third option: undo all of your changes, and just update 1 line in my code:
- return a.is(b)
+ return a.equals(b)

this pulls in the entirety of immutable.js as a dependency, and puts the burden on the consumer to use a tree shaking optimizer to remove it. We don't want to do that.

I see. Got the point. I will remove that from deps.

Alternatively, install immutable-is as a dependency and use that instead of copy+pasting code. Whatever you prefer!

I think this might be the best and neatest option for us. I will take this up.

Now I removed immutable.js from deps and added immutable-is 💪

Nice work @IzumiSy! Can you also do what I should have done, and add test for a few immutable collections (or all of them!) to make sure that they work correctly with the new behavior? Ie. Create a store with Immutable properties, update them, make sure that React caused the DOM to update.

Done. You can see if my test works or not by switching is(a, b) around a === b.

Finally we got the cleanest changes. Thanks!

And a third option: undo all of your changes, and just update 1 line in my code

I missed this comment 🙏

Published 3.2.1


Please release the test code

Hi @aupendu , many thanks for your interest in our work. We plan to release the code before the conference.


Warn rather than error for deprecations

Description
Would be nice to have the deprecations warn rather than error until the next major version due to transient dependency inclusion.
Related PRs
List related PRs against other branches:

branch
PR

feature/multi-column-sort
#619

Impacted Areas in Application
error message for deprecations.

Thanks! 😊


Such VTK, many 3D! So render, very mesh, much analyze!

Downloaded asset works. Good to merge.


New version: ONNX v0.1.1

Registering package: ONNX
Repository: https://github.com/FluxML/ONNX.jl
Created by: @ayush-1506
Version: v0.1.1
Commit: 3244553c6693ac0f753ccbf1a9147bf0af4f0fbf
Reviewed by: @ayush-1506
Reference: https://github.com/FluxML/ONNX.jl/issues/23

Please add (at least)
[compat]
julia = "1"

to your project file and retrigger the bot.


Add macOS M1 runner testing configuration for PRs and scheduled tests

Description
This PR configures the workflows to include tests for the new GitHub Actions M1 runners.
Type of change
Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

[ ] New feature (non-breaking change which adds functionality)
[ ] Optimization (back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)

Key checklist:

[x] No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
[x] All tests pass: $ python run-tests.py --all (or $ nox -s tests)
[x] The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).
Further checks:

[x] Code is commented, particularly in hard-to-understand areas
[x] Tests added that prove fix is effective or that feature works

Python 3.8 and Python 3.9 are not yet available in the version manifest, I had flagged this earlier today at https://github.com/actions/setup-python/issues/808

Looks like they did, feel free to approve and merge


what is envHasBigInt64Array good for?

Why do you have this?
https://github.com/epoberezkin/fast-deep-equal/blob/a8e7172b6c411ec320d6045fd4afbd2abc1b4bde/src/index.jst#L6
is it being used for something?

duplicate


Add vsock support

Signed-off-by: Hui Zhu<EMAIL_ADDRESS>
@karthikravis

Replaced by https://github.com/grpc/grpc/pull/21121.


SMS body doesn't fill on Android 10 (emulator)

I am trying to pre-fill sms body in my flutter application, but it doesn't work on Android 10 (SDK 29) emulator. I  tried older versions and it works just fine, on iOS aswell.
String url = 'sms:12345?body=message';
await launch(url);

Do you think you can solve this issue?
Thank you.

Hi @TenPetr
What flutter version do you use?
I tried with v1.12.13+hotfix.8 and Android 10 (SDK 29) emulator, pre-fill sms body works

Hi @TenPetr
What flutter version do you use?
I tried with v1.12.13+hotfix.8 and Android 10 (SDK 29) emulator, pre-fill sms body works

I am using v1.12.13+hotfix.7 right now.

@TenPetr
do you use the last version of url_launcher 5.4.2?

@TenPetr
do you use the last version of url_launcher 5.4.2?

I am currently using version url_launcher: 5.4.1, I will try to update package.

True, with url_launcher 5.4.2 and Android 10 the sms body is not filled
(I couldn't reproduce it before because mistakenly was running with Android 9)

True, with url_launcher 5.4.2 and Android 10 the sms body is not filled
(I couldn't reproduce it before because mistakenly was running with Android 9)

Yes, same here. I have just updated the package and it doesn't work.

same issue here

@TahaTesser any update

Using url_launcher 5.4.11 and seeing the same issue. Has anyone been able to resolve this?

Facing the same issue, with the following configuration :
url_launcher: ^5.5.0
Flutter 1.20.0-7.3.pre
Running on Android 10 sdk 29 emulator
Is there any possible workaround or fix ?

It looks like it's a bug in Android 10, which ignores the body parameter if the URL scheme doesn't start with two forward slashes (//) after sms:. You can test this by going to this test site on your Android device and playing around with de URL from the 'TXT TO FRIEND' button.
Adding the two slashes to the URL, which would get you sms://12345?body=message, should be able to fix your problem.
Tests with Android API 30 indicate this problem to be solved.

@valconetti it will work on andriod 9/8?

Hi @kw2019ltd
I checked it, and it should work on Android 8 and 9, too.

Closing per @valconetti 's comment. url_launcher seems to be properly forwarding the URL to the platform, abstracting the different URL schemes is out of scope for url_launcher, users should make sure to use URL schemes that are valid for the underlying platform.


New Renderer interface and OpenGL 4.3 renderer

What this implements
This PR will implement a new interface for the renderers, and a storage for the compiled graphics apis.
Most of the changes are in include/Engine/Renderer and src/Engine/Renderer
Renderer interface
This interface currently has:

Abstract gp1::renderer::RendererData
Abstract gp1::renderer::DynamicRendererData extends gp1::renderer::RendererData adds m_Dirty, m_Editable and m_Dynamic variables.
Abstract gp1::renderer::Mesh extends gp1::renderer::DynamicRendererData
gp1::renderer::StaticMesh extends gp1::renderer::Mesh
Abstract gp1::renderer::Texture extends gp1::renderer::DynamicRendererData
gp1::renderer::Texture2D extends gp1::renderer::Texture
gp1::renderer::Texture2DArray extends gp1::renderer::Texture
gp1::renderer::Texture3D extends gp1::renderer::Texture
gp1::renderer::TextureCubeMap extends gp1::renderer::Texture
gp1::renderer::Shader
gp1::renderer::ShaderProgram extends gp1::renderer::DynamicRendererData
gp1::renderer::Material extends gp1::renderer::RendererData
gp1::renderer::ReservedUniformBuffers extends gp1::renderer::RendererData
Abstract gp1::renderer::Uniform extends gp1::renderer::DynamicRendererData
gp1::renderer::Uniform[Type] extends gp1::renderer::Uniform where [Type] is one of the names in the enum gp1::renderer::EUniformType, but the vector and matrix types have their underlying data type shrunk to one character.
gp1::renderer::UniformBuffer extends gp1::renderer::RendererData
Abstract gp1::renderer::DebugRenderer
Abstract gp1::renderer::Renderer

The function [TypeName]::Create(); where [TypeName] is the name of the type without [ or ] is used to create a new instance of the type extending gp1::renderer::RendererData excluding gp1::renderer::DynamicRendererData but including those extending it, optionally you can also use the function gp1::renderer::Renderer::Create[TypeName]();, but it is not recommended.
So to create a gp1::renderer::StaticMesh you would call gp1::renderer::StaticMesh::Create(); and for OpenGL it would return a new instance of gp1::renderer::opengl::OpenGLStaticMesh.
The function gp1::renderer::Renderer::IsCompatible(); can be used to check if the renderer is compatible on the current system.
This interface as seen uses inheritance to implement per graphics api functions and variables, thus it should not require RTTI to work.
gp1::renderer::ReservedUniformBuffers is used to store global uniforms like the camera's matrices.
Graphics API storage
The class gp1::renderer::Renderers is used to store graphics apis at the start of the application.
And to retrieve and construct one of the renderers you can call one of the following:

s_Renderers->GetBestRenderer(); This gets the best compatible renderer using each entry's priority from high to low and using the IsCompatible(); function of the renderers.
s_Renderers->GetRenderer(const std::string&); This gets the renderer of the given name.

Application
I have added the ability to set the renderer to use from an .ini file located in Config/Application.ini from the run path.
[Renderer]
Selected = best

the options are "best" for the best compatible renderer and the names of the renderers listed below.
This is the list of the current renderers:
API: Name, Priority

OpenGL: "opengl", 0
Vulkan: "vulkan", 1

Still reviewing most of the code, but question for you: Why the use of shared_ptr everywhere? That can be a bit dangerous as it can hide bugs where we operate on an object's member after the object has been destroyed without realizing it.
In general, cases where a class member should be a shared_ptr are rare. If you have a bunch of objects that all need access to some resource, and you want that resource to be destroyed when the last of those objects is destroyed, then it makes sense. Otherwise, the object that controls the lifetime of the object should have a unique_ptr, and objects that just need a reference to it can either access it from the owning object, or store it as a raw pointer.

How can shared_ptr be dangerous when you can't get a shared_ptr to a destroyed object when using it in a normal manner, i.e. not using the .get() function.
But in general I wanted to test it against other ways, so therefore they might all become regular pointers after I test it's performance compared to shared_ptr.

That's exactly why shared_ptr is dangerous. It hides bugs. If we're accessing something after it should've been destroyed, we won't know. That's a problem. We WANT things to crash if we try to access a destroyed object. That means there's a fundamental problem in our code. If a class's member should be destroyed when it is destroyed, it should be a unique_ptr.

This PR looks like a solid starting point besides the above issue.

I haven't had much time to do any work, I have been doing other stuff for a bit.
But now I'll try to remove all std::shared_ptrs and convert them to raw pointers.
Maybe I will do what you told me with using std::unique_ptrs and just passing around raw pointers everywhere, but idk.

Sure thing, not trying to say there can be no std::shared_ptrs, just make sure that the ptr type we're using makes sense with the lifetime and ownership we're looking for.

Yeah, meshes should probably be shared_ptr as it'll make it much easier to cache and reuse them.

Yeah that's why I sorta tried out std::shared_ptrs so I wouldn't have to deal with their lifetime and ownership, but there's obviously some problems with them.
So I say I'll see what the fps gain would be with 10000 textured spinning quads rendered separately using raw pointers and if it's small like below a few milliseconds I would say that using those std::shared_ptrs would be fine as long as people don't store std::shared_ptrs in classes/structs that only need to know they might exist (use std::weak_ptr in those cases) and that we don't make cyclic dependencies with std::shared_ptrs which are always bad.
Windows Release mode 10000 textured spinning quads:
std::shared_ptr: ~36.2ms render latency
smart_pointers::shared_ptr: ~36.3ms render latency

Here's a reference of what the render latencies currently are. (smart_pointers::shared_ptr was my own attempt at a non thread safe shared_ptr type)

My concern has nothing to do with performance. It’s about lifetime and
hiding errors.
If we want to use shared_ptr by default and just be very tight about taking
weak_ptrs when appropriate, that works too.
On Sat, May 29, 2021 at 17:18 Markus @.***> wrote:

Yeah that's why I sorta tried out std::shared_ptrs so I wouldn't have to
deal with their lifetime and ownership, but there's obviously some problems
with them.
So I say I'll see what the fps gain would be with 10000 textured spinning
quads rendered separately using raw pointers and if it's small like below a
few milliseconds I would say that using those std::shared_ptrs would be
fine as long as people don't store std::shared_ptrs in classes/structs
that only need to know they might exist (use std::weak_ptr in those
cases) and that we don't make cyclic dependencies with std::shared_ptrs
which are always bad.
Windows Release mode 10000 textured spinning quads:
std::shared_ptr: ~36.2ms render latency
smart_pointers::shared_ptr: ~36.3ms render latency
Here's a reference of what the render latencies currently are. (
smart_pointers::shared_ptr was my own attempt at a non thread safe
shared_ptr type)
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/TheChernoCommunity/GameProject-1/pull/38#issuecomment-850900301,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC5FTUQZSMCGXTO26MIAZ5LTQFK25ANCNFSM443I5P6A
.

I should say that the std::shared_ptrs owners are the caller.
So for instance calling gp1::renderer::StaticMesh::Create(); will obviously create a new StaticMesh but the caller is now the owner of that static mesh, and the renderer only stores a std::weak_ptr to it so it knows it's been created by the renderer and knows that it might exist.
Also when creating a new entity and attaching it to a scene, the caller of those functions has full ownership of them, so this means that it's up to the caller when the entity should get removed from the scene (which happens automatically the next time Update() is called on the scene).
But I can obviously see that the entity wouldn't get removed from the scene until all std::shared_ptrs to it are destroyed, but that could be fixed at some other point in time.

Update: After battling access violations and stuff not loading properly it works. The downside is that the fps didn't increase whatsoever in release mode (debug mode got increased by 20 fps lol), so should I change the std::shared_ptrs in the renderer branch, or should we just stay with std::shared_ptrs until they really make problems.

Again, this was not about performance at all, although shared_ptr can have significant performance implications. It's about safety and ownership. The two most popular C++ style guides on this subject:

R20 and R21 of the ISO C++ Style Guide
Ownership and Smart Pointers in the Google C++ Style Guide

This is because shared_ptr, when misused, quickly becomes a hidden global variable. This makes things difficult to reason about and bugs difficult to diagnose. Using shared_ptr does NOT mean, "I'm going to use this in more than one place," but instead, "I want this resource to be destroyed precisely when all of these scopes that accessed it end." Using shared_ptr when you don't intend to have actual shared ownership of an object and its lifetime is wrong and bad, period. The idea that you should never use raw pointers is taking the wrong lesson from modern C++ and smart pointers. This is a particularly egregious case, because most of the resources that are shared_ptrs here aren't even being mutated, meaning they could just be const T*s.
Everything I've said above, however, doesn't matter if there's just a reason I'm not seeing that every entity that reads a mesh needs mutate access and control of its lifetime, for example. Wouldn't it make more sense to have a separate class that owns the mesh, so that every object that needs read access to a particular mesh can simply get a const Mesh* to it, and we can have direct and explicit control over the resource's lifetime?
All that said, it looks like I'm not convincing anyone here. So while I maintain that the above is important and accurate, I get that this is a blocker and things need to keep moving. So if a third party says this looks good, then I'll merge it and we can use it as a starting point.

Alright give me a few days so I can change them to std::unique_ptrs and make it remove itself from the areas where I've used std::weak_ptrs to indicate that the object might exist.

To be clear, if there's a place where you're sure shared_ptr makes sense and I'm not seeing it, please don't change that, just explain. I'm not saying never use shared_ptr, just that unique_ptr should be the default unless shared ownership is truly needed.

Here we go if you see anything you want changed just tell me :)

In the interest of keeping things moving, I'm going to avoid nit picking on this PR any more since it's a big one and lots of work is dependent on this. Overall things look good, so we'll deal with any issues in future PRs.


Should we delete the archived repos?

It's been a while since we made this project a monorepo. The old archived repos are just cluttering up the repos list now, and all their commit history is in the monorepo anyway. Time for some spring cleaning?

Done


New /settings/ endpoint

closes #1659
Checklist before merging

[ ] I added an appropriate entry to CHANGELOG.md
[ ] I merged main into the current branch.

How does this look when we have an OAuth client: are we ever printing their SECRET?
The reason I ask is that the key (OAUTH_CLIENTS_CONFIG) does not include the SECRET or PASSWORD strings.


[BUG] Upgrade fails after 4th node

Describe the bug
During a 1.1.1 -> 1.1.2 upgrade one of our nodes locked up and rebooted.
This caused the upgrade to fail with some nodes running 1.1.1 and others running 1.1.2 and different versions of kubernetes.

What is the procedure to restart the upgrade? The upgrade button actually offers us the option to upgrade to v1.2.1 but we do not thing that will work with v1.1.1 nodes still on the system.
To Reproduce
Steps to reproduce the behavior:

Execute a v1.1.1 upgrade to v1.1.2.
Reboot a node that locks up via idrac
Upon reboot the entire upgrade fails leavling the cluster in an unexpected state. It is working but we cannot finish the upgrade.

Expected behavior
A successful upgrade to v1.1.2 that is able to recover, or a just drop the one failed node and continue the upgrade for the others.
Support bundle
Available upon request, we have it in onedrive and can share it to you.
Environment

v1.1.1 -> on the way to v1.1.2
BareMetal Epyc Dell R7xx series
14 nodes 128 core with 1TB of memory and 3 NVME drives on each server.

Additional context
We had a tough time doing this upgrade and did over the course of two days eventually resolve issues with promethous and alert monitoring. Once those were resolved the process started back up and we made progress.

Can you send a support bundle? That can help us determine the cluster state.

Can you send a support bundle? That can help us determine the cluster state.

Support bundle has been sent to the harvester-support-bundle email address at suse

@Sepiidae confirm we received it. @starbops can you help check the bundle? Thanks!

Hi @Sepiidae! Here's my observation from the support bundle:

Current situation: The upgrade was ended with a failure. All control-plane nodes were upgraded successfully, but the embedded Rancher Manager still waits for the harvester-l8bht (worker) node to be uncordoned, and it blocks the user from starting the upgrade again.
$ kubectl get clusters.provisioning.cattle.io/local -n fleet-local -o yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
metadata:
  annotations:
    objectset.rio.cattle.io/id: provisioning-cluster-create
    objectset.rio.cattle.io/owner-gvk: management.cattle.io/v3, Kind=Cluster
    objectset.rio.cattle.io/owner-name: local
    sim.harvesterhci.io/creationTimestamp: "2022-12-22T02:06:19Z"
  creationTimestamp: "2022-12-22T02:06:19Z"
  finalizers:
  - wrangler.cattle.io/provisioning-cluster-remove
  - wrangler.cattle.io/rke-cluster-remove
  generation: 1
  labels:
    objectset.rio.cattle.io/hash: 50675339e9ca48d1b72932eb038d75d9d2d44618
    provider.cattle.io: harvester
  name: local
  namespace: fleet-local
  resourceVersion: "3152"
  uid: deab9647-de9c-44e5-9db4-0fb339f20bb7
spec:
  kubernetesVersion: v1.24.11+rke2r1
  localClusterAuthEndpoint: {}
  rkeConfig:
    provisionGeneration: 1
    upgradeStrategy:
      controlPlaneDrainOptions:
        deleteEmptyDirData: false
        disableEviction: false
        enabled: false
        force: false
        gracePeriod: 0
        skipWaitForDeleteTimeoutSeconds: 0
        timeout: 0
      workerDrainOptions:
        deleteEmptyDirData: false
        disableEviction: false
        enabled: false
        force: false
        gracePeriod: 0
        skipWaitForDeleteTimeoutSeconds: 0
        timeout: 0
status:
  clientSecretName: local-kubeconfig
  clusterName: local
  conditions:
  - status: "True"
    type: Ready
  - lastUpdateTime: "2022-12-22T02:06:19Z"
    status: "False"
    type: Reconciling
  - lastUpdateTime: "2022-12-22T02:06:19Z"
    status: "False"
    type: Stalled
  - lastUpdateTime: "2023-12-21T18:55:55Z"
    status: "True"
    type: Created
  - lastUpdateTime: "2023-12-21T19:30:18Z"
    status: "True"
    type: RKECluster
  - status: Unknown
    type: DefaultProjectCreated
  - status: Unknown
    type: SystemProjectCreated
  - lastUpdateTime: "2022-12-22T02:06:34Z"
    status: "True"
    type: Connected
  - lastUpdateTime: "2022-12-22T02:07:58Z"
    status: "True"
    type: Provisioned
  - lastUpdateTime: "2023-12-21T19:30:18Z"
    message: 'uncordoning worker node(s) custom-4e46b8d06b63: waiting for uncordon
      to finish'
    reason: Waiting
    status: Unknown
    type: Updated
  observedGeneration: 4
  ready: true

Root cause: The pre-drain job for the harvester-l8bht node failed. The Job logs were missing, so it's hard to tell what exactly caused the failure. However, I noticed the RKE2 of the node was upgraded to v1.24.11, which is quite peculiar. And from the rancher-8584878865-2kmtz Pod's log, we could see the relevant lines showing it's upgrading the node's RKE2:
...
2023-12-21T18:55:52.579589010Z 2023/12/21 18:55:52 [INFO] [planner] rkecluster fleet-local/local: waiting: draining worker node(s) custom-4e46b8d06b63: draining node
2023-12-21T18:55:55.666820548Z 2023/12/21 18:55:55 [INFO] [planner] rkecluster fleet-local/local: waiting: configuring worker node(s) custom-4e46b8d06b63: waiting for plan to be applied
...
2023-12-21T18:55:59.478504815Z 2023/12/21 18:55:59 [INFO] [planner] rkecluster fleet-local/local: waiting: configuring worker node(s) custom-4e46b8d06b63,custom-564741db1fda,custom-869a9465598d and 7 more
...
2023-12-21T18:57:16.280398319Z 2023/12/21 18:57:16 [INFO] [planner] rkecluster fleet-local/local: waiting: uncordoning worker node(s) custom-4e46b8d06b63: waiting for uncordon to finish
...

How to initiate the upgrade again
We need to resolve the current stuck on the embedded Rancher Manager by manually assisting it to complete the upgrade workflow. For detailed steps, please refer to this comment. After that, we could start the upgrade again.

Hi @Sepiidae! Here's my observation from the support bundle:

Current situation: The upgrade was ended with a failure. All control-plane nodes were upgraded successfully, but the embedded Rancher Manager still waits for the harvester-l8bht (worker) node to be uncordoned, and it blocks the user from starting the upgrade again.$ kubectl get clusters.provisioning.cattle.io/local -n fleet-local -o yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
metadata:
  annotations:
    objectset.rio.cattle.io/id: provisioning-cluster-create
    objectset.rio.cattle.io/owner-gvk: management.cattle.io/v3, Kind=Cluster
    objectset.rio.cattle.io/owner-name: local
    sim.harvesterhci.io/creationTimestamp: "2022-12-22T02:06:19Z"
  creationTimestamp: "2022-12-22T02:06:19Z"
  finalizers:
  - wrangler.cattle.io/provisioning-cluster-remove
  - wrangler.cattle.io/rke-cluster-remove
  generation: 1
  labels:
    objectset.rio.cattle.io/hash: 50675339e9ca48d1b72932eb038d75d9d2d44618
    provider.cattle.io: harvester
  name: local
  namespace: fleet-local
  resourceVersion: "3152"
  uid: deab9647-de9c-44e5-9db4-0fb339f20bb7
spec:
  kubernetesVersion: v1.24.11+rke2r1
  localClusterAuthEndpoint: {}
  rkeConfig:
    provisionGeneration: 1
    upgradeStrategy:
      controlPlaneDrainOptions:
        deleteEmptyDirData: false
        disableEviction: false
        enabled: false
        force: false
        gracePeriod: 0
        skipWaitForDeleteTimeoutSeconds: 0
        timeout: 0
      workerDrainOptions:
        deleteEmptyDirData: false
        disableEviction: false
        enabled: false
        force: false
        gracePeriod: 0
        skipWaitForDeleteTimeoutSeconds: 0
        timeout: 0
status:
  clientSecretName: local-kubeconfig
  clusterName: local
  conditions:
  - status: "True"
    type: Ready
  - lastUpdateTime: "2022-12-22T02:06:19Z"
    status: "False"
    type: Reconciling
  - lastUpdateTime: "2022-12-22T02:06:19Z"
    status: "False"
    type: Stalled
  - lastUpdateTime: "2023-12-21T18:55:55Z"
    status: "True"
    type: Created
  - lastUpdateTime: "2023-12-21T19:30:18Z"
    status: "True"
    type: RKECluster
  - status: Unknown
    type: DefaultProjectCreated
  - status: Unknown
    type: SystemProjectCreated
  - lastUpdateTime: "2022-12-22T02:06:34Z"
    status: "True"
    type: Connected
  - lastUpdateTime: "2022-12-22T02:07:58Z"
    status: "True"
    type: Provisioned
  - lastUpdateTime: "2023-12-21T19:30:18Z"
    message: 'uncordoning worker node(s) custom-4e46b8d06b63: waiting for uncordon
      to finish'
    reason: Waiting
    status: Unknown
    type: Updated
  observedGeneration: 4
  ready: true

Root cause: The pre-drain job for the harvester-l8bht node failed. The Job logs were missing, so it's hard to tell what exactly caused the failure. However, I noticed the RKE2 of the node was upgraded to v1.24.11, which is quite peculiar. And from the rancher-8584878865-2kmtz Pod's log, we could see the relevant lines showing it's upgrading the node's RKE2:...
2023-12-21T18:55:52.579589010Z 2023/12/21 18:55:52 [INFO] [planner] rkecluster fleet-local/local: waiting: draining worker node(s) custom-4e46b8d06b63: draining node
2023-12-21T18:55:55.666820548Z 2023/12/21 18:55:55 [INFO] [planner] rkecluster fleet-local/local: waiting: configuring worker node(s) custom-4e46b8d06b63: waiting for plan to be applied
...
2023-12-21T18:55:59.478504815Z 2023/12/21 18:55:59 [INFO] [planner] rkecluster fleet-local/local: waiting: configuring worker node(s) custom-4e46b8d06b63,custom-564741db1fda,custom-869a9465598d and 7 more
...
2023-12-21T18:57:16.280398319Z 2023/12/21 18:57:16 [INFO] [planner] rkecluster fleet-local/local: waiting: uncordoning worker node(s) custom-4e46b8d06b63: waiting for uncordon to finish
...

How to initiate the upgrade again
We need to resolve the current stuck on the embedded Rancher Manager by manually assisting it to complete the upgrade workflow. For detailed steps, please refer to this comment. After that, we could start the upgrade again.

Morning, sorry for the long response, but our University closed down for the Holiday and we got kicked out of the office lol.
We are now in the current status.
apiVersion: provisioning.cattle.io/v1
kind: Cluster
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"provisioning.cattle.io/v1","kind":"Cluster","metadata":{"annotations":{},"name":"local","namespace":"fleet-local"},"spec":{"kubernetesVersion":"v1.24.7+rke2r1","rkeConfig":{"controlPlaneConfig":null}}}
objectset.rio.cattle.io/applied: H4sIAAAAAAAA/4yQzU7DMBCEXwXt2Slt079Y4oAQ4sCVF9jYS2Ow15G9CYfK746SVqJC4udo78xovjlBIEGLgqBPgMxRUFzkPD1j+0ZGMskiubgwKOJp4eKts6ChT3F02UV2fKyMH7JQqkwiFAL1ozV+MKXqOL6DhoCMRwrEciUYa3Xz7NjePZwj/8xiDAQafDTo/yXOPZrJAUXB3NdFfnGBsmDoQfPgvQKPLflfR+gwd6Bhu9ztt3XdUGNwc7Crdr9u6jW1y/pg91vb2LXdbHarA6jzYpbSVwho6DCNNIMWBd9Yrtu+eiKpzpeiIPdkpnbzx2Wq+0G6R7Z9dCygT2WSCcpwwciURrJPxJRmZtDLUj4DAAD//5CVWGcAAgAA
objectset.rio.cattle.io/id: provisioning-cluster-create
objectset.rio.cattle.io/owner-gvk: management.cattle.io/v3, Kind=Cluster
objectset.rio.cattle.io/owner-name: local
objectset.rio.cattle.io/owner-namespace: ""
creationTimestamp: "2022-12-22T02:06:19Z"
finalizers:

wrangler.cattle.io/provisioning-cluster-remove
wrangler.cattle.io/rke-cluster-remove
generation: 4
labels:
objectset.rio.cattle.io/hash: 50675339e9ca48d1b72932eb038d75d9d2d44618
provider.cattle.io: harvester
name: local
namespace: fleet-local
resourceVersion: "728721089"
uid: bc931ef9-ce52-4999-9dcd-6fd9475510aa
spec:
kubernetesVersion: v1.24.11+rke2r1
localClusterAuthEndpoint: {}
rkeConfig:
chartValues: null
machineGlobalConfig: null
provisionGeneration: 1
upgradeStrategy:
controlPlaneDrainOptions:
deleteEmptyDirData: false
disableEviction: false
enabled: false
force: false
gracePeriod: 0
ignoreDaemonSets: null
postDrainHooks: null
preDrainHooks: null
skipWaitForDeleteTimeoutSeconds: 0
timeout: 0
workerDrainOptions:
deleteEmptyDirData: false
disableEviction: false
enabled: false
force: false
gracePeriod: 0
ignoreDaemonSets: null
postDrainHooks: null
preDrainHooks: null
skipWaitForDeleteTimeoutSeconds: 0
timeout: 0
status:
clientSecretName: local-kubeconfig
clusterName: local
conditions:
status: "True"
type: Ready
lastUpdateTime: "2022-12-22T02:06:19Z"
status: "False"
type: Reconciling
lastUpdateTime: "2022-12-22T02:06:19Z"
status: "False"
type: Stalled
lastUpdateTime: "2023-12-21T18:55:55Z"
status: "True"
type: Created
lastUpdateTime: "2024-01-03T14:54:18Z"
status: "True"
type: RKECluster
status: Unknown
type: DefaultProjectCreated
status: Unknown
type: SystemProjectCreated
lastUpdateTime: "2022-12-22T02:06:34Z"
status: "True"
type: Connected
lastUpdateTime: "2022-12-22T02:07:58Z"
status: "True"
type: Provisioned
lastUpdateTime: "2024-01-03T14:54:18Z"
message: 'uncordoning worker node(s) custom-4e46b8d06b63: error undraining machine
custom-4e46b8d06b63: [error when evicting pods/"instance-manager-r-cdf8b2b5"
-n "longhorn-system": global timeout reached: 10m0s, error when evicting pods/"instance-manager-e-5c2d22b4"
-n "longhorn-system": global timeout reached: 10m0s]'
reason: Waiting
status: Unknown
type: Updated
observedGeneration: 4
ready: true

It looks like post drain is erroring
➜  rancher git:(main) ✗ ./post-drain.sh harvester-l8bht

NODE=harvester-l8bht
++ kubectl get node harvester-l8bht -o yaml
++ yq -e e '.metadata.annotations."cluster.x-k8s.io/machine"' -
MACHINE=custom-4e46b8d06b63
++ kubectl get secret -n fleet-local custom-4e46b8d06b63-machine-plan -o yaml
++ yq -e e '.metadata.annotations."rke.cattle.io/post-drain"' -
Error: no matches found
data=null

We have followed your instructions but the node will not progress to a non null rke-post-drain status
harvester-l8bht (custom-4e46b8d06b63)
rke-pre-drain: {"deleteEmptyDirData":true,"disableEviction":false,"enabled":true,"force":true,"gracePeriod":0,"ignoreDaemonSets":true,"ignoreErrors":false,"postDrainHooks":[{"annotation":"harvesterhci.io/post-hook"}],"preDrainHooks":[{"annotation":"harvesterhci.io/pre-hook"}],"skipWaitForDeleteTimeoutSeconds":0,"timeout":0}
harvester-pre-hook {"deleteEmptyDirData":true,"disableEviction":false,"enabled":true,"force":true,"gracePeriod":0,"ignoreDaemonSets":true,"ignoreErrors":false,"postDrainHooks":[{"annotation":"harvesterhci.io/post-hook"}],"preDrainHooks":[{"annotation":"harvesterhci.io/pre-hook"}],"skipWaitForDeleteTimeoutSeconds":0,"timeout":0}
rke-post-drain: null
harvester-post-hook: null
Any hints on getting it to move forward?


Prevent Close() from being called after Finish()

Close() writes hash state to the filesystem.  If Finish has already been called, return the underlying error to prevent this.

LGTM!

Pending circle but LGTM


release tools update to 1.19

Signed-off-by: Humble Chirammal<EMAIL_ADDRESS>Additional note for reviewer:
It seems to me that, this mention of 1.18 is causing the CI to fail while doing test-vendor in imported project.
https://github.com/kubernetes-csi/csi-driver-iscsi/pull/137#issuecomment-1237703595

/release-note-none

/assign @pohly

Looks like its not yet ready to bump the version to 1.19.. we can put this on hold for now.. I am marking this as WIP.

/test pull-kubernetes-csi-release-tools-external-snapshotter

/test pull-kubernetes-csi-release-tools-external-snapshotter


MFA enforcement for SSH

Is your feature request related to a problem? Please describe.
Imagine a scenario that only the Linux VM Sign In resource requires MFA. In such case, the initial "az login" would not trigger MFA, and then the subsequent "az ssh vm ..." attempt would fail and CLI would currently suggest end user to redo "az login". But that would end up in a loop.
This is not a new ask from SSH feature only. It could happen for any MFA-protected resource.
Describe the solution you'd like
Adding a new optional scope parameter to the initial login, i.e. "az login --scope foo".
The error message would better also provide a full command with the specific scope, so that end user can just do a copy-and-paste to trigger a new interactive auth login.
Describe alternatives you've considered
n/a
Additional context
This is derived from an internal email conversation.
@jiasli , CC: @fengzhou-msft

Adding a new optional scope parameter to the initial login


AutoComplete enhancement: press enter on selecting suggestion.

VSCode Version: 1.12.2 (1.12.2)
OS Version: macOS Sierra 10.12.4

Steps to Reproduce:

Enter some words, like toLowerCase.
Suggestion will show up.

If I press enter, and I have acceptSuggestionOnEnter enabled, "editor.acceptSuggestionOnEnter": true,. The current behavior is it will create a newline, but I think since we are selecting the suggestion, the correct behavior should be dismiss the suggestions ( which this is Sublime's default behavior).

dupe of #24420


Recast not installed imports statment node

Was hoping to use this good idea 3 errors
On a windows machine .fs will not install use fs-extra
add recast to package json
change import statement
ERROR
`ERROR #11321  PLUGIN
"gatsby-plugin-static-props" threw an error while running the onCreatePage lifecycle:
Cannot use import statement outside a module
22 |   require("jsdom-global")()
23 |

24 |   const PageComponent = require(page.component)
|                         ^
25 |   const getStaticProps = PageComponent.getStaticProps || new Function()
26 |
27 |   try {

File: node_modules\gatsby-plugin-static-props\gatsby-node.js:24:25`

@victorvoid could you please republish npm package? There is old version in npm, without recast

Thanks!! Published ✌🏻

Thank You!


Reproduce figures of the paper

Dear authors,
Thank you for providing this excellent paper and the complementary code-base.
However, I would like to reproduce the figures from the paper exactly, e.g., the ones in which the avg and the std of the normalized bias for the Ant-env is depicted. Is this code also publicly available or should I rather try to write this myself? Unfortunately, I could not find it in this repository.
Best,
Martin

Hi Martin! Thanks for posting the issue! You don't need to write it
yourself! I can upload all the plotting utilities and data we have so that
you can easily reproduce the figures! I will try get this done in the next
few days!
MarWaltz @.***> 于 2021年7月22日周四 下午9:00写道：

Dear authors,
Thank you for providing this excellent paper and the complementary
code-base.
However, I would like to reproduce the figures from the paper exactly,
e.g., the ones in which the avg and the std of the normalized bias for the
Ant-env is depicted. Is this code also publicly available or should I
rather try to write this myself? Unfortunately, I could not find it in this
repository.
Best,
Martin
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_watchernyu_REDQ_issues_5&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=eaSqLVOFESkIqfDT9hWv1w&m=bGIAYQXxVHHTvQFuhc4GlDXNCLcwyKl73Ra1iWlsdzE&s=P9vTm5eL3_L2Q0BUQYftaE_9eVP3gjoMngx9WjFejrU&e=,
or unsubscribe
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ADTQ5ZAAYXIYEKCDEIYZFH3TZAI6ZANCNFSM5AZ6VEDQ&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=eaSqLVOFESkIqfDT9hWv1w&m=bGIAYQXxVHHTvQFuhc4GlDXNCLcwyKl73Ra1iWlsdzE&s=NhrmhOhFL48CWHOa6RA1B6rqD-axiEJF7eAJZSpaOHA&e=
.

Great, thanks a lot!

@MarWaltz Hi! We have uploaded data and the code for reproducing the figures in the paper! We have also added the Q bias evaluation methods in the latest codebase, so now you can log these bias information while running experiments. Currently the plotting uses Seaborn 0.8.1 version (using the tsplot function. Note this function is deprecated in newer version of Seaborn. If you want to use newer version you can try modify the code slightly, shouldn't be too hard. If I have time this week I will also try update code to support newer version of Seaborn and write some more tutorial). The link to download the data and how to plot the figures are discussed on the README page.
Hope that helps! If you encounter any issue please let us know!

Dear Che Wang,
Thanks again for providing the details.
I came back to your approach of bias quantification, and I think I have kind of a misunderstanding. In the paper, Section 3, you define a Q-function Q^{\pi} for a policy \pi. Further, Q^{\phi} is an estimate of it, and the bias is consequently Q^{\phi} - Q^{\pi}. In the experiment, you consider the current behavior policy \pi induced by your current actor, which contains some noise for exploration (I looked it up in your code). This is the policy for generating your MC trajectories, which serve as the ground truth and are used to compute returns by definition.
But is comparing these returns created by the continuous analog of an \epsilon-greedy strategy with the learned Q's of the critic a valid approach? In Q-Learning, or the critic training in this case, the Q directly approximates Q* (Sutton & Barto, 2018, p. 131), regardless of the behavior policy. So I am wondering how the pursued approach can be justified.
Again, I believe that I have some conceptual misunderstanding, and I would be glad if you could help me with this point.
Best,
Martin


Changing the thanos_store_nodes_grpc_connections implementation

Intro
I wanted to make a brand new issue to include all information from previous issues/pull-requests and slack discussions into one fresh issue. I will provide some extra context and background information. This should resolve some discussions we are having and move this forward. I feel this is useful as this had quite some discussions before over quite some period.
Is your proposal related to a problem?
This issue relates to:
https://github.com/thanos-io/thanos/pull/4169
https://github.com/thanos-io/thanos/issues/4124
https://github.com/thanos-io/thanos/issues/3014
https://github.com/thanos-io/thanos/issues/4177
Background and context
First we need to start with the reason --store-strict was implemented. This can be read here: https://thanos.io/tip/proposals/202001_thanos_query_health_handling.md/
Thanos Query updates the list of healthy store nodes every 5 seconds. It does that by sending the Info() call via gRPC.
How --store works
The last successful check is noted in the LastCheck field. At this point if that fails, we note the error and remove the store node from the set of active store nodes. If it succeeds then it becomes a part of the active store set.
After --store.unhealthy-timeout passes since LastCheck then it also gets removed from the UI. At this point we would forget about it completely.
With regards to the metric thanos_store_nodes_grpc_connections - this gets instantly removed if the store is unhealthy and it does not wait for the --store.unhealthy-timeout to pass.
How --store-strict works
This only allows static defined store nodes.
If the health check fails, we note the error but do not remove the store from the active store set AND Thanos Query will always retain the last successfully retrieved information via the Info() gRPC method.
Thus, they will always be considered as part of the active store set.
With regards to the metric thanos_store_nodes_grpc_connections -  This simply does not get updated until the node returns into a healthy state.
My take about static targets & strict store
The feature of --store-strict IMO unintendedly created a bug because we 'ignore' the activity of the store node to keep it in the active store set even if its unhealthy. This then also does not remove the metric anymore, but also does not reflect the actual state.
Regardless of anything, this should be fixed.
However I do not think the 'liveness' of the thanos_store_nodes_grpc_connections metric should be limited to the --store-strict argument.
Yes, it implies strictness. But --store-strict ALSO adds different features that might not be useful or even breaking to users who's sole intent is to have a solid thanos_store_nodes_grpc_connections metric.
For example the Query component will always query unhealthy store nodes. Which might poses different issues for users. A user needs to deal with this, in order to have a working thanos_store_nodes_grpc_connections metric. Which seems not logical.
To sum this up; A user should not have to use --store-strict for a working metric while the intend for --store-strict is created for one goal:

Update the health check logic to properly handle the case when a node disappears in terms of a caching layer.

To further add; it implies only --store-strict expects stores to be up. Let's be honest: we also expect --store to have atleast 1 store node that is not unhealthy.  The reason we more explicitly want to keep stores up on --store-strict is to deal with the caching layer. Regardless we still want to check metrics for stores, even on "non-strict mode"
Static target vs Dynamic
To move towards a solution I wanted to describe something about static vs dynamic targets.
I believe that any target that uses dns+ or dnssrv+ is dynamic. Any other target is static.
If we start the Query Component with --store=somehost:10902 then we have a 'source of truth' and we can (and should?) keep the metric thanos_store_nodes_grpc_connections alive and update its value to reflect the status. See my description above about the working of --store - it instantly removes the metric if the store node is unhealthy at the moment.
Describe the solution you'd like
(refer as solution 1)
Let's not bother (I mean this in a good way) between --store vs --store-strict for a metric.

If a target is static, we should keep the metric and update its value to reflect the actual status
If a target is dynamic (so; dns+ or dnssrv+ , we should keep current behaviour)

Why remove dynamic store node metrics
Because it's not reliable.

Store node goes down
Query goes does down
Query goes back up
Query has no knowledge of that dynamic store and thus would NOT have any metric about it

We need to update documentation that one could only have stable monitoring on thanos_store_nodes_grpc_connections  if they define static targets.
Describe alternatives you've considered
(refer as solution 2)
Don't make a difference between static vs dynamic targets and always keep the metric alive. It still should have the clear documentation that this is only reliable if static targets are used.
Another downside is that if users who have a lot of fluctuating targets, it could 'spam' the metric full. That said, I doubt if this is an actual problem as the metric is fairly generic. It only contains the external_labels and store_type labels. Not an ip, hostname and such.
(refer as solution 3)
Keep --store as it is and only implement this metric behaviour on --store-strict.
This could make sense if new features extend on the more 'strict' part and assumptions and we deal with --store as volatile store nodes.
Additional context
There are 3 options, I just want to make clear that either way it would resolve everyones problems. We just have to pick the most logical one. Obviously this is open for other suggestions.

@wiardvanrij thanks for digging into this and creating the new issue. I think having all this context is incredibly helpful and gives the correct amount of background for us to make in an informed decision 🙂
That being said, I'm good with Solution 1 or 2, though I think Solution 1 is likely more straightforward.

To sum this up; A user should not have to use --store-strict for a working metric while the intend for --store-strict is created for one goal:
Update the health check logic to properly handle the case when a node disappears in terms of a caching layer.
To further add; it implies only --store-strict expects stores to be up. Let's be honest: we also expect --store to have atleast 1 store node that is not unhealthy. The reason we more explicitly want to keep stores up on --store-strict is to deal with the caching layer. Regardless we still want to check metrics for stores, even on "non-strict mode"

Well, that's only partially true. "Update the health check logic to properly handle the case when a node disappears in terms of a caching layer." - let's be specific. I think strict was added to make sure that:

We have response caching.
We have a store that is down, and ofc k8s service discovery take down store out of DNS result, so Querier can't see that there was previously any store if dns+, dnssrv+ are used.
Despite this, partial response is not triggered, Querier thinks we had consistent data so returns success.
Data with gap is cached.

Overall we ensure that we return API errors (or warnings, depending on PartialResponseStrategy) when the strict store is down. We also cache last successful metadata yues. That's why it's called "strict".

With regards to the metric thanos_store_nodes_grpc_connections - This simply does not get updated until the node returns into a healthy state.

We don't have a connection so I would fix this and put 0 for not alive connections.
Summary
I like solution 1 + let's fix thanos_store_nodes_grpc_connections for strict nodes.
So the end state could look like this:
I would advise against solution 2. It can give a false impression that you can avoid using absent but you can't as the metric is cached and if process crashes you will miss alert (metric will "unregister" such lost store API automatically)

--endpoint (because with @hitanshu-mehta we want to move it to be more generic): best-effort endpoint that is expected to be there or not.
If such endpoint is unhealthy (does not respond to Info) we DON'T fail query.
Yet we:

register the connection metric (as 0) as soon as we know the address if static.  If dynamic then we don't and metric is more fluid for purpose. For static ones we also never unregister.
I would also keep last cached metadata on store page

@bwplotka one clarification question on this point. Can we define what we mean by "dynamic" here? Based on the current definition for passing in --store
--store=<store> ...
Addresses of statically configured store API
servers (repeatable). The scheme may be
prefixed with 'dns+' or 'dnssrv+' to detect
store API servers through respective DNS
lookups.

I'd say the dns+ and dnssrv+ would be dynamic and anything else (IP or hostname + port) are static. Also, based on how I believe --store.sd-files=<path> works, that takes a list of IP/hostname combinations as well so those should also be considered static.
Any disagreement there?
Thanks!

Still relevant


use full tokenizer

see here for different tokenizers: https://github.com/nextapps-de/flexsearch/tree/0.6.32#tokenizer

I already gave a functional review in https://github.com/skohub-io/skohub-vocabs/issues/153

@dr0i Please deploy on production and merge.

Deployed and merged. Closed.


Updates to the multi-version fonction and 'contract.js' ruleset

improved regexp to support corner cases, see https://app.asana.com/0/1206872740398257/1206657704786474/f
added an exception list for paths that include version numbers by design such as Meraki 'dhcp/v4'
simplified name of rule 'multi-versions-server-url-missing-version]' to 'server-url-missing-version'
simplifications, clarifications and more consistent spectral messages

hum, the CI tests seems to be failing :-(
I'll have to check over...
FYI, I had checked that the Meraki, Catalyst SD-WAN, Catalyst Center and Crosswork documents are analyzed correctly using the updated Contract ruleset.

final PR scope:

improved regexp to support corner cases, see https://app.asana.com/0/1206872740398257/1206657704786474/f
added an exception list for paths that include version numbers by design such as Meraki 'dhcp/v4'


how can I add .html page on the "just-the-docs" page

Dear.
I'm new to github based webpage.
I added new cccc.md and it looks work.
https://adioshun.github.io/just-the-docs/docs/utilities/cccc/
Now, I am trying to add new test.html (by convert jupyter.ipynb to test.html)
I just upload it on the same folder.
https://github.com/adioshun/just-the-docs/blob/master/docs/utilities/test.html
What should I do next? do I need to add  front-matter like the cccc.md??

@adioshun you'd need to add the front matter to test.html. Please refer to the Jekyll documentation for more info:
https://jekyllrb.com/docs/pages/
https://jekyllrb.com/docs/front-matter/

@pmarsceill  thanks.


[Testing] Hold - Approve confirmation modal show full amount as full and partial option

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: v9.0.35-0
Reproducible in staging?: Y
Reproducible in production?: Y
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause Internal Team
Action Performed:
Precondition:

Login as the owner of the workspace
Create a workspace
Invite the approver and employee
Navigate to more features
Enable "workflows"
On the "Workflow" editor - enable "Add Approvals"
Set the Approver account as the Approver

Steps:

As Employee submit 2 distance expenses to the workspace chat
As Employee edit the first distance request by changing start or stop destination
As Employee hold the first distance request
As Approver click on green Approve button
Verify confirmation modal appears

Expected Result:
Confirmation modal should show full and partial amount that should be approved
Actual Result:
Confirmation modal shows same amount (full amount as full and partial option)
Workaround:
Unknown
Platforms:

[x] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos

https://github.com/user-attachments/assets/6b4014cc-aba9-486a-820d-3ce98bb46df7

View all open jobs on GitHub

We think that this bug might be related to #wave-collect - Release 1

@strepanier03  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Proposal
Please re-state the problem that we are trying to solve in this issue.
Hold - Approve confirmation modal show full amount as full and partial option
What is the root cause of that problem?
Offending PR: https://github.com/Expensify/App/pull/49144
The PR misidentified the root cause which fixed the problem it was intended to resolve but introduced issues elsewhere
What changes do you think we should make in order to solve the problem?

Revert changes in the offending PR.
Add missing * coefficient multiplication for the unheld total.

     if (hasUpdatedTotal(iouReport, policy) && hasPendingTransaction) {
-        const unheldTotal = reportTransactions.reduce((currentVal, transaction) => currentVal + (!TransactionUtils.isOnHold(transaction) ? transaction.amount : 0), 0);
+        const unheldTotal = reportTransactions.reduce((currentVal, transaction) => currentVal - (!TransactionUtils.isOnHold(transaction) ? transaction.amount : 0), 0);

-        return [CurrencyUtils.convertToDisplayString(unheldTotal, iouReport?.currency), CurrencyUtils.convertToDisplayString((iouReport?.total ?? 0) * coefficient, iouReport?.currency)];
+        return [CurrencyUtils.convertToDisplayString(unheldTotal * coefficient, iouReport?.currency), CurrencyUtils.convertToDisplayString((iouReport?.total ?? 0) * coefficient, iouReport?.currency)];
    }

Will prioritize this tomorrow.

I've tested this twice now and I'm unable to reproduce this.

This has been labelled "Needs Reproduction". Follow the steps here: https://stackoverflowteams.com/c/expensify/questions/16989

Waiting for repro.

still testing

Issue not reproducible during KI retests. (First week)

Issue not reproducible during KI retests. (Second week)

Awesome, thanks @mvtglobally - I'm going to close this out.


Import History showing (rows - 1) results

The Import History is not showing the full set of results, it is one line short, due to the change made in release 0.8.21. The for loop on line 64 of Import_History.php should read:
for ($i = 2; $i <= ($rows + 1); $i++) {

otherwise only ($rows - 1) rows of data are shown.

Thanks!


Update the import paths for Caddy

This pull request makes caddy-sstp compatible with Caddy v1.0.1+ (see epicagency/caddy-expires#6 for reference) by updating the import paths for Caddy.

Thanks!


Orion graph wrong order of execution with Dask

Snippet of code to reproduce the problem.

Code
import random
import time
from typing import List

from prefect import State, flow, task
from prefect.futures import PrefectFuture
from prefect.task_runners import ConcurrentTaskRunner

from prefect_dask import DaskTaskRunner

from flows_2.sub_flows.result import Result

my_network_graph = {
    "task_1": [],
    "task_2": [],
    "task_3": ["task_1", "task_2"],
    "task_4": ["task_3"],
    "task_5": ["task_4"],
}

@task(name="write_to_storage")
def write_result(result: Result):
    print(f"{result.number} is upload to storage: {result.fake_path}")

@task(name="compute_result")
def compute_result(
    task_name: str,
    upstream_futures: List[Result],
) -> Result:

    print(f"Hello from: {task_name}")
    print(f"My upstream computed dependencies: {upstream_futures}")
    # time.sleep(random.randint(10, 50))
    result = random.randint(1, 100)
    print(f"computed value is {result}")
    return Result(name=task_name, number=result, fake_path=f"gs:fake_path:{result}")

def create_pool_futures_write_results(results: List[Result]) -> List[PrefectFuture]:
    tasks = []
    for result in results:
        tasks.append(
            write_result.with_options(name=f"write_to_storage_{result.name}").submit(
                result=result,
            )
        )
    return tasks

# this function simulate predefined DAG
def create_pool_futures_compute_results() -> List[PrefectFuture]:
    tasks = []
    appended_tasks = {}

    for key, value in my_network_graph.items():
        if len(appended_tasks) == 0:
            tasks.append(
                compute_result.with_options(name=f"compute_result_{key}").submit(
                    task_name=key,
                    upstream_futures=[],
                    wait_for=[],
                )
            )
            appended_tasks[key] = tasks[-1]
        else:
            upstream_futures = [
                appended_tasks[_value] for _value in my_network_graph[key]
            ]
            tasks.append(
                compute_result.with_options(name=f"compute_result_{key}").submit(
                    task_name=key,
                    upstream_futures=upstream_futures,
                    wait_for=upstream_futures,
                )
            )
            appended_tasks[key] = tasks[-1]
    return tasks

@flow(task_runner= DaskTaskRunner())
def futures_execution_flow():

    tasks_computes = (
        create_pool_futures_compute_results()
    )  # executing the features tasks in the flow

    states_features: List[State] = [
        tasks_compute.wait() for tasks_compute in tasks_computes
    ]  # Wait for all tasks to finish

    results = [state.result() for state in states_features]

    tasks_writing = create_pool_futures_write_results(results=results)
    states_writing_tasks: List[State] = [
        task_writing.wait() for task_writing in tasks_writing
    ]  # Wait for all tasks to finish

if __name__ == "__main__":
    futures_execution_flow()

Put in separete file.
from pydantic import BaseModel

class Result(BaseModel):
    name: str
    number: int
    fake_path: str

Orion graph for DaskTaskRunner

I'd rather wait the same graph like for ConcurrentTaskRunner:
Which shows that it's impossible to execute write tasks before computed one

For the moment I don't arrive to reproduce the problem, which I have with my production code.
But I have the problem due to order of execution, which we can observe on graph: write tasks executed before computed one.
I provide a traceback with Dask execution (no problem with Concurrent runner):

 Traceback
   Crash detected! Execution was interrupted by an unexpected exception: Traceback (most recent call last):
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 108, in execute
    self._adapt_connection._handle_exception(error)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 236, in _handle_exception
    raise error
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 90, in execute
    self.await_(_cursor.execute(operation, parameters))
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 68, in await_only
    return current.driver.switch(awaitable)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 121, in greenlet_spawn
    value = await result
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/cursor.py", line 37, in execute
    await self._execute(self._cursor.execute, sql, parameters)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/cursor.py", line 31, in _execute
    return await self._conn._execute(fn, *args, **kwargs)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/core.py", line 129, in _execute
    return await future
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/core.py", line 102, in run
    result = function()
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/prefect/orion/orchestration/rules.py", line 373, in validate_proposed_state
    await self._validate_proposed_state()
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/prefect/orion/database/dependencies.py", line 117, in async_wrapper
    return await fn(*args, **kwargs)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/prefect/orion/orchestration/rules.py", line 414, in _validate_proposed_state
    await self.session.flush()
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/session.py", line 405, in flush
    await greenlet_spawn(self.sync_session.flush, objects=objects)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 128, in greenlet_spawn
    result = context.switch(value)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3444, in flush
    self._flush(objects)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3584, in _flush
    transaction.rollback(_capture_exception=True)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3544, in _flush
    flush_context.execute()
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 453, in execute
    n.execute_aggregate(self, set_)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 747, in execute_aggregate
    persistence.save_obj(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
    _emit_insert_statements(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements
    c = connection._execute_20(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
    ret = self._execute_context(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 108, in execute
    self._adapt_connection._handle_exception(error)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 236, in _handle_exception
    raise error
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/dialects/sqlite/aiosqlite.py", line 90, in execute
    self.await_(_cursor.execute(operation, parameters))
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 68, in await_only
    return current.driver.switch(awaitable)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 121, in greenlet_spawn
    value = await result
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/cursor.py", line 37, in execute
    await self._execute(self._cursor.execute, sql, parameters)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/cursor.py", line 31, in _execute
    return await self._conn._execute(fn, *args, **kwargs)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/core.py", line 129, in _execute
    return await future
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/aiosqlite/core.py", line 102, in run
    result = function()
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: INSERT INTO task_run_state (id, created, updated, type, timestamp, name, message, state_details, data, task_run_id) VALUES (:id, :created, :updated, :type, :timestamp, :name, :message, :state_details, :data, :task_run_id)]
[parameters: {'id': '1ef8ce60-06d2-4cdb-b197-fe4fb0bd3e26', 'created': '2022-12-20 16:32:14.841743', 'updated': '2022-12-20 16:32:14.841767', 'type': 'COMPLETED', 'timestamp': '2022-12-20 16:32:14.830388', 'name': 'Completed', 'message': None, 'state_details': '{"flow_run_id": "583b1ae3-8ad0-4745-9269-53a1ac000b43", "task_run_id": "4c8f1cd3-8992-4af0-867e-b0cacf8b3272", "child_flow_run_id": null, "scheduled_time": null, "cache_key": null, "cache_expiration": null, "untrackable_result": false}', 'data': '{"type": "literal", "value": null}', 'task_run_id': '4c8f1cd3-8992-4af0-867e-b0cacf8b3272'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/prefect/orion/models/task_runs.py", line 326, in set_task_run_state
    await context.validate_proposed_state()
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/prefect/orion/database/dependencies.py", line 117, in async_wrapper
    return await fn(*args, **kwargs)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/prefect/orion/orchestration/rules.py", line 379, in validate_proposed_state
    task_run_id=self.run.id,
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 482, in __get__
    return self.impl.get(state, dict_)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 942, in get
    value = self._fire_loader_callables(state, key, passive)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 973, in _fire_loader_callables
    return state._load_expired(state, passive)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/state.py", line 712, in _load_expired
    self.manager.expired_attribute_loader(self, toload, passive)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 1451, in load_scalar_attributes
    result = load_on_ident(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 407, in load_on_ident
    return load_on_pk_identity(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 530, in load_on_pk_identity
    session.execute(
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1713, in execute
    conn = self._connection_for_bind(bind)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1549, in _connection_for_bind
    TransactionalContext._trans_ctx_check(self)
  File "/Users/aliadov/Library/Caches/pypoetry/virtualenvs/ia-analysis-RNGpen2B-py3.8/lib/python3.8/site-packages/sqlalchemy/engine/util.py", line 199, in _trans_ctx_check
    raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't operate on closed transaction inside context manager.  Please complete the context manager before emitting further commands.

During handling of the above exception, another exception occurred:

AttributeError: 'NoneType' object has no attribute 'is_running'

The problem with Dask was resolved with explicitly exported env variable:
prefect config set PREFECT_API_URL=http://<IP_ADDRESS>:4200/api
Problem was: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
But the UI problem still persist.

At the moment, I don't have a quick solution, but I wanted to say thanks for reporting this! I'll be sure to look into this once I have a bit more time.

I have the same UI problem using the RayTaskRunner; using the ConcurrentTaskRunner results in a correct graph, but using the RayTaskRunner results in a different graph that doesn't correctly show the dependencies of the tasks in the flow. However, the actual execution of the flow seems to be in the correct order.


Retrieve president from database

Currently, the data about the president in the about/current-team page is hardcoded. It should be instead retrieved from Supabase. This also requires the definition of function on the website that retrieves the data of the latest president from the presidents table.

I just wrote a sql function get_latest_president that returns president's id, name, surname, linkedin infos. I had a problem while creating server load functions. I will try to check in my free time.

I think you can leverage the function get_president (which we might as well rename get_president_id) which returns the id of the the current president. For the load function, You can do something similar to what is already there:
export const load: PageServerLoad = async ({ locals }) => {
    const res = await locals.supabase.rpc("get_complete_subteams");
    if (res.error)
        throw error(500, {
            details: res.error.details,
            message: res.error.message,
        });
    // retrieve president
    const president = await supabase.rpc("get_latest_president");
    // do error check similar to the one above
    return {
        subteams: res.data as unknown as CompleteSubteam[],
        ppBucket: locals.supabase.storage.from("people-pics").getPublicUrl("")
            .data.publicUrl,
        // president data
       president,
    };
};

I didn't use the get_president function because I couldn't see any function to pass president's id which gives me president's other data(name, surname etc). That's why I have written a new function to get all data about president. That's basically includes get_president function inside of it. If we have easier way to do it, let me know

I meant exactly using the get_president function inside yours to simplify it


262 add hooks for technological correct profile and contour line orientation

Added attributes and hooks for technologically correctly orientated contour lines and profiles.
Closes pyroll-project/pyroll-export#6

I'm still missing tests for this new hook.
Something like plotting of the profiles and contour lines would be suitable.

Added a test which contains both

plotting of the profiles and contours
and a assert check wheter the technological orientation works correctly


Fix badges

And tox.

Codecov Report
Base: 89.42% // Head: 89.42% // No change to project coverage :thumbsup:

Coverage data is based on head (c60f7bf) compared to base (339f684).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #31   +/-   ##
=======================================
  Coverage   89.42%   89.42%
=======================================
  Files          19       19
  Lines        1352     1352
  Branches      189      189
=======================================
  Hits         1209     1209
  Misses        104      104
  Partials       39       39

Flag
Coverage Δ

unittests
89.42% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Fix Authorization Failure While Reading Tables From Unity Catalog [databricks]

This PR replaces ThreadPoolExecutor creation with a utility function from Spark. The idea behind this replacement is to benefit from the custom implementation in different flavors of Spark.
We also reverted the changes made in 5f40711 because they were interfering with some confs that were set by Databricks runtime which are essential to run with Unity Catalog.
fixes #10318

build


Create database if eve.sqlite does not exist

If no database file is found (i.e., eve.sqlite does not exist), a new one should be created and saved based on db-script.sql at the start of the Go server.

The same code shouldnt be saved twice. See script.sql vs. Script.go.
Maybe read the first file in the go file?

#80 Solved


fix #4837 Update glob due to vulnerability in dep

Description of the Change
The minimatch npm package version <=3.0.4 has a security vulnerability explained here: https://github.com/advisories/GHSA-f8q6-p94x-37v3
Mocha has the glob package 7.2.0 as a dependency, which in turn has minimatch ^3.0.4 as a dependency
This results in a high security risk warning for consumers of mocha. See https://github.com/mochajs/mocha/issues/4937 for an example.
This PR changes the glob dependency to 9.2.1 which no longer depends on the insecure version of minimatch.
Alternate Designs
N/A
Why should this be in core?
It's not possible to fix https://github.com/mochajs/mocha/issues/4937 without updating core.
Benefits
Fix security concerns raised in https://github.com/advisories/GHSA-f8q6-p94x-37v3
Possible Drawbacks
All tests pass for me, and I see no drawbacks.
Applicable issues
Applicable Issue is https://github.com/mochajs/mocha/issues/4937
This should be suitable for a patch release.

@juergba this should be ready to be tested again. 2 things I have concerns about. #1 The tests referenced coffee-script, with a hyphen, but when I ran the tests, I got a "module not found error" and it looks like the hyphen package is depreciated, and mocha's package.json references "coffeescript" with no hyphen, so I updated those tests. Additionally, I got an error during linting that   "eslint-plugin-n" is missing. I though it was my local setup, but it also happened when the tests ran in the CI process. Anyways, I added that package to the dev deps. Let me know if you have any questions or concerns

@jb2311 thank you for this PR.
We will see wether our CI tests will pass. Tbh I don't trust your package-lock.json.

Coverage: 94.331% (+0.002%) from 94.329% when pulling e20dc663fd0b191fa278783407f1ba992b6e9361 on jb2311:upgrade-glob into 9f24d0d03fd3dc1d15681d9b1548f77ddbeb9ef3 on mochajs:master.

esline-plugin-node is no longer being maintained, so eslint-plugin-n (https://github.com/eslint-community/eslint-plugin-n) is a fork that is still active. However, I have no idea why it's erroring out without it suddenly. It happened on the first test run, when I did nothing but update the glob package https://github.com/mochajs/mocha/actions/runs/4368808387/jobs/7681675449

@juergba updated

@juergba okay, I think the whitespace should be appropriate.


Switch to native runner

This switches from the stack runner to native runner. rel https://github.com/paritytech/substrate/pull/7260
Right now for big contracts, only native runner is usable, due to a state overlay memory leak (#155). In normal operations, due to lack of state overlay and reversion logic being handled outside of runtime, native runner should be generally faster.
The only drawback of native runner is that it's much harder to get exactly gas usage match with Ethereum blockchain.

The native runner still has some issues. So let's keep using the stack runner for this PR and do the final switch in a separate PR. Otherwise this PR will be really big and standing for a long time.