Bugs with GDExtension API validation logic (--validate-extension-api command and CI)

Tested versions

Reproducible in 4.4.dev (826de7976a6add282c7b14d4be2a7e6d775821d8), #90993, and older versions too

System information
Any
Issue description
We have some bugs in the GDExtension API validation checks that we perform on CI, which are starting to hamper development as we need to work them around in non-trivial ways.
#90993 is a good example of that as it's impacting a complex method, RenderingDevice::draw_list_begin which changed multiple times since Godot 4.0:

In 4.1-stable, its 9th argument was changed from an Array to a TypedArray<RID> (Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments/9': type changed value in new API, from "Array" to "typedarray::RID".).
In 4.3-stable, its 9th argument was removed (Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments': size changed value in new API, from 10 to 9.).
In #90993 (intended for 4.4-stable), a new 9th argument with a different name, type and default value was added (Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments': size changed value in new API, from 9 to 10.).

To make CI checks pass, I had to add changes to 4.0-stable_4.1-stable.expected and 4.2-stable_4.3-stable.expected that reflects changes done in 4.4, which seems wrong to me. And also contradicts previous messages in the same .expected files for the API breakage done in their respective versions.
diff --git a/misc/extension_api_validation/4.0-stable_4.1-stable.expected b/misc/extension_api_validation/4.0-stable_4.1-stable.expected
index 5c3bf07fb2..2b7f9dc662 100644
--- a/misc/extension_api_validation/4.0-stable_4.1-stable.expected
+++ b/misc/extension_api_validation/4.0-stable_4.1-stable.expected
@@ -349,3 +349,11 @@ Validate extension JSON: Error: Hash changed for 'classes/EditorUndoRedoManager/
 Validate extension JSON: Error: Hash changed for 'classes/UndoRedo/methods/create_action', from 0AEC1BFC to E87757EB. This means that the function has changed and no compatibility function was provided.

 Added a optional parameters with default values. No adjustments should be necessary.
+
+
+GH-90993
+--------
+Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments/9': type changed value in new API, from "Array" to "int".
+
+FIXME: Hack to workaround a bug in our validation logic with a method that changed arguments count, type, and default values multiple times.
+The actual last change triggering this error happened in 4.4 (GH-90993).
diff --git a/misc/extension_api_validation/4.2-stable_4.3-stable.expected b/misc/extension_api_validation/4.2-stable_4.3-stable.expected
index ce8f24c7a9..3de9736c8d 100644
--- a/misc/extension_api_validation/4.2-stable_4.3-stable.expected
+++ b/misc/extension_api_validation/4.2-stable_4.3-stable.expected
@@ -380,3 +380,12 @@ Validate extension JSON: Error: Field 'classes/Image/methods/get_mipmap_offset/r

 Type changed to int64_t to support baking large lightmaps.
 No compatibility method needed, both GDExtension and C# generate it as int64_t anyway.
+
+
+GH-90993
+--------
+Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments/9': default_value changed value in new API, from "Array[RID]([])" to "0".
+Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments/9': type changed value in new API, from "typedarray::RID" to "int".
+
+FIXME: Hack to workaround a bug in our validation logic with a method that changed arguments count, type, and default values multiple times.
+The actual last change triggering this error happened in 4.4 (GH-90993).
diff --git a/misc/extension_api_validation/4.3-stable.expected b/misc/extension_api_validation/4.3-stable.expected
index 24c7702090..5c869d9914 100644
--- a/misc/extension_api_validation/4.3-stable.expected
+++ b/misc/extension_api_validation/4.3-stable.expected
@@ -14,3 +14,11 @@ Validate extension JSON: Error: Field 'classes/ShapeCast2D/properties/collision_
 Validate extension JSON: Error: Field 'classes/ShapeCast3D/properties/collision_result': getter changed value in new API, from "_get_collision_result" to &"get_collision_result".

 These getters have been renamed to expose them. GDExtension language bindings couldn't have exposed these properties before.
+
+
+GH-90993
+--------
+Validate extension JSON: Error: Field 'classes/RenderingDevice/methods/draw_list_begin/arguments': size changed value in new API, from 9 to 10.
+
+draw_list_begin added a new optional debug argument called breadcrumb.
+There used to be an RID argument as arg #9 which was removed in GH-84976 but now we're adding a new one in the same location.

Aside from the above issue, we have a few other long-running problems that trigger recurring warnings in the logs, but somehow don't seem to break the CI checks, or we'd have fixed them ages ago:
ERROR: New API lacks base array: enums
   at: compare_dict_array (./core/extension/extension_api_dump.cpp:1362)

Happens when checking the 4.0-stable_4.1-stable.expected and 4.1-stable_4.2-stable.expected files.
ERROR: Validate extension JSON: Missing field in current API 'classes/RenderingDevice/methods/draw_list_end': arguments. This is a bug.
   at: compare_dict_array (./core/extension/extension_api_dump.cpp:1443)
ERROR: Validate extension JSON: Missing field in current API 'classes/RenderingDevice/methods/compute_list_begin': arguments. This is a bug.
   at: compare_dict_array (./core/extension/extension_api_dump.cpp:1443)
ERROR: Validate extension JSON: Missing field in current API 'classes/RenderingDevice/methods/compute_list_end': arguments. This is a bug.
   at: compare_dict_array (./core/extension/extension_api_dump.cpp:1443)

Happens for all stable version checks, but somehow not in the dev version (currently 4.3-stable.expected). This was the same in the previous dev cycle where it didn't occur for 4.2-stable.expected, so it's not that it was fixed in 4.3 / 4.4.dev.
Steps to reproduce
./misc/scripts/validate_extension_api.sh godot

Minimal reproduction project (MRP)
n/a

I've been looking into what's happening with PR #90993 and trying to figure out how the validate_extension_api.sh script is supposed to work.
It looks like it will concatenate all the *.expected files between the reference version (the version whose extension_api.json it's validating) and the current version, in order to attempt to account for all the progressive changes since the reference version.
For example, when the reference version is 4.1-stable, it'll concatenate the following *.expected files:

4.1-stable_4.2-stable.expected
4.2-stable_4.3-stable.expected
4.3-stable.expected

This should theoretically work, but has problems specifically with adding/removing arguments, because the validation messages will change. If the property had simply changed type in each release, we would have been fine, because all the validation messages about the type changes would be in the concatenated file, and all would be ignored.
Here's the first bunch of solutions that come to my mind:

Put the additional type change messages in 4.3-stable.expected, rather than scattering them into the old 4.2-stable_4.3-stable.expected files like @akien-mga describes as his workaround above. We really shouldn't change the old *.expected files after we've moved on to the next minor version, and adding the messages in 4.3-stable.expected would work because it'll always be concatenated together with the other files. This isn't great, because it's not obvious that this needs to be done, but it's also not the worst and we can document it.
Attempt to make the script smarter and recognize removed arguments. Like, we could look for messages about an arguments size change, and then automatically ignore any validation messages that refer to indexes higher than the size change. However, this wouldn't be a lot of fun to do in a shell script and with our relatively unstructured data. Completely reworking how this process works (like moving to Python or using more structured data for the *.expected files) seems like it'd be too much.
Rethink which things we consider to be errors. From the perspective of GDExtension compatibility, only a few things really matter like function hashes changing, constant values changing, and maybe a couple other things. I know the argument count and default values affect GDScript and C#, so the errors from this script can be helpful in calling out these changes for reviewers. But do we actually need them? If we could live without, then maybe we stop considering so many things to be errors in general?

There's probably more possible solutions that I haven't thought of.


fix(tx-builder): Content editable firefox bug

What it solves
Resolves https://github.com/gnosis/safe-react-apps/issues/400
How this PR fixes it
There is a bug in Firefox with contenteditable spans that makes copying the editable content copy the span itself as well
https://stackoverflow.com/questions/26057848/cut-paste-from-to-contenteditable-to-itself-does-not-work-in-firefox
We are changing the span to a div

All good, thanks!


Is this still maintained?

It's my first attempt to work with SAFE and I'm trying to make this work, however upon start, I get this:
let view = elmishView "Login" <| fun { Model = model; Dispatch = dispatch } ->
    let buttonActive =
        if String.IsNullOrEmpty model.Login.UserName ||			<--- line 83

Cannot read property 'UserName' of undefined
    at render (Login.fs:83)

Also, the tutorial from the README doesn't really match the existing code.
I see that the online demo is working.
Any ideas on how to fix it?
Thanks,
Daniel

Hi Daniel. The repository hasn't been touched in a while and needs some TLC.
If you want something that will get you some hands-on experience, have a look at the SAFE Dojo project.
We're also in the process of creating some other samples to learn from - watch this space.

Thanks, I'm trying to build a proof of concept app, using Fable and F#.
So I'll be watching this.

I would suggest you try out the SAFE Template itself to get up and running with a basic shell of an app and work from there rather than use this as a baseline.

#432 is for upgrading the application to SAFE v5


HeaderTooLarge when testing

I tried deploying it. I had this error:
Traceback (most recent call last):
File "/app/model.py", line 10, in
model = AutoModelForCausalLM.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2903, in from_pretrained
) = cls._load_pretrained_model(
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 3246, in _load_pretrained_model
state_dict = load_state_dict(shard_file)
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 447, in load_state_dict
with safe_open(checkpoint_file, framework="pt") as f:
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge
Any idea ?

same as #1
Please make sure your model file path is correct, you can refer to the following article
https://soulteary.com/2023/07/21/use-docker-to-quickly-get-started-with-the-official-version-of-llama2-open-source-large-model.html


Question about the shared codebook

Hi, thanks for your wonderful work! I'm studying your work with big interest!
Is there any reason to use shared code book for multiple scales?
Intuitively, it seems there might be a possibility of performance gain via seperate codebook. (distinguishing the role of each codebook)
I wonder if you've tried unshared codebook.
Again, thank you!

Thanks @jiho314. Although didn't try this, I also believe that an unshared one can yields better results because sharing the codebook would limit the model's expressive power. We use the shared codebook just for convince.

Oh I got it. Thanks for your kind reply!


Provision project-clone container separately from devfile Components

What does this PR do?
Don't add project-clone container to flattened DevWorkspace as used internally, instead add it as an init container on the deployment directly.
What issues does this PR fix or reference?
Mounted flattened devfile ($DEVWORKSPACE_FLATTENED_DEVFILE) contains project-clone. Accidentally applying this template causes the workspace to fail to start (DWO adds a second project-clone container).
Is it tested? How?

Start a DevWorkspace
Open exec into workspace deployment
cat $DEVWORKSPACE_FLATTENED_DEVFILE should not container project-clone component, command, or event.

PR Checklist

[ ] E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)

[ ] v8-devworkspace-operator-e2e: DevWorkspace e2e test
[ ] v8-che-happy-path: Happy path for verification integration with Che

Fixes https://github.com/devfile/devworkspace-operator/issues/709

/test v8-devworkspace-operator-e2e, v8-che-happy-path

/test v8-devworkspace-operator-e2e

/test v8-devworkspace-operator-e2e

/test v8-devworkspace-operator-e2e


Zeebe Simple Monitor Helm Chart

I am in need of a zeebe simple monitor helm chart but I see you do not have it. It is possible to include it in this suite? (as an extra service)

@Zelldon Would it be possible if I do it myself? I think it would be much easier to do it here than create my own and have 2 forks

Hey @michelefa1988 thanks for raising this but this is out of scope for this project. We want to have here only the official supported services and charts by camunda. For community supported charts, take a look at https://github.com/camunda-community-hub/community/issues/19 There are plans to create a new repo for such.


[Tandoor Recipes] CSRF verification error when logging in

Just saw that the add-on supports ingress, so I went to try it. And it works up to a point: I can't manage to login due to this error:

It seems to be this issue: https://github.com/TandoorRecipes/recipes/issues/518

Hi, I started to implement ingress but it is broken in part as the way the app works does not make it easy to implement. Either I remove it completely, or I'll continue working on it in the future, but for the moment you can only access directly with ip:port
Sorry about that

Has ingress addition bee completed as of now? Or still in progress?


Prototype Refactor

cross https://github.com/dimagi/commcare-odk/pull/454

@phillipm

@wpride Done with a first pass, added lots of questions.
I have one high-level question that has to do with my limited understanding with how prototype factories are used: The mStaticHasher is global. If there are multiple prototype factories around (does this happen) is there a problem that PrototypeFactory.setStaticHasher can swap out the hasher being used at anytime?

Also, I'm looking at fixing the failing test results reported by jenkins.

@wpride Also, please run the Android Studio code formatter (with the commcare style settings) on all the new files/large code blocks that you add.

@phillipm thanks for the feedback, think I've responded or fixed everything or referred to @ctsims  for some legacy code questions
As to the PrototypeManager, my understanding is that we basically use this for two things: maintain a prefix tree of classes we're interesting in hashing, and either statically retrieve/rebuild a PrototypeFactory with these prefixes when need.
I've made one more change that I think will work with this model: instead of having a separate NamedPrototypeFactory, we can just set the default PrototypeFactory static hasher so that we'll always use this hasher, even when we rebuild from the PrototypeManager.
I think the potential issue comes in when someone wants to use a PrototypeFactory super class - then you'd have to change the PrototypeManager to use rebuild and return this instead. Its worth noting that ODK already doesn't use this PrototypeManager (we do something similar in the DbUtil class). Possibly the next step of refactoring/unifying should be merging these classes and having a Manager that you can tell what PrototypeFactory class you want to return.

@damagatchi  retest this please

@wpride Have you checked out the failing test case?
org.javarosa.core.util.test.ExternalizableTest.doTests is throwing ```java.lang.AssertionError: Serialization Failure: Exception! java.lang.IndexOutOfBoundsException null

@wpride Minor comments, good after!

Looks good to me after addressing my surface level comments

Addressed with a couple questions for @ctsims ctsims


Grafana's rendering feature on FlowCharting panel does not work

I would like to create .PNG snapshot images for reporting purpose and tried to invoke the rendering URL on one FlowCharting panel:

I tried it with the deprecated PhantomJS renderer and with the new grafana-image-renderer plugin as well, booth give TimeoutError on FlowCharting panels.
This is the debug printout of grafana-image-renderer plugin when I invoke the rendering:
{"level":"info","message":"Render request received', 'url', http://grafana:3000/d-solo/EYdpLlaZk/borg-warner?orgId=1&from=1580291897741&to=1580313497742&panelId=2&width=1000&height=500&tz=Europe%2FBudapest&render=1"}
{"0":"url","1":"http://grafana:3000/d-solo/EYdpLlaZk/panel?orgId=1&from=1580291897741&to=1580313497742&panelId=2&width=1000&height=500&tz=Europe%2FBudapest&render=1","2":"error","3":{"name":"TimeoutError"},"level":"error","message":"Render request failed"}
{"message":"::ffff:<IP_ADDRESS> - - [29/Jan/2020:15:59:25 +0000] \"GET /render?domain=grafana&encoding=&height=500&renderKey=xxx&timeout=60&timezone=Europe%2FBudapest&url=http%3A%2F%2Fgrafana%3A3000%2Fd-solo%2FEYdpLlaZk%2Fpanel%3ForgId%3D1%26from%3D1580291897741%26to%3D1580313497742%26panelId%3D2%26width%3D1000%26height%3D500%26tz%3DEurope%252FBudapest%26render%3D1&width=1000 HTTP/1.1\" - - \"-\" \"Go-http-client/1.1\"\n","level":"info"}

Is it possible to create a snapshot of one FlowChart panel that way? Is it a bug or something should be implemented to make these panels renderable?

Need to try, never tested

The feature supports to share with externals a quasi online view of a dedicated FlowChart panel. Does anybody have any idea how to solve it?

For information, work in progress, i added this.renderingCompleted(), render function give a result but without colors ;

I add link with grafana issue
https://github.com/grafana/grafana-image-renderer/issues/67

@algenty -- you may want to add a slight delay after you think it is done before calling renderingCompleted()

Hi @ryantxu,
How do you do ?
I'll try it
A little question, with "Direct link rendered image" called, which event is called ? 'render' ? 'snapshot' ?
In code :
// Events Render
ctrl.events.on('render', () => {
this.render();
});

It works with a delay, but i'am not satisfied with this method because display is longer

Thank you, it works like a charm :)

Thank You very much guys!
Stay healthy!

You'r welcome :)
Be carefull, disable animation in flowchart section  if u use rendering
thx @ryantxu for your help


README

README
User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
Acceptance Criteria
GIVEN the prework project repository
WHEN I view the README for the repository
THEN I can see a professional README that provides information on the project and how to use it

the README is complete and viewable.


Add LND and AKRO icons

For the tokens LendingBlock and Akropolis

@monsieurleberre Thanks for the wait. I took some time to look at your pull request and will provide you with a detailed feedback.
For these two icons the icon and color versions do not match the style (icons are not in a circle and do not follow design guidelines). For example see AKRO from your pull request in the screenshot below.

Also, for both icons relative size between image and circle is off. See the icon for AKRO from your pull request below. It should be smaller, and could be somewhat aligned regarding circle to icon ratio with the ADA icon as it is quite similar in surface distribution.

Regarding LND specifically, icon color does not seem to be correct. Based on the website it is #1E1E1E instead of full black—sidenote, optical centering seems a bit off, so I will fix this as well.
Please open an issue with the base SVG files, so I can fix the aforementioned issues and add both icons.


Auto Location List should remember your place when you exit and return to the window

If you go to the auto location window, go down to the third line and press enter, it will take you to that location in the file.    It should remain where it was before.
This is something I miss from the straight flake8 plugin.  When there's lots of warnings, possibly some you've decided to skip, it can get pretty tedious to have to find the message you really want to fix.

The loclist window is managed by Vim, not by syntastic.  A better approach is to navigate through the list without switching focus to the error window, cf. FAQ.

@lcd047 hrmm, that will probably work, but now I wonder, if there's a way to activate the list without jumping to it, and instead using :lnext, etc

Nevermind. It does that by default.
Thank you, sir.


Refactor the save search template

This PR refactors the save search template, which was highlighted in a tech debt ticket from Sam as being a bit messy.
That assessment was I think borne out by how long this refactor took... I took this refactoring on because it was blocking my work to use WTForms more widely in the frontend apps (see alphagov/digitalmarketplace-frontend-toolkit#400).
I used WTForms to move most of the validation logic out of the view and into the CreateProjectForm, so overall the number of lines is fairly static; however with alphagov/digitalmarketplace-frontend-toolkit#400 the new emphasis on convenience methods for WTForms will mean a future reduction in lines.
When making changes I made sure that unit tests and functional tests remained green, and there should be as few visual regressions as possible, so some peculiarities remain in the design of this page:

The validation banner that appears at the top of the page when there is a form error is different to almost every other banner on this site; instead of showing the question name it shows the validation error message.

The search query is passed to the view both as a URL query string and (when POSTing) as a form field. As far as I could tell the search_query form field was never used in the view, but I did not remove it because doing so would mean that the tests would diverge from the code, and I didn't want to touch the tests.

Previously the form validation did not check whether the POSTed project was present in the list presented to the user in the selection box. It turned out that all the unit tests had been exercising this "feature" because the global project fixture was not in the global project list fixture. This is the one area where I did touch the tests, because whilst this might seem like a feature it is also a clear bug.

Notionally from 30,000ft, a hundred thumbs up. Will look closer later.


[Bug][c2cpg] import node can be find, but can not find which file import it.

joern> cpg.imports.id(455580L).l
val res10: List[io.shiftleft.codepropertygraph.generated.nodes.Import] = List(
Import(
id = 455580L,
code = "#include "asn-util.h"",
columnNumber = Some(value = 1),
explicitAs = None,
importedAs = Some(value = "asn-util.h"),
importedEntity = Some(value = "asn-util.h"),
isExplicit = None,
isWildcard = None,
lineNumber = Some(value = 19),
order = -1
)
)
joern> cpg.imports.id(455580L).file.l
val res11: List[io.shiftleft.codepropertygraph.generated.nodes.File] = List()
joern>

joern verison is  2.0.39

Do you have the source code you scanned somewhere available?

this is the source code


flight.base_url not taken into account for routes

A request to /base/books would not be successfully handled by the following setup
Flight::set('flight.base_url', '/base');
Flight::route('/books', function() {});

What is your server configuration and folder structure like? Where are you placing your index.php file?

I am not quite sure about your question.
You just need to drop the code above into an empty file, route all /base requests to this file and you will notice how it does not work.
My point is that base_url is only used for redirects (Engine._redirect()) but not for anything else such as routes.

Makes sense... I would suggest to add a new option e.g. flight.base_route.

Combination of flight.base_url and flight.base_route could work, but would be nice to handle files moved to a subdirectory globally by a path prefix used for routes, views, ...

index.php
-- core
--- flight
-- site
--- views
--- assets
--- ???

So files and routes inside of site should be handled by a global "site" prefix
Flight::set('flight.path_prefix', 'site');
Flight::set('flight.views.path', 'views');  // = default

So render('home', [...]) should load the view site/views/home.php.
It would also help to route assets like JS, CSS or images from a subdirectory.

I just tested this myself and it is working as expected.


Investigate use of Python 2 and Python 3 dependencies for building Salt packages

For Nitrogen still shipping Python 2 based Salt and dependencies.
For Oxygen, will have to generate Python 2 and Python 3 explicit packages.


remove ImageNet-C stimuli identifier to disable caching activations

the activations are not used a second time, like for the ImageNet benchmark. so we would only store unnecessary information
caching for temporal models like CORnet-S breaks when checking that coordinates are present because logits-t0 doesn't match logits

test failures primarily from S3 outage


✨(search) allows different search and courses page

Add RICHIE_SEARCH_REVERSE_ID setting that allows to have different search page and courses parent page.
Closes #1810

Closing in favor of this PR https://github.com/django-cms/django-cms/pull/7489 to Django CMS.


Consolidate and better tests for expression re-rewriting / aliasing

~Draft as it builds on https://github.com/apache/arrow-datafusion/pull/3710~
Which issue does this PR close?
re https://github.com/apache/arrow-datafusion/issues/3704
Rationale for this change
There are at least three places that we have rediscovered the issue of names changing when rewriting expressions such as https://github.com/apache/arrow-datafusion/issues/3704, #3555 and https://github.com/apache/arrow-datafusion/blob/master/datafusion/optimizer/src/simplify_expressions.rs#L316
I hope we can avoid other tricky bugs in the future
What changes are included in this PR?

introduce a function rewrite_preserving_name that handles aliasing during rewrites
Change code to use rewrite_preserving_name
Add tests

Are there any user-facing changes?
no

I think there are may place which use the ExprRewriter to rewrite the expr in the optimizer like ConstEvaluator and other.

I think there are may place which use the ExprRewriter to rewrite the expr in the optimizer like ConstEvaluator and other.
Do we need to fix all of them?

That is an excellent point @liukun4515  -- I think we probably do. I will file a follow on ticket to track and do so

Github actions is having issues I think -- the CI failures are not related to changes in this PR

I think there are may place which use the ExprRewriter to rewrite the expr in the optimizer like ConstEvaluator and other.
Do we need to fix all of them?

That is an excellent point @liukun4515 -- I think we probably do. I will file a follow on ticket to track and do so

I have less knowledge about the alias for the expr, can you explain when and where we should do the alias for the expr which will be rewrited.

I have less knowledge about the alias for the expr, can you explain when and where we should do the alias for the expr which will be rewrited.

@liukun4515  I have filed https://github.com/apache/arrow-datafusion/issues/3794 which describes the issue more generally and explains why and when aliases are needed

I have less knowledge about the alias for the expr, can you explain when and where we should do the alias for the expr which will be rewrited.

@liukun4515 I have filed #3794 which describes the issue more generally and explains why and when aliases are needed

I will take a look your issue with this pr https://github.com/apache/arrow-datafusion/pull/3788, but it will take more time for this.

Unless there are objections, I plan to merge this PR as it doesn't change any behavior and adds additional test coverage. We can continue to improve the code in other places it may be renaming incorrectly as part of https://github.com/apache/arrow-datafusion/issues/3794

Thanks @alamb , LGTM. Let's merge!

🥳


created a sample map that uses Mappa leaflets to highlight places a p…

…erson has been using data put in a simple csv. Json data for places is input in another file.

#7


Working representation of the resolved issue #97, #106 and #111

Hi @shubhamkakkar . There are too many files to be checked (52, whoah 😅, that's why there's never a good idea to push a PR for multiple issues at the same time -- one issue, one PR, on branch on your fork is the best idea for being more organized).
I got some really fast feedback so we can actually work with this PR:

Please remove the .idea files.
Only change files that are related to your issue or stuff that's related to the issue ( there's a lot of changes for lines that have nothing to do with the screen itself --> paths changed and stuff like that)

Thanks for your hard work!

I am actually closing this PR, and would make different PR for each working representation and also for segment component.


Failed to get it work with Debian Transmission package installation

Describe the bug
I installed Transmission via Debian package. The version is old, dating back to 2020.
apt list transmission-daemon
Listing... Done
transmission-daemon/stable,now 3.00-2.1+b1 arm64 [installed]

I'm trying to replace the original web UI with Transmissionic and followed the installation guide. Tried both methods.

Method 1: I appended export TRANSMISSION_WEB_HOME=... to /etc/profile. Then rebooted VM.
Method 2: There is no /usr/share/transmission/public_html folder as described, but only /usr/share/transmission/web. So I replaced the web folder instead. Then rebooted VM.

Neither is working and I keep seeing the original web UI.
To Reproduce

Provision a Debian 12 VM.
sudo apt update
sudo apt install transmission-daemon -y
Install Transmissionic web UI via either method described above.
Visit web UI and see the original.

Platform:

OS: Debian 12 VM
Browser: Microsoft Edge

hi there. did you switched on the remote mode in the main transmission app?


chore: update default pb config to match new structure

RVT-4060

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#1340  👈
#1339
#1338
#1337
#1336
#1335
#1334
#1333
#1332
#1329
#1328
#1327
#1322
#1321
#1320
#1319
#1318
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @NathanFlurry and the rest of your teammates on  Graphite


ci: Basic tests

What ❔
Adds basic tests
Checklist

[x] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.
[ ] Code has been formatted via zk fmt and zk lint.

closing since basic CI is already implemented in dev branch


✨ Dagger CLI: dagger develop with recursive code gen for multi module repositories

What are you trying to do?
Creating a "Daggerverse" that has many modules residing in the same Git repository is a common practice today. However, not everyone will commit the generated code up to the repository as it is not required for general dagger operations.
I need a way to tell dagger to recursively go into each of my modules and generate code with a single command so that I can lint the dagger modules without running into errors based on missing generated code.
Why is this important to you?
It is a tedious process to lint dagger module code as the generated code is required for linting to work. Especially in CI systems where a fresh copy of the repository is downloaded and the generated code is missing until dagger develop is run for each module. I want to be able to run golangci-lint on all my modules with as little effort necessary on my part.
How are you currently working around this?
cd mymodule
dagger develop
golangci-lint run --skip-dirs=internal --skip-files=dagger.gen.go

repeat for each module. Tedious and repetitive for a repository that could potentially house 100s of modules.

There is a bit more context to this here - https://discord.com/channels/707636530424053791/1251234150468288625
You can accomplish this using the source.AsModule().GeneratedContextDirectory() API, however, that is not well documented.
I think first class module linting support in the CLI can also help out module devs.


Automatted Instalation Enhancements

While building a very basic cPanel to EE (hosted on Vultr) automation script, I have had trouble finding a way for automated installation.
Current version has the ability to define these parameters
ee site create domain.ext --wp --user --email

What I want instead is to automated the migration and for that something like this would be needed
ee site create domain.ext --wp --user --email --db_name --db_user --db_pass --db_prefix

Any luck with that?

@ahmadawais we have already done some work on this in v4 esp around. Overall this should be present in first EE v4 release.

We are already tracking this feature in #1029


react-redux: mapDispatchToProps can return an object

This fixes #2284

There are test errors, you'll need to fix that first.

cc @villesau


fix: handle colima host

k3d mostly works flawlessly on github.com/abiosoft/colima, but host.k3d.interal points to nowhere useful. We detect colima runtime by the name in docker info output, then resolve the host IP by querying host.lima.internal.
How to test:
$ colima start
$ k3d cluster create
$ nc -l <IP_ADDRESS> 1234
In another terminal
$ kubectl run -it --rm --image alpine:latest test --  /bin/sh -c 'echo "hello there" | nc host.k3d.internal 1234'

What
Fixes HostIP detection when running on Colima.
Why
To fix host.k3d.internal
Implications
Unsure. Works on my machine with the patch...

That's a nice addition, thank you @thejan2009 ! :+1:


Add a command to retrieve all source roots of the project

For copilot to be able to find/handle files in a multi module project, it needs to know where to look for sources. The command should return a list of all source folders (absolute path) so that all ”main” sources are first (most likely to contain what copilot is looking for) and then all ”test” source folders

It can be achieved using:
ModuleManager.getInstance(project).modules.forEach { module: Module ->
   val moduleRootManager = ModuleRootManager.getInstance(module)
   moduleRootManager.getSourceRoots(JavaSourceRootType.SOURCE)
   moduleRootManager.getSourceRoots(JavaSourceRootType.TEST_SOURCE)
}


Microphone permission not working

It appears that the microphone permission is not working. To reproduce this, I simply modified the PermissionsTestViewController.m file and changed the  [permissions showPhotoPermissionsWithTitle...] call to  [permissions showMicrophonePermissionsWithTitle...]
No other changes.  Now, the UIAlert fires. But no matter which option I select -- 'Not Now' or 'Give Access' -- user action is always denied and system action is always 'no action taken'.  And hasPermission is, therefore, always NO.
Any ideas?

Thanks @kwade101.
I've been super busy lately, but I'll take a look soon.

@rsattar
Problem is inside ClusterPrePermissions.m at line 218, you use AVMediaTypeVideo for microphone, but it's AVMediaTypeAudio type :)

Thanks everyone. Pushed 0.6.0 that has a fix for this. Sorry for the delay.


fidelity of floating point emulation

I am writing a sleigh spec for a "new" processor. To check my work I am executing the pcode and verifying it against
a "known good" model (the actual hardware, for the problem below).
The issue I have run into is that floating point division (FLOAT_DIV) is generating minor errors compared to the actual hardware.
Interestingly, the problem reproduces on x86. So, I expect the issue is generic for FLOAT_DIV and would apply to all architectures. The problem can be reproduced with a "tiny" rom image, and a python script...
bash$ printf '%b%b%b%b' "\xf3" "\x0f" "\x5e" "\xc1" > /tmp/x86.rom
Loading this into ghidra at 0x0 as x86:LE:64:default should give "DIVSS XMM0,XMM1" as disassembly.
This can then be executed in conjunction with the small python script (below):
./analyzeHeadless /tmp x86 -noanalysis -import /tmp/x86.rom -loader BinaryLoader -loader-baseAddr 0x00000000  -loader-blockName myblock -processor x86:LE:64:default  -postScript /tmp/x86.py
In the output, it will print:
0x40bb8000
the correct value, as executed on x86, (and my target hardware) is: 0x40bb8001 <- note ends in 1...
Can someone familiar with the inner workings of the float code comment on whether this can be fixed by setting a rounding mode or some other "global" control? I am using ghidra_9.1.2_PUBLIC and jdk-11.0.8
cat > /tmp/x86.py

def main():
    emuHelper = EmulatorHelper(currentProgram)
    emuHelper.writeRegister("XMM0", 0x42580000)
    emuHelper.writeRegister("XMM1", 0x411374bc)
    emuHelper.writeRegister("RIP", 0)
    emuHelper.writeRegister("RSP", 0)
    emuHelper.step(monitor)
    print "{:#010x}".format(emuHelper.readRegister("XMM0"))

main()

The emulator floating point arithmetic is done by the class ghidra.pcode.floatformat.FloatFormat.  The method opDiv converts each target value to native Java double, does the division, then returns getEncoding of the result.  The bug is in getEncoding.
The mantissa of Java's double precision result is truncated from 52 to 23 bits, it ought to be rounded.
see here

@jpleasu  May be of some interest/related to this issue, but I played around a bit with pcodetest and FP operations and it was not going well and couldn't get reliable results. (#1479)

@mumbel  Thanks for the pointer -- you're right, they're related.  EmulatorHelper is used by ProcessorEmulatorTestAdapter, the base class for the pcode tests.  Your convert tests make for a really good goal.

Thanks for your comments! It's taken me a while to circle back to this...
@jpleasu the pointer you gave into the code helped me greatly...
I built a new version of ghidra with this one line patch:
git diff
diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcode/floatformat/FloatFormat.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcode/floatformat/FloatFormat.java
index 2b97ac6..b1e7c33 100644
--- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcode/floatformat/FloatFormat.java
+++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/pcode/floatformat/FloatFormat.java
@@ -370,6 +370,8 @@ public class FloatFormat {
                if (exp != 0 && jbitimplied)
                        signif <<= 1; // Cut of top bit for normal case only (which should be 1)

+               signif = signif + ((1L << (64 - frac_size)) >> 1);
+
                long res = (signif >>> (64 - frac_size)) << frac_pos;
                res |= (long) exp << exp_pos;
                return setSign(res, data.sign);

and that proved to be correct for 100 or so tests cases that I have in a practical use case...
Caveat emptor though :-) I cannot claim that the test cases I have constitute coverage... and I'm not truly confident in the fix as I don't understand the ramifications of:
                if (exp != 0 && jbitimplied)
                        signif <<= 1; // Cut of top bit for normal case only (which should be 1)

Anyway, I hope posting this comment may help others.

Sorry for the lack of updates - I'm almost done with a pretty thorough set of fixes for Ghidra's emulation of floating point.  It should go in for review this week.
Ghidra's emulation of floating point is incorrect in two ways -- the first, which you ran into, is the rounding.  IEEE754 prescribes rounding to the nearest even for binary floating point values, but Ghidra's emulator truncates.  Ghidra doesn't need to round when
emulating doubles (since native doubles are used), but anything else and Ghidra will be wrong.  The second issue is incomplete support for "subnormal" numbers.
I test against pseudo-random values to catch rounding errors and for the many corner cases I include +- zero, +- infinity, nan, +-min, max subnormal, max subnormal +- min, and +-max.

@jpleasu thanks for the update.
I have found an issue that I think is related... This time in int2float. I have an instruction in my sleigh code which (paraphrasing is):
rd = int2float(ra)
where ra and rd are 32b. When ra is set to (0x80000000 | 0x12) the result in rd is "ce800000" -- on the actual hardware, the result is cf000000 [some c on x86 also produces cf000000]. As you can see the mantissa is 0, but the exponent field (bits [30..23]) is wrong 157 (0x9d) vs the correct 158 (0x9e)... Which (I'm guessing) means the exponent hasn't been correctly adjusted somewhere.
I'm wondering if this is on a common code path with the float issues you are chasing down(??) and is just another aspect of this issue.
FWIW: the int2float conversion of 0x12 is correct --- when I wrote the test harnesss, I threw in the or'ing of 0x80000000 to create an additional test and this raised the issue!

Yes, it's a common path and the new emulator produces the correct result, I tested with:
OpBehaviorFloatInt2Float op = new OpBehaviorFloatInt2Float();
long result = op.evaluateUnary(4, 4, 0x80000000 | 0x12);
System.err.printf("%x\n", result);
// produces:
// cf000000

evaluateUnary calls to FloatFormat#opInt2Float which calls getEncoding, where erroneous truncation happens.

@jpleasu Thanks for explaining the issue with int2float !

Corrected for 9.2.3 release

Corrected for 9.2.3 release


getting error popup

Does this occur consistently?
Repro steps:

Action: startArmLanguageServer
Error type: Error
Error Message: ARM Template Language Server: An error occurred starting the language server.
Connection to server got closed. Server will not be restarted.
Version: 0.15.11
OS: win32
OS Release: 10.0.19044
Product: Visual Studio Code
Product Version: 1.87.2
Language: en

Call Stack
extension.bundle.js:16:294550extension.bundle.js:16:294550
rejected extension.bundle.js:16:285678

If you'd like me to investigate, I need more info.  Does this occur consistently?  If so, are there repo steps you can share?  Thx.


Invalid loader type of void when loader function has promise.then(data => data.x)

Describe the bug
When the loader method has promise chaining like loader: async () => myDataLoader().then((d) => d.msg),, the inferred type includes void, as in my simple example string | void.
When the loader has this restructured to first set the result into a field and then return the field, it is OK.
Your Example Website or App
https://www.typescriptlang.org/play?#code/JYWwDg9gTgLgBAbzgYygUwIYzQJQgV2wBoV0tcIIY9C04BfOAMyghDgCIABGDAOwDOvZAGsA9GWQwAtK1pQOAbgBQy5BEHxWVGtjgBeUpmx4dBbAAoE9AJQrlGAQE8+yZvlcxgGuCCcARLAwAGQgMABM0KAsbAC44AAVWEGABNAAeJBABAHN4oShgPhyGAD5EZTg4dBh8KD5E5NS0ADp0AQgAGwA3NCtfXPiOAAs0Ts6IDgY7ZXpVdU04Is6iiloDI3JdPoRKuBy0GASMdD5qczR4mINy7XPaIj2wLGGhsSZKDkeqiYio+McLjc1305T8gV4oT+0RsLRgoz4Fgs4RsNzg4Ra2RyNketnsCyEcAARlAMKJDtsNqhjGtLLsqgcjic0GdtldUaDqpR7sQni83kSTl89r9IlAAc5XHAQeV6VUUBpCeEghsMAB3DDAeDgoJQsUxOEIpEotEYrEzPZVGp1BrK3gqKr0XEWgnwAD6y1WEIwG09fFprXwaT1UW9MRUrrgbpJZJEh29Gxj5J5geDYTFYZmQA
Steps to Reproduce the Bug or Issue

Create route
Add loader
Add loader method with promise.then(....

Expected behavior
The type is inferred correctly
Screenshots or Videos
No response
Platform
@tanstack/react-router: 1.26.3
Additional context
No response

Not sure why but the issue goes away when removing async from this function (that doesn't have await in it anyway):
https://www.typescriptlang.org/play?#code/JYWwDg9gTgLgBAbzgYygUwIYzQJQgV2wBoV0tcIIY9C04BfOAMyghDgCIABGDAOwDOvZAGsA9GWQwAtK1pQOAbgBQy5BEHxWVGtjgBeUpmx4dBbAAoE9AJQrlGAQE8+yZvlcxgGuCCcARLAwAGQgMABM0KAsbAC44AAVWEGABNAAeJBABAHN4oShgPhyGAD5EZTg4dBh8KD5E5NS0ADp0AQgAGwA3NCtfXPiOAAs0Ts6IDgY7ZXpVdU04Is6iiloDI3JdPoRKuBy0GASMdD5qczR4mINy7XPaIj2wLGGhsSZKDkeqiYioq5sN18ASCoT+0RsLRgoz4Fgs4UB+nK4Ra2RyNketnsCyEcAARlAMKJDtsNqhjGtLLsqgcjic0GdtgCgXdtt84M9oW88ScvntfpEoPFHC43NckRUqlUcfBwkENhgAO4YYDwPyBXhgwUxKEwuEIoEotEzPZVGp1Bpy3gqKr0TEmmVwAD6y1WGowG1dfEprXwaS1UXdMWUYjEUrgAD0APxqDS4p0EokiQ7ujaJ4n3bAtP1oANQIMzIA
I generally wouldn't mix async/await with promise chaining - choose one and stick with it :)

That solves it for now, but probably more devs will face this inconsistency. await Promise.resolve('foo') should be the same type as await Promise.resolve().then(() => 'foo), both should be Promise<string>.
Additional hint, was using previously version 1.17.4 and this behaviour was OK. The regression was done between this and 1.26.3


Update proj3 with scorecard

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (df2d1f3) 81.28% compared to head (50a1741) 81.28%.

:exclamation: Current head 50a1741 differs from pull request most recent head 860e659. Consider uploading reports for the commit 860e659 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #61   +/-   ##
=======================================
  Coverage   81.28%   81.28%
=======================================
  Files          27       27
  Lines        2191     2191
=======================================
  Hits         1781     1781
  Misses        410      410

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Start app from notifications + auto-launch at startup + changelog hide option + translations

Hello !!!
I've done a few changes that I hope you will integrate :

A "Do not show again" checkbox for the changelog. It's cool to see the changes but it's annoying to have them at each start. It is automatically unchecked when the app is updated.

Launch the app when clicking "Open Fit" in the notification bar

Add a checkbox to be able to auto-launch the app on startup of the smartphone

Add some French translations

Best regards,
Régis

awesome thanks @rdupuis  took me long enough to test and verify.


Cluster Awareness

Let's say you have multiple processes logging to the same file:
var cluster = require("cluster");
var log = require("bristol");

var num_cpus = 2;

if (cluster.isMaster) {
    for (var i = 0; i < num_cpus; i++) {
        cluster.fork();
    }

} else {
    log.addTarget("file", { file: "log.txt" });
    log.addTarget("console")
        .withFormatter("human")
        ;
    log.info(process.pid, "Lorem Ipsum");

}

Then run the script, and while the script is running, fire up lsof:
$ lsof ./log.txt
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
node    49511 doug   13w   REG    1,2<PHONE_NUMBER>2 ./log.txt
node    49512 doug   13w   REG    1,2<PHONE_NUMBER>2 ./log.txt

Two processes, two different file descriptors, same file.  Under UNIX, there is no guarantee that writes to a file are atomic, thus running the risk of partial lines being written to one file descriptor before the next line is written, thus creating a "staircase" effect.
Opening files in the master process then spawning the child process won't help. (I tried)
The best solution I found so far is to have a single process write files.  This can be done with process.send() as follows:
//
// In the master process
//
process.on('message', function(m) {
   // Write to file
});

//
// In the child process, JSON data structure is sent to the master process
//
process.send({ foo: 'bar' });

There are probably other ways to accomplish the same.  This particular technique worked for me in my projects.

I'm going to close this for now. While there's a good solution to this, it would appear to be not worth the effort at this point. There's a strong consensus that for most use cases, if you're running Node on a machine with multiple CPUs, you should either:

(if you're on a cloud provider) spend the same amount of money to turn that machine into multiple machines with single CPUs and run multiple instances of your app, or:
(if you're on discrete hardware) use a hypervisor or container scheduler to run multiple instances of your app on that machine as though they are individualized machines.

With the above, you multiply your durability for literally free, and gain the advantage of a less complex codebase as a result.
I'll reopen if this becomes a popular request!


NodeRed updates .config.json with local path instead of npm path

This is related to a previously reported issue (https://github.com/node-red/node-red/issues/1756) and happens only when developing custom nodes locally, & then deploying them.
Steps To Reproduce:

npm install a node locally (e.g., npm install /path/to/node/package),
run node-red locally
verify that the module appears in the palette & functions fine)
stop node-red
npm uninstall that same module: npm uninstall {path/to/package}
clear browser cache & history
search entire node red project for the module (Ctrl+F) to ensure it is completely gone from the project (make sure it's gone from .config.json, the node_modules folder, & the package.json files)
Install the module of the same name, but from npm (not locally), e.g.,
npm install node-red-contrib-c8y-events
start node-red
observe that the .config.json updates, setting the "file" parameter equal to the local path, instead of the npm path (/usr/local/lib/..etc)

What happens?

When updating the .config.json file at runtime, Node-Red appears to "remember" the local path of the locally developed node, and it injects that path to the flow, instead of the correct path from npm.

The node will appear in the palette, but if dragged onto a flow, it will not work, and the console will reveal the following error:

RED.events.emit error: [nodes:add] TypeError: Cannot read property 'module' of undefined
red.min.js:16 TypeError: Cannot read property 'module' of undefined
at Array. (red.min.js:16)
at Object.emit (red.min.js:16)
at Object.X [as add] (red.min.js:16)
at HTMLDivElement.drop (red.min.js:16)
at e.(:443/anonymous function).(anonymous function)._trigger (http://localhost:443/vendor/vendor.js:16:10003)
at e.(:443/anonymous function).(anonymous function)._drop (http://localhost:443/vendor/vendor.js:16:40380)
at e.(:443/anonymous function).(anonymous function)._drop (http://localhost:443/vendor/vendor.js:16:5029)
at e.(:443/anonymous function).(anonymous function). (http://localhost:443/vendor/vendor.js:16:42279)
at Function.each (vendor.js:2)
at Object.drop (vendor.js:16)

What do you expect to happen?
When installing a new module from npm, the .config.json should update with the npm path of that module -- not the local path from earlier development.
Environment:

[ ] Node-RED version: v0.18.7
[ ] node.js version: v9.11.1
[ ] npm version: 6.1.0
[ ] Platform/OS: MacBook Pro: MacOS High Sierra Version 10.13.4 (17E202)
[ ] Browser: Google Chrome Version 66.0.3359.181

Comments:
This could also reveal some sort of problem with npm? In which case I'll submit an issue to them instead.  Interested to hear your thoughts.

As I said on the previous issue, Node-RED only writes to .config.json what it finds. If it has written the 'local path', then that is where it has found the files.
Comparing the two npm install commands you've listed:
npm install /path/to/node/package
npm install node-red-contrib-c8y-events

There is no difference in where npm will install the files. The only difference is where it gets the package from in order to install.
If you are in the directory /home/pi/.node-red, running either of those npm install commands will result in the module being installed in /home/pi/.node-red/node_modules/node-red-contrib-c8y-events.
npm will only install to /usr/local/lib/.. if you specify the -g option with the install command in order to install it as a global module.

Hey Nick,
Thanks for your patience & explanation. I see what you are saying. I muddled the clarity of my original issue by incorrectly assuming the cause.  Unfortunately, I do still have a problem, which I will restate clearly in another issue.  Sorry about it!  This time I'll be really really clear & won't make any assumptions.  I think this next issue will be the last one I post. 3rd time is the charm!
Best,
Allie

No problem. If you wanted to discuss on slack or the forum before raising an issue then that may be easier. Links at the bottom of nodered.org

Oh great! I actually just sent you a direct email haha
I will check out the forums as well


Rename forge--{ list => ls }-notifications-all

The function forge--list-notifications-all was recently renamed to forge--ls-notifications-all (https://github.com/magit/forge/commit/0394d232e04722a0421b024828de0301c9a1153e), causing gh-notify--forge-get-notifications to break. This patch fixes this.

Superseded by https://github.com/anticomputer/gh-notify/pull/14.


Hardware wallet integration

Provide a way to use hardware wallets with KEEP token dashboard.

Dropping slides from Figma into here! I researched some dApps (mainly Oasis) to see how they were handling this. I've currently included MetaMask, Ledger (Live and Legacy), Coinbase, and Trezor.
MetaMask: Should link to the download page in an external window if it's not installed and have a web3 error
Coinbase: Will show a QR code, link to download Coinbase Wallet
Ledger: Shows Ledger Live and Ledger Legacy and a 'connect' button that I assume looks for something plugged into the USB drive?
Trezor: Links out to an external wizard

The far left modal window shows all the options supported, and the other two are links from when you click either Ledger or Coinbase, respectively.

Full screen for context (you can see there's an error message that's saying "Please connect a wallet" in the top lefthand corner).

I think that the modal should show initially, though, and not require the user to click on the "Please connect to a wallet" link.
I'm thinking this link would be there for when the user closes out of this modal but still needs to link a wallet.

The copy has changed in Figma for most of these, btw. Also, addition of images to the hardware wallet modal flows:


refguide-check fails on master

Recent PRs fail the refguide-check CI run, e.g. https://travis-ci.org/scipy/scipy/jobs/422178274
Most failures are due to some changes in the amount of whitespace numpy decides to use when printing arrays.
Usually, plain doctests are annoying to work with precisely because they tend to fret on whitespace. To counter that, our refguide-check tries to detect floating-point comparisons and delegate them to numpy. It seems that the detection fails in several classes of cases:

[ ] printing n-dim arrays. We added some logic for detection of the result of printing an array in gh-9178, but it does not handle ndim>1.

For example:
File "build/testenv/lib/python3.6/site-packages/scipy/cluster/hierarchy.py", line 1788, in inconsistent
Failed example:
    print(Z)
Expected:
    [[ 5.          6.          0.          2.        ]
     [ 2.          7.          0.          2.        ]
     [ 0.          4.          1.          2.        ]
     [ 1.          8.          1.15470054  3.        ]
     [ 9.         10.          2.12132034  4.        ]
     [ 3.         12.          4.11096096  5.        ]
     [11.         13.         14.07183949  8.        ]]
Got:
    [[  5.           6.           0.           2.        ]
     [  2.           7.           0.           2.        ]
     [  0.           4.           1.           2.        ]
     [  1.           8.           1.15470054   3.        ]
     [  9.          10.           2.12132034   4.        ]
     [  3.          12.           4.11096096   5.        ]
     [ 11.          13.          14.07183949   8.        ]]

The simplest way of fixing these is probably to change in the examples, >>> print(Z) to just >>> Z
Otherwise, the relevant logic is https://github.com/scipy/scipy/blob/master/tools/refguide_check.py#L570

[ ] Somewhat trickier: numpy arrays hidden in textual output.  Note the whitespace in the slack and x arrays.

scipy.optimize.linprog
----------------------
File "build/testenv/lib/python3.6/site-packages/scipy/optimize/_linprog.py", line 1059, in linprog
Failed example:
    print(res)
Expected:
         fun: -22.0
     message: 'Optimization terminated successfully.'
         nit: 1
       slack: array([39.,  0.])
      status: 0
     success: True
           x: array([10., -3.])
Got:
         fun: -22.0
     message: 'Optimization terminated successfully.'
         nit: 1
       slack: array([ 39.,   0.])
      status: 0
     success: True
           x: array([ 10.,  -3.])

Maybe it's best to just change doctest output. Alternatively, one can teach the doctester to search for array([<whatever>]).

[ ] namedtuples and similar objects:

File "build/testenv/lib/python3.6/site-packages/scipy/stats/morestats.py", line 97, in bayes_mvs
Failed example:
    std
Expected:
    Std_dev(statistic=2.9724954732045084, minmax=(1.7823367265645143, 4.945614605014631))
Got:
    Std_dev(statistic=2.9724954732045084, minmax=(1.7823367265645143, 4.9456146050146312))

Again, there's some rudimentary logic for handling these sorts of issues, https://github.com/scipy/scipy/blob/master/tools/refguide_check.py#L587
but it is also too simplistic.

I'd add that I'm willing to work on at least some of these --- time's a bit tight though, and there is no definite ETA from my side.

Or, maybe, just maybe these issues disappear in a puff of smoke if we use the newest numpy in the wheel run (it seems to be 1.13 ATM)

I think we aim to target the refguide to the numpy output as it is in the latest version.

All refguide tests pass

$ python runtests.py -g --refguide-check
Building, see build.log...
Build OK (0:00:11.520894 elapsed)
Running checks for 30 modules:
scipy.cluster .. scipy.cluster.hierarchy
................................................................
scipy.cluster.vq .......... scipy.constants
...............................................................................
...............................................................................
........... scipy.fftpack
................................................................
scipy.fftpack.convolve ......... scipy.integrate
........................................................ scipy.interpolate
...............................................................................
................... scipy.io .......................... scipy.io.arff
......... scipy.io.wavfile ....... scipy.linalg
...............................................................................
...............................................................................
............................... scipy.linalg.blas
...............................................................................
..................................................................
scipy.linalg.lapack
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...........................................................
scipy.linalg.interpolative ........................ scipy.misc ............
scipy.ndimage
...............................................................................
................................................................... scipy.odr
.................................. scipy.optimize
...............................................................................
....................................... scipy.signal
...............................................................................
...............................................................................
...............................................................................
...............................................................................
.................. scipy.signal.windows
.................................................. scipy.sparse
............................................................................
scipy.sparse.csgraph ................................................
scipy.sparse.linalg
..................................................................
scipy.spatial .................................. scipy.spatial.distance
............................................................
scipy.spatial.transform ...... scipy.special
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
................................. scipy.stats
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
............................... scipy.stats.mstats
...............................................................................
.................................................................

if I use
>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.2.0.dev0+80c9b7e 1.14.2 sys.version_info(major=3, minor=5, micro=2, releaselevel='final', serial=0)

Just out of interest how does the refguide work? Does it do regex's against text, or does it convert to numpy array and use np.testing.assert_allclose?

@andyfaff the checker's here: https://github.com/scipy/scipy/blob/master/tools/refguide_check.py#L533
Basically, it first tries plain doctests; if that fails, it does np.allclose.
The rest are details, and it's exactly where things go south now.
It generally works rather well, and kept working for several years with essentially zero maintenance, until (I suppose) numpy improved their printing. We might mostly get away with just tweaking the expected output.

It generally works rather well, and kept working for several years with essentially zero maintenance, until (I suppose) numpy improved their printing. We might mostly get away with just tweaking the expected output.

@ev-br does numpy have their own doctest code for examining their output? If so, one would hope that it's kept up with the changes to the printing that have been going on. If there is, and it has, then we could use it.

We fixed the printing to latest numpy format in gh-8279, so I think we only need to tell CI to use latest numpy: gh-9202


[helm/alertmanager] Reference external secret for Alertmanager config

There are cases like using API tokens on webhooks, that defining those in plain text on values.yaml is not an option

Ping @gianrubio

@brancz @gianrubio friendly ping.
Without that it's impossible for us to add Alertmanager integrations that require authentication as I said before.

@gianrubio @brancz is there any interest on this?

There seems to be an issue with the testing btw:
go: disabling cache (/.cache/go-build) due to initialization failure: mkdir /.cache: permission denied

@gianrubio rebased.
Test fails because of:
--- FAIL: TestPrometheusReloadRules (313.08s)
	prometheus_test.go:470: waiting for alert 'secondAlert' to fire: timed out waiting for the condition: expected 1 query result but got 0

@gianrubio bump the versions again

@gianrubio friendly ping, this is a trivial patch but is very much required.


Segmeth error "IndexError: list index out of range"

Really useful tool and am looking forward to using it. However, I can't get beyond the first step. I am trying to run segmeth but I get the below error. Would greatly appreciate any leads to resolve it. Thank you.
methyartist version: 1.2.3 (installed using conda)
Command: methylartist segmeth -b ../sample.haplotagged.bam --ref genome.fa --motif CG -i ../genome.bed -p 32
Std out:
2023-06-19 15:10:17,156 found mod names: m,h
2023-06-19 15:10:17,156 segmeth output filename: genome.sample.haplotagged.segmeth.tsv
[W::hts_idx_load3] The index file is older than the data file: ../sample.haplotagged.bam.bai
[W::hts_idx_load3] The index file is older than the data file: ../sample.haplotagged.bam.bai
2023-06-19 15:10:17,640 warning: collision between mods, this is a bug! report me!
2023-06-19 15:10:17,640 warning: collision between mods, this is a bug! report me!
Error message:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/XXX/XXX/miniconda3/envs/methylartist/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/XXX/XXX/miniconda3/envs/methylartist/bin/methylartist", line 729, in get_segmeth_calls
for row in parse_methbam(bam_fn, reads, chrom, seg_start, seg_end, meth_thresh=0.8, can_thresh=0.8, restrict_ref=args.ref, restrict_motif=args.motif):
File "/XXX/XXX/miniconda3/envs/methylartist/bin/methylartist", line 676, in parse_methbam
genome_pos = ap[base_pos[i]]
IndexError: list index out of range
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "XXX/XXX/miniconda3/envs/methylartist/bin/methylartist", line 4826, in
main(args)
File "/XXX/XXX/miniconda3/envs/methylartist/bin/methylartist", line 4546, in main
args.func(args)
File "/XXX/XXX/miniconda3/envs/methylartist/bin/methylartist", line 2179, in segmeth
meth_result, seg = res.get()
File "/XXX/XXX/miniconda3/envs/methylartist/lib/python3.10/multiprocessing/pool.py", line 774, in get
raise self._value
IndexError: list index out of range

Hi, hope we can get it working for you. Let's upgrade from 1.2.3, that's quite old, the conda build for methylartist broke quite awhile back and I don't have the bandwidth to troubleshoot nor was it obvious how to remove something from bioconda. In your methylartist env could you run:
git clone https://github.com/adamewing/methylartist.git
cd methylartist
pip install -e $PWD

Try the same command with the updated version and let me know how you go.

Thanks for the quick reply Adam. The latest version works.


should we use greedily decoded sentence for baseline reward computation?

In the paper it is mentioned that the idea is to baseline the REINFORCE algorithm with  the  reward  obtained  by  the  current  model  under  the inference algorithm used at test time.
What if we want to use topk sampling at the test time? should we still use greedy decode output for baseline reward?
Thanks

Model trained with self crtitical usually can't work with topk because the posterior is too peaky.

Model trained with self crtitical usually can't work with topk because the posterior is too peaky.

@ruotianluo Thanks for you quick reply.
In this regard:

does it make sentence to have baseline reward using greedy, however during inference we use topk?
Do people usually use beam search during inference for a model trained by self critical?
In general what is the best approach if I want to use RL to finetune my model? Should I only use beam or greedy during inference?

can't understand.
Yes.
Best approach of what?

1. can't understand.

2. Yes.

3. Best approach of what?

1- You mean if we train self-critical, using topk sampling during inference time won't work, right?
3- In other words, only beam search/greedy decoding (during inference) work with self-critical?

Sorry for the ambiuty

Won't work means it will not generate very different captions because the posterior is too peaky.
Any decoding would work in terms of generating a caption; however, they may not work as you would expect.

You can certainly try,


update the code of script to support latest mingw gcc versions

got this error :

/ _ \_____      _____ _ __/ _\ |_ __ _  __ _  ___ _ __

/ /)/ _ \ \ /\ / / _ \ '_\ | / _ |/ _ |/ _ \ '|
/ / () \ V  V /  __/ |  \ \ || (| | (| |  / |
/    _/ _/_/ _||  _/__,|_, |___||
_ __  _   _                           |/
| ' | | | |
| |) | || |  A payload stager using PowerShell
() ./ _, |  Created by z0noxz
||    |_/
Traceback (most recent call last):
File "power.py", line 2962, in
main(sys.argv[1:])
File "power.py", line 2949, in main
compile_source()
File "power.py", line 2694, in compile_source
Print.add_name_value("File signature", hashlib.md5(open(output, "rb").read()).hexdigest(), Print.info)
IOError: [Errno 2] No such file or directory: 'windowsbd'
with gcc-mingw-w64-i686 version 8.3.0
after downgrading to 4.9.1 version in debian jessie repository the problem disapeared .
your script requires all these packages
gcc-mingw-w64-i686
gcc-mingw-w64-x86-64
binutils-mingw-w64-i686
binutils-mingw-w64-x86-64
and right now it will only work if these packages came from jessie repository .

I have very limited time for this at the moment. If anyone else has the time,
feel free to send a pull request.
On 2019-12-14 15:48:01, peterpt wrote:

got this error :

/ _ \_____      _____ _ __/ _\ |_ __ _  __ _  ___ _ __

/ /)/ _ \ \ /\ / / _ \ '_\ | / _ |/ _ |/ _ \ '|
/ / () \ V  V /  __/ |  \ \ || (| | (| |  / |
/    _/ _/_/ _||  _/__,|_, |___||
_ __  _   _                           |/
| ' | | | |
| |) | || |  A payload stager using PowerShell
() ./ _, |  Created by z0noxz
||    |_/
Traceback (most recent call last):
File "power.py", line 2962, in
main(sys.argv[1:])
File "power.py", line 2949, in main
compile_source()
File "power.py", line 2694, in compile_source
Print.add_name_value("File signature", hashlib.md5(open(output, "rb").read()).hexdigest(), Print.info)
IOError: [Errno 2] No such file or directory: 'windowsbd'
with gcc-mingw-w64-i686 version 8.3.0
after downgrading to 4.9.1 version in debian jessie repository the problem disapeared .
your script requires all these packages
gcc-mingw-w64-i686
gcc-mingw-w64-x86-64
binutils-mingw-w64-i686
binutils-mingw-w64-x86-64
and right now it will only work if these packages came from jessie repository .
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/z0noxz/powerstager/issues/16


Appstudio update test-custom-default-branch-xicf

Pipelines as Code configuration proposal

Pipelines as Code CI is skipping this commit.User rhtap-qe-bots is not allowed to run CI on this repo.

Pipelines as Code CI is skipping this commit.User rhtap-qe-bots is not allowed to run CI on this repo.


Put Peruvian Earthquakes in Quakeledger

So that there are EQs available that TS-Service and Shakyground can also work in Peru.

Works like a charm :)


Non-leader identifiers are being incorporated into drug-chemical conflation

Non-leader identifiers without a Biolink type were being incorporated into drug-chemical conflations, causing NodeNorm to crash. I looked through a few examples and was unable to replicate this in the current Babel run, but we should investigate more thoroughly and make sure this doesn't happen again.
This bug was originally reported at https://github.com/TranslatorSRI/NodeNormalization/issues/221. I've closed that issue because I added a workaround for NodeNorm to suppress this error, but it really shouldn't be happening at all.

This seems to be fixed with the updated conflation generation code, now. Closing.


Compare Hex Files

I can no longer compare hex files using the "Select for Compare" and "Compare with Selected" feature when using the ms-vscode.hexeditor extension. The comparison opens but they are both blank.
VS Code version 1.58.2
ms-vscode.hexeditor version 1.8.2

Here's what it looks like when I do it.


Support for Pylontech Force H2 battery

Hi,
first of all, thans for great HA integration.
Pylontech Force H2 battery pack uses Solarman v5 logger, so i checked available addresses on local tcp/modbus using jmccrohan/pysolarmanv5 register_scan.py.
Found valid addresses in range of 03972-06911 (scanned 0-9999)
Not sure if, everything is correct. Must do some more testing and observation, since documentation for this is none.
More detail and list of all availbe address blocks: https://gist.github.com/pavelmaca/170c282bdb9e20b6c624e8c7633be3b2
Link to my WIP custom_parameters.yaml for standalone battery:
https://gist.github.com/pavelmaca/170c282bdb9e20b6c624e8c7633be3b2#file-custom_parameters-yaml
Logger SN: 39xxxxxxxx
FW version:  V1.6

Hi,
nice work. Your custom_parameters.yaml working with my pylontech force H1 !! (Firmware versionMW3_15_B505_1.27
). Logger SN: 40xxxxxxxx
Thanks

It also works for me... (Force H2 V2).
I modified for 5 Modules and changed Charge/Discharge (Today/Total) registers to 32 bit (rule: 3)


pylxd fails to correctly identify existing profiles.

This is with Python 2.7 and pylxd 2.2.5 and lxc version 2.21
I'm running into a situation where pylxd successfully gets a list of all my profiles, but then fails to recognize some of them as existing.
The script below simply iterates through the listing of profiles and then asks if that profile exists:
from pylxd import Client
from pylxd.models import Profile

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

client = Client(endpoint='blanked out',
                 cert=("/home/FUZZBUTT/snesbitt/snap/lxd/5758/.config/lxc/client.crt",
                       "/home/FUZZBUTT/snesbitt/snap/lxd/5758/.config/lxc/client.key"),
                verify=False)
client.authenticate('blanked out')

for profile in Profile.all(client):
    print("Profile Name: %s. Exists: %s" % (profile.name, Profile.exists(client, profile.name)))

The result of this script is:
/home/FUZZBUTT/snesbitt/projects/ansible/venv_2.7/bin/python2 /home/FUZZBUTT/snesbitt/.IntelliJIdea2018.1/config/scratches/scratch.py

/home/FUZZBUTT/snesbitt/projects/ansible/venv_2.7/bin/python2 /home/FUZZBUTT/snesbitt/.IntelliJIdea2018.1/config/scratches/scratch.py
Profile Name: default. Exists: True
Profile Name: devAnsibleServer. Exists: True
Profile Name: dev. Exists: True
Profile Name: maas. Exists: True
Profile Name: tmpl_consulserver. Exists: False
Profile Name: test-test. Exists: True
Profile Name: test_test. Exists: True
Profile Name: test_testtesttest. Exists: True
Profile Name: test-testtesttest. Exists: True
Profile Name: tmpl_consulserver1. Exists: True
Profile Name: tmpl-consulserver1. Exists: True

Note that pylxd marks tmpl_consulserver as not existing.
I'm also noting that the output is a bit flaky and the results can change after adding a new profile. The one constant is that tmpl_consulserver is always marked as not existing.
(edit:ajkavanagh just fixed up formatting)

Well that's a bit strange.  It might be a bug in the underlying LXD implementation; before I 'blame' them, is there anything different, say, between the profiles of tmpl_consulserver and the other variants (with - and _.*1) for example?  i.e. please could you past the contents of those profiles.
The "reason" exists comes back as False is if a NotFound exception is raised via the API handling, which would imply a 404 from the LXD daemon.  Which as you say, is a bit strange and flaky.
Thanks.

Sorry about the late reply and poor original formatting.
Naturally my test fixture for this was deleted as part of ongoing work, and I can't recreate
Let's close for now.

Okay, thanks for letting me know.  I'll close it for now, but please feel free to re-open it if you come across it again.  The module's tests with sockets isn't very complete at the moment; it's something that I'm working on, as there seem to be a few bugs/issues arising with TCP/IP & web-sockets, whereas most of my testing is with the local unix sockets.


USE bug; variables not converted to uppercase

Reported by Ansley B Manke on 3 Sep 2002 18:48 UTC
! When a datafile is changed outside of Ferret and re-USEd,
! variables that overlap with those in other SET datasets
! are not converted to uppercase.  This makes them inaccessible
! to the user.
! Create three simple files containing variables with lowercase names.
set mode verify
let a = 1
let b = 2
save/q/clob/file=f.nc a
save/q/clob/file=f2.nc a,b
save/q/clob/file=g.nc b
can var a b
! note see http://www.gfdl.gov/~jps/GFDL_VG_NetCDF_Utils.html#ncrename
sp ncrename -h -v A,a f.nc
sp ncrename -h -v A,a -v B,b f2.nc
sp ncrename -h -v B,b g.nc
! Use two of the files and cancel the first one.
use f.nc
use g.nc
can dat 1
! Now overwrite the first file and re-use it.
sp mv -f f2.nc f.nc
use f.nc
show dat
! Note that variable "b" is lowercase in dataset 1.
! It is now inaccessible: e.g. the following gives an error.
list b[d=1]
Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/427

Comment by<EMAIL_ADDRESS>on 3 Sep 2002 18:52 UTC
Kevin fixed this; see cd_scan_vars.F


Add refresh_events option

This is a better approach of the previous pr I made (#453). This time instead of hard coding the events I decided to let the end user choose what events should run the IndentBlanklineRefresh command.
Option name: refresh_events
Default value: { "CursorMoved" }
As I mentioned in the commit message please add this to the help tag since I don't know much about that. Thanks!

I get what you are going for, but this would be a more complicated change.
This is not the only place autocmds are set. They are different based on if context is on or not.
I also don't think it is that necessary to have a configuration option for this. Users can just overwrite the autogroup if they want to customize the events.
But I'd still like to merge your first version, just fixing the default autocmd.


[Tracking Issue] Publish lab device benchmark results

We have some emulators/devices in the lab to run benchmarks and their results can also be published to https://perf.iree.dev.
The computation resources are very restricted on these devices so we only run cron jobs to fetch main branch commit and run benchmarks. Then the results can be periodically pushed to the dashboard.
### Tasks
-  [ ] #14299
- [x] Publish results to the dashboard
- [ ] #14301

Closed as the this is considered done


net: sockets: tls: Prevent infinite block during handshake

In case peer goes down or we disconnect from the network during the TLS handshake, the TLS socket may block indefinitely during connect()/accept(), waiting for data from the peer. This should be avoided, hence use the preconfigured timeout for the TLS handshake, same as we use for TCP-level handshake.

Thank you. Please consider updating the comment in sendto_dtls_client explaining, why K_FOREVER is ok there instead of the TODO.

Thank you. Please consider updating the comment in sendto_dtls_client explaining, why K_FOREVER is ok there instead of the TODO.

Added additional comment but left the TODO, as it's about something else (blocking/non-blocking socket behavior).


process.stop no method error

I've been receiving the following error when calling the process.stop method on the childprocess instance.
Here's how I'm using it:
Inside my ProcessHelper class, I have two methods, start and stop. I'm calling them to start a server in the background and run some tests.  When I'm down with my tests I need to shut down the process.
def start
process = ChildProcess.build(*parameters)
process.start
end
#doing something with the process running in background
def stop
process.stop
#need to stop the process explicitly since it does not exit on its own so wait will not work
end
The error I'm getting is:
undefined method `stop' for # < ProcessHelper:0x007fd29555d5b0 @ params=["appium"]> (NoMethodError)

The exception you've included indicates that ProcessHelper doesn't have a stop instance method. If you believe it's a bug in ChildProcess, please provide a full example with all the code necessary to reproduce the problem.


Datepicker and Bootstrap 4 compatibility

I'm integrating the Datepicker with Bootstrap 4 alpha but while in your demo site it renders perfectly ones I integrate into my Bootstrap 4 application it doesn't. I compared the code and it seems that it renders Boostrap 3 code like glyphicons and btn-default. Anything I should do to make sure that the html output is compatible with Bootstrap 4?

Dup with https://github.com/valor-software/ng2-bootstrap/issues/507

I am working on new version of datepicker, so it will be fixed soon
Closing as duplicate


Add WPS238: Forbids too many (3+) raise statements in a function diagnostic

done here https://github.com/strmwalker/ruff/pull/5


Request: ability to throw error() in +server.js endpoints just like in load functions

Describe the problem
One thing I like about #5748's upcoming changes is the ability to throw errors in load with the error() helper function. Currently, that doesn't apply to endpoint handlers in +server.js, which must return a Response object with a suitable HTTP error code. I'd like to request that an error() helper function be made available to +server.js handlers as well.
My use case is an API project that I implemented using Svelte-Kit. I have a lot of auth code that's the same across multiple API endpoints, which I've extracted into helper functions: for example, the allowSameUserOrAdmin function checks the JWT found in the Authorization header, and ensures that: 1) it's valid, correctly signed, and not expired, and 2) it belongs to either the username whose settings are being updated, or to a site admin who has access rights to change anything.
Currently, all my auth functions return an object of the same "shape" that endpoint handlers currently return: { status: 401, body: "Unauthorized" } (or 403, or ...) for failures, or { status: 200, body: '' } if the user is authorized to perform the operation. Then the rest of the code in the "edit user account" endpoint looks something like this:
const authResult = await allowSameUserOrAdmin(db, { params: { username }, headers });
if (authResult.status === 200) {
    const result = await updateUser(db, user.id, userDetails));
    return { status: 200, body: result };
} else {
    return authResult;
}

I have boilerplate like this all over my endpoint handlers. But if my auth helper functions could throw errors (that had HTTP statuses and messages attached to them, just like the error() helper for load functions), I could simplify that boilerplate down to just two lines:
await allowSameUserOrAdmin(db, { params: { username }, headers });
return updateUser(db, user.id, userDetails));

Describe the proposed solution
An error() function would be made available to +server.js code, and would work similarly to how load() functions work: if the handler function throws a SvelteKitError (or whatever specific exception the error() function constructs) instead of returning a Response object, then the Svelte-Kit server code will catch that exception and turn it into a Response with appropriate status and body (taken from the values passed into error()). Any exceptions that did not come from the error() function would be handled the same way they currently are, by producing a 500 Internal Server Error response.
Alternatives considered
Once #5748 goes through, I can change my auth functions to return a Response with appropriate status codes, but then the if (authResult.status === 200) boilerplate would remain essentially unchanged. Or I could create my own AuthException exceptions and throw them, and reduce the boilerplate down to a throw-catch:
try {
    await allowSameUserOrAdmin(db, { params: { username }, headers });
    return updateUser(db, user.id, userDetails));
} catch (e) {
    if (e instanceof AuthException) return makeResponseFrom(e);
    else throw e;
}

But that's the same number of lines as my original boilerplate, so doing it myself isn't a huge gain. To really be able to simplify my boilerplate, I would need Svelte-Kit to handle the "catch and makeResponseFrom" logic.
Importance
would make my life easier
Additional Information
No response

What I like about the current(future) API is that it's very simple: Return any form of Response object, that's it. I'm not sure muddying the waters helps here. Your code could be rewritten as follows to make it less repetitive:
return allowSameUserOrAdmin(db, { params: { username }, headers }, () => { /* code that should run when allowed */ });

where allowSameUserOrAdmin is something like
function allowSameUserOrAdmin(db, params, cb) {
   if (notAuthed(db, params) {
      return Response('Nope', { status: 400 });
   } else {
      return cb();
   }
}

I was envisaging that we would support the same helpers, since a) as you say, it provides consistency (e.g. reusing auth logic between +page.server.js and +server.js), and b) it would actually be more to not support them.
The same goes for setHeaders — even though it's redundant (you can return a Response with whatever headers you want) it would be weird to prevent its usage on that basis.
In fact this is described in #5748, in the '+server.js API' section of the OP:

As conveniences (and to keep things consistent), the error and redirect imports and the setHeaders helper can be used, though strictly speaking they're not necessary since you could construct your own error/redirect Response objects with whatever headers you like.

Great! I missed that, but that's exactly what I was wanting to have. Shall I close this issue as redundant? Or would you prefer for me to leave it open for some reason?

I'm just going to go ahead and close it for now since it's part of the design doc. If, for some reason, we change that design before release and this feature doesn't get implemented, we can reopen it or create a new one. Or Rich can reopen it if he thinks it should stick around. :)


Better EnumConstraint error message

Currently EnumConstraint uses print_r which produces messages like:
Does not have a value in the enumeration Array\n(\n    [0] => GBP\n    [1] => USD\n)

Which is not the most friendly messaging in the world. I'm changed this to use implode to make it more readable:
value is not in enumeration: [GBP, USD]

(There's probably broken tests as a result of this, I'll fix these if we're happy to accept the change)
Thoughts?

@wpillar please see the TravisCI console for failures.


Can the processed dictionary created on-the-fly by the command line be saved for re-use?

First of all, thank you for developing this tool. It is amazingly useful and fast!
I have been using the command line version since my programming skills are limited (my skills being in vba, sql and some java).  I do have visual studio 2017 installed so perhaps that could help if I need to modify the project on my end..
So my hope would be that I could find a way to do the following:

process the frequency dictionary once and save it for re-use
(I want to confirm that there is no way to have the processed dictionary be loaded into a db like mysql..I assume this won't work because mysql cannot create the proper indexes..correct?)

use a command line switch to set the number of matches returned (in frequency order of course) for edit distance 1 and edit distance 2 etc. So let's say I want the "top 5" and I set my max-edit distance to 3 then I would get 15 results (assuming there are >=5 matches for each edit distance). As it is now I may get a few for distance 1, a lot for edit distance 2 and a massive list for edit distance 3. I have been attempting to cleanup names from the census with have transcription errors and many times the correct name is the 1st or 2nd result in edit distance 2 or 3 (not edit distance 1). If I could get the top few matches from each edit distance then I have a phonetic algorithm that narrows the results.

ultimately I would really want to create an excel function that could call upon the command line for matches where the processed dictionary is already loaded into memory and that environment is accessible to vba

As a first step - items 1 and 2 are most important (saving the processed dictionary and setting the max # matches ordered by frequency)
Do you think this is possible? and could the dictionary ever be moved into a db? Thanks for your help and for sharing this excellent tool.
As a side note, could this ever be successfully migrated to nodejs to create an api?

process the frequency dictionary once and save it for re-use

You have to serialize/deserialize the delete data structure of symspell and save it to a file/load it from a file. Those files can then also be stored in MySQL as BLOB. You can implement your own serialization, use JSON or something like flatbuffers

use a command line switch to set the number of matches returned (in frequency order of course) for edit distance 1 and edit distance 2 etc.

This is certainly possible to implement as an additional command line parameter. Alternatively one could  implement this as a post processing step to filter the complete result list accordingly.

I would really want to create an excel function that could call upon the command line for matches where the processed dictionary is already loaded into memory and that environment is accessible to vba

SymSpell.CommandLine allows pipes and redirects for Input & Output. So does VBA. Could this be a way how those two programs communicate while the directory is already loaded?
https://support.microsoft.com/en-us/help/177696/how-to-use-named-pipes-in-a-visual-basic-32-bit-program
I will mark this as future enhancements, but can't give any timeline.

Thanks for the quick reply.
I'll investigate what you said about saving down the data structure. At first glance it is not obvious where to begin but I will look through the code to see what I can find that relates to what you said.
The reason I had asked about number 2 (limiting the # of responses to the top x per edit distance) is that my dictionary is several million entries and when I send a list of, say, 10000 names to be processed and I allow up to an edit distance of 3 I get back millions and millions of matches. This is particularly true if my list has some long names. What I found is that in most cases the answer I want is in the top 5 matches - but sometimes that match has an edit distance of 3 which means I have to deal with the millions of matches.
I'll look more closely at pipes and redirects.
Thanks again for a fantastic tool.

I also forgot to ask if you know of anyone who has tried to use nodejs to create an api using your work. I found 1 very old attempt but it is unusable.

As far as I understand Node.js is an JavaScript run-time environment that executes JavaScript code outside the browser. So it could be used to run one of the SymSpell JavaScript ports.
But if you just want an embedded web server to provide a REST-API to SymSpell, this can be easiy implemented using the HTTPListener class.


query Multiple Tables

Project requirements
Create a table named owners with the following columns:
id: integer (set it as autoincremented PRIMARY KEY)
full_name: string
age: integer
Create a table named species with the following columns:
id: integer (set it as autoincremented PRIMARY KEY)
name: string
Modify animals table:
Make sure that id is set as autoincremented PRIMARY KEY
Remove column species
Add column species_id which is a foreign key referencing species table
Add column owner_id which is a foreign key referencing the owners table
Remember all this goes in schema.sql file.
Insert the following data into the owners table:
Sam Smith 34 years old.
Jennifer Orwell 19 years old.
Bob 45 years old.
Melody Pond 77 years old.
Dean Winchester 14 years old.
Jodie Whittaker 38 years old.
Insert the following data into the species table:
Pokemon
Digimon
Modify your inserted animals so it includes the species_id value:
If the name ends in "mon" it will be Digimon
All other animals are Pokemon
Modify your inserted animals to include owner information (owner_id):
Sam Smith owns Agumon.
Jennifer Orwell owns Gabumon and Pikachu.
Bob owns Devimon and Plantmon.
Melody Pond owns Charmander, Squirtle, and Blossom.
Dean Winchester owns Angemon and Boarmon.
Remember these insertions and modifications happen in data.sql file.
Write queries (using JOIN) to answer the following questions:
What animals belong to Melody Pond?
List of all animals that are pokemon (their type is Pokemon).
List all owners and their animals, remember to include those that don't own any animal.
How many animals are there per species?
List all Digimon owned by Jennifer Orwell.
List all animals owned by Dean Winchester that haven't tried to escape.
Who owns the most animals?
Remember all these should be written in queries.sql file.
Take a screenshot of the results of your queries.
Remember to add the screenshot with the results of your queries to your Pull Request description.

[ ] @Abdulwhbhussain consider updating the screenshot after effecting the change

[ ]  @Abdulwhbhussain consider updating the screenshot after effecting the change

Already done!


Can't take Screenshots on Android 6 &7

Please fill out the blanks below describing your issue
What are the steps to reproduce this issue?

…
…
…
…

What happens?
In the Android Studio Emulator and on my own device the Screenshot is made on Android 5 (Lollipo). On other devices (Android 6 & 7) I can't find the Screenshot. Do you know where I can find it?
What were you expecting to happen?
…
Any logs, error output, bugreport etc?
No errors.
Any other comments?
…
What versions of software are you using?
Device Information: …
Android Version: …
Configuration Information: …

@KrasserTommy Let me cross-check. Thank you for reporting this.

@KrasserTommy Are you talking about the filepath? If so then have a look at issue #5

I don't think so. On my Lollipop Device the Picture is saved to the Pictures Folder. On my Android 6 and 7 Device the folder is empty...

I am guessing you haven't provided the write permissions. Can you check your logcat output, where it would show if that is the case.

@KrasserTommy Any updates ? I was able to take screenshots in both Android 6 and 7.


[Improvement][Shell] kill shell command when using shell to run flume would cause flume exit not gracefully

/usr/local/service/flume/bin/flume-ng agent -c /usr/local/service/flume/conf/ -f /data/dolphinscheduler/data/hadoop/resources/appsflyer/appsflyer_flume.properties -n agent -Dflume.root.logger=INFO,console

Start the Flume agent with above shell command. After it has written a few files to HDFS, kill it using webui kill command. Check the hdfs dir. There will be a .tmp file. If we restart the Flume agent, it will resume writing data to HDFS, but it will not close/cleanup the .tmp file.
I think the correct scenario is that use kill -15 instead of of kill -9, so that flume agent can exit gracefully and it can perform these tasks before stopping the agent.
Which version of DolphinScheduler:
-[1.3.6]

I think it would be better to provide the two-way, let users choose the kill way when they need to stop the workflow. Because sometimes we need force kill.


Screenshot compare tool in RBFW

Moving from the TML framework, we see that the sys.imagecompare is a very good experience tool,
that very important and useful for the automation testing.
The lib support to compare 2 images with a wide variety of options (threshold, test hook, arIgnoreMasks, ...)
We will really appreciate this and hope the RBFW will also support the same functionality,
Thank you.

Proposal:
https://github.com/manykarim/robotframework-imagecompare
https://pypi.org/project/robotframework-imagecompare/

Hi Nguyen,
please integrate this library.
We have already multiple requests for image comparison.
Thank you,
Thomas

Hi Team,
This libraries is available in https://github.com/test-fullautomation/RobotFramework_AIO/commit/bb0873ee9bb9130bb0873ee9bb91303612ba2f26b68a54751bde5f53612ba2f26b68a54751bde5f5

Hi Nguyen,
with which pull-request was this provided?
Thank you,
Thomas

Hi @test-fullautomation ,
The commits was merged in this pull request:
https://github.com/test-fullautomation/RobotFramework_AIO/pull/63/commits
Commit messsage: "agt2hc/feat/64/screenshot_compare_tool_in_rwfaio"

@HolQue ,
We discussed some yesterday about a screen compare tool.
This one solves the issue? I did not try, but it provides threshold, test hook, arIgnoreMasks, ...
Should be good for us.
Thank you,
Thomas

Hi Thomas,
status: pending
I have to evaluate. Therefore I have doubts for this feature regarding milestone 0.8.0

Hi  Holger,
I shift this to 0.9.0.
Thank you,
Thomas