context is not protected, then is made sync

In github actions, cp semaphore tests are failed sometimes. In windows dump file, we saw that the seg fault is occured at then continuation. As we review the code, context lifecycle is not protected. As there is no user code, instead of extending lifecycle, it is also made sync. So that the client will not destroyed until this continuation.

Windows test PASSed.

Linux test PASSed.


Made the web server a class with methods that are project-overrideable.

This change helps make server development easier by providing a convenient project hook for overriding and extending the server code.

Very nice @yavin5 love it 👍 thank you!


cmd/compile: only combine loads if there is a single use of each

When we combine loads into a single load, we need to make sure that each load we're combining only has one use.  We'd like to keep the invariant that if you do:
 a := b[0]
 ...use a here...
 ...use a there...

then we're guaranteed that we use the same value of a in both places, regardless of races or whatever other nastiness is happening.  In other words, we can't issue the b[0] load twice, we have to issue it only once and use the result for all occurrences of a.
The load combining rules currently violate that rule.  We need to either make sure that all loads we're combining have only one use (the use of the combination value), or we extract the individual loads back from the combined load in case they are used elsewhere.
The obvious Uses==1 condition on the rewrite rule doesn't quite work, as mid-rewrite those loads have other dead uses that haven't been cleaned up yet.  We need a more exact Uses count to make that condition work.
@tzneal

Fixed by https://go-review.googlesource.com/c/22326/


Unify linkscript for all targets

Description
We currently use one linker script per target that embeds the memory definitions and the placement of code sections in memory. The placement part is pretty similar across devices and has no reason to be kept duplicated.
This PR splits things into per-target memory definitions and a single script.ld that places code sections, and can be updated for all devices at once when we need it to.
Perhaps even the memory definitions could be merged together also (in another PR).
Changes include

[x] Other (for changes that might not fit in any category)

(cleanup/refactor)

This is breaking app-bitcoin-new build on nanos, so probably something doesn't works as expected.
We probably should check that this doesn't change app binaries.

I suppose it's because the G_cx symbol is no longer allocated in the CXRAM section?
Note that I use the CXRAM section explicitly on NanoS, when I need memory but I'm no using any crypto functions that might touch CXRAM... 😇

Codecov Report
Patch and project coverage have no change.

Comparison is base (d31f67c) 69.09% compared to head (a22130b) 69.09%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #398   +/-   ##
=======================================
  Coverage   69.09%   69.09%
=======================================
  Files          11       11
  Lines         864      864
=======================================
  Hits          597      597
  Misses        267      267

Flag
Coverage Δ

unittests
69.09% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Yes, I forgot the placement of G_cx. Moved it to the Nano S's layout only.


Allow to have the tabs on the left side

This would be useful if you have a lot of tabs open and don't want multiple lines of tabs.
Example:

What this about? It's kinda unclear to me now. The Example is also broken

Having tabs on the left side of the screen instead of on the top.
(it looked like this but way nicer iirc)


Feature: Ability to tag created volumes

Unless I've missed this somewhere, it would be a welcome addition. Would likely require a second request to aws

aws.tags is only tagging the instance but not the volumes. In fact I think it's impossible to tag the volumes within the same call. Most likely it will need to pull all volumes and apply the same tags to each one separately

It used to be that tagging required a separate call but it is now possible to tag volumes as part of the instance run call.
https://aws.amazon.com/blogs/aws/new-tag-ec2-instances-ebs-volumes-on-creation/


Create a menu page

Hi
I am a contributor at GSSOC'22 and I want to create the menu page in flutter

A mentor will review this pull request shortly , please have patience.

Can i start working on this page ? can this issue be assigned to me?

Assigned to you make sure you follow code conventions and quality of code 🙃

Hey @Lakhankumawat I wanted to work on this issue, Please consider me next in the queue if the issue is not resolved. I'm a GSSOC22 participant.

I would recommend the contributor of this issue to fetch upstream and resolve conflicts before creating a new Pull Request.

hi @ChetasShree  you have 2 more days then this issue will be assigned to someone else

@Lakhankumawat this issue is still open, please assign it to me.

/assign

Issue assigned to @Chitrakshi2531  due to overtime


UI mockups /screenshots

It will be great to see some screenshots of what is being developed. If not those, even UI mockups will be great to see on the readme.

Hi @kasriniv! There is a mock up of what a 'Topics' page may look like in the proposal if you havent seen it already: https://github.com/matthew-chirgwin/proposals/blob/ui-proposal/009-strimzi-ui.md . Once all the foundational/infrastructure type work is sorted, and proposals agreed, I am hoping we will have issues for the various UI screens and add mockups/designs as a part of those issues.

Hi @matthew-chirgwin, before polluting the issues unnecessarily I thought I might just ask in this tangentially related issue :)

Once all the foundational/infrastructure type work is sorted, and proposals agreed, I am hoping we will have issues for the various UI screens and add mockups/designs as a part of those issues.

Do you already accept such issues with feature requests? Things like:

Ability to reset consumer groups
Ability to delete topics
Cluster visualization
...

if so, then I could go ahead and create them.

Hi @chgl ! I see no harm in raising them now. As mentioned in the proposal, there are a set of screens and capabilities we will be implementing first, starting with a topics listing page (with the ability to create/delete following for example). After that however, others could be added, with agreement from the @strimzi/maintainers . I think raising the issues, and starting discussions early is not a bad idea.


Correct VS 17.4.33403.182 display version number

This change corrects the display version number for 17.4.33403.182 to 17.4.5.
VSWhere identifies build version 17.4.33403.182 with the 17.4.5 display version. The entry on February 14, 2023 incorrectly identifies it as 17.4.4, which introduced an incorrect duplicate entry for 17.4.4.
Addresses #8969.

@TerryGLee
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Minor bugs in example/thermometer-notebook.jl

I had to make two little changes to make the example notebook work for me again.
@@ -142,7 +144,7 @@ struct UnitsEvent <: HSM.AbstractHsmEvent
 end

 # ╔═╡ 5e3ff2ba-5deb-464e-b087-a2bbec22c254
-struct TemperatureEvent <:AbstractHsmEvent
+struct TemperatureEvent <: HSM.AbstractHsmEvent
        temperature::Float32
 end

and
@@ -412,7 +414,7 @@ new_temp
 # ╔═╡ 925fdbca-f263-4d28-9bbf-ab23a5810471
 with_terminal() do
        with_logger(logger) do
-               handle_event!(thermometer_state_machine, TemperatureEvent(new_temp))
+               HSM.handle_event!(thermometer_state_machine, TemperatureEvent(new_temp))
                println("display: $(thermometer.display)")
                println("temperature: $(thermometer.temperature)")
        end

I don't feel comfortable sending a pull request for this one, because my Pluto instance changed so many other things when it installed the dependencies for the notebook.  I think it'd be less intrusive if you made these changes yourself.

It wasn't less intrusive. Probably because that notebook was ~2 years old. Now works with Julia 1.11 and whatever version Pluto is as of today.


ImportError when running Streamlit on Raspberry Pi 4

Describe the bug
I'm encountering an issue when running my Streamlit application on a Raspberry Pi 4. Here are the error details:
Environment:
Raspberry Pi 4
Operating System: [Raspberry Pi OS]
Python version: 3.10
Virtual environment: hummingbot-env
Steps Taken:
I created a virtual environment (hummingbot-env) and installed all required dependencies from requirements.txt.
I attempted to update Pydantic and all associated dependencies to the latest available versions.
I purged pip cache and reinstalled all dependencies.
I checked for version conflicts and ensured compatibility between installed dependencies.
I tried installing a specific version of Pydantic (1.8.2) to ensure compatibility with other project dependencies.
Despite these efforts, the error persists. I'm still stuck at importing default_ref_template from pydantic.schema.
Steps to reproduce bug
Traceback (most recent call last):
File "/home/msi/TRADING_BOT/hummingbot/hummingbot-env/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 589, in _run_script
exec(code, module.dict)
File "/home/msi/hummingbot/dashboard/main.py", line 6, in
from backend.utils.os_utils import read_yaml_file, dump_dict_to_yaml
File "/home/msi/hummingbot/dashboard/backend/utils/os_utils.py", line 10, in
from hummingbot.strategy_v2.controllers.directional_trading_controller_base import DirectionalTradingControllerBase, DirectionalTradingControllerConfigBase
File "/home/msi/TRADING_BOT/hummingbot/hummingbot-env/lib/python3.10/site-packages/hummingbot/strategy_v2/controllers/init.py", line 1, in
from hummingbot.strategy_v2.controllers.controller_base import ControllerBase, ControllerConfigBase
File "/home/msi/TRADING_BOT/hummingbot/hummingbot-env/lib/python3.10/site-packages/hummingbot/strategy_v2/controllers/controller_base.py", line 10, in
from hummingbot.client.config.config_data_types import BaseClientModel, ClientFieldData
File "/home/msi/TRADING_BOT/hummingbot/hummingbot-env/lib/python3.10/site-packages/hummingbot/client/config/config_data_types.py", line 8, in
from pydantic.schema import default_ref_template
ImportError: cannot import name 'default_ref_template' from 'pydantic.schema' (/home/msi/TRADING_BOT/hummingbot/hummingbot-env/lib/python3.10/site-packages/pydantic/schema.py)

I had this issue too with my mac mini server - try to pull the latest source because there were some conflicts with versions of dependencies


Meter size change

As far as I can see there is no way of changing the meter size without modifying the meter.cpp file.
Or am I missing something?

Did you solve how to modify the meter?


Expression box in safari acts really weird[BUG]

https://user-images.githubusercontent.com/29583265/143012808-1dae9c38-d5cd-4aa7-98c6-b1f7e61b50e8.mp4
I saw several topics about this and it seems they are fixed but for chrome not safari. Cursor acting weird sometimes it jumps at start sometimes at the end + cursor not respond to position where I am typing.

We don't test with Safari. Contributions are always welcome.


Renderer: split shaders and nodes in conceptually atomic, simpler units

A number of nodes execute shader programs that do a number of things at once.
For example:

PrePostCompositeNode takes care of reflections, haze, volumetric fog (disabled), ambient occlusion and outline.
InitialPostProcessingNode takes care of the bloom effect, chromatic aberration (not working), vignette and light shafts
FinalPostProcessingNode adds film grain, depth-of-field and motion blur to the picture.

While perhaps performance-friendly this way of doing things 1) considerably reduces the debugging capability of tools examining the content of FBOs at any given point in the render graph 2) prevents the insertion of nodes between these individual effects 3) in some cases generates unduly large shaders that are harder to debug and mantain.
This issue advocates for:

the splitting of shaders and the associated nodes, toward more atomic shaders and nodes
the development of generic, auxiliary nodes such as TintNode and CompositingNode

Notice that more atomic shaders and nodes, whenever possible, doesn't necessarily mean simpler shaders and nodes. For example, an atomic VignetteNode could provide currently unavailable functionality, such as the replacing or tinting of the vignette texture to signify different things affecting the player: damage, cold, heat, binoculars, underwater gogles and even dream-like states.

The first step towards this goal in my opinion would be to modularize shaders. The current post-processing shader and node design is a mess, with some nodes performing half of their processing in their own shader, feeding the intermediate result as an FBO into a sampler in the post processing composite node where it is sometimes processed further before finally being rendered to screen.
Instead, an approach (specific to post-processing, but can be generalized) which depends on resolution of #3035, could be as follows:

Each Node should be able to declare its own shader, which fully computes and passes the output into gl_FragData.
Each post-processing (pp) node will be a child of a pp stage node (PrePostCompositeNode, InitialPostProcessingNode, or FinalPostProcessingNode).
For every pp stage node (for absolutely any node with children, rather), its children will be evaluated in the order in which they are added (possibly with the "input plugs" and "output sockets" system or similar as detailed in #3035).
Each pp stage node defines a set of framebuffers (via an FBO) that must be filled by the child pp nodes. For example, the existing pp stage nodes, based on their shader definitions, would define the following framebuffers:

Pre: colorOpaque, colorTransparent, depth, normals, lightBuffer
Initial: color
Final: color, blur, depth

After each child pp node of a pp stage node is processed, the pp stage node aggregates the results and preps them for the next stage (if any). For example, the Pre pp stage node will mix the colorOpaque and colorTransparent buffers into a single color buffer to be used in the next stage.
Generalizing this approach to all nodes, we get a tree-like structure with the OutputToScreen/ImageNode as the root. A PostprocessingNode could be the last child of the root node. The root node, after processing all its child nodes in order, will output the composited results to screen or an image (as a screenshot).

The first step towards this goal in my opinion would be to allow user-defined modular shaders. The current post-processing shader and node design is a mess, with some nodes performing half of their processing in their own shader, feeding the intermediate result as an FBO into a sampler in the post processing composite node where it is sometimes processed further before finally being rendered to screen. Modular shaders would also help break down currently mushed-together nodes, like all the post-processing nodes.

The first step towards this goal in my opinion would be to allow user-defined modular shaders.

Way too ambitious at this stage. It is a GSOC-level type of project and I'd want to see more details. The current primary goal of the renderer is to a) transform it into a Render Graph based architecture b) open its API. Then the idea is to go into shaderland and see what we can do to clean it up: we don't change it, we just clean it up. Then we can think about how to radically change it.

The current post-processing shader and node design is a mess

I can agree that shader management is a mess: we haven't touched it yet. If you want to work on the renderer you'll need to cope with the current node-based architecture and the concept of the render graph being worked on in PR #3088.

with some nodes performing half of their processing in their own shader, feeding the intermediate result as an FBO into a sampler in the post processing composite node where it is sometimes processed further before finally being rendered to screen.

From my perspective this is a perfectly normal, modular, rendering pipeline and it doesn't come even close to the complexity of the composites I have worked with in the film industry. If you wanted to criticize it (and it better be constructive criticism) you might want to try again.

Instead of shader programs being hardcoded in a ShaderManager, each Node should be able to define its own shader.

Shaders aren't hardcoded in the ShaderManager, rather they are in .glsl files. But it is true that the ShaderManager currently has an hardcoded list of shaders which it processes before they become available to the nodes. It is certainly one of my goals to make sure that nodes can bring in their own shaders, we might be on the same page about this. We just haven't gotten to that point.
@eviltak: you reeeeallly want to have a more positive attitude if you want to work on the renderer. You might want to ask @vampcat for advice.

If you wanted to criticize it (and it better be constructive criticism) you might want to try again.

Sure, I'll take up the challenge. The ultimate goal of giving modules the ability to add their own nodes on the render pipeline hinges on everything being modular, which this issue slightly addresses by setting the goal of "atomizing" everything. I may not be as experienced as you in this regard, but I do know for sure that in it's current state, the rendering pipeline is far from modular.
Take the PrePostCompositeNode as an example. It processes the results of five different effects (all of which have their own separate nodes) by first loading intermediate output from their FBOs into samplers and then finally aggregating them together.
Now say I want to add another node to the pre-post processing stage implementing, say, SSDO. There is no way I can do that without modifying PrePostCompositeNode and its corresponding shader - if the SsdoNode was intended to be part of a module, I would not be able to do it at all without modifying engine code. To have the desired effect of simply plugging in and taking out nodes, this blurring of Node responsibility must be averted.
It's not like PrePostCompositeNode must process the results of five different effects, either. That responsibility can be handed over to their respective shaders, and all PrePostCompositeNode has to do is the last 6 lines in the shader. The workflow can go as follows:

PrePostCompositeNode builds the opaque, transparent, depth, normal, and light buffers and passes them on to its neighbor in the RenderGraph, which would be (following the order in prePostComposite_frag.glsl) a (newly created, "atomized" node) VolumetricFogNode.
VolumetricFogNode computes and updates the opaque and transparent color at each fragment and passes the updated buffers to its neighbor, a (newly created) LocalReflectionsNode, which again does the same and passes the buffers to its neighbor, and so on.
After the pre post composite pass is complete, all that is left to do is mix the opaque and transparent buffers into a new color buffer (essentially the last 6 lines of prePostComposite_frag.glsl).

Come to think of it, the issue does advocate something like the above, which is exactly what I meant by "modularizing" shaders, so I am not sure where the backlash comes from.
Of course, this problem may only be limited to the post-processing nodes, but I did specify:

The current post-processing shader and node design is a mess
And so did your issue (indirectly).

@eviltak: you reeeeallly want to have a more positive attitude if you want to work on the renderer.
Not reeeeallly sure where you got the negativity from. The only negative thing in my comment was "the current post-processing shader and node design is a mess" (emphasis again to prevent misconceptions, I was talking purely about the post processing node pipeline) which is just another more direct way of saying
While perhaps performance-friendly this way of doing things 1) considerably reduces the debugging capability of tools examining the content of FBOs at any given point in the render graph 2) prevents the insertion of nodes between these individual effects 3) in some cases generates unduly large shaders that are harder to debug and mantain.

I may not have all your years of experience to know that this is considered modular in the CG industry (if it is, I must admit I will be slightly disappointed), but I do know that it is something to be overcome in order to achieve the goals outlined in this issue.

@eviltak I won't be commenting much on this, but you'll have to learn (probably the hard way) that just being decent at coding is not enough - learning how to communicate (not just string together some words in english, but rather communicate properly and effectively) is way more important. Before manu posted his message, I had half a mind to say something along the same lines, but I refrained - primarily because I was in your shoes a few weeks ago.
 The worst part? You don't even realize it. Why do I say it so bluntly? Because so was I, and I'm glad someone pointed it out.

@vampcat Got it. I'm sorry if I overstepped boundaries @manu3d and @vampcat. I shall make conversations more genial and productive.

The ultimate goal of giving modules the ability to add their own nodes on the render pipeline hinges on everything being modular, which this issue slightly addresses by setting the goal of "atomizing" everything.

Yes and no. A renderer API would, as a first step, allow modules to add nodes to the RenderGraph. This doesn't need modular shaders. Atomic nodes, concentrating on simple tasks such as tinting and compositing, would then create a finer modularity. So, I disagree that this issue "slightly" addresses the issue. It addresses it heads on by prescribing a concrete way forward:

the splitting of shaders and the associated nodes, toward more atomic shaders and nodes
the development of generic, auxiliary nodes such as TintNode and CompositingNode

Now say I want to add another node to the pre-post processing stage implementing, say, SSDO. There is no way I can do that without modifying PrePostCompositeNode and its corresponding shader - if the SsdoNode was intended to be part of a module, I would not be able to do it at all without modifying engine code. To have the desired effect of simply plugging in and taking out nodes, this blurring of Node responsibility must be averted.

This paragraph, and the other 20 lines that follow it, are precisely what I mean in: "2) prevents the insertion of nodes between these individual effects". So, you are trying to crash trough and open door.

Not reeeeallly sure where you got the negativity from. The only negative thing in my comment was "the current post-processing shader and node design is a mess"

That is no small negative assertion. Never ever call somebody else's work a mess: you don't know what's behind it. Go back a couple of years and take look how the whole renderer looked like. You are ignorantly neglecting the progress that I, vampcat and tdgunes before him managed to do on top of begla's excellent prototyping work. We are still in the phase of transforming that monolithic (but highly functional) prototype into a production-ready, extensible part of the codebase. Calling any parts of it a mess is like arguing that a mechanic shop should be spotless clean during working hours.
Look, you started with the wrong foot, both here and in previous contributions, by being overly assertive. It's not the end of the world and we can all turn things around.
The golden rule always applies. Imagine you made something, it's far from finished and somebody, a complete stranger, wanted to make changes to it. He or she might have good ideas about it, he might even think along the same lines with you. But would you want to work with him if he is inconsiderate about the work you and others before you put into it? I don't think so, you'd just keep going your way until a better person comes along.
The kind of feedback you are giving is the kind of feedback you might be able to give to people you worked closely with, for years. With people you don't know, independently of the experience they might have, you want to add large amount of padding in your communication. For example:

I see how this came to be this way. I wonder if it could be improved by...
I'm unfamiliar with this code and I can't quite understand why this was done this way...
I have a suggestion but I'm not sure how it fits in the bigger picture...
I'd like to take care of this aspect by doing A, B and C, what do you guys think about it?
etc...

Keep in mind the kind of contributions/interactions you'd like to receive/have from a complete stranger and you'll be fine. As you gain trust and experience it will get easier and more relaxed.
 We are here to have fun, learn and help each other.

@manu3d understood. Sothis issue focuses more on the node pipeline rather than the shaders, which will be the gradual next step after this issue is resolved.

You are welcome @eviltak.
And yes, what you mention about the focus of this issue is fair.


Enhancement: Negate lua altogether

Please check for duplicates or similar issues before creating this issue.
What is your suggestion, and why should it be implemented?
 In particular, linc_luajit. So instead we just add song events for all the modchart functions.
Song Events

Work on all platforms
Are easy to implement

Lua Modcharts

Only work on windows
Are in a completely different language
Stupid
No Brackets (seriously wtf why is that)

already working on this, too. just putting this here in case anybody wants to race. :)

No. Modcharts and song events can coexist. Song events aren’t advanced enough to replace modcharts, and modcharts are too advanced to replace song events.

Song events aren’t advanced enough to replace modcharts

I can understand this, since all of the modchart functions require more than one input, and uhh

Perhaps we could add one or two, like Asset Changes (changeDadCharacter(), changeBoyfriendCharacter()), which are really simple and don't require several inputs.
ALTHOUGH
We could parse arguments with commas or spaces

modcharts are too advanced to replace song events.

Still trying to wrap my head around what this means because today's a smooth brain day for me
I think you're saying that the method of recreating a song event through a modchart is more complex than song events currently are. idk. im dumb.
But yeah i can see what you mean. 🅰

yeah, asset changes are pretty easy, ill see if i can funny pr it

I’d be fine with hscript AND lua mod charts. Based on the file extension it would switch, can use existing code and provide flexibility for those who prefer haxe over lua.

I’d be fine with hscript AND lua mod charts. Based on the file extension it would switch, can use existing code and provide flexibility for those who prefer haxe over lua.

.lua for lua
.hsc / .hscript for hscript

I’d be fine with hscript AND lua mod charts. Based on the file extension it would switch, can use existing code and provide flexibility for those who prefer haxe over lua.

.lua for lua
.hsc / .hscript for hscript

Yes

This, PLEASE! It sucks not having modcharts on Linux

I’m working on lua and hscript support for my engine might pr into kade


        Like :meth:`__setattr__`, this method converts :exc:`KeyError` to
        :exc:`AttributeError`.

        Examples
        --------
        >>> s = Struct(a=10)
        >>> s.a
        10
        >>> type(s.get)
        <... 'builtin_function_or_method'>
        >>> try:
        ...     s.b
        ... except AttributeError:
        ...     print("I don't have that key")
        ...
        I don't have that key
        """
        try:
            result = self[key]
        except KeyError:
>           raise AttributeError(key) from None
E           AttributeError: language

/azureml-envs/azureml_65a72d7faa653dcf6da20162d218b48c/lib/python3.9/site-packages/nbformat/_struct.py:125: AttributeError
----------------------------- Captured stdout call -----------------------------
INFO:papermill:Input Notebook:  /mnt/azureml/cr/j/8e37d7f372a94e03bf052e14ca5b0be0/exe/wd/examples/00_quick_start/npa_MIND.ipynb
INFO:papermill:Output Notebook: output.ipynb
------------------------------ Captured log call -------------------------------
INFO     papermill:execute.py:84 Input Notebook:  /mnt/azureml/cr/j/8e37d7f372a94e03bf052e14ca5b0be0/exe/wd/examples/00_quick_start/npa_MIND.ipynb
INFO     papermill:execute.py:85 Output Notebook: output.ipynb
============================== slowest durations ===============================
1.26s call     tests/unit/examples/test_notebooks_gpu.py::test_gpu_vm
0.03s call     tests/smoke/examples/test_notebooks_gpu.py::test_npa_smoke

In which platform does it happen?

How do we replicate the issue?

See: https://github.com/recommenders-team/recommenders/actions/runs/6884390439/job/18726805379
Expected behavior (i.e. solution)

Other Comments

I'm working on this, along w/ notebook fixes for #2035

Is this issue fixed? @loomlike @miguelgfierro

@SimonYansenZhao staging is passing, running main right now:

CPU: https://github.com/recommenders-team/recommenders/actions/runs/7307018860
GPU: https://github.com/recommenders-team/recommenders/actions/runs/7307015824
Spark: https://github.com/recommenders-team/recommenders/actions/runs/7307019104

If they all pass, we can close this issue.


Sincerely recommend to add a funtion: Avoid flashing your removable hard disk driver.

I was too stupid to remember I connected my removable hard disk and USB sticker at the same time, therefore, as you may guess, I just pressed "flash" as usual as I do.......wellllllll, now Iam recovering datas from hard disk driver with tremendous self-accusation.
So, Iam thinking when etcher has checked the target driver is bigger than 100GB, it ought to force user to type some words like "I will flash my hard disk".
I have to say this is my first time to make a huge mistake flash a wrong driver, since I believe etcher is smart enough to be trusted.

We've had suggestions like this before... the problem always comes down to where to draw the "cutoff point" between Hard Drives and SD cards - for example I personally own an 80GB external HDD, and a 200GB micro SD card.
Etcher does clearly show the size of each drive in the drive-selection window, and also in the main window after selecting a drive. I think it's always a very delicate balancing act between making something "safe" and making something "have too many annoying pop-ups which people will just ignore anyway".

Perhaps a warning for drives over X size (64gb?) Something like -- "This
drive looks quite large. Please make sure you've not accidentally selected
a removable hard drive. [All OK, Continue] [Abort Flashing]"
cc @jviotti @jhemsmeier
--
Alexandros Marinos
Founder & CEO, Resin.io
+1<PHONE_NUMBER>
@alexandrosm
On Tue, Sep 19, 2017 at 6:06 AM, Andrew Scheller<EMAIL_ADDRESS>wrote:

We've had suggestions like this before... the problem always comes down to
where to draw the "cutoff point" between Hard Drives and SD cards - for
example I personally own an 80GB external HDD, and a 200GB micro SD card.
Etcher does clearly show the size of each drive in the drive-selection
window, and also in the main window after selecting a drive. I think it's
always a very delicate balancing act between making something "safe" and
making something "have too many annoying pop-ups which people will just
ignore anyway".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/1728#issuecomment-330532759,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCMU4fUzjYuddjssu86XHO72du4azks5sj7xRgaJpZM4PYk4d
.

I think that's a great idea! 👍
—
Juan Cruz Viotti,Software Engineer, Resin.io (https://resin.io)
twt: @resin_io
--- original message ---
On Thu, Sep 21, 2017 at 07:44 pm<EMAIL_ADDRESS>Alexandros Marinos wrote:
Perhaps a warning for drives over X size (64gb?) Something like -- "This
drive looks quite large. Please make sure you've not accidentally selected
a removable hard drive. [All OK, Continue] [Abort Flashing]"
cc @jviotti @jhemsmeier
--
Alexandros Marinos
Founder & CEO, Resin.io
+1<PHONE_NUMBER>
@alexandrosm
On Tue, Sep 19, 2017 at 6:06 AM, Andrew Scheller<EMAIL_ADDRESS>wrote:

We've had suggestions like this before... the problem always comes down to
where to draw the "cutoff point" between Hard Drives and SD cards - for
example I personally own an 80GB external HDD, and a 200GB micro SD card.
Etcher does clearly show the size of each drive in the drive-selection
window, and also in the main window after selecting a drive. I think it's
always a very delicate balancing act between making something "safe" and
making something "have too many annoying pop-ups which people will just
ignore anyway".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/resin-io/etcher/issues/1728#issuecomment-330532759,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABLUCMU4fUzjYuddjssu86XHO72du4azks5sj7xRgaJpZM4PYk4d
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
--- end of original message ---

I'm careful when writing images to the SSD card. Especially since I've got a 256Gb Transcend Jetdrive that I use for data storage.
This drive always shows up as a possible target to write images to.
Is it possible to (manually) create a blacklist of targets that Etcher won't show?

I hadn't heard of that before, so just googled and found https://www.amazon.co.uk/Transcend-256GB-JetDrive-Storage-Expansion/dp/B01BE0C2PU
Looks like it's basically a half-size SD card? (and if it plugs into the Mac's SD slot, then it's no surprise that Etcher detects it as a removable drive!)
Could you send us your diskutil output, so that we can see if there's a way to uniquely identify this particular drive?

It's a nice way to add additional storage. It's indeed a half size SD card, directly inserted into the designated slot.
Identifying this drive from diskutils output is not easy as far as I can tell.  The only recognisable attribute is it's name, which is chosen when formatting the device. See disk utils output below:
$ diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *480.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                 Apple_APFS Container disk1         479.1 GB   disk0s2

/dev/disk1 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +479.1 GB   disk1
                                 Physical Store disk0s2
   1:                APFS Volume Macintosh HD            442.2 GB   disk1s1
   2:                APFS Volume Preboot                 20.9 MB    disk1s2
   3:                APFS Volume Recovery                519.9 MB   disk1s3
   4:                APFS Volume VM                      2.1 GB     disk1s4

/dev/disk2 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *259.7 GB   disk2
   1:                  Apple_HFS Transcend JetDrive      259.7 GB   disk2s1

Even though it's reported as an 'internal' drive, many other people may use this slot for regular SD cards. Sometimes I use an SD card in a USB adapter that's also formatted in HFS format, so the filesystem type isn't a good discriminator.
The output from system_report might be more useful. It shows that this drive has 'owner ship' enabled. (ignore ownership = no).Excluding drives with ownership enabled is quite like 95% accurate, but not foolproof.
Hence, a blacklist in the advanced config is probably the best solution.
$  system_profiler SPStorageDataType
Storage:

    Macintosh HD:

      Available: 34,28 GB (34.276.270.080 bytes)
      Capacity: 479,11 GB (4<IP_ADDRESS>6 bytes)
      Mount Point: /
      File System: APFS
      Writable: Yes
      Ignore Ownership: No
      BSD Name: disk1s1
      Volume UUID: 14820296-AF63-3B3D-91F5-551DC0FD5A0A
      Physical Drive:
          Device Name: OWC Aura
          Media Name: AppleAPFSMedia
          Medium Type: SSD
          Protocol: SATA
          Internal: Yes
          Partition Map Type: Unknown

    Transcend JetDrive:

      Available: 237,69 GB (237.691.359.232 bytes)
      Capacity: 259,68 GB (259.677.814.784 bytes)
      Mount Point: /Volumes/Transcend JetDrive
      File System: Journaled HFS+
      Writable: Yes
      Ignore Ownership: No
      BSD Name: disk2s1
      Volume UUID: 02AD767B-6949-33F0-B031-592788C0B5CF
      Physical Drive:
          Device Name: Built In SDXC Reader
          Media Name: Apple SDXC Reader Media
          Protocol: Secure Digital
          Internal: Yes
          Partition Map Type: MBR (Master Boot Record)

The trouble with an explicit drive blacklist (I'm assuming you're referring to e.g. blacklisting /dev/disk2 in your example above) is that e.g. on Linux, the drives get assigned device-names depending on the order they were detected. So if you have USB drives GreenDisk, BlueDisk and RedDisk, and then  plugged in BlueDisk first, then RedDisk, and finally GreenDisk, you might have BlueDisk as /dev/sdb (/dev/sda is usually the internal HDD), RedDisk as /dev/sdc, and GreenDisk as /dev/sdd; so if you wanted to explicitly blacklist GreenDisk, then you couldn't really blacklist /dev/sdd because the next time you boot up (or if you unplugged and replugged the drives in a slightly different order), GreenDisk might have a different device-name. I'm not sure if I've explained that very well, but hopefully you see what I mean.
I dunno about Mac, but I think Windows also auto-assigns drive letters (D:, E:, F:, etc.) based on the order that devices are plugged in?

The Mac is more or less the same. Filtering has to be done on the volume name and not the name of the physical device.
In macOS each volume get's a name, comparable to the labels in a Linux filesystem. The devices also have a UUID. Linux has a similar mechanism, but I don't know if Windows does.
One difference between the Jetdrive and other SD cards is that the drive is always present. It's used is if it were an internal disk. Even though this doesn't make the device name predictable, it's quite consistent during reboots if no other external drives are attached.
How about creating a (hidden) file detector? If a volume contains a specific hidden file, such as .Exclude_from_Etcher this could be detected and used to exclude the drive from the list?
Would this work for Windows as well?

it's quite consistent during reboots if no other external drives are attached.

The "if" in that sentence is exactly the sort of thing I'm worried about... ;-)
(it's far too easy for somebody to reboot with some other external drive still plugged in)

How about creating a (hidden) file detector? If a volume contains a specific hidden file, such as .Exclude_from_Etcher this could be detected and used to exclude the drive from the list?

We could look for specific files, but that then gets tricky as it assumes that the partition is already mounted before Etcher tries reading it, and a file-on-a-filesystem is a partition-level thing, whereas Etcher operates on a disk-level basis; so if there are multiple partitions on a disk Etcher would need to check each of them (which again, it can't do if the OS doesn't auto-mount all the partitions, e.g. a disk with ext4 partitions connected to a Windows PC).
I like your idea of using the UUID though, but as you say, any feature we add to Etcher ought to work the same across all three platforms.
If we add a blacklist feature, I'm thinking some kind of feature-combination might work? E.g. blacklist a disk if it has label XXXX AND size YYYY ? (OR uuid ZZZZ ?)  But that quickly gets complicated, which means it might fail, and we wouldn't want people to blame Etcher if they use it to overwrite a drive that they thought they'd blacklisted.
Looking at the problem from a different angle, is there any other information we could display in the drive-selection window that would make it "easier" for you to avoid selecting the "wrong" drive?

All major operating systems have a concept similar to UUID to unique identify drives. In any case, we shouldn't make users manually pass UUIDs to the application. The drive selector modal should have a "blacklist" button alongside each of the drives that transparently fetches its unique id and saves it. Then, the user can potentially open a window from the settings page to see the blacklisted drives and whitelist any of those.

Those UUIDs are usually per volume / mountable partition or the GPT's UUID. Both of which should be accessible to us:

Windows tracks also this through /System Volume Information/IndexerVolumeGuid on the volume
Mac OS has /.fseventsd/.fseventsd-uuid (also per volume afaik)
Linux: not sure how, or if at all (?)

We could go with the serial number as well. If we add support for it on drivelist, then its a piece of cake.

There are indeed a lot of things to consider. Coming up with something that is 100% fault proof might be difficult. But combining different techniques could be feasible with a priority system?
If a UUID is available use it to blacklist a device. If not use serial number?
Linux supports UUID's on volume level and partition level, e.g.:
# ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 okt  2 19:49 64124f0a-173b-49fc-a134-3898709a878a -> ../../sda3
lrwxrwxrwx 1 root root 10 okt  2 19:49 BF12-60C5 -> ../../sda1
lrwxrwxrwx 1 root root 10 okt  2 19:49 d5ba79d7-44e9-469d-bc25-57345c45477e -> ../../sda2

# ls -l /dev/disk/by-partuuid/
total 0
lrwxrwxrwx 1 root root 10 okt  2 19:49 011e5e86-102f-4990-a5bb-9059e39a0559 -> ../../sda2
lrwxrwxrwx 1 root root 10 okt  2 19:49 0a194633-0ae2-49dc-977a-338d768f3a28 -> ../../sda3
lrwxrwxrwx 1 root root 10 okt  2 19:49 7d3cc72e-d7aa-4d8d-8156-c062793c985a -> ../../sda1

And serial numbers are available  in Linux to:
# lsblk -o NAME,SERIAL  -a
\NAME   SERIAL
sda    5VJB25AF
├─sda1
├─sda2
└─sda3
sdb    5VJB1Y5N
├─sdb1
├─sdb2
└─sdb3
loop0
loop1
loop2
loop3
loop4
loop5
loop6
loop7

or:
# ls -l /dev/disk/by-id/ | grep ata
total 0
lrwxrwxrwx 1 root root  9 okt  2 19:49 ata-ST9500420ASG_5VJB1Y5N -> ../../sdb
lrwxrwxrwx 1 root root 10 okt  2 19:49 ata-ST9500420ASG_5VJB1Y5N-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 okt  2 19:49 ata-ST9500420ASG_5VJB1Y5N-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 okt  2 19:49 ata-ST9500420ASG_5VJB1Y5N-part3 -> ../../sdb3
lrwxrwxrwx 1 root root  9 okt  2 19:49 ata-ST9500420ASG_5VJB25AF -> ../../sda
lrwxrwxrwx 1 root root 10 okt  2 19:49 ata-ST9500420ASG_5VJB25AF-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 okt  2 19:49 ata-ST9500420ASG_5VJB25AF-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 okt  2 19:49 ata-ST9500420ASG_5VJB25AF-part3 -> ../../sda3

Serialnumber detection seems to be possible on the Mac as well on device level.
Card Reader:

    Built in SD Card Reader:

      Vendor ID: 0x14e4
      Device ID: 0x16bc
      Subsystem Vendor ID: 0x14e4
      Subsystem ID: 0x96bc
      Revision: 0x0010
      Link Width: x1
      Link Speed: 2.5 GT/s

        SDXC Card (Class 10):

          Product Name: J`JDL
          Manufacturer ID: 0x74
          Revision: 2.0
          Serial Number: 637536167
          Manufacturing Date: 2016-06
          Specification Version: 3.0
          Capacity: 259.71 GB (259,710,255,104 bytes)
          Removable Media: Yes
          BSD Name: disk2
          Partition Map Type: MBR (Master Boot Record)
          Volumes:
            Transcend JetDrive:
              Available: 237.69 GB (237,691,351,040 bytes)
              Capacity: 259.68 GB (259,677,816,832 bytes)
              Writable: Yes
              File System: Journaled HFS+
              BSD Name: disk2s1
              Mount Point: /Volumes/Transcend JetDrive
              Content: Apple_HFS
              Volume UUID: 02AD767B-6949-33F0-B031-592788C0B5CF

I have no experience with Windows and agree that you don't want people to blame Etcher if they use it to overwrite a drive that they thought they'd blacklisted.
Would reversing the process help? All devices are blacklisted until a user whitelists one? That's more or less what users do in the current flow anyway.
Given the complexity I wouldn't be disappointed if you decide to leave things as they are. Upfront I didn't realise the complexity at all.
Thanks anyway for discussing this extensively.

Thanks anyway for discussing this extensively.

Thank you for your discussion too, and thanks for the research and info you've provided :+1:

Given the complexity I wouldn't be disappointed if you decide to leave things as they are. Upfront I didn't realise the complexity at all.

I suspect a lot of people don't realise how much hard work it takes, to make Etcher appear so "simple" ;-)

Should this be converted into a feature request ticket?

I just did this... tgif bit me in the a**. Whats a good recorevery software? thiking ddrescue in linux


IONIC SERVE gets ERR_CONNECTION_RESET

Ionic version:  (check one with "x")
[ ] 1.x (For Ionic 1.x issues, please use https://github.com/ionic-team/ionic-v1)
[ ] 2.x
[X] 3.x
I'm submitting a ...  (check one with "x")
[X] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior:
Hello. Thanks in advance for future help and sorry for my english, it is not my first language.
When running ionic serve in my application I get ERR_CONNECTION_RESET on Chrome.
This was working long time ago. And it is still working on other branch from the same application which has much less code of course.
I'm not being able to detect the problem. My command console gets
"TypeError: Cannot read property 'toString' of undefined
at injectLiveReloadScript ... AppData\Roaming\npm\node_modules\ionic\node_modules@ionic\cli-utils\lib\ionic1\serve.js:115:29)
at ReadFileContext.fs.readFile [as callback] (...\AppData\Roaming\npm\node_modules\ionic\node_modules@ionic\cli-utils\lib\ionic1\serve.js:158:33)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:366:13)"

Nevermind, somehow index.html was being deleted...


provideCellStatusBarItems  Test Fail on MacOS

From https://dev.azure.com/monacotools/Monaco/_build/results?buildId=181401&view=logs&j=9833cdaa-2e92-5f8d-56a8-239fd18fd628&t=9bc48eac-eb66-52c8-7250-d6490c5f3a15
  1) statusbar
       provideCellStatusBarItems called on metadata change:
     Error: asPromise TIMEOUT reached
      at asPromise (extensions/vscode-api-tests/src/utils.ts:128:16)
      at Context.<anonymous> (extensions/vscode-api-tests/src/singlefolder-tests/notebook.api.test.ts:291:34)
      at processImmediate (node:internal/timers:466:21)

Was just looking at this. That might make sense too, but it also seems to make sense to test this as an API contract.

Is it going to make a difference if I move off the deprecated WorkspaceEdit api?

The issue you have mentioned with the editor service, I wonder if that could be in play here because this failure is just implying that the cell did not get rendered and initially request statusbar items.


how to run plumber api + deploy

Hi,
How can I launch the R app from the terminal? I am not sure what this statement means: "To test the application can be launched from R, run launchAppDev()."
Also is it possible to deploy using Heroku without using docker?
Sorry, new to web dev :)
A

Hi you can run this command from the command line from the top-level directory of reactor:
R -e "library(reactor); launchAppDev()"
This will pass the expression to R and execute it. It should launch your browser and display the simple react app that is connected to R via plumbr.
I wish I could help you with deployment. I, too, am new to webdev and this repository is a collection of what I've learned to get started. If I find out how to deploy to Heroku I'll update the Readme.

@Zelazny7 it works! thanks. Didnt know you could put a react app within an r package lol. mind blown. Just incase you are interested: I found another app that use vue on the front end and r on the back-end.
Best,
A

Glad it worked for you. As a person new to web development I would be very interested in a link to the app that uses Vue as well. I have used React in the past and understand it's model but I've heard Vue is great, too.

@Zelazny7 sorry! forgot to paste the link: https://github.com/lautiamkok/nuxt-plumber.
A


Possible mistake in the assertions about the angles of the Triangle class

Dear Dr Beg,
Some assertions made about the angles() method of the Triangle class in lecture 5 are possibly wrong, because they don't agree with the specification for the method.

add a method to the Triangle class called  angles(). It should return a tuple of three floats, each representing the interior angle in radians of the vertices of the triangle, ordered so that the first angle is at the first vertex of the triangle, and so on.

However, in the assertions, it appears that the angles of the first and second vertices are accidentally swapped.

I appreciate your time into this issue in advance.
Yours truly,
esemsc-df329480 (Guanyuming He)

Dear Dr Beg,
It just occurred to me that perhaps you wanted each angle to correspond to an edge, instead of a vertex. Nevertheless, there's some inconsistency with the specification.

Hi @esemsc-df329480, thank you for letting us know. I agree with you that the problem description is a bit vague and can be understood differently. Would you like to suggest to us how you would formulate that question and we can consider including it for the next cohort?

Dear Dr Beg,
I think your description is good and clear. According to it, the first angle, angles()[0], should be the one at the first vertex, $(0,0)$; and it should be $\pi/2$. The problem is that $\pi/2$ is asserted to be angles()[1] instead.
With that said, however, since each vertex is assigned a name in __init__ ($a$, $b$, and $c$), I can think of a way to make the statement 100% accurate, using mathematical language. For example, we could rephrase the specification of angles() to:

The angles() method should return the tuple $(\angle cab, \angle abc, \angle bca)$, in radians.

Cheers. :)

That's a great suggestion. Alternatively, we can adjust the tests so it does not matter what the order of the angles in the tuple is.
Thank you! I will leave this issue open to remind us to update the materials.


Clear out-of-date extension global storage folders

Extensions may store some files in the global storage. After several iterations, the global storage may contain a lot of out-of-date folders there. And things will be even worse when the pre-release versions are installed:

Will VS Code consider supporting clean those out-of-date folders when extensions update? Or is it the extension's responsibility to clean them?

Similar to https://github.com/microsoft/vscode/issues/32461
//cc @dbaeumer

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

My bad, looks like that those folders for different versions are generated by the extension, not VS Code. So it should be the extension's responsibility to manage those caches.


Ideas for v2

bn254 g2, bn254 pairing?
consider creating "addons" or "others" module

move jubjub and pasta curves into "others" file
research other curves that can be included

brainpool? something else?

Mirror module system related changes from v2 noble-hashes

We are currently looking into replacing our rustbn-wasm build with this library for the bn254 (bn128) ADD, MUL, PAIRING precompiles from EIP-196 and EIP-197, so I agree that adding support for "bn254 g2, bn254 pairing" would be great idea! (respectively pretty valuable to us)

(side question: do you think we could use Noble for the BLS precompiles (so https://eips.ethereum.org/EIPS/eip-2537 ?)

@holgerd77 the performance would be decreased compared to wasm. For example, bls12 does 100 pairings/sec on fast mac. For comparison, ETH can do 4000 signatures/sec on the same cpu. Your current wasm solution is likely several times faster.
How constrained are you in terms of performance?

Cool!
For BLS I've compiled down the following list of potential function mappings, if you have something to add:
mcl.add -> bls12_381.fields.Fp.add
mcl.mul -> bls12_381.fields.Fp.mul
mcl.mulVec -> ? (bls12_381.fields.Fp.mulN ?)
mcl.millerLoop -> bls12_381.millerLoop
mcl.finalExp -> ?
mcl.G1/G2  -> bls12_381.G1/G2
mcl.Fp(2) -> bls12_381.fields.Fp (2)
mcl.verifyOrderG1/G2 -> bls12_381.G1.isWithinCurveOrder

We will go our "classic" route here, and provide the JS implementation as default and then let people opt-in (dependency-inect) WASM if they want to.
Actually with the pairing and if we could replace bn254 we would be completely "WASM free" for the EVM which would be a pretty big deal! 🤩

I will make it a priority then.

Great to hear (if you need financial support for this let me know)! 🙏 🤩

bn254 pairings have been added.

Ok, I am on it, see above PR!
Have now created a new build here https://github.com/holgerd77/noble-curves/tree/build-3ed792f . When building our EVM code I am getting two BLS related errors now, not sure if something to fix on your or on our side, will look into it:

Update: think this is just because of conflicting noble versions (now having noble from my branch + noble from ethereum-cryptography in). Can solve this with any annotation and remove later.

toHex() on the point seems not implemented (throws with "not implemeneted")

Same with toRawBytes()

In between state: I have now got multiplication and addition working. This should be tested enough, there are 100s (~1000 or so?) tests in the official ethereum/tests which all pass now. Will now move over to pairing.

Hi Paul,
I am currently trying to create a valid G2 point, which is not working yet. Can you have a look at the following code, guess you will spot quickly what is wrong?
import { hexToBytes } from '@ethereumjs/util'
import { bn254 } from '@noble/curves/bn254'

// From ecpairing_two_point_match_1 test
/* const input =
  '0x00000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000002198e9393920d483a7260bfb731fb5d25f1aa493335a9e71297e485b7aef312c21800deef121f1e76426a00665e5c4479674322d4f75edadd46debd5cd992f6ed090689d0585ff075ec9e99ad690c3395bc4b313370b38ef355acdadcd122975b12c85ea5db8c6deb4aab71808dcb408fe3d1e7690c43d37b4ce6cc0166fa7daa000000000000000000000000000000000000000000000000000000000000000130644e72e131a029b85045b68181585d97816a916871ca8d3c208c16d87cfd45198e9393920d483a7260bfb731fb5d25f1aa493335a9e71297e485b7aef312c21800deef121f1e76426a00665e5c4479674322d4f75edadd46debd5cd992f6ed090689d0585ff075ec9e99ad690c3395bc4b313370b38ef355acdadcd122975b12c85ea5db8c6deb4aab71808dcb408fe3d1e7690c43d37b4ce6cc0166fa7daa' */
console.log('test')

const inputG2 =
  '0x198e9393920d483a7260bfb731fb5d25f1aa493335a9e71297e485b7aef312c21800deef121f1e76426a00665e5c4479674322d4f75edadd46debd5cd992f6ed090689d0585ff075ec9e99ad690c3395bc4b313370b38ef355acdadcd122975b12c85ea5db8c6deb4aab71808dcb408fe3d1e7690c43d37b4ce6cc0166fa7daa'

const inputG2Bytes = hexToBytes(inputG2)
console.log(inputG2Bytes.length)

const pFP1 = bn254.fields.Fp2.fromBytes(inputG2Bytes.slice(0, 64))
const pFP2 = bn254.fields.Fp2.fromBytes(inputG2Bytes.slice(64, 128))

const pG2 = bn254.G2.ProjectivePoint.fromAffine({
  x: pFP1,
  y: pFP2,
})
pG2.assertValidity()

So this is the test input from one of the tests, input is the full input passed to the precompile and inputG2 the part I am using for G2 (obviously contained in input).
I am trying to get a positive assertValidity() check. Atm it is always throwing there.

Ok, I think I've found the respective code parts in your tests, will try to take it from there.

Ok, all official tests passing (I basically did the same as you did in your EVM setup just not in elegant 😂).
https://github.com/ethereumjs/ethereumjs-monorepo/pull/3564/
Think this should be enough for a release, at least from my side! So the test suite from ethereum/tests should really be complete, since the EIPs are out for years.
The only thing which irritated me mildy is that I had to flip the elements for the G2 point creation, see:
https://github.com/ethereumjs/ethereumjs-monorepo/pull/3564/files#diff-d8e4e56024eab6bc6446b026499696dcfcc4cbb9b6bdfe6c9fd35e983052d026R83
Not sure if this is an inconsistency in your library or in the EIP or otherwise has some reason I do not understand. If this is something to be adjusted on your side you can simply do along release I guess without another testing round.
Again, thanks a lot for all the work on this as well as this thorough EVM-specific test setup! 🙏 🎉

It's the same issue as in with toHex: there is no standard for this. Someone can serialize imaginary part of the complex number (G2) first, someone does the opposite.
For comparison, BLS was specced with hash-to-curve, so it's much better.
I will see what can be improved for your cases and prepare new release today/tomorrow.

There would be no serialization by default for now. Reasoning:

Some libraries do LE (little-endian), some do BE (big-endian, byte-swapped)
Some libraries add flags while serializing points (similar to BLS), some don't
Some libraries use imaginary part last in G2, some use it first

EVM picked: BE, no-flag, imag-first. Which is not compatible with something like ZEC and less feature-full. For example, no one else does BE.
I will add it to the readme. The community can then figure out the optional path of moving forward.

1.5.0 is out. Will ask to include it in eth-cryptography

@holgerd77 does it make more sense to include the update into eth-cryptography after it gets audited? Everything in eth-cryptography is audited.

Cool, will test! 🤩
Not including yet into eth-cryptography might counterbalance some of our tree shaking efforts for the moment (not fully sure), since we then might get double code parts from the different versions in.
Guess nevertheless makes sense (not to include yet). So, as you said, audit is not so far away, right?

Audit is this month. So even if you do major releases today, you can do x.y.1 patch to improve tree-shaking later this month.

Ah, that’s great, no no, we are still a month away from releases

@paulmillr Do you think this library will be able to be compiled with Static Hermes?

I’m not sure what are the constraints. AFAIK, many people use it with react native.

I’m not sure what are the constraints

Sound types

AFAIK, many people use it with react native.

Static Hermes isn't released yet. Static Hermes compiles JS to native instructions ahead of time.
Here is some context:
https://tmikov.blogspot.com/2023/09/how-to-speed-up-micro-benchmark-300x.html
https://speakerdeck.com/tmikov2023/static-hermes-react-native-eu-2023-announcement
https://x.com/tmikov/status/1700353858763911570?s=20
Basically noble will be THE fastest cryptography library on React Native without native code

What's "sound types" and how does one detect which types are unsound? Is there automatic tool?

https://speakerdeck.com/tmikov2023/static-hermes-react-native-eu-2023-announcement?slide=12

@paulmillr

What's "sound types" and how does one detect which types are unsound? Is there automatic tool?

Essentially if you have type number annotation on a variable in JS runtime it can become undefined or string or whatever. Instead of silently failing or basically continuing to work Static Hermes will throw just like in any strongly typed language. So if the noble depends on JS looseness it must be refactored.

I'm open to a speed up, but there should be some simple tool to check all files for "sound type" errors.

@paulmillr that's a good point. I assume there will be something like that. I'm gonna try that out

@paulmillr Ok I went ahead and tried to compile https://github.com/paulmillr/noble-secp256k1/blob/main/index.js . I chose secp256k1 because I'm already familiar with the lib.
Here are steps to try it out:
Step 1: Clone Hermes and checkout to static_h
mkdir hermes_build
cd hermes_build
git clone https://github.com/facebook/hermes

Step 2: Build Hermes
Taken from: https://github.com/facebook/hermes/blob/static_h/doc/BuildingAndRunning.md
cmake -S hermes -B build -G Ninja
cmake --build ./build

To build release:
cmake -S hermes -B build_release -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build ./build_release

Step 3: Run the index.js with Hermes

Remove (or refactor to module.exports) the last 2 export lines since Hermes doesn't support ES6 for now
Run the file with Hermes

build/bin/shermes --typed -exec ../noble-secp256k1/index.js

The --typed instruction will try to compile the file. That's essentially how you verify if you have sound or unsound types. More info about that

I got bunch 13 errors after removing the `export` instruction.  Click to expand
../noble-secp256k1/index.js:516:1: error: 'export' statement requires module mode
export { getPublicKey, sign, signAsync, verify, CURVE, // Remove the export to easily use in REPL
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:517:39: warning: the variable "ProjectivePoint" was not declared in anonymous function
getSharedSecret, etc, utils, Point as ProjectivePoint, Signature }; // envs like browser console
                                      ^~~~~~~~~~~~~~~
Emitted 1 errors. exiting.
➜  hermes_build build/bin/shermes --typed -exec ../noble-secp256k1/index.js
../noble-secp256k1/index.js:9:20: warning: local variable may be used prior to declaration, assuming 'any'
const crv = (x) => mod(mod(x * x) * x + CURVE.b); // x³ + ax + b weierstrass formula; a=0
../noble-secp256k1/index.js:9:24: warning: local variable may be used prior to declaration, assuming 'any'
const crv = (x) => mod(mod(x * x) * x + CURVE.b); // x³ + ax + b weierstrass formula; a=0
../noble-secp256k1/index.js:10:14: warning: ft: typing of pattern parameters not implemented, :any assumed
const err = (m = '') => { throw new Error(m); }; // error helper, messes-up stack trace
             ^~~~~~
../noble-secp256k1/index.js:21:39: warning: local variable may be used prior to declaration, assuming 'any'
const toU8 = (a, len) => au8(str(a) ? h2b(a) : u8n(au8(a)), len); // norm(hex/u8a) to u8a
                                      ^~~
../noble-secp256k1/index.js:22:17: warning: ft: typing of pattern parameters not implemented, :any assumed
const mod = (a, b = P) => { let r = a % b; return r >= 0n ? r : b + r; }; // mod division
                ^~~~~
../noble-secp256k1/index.js:22:41: warning: local variable may be used prior to declaration, assuming 'any'
const mod = (a, b = P) => { let r = a % b; return r >= 0n ? r : b + r; }; // mod division
                                        ^
../noble-secp256k1/index.js:22:65: warning: local variable may be used prior to declaration, assuming 'any'
const mod = (a, b = P) => { let r = a % b; return r >= 0n ? r : b + r; }; // mod division
                                                                ^
../noble-secp256k1/index.js:30:5: error: ft: static/async/generator methods unsupported
    static fromAffine(p) {
    ^
../noble-secp256k1/index.js:33:5: error: ft: static/async/generator methods unsupported
    static fromHex(hex) {
    ^
../noble-secp256k1/index.js:52:5: error: ft: static/async/generator methods unsupported
    static fromPrivateKey(k) { return G.mul(toPriv(k)); } // Create point from a private key.
    ^
../noble-secp256k1/index.js:109:12: warning: ft: typing of pattern parameters not implemented, :any assumed
    mul(n, safe = true) {
           ^~~~~~~~~~~
../noble-secp256k1/index.js:149:11: warning: ft: typing of pattern parameters not implemented, :any assumed
    toHex(isCompressed = true) {
          ^~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:154:16: warning: ft: typing of pattern parameters not implemented, :any assumed
    toRawBytes(isCompressed = true) {
               ^~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:160:7: warning: ft: typing of object declarators not implemented, :any assumed
const { BASE: G, ZERO: I } = Point; // Generator, identity points
      ^~~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:178:27: error: ft: incompatible binary operation: + cannot be applied to string and union
const b2n = (b) => BigInt('0x' + (b2h(b) || '0')); // bytes to number
                          ^~~~~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:184:18: warning: ft: typing of pattern parameters not implemented, :any assumed
const concatB = (...arrs) => {
                 ^~~~~~~
../noble-secp256k1/index.js:185:19: warning: local variable may be used prior to declaration, assuming 'any'
    const r = u8n(arrs.reduce((sum, a) => sum + au8(a).length, 0)); // create u8a of summed length
                  ^~~~
../noble-secp256k1/index.js:187:5: warning: local variable may be used prior to declaration, assuming 'any'
    arrs.forEach(a => { r.set(a, pad); pad += a.length; }); // ensure they have proper type
    ^~~~
../noble-secp256k1/index.js:190:19: warning: ft: typing of pattern parameters not implemented, :any assumed
const inv = (num, md = P) => {
                  ^~~~~~
../noble-secp256k1/index.js:191:23: warning: local variable may be used prior to declaration, assuming 'any'
    if (num === 0n || md <= 0n)
                      ^~
../noble-secp256k1/index.js:192:47: warning: local variable may be used prior to declaration, assuming 'any'
        err('no inverse n=' + num + ' mod=' + md); // no neg exponent for now
                                              ^~
../noble-secp256k1/index.js:193:22: warning: local variable may be used prior to declaration, assuming 'any'
    let a = mod(num, md), b = md, x = 0n, y = 1n, u = 1n, v = 0n;
                     ^~
../noble-secp256k1/index.js:193:31: warning: local variable may be used prior to declaration, assuming 'any'
    let a = mod(num, md), b = md, x = 0n, y = 1n, u = 1n, v = 0n;
                              ^~
../noble-secp256k1/index.js:199:30: warning: local variable may be used prior to declaration, assuming 'any'
    return b === 1n ? mod(x, md) : err('no inverse'); // b is gcd at this point
                             ^~
../noble-secp256k1/index.js:216:32: warning: ft: typing of pattern parameters not implemented, :any assumed
const getPublicKey = (privKey, isCompressed = true) => {
                               ^~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:217:18: error: ft: named property access only allowed on objects, found class constructor
    return Point.fromPrivateKey(privKey).toRawBytes(isCompressed); // 33b or 65b output
                 ^~~~~~~~~~~~~~
../noble-secp256k1/index.js:217:53: warning: local variable may be used prior to declaration, assuming 'any'
    return Point.fromPrivateKey(privKey).toRawBytes(isCompressed); // 33b or 65b output
                                                    ^~~~~~~~~~~~
../noble-secp256k1/index.js:226:5: error: ft: static/async/generator methods unsupported
    static fromCompact(hex) {
    ^
../noble-secp256k1/index.js:270:30: warning: ft: typing of pattern parameters not implemented, :any assumed
const prepSig = (msgh, priv, opts = optS) => {
                             ^~~~~~~~~~~
../noble-secp256k1/index.js:271:43: error: ft: unknown array property
    if (['der', 'recovered', 'canonical'].some(k => k in opts)) // Ban legacy options
                                          ^~~~
../noble-secp256k1/index.js:271:58: warning: local variable may be used prior to declaration, assuming 'any'
    if (['der', 'recovered', 'canonical'].some(k => k in opts)) // Ban legacy options
                                                         ^~~~
../noble-secp256k1/index.js:273:9: warning: ft: typing of object declarators not implemented, :any assumed
    let { lowS } = opts; // generates low-s sigs by default
        ^~~~~~~~
../noble-secp256k1/index.js:273:20: warning: local variable may be used prior to declaration, assuming 'any'
    let { lowS } = opts; // generates low-s sigs by default
                   ^~~~
../noble-secp256k1/index.js:274:9: warning: local variable may be used prior to declaration, assuming 'any'
    if (lowS == null)
        ^~~~
../noble-secp256k1/index.js:275:9: warning: local variable may be used prior to declaration, assuming 'any'
        lowS = true; // RFC6979 3.2: we skip step A
        ^~~~
../noble-secp256k1/index.js:280:15: warning: local variable may be used prior to declaration, assuming 'any'
    let ent = opts.extraEntropy; // RFC6979 3.6: additional k' (optional)
              ^~~~
../noble-secp256k1/index.js:283:19: warning: local variable may be used prior to declaration, assuming 'any'
            ent = etc.randomBytes(fLen); // if true, use CSPRNG to generate data
                  ^~~
../noble-secp256k1/index.js:295:19: warning: local variable may be used prior to declaration, assuming 'any'
        const q = G.mul(k).aff(); // q = Gk
                  ^
../noble-secp256k1/index.js:304:13: warning: local variable may be used prior to declaration, assuming 'any'
        if (lowS && moreThanHalfN(s)) { // if lowS was passed, ensure s is always
            ^~~~
../noble-secp256k1/index.js:377:38: warning: ft: typing of pattern parameters not implemented, :any assumed
const signAsync = async (msgh, priv, opts = optS) => {
                                     ^~~~~~~~~~~
../noble-secp256k1/index.js:378:11: warning: ft: typing of object declarators not implemented, :any assumed
    const { seed, k2sig } = prepSig(msgh, priv, opts); // Extract arguments for hmac-drbg
          ^~~~~~~~~~~~~~~
../noble-secp256k1/index.js:378:49: warning: local variable may be used prior to declaration, assuming 'any'
    const { seed, k2sig } = prepSig(msgh, priv, opts); // Extract arguments for hmac-drbg
                                                ^~~~
../noble-secp256k1/index.js:379:27: warning: local variable may be used prior to declaration, assuming 'any'
    return hmacDrbg(true)(seed, k2sig); // Re-run drbg until k2sig returns ok
                          ^~~~
../noble-secp256k1/index.js:379:33: warning: local variable may be used prior to declaration, assuming 'any'
    return hmacDrbg(true)(seed, k2sig); // Re-run drbg until k2sig returns ok
                                ^~~~~
../noble-secp256k1/index.js:381:27: warning: ft: typing of pattern parameters not implemented, :any assumed
const sign = (msgh, priv, opts = optS) => {
                          ^~~~~~~~~~~
../noble-secp256k1/index.js:382:11: warning: ft: typing of object declarators not implemented, :any assumed
    const { seed, k2sig } = prepSig(msgh, priv, opts); // Extract arguments for hmac-drbg
          ^~~~~~~~~~~~~~~
../noble-secp256k1/index.js:382:49: warning: local variable may be used prior to declaration, assuming 'any'
    const { seed, k2sig } = prepSig(msgh, priv, opts); // Extract arguments for hmac-drbg
                                                ^~~~
../noble-secp256k1/index.js:383:28: warning: local variable may be used prior to declaration, assuming 'any'
    return hmacDrbg(false)(seed, k2sig); // Re-run drbg until k2sig returns ok
                           ^~~~
../noble-secp256k1/index.js:383:34: warning: local variable may be used prior to declaration, assuming 'any'
    return hmacDrbg(false)(seed, k2sig); // Re-run drbg until k2sig returns ok
                                 ^~~~~
../noble-secp256k1/index.js:385:33: warning: ft: typing of pattern parameters not implemented, :any assumed
const verify = (sig, msgh, pub, opts = optV) => {
                                ^~~~~~~~~~~
../noble-secp256k1/index.js:386:9: warning: ft: typing of object declarators not implemented, :any assumed
    let { lowS } = opts; // ECDSA signature verification
        ^~~~~~~~
../noble-secp256k1/index.js:386:20: warning: local variable may be used prior to declaration, assuming 'any'
    let { lowS } = opts; // ECDSA signature verification
                   ^~~~
../noble-secp256k1/index.js:387:9: warning: local variable may be used prior to declaration, assuming 'any'
    if (lowS == null)
        ^~~~
../noble-secp256k1/index.js:388:9: warning: local variable may be used prior to declaration, assuming 'any'
        lowS = true; // Default lowS=true
        ^~~~
../noble-secp256k1/index.js:389:21: warning: local variable may be used prior to declaration, assuming 'any'
    if ('strict' in opts)
                    ^~~~
../noble-secp256k1/index.js:396:21: error: ft: class Signature constructor expects 3 arguments, but 2 supplied
        sig_ = rs ? new Signature(sig.r, sig.s).assertValidity() : Signature.fromCompact(sig);
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:396:78: error: ft: named property access only allowed on objects, found class constructor
        sig_ = rs ? new Signature(sig.r, sig.s).assertValidity() : Signature.fromCompact(sig);
                                                                             ^~~~~~~~~~~
../noble-secp256k1/index.js:398:53: error: ft: named property access only allowed on objects, found class constructor
        P = pub instanceof Point ? pub.ok() : Point.fromHex(pub); // Validate public key
                                                    ^~~~~~~
../noble-secp256k1/index.js:405:11: warning: ft: typing of object declarators not implemented, :any assumed
    const { r, s } = sig_;
          ^~~~~~~~
../noble-secp256k1/index.js:406:9: warning: local variable may be used prior to declaration, assuming 'any'
    if (lowS && moreThanHalfN(s))
        ^~~~
../noble-secp256k1/index.js:406:31: warning: local variable may be used prior to declaration, assuming 'any'
    if (lowS && moreThanHalfN(s))
                              ^
../noble-secp256k1/index.js:410:24: warning: local variable may be used prior to declaration, assuming 'any'
        const is = inv(s, N); // s^-1
                       ^
../noble-secp256k1/index.js:412:24: warning: local variable may be used prior to declaration, assuming 'any'
        const u2 = mod(r * is, N); // u2 = rs^-1 mod n
                       ^
../noble-secp256k1/index.js:413:13: warning: local variable may be used prior to declaration, assuming 'any'
        R = G.mulAddQUns(P, u1, u2).aff(); // R = u1⋅G + u2⋅P
../noble-secp256k1/index.js:421:18: warning: local variable may be used prior to declaration, assuming 'any'
    return v === r; // mod(R.x, n) == r
                 ^
../noble-secp256k1/index.js:423:39: warning: ft: typing of pattern parameters not implemented, :any assumed
const getSharedSecret = (privA, pubB, isCompressed = true) => {
                                      ^~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:424:18: error: ft: named property access only allowed on objects, found class constructor
    return Point.fromHex(pubB).mul(toPriv(privA)).toRawBytes(isCompressed); // ECDH
                 ^~~~~~~
../noble-secp256k1/index.js:424:62: warning: local variable may be used prior to declaration, assuming 'any'
    return Point.fromHex(pubB).mul(toPriv(privA)).toRawBytes(isCompressed); // ECDH
                                                             ^~~~~~~~~~~~
../noble-secp256k1/index.js:438:34: warning: ft: typing of pattern parameters not implemented, :any assumed
    hmacSha256Async: async (key, ...msgs) => {
                                 ^~~~~~~
../noble-secp256k1/index.js:444:55: warning: local variable may be used prior to declaration, assuming 'any'
        return u8n(await s.sign('HMAC', k, concatB(...msgs)));
                                                      ^~~~
../noble-secp256k1/index.js:448:19: warning: ft: typing of pattern parameters not implemented, :any assumed
    randomBytes: (len = 32) => {
                  ^~~~~~~~
../noble-secp256k1/index.js:452:43: warning: local variable may be used prior to declaration, assuming 'any'
        return crypto.getRandomValues(u8n(len));
                                          ^~~
../noble-secp256k1/index.js:464:5: warning: ft: unsupported property for typed object, assuming 'any'
    precompute(w = 8, p = G) { p.multiply(3n); w; return p; }, // no-op
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../noble-secp256k1/index.js:464:16: warning: ft: typing of pattern parameters not implemented, :any assumed
    precompute(w = 8, p = G) { p.multiply(3n); w; return p; }, // no-op
               ^~~~~
../noble-secp256k1/index.js:464:23: warning: ft: typing of pattern parameters not implemented, :any assumed
    precompute(w = 8, p = G) { p.multiply(3n); w; return p; }, // no-op
                      ^~~~~
../noble-secp256k1/index.js:464:32: warning: local variable may be used prior to declaration, assuming 'any'
    precompute(w = 8, p = G) { p.multiply(3n); w; return p; }, // no-op
                               ^
../noble-secp256k1/index.js:464:48: warning: local variable may be used prior to declaration, assuming 'any'
    precompute(w = 8, p = G) { p.multiply(3n); w; return p; }, // no-op
                                               ^
../noble-secp256k1/index.js:464:58: warning: local variable may be used prior to declaration, assuming 'any'
    precompute(w = 8, p = G) { p.multiply(3n); w; return p; }, // no-op
                                                         ^
../noble-secp256k1/index.js:472:20: warning: ft: empty array with no context, assuming 'any' array
    const points = []; // 10x sign(), 2x verify(). To achieve this,
                   ^~
../noble-secp256k1/index.js:474:13: warning: local variable may be used prior to declaration, assuming 'any'
    let p = G, b = p; // a lot of points related to base point G.
            ^
../noble-secp256k1/index.js:491:13: warning: local variable may be used prior to declaration, assuming 'any'
    let p = I, f = G; // f must be G, or could become I in the end
            ^
../noble-secp256k1/index.js:491:20: warning: local variable may be used prior to declaration, assuming 'any'
    let p = I, f = G; // f must be G, or could become I in the end
                   ^
../noble-secp256k1/index.js:508:38: error: ft: indexed access only allowed on array and tuple, found void
            f = f.add(neg(cnd1, comp[off1])); // bits are 0: add garbage to fake point
                                     ^~~~
../noble-secp256k1/index.js:511:38: error: ft: indexed access only allowed on array and tuple, found void
            p = p.add(neg(cnd2, comp[off2])); // bits are 1: add to result point
                                     ^~~~
Emitted 13 errors. exiting.

I think it's should be rather easy to fix these.

@paulmillr have you had a chance to check static hermes out?

@alexandrius the build commands do not produce any shermes file

@paulmillr did you checkout to static_h branch? I missed the part in the command

I have removed all initial errors (see https://gist.github.com/paulmillr/d9d5fdf8d8c9d7c954e9f26fdd30bc9d) but then compiler emitted 20 more errors.
Seems like it's unreasonably complex for now.

 Even this code doesn't work and is constantly segfaulting. https://gist.github.com/paulmillr/8002c140bec8ea64fd3b5efcf867eaf1


Clearer info docstring

Fixes #7818. @hoechenberger @larsoner @agramfort OK with you?

@cbrnr care to address #7774 at the same time?

This already helps , thanks @cbrnr ! Feel free to tackle the cross references next if you're up for it


try wasm-bindgen fixes

The command that failed was this one:
/home/runner/.cargo/bin/cargo +stable build --target wasm32-unknown-unknown --manifest-path ./example-projects/reqwest-wasm-example/Cargo.toml

So I think the failure is caused by this line: https://github.com/ozabalaferrera/cloudevents-sdk-rust/blob/09661ddaf72dfec2133962ae4dc6d322823558a1/example-projects/reqwest-wasm-example/Cargo.toml#L5
See: https://github.com/smoelius/cloudevents-sdk-rust/actions/runs/12137884805/job/33842119071
Good luck!

Thanks @smoelius ! It looks like I had to change the edition on both the root, project manifest and the example project manifest. Other combinations of edition and resolver didn't seem to do it. I squashed to get this ready for a PR to the official repo. Thanks again.


Newer GEMINI installer changed options

And this makes installation fail:
/opt/bcbio/anaconda/bin/python -Es gemini_install.py --help
usage: gemini_install.py [-h] [--nodata] [--gemini-version GEMINI_VERSION]
                         tooldir datadir

Automated installer for gemini framework.

positional arguments:
  tooldir               Directory to install 3rd party software tools
  datadir               Directory to install anaconda python and gemini data
                        files

optional arguments:
  -h, --help            show this help message and exit
  --nodata              Do not install data dependencies
  --gemini-version GEMINI_VERSION
                        Install one specific gemini version with a fixed
                        dependency chain.

Luca;
Thanks much for reporting this and apologies about being slow in getting the GEMINI install up to date with the latest release. If you upgrade to the latest development it should now install cleanly. We're hoping to have a 0.9.5 release next week including this fix. We install the GEMINI code itself with conda now and then only install the data if requested. Please let us know if you run into any problems with installing/upgrading now.


Search service implementation

This PR closes #135
Worked on using the REST API's search endpoint instead of the mock search service.
Replaced the search method in search.service.ts with a real implementation.
Refactored parts of the data builders, to make sure the type of the requested item does not need to be known and can be retrieved from the object's type.
Made sure the right SearchResultObjects are created, based on the DSpaceObject they wrap.
Fixed all (including e2e) tests that were failing due to the changes I made.
I'm going to add implementation for other methods in the Search Service.
This PR is still in progress.

This PR closes #135

@atarix83 Thanks for your feedback, I made some new commits to solve some of these issues.

because of we're using in mydspace implementation I should move search service and all components related, to the shared directory

I don't think this is necessary. You can simply import the search module into the mydspace module if you need it there. If we move this to the shared folder, everything might end up in there and the shared folder will lose its purpose.

I should change the uiSearchRoute param in SearchService according to the ui route, as we made here

You're right. I changed this to make sure the current route is used instead of the hardcoded "/search" path. I reused your code for this.

In mydspace we are using the param configuration of the Search rest endpoint,
so we need to pass it to SearchService. I think it should be a property of the SearchOptions class

I believe I already did that, somewhere in the process. Both the sorting and pagination parameters as well as the current filters, are saved in the (Paginated)SearchOptions object now.

In the mock version of the search module, loading spinner was shown during the search. Now it doesn't happen but i think it's useful

I made sure the loader component is also shown when the result object does not exist yet. I saw this was already done in the collection-page component, but not in any other components. This might explain why the spinner doesn't appear on the home page either.

another improvement should be to show a message whether the search does not return any results.

I added a message when the result set is an empty array.

When the search doesn't return any reults this error occurs

I could not reproduce this error (anymore). I did solve another issue where the result list would not exist if there were no results. This didn't cause any errors in my console, but I wasn't able to determine when the request was finished or not, without fixing this first.

While implementig the mydspace page to retrieve filters config and facet values,
we have used only the response from /api/discover/search/objects endpoint.
This should save other further REST call to build search results.
If this could be useful the full code is available here

In my opinion, it should not matter what endpoint is called. What I did is call the specific endpoints I need. If this data was already returned by another request, it should have been cached. I know it wasn't implemented that way yet, but I don't think this should be the SearchService's concern. If you would want to show the facets on another page, you would still have to ask the SearchService for this data. It would not make sense to request the whole /objects endpoint in this case.

@tdonohue:

If I visit a Collection Homepage, the "Recent Submissions" never loads.  In the Chrome Developer Tools Console, I see this error message: Error: A browse endpoint for items on dc.date.issued isn't configured.  (I tested this on the latest master and it works perfectly there, so this error seems to be caused by this PR)

Fixed by c125f36f43403c320258bb0ef4571ee3500770bc, forgot this. in a refactor

In the Search filters, the "Date" filter doesn't work (it lists possible values, but selecting one always returns "no results"). All other filters function properly.

This will be fixed in the PR that adds the proper date widget as described in the mockups.

"Sort By" option in Settings doesn't seem to affect the results listing. It changes the URL in the Chrome address bar (e.g. sortDirection=DESC), but the results listing remain in the same order.

Fixed. Reason was a peculiarity with typescript string enums. Apparently you can't fetch a key by its value (only for strings, it works for other enums). This was fixed by making keys and values identical.

"Results per page" option in Settings works fine in the list view. However, it doesn't seem to function properly in the grid view. If I select a different option, it always resets itself back to whatever value it last was in the list view. For example, if I switch to "list view" and select "5" results per page, this results in multiple pages.  If I then switch to "grid view" and select a higher value (e.g. "12"), pagination remains.

Lotte ran in to some issues with the different page sizes for grids and lists. Since you already noted that it wasn't very clear to an end user what was going on, we decided to scrap the idea, but there were some remnants left over. I've removed those now, and both views, now use the same page sizes again.
@PTrottier:
These will probably have happened automatically by the IDE. Trailing commas are no problem in JS, some people even encourage their use, as it means you can add a next property without having to modify the previous line, so you get a cleaner diff in git.
A missing newline at the end of a file can cause issues with certain tools, so I've added it.

I've re-tested this PR today and all the bugs I'd previously reported (other than the date filter, as noted) were fixed. This works great now, so I've given it a 👍 .
However, I will note one other oddity I just encountered. It's not really a "bug", but it's definitely unexpected behavior. If I use the demo REST API with this PR and search on "standards-based metadata procedure", one might expect that the top match would be the document with this title:
"Standards-based metadata procedures for retrieving data for display or mining utilizing persistent (data-DOI) identifiers"
But, it's not the top match. Instead, the results come back ordered alphabetically, so that the "best match" is listed third (out of three). The other two matches also do contain these words (in either title or abstract), but not all words even appear in the title. Here's the three titles in the order they are returned:

"Science, institutional archives and open access: an overview and a pilot survey on the Italian cancer research institutions" (Zero matches in title, but all words are in abstract)
"Standards-based curation of a decade-old digital repository dataset of molecular information" (One word in title, and others are in abstract)
"Standards-based metadata procedures for retrieving data for display or mining utilizing persistent (data-DOI) identifiers" (This is the document I'd expect to see first as what I searched on is the beginning of this title)

@tdonohue The default sort order was set to dc.title (ascending) in the SortOptions constructor. I removed the default parameters, and made sure that it's set by the responsible components and services. The search page is now sorted by score (descending) by default. The other components still order by dc.title (ascending).


design: cleanup blog post page headers

This PR cleans up the blog post page header layout and styles, and fixes a horizontal scroll issue on small screens that I introduced last night.

Before
After

The deploy preview looks great 🚀, I'm going to merge this.


Playground bugfix: The demo files appeared with an extra newline at the start.

Pre-submit checklist:

Branch

[X] Commit sequence broadly makes sense
[X] Key commits have useful messages
[ ] Relevant tickets are mentioned in commit messages

PR

[X] Self-reviewed the diff
[X] Useful pull request description
[X] Reviewer requested

If you updated any cabal files or added Haskell packages:

[ ] $(nix-build default.nix -A dev.scripts.updateMaterialized) to update the materialized Nix files
[ ] Update hie-*.yaml files if needed

If you changed any Haskell files:

[X] $(nix-shell shell.nix --run fix-stylish-haskell) to fix any formatting issues

If you changed any Purescript files:

[ ] $(nix-shell shell.nix --run fix-purty) to fix any formatting issues

Pre-merge checklist:

[ ] Someone approved it
[ ] Commits have useful messages
[ ] Review clarifications made it into the code
[ ] History is moderately tidy; or going to squash-merge

This one line change has now taken more than half a day for hydra to try (and fail) to build. 😭

Try rebasing on master, that will help.

Okay, thanks. Here we go. Fingers crossed...


The diff coverage is 53.65%.

:exclamation: Current head 8621d2a differs from pull request most recent head c79c273. Consider uploading reports for the commit c79c273 to get more accurate results

@@             Coverage Diff              @@
##                dev   #14055      +/-   ##
============================================
+ Coverage     38.20%   38.32%   +0.12%
- Complexity     4434     4440       +6
============================================
  Files          1220     1222       +2
  Lines         42706    42624      -82
  Branches       4735     4693      -42
============================================
+ Hits          16314    16335      +21
+ Misses        24587    24482     -105
- Partials       1805     1807       +2

Impacted Files
Coverage Δ

...nscheduler/api/configuration/AppConfiguration.java
93.93% <ø> (ø)

...inscheduler/api/controller/ExecutorController.java
17.94% <ø> (ø)

...lphinscheduler/api/controller/LoginController.java
51.72% <0.00%> (-16.46%)
:arrow_down:

...che/dolphinscheduler/api/python/PythonGateway.java
16.94% <0.00%> (ø)

...inscheduler/api/service/impl/UsersServiceImpl.java
70.78% <ø> (ø)

...e/dolphinscheduler/common/constants/Constants.java
75.00% <ø> (ø)

...ler/plugin/task/api/am/YarnApplicationManager.java
0.00% <0.00%> (ø)

...rver/worker/processor/WorkerTaskKillProcessor.java
0.00% <0.00%> (ø)

...er/api/security/impl/sso/CasdoorAuthenticator.java
72.22% <72.22%> (ø)

...cheduler/api/service/impl/ExecutorServiceImpl.java
51.49% <78.26%> (+1.11%)
:arrow_up:

... and 5 more

... and 36 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Bump @guardian/support-dotcom-components

What does this change?
Bumps @guardian/support-dotcom-components to v2.0.1.
This is a major change as per our recommendations, and consumers should
ensure that they provide a compatible version of the package.
Why?
The latest is v2.0.1 and this is what is used in dotcom-rendering.

Can we drop this as a dependency? It's not used here and not a peer of a dep as far as I can see?


mQc: Notify in list cards

When we added new card to list, we need post and get data to/from database, but if we use notify.... (like Den said) after added car this card not added to database, respectively we need get all items from database after called onResume or maybe get all database after app started and post all items before close app, but it's bad solution (i think)

Cursorloader


Update dependencies

As seen here, the fabpot/php-cs-fixer package is not maintained anymore.

@aledeg it's not much of an issue as it's a dev dependency, but PRs are welcomed :)

@theofidry It just raises a warning while using composer. It is not breaking anything but I like to have no warnings :).
I'll try to find time to submit a PR.


Error : OAuth Server-to-Server credential support

When we are trying to use v6.0.0-beta.11 or v6.0.0-beta.12 or v6.0.0-beta.13...
we are getting below Error :

Any update to support the oauth-server-to-server credential Type

This issue has come up in a few different places. We placed it near the top of the backlog and will be taking a look at this soon. We will update this issue when it's fixed!

Currently it is not possible to add JWT credentials to the Experience Platform API and the beta versions of the uploader still only return the same error of the original poster. Is there a way to use the API with the OAuth S2S credentials at all?

Is it currently possible to use reactor-uploader with oauth s2s credentials on any branch?

Thank you everyone for your patience. Does the issue resolve for you using<EMAIL_ADDRESS>(@adobe/reactor-uploader@next)?

Thank you everyone for your patience. Does the issue resolve for you using<EMAIL_ADDRESS>(@adobe/reactor-uploader@next)?

Yes, after redownloading @adobe/reactor-uploader@next I was able to upload a package without any issues.

closed by #66


TS type definitions

It'd be nice to have typescript definitions for lightstep tracer, since it extends what's in opentracing spec.
Would this repo be a good place to host such typedefs?

@DeTeam , admittedly I'm not a TypeScript expert but if it's conventional to host typedefs along with the JS code, I'd be glad to accept the addition!

@bcronin a nice way to get correct type is to have the project in typescript. 😄

Here are some definitions I wrote for a project (some options are missing):
declare module 'lightstep-tracer' {
    import * as opentracing from 'opentracing'

    export interface TracerOptions {
        /** the project access token */
        access_token: string

        /** the string identifier for the application, service, or process */
        component_name: string

        /**
         * controls the level of logging to the console
         *
         * - 0 - the client library will never log to the console
         * - 1 - error reporting will be throttled to the first error per minute
         * - 2 - all errors are logged to the console
         * - 3 - all errors, warnings, and info statements are logged to the console
         * - 4 - all log statements, including debugging details
         *
         * @default 1
         */
        verbosity?: number

        /** custom collector hostname */
        collector_host?: string

        /** custom collector port */

        collector_port?: number
        /** custom collector base path (if served behind a reverse proxy) */
        collector_path?: string
    }

    export class Tracer extends opentracing.Tracer {
        constructor(options: TracerOptions)

        public startSpan(name: string, options?: opentracing.SpanOptions): Span
    }

    class Span extends opentracing.Span {
        public generateTraceURL(): string
    }
}

@mlavina you're welcome to take my code above and do that :)

Since the above MR is merged, can this issue be closed?


Initial ctc

By running example.sh, I was able to train a very small CTC model on CPU.
environment.yml contains the conda environment I've been using.

To clarify, this project has no pytorch dependency. I just messed up.