[FEATURE] Support v2 volume without dedicated CPU cores

Is your feature request related to a problem? Please describe (👍 if you like this request)
Currently, each instance-manager pod needs a dedicated CPU core for supporting v2 volumes. It makes v2 volume feature is unable to run in a computing resource limited environment.
SPDK supports scheduler feature using lightweight threads for avoiding the high CPU consumption. Even though the feature is still an experimental feature, it is still worth trying and integrating it into v2 volume.
Describe the solution you'd like

Describe alternatives you've considered

Additional context

cc @DamiaSan

Assigned to @DamiaSan first, but loop @PhanLe1010 for the info.

More info about the dynamic scheduler https://spdk.io/doc/scheduler.html#:~:text=thread distribution configuration.-,dynamic,values of scheduler parameters can be displayed by using framework_get_scheduler RPC.

They are also working on interrupt mode, in next release nvmf will handle it: https://spdk.io/doc/changelog.html

Hi @derekbit , just a question that came out from the Europe Longhorn/Harvester sync-up: why don't we continue to use the v1 engine for resource limited environment? Because v2 engine has the hugepage requirement too, so even if we will be able to reduce the CPU usage, the memory will continue to be an heavy requirement. Maybe in the future, also if we wiil have the v2 as the default engine, the v1 could continue to be the best option for low resource environment.
cc @innobead @PhanLe1010

I have made some test with dynamic scheduler of SPDK: if I execute nvmf_tgt -m [0,1,2], which instructs SPDK to use 3 CPU cores, we see a %CPU of 300 in top assigned to the process reactor_0, which is the spdk tgt.  Using spdk_top, we see 3 nvme threads, each one on a different core, which continuously poll for data.
Now, if we execute the RPC framework_set_scheduler with dynamic parameter,  with spdk_top we can observe:

nvmf threads on core 1 and 2 are moved to core 0
core 1 and 2 enter in interrupt mode
nvmf threads reduce a lot their activity
app thread (main thread) reduce a bit its activity
core 0 reduce a bit its load

With top  we see a %CPU of 100 assigned to the process reactor_0.
So with dynamic scheduler we can reduce the global amount of CPU used by SPDK, but one core continue to be at 100%, because app_thread continue to work in polling mode (it reduces its activity, but the difference seems to be not so significant).

I have made some test with dynamic scheduler of SPDK: if I execute nvmf_tgt -m [0,1,2], which instructs SPDK to use 3 CPU cores, we see a %CPU of 300 in top assigned to the process reactor_0, which is the spdk tgt. Using spdk_top, we see 3 nvme threads, each one on a different core, which continuously poll for data. Now, if we execute the RPC framework_set_scheduler with dynamic parameter, with spdk_top we can observe:

nvmf threads on core 1 and 2 are moved to core 0
core 1 and 2 enter in interrupt mode
nvmf threads reduce a lot their activity
app thread (main thread) reduce a bit its activity
core 0 reduce a bit its load

With top we see a %CPU of 100 assigned to the process reactor_0.
So with dynamic scheduler we can reduce the global amount of CPU used by SPDK, but one core continue to be at 100%, because app_thread continue to work in polling mode (it reduces its activity, but the difference seems to be not so significant).
I suggest to wait for 24.09 release, where they added support for interrupt mode in the NVMe-of TCP transport, to see if something change.

Thanks @DamiaSan for the new data. I am starting learning up with v2 engine so may be it will take sometime for me to understand what you are referring to here. Looking forward to learning from you!

Hi @derekbit , just a question that came out from the Europe Longhorn/Harvester sync-up: why don't we continue to use the v1 engine for resource limited environment? Because v2 engine has the hugepage requirement too, so even if we will be able to reduce the CPU usage, the memory will continue to be an heavy requirement. Maybe in the future, also if we wiil have the v2 as the default engine, the v1 could continue to be the best option for low resource environment. cc @innobead @PhanLe1010

Ah. I see. Yeah, indeed beside of the CPU consumption of v2 engine, there is a also RAM consumption requirement... cc @innobead @derekbit @shuo-wu What is your thought?
That being said, I think v1 engine will continue to stay in the long future though so users should not panic.

The following is my current understanding (WIP)
Static Scheduler
Longhorn SPDK Branch (longhorn/spdk/longhorn-24.05)
The static scheduler does not implement any thread-to-core adjustments or frequency management.
SPDK Upstream (Master Branch)
In the upstream master branch, balance_static restores the initial state of thread-to-core assignments.
The function iterates over cores and threads to reassign each thread to its original core. It then disables further adjustments by setting the scheduler period to 0, turning off any dynamic scheduling attempts.
Dynamic Scheduler
The dynamic scheduler balances threads across cores in real time, aiming to optimize CPU utilization based on load. It involves:

Moves idle threads to the main core.
Balance active threads across available cores.
Switches between interrupt and polled mode for cores depending on activity.
Adjusts main core frequency up or down depending on workload demands.

With Kubernetes
CPU Limits
Kubernetes enforces CPU limits for each pod, limiting the CPU resources available to static and dynamic schedulers. If the pod is moved to another node (not the case for Longhorn) or if the cores are overutilized or underutilized, the static scheduler will not adapt to the new conditions. This can result in inefficient resource utilization since thread assignments are fixed post-initialization. In contrast, the dynamic scheduler adjusts its thread-to-core assignments based on the real-time load and vCPU utilization.
CPU Access
In a Kubernetes pod, SPDK's access to CPU cores is limited to the resource allocated by Kubernetes:

Access to virtualized CPUs:

When SPDK runs inside a Kubernetes pod, it does not have direct access to the host's physical CPUs. Instead, it operates within the limits set by Kubernetes for the pod. The SPDK scheduler can only distribute and manage threads across the vCPUs that Kubernetes allocated to the pod.

Impact on SPDK scheduling:

Static Scheduler: The static scheduler will assign threads to the virtual cores provided by Kubernetes, and these assignments remain fixed.
Dynamic Scheduler: The dynamic scheduler attempts to monitor workload and adjust thread-to-core assignments dynamically. However, its effectiveness is limited because it can only see and manage the virtual cores that Kubernetes provides, not the actual host cores.

Summary
The primary difference between the static and dynamic schedulers lies in how they manage thread distribution:

Dynamic Scheduler: Continuously balance threads across cores. Where resources are shared by other workloads, It adapts to CPU load within Kubernetes limits, providing better resource utilization when the workload fluctuates.
Static Scheduler: Assigns threads to specific cores initially and does not adjust dynamically.

In terms of overall CPU consumption, both schedulers can have comparable efficiency:

gthread and Lightweight Thread Creation: Both schedulers use the same approach for thread-to-core assignment.
Efficiency Comparison: Both schedulers aim to optimize the usage of available CPU cores, but the static scheduler does so by fixing thread assignments, while the dynamic scheduler continually evaluates and adjusts thread distribution.
When Efficiency is Comparable: In steady-state workloads, where the load is predictable or changes infrequently, the dynamic scheduler may not provide additional benefits, and both schedulers will consume CPU resources similarly, The static scheduler will simply assign thread and allow them to run without further adjustments.
When Efficiency Differs: The dynamic scheduler is more efficient in scenarios where workloads fluctuate or loads are unpredictable. The dynamic adjustments prevent inefficiencies such as idle threads or overloaded cores. In contrast, the static scheduler may be less efficient in these scenarios, but in steady conditions, both schedulers will perform similarly in terms of CPU consumption.

In a Kubernetes environment, both schedulers are subject to the CPU constraints and sharing policies imposed by the Kubernetes scheduler. When pod resources are tightly constrained, the dynamic scheduler's ability to balance load across cores becomes less relevant, and both schedulers may end up consuming similar CPU resources since the Kubernetes scheduler will enforce limits on CPU resources anyway.
Reference

https://github.com/spdk/spdk/blob/master/lib/event/scheduler_static.c#L45-L68
https://github.com/longhorn/spdk/blob/longhorn-v24.05/module/scheduler/dynamic/scheduler_dynamic.c#L283-L351

Why top shows 100% of CPU

SPDK uses a polled-mode design; a reactor thread running on each CPU core assigned to an SPDK application schedules SPDK lightweight threads and pollers to run on the CPU core. Therefore, the standard Linux top utility is not effective for analyzing the CPU usage for polled-mode applications like SPDK because it just reports that they are using 100% of the CPU resources assigned to them.

https://spdk.io/doc/spdk_top.html

I suggest to wait for 24.09 release, where they added support for interrupt mode in the NVMe-of TCP transport, to see if something change.

https://github.com/longhorn/longhorn/issues/9700

A clarification about dedicated CPU cores: SPDK doesn't need any "real" dedicated CPU cores, the locks that we see here are used to make dedicated cpu cores between multiple SPDK applications, not to prevent cpu usage by other kind of processes. If you run multiple spdk tgt on the same host, you must give them not overlapping cpu masks.
However, it is recommended to run SPDK target on a dedicated CPU set, as we can read here, but it is not mandatory.

Dynamic Scheduler: The dynamic scheduler continuously balances threads based on the workload's needs, but its ability to optimize across multiple workloads within Kubernetes is limited. It adapts to changes in CPU load within the constraints of the pod's allocated resources. This means in the multi-core situation there will be at least one single dedicated core. If idle threads are moved back to the main core, the other core is freed up.

Have you made some tests to confirm this limited ability?
What about the dedicated core, do you mean that also with dynamic scheduler at least one core is always at 100% right?
In addition to @c3y1huang considerations, we can say that dynamic scheduler, due to its ability to reduce CPU usage, will be less responsive to a fast workload growth because it could have to restart the polling mode in some cores and move on them some threads.

A clarification about dedicated CPU cores: SPDK doesn't need any "real" dedicated CPU cores, the locks that we see here are used to make dedicated cpu cores between multiple SPDK applications, not to prevent cpu usage by other kind of processes. If you run multiple spdk tgt on the same host, you must give them not overlapping cpu masks. However, it is recommended to run SPDK target on a dedicated CPU set, as we can read here, but it is not mandatory.

Thank you, after reading the code again, I see you are right - it's not mandatory to dedicate CPU cores. However, my understanding is that each CPU core SPDK uses is assigned to a reactor thread. This reactor thread manages and runs all the lightweight threads and pollers. Since the reactor thread continuously polls, the CPU core is primarily dedicated to SPDK tasks, leaving little room for other workloads or non-SPDK threads. In other words, the CPU core doesn't enter an idle state, nor is it "handed off" to other processes. As a result, any other workload assigned to the same core would likely experience CPU starvation, as the SPDK polling loop doesn't yield CPU time to non-SPDK processes. Do you think this understanding is accurate?

Dynamic Scheduler: The dynamic scheduler continuously balances threads based on the workload's needs, but its ability to optimize across multiple workloads within Kubernetes is limited. It adapts to changes in CPU load within the constraints of the pod's allocated resources. This means in the multi-core situation there will be at least one single dedicated core. If idle threads are moved back to the main core, the other core is freed up.

Have you made some tests to confirm this limited ability? What about the dedicated core, do you mean that also with dynamic scheduler at least one core is always at 100% right?

Based on my understanding from the SPDK documentation and a bit of reading through the SPDK code, the reactor thread schedules lightweight threads and pollers to run on the assigned CPU core. With the dynamic scheduler, the reactor thread will either poll on the main core or distribute the workload across other SPDK cores. This suggests that at least one core will consistently operate at 100% CPU usage. Does this align with the behavior observed in your tests?

In other words, the CPU core doesn't enter an idle state, nor is it "handed off" to other processes. As a result, any other workload assigned to the same core would likely experience CPU starvation, as the SPDK polling loop doesn't yield CPU time to non-SPDK processes. Do you think this understanding is accurate?

I think that, if another process uses the same core as SPDK, as you say this process won't have so much CPU to perform its workloads; but also SPDK polling will result slower, because they will share the same resource. You are right saying that SPDK polling doesn't yeld CPU, but the processor must handle all the process running over it.

This suggests that at least one core will consistently operate at 100% CPU usage. Does this align with the behavior observed in your https://github.com/longhorn/longhorn/issues/7064#issuecomment-2358192754?

Yes, exactly.

@derekbit @innobead , We should reconsider proceeding with this issue because:

Both the static and dynamic schedulers will consume at least one CPU core for the reactor.
There is no hard requirement for CPU cores dedication, however, the SPDK core may still be heavily utilized by reactors, potentially leaving limited resources for other non-SPDK processes, regardless of the scheduler type.

Waiting for a decision. Putting this to the backlog to work on other issues.

After discussing with @innobead @derekbit , we need to clarify the exact behavior of the reactor and poller when a core is set to interrupt_mode. I will work on this after hack week.

Dynamic Scheduler

Gather Core Info

The balance() function iterates through each SPDK core and updates the g_core structure with the current stats, which are stored in cores_info.

Initialize Main Core
After gathering core information, the main_core is set based on g_main_lcore.
Balance Threads

Check Thread Load (_get_thread_load(thread_info))

Retrieve Busy and Idle Time:

busy = thread_info->current_stats.busy_tsc - The time the thread has been active (busy).
idle = thread_info->current_stats.idle_tsc - The time the thread has been idle.

Calculate Busy Percentage (_busy_pct(busy, idle)):

Handle Zero Total Time:

If (busy + idle) == 0, the thread hasn’t been active at all, so the load is 0%.

Calculate the Busy Percentage:

If both busy and idle time are non-zero, the busy percentage is calculated as busy * 100 / (busy + idle).

Balance Idle Threads (_balance_idle())

Determine Load: If a thread's load is below the threadshold defined by g_scheduler_load_limit (default 20), the thread is considered idle.
Move Idle Threads: Idle threads are moved to the main core (g_main_lcore).

Balance Active Thread (_balance_active())

Determin Load: If the load is greater than or equal to the global load limit (g_scheduler_load_limit), the thread is considered active and requires balancing.
Find Optimal Core: The function _find_optimal_core(thread_info) selects the core with the least busy time, as indicated by g_cores[i].busy, to host the active thread.
Move Active Threads: The active thread is moved to the optimal core.

Move Thread (_move_thread())

Update Destination Core:

Increase the busy time (busy_tsc).
Decrease idle time.
Increase the thread count.

Update Source Core:

Decrease the busy time.
Increase the idle time.
Decrease the thread count.

Update the Thread's Logical Core: The logical core assignment is updated with thread_info->lcore = dst_core, reflecting the new core.

Switch Core Modes

After balancing threads, each core is assessed to determine whether it should switch its mode.

Idle Core Mode:

If a core has no threads (g_cores[i].thread_count == 0) and there are no threads in the reactor queue (TAILQ_EMPTY(&reactor->threads)), the core switches to interrupt mode:

Set core->interrupt_mode = true.
Reduce the core's frequency to the minimum using governor->set_core_freq_min(core).

Active Core Mode:

If the core has threads and is not the main core, the core switches to polled mode:

Set core->interrupt_mode = false
Set the core frequency to the maximum with governor->set_core_freq_max(core).

Mode Handling In Reactor Loop:

Interrupt Mode: If core->interrupt_mode = true, the reactor's loop invokes reactor_interrupt_run() instead of _reactor_run(). This mode waits for activity on file descriptors (external notifications).
Polled Mode: If the core is in poll mode, the loop invokes _reactor_run(). This function processed events in the reactor's queue:

For each thread, spdk_thread_poll is called to process events.
If th thread is in interrupt_mode, it checks for file descriptor events but returns immediately if no events are pending.
Message in the queue are processes (msg_queue_run_batch()).
Active pollers are executed using thread_execute_poller() where the callback function (poller->fn(poller->arg)) is invoked.
Timed pollers are executed using thread_execute_timed_poller().
After processing, the thread context is restored.

Adjust Main Core Frequency

If there are busy threads on the main core, the frequency is set to maximum (governor->set_core_freq_max(g_main_lcore)).
If the main core is more busy than idle, its frequency is increased (governor->core_freq_up(g_main_lcore))
If the main core is more idle than busy, its frequency is decreased (governor->core_freq_down(g_main_lcore))

Additionally, The app-thread runs on the main core to handle tasks and subsystem messages. This means the scheduler is not responsible for switching the main core to interrupt mode. The app-thread is also initialized with at least one poller to monitor incoming RPCs (SPDK_POLLER_REGISTER(rpc_subsystem_poll_servers, NULL, RPC_SELECT_INTERVAL), which means it maintains a persistent poller regardless of the scheduler type.
Based on these findings, I believe the scheduler type is less relevant to this issue. We should investigate longhorn/longhorn#9834 to see if that is more relevant.

Waiting for further instruction. I'm moving the state to icebox to focus on other issues.


Add option to disable automatic migrations

Some people would prefer a CLI command to migrate the database, as opposed to the migration occurring on file save.

Okay, so my current plan or this feature:

Add migration.disallowDevHazards array option, which is like migration.allowHazards but only active when working with a local Postgres instance. Also add a --no-hazards option to pg-nano dev, which disallows all hazards for that session.
When a migration fails due to a hazard, allow the developer to press Enter to tell pg-nano dev to try again with all hazards allowed.


Visible cells nice to have

It would be convenient for a public method on gridView that returns the currently visible cells. Right now I have resorted to making the itemSubviews method public instead of private. Anyone concur with me on this??

You can also get the visible cells like this:
CGPoint contentOffset = ((UIScrollView*)[[gridView subviews]objectAtIndex:0]).contentOffset;
for (GMGridViewCell *view in [[[gridView subviews]objectAtIndex:0] subviews])

{
...
}
But everytime you enter sorting mode the cells order seems to change in the subviews array. So this is not best solution. I'll try to make itemSubviews public and see if its better for my purpose.
Maybe I'll create a getCellsInRange Method or something like that, where you can also get a smaller amount of cells than the visible ones. Could be useful for some fancy animations. :)


Filter.Term vs Filter.Terms parameter types

The api is inconsistent in between Term and Terms.
For example I can do this
Filter<T>.Term("age", 21);

but not
Filter<T>.Terms("age", new int[] { 21, 22 });

Please note that this is an example of the inconsistencies...

Agreed, it should be Terms<K>
We have a few other inconsistencies (https://github.com/elastic/elasticsearch-net/issues/1549) that we're aware of- this is something we're definitely going to address for 2.0 alpha.
Thanks for raising this.

See last attached commit


List index out of range after loading server list

Arch x86_64
discurses from pip
I'm in 27 servers if that makes a difference.
Traceback (most recent call last):        │+ /r/Android: 17                                                                                │
  File "/usr/lib/python3.6/site-packages/discord/client.py", line 307, in _run_event                                                                                │
    yield from getattr(self, event)(*args, **kwargs)lehome: 4                                                                                │
  File "/usr/lib/python3.6/site-packages/discurses/discord.py", line 36, in eh                                                                                │
    f(*args, **kwargs)                    │+ Discord Electronics Server: 16                                                                                │
  File "/usr/lib/python3.6/site-packages/discurses/ui/message_list.py", line 42, in _on_message                                                                          │
    if message.timestamp.date() != self.list_walker[-1].message.timestamp.date():                                                                                │
IndexError: list index out of range       │+ Facepunch CD/TG/A: 21

Sorry, but I dont really have time to update this project anymore. If someone wants to take over, that would be awesome!

I have submitted a PR. This is a dirty hack but I guess that it temporarily solves the issue.

#24 merged, should close this


creation of project structure

I created the project folder to help guide contributors

GitHub doesn't recognize empty folders and .gitkeep are just placeholders to help keep them, we will delete them soon


Cryo 73

Have updated the Customize and Access Data.ipynb to include ATL10 and MOD29 and removed the Access All Data.ipynb. The Customize and Access Data-Rendered.ipynb has not been modified as I wanted to make sure the changes I made to the notebook were ok first, and I also don't know how to create a Rendered version (yet).

@jroebuck932 This notebooks looks great! I added two minor comments in ReviewNB. I ran everything end to end including the Visualize and Analyze notebook and it all ran successfully.
As far as the Customize and Access Data-Rendered.ipynb: We provide the notebook outputs in these rendered versions so you could look at the results fully without needing to run the notebook. So, here's what you can do next:

Update the (non-rendered) notebook with those small changes I mentioned in the review comments.
Clear the outputs and save the notebook
Delete the existing Customize and Access Data-Rendered.ipynb notebook
Commit and your changes (git rm the Rendered notebook and git add your updates to the non-rendered)
Copy the non-rendered notebook and call it Customize and Access Data-Rendered.ipynb
Open up Rendered and run it end to end
Clear only the outputs on the authentication step to protect privacy with your EDL credentials (even though it doesn't spit out the password, it still shows the character length, your UID, email, etc)
Save as-is so that all of the other outputs are there
Commit those changes
Push everything!

Then we should be all good to go to merge!

@asteiker thanks for the review. I have made all the suggested edits and added the rendered version of the notebook.


Can Pointer.type be typed as a string union?

It breaks modularity a bit, but it could be nice if event.pointer.type wasn't of type string, but instead `pdom'|'mouse'|'touch' etc. @jonathanolson could we have this type directly in Pointer?
The type checker is passing, so I thought I'd commit as a proposal.
I ran into this over in https://github.com/phetsims/molarity/issues/245 where the pointer type was changed for PDOMPointer, but I missed a couple of cases.  This probably won't happen again, but it may be nice to improve the typing here, so long as we don't expect users of scenery to add in their own Pointers ad hoc.

This looks great, thanks!


Make install an optional dependency and add 32-bit arm targets

Resolves #1284
Believe it or not, some Linux embedded device will not have install installed. This is not uncommon when they have a custom distribution build with Buildroot or Yocto, such devices typically have cp and chmod though, so when install is not available, there's now a fall back to cp and chmod.
Then I added 32-bit arm targets, that includes old Raspberry Pi models but also various embedded linux devices such as Xilinx Zynq family.

It is tested on a Zynq-7000 which has this output for uname -a:
Linux hostname 5.15.19-xilinx-v2022.1 #1 SMP PREEMPT Wed Aug 24 06:33:16 UTC 2022 armv7l GNU/Linux

This device does not have install.

Now also tested on Raspberry PI 0 which has this uname -a output
Linux hostname 6.1.21+ #1642 Mon Apr  3 17:19:14 BST 2023 armv6l GNU/Linux

Requested changes implemented


Updated API definitions for certification 2.0

A new pull request has been opened to add connectors to certified-connectors, the previous pull request has been canceled

Closing at request of partner. @NicolasValenzuela Please re-open/start a new PR when ready.


Add pytorch GPU image for inference

What this PR does / why we need it:
Add Pytorch GPU image for inference on GPUs.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #537
Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

This change is

Great you added the simpletransformers library.

Great you added the simpletransformers library.

I did !

Any indication of when we can expect a release with this change? Without GPU support we can't continue our migration to kfserving.

/retest

Looks like this fails the pytorch e2e test.

/retest

/retest

/retest

Looks like this fails the pytorch e2e test.

@ellis-bigelow this is good to go we have gpu e2e test now! it was getting OOM :)

Awesome thank you!

/lgtm
/approve

/retest

Thank you for taking care of this! We are very happy to see this moving forward. Can't wait to give it a try.


some new tutorials to go with the anaconda jupyter image

/PTAL @jeff-phillips-18 . Thanks for the update @AlbertDeFusco


ṕ does not exist as a quick.

In this image from the readme:

There's a quick called prior, which is translated as ṕ. However, there's no ṕ documented in the list of quicks, nor is there a ṕ character in the codepage.
By prior, did you mean one of the grouping quicks? Those being ¢Çç for last link as nilad/moand/dyad, $¥ for last two links as a monad/dyad, Ɗɗ for last three links as a monad/dyad and Ʋʋ for last four links as a monad/dyad.
Other than that, I'm not quite sure what prior is supposed to mean. If it's not being converted to something else before executing Jelly, it'll be ignored, making no difference to the way the program is executed.

Also, after reading through tokens.py, I noticed there were a few things you marked as don't know what this does.

Ṿ is like how python's repr(x) returns a string that, when evaluated using eval, should return a value the same (if not, close to) x. In this case, the string representation is returned as Jelly syntax. Here's the implementation of how different types are converted
Ẏ flattens a list of nested lists by 1 level; where F completely flattens the structure of a list to be 1-dimensional, Ẏ "un-nests" each item in a list once. Try it online! - a layer of brackets is removed from each item.
G (Attempt to format z as a grid) is a built-in that essentially joins all sublists on spaces and that on newlines. This is mostly for code golf challenges with strict output requirements. The best example I could find from the Code Golf Stack Exchange was this answer by Dennis. Without the G at the end, the output is a list of lists (Try it online!), but with the G, it's displayed as an evenly spaced grid (Try it online!). Outside of golf, you can pretty much ignore it.
k is a little hard to explain, but one way to potentially think about it is as so:

k takes two arguments: a normal list, and a shape list. The shape list typically consists of 1s and 0s, but you only need to consider the truthiness and falsiness of each value.
The shape list has all consecutive 0s grouped, with the first 1 that follows included in that group. For example:
[1,0,0,0,1,1,0,0,1,1]

would be grouped as
[1, [0,0,0,1], 1, [0,0,1], 1]

The normal list is then molded to the shape of the grouped shape list. Now due to internal implementations, if there's a 1 at the end of the shape list, an empty list will be appended to the result. I'm not quite sure why it does that, and if there's one thing rewriting the Jelly regex parser in a regexless way has taught me, it's that Jelly, while good as a language, really could have more code documentation :p. The partition_at function is one of those functions that doesn't really tell you why it does things. It's implementation can be found here

Anyhow, that seems to be all the atoms you marked with the don't understand comment. Hopefully that helps clarify things if someone hasn't already beaten me to the punch elsewhere.

@lyxal It's defined in their Jellyfish fork; if I'm reading it correctly it appears to be a dyad-only version of Ɲ. I would nevertheless recommend not using it to show off something you advertise as a tool for base Jelly.
Also, this is just a personal this-is-bugging-me thing, but would it be too late to rename R from iota to range? J corresponds to the conventional APL iota, and it's not like you're using range for r (for which I might recommend something like range2 or ..=).

@lyxal It's defined in their Jellyfish fork; if I'm reading it correctly it appears to be a dyad-only version of Ɲ. I would nevertheless recommend not using it to show off something you advertise as a tool for base Jelly.

@lyxal What @UnrelatedString said here ^^ is correct. I have updated the README to say that Jello wraps the Jellyfish executable which is a fork of Jelly.

Also, this is just a personal this-is-bugging-me thing, but would it be too late to rename R from iota to range? J corresponds to the conventional APL iota, and it's not like you're using range for r (for which I might recommend something like range2 or ..=).

I have added overloads for range and range0 (the corresponding names for iota and iota0)

Oh, and if you're wondering about when ṾV isn't quite the identity function, it isn't equipped to handle edge cases that break Jelly's type system like ⁾0Ƭ+/ṾV.


Drag with offsetY works incorrectly in Firefox

Are you requesting a feature or reporting a bug?
bug
What is the current behavior?
drag action with offsetY works incorrectly in Firefox
What is the expected behavior?
drag should work as expected
Provide the test code and the tested page URL (if applicable)
Tested page URL:

Test code
test('test', async t => {
    await t
        .setTestSpeed(0.01)
        .drag(div, 300, 300)
});

Looks like I've missed DOCTYPE which is required according to the https://developer.mozilla.org/en-US/docs/Glossary/Doctype

In HTML, the doctype is the required "" preamble found at the top of all documents. Its sole purpose is to prevent a browser from switching into so-called “quirks mode” when rendering a document; that is, the "" doctype ensures that the browser makes a best-effort attempt at following the relevant specifications, rather than using a different rendering mode that is incompatible with some specifications.


Broken link for the F&O BI blog

the link under getting started, "Finance and Operations: Business intelligence (blog)" Leads to a "page not found" page

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: adf5c448-2cd0-1538-d380-4af101563e90
Version Independent ID: e6497148-5f30-b30a-98ca-853181932447
Content: Business intelligence (BI) and reporting home page - Finance & Operations | Dynamics 365 | #MSDyn365FO
Content Source: articles/dev-itpro/analytics/bi-reporting-home-page.md
Service: dynamics-ax-platform
GitHub Login: @sericks007
Microsoft Alias: sericks

@maveloso, thanks for making us aware of this issue!  I've fixed the link, and the updated topic will be published later today.
In the meantime, you can find the blog here: https://community.dynamics.com/365/financeandoperations/b/powerbianalyticsandreporting


update regions for variability

typos in PR merged by @gleckler1 please fix

typo corrected, sorry! @gleckler1 @doutriaux1

@gleckler1 wanna assign a milestone here? @lee1043 is there an open/merged PR that goes with this issue - if so should it be closed?

@lee1043 @gleckler1 can we close now that typo is fixed?

@doutriaux1 Yes we can!


hardware support: blink(1) by ThingM

add support for blink(1). it also uses usb-hid and is open-source.
help needed: I don't own this device, but seems it would be easy enough to add support for it.

not planned. next version will use BLE beacons


[FEATURE] Remove recursive downloading from Pipline Runs

There are endpoints for downloading CSVs and downloading media so I think that recursiveDownloading of the folder and filtering afterwards no longer needs to be used.  This should simplify some of the logic within these tasks.

@BryonLewis is this issue still relevent?
In run_pipeline I see:
    # Download source media
    input_folder: GirderModel = gc.getFolder(input_folder_id)
    input_media_list = download_source_media(gc, input_folder_id, input_path)

I only see downloadFolderRecursive used to fetch the folder that contains model(s) if the pipeline run is for a trained model.

I think this can be closed now because of the latest update.


stack overflow in refreshAll(E) fixed (recursive call) + warning fixed

invoking refreshAll on KotlinEntityDataStore caused infinite recursion

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Rafał Ciurkot seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.

Great thanks! looks like you just need to associate the committers account with your github to fix the cla warning, then will merge

@npurushe I've just fixed the CLA warnign

Great thanks!


fix(storage-resize-images): update resized metadata content disposition

fixes: https://github.com/firebase/extensions/issues/807

Codecov Report

Merging #839 (99c4806) into next (3b1cf99) will decrease coverage by 0.04%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             next     #839      +/-   ##
==========================================
- Coverage   50.19%   50.16%   -0.03%
==========================================
  Files          30       30
  Lines        1329     1330       +1
  Branches      197      198       +1
==========================================
  Hits          667      667
- Misses        661      662       +1
  Partials        1        1

Impacted Files
Coverage Δ

...torage-resize-images/functions/src/resize-image.ts
38.89% <0.00%> (-0.54%)
:arrow_down:

Testing
Image Resizing

Updated Content Disposition


daxm PR

Added ZDX Portal Talker
Added my logger and helper methods (goes along with ZDX Portal Talker)
Worked on init to advertise methods/classes that should be publicly accessible.
Created pip_version.Dockefile to provide method to use pip version in Docker.
Moved working python code into sub-dir to isolate from other repo uses.
Built working Poetry build structure (renamed previous files to *.old).

@daxm  Please update changelog.


How to override and customize styles with sass

I want to know if its possible to add sass to the project

@sizmamani - Currently not.

This is possible by doing the following:

Install the following packages:
npm install sass-loader node-sass --save-dev

Set the lang property of the script tag

Thanks @ryanmstokes


Update dataset.py

"--query_prefix" and "--passage prefix" already contain a space. So we should remove the space between prefix and query/doc during preprocessing.

thanks for the edits!


Warning 1205 missing

I'm getting warning 1205 "Section [Driver_files] referenced from DelFiles and CopyFiles directive". Why isn't it listed here? Why is this a warning?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 48275f87-4086-1ca5-a190-c741f29ad348
Version Independent ID: 5481885d-3063-527b-a85e-6a898dbcb990
Content: INF Validation Errors and Warnings - Windows drivers
Content Source: windows-driver-docs-pr/devtest/inf-validation-errors-and-warnings.md
Product: windows-hardware
GitHub Login: @EliotSeattle
Microsoft Alias: eliotgra

Thanks for pointing this out, @BrianCatlin. I've added the following to the table:
1205: Section [Driver_files] referenced from [Directive1] and [Directive2] directive
This warning is generated whenever two different directives point to the same section.
I just created the pull request, so it should be live shortly.

You should point out that multiple directives pointing to the same section is frequently an error, but that is not always the case (false positive)


Run nixosTests on darwin

Description of changes
Towards #108984
I had to add lazyFunction in order to half-support make-test-python.nix calls which don't pass hostPkgs along.
All tests should be migrated to use the simpler runTest IMO.
Draft because this needs testing and validation, but I don't have much time to spend on this. Figured I'd dump my knowledge in the form of a draft PR (and because I've changed things #191540).
TODO

[ ] Eval on hydra against eval regressions

[ ] in existing nixos.test.*
[ ] future breakage in the macos tests (but only nixos.tests.simple and test-framework testing tests?)

[ ] Docs
[ ] Add tests to (TBD) nixosTests.nixos-test-runner suite (categorize tests with recurseIntoAttrs)
[ ] Consider moving more of the logic into nixos/lib/testing. The nixosTests system logic is also useful there.
[ ] Get #194514 done and rebase

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

I've removed the cross compilation changes, which I wasn't confident about. However I did leave the preliminary support for running NixOS tests using cross-compiled pkgs in place.

I'm on M1 Mac and QEMU command line currently doesn't support running non-native architecture (needs some tweaks, if that's possible). Would it be possible to warn user about this somehow?

I don't know about what can be tweaked and I have no ambition to research this (sorry). A warning could be added if we're confident that this will never be supported, but until we're confident I don't want to discourage anyone from picking up where I/we left off.

Would it be possible to add some of these tests on Hydra, so that we can be sure that this will not get broken in the future?

I would think so and this is on the todo list. :+1:

Is it possible to use nixosTest with this branch?  I tried the following:
{ inputs.nixpkgs.url =
    "github:NixOS/nixpkgs/e39a5efc4504099194032dfabdf60a0c4c78f181";

  outputs = { nixpkgs, ... }: {
    checks.aarch64-darwin.default =
      nixpkgs.legacyPackages.aarch64-darwin.nixosTest {
        name = "test";

        nodes.machine.virtualisation.host.pkgs =
          nixpkgs.legacyPackages.aarch64-darwin;

        testScript = "";
      };
  };
}

… but that fails with:
error: assertion '(stdenv).isLinux' failed

       at /nix/store/3zj9cv9pj7q2y4g0q1bfxzmv4wq2i4zj-source/pkgs/os-specific/linux/kernel/generic.nix:71:1:

           70|
           71| assert stdenv.isLinux;
             | ^
           72|
(use '--show-trace' to show detailed location information)

Is it possible to use nixosTest with this branch?

It doesn't have a good reason not to, but the immediate problem is that doesn't let you set anything related to either pkgs or hostPkgs.
It should probably the logic I've highlighted in my previous comment in one way or another.
In your example, perhaps you might get away with setting nodes.machine.nixpkgs.pkgs to a linux one by hand?
Really it should be automated though, like nixosTests plural.

That works!  The following modified version built for me:
{ inputs.nixpkgs.url =
    "github:NixOS/nixpkgs/e39a5efc4504099194032dfabdf60a0c4c78f181";

  outputs = { nixpkgs, ... }: {
    checks.aarch64-darwin.default =
      nixpkgs.legacyPackages.aarch64-darwin.nixosTest {
        name = "test";

        nodes.machine = {
          nixpkgs.pkgs = nixpkgs.legacyPackages.aarch64-linux.pkgs;

          virtualisation.host.pkgs =
            nixpkgs.legacyPackages.aarch64-darwin;
        };

        testScript = "";
      };
  };
}

Oh and nixosTest should be superseded by a new function that takes a module argument instead of behaving like callPackage. Sadly those behaviors are mutually exclusive.
I did add nixpkgsFlake.lib.nixos.runTest, which takes a module, but leaves the pkgs and hostPkgs for you to set. Not exactly the answer to your question, but perhaps a solution?

@Gabriella439 I tried building your example flake on an M1 but getting the following error:
nix build .#checks.aarch64-darwin.default
error: a 'aarch64-darwin' with features {hvf, nixos-test} is required to build '/nix/store/icqc6g8lf1f1lnl1dzipfaw5rlmp4vsx-vm-test-run-test.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

(currently on macOS 12.4)
Is there any other setup necessary to get hvf working?

Oh yeah, I had to also add this to my nix.conf:
extra-system-features = hvf

Oh yeah, I had to also add this to my nix.conf:
extra-system-features = hvf

Ah thanks so much! I did try this before, but the option was being ignored when put in ~/.config/nix/nix.conf. After I moved it to /etc/nix/nix.conf, it built successfully.

Created #7619 to mirror the solution we have for kvm on linux.

Created https://github.com/NixOS/nix/pull/7619 to mirror the solution we have for kvm on linux.


add B_SEC_* tests from SBSA ACS?

SBSA ACS has tests for B_SEC_0[1-5] features. By name they belong to BSA ACS.
They are described only in BSA specification.

ok, just another argument to merge SBSA ACS to BSA ACS :D


Sm/qa pr50

What does this PR do?
replicates PR #50  from Fabien for CI reasons, plus a few tweaks
What issues does this PR fix or reference?
https://github.com/forcedotcom/cli/issues/827
@W-8825037@

@mshanemc I assume that this has gone through command review board and is approved?

@mshanemc I assume that this has gone through command review board and is approved?

it's been reviewed by Juliet, with her changes incorporated on the original PR.


[Discussion] Removal of the team leader feature

The team at Portainer have decided to depreciate the team leader feature, and propose removing it from Portainer.
Our thoughts are:

The team leader feature in it's current state is broken, meaning it can't be used to create new users.
Since the introduction of external authentication, Portainer recommends it's use for automatic account creation and the added security it provides. Thus the team leader feature would not make to use.

What are the thoughts of the community?
Is this an important feature for you in the core Portainer software?
If you are using the team leader feature, can you provide us your use cases?

Hello together,
first of all i really love portainer! It is easy to use and the ability to connect to container terminal via browser is pretty cool.
If you are using the team leader feature, can you provide us your use cases?
We uses containers to simulate "real" endpoints for our testenvironment
We are a team of 14 people divided into 8 subteams. So that we also defined 8 teams in portainer with 1 teamleader. Every subteam has a amount of specific "endpoints". In case that students should support a team (because of reasons), a teamleader can give another user the access right to use his endpoints by adding him to his team. Since the work is done the teamleader can remove the access right by removing him from the list.
We uses this function a lot.
Also, I dont see the point why a team leader should create new users? An admin can do this
best regards

I have no use case for this. I didn't know it even existed. For that reason, I support your decision toward deprecating.

We use this feature in a different way:

when people subscribe to our platform they become a team leader
we create a group/team for them, which has access to all the different endpoints
within that team, they can also create (local) users to collaborate with them

Following feedback on this issue & from elsewhere in the community, there is a definite case to keep this feature. It will need some work, so I have opened an enhancement request here #3139


2024 Crescendo VictiScout Update

Update to conform to game
- Add Type of drive
- Add Chasis Size (updateable in table view)
- Add Height (updateable in table view)
- Add Climb Method dropdown
- Add Auto Starting position dropdown
- Add Note Scoring

Add Shot Accuracy
- Add Auto Shot Accuracy field in table view
- Add Tele-Op Shot Accuracy field in table view
- Add Sort By Auto Shot Accuracy button in table view
- Add Sort By Tele-Op Shot Accuracy button in table view

Add Export to CSV button

Clean Up UI
- Hide Input file
- Hide Convert file
- Hide input file textbox
(Hides 10,000 px off the screen, breaks if deleted)
Fix #178, #170

Anything I haven't  resolved still needs to be fixed give me a bit and ill get it done :)

I'm still a little hung up on the hidden inputs and I would prefer that this question gets resolved before merging.
Do we not want to have support for merging and transferring files anymore? I would question this decision as I recall those being pretty fundamental features of the app and the primary reason for the data screen existing in the first place. But if we don't want them anymore, then we should remove them outright including the fairly substantial logic that manages them.


[Ubuntu Core] List probable tasks and estimate needed time for them.

We need to work out the future feature and release plan. So please investigate what to do and break out the work to support Ubuntu Core, list probable tasks and estimate needed time for them.

@halton please take over this issue.

The rough estimation on this task is 3-week engineering work


Parallels Coherence Windows Applications Aren't Tiled

When Parallels desktop windows coherence mode applications are opened they aren't tiled, but I'd like them to be. Here are the results of the yabai -m query --windows command for the "Messenger (Beta)" windows application.
Before running yabai -m window --toggle float:
{
	"id":5697,
	"pid":8659,
	"app":"Messenger (Beta)",
	"title":"Messenger",
	"frame":{
		"x":407.0000,
		"y":501.0000,
		"w":800.0000,
		"h":435.0000
	},
	"level":3,
	"role":"AXWindow",
	"subrole":"AXDialog",
	"movable":1,
	"resizable":1,
	"display":1,
	"space":3,
	"visible":1,
	"focused":0,
	"split":"none",
	"floating":1,
	"sticky":0,
	"minimized":0,
	"topmost":1,
	"opacity":1.0000,
	"shadow":1,
	"border":0,
	"zoom-parent":0,
	"zoom-fullscreen":0,
	"native-fullscreen":0
}

And after running yabai -m window --toggle float on the window in question:
{
	"id":5697,
	"pid":8659,
	"app":"Messenger (Beta)",
	"title":"Messenger",
	"frame":{
		"x":709.0000,
		"y":323.0000,
		"w":882.0000,
		"h":474.0000
	},
	"level":0,
	"role":"AXWindow",
	"subrole":"AXDialog",
	"movable":1,
	"resizable":1,
	"display":1,
	"space":3,
	"visible":1,
	"focused":0,
	"split":"none",
	"floating":0,
	"sticky":0,
	"minimized":0,
	"topmost":0,
	"opacity":1.0000,
	"shadow":1,
	"border":0,
	"zoom-parent":0,
	"zoom-fullscreen":0,
	"native-fullscreen":0
}

The window was also moved a bit between those two tests.
Let me know if you'd like me to do any more testing to help figure out why yabai isn't finding these windows, I suspect there's something non-standard about the way Parallels passes application windows through to MacOS, but I'm not sure how to go about doing more testing.
Edit: here's a link to an explanation of what coherence mode is: https://kb.parallels.com/en/4670

The window is not tiled because it has subrole AXDialog and not AXStandardWindow.
You can make this specific window automatically tile by using window rules: https://github.com/koekeishiya/yabai/blob/master/doc/yabai.asciidoc#rule
Add something like this to your config file: yabai -m rule --add app="^Messenger (Beta)$" title="^Messenger$" manage=on

Ok thanks!

Quick update, for those that also run into this, I've added this line to my yabai config: find ~/Library/Parallels/Applications\ Menus/ -maxdepth 3 -type f | awk -F'/' '{ print $NF; }' | awk '{$1=$1};1' | tr "\n" "\0" | xargs -0 -I{} yabai -m rule --add app="^{}\$" title=".*" manage=on
Basically, it recursively lists all the files inside your parallels coherence app shortcuts folder, then creates a rules that enable management for any windows that have the same app name as one of those shortcuts. It's definitely not perfect, I've just started using it, but let me know if you have any suggested improvements let me know.


[CARBONDATA-3756] Fix stage query bug it only read the first blocklet of each carbondata file

Why is this PR needed?
The query of stage files only read the first blocklet of each carbondata file.
So when the file contains multiple blocklets, the query result will be wrong.
What changes were proposed in this PR?
The query of stage files should read the all blocklets of all carbondata files.
Does this PR introduce any user interface change?

No

Is any new testcase added?

No, who tell me how to add a test case.

Build Success with Spark 2.4.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbon_PR_Builder_2.4.5/863/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/2571/

LGTM


[Suggestion] Do not set stroke color on path

As per the Contributing Guidelines of SimpleIcons, SVGs should never rely on a stroke. Hence, I think we should consider not settings the stroke color on the <path> of the icon.

@ericcornelissen Thanks for your contribution!


How to deploy this tool in Cloud foundry

I am exploring the option to run the tool in cloud foundry as I have no VMs. PLease help.

Please refer to the documentation on the README of the project. This tool is just a plain old cli tool. You can run it on any computer, with appropriate configuration for your specific use case.


Project status

Hey maintainers,
This looks like a good project with great potential, but there hasn't been a commit for a while on the master branch. Do you have any further development  planned or should we consider this project inactive?

Inactive though I may be updating it in bulk at some point in the future. It's sort of an open project though. I'd endorse anyone taking the reigns for improvements.

On Dec 19, 2015, at 10:55 AM, Pierre Lebrun<EMAIL_ADDRESS>wrote:
Hey maintainers,
This looks like a good project with great potential, but there hasn't been a commit for a while on the master branch. Do you have any further development planned or should we consider this project inactive?
—
Reply to this email directly or view it on GitHub.


Typo - Chapter 2 - Better evaluation using code-validation

There's a typo in the book: page 69. The code is presented as:
from sklearn.model_selection import cross_val_score
scores = cross_val_score(tree_reg, housing_prepared, housing_labels, scoring="neg_mean_squared_error", cv=10)
rmse_scores = np.sqrt(-scores) ### here is the typo, the variable should have been names: tree_rmse_scores

while on the second page, we have:
display_scores(tree_rmse_scores)

Variable: tree_rmse_scores does not exists.
On Github the code is correct: ch2 code project: In [79]

Hi @etudor ,
Thanks for your feedback.  Indeed this error exists in the first paper version.  I fixed it a few weeks ago, and the latest printed version is okay now (as well as the digital versions).
I hope you are enjoying the book!
Cheers,
Aurélien


Inaccurate player routefinder to overlapping target

Currently we use osrs's implementation. Early osrs and pre-2007 runescape, players and npc's handled routefinding to overlapping targets, the exact same way: Moving to a random direction
Example 1: May 11, 2006 video
https://github.com/user-attachments/assets/2ef2602a-3b01-434a-8975-6b0da619d179
Example 2: June 29, 2007 video
https://github.com/user-attachments/assets/08407718-f051-436b-930a-f8b24204fca2

lots of unknowns about this without being able to test on an original osrs server or something but this can be cheaply done with something like this:


update circleci config

Combines test and build steps, reducing job startup time
Improves node_modules caching
Names commands for better readability on dashboard
Removes extraneous prod config

Closing based on objection from the team, may revisit later


nixos/security/ca: enable support for compatibility bundles

Description of changes
Certain software stacks have no support for OpenSSL non-standard PEM format and will fail to use our NixOS CA bundle.
For this, it is necessary to fallback on a 'compatibility' bundle, which will contain no additional trust rules.
Open questions:

Add a NixOS test?
Make it configurable at the cacert package level to save on space?
Document it in the release notes as a highlight?

Depends on https://github.com/lukegb/buildcatrust/pull/4. Depends on stable release of buildcatrust.
Closes #286722.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

I will retarget staging.

You missed a reference to enableCompatibleBundle :)

You missed a reference to enableCompatibleBundle :)

"I had one job"


feat: add keep git dir to go sdk client git

https://github.com/dagger/dagger/issues/3657

It seems worth asking whether we even need this to be a param though. I was surprised the .git dir was being dropped in the first place. Should we just make it true?

I like this. I can see reasons for having it as a param... but I think I'd prefer it be the negation: dropGitDir or something similar. Most of the time, you'll want it.

I like this. I can see reasons for having it as a param... but I think I'd prefer it be the negation: dropGitDir or something similar. Most of the time, you'll want it.

However, the current LLB instruction is llb.KeepGitDir(), should we not try to follow the same spec?

It seems worth asking whether we even need this to be a param though. I was surprised the .git dir was being dropped in the first place. Should we just make it true?

I assume .git busts the cache way too often, hence why it's ignored by default?


[WIP] Template Mod Generator

This is a very work in progress PR to look at creating a web based template mod generator. This generator will take some input from a user and then generate a ready to go project template.
Features required for MVP:

[ ] Generate and download a zip file similar to the example mod.
[ ] Mod name, and package input
[ ] Minecraft version selection
[ ] Generate a Kotlin mod (with groovy buildscript)
[ ] Data generation. (Disabled when kotlin selected?)
[ ] Github Actions option.
[ ] Split sources option.
[ ] Support older minecraft versions, and target their JVM version.

Im creating this PR so early to try and gain feedback, I have a lot of ideas that can also be added after an inital release. If anyone wants to help out please let me know 👍

I think abstracting this away from the browser isnt a major amount of work. Proving an interface for the metadata API would be useful to remove the dependency on the XML parsing and allow a server or cli tool to cache the requests easier.
This kind of abstraction would also solve the automated testing problem (we need to make sure all the combinations compile).

Could the actual template generation management be split into a library for external use?

Yes, I think thats the plan. Might not happen right away, as some time might be needed to ensure a stable API. The plan is to allow this to run in the browser (on fabricmc.net) and then others can implement this into IDEs or CLI tools as they wish.

Just saw this in the Fabric Discord and this looks really interesting. Maybe it'd be useful if there were description, list of authors and license sections (and maybe contact information too) alongside the name? (basically fabric.mod.json) This could get bloated pretty quick though.
Also, maybe an option to enable/use access wideners would be nice to have? This would likely be placed alongside the rest of the advanced options.
By the way, are Mixins set up automatically?

I think this is basically ready to be played with. Its not perfect but should be good enough. I think the besy way to release this will be softly first by not linking to it from any page or making an annocument. This will allow some time for the curious to have a play with it and provide some feedback.
Once im happy its working well a link can be added to the develop page and an announcement about it made.


GSD Request

--- GSD JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v5.10 to before v5.10.85",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=ccdf07219da6bd1f43c6ddcde4c0e36993c7365a",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=4b7e90672af8e0c78205db006f1b0a20ebd07f5f"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "ccdf07219da6bd1f43c6ddcde4c0e36993c7365a",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "4b7e90672af8e0c78205db006f1b0a20ebd07f5f",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "devlink: fix netns refcount leak in devlink_nl_cmd_reload()\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.10.85 by commit 4b7e90672af8e0c78205db006f1b0a20ebd07f5f, it was introduced in version v5.10 by commit ccdf07219da6bd1f43c6ddcde4c0e36993c7365a. For more details please see the references link."
}
--- GSD JSON ---

This issue has been assigned GSD-2021-1002509


MINOR: The expected timoout of 5seconds might fail in launching conta…

…iner and heartbeats.
The increase in Heartbeat timeouts guarantees some slowness in VMs

This patch is not fixing the issue on all cases, so closing it.


Update test_data_guidelines to mention github file limits

As per request from @ewels

@nf-core-bot fix linting


Designing Wizards and Validations With Redux Form

Hi All,
I have an form wizard, which contains three wizards Wizard1,Wizard2,Wizard3. And each of these wizards have been tied up with the reduxForm like this:
Wizard1 = reduxForm({
    form : 'create-form-1',
    fields : ['description']
})(Wizard1);

Wizard2 = reduxForm({
    form : 'create-form-2',
    fields : ['someField']
})(Wizard2);

Now each wizard needs to be validated before other wizard gets displayed. So there is a WizardHandler component, which does takes care of "rendering" from one wizard to another wizard. So in WizardHandler I need to validate Wizard1 fields before it displays Wizard2.
On this use case I have couple of questions:

How do WizardHandler get the values from create-form-* form fields?
Is it a good design for solving the wizards with respect to React and Redux-Form?

Thanks for your help!

How do WizardHandler get the values from create-form-* form fields?

in your connected component, in wire up form in mapStateToProps.  then you can ref this.props.form['create-form-*']

Is it a good design for solving the wizards with respect to React and Redux-Form?

no red flags.  you tell us when you're done :).  it may be easier to change some other piece of state post-Wizard1 form submit that indicates that wizard2 be shown, maybe?  follow your heart.

oh, and http://erikras.github.io/redux-form/#/api/get-values?_k=eo04uv

in a connected component, wire up state.form in mapStateToProps. then you can ref this.props.form['create-form-*']

This one made it work. But the problem is then in order to get the value, I need to do this:
this.form.fieldValue != undefined ? this.form.fieldValue.value : ""
This looks like lot of work! Any other easier way to get?

Your other option is to use the same form name on multiple pages, as detailed here: https://github.com/erikras/redux-form/issues/260#issuecomment-155852315

Here's a new example just for you. Wizard Form

@erikras : This is great! Thanks for your time!

A short question for better understanding redux form: Usually form fields are initialized by providing them as an array to reduxForm. In the wizard-example this is not the case. I tried to recreate this and add some more functionality and flexibility and ran into problems with accessing the form fields only being actually considered (e.g. in validation) after second render. I assume this relates to not providing a fields array. Can you explain?

@RT-TL, what's your code and what's the version you're using?
As usual, an working demo is always better. I recommend http://webpackbin.com.

@gustavohenke Thanks for the hint to use webpackbin. Nice tool that I didn't know before. In the meantime I was able to solve my problem. It was caused by a wrong understanding of how the validation works. I was not aware that when using form-level-validation I allways need to validate not-rendered fields as well so that the whole form is not set to "validat:true".


Inherents are checked with ValidateUnsigned

I see that ValidateUnsigned is not needed for inherents (for instance, pallet-timestamp works that way).
However, when pallet implements both ProvideInherent and ValidateUnsigned, ValidateUnsigned::validate_unsigned() is called on inherents for further validation.
This doesn't make a lot of sense to me since there is already ProvideInherent::check_inherent() for that purpose.
Maybe ProvideInherent::is_inherent() is needed before calling ValidateUnsigned::validate_unsigned() and ::pre_dispatch() on extrinsics that are not signed and depending on response either one or the other kind of validation should be used?

Yes I think we can improve on the way to declare inherent extrinsics and unsigned transaction to avoid having this trait ValidateUnsigned which needs to take care of both.
Currently ValidateUnsigned should be understood as the trait to validate unsigned extrinsics, so both unsigned transactions and inherents.
here is the implementation https://github.com/paritytech/substrate/blob/cc4bf9134b492d3a11f45c6b1cfa4fdf307e63e2/frame/support/procedural/src/construct_runtime/expand/unsigned.rs#L46-L70


Allow all policy types to use hub templates

The framework-addon will handle the reporting of the hub template errors, it is not necessary for each controller to consider them.
Refs:

https://issues.redhat.com/browse/ACM-10858

Consider https://github.com/open-cluster-management-io/governance-policy-framework-addon/pull/136 first - it might currently need some adjustments though.

Consider https://github.com/open-cluster-management-io/governance-policy-framework-addon/pull/136 first - it might currently need some adjustments though.

/unhold


Add nwadu1's file

I copied and pasted this content from my second issue.

I have done many things. I have learned about issues, pull requests, and the structure of a GitHub repository. I have learned about branching, I created a commit, I viewed and responded to pull request reviews. I also edited an existing file, enabled github pages and made my first contribution.


Add an option to have containerless by default

Currently, you can add the 'containerless' attribute to elements, the issue with that is that you can find yourself in places where you need to add this attribute every single time which can be tedious, repetitive and very annoying but most importantly it can make the view unreadable.
So we need a way to tell Aurelia that we want containerless to be the default for all elements and to reverse that option locally we would use something like the following:
<element containerless=false></element>

Hmm. I actually thought when you first stated this that you were referring just to making the compose element containerless by default. I'm not sure doing this for all elements would be a good idea. That would break a lot of things. 3rd party elements would function incorrectly, it wouldn't be very compatible if you wanted to re-use your aurelia element as a web component, etc.
If what you mean is that for a particular element, you want it to always be containerless when it is used, you can actually do that today by decorating the class with the @containerless decorator. Is that what you want?

@EisenbergEffect, yeah! that's what I wanted/meant! ;)

Excellent.


feat(iroh-net)!: Implement http proxy support

Headers are based on how curl and env variables onreqwest.
Setting any of

HTTP_PROXY
http_proxy
HTTPS_PROXY
https_proxy

will make all relay code use these to proxy outgoing connections.
Closes #2295
Breaking Changes

Added iroh_net::relay::http::ClientError::Proxy enum variant

TODOs

[x] config & parsing env variables
[x] the todos in the code
[x] https proxy
[x] testing: tested manually on two machines using squid

Is it a good idea for a library to read environment variables?

reqwest is already used, and does exactly this. They do cover a wider variety of supported variables, like NO_PROXY and system specific configurations which we can add over time, but for now this allows existing users who need this to work with iroh, where they can't currently

Environment variables for proxy configuration are extremely common. Almost every tool that supports proxies supports it through environment variables, especially if it has a CLI.
Fundamentally I think it makes sense because it is an environment configuration, that is the same for every tool on the machine, in almost every case, and it doesn't seem valuable to make the user comprehend separate configurations for every app they use that needs to access the network if it isn't necessary.

I see your point.
Maybe we should have a Proxy config struct that could be passed into the ClientBuilder or something like that. We could have a Proxy::from_env() function, and if it helps, you could have proxies disabled by default in the library.
Then the Iroh CLI will configure the client with the Proxy::from_env().
It would actually be handy to have the Proxy struct, and the environment variable parsing logic in a separate crate so that it could be re-used.

the environment now needs to be read explicitly in iroh-net, but iroh will by default read it


How to generate polygons from segments ?

The demo shows the way how to segment image to masks matrix but doesn't show how to convert masks to polygons ?
Could you teach me how to do it in Python ?
The 'scikit-image' has from skimage.measure import find_contours, approximate_polygon. Should I follow this tool ?

The paper mentions the process in section "4. SNIC-based polygonal partitioning". However I haven't implemented it yet. I will have to see, if I can reserve some time to implement it over the course of the week.
Just a few points, that may help you in the mean time:
Contour Tracing - You could use find_contours, but you would need to run it once for every segement. I think it would be better to use a line tracing algorithm that outputs a single graph over the image (this will also be more useful for the next steps).
Simplifying Polygons - While approximate_polygon does the right thing for a single polygon, is problematic in the way that we need shared edges between superpixels. If we simplify each polygon independently, we will likely obtain regions in the image which aren't assigned to any super-pixel.
To archive this, the paper starts out by choosing vertices that are adjacent to three or more super-pixels (or two superpixels and the image borders), and simplifying the edges between these vertices. This ensures consistent edges between the super-pixels. The paper here explicitely mentions the Ramer–Douglas–Peucker algorithm.

I just pushed the code to the polygonize branch. Writing a efficient the edge tracing took a bit longer and the code still needs clean up and proper documentation. However, the algorithm itself should work and you can already use it. Feel free to ask if something is unclear or report bugs with the new code if you find some.
I have provided a basic example at https://github.com/MoritzWillig/pysnic/blob/9d9fa28b47e24477cca0718c2f5fdbe2c4b398f5/pysnic/examples/polygons.py

Hi! Instead of only plotting,
for vertices, edges in graphs:
    fig = plt.figure("SNIC with %d segments" % len(centroids))
    plt.imshow(mark_boundaries(color_image, np.array(segmentation)))
    for x,y in vertices.keys():
        plt.scatter(x + 0.5, y + 0.5)
    for edge in edges:
        plt.plot([p[0] + 0.5 for p in edge], [p[1] + 0.5 for p in edge])
plt.show()

is it possible to save the results from polygonize into shapefile to be opened in GIS softwares like QGIS?
Thank you very much,
Leandro

Thanks for your question. Unfortunately, I'm not sure what behaviour to expect here; I guess everything should be overlaid by polygons, e.g. covered in blue? (I assume you wrote the `convert_to_shapely_geometry' yourself, as I can't find any reference to it online).
In your images I see lines connecting the start- and end-points of individual edge sequences. So, I think I missed to describe the data format more clearly: edges does not include lines enclosing individual areas/polygons, but are only connections from one vertex - a point where 3 or more segments meet - to another.
You still need to assemble polygons by tracing along the edges. (For this, you might need to lookup additional pixel information using the unsimplified edges). You also need to take care of adding possible fully enclosed segments as holes to the enclosing polygons. Maybe you have an idea on how to implement this or, otherwise, I'll see what I can do (but this might take a while as I'm currently a bit short on time...).


Consolidate different Button-type components to use Button variants

📝 Provide a description of the new feature
Currently we have a variety of Button-type components

Button
InteractiveButton
PillButton
SidebarButton

Recently, we introduced class-variance-authority to the Button component and a handful of variants
➕ Additional Information
Consider consolidating all of the above components into a single Button component, with some clear variants for each use-case sidebar, interactive, pill, etc

With the pending switch to use primer design components, this refactor may be no longer required


Failing to parse JSON response from Cosmos ListSQLRoleDefinitions

Bug Report
I am trying to retrieve a list of SQL Role Definitions setup against a Cosmos DB. When I call ListSQLRoleDefinitions I'm getting the following error:
documentdb.SQLResourcesClient#ListSQLRoleDefinitions: Failure responding to request: StatusCode=200 -- Original Error: Error occurred unmarshalling JSON - Error = 'json: cannot unmarshal number into Go struct field SQLRoleDefinitionListResult.value of type documentdb.RoleDefinitionType'
I'm assuming this is a similar issue to #14509 where the Swagger spec has an issue.
The JSON returned is
{
  "value": [
    {
      "id": "/subscriptions/.../resourceGroups/.../providers/Microsoft.DocumentDB/databaseAccounts/.../sqlRoleDefinitions/00000000-0000-0000-0000-000000000001",
      "name": "00000000-0000-0000-0000-000000000001",
      "properties": {
        "roleName": "Cosmos DB Built-in Data Reader",
        "type": 0,
        "assignableScopes": [
          "/subscriptions/.../resourceGroups/.../providers/Microsoft.DocumentDB/databaseAccounts/..."
        ],
        "permissions": [
          {
            "dataActions": [
              "Microsoft.DocumentDB/databaseAccounts/readMetadata",
              "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/executeQuery",
              "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/readChangeFeed",
              "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/items/read"
            ],
            "notDataActions": []
          }
        ]
      },
      "type": "Microsoft.DocumentDB/databaseAccounts/sqlRoleDefinitions"
    },
    {
      "id": "/subscriptions/.../resourceGroups/.../providers/Microsoft.DocumentDB/databaseAccounts/.../sqlRoleDefinitions/00000000-0000-0000-0000-000000000002",
      "name": "00000000-0000-0000-0000-000000000002",
      "properties": {
        "roleName": "Cosmos DB Built-in Data Contributor",
        "type": 0,
        "assignableScopes": [
          "/subscriptions/.../resourceGroups/.../providers/Microsoft.DocumentDB/databaseAccounts/..."
        ],
        "permissions": [
          {
            "dataActions": [
              "Microsoft.DocumentDB/databaseAccounts/readMetadata",
              "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/*",
              "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/items/*"
            ],
            "notDataActions": []
          }
        ]
      },
      "type": "Microsoft.DocumentDB/databaseAccounts/sqlRoleDefinitions"
    }
  ]
}

Here's the code I'm using to generate the error
package main

import (
	"context"
	"fmt"

	"github.com/Azure/azure-sdk-for-go/services/cosmos-db/mgmt/2021-06-15/documentdb"
	"github.com/Azure/go-autorest/autorest/azure/auth"
)

func main() {
	ctx := context.Background()
	client := documentdb.NewSQLResourcesClient(<subscription_id>)
	authorizer, err := auth.NewAuthorizerFromEnvironment()
	client.Authorizer = authorizer
	defs, err := client.ListSQLRoleDefinitions(ctx, <resource_group>, <cosmos_account>)
	if err != nil {
		fmt.Println(err.Error())
	}
	fmt.Println(defs)
}

import path of package .../services/cosmos-db/mgmt/2021-06-15/documentdb
SDK version v56.2.0

Hey @lirenhe I'm assuming it's this line that's causing us issues rbac.json.
The swagger spec defines the type as a string, but the API is actually returning an integer. What do we need to do to get this fixed?

Hey @lirenhe, should I be raising this as in issue in the azure-rest-api-specs github repo instead?

Hey @lirenhe, should I be raising this as in issue in the azure-rest-api-specs github repo instead?

Hey @jarrodd07 , since this is caused by mismatch between swagger and actual response, this should be an issue of the swaggers instead of our SDK. Could you please open an issue in the azure-rest-api-spec repo?

Close this item as this is a Swagger issue. After the swagger is fixed, the new release of CosmosDB would contain the fix.
@jarrodd07, feel free to reopen it if you have further questions. Thanks


No imperative way to toggle Hamburger

Hey there! Saw this package on reddit a few days ago and thought I'd give it a try, I really like the animations.
I tried this out in my project and ran into an issue and wondering if you could help. I have it working to open and close my menu just fine, but inside my menu I want to be able to click on a navigation item and have that close the menu after I click on it. I currently can't fire off a function to affect the Hamburger state, essentially.
What I am proposing is maybe a new property (named toggled perhaps?) that accepts a boolean value, so that I can control the toggling state of the Hamburger from clicking on something that isn't the Hamburger.
Example:
// hook for showing/hiding menu
const [menuIsToggled, setMenuIsToggled] = useState(false);

// elsewhere in my code, a function that closes the menu after a navigation item is clicked
const closeMenuAndNavigate = () => {
  // navigate to new page, then close the menu
  navigate('/');
  setMenuIsToggled(false); // close the menu
}

// Later on, I can control the Hamburger's internal state with a `toggled` prop,
// and still use `onToggle` to trigger the change
<Hamburger toggled={menuIsToggled} onToggle={() => setMenuIsToggled(!menuIsToggled)} />

Look forward to hearing back. Thanks!

Hey, thanks for opening an issue. I'm currently working on something like this here: https://github.com/luukdv/hamburger-react/commits/custom-state
When finished, you'll be able to provide your own state + setter through a toggled and toggle prop. In your case, that would look like the following:
const [menuIsToggled, setMenuIsToggled] = useState(false);

const closeMenuAndNavigate = () => {
  navigate('/');
  setMenuIsToggled(false);
}

<Hamburger toggled={menuIsToggled} toggle={setMenuIsToggled} />

Let me know if that works for you!

@luukdv that would work perfectly for me, glad to see you were already working on it! Thanks for the reply, I will keep an eye for that branch to be merged

🙌  thanks! less than 24 hours turnaround time that's pretty awesome 😄

Hehe. :shipit: No problem!


slider: handle mouse wheel events

Make slider + vertical_slider handle mouse wheel events -- i.e. when hovered, scrolling the mouse wheel will modify the slider.

Question: currently the slider is incr/decremented by self.step for both ScrollDelta::Pixels and ScrollDelta::Lines. This seems appropriate for Lines, but I'm not sure about Pixels. (It felt fine to me when testing, but I'm not sure what the "expected" behaviour is.)

Oh. I was actually trying to emulate Windows-specific behavior here, specifically the app I was replacing with my Iced project. Of course, users can adjust, but the initial friction will be there.
So I suppose there's no way for a widget inside another to prevent the outer widget from capturing its events?


Hot reload does not work in VSCode

When I pressed cmd+s , nothing changed and got the error below:
NoSuchMethodError: The getter 'isClosed' was called on null.
Receiver: null
Tried calling: isClosed

It's so ridiculous to see hot reload works fine in Android Studio.

please add the output of flutter doctor

Same here. Works fine on Android Studio and IntelliJ. flutter doctor -v output below
[√] Flutter (Channel beta, v0.4.4, on Microsoft Windows [Version 10.0.15063], locale en-GB)
• Flutter version 0.4.4 at C:\Flutter\flutter
• Framework revision f9bb4289e9 (2 weeks ago), 2018-05-11 21:44:54 -0700
• Engine revision 06afdfe54e
• Dart version 2.0.0-dev.54.0.flutter-46ab040e58
[√] Android toolchain - develop for Android devices (Android SDK 27.0.3)
• Android SDK at C:\AndroidSetup\android-sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-27, build-tools 27.0.3
• ANDROID_HOME = C:\AndroidSetup\android-sdk
• Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-915-b01)
• All Android licenses accepted.
[√] Android Studio (version 3.0)
• Android Studio at C:\Program Files\Android\Android Studio
• Flutter plugin version 23.2.1
• Dart plugin version 171.4424
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-915-b01)
[√] IntelliJ IDEA Community Edition (version 2018.1)
• IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.1
• Flutter plugin version 24.1.2
• Dart plugin version 181.4203.498
[√] VS Code, 64-bit edition (version 1.23.1)
• VS Code at C:\Program Files\Microsoft VS Code
• Dart Code extension version 2.12.1
[√] Connected devices (1 available)
• SM G935F • ce01171141a5b8a20c • android-arm64 • Android 7.0 (API 24)
• No issues found!

flutter doctor -v output :
[✓] Flutter (Channel master, v0.0.0-unknown, on Mac OS X 10.13.4 17E202, locale en-CN)
    • Flutter version 0.0.0-unknown at /Users/xavier/tools/flutter
    • Framework revision e816b89a1f (13 hours ago), 2018-05-25 11:03:47 -0700
    • Engine revision abd74ed5ed
    • Dart version 2.0.0-dev.55.0.flutter-97b6c2e09d

[✓] Android toolchain - develop for Android devices (Android SDK 27.0.3)
    • Android SDK at /Users/xavier/tools/android_home
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-27, build-tools 27.0.3
    • ANDROID_HOME = /Users/xavier/tools/android_home
    • Java binary at: /Applications/Android Studio 3.2 Preview.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b02)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 9.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 9.3.1, Build version 9E501
    • ios-deploy 1.9.2
    • CocoaPods version 1.5.2

[✓] Android Studio
    • Android Studio at /Applications/Android Studio 3.2 Preview.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b02)

[!] IntelliJ IDEA Community Edition (version 2018.1.4)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.io/intellij-setup/#installing-the-plugins

[✓] VS Code (version 1.23.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Dart Code extension version 2.12.1

[!] Connected devices
    ! No devices available

! Doctor found issues in 2 categories.

@dashMrl Could you enable Flutter Run logging and provide the log? Where are you seeing the error you posted? Is there a full stack trace?
I think there's also something weird about your flutter doctor output; the version number is missing? Here's what mine looks like:
[✓] Flutter (Channel master, v0.4.5-pre.52, on Mac OS X 10.13.4 17E202, locale en-GB)
    • Flutter version 0.4.5-pre.52 at /Users/danny/Dev/Google/flutter
    • Framework revision 8717a5e6f7 (7 days ago), 2018-05-19 03:19:35 +0300

It may be worth making the sure working tree is clean (or updating to a different changeset).

Also; if you run Flutter: Hot Reload from the command palette or press the Restart button on the debug toolbar, do you get the same issue, or does it work?

Also:

The flutter doctor output suggests you don't have the Android Studio flutter plugin installed - is this the same machine where it works in Android Studio?

If you run Flutter: Hot Reload from the command palette or press the Restart button on the debug toolbar, do you get the same issue, or does it work?

Here is the log file:
flutterRunLogFile.txt , cleaned the cache before pressing F5
I'm using Android Studio 3.2 Canary 15, and created environment variables  for zsh :
export FLUTTER_ROOT=$HOME/tools/flutter
export PUB_HOSTED_URL=https://pub.flutter-io.cn

I got these messages when the project is opened on Android Studio 3.2 Canary :

3:39 PM  Unsupported Android Studio version: Canary versions of Android Studio are not supported by the Flutter plugin.

@DanTup
I fotgot mentioning I cloned the repo with depth option ：--depth=1, so all the tags were lost

@dashMrl Thanks! Do you see any similar issues if you run from the command line and press r to reload? Could you also try enabling the Observatory Log to see whether we stack traces for the errors in there?

@DanTup Thanks for your awesome works! I pulled the latest commit from branch master just now, and found Hot Reload is working both on VSCode and Android Studio 3.2 Canary 15.
And here is the  Observatory Log file :
observatoryLogFile.txt

I'm not exactly sure what the issue is or what fixed it, but glad it's working. I'm not sure if that log is from when it was broken or working, but there's nothing obviously bad in there (there is a rogue Resume call, but it shouldn't cause any issues like this).
The issue Tim linked above I think is unrelated - that's caused by the Visual Studio Live Sharing extension interfering with debugging, so would only affect VS Code (though should be fixed by a workaround in Dart Code going out later in the week and a change in the VSLS extension going out tomorrow).

But the Dart and Flutter extensions are installed in VSCode (...and they are not disabled).

It's safe to ignore that doctor says you don't have them in the current beta; there's a bug (fixed in dev/master) in the extension detection but it won't affect anything.
There's a lot of stuff in this issue that's probably unrelated to yours so it's probably best to open a new issue.  Any message?
Capture a Flutter Run log file using the Dart: Capture Logs command in VS Code
Try temporarily switching to the dev or master channels to see if the issue may have already been fixed

@DanTup Thanks for your advice, but I've solved the problem. It was my fault! I'm a very beginner who just experimenting with the framework.
My "main.dart" file looks like this:

import 'package:flutter/material.dart';
void main() {
var app = MaterialApp(
home: Scaffold(
appBar: AppBar(
title: Text('Flutter Experiment'),
),
),
);
runApp(app);
}

...so I need a restart, not a reload when I change the text in the Text Widget!

Hot reload doesn't re-execute all code, it just updated the code in memory and then invalidates the root widget so build() is executed again for the whole tree.
Anything that is not updated by build() will not updated in the UI.
Affected are for example model dialogs that have their own root widget or code that is executed before runApp(app) like in your example
  var app = MaterialApp(
    home: Scaffold(
      appBar: AppBar(
        title: Text('Flutter Experiment'),
      ),
    ),
  );

So there are a few less common situations where you need to restart instead of reload.
See also https://flutter.io/hot-reload/#recent-code-change-is-included-but-app-state-is-excluded

Glad you got it sorted :) When I implemented hot reload in VS Code I also didn't realise changes in main method wouldn't reload (the Hello World example is all inlined) so I spent a while scratching my head over it too (and it's not just us). Hopefully we can communicate this sort of thing better in future (we have tried some things previously, but they weren't reliable enough).

@zoechi Thanks for the explanation and thanks to all of you for the framework! I really like it!

by flutter run -v, look up:
NoSuchMethodError: The getter 'isClosed' was called on null. Receiver: null

How to do? vscode can't hot reload.

thank you @DanTup . I had added -v to the args. look:

"version": "0.2.0",
  "configurations": [
    {
      "name": "wk-flutter",
      "request": "launch",
      "type": "dart",
      "args": ["-v"]
    }
  ]

I had submited the issus. the link is https://github.com/flutter/flutter/issues/22253.
thanks again.

When I run the debug command in flcode directly to generate the debug command, it will cause my iPhone X emulator to be white screen. If I use the custom lanch.json, then I will bring the -v parameter and it will run. . But it will burst out of this error #22253

Now，the hot replace is not work.I am very worry.

This is my flutter doctor -v result:
[✓] Flutter (Channel beta, v0.8.2, on Mac OS X 10.13.6 17G65, locale en-CN)
    • Flutter version 0.8.2 at /Users/zheng/android/flutter
    • Framework revision 5ab9e70727 (3 weeks ago), 2018-09-07 12:33:05 -0700
    • Engine revision 58a1894a1c
    • Dart version 2.1.0-dev.3.1.flutter-760a9690c2

[✓] Android toolchain - develop for Android devices (Android SDK 28.0.0)
    • Android SDK at /Users/zheng/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.0
    • ANDROID_HOME = /Users/zheng/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
    • All Android licenses accepted.

[!] iOS toolchain - develop for iOS devices (Xcode 10.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.0, Build version 10A255
    ✗ libimobiledevice and ideviceinstaller are not installed. To install, run:
        brew install --HEAD libimobiledevice
        brew install ideviceinstaller
    ✗ ios-deploy not installed. To install:
        brew install ios-deploy
    • CocoaPods version 1.6.0.beta.1

[✓] Android Studio (version 3.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 28.0.2
    • Dart plugin version 181.5616
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)

[✓] VS Code (version 1.27.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 2.18.0

[✓] Connected devices (1 available)
    • iPhone X • 6DBBD95F-4D36-47CF-892C-171BFDE35AB4 • ios • iOS 12.0 (simulator)

! Doctor found issues in 1 category.

@zsirfs If you'd added -v to args then you may be hitting a bug in VS Code where running Flutter in verbose mode could result in connecting to Observatory too early. Try installing this beta version and see if the issue goes away.

thank you reply to me. let me try.

I had the same Issue with VS code because I was firing the emulator up from VS code debug >> start debugging on the Menu tab. I solved the Issue by stopping the app and re-running it from the console using 'flutter run' the pressing r to reload

@craxet running from the terminal means you won't be able to use the debugger. If you're having issues when launching with F5/Start Debugging from VS Code, please open an issue https://github.com/Dart-Code/Dart-Code and attach a log captured with the Dart: Capture Logs command.

Stop flutter run with "q" key on terminal and just press "F5" (on VS Code).
Enjoy!

This was likely due to an old bug, but if anyone is getting this recently and ended up in this page (as I did), the inability to hot reload/debug in VSCode was caused by a bug on VSC 1.32.1 and already fixed in the latest insider's version.

VS Code 1.32.2 has now been released and contains the fix for the recent issue mentioned above.