Add Quick info panel

From @Krzysztof-Cieslak on October 11, 2015 18:9
Quick Info panel is "small margin located at the bottom of the code editor. It automatically shows errors and warnings under the cursor, eliminating the need to use mouses to see tooltips with the same info."
That's how it looks like in Ionide Atom -
 Same feature is also implemented in Visual F# Power Tools - https://fsprojects.github.io/VisualFSharpPowerTools/quickinfopanel.html
IMO, it might be interesting addition to language services API.
Copied from original issue: Microsoft/vscode-extensionbuilders#6

From @reflectiondm on October 11, 2015 21:3
Great thought!

From @egamma on October 12, 2015 7:40
There are the go to next/previous error actions (F8). They show you the error/warning in place using the peek UI also avoid the need for hovering:

From @Krzysztof-Cieslak on October 12, 2015 8:48
Right, this handles error and warnings. But some editors / plugins are using it also for different information (I haven't mentioned it in 1st post, my bad) such as type / function signatures (what is useful in languages with strong type inference - F# / OCaml / Haskell etc.)

From @jrieken on October 14, 2015 12:41
Agreed, we have setStatusBarMessage already today but we don't expose calling language service features from extensions.

From @jrieken on October 14, 2015 12:50
somehow related to #10, at least my previous comment...

From @jrieken on October 31, 2015 13:3
So, with 0.10 there be will some sweet changes that enable this. We will expose the brains-behind language service features as commands. For instance all folks that can compute the definition of a symbol can be trigger like so:
commands.executeCommand('vscode.executeDefinitionProvider', <args>)
    .then(locations => {...})

The arguments will correspond to those the provider needs, usually a document and a position or range. This isn't a UI operation but just the back-office operation. An extension could do the following:

listen to selection changes
trigger the commands to compete extra information/hover
show the results in the status bar -> :dancers:

Should be done as outlined in my previous comment.


[Abide] URL Pattern not taking www into account

URL Pattern only takes http and https into account
but there should be a way to add most common www into account
How to reproduce this bug:
1. Step one
Create a test case similar to
http://codepen.io/IamManchanda/pen/MpoaJY
2. Step two
Go to URL Pattern field
3. Step three
Type www.facebook.com
4. Step four
Press on that dreaded Submit Button
What should happen:
This shouldn't throw error
What happened instead:
This throws error

Basically looking for a combo of domain and url pattern
because with a domain patern its opposite situation => http://codepen.io/IamManchanda/pen/pewLvY
I think we need a combo pattern of both domain and url !

URL is not valid without http or https... perhaps a more general solution would be to allow logical combinations of validatons? e.g. domain OR url ?

Yup combination is what i want .... is this possible right now ?? @kball

Nope. Do you want to implement it? ;)

Yup why not @kball but how do i apply it ..... in codes
Looking for something like
patterns: {
  website: domain || url
}

There's a couple ways you could do it... you could address the general problem (change abide's data-validator option to accept logical sets of validators, and handle that...) or you could simply apply the specific problem by writing a pattern that responds to the 'test' method (either a regex or an object with a test method that does what you want.  You can see how these are being applied here: https://github.com/zurb/foundation-sites/blob/develop/js/foundation.abide.js#L365
So you could probable do something like
patterns: {
  website: {
    test: (text) => {return Abide.defaults.patterns['domain'].test(text) || Abide.defaults.patterns['url'].test(text);}
   }
}

PR is up #9984 .... thanks @kball for your kind help!


I can't find my device after flashing what now?

Before opening a new issue, please read the following
Are you reporting a successful flash?
Please add it to the wiki instead!
Do you have a question about configuring your newly flashed device?
Please ask in your alternative firmware's support channel instead!
Did you have issues getting a device to flash?
Please be sure to include your device name, model, and firmware version as well as your logs!
Please remember to close your issue when the problem has been addressed!

Please elaborate, what do you mean you can't find it?

I have had a similar experience twice now. Tuya convert works fine, update ssid/pew in tasmota thru <IP_ADDRESS> and then the device doesn’t show up.
For some reason the device doesn’t seem to go to dhcp and latches on to an odd ip range outside my router range (10.##.##.##.##) Found the device connecting correctly to AP thru unify APP app, then was able to get MAC address and thru the router force it to the correct ip range thru dhcp reservation.
Hope I explained that well enough.

I think I have the Problem with da Blitzwolf SPH6 Device

Hi, @kinselp Do use the Modul with the external antenna?
Check that the Module not as a Acces Point in your Network.

Closing until we receive more information


memory corruption in pkt_alloc

When pkt_alloc is called with a timeout (CLONE_TIMEOUT=100ms)  and there is no free buffer, k_mem_slab_alloc returns a 0 value (OK) but the returned pkt is NULL. Since afterwards the pkt pointer is not checked it results in memory corruption.
I encountered this problem in the following scenario: I have a multicast listener (mdns ipv4) and bombard the device with multicast traffic for this listener, the aim being to exhaust the pkt buffer pool. From the code (net_conn_input) I see that multicast packets are cloned as follows:
mcast_pkt = net_pkt_clone(pkt, CLONE_TIMEOUT);
This eventually calls pkt_alloc, which calls k_mem_slab_alloc. When the slab is exhausted and because our timeout is not K_NO_WAIT, the function k_mem_slab_alloc returns a return code of zero but also a NULL pkt.

Thanks for the analysis. I checked the net_pkt_clone() call in connection.c and at least it is checking the return value. Can you point me to the line where you found the issue?

The problem happens in function pkt_alloc in subsys/net/ip/net_pkt.c, called with CLONE_TIME. The function calls k_mem_slab_alloc, which in my case returns 0, indicating success but the pkt pointer is NULL. Because pkt_alloc relies on the return code it does not check the pkt pointer and uses the returned NULL in the subsequent call to  memset(pkt, 0, sizeof(struct net_pkt));. I reconstruct the problem under heavy multicast traffic, which causes a lot of packet cloning and finally apparently the exhaustion of the packet buffers. The strange thing for me is the function k_mem_slab_alloc:
The problematic code (below) somehow assumes that _current->base.swap_data; has a valid memory pointer. Shouldn't the logic somewhat like:  return *mem ? result : -ENOMEM
/* wait for a free block or timeout */
	result = z_pend_curr(&lock, key, &slab->wait_q, timeout);
	if (result == 0) {
		*mem = _current->base.swap_data;
	}
	return result;

Ok, this seems to indicate issue in k_mem_slab_alloc() if 0 is returned but the returned memory pointer is NULL.
Assigning to @andyross for analysis.

Are you also using k_mbox_*(), k_stack_*(), k_pipes_*(), k_queue_*(), or k_msgq_*()? These also touch base.swap_data`.

yes, we at least use also the k_msgq , work queues and threads

Could you provide something that can reproduce the issue via QEMU (which will help with debugging)? Just by looking at the code, mem slab should not return a null pointer in k_mem_slab_alloc() unless a null pointer is being freed in k_mem_slab_free(). I also made a mock up to test alloc/free with timeouts but it ran for hours without returning pointers outside the mem slab block. So anything that can help us to reproduce the issue would be very helpful.

Sorry, I do not have the resources to reproduce this, except the scenario I described. Could it be, that there simply was no call to k_mem_slab_free in the given timeout? Another possible direction for me is, that our bsp code (aarch64) somehow messes things up.

It should return -EAGAIN if alloc times out (as set in kernel/include/kswap.h, line 99 when the allocating thread first is being swapped out).
I ran my mock-up app with qemu_cortex_a53 for all morning without any issue, but this is missing a lot of interactions that exist in your app.

Another thing, is it always happening to the same thread? If so, maybe you can setup a watch point to catch when the thread's base.swap_data is set to zero. k_mem_slab_free() should not set it to zero, or else it would be freeing a null pointer. The k_msgq may be setting it to zero, depending on your app.

Looks like https://github.com/zephyrproject-rtos/zephyr/pull/29615 was submitted to manage exactly the same issue.  That's more of a workaround than a fix, though.

I am seeing this issue with latest Zephyr (e1e06d05fa + some cherry-picked fixes), its quite easy to reproduce and my test case is below:
Test

Zperf is running UDP download over Wi-Fi and iperf client pumping at 50Mbps, the throughput reported is ~15Mbps
In the background, a separate thread is running a command to get some driver stats, this command causes the RX queue to be full as it blocks the
driver. Running it 4/5 times is enough

Context

Driver is using net_pkt_rx_alloc_with_buffer with 100ms timeout, when this fails net_pkt_clone also fails as described
above
Disable timeout fixes the issue

Ozone callstack:
[00:00:52.087,463] <err> net_pkt: Data buffer (1498) allocation failed.
[00:00:52.087,493] <wrn> net_conn: pkt cloning failed, pkt 0x20066df4 dropped
[00:00:52.187,744] <err> net_pkt: Data buffer (1498) allocation failed.
[00:00:52.187,744] <wrn> net_conn: pkt cloning failed, pkt 0x20067168 dropped
[00:00:52.288,024] <err> net_pkt: Data buffer (1498) allocation failed.
[00:00:52.288,024] <wrn> net_conn: pkt cloning failed, pkt 0x200671ac dropped
[00:03:38.462,005] <err> os: ***** BUS FAULT *****
[00:03:38.462,005] <err> os:   Precise data bus error
[00:03:38.462,005] <err> os:   BFAR Address: 0xffffff73
[00:03:38.462,036] <err> os: r0/a1:  0xffffff65  r1/a2:  0x20003700  r2/a3:  0x20003760
[00:03:38.462,036] <err> os: r3/a4:  0x20011338 r12/ip:  0x00006f4c r14/lr:  0x0006c6c7
[00:03:38.462,036] <err> os:  xpsr:  0xa1000000
[00:03:38.462,036] <err> os: Faulting instruction address (r15/pc): 0x0006c33a
[00:03:38.462,066] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
[00:03:38.462,066] <err> os: Current thread: 0x20003700

This varies but I see two callstacks consistently, none of them are directly related to the code.

z_impl_zsock_select....->z_sched_prio_cmp (thread1 pointer is corrupted, thread2 is the background thread)
z_impl_k_poll->register_events...->clear_event_registrations...->sys_dlist_remove (list is corrupted)

Enabling all network debugs was unhelpful.
This looks like the some thread switching while we wait for network buffers to be free is causing the issue, for now not using timeout works well, so, proceeding with that . I can provide any details needed.
@rlubos @carlescufi  please re-open the issue or let me know if you need me to raise a new bug.

@krish2718 I'd rather open a new issue, instead of resurrecting an over 2-year-old report. However - I think'll be really difficult to help here, w/o providing some way to reproduce the issue, ideally in a form of a minimal sample or test to run in a simulated environment.
Also, please note, that based on my past experiences, I'm more inclined to believe that such kernel misbehaviors, are most likely a result of some external memory corruption (stack overflows etc.) rather than a problem within the kernel itself.


Review error handling

We added basic error handling so that (e.g.) if there's a failure pulling image-builder we don't try to build an image. We need to review this, and make sure that any failure will result in an appropriate error message back to the user.

So I don't lose the reference, here was the fix in PD 1.7 to allow correctly failing tasks: #4341

Covered via testing.


Update management plane "Generate SDK" to remove extra steps

After changes done last week, extra steps where added to the generation of the management plane SDKs (for languages different than .NET) that don't make sense because the service team will only have the SDK when they request a release.
Also, the link used for the Release request documentation doesn't make sense anymore because it takes people to create a release plan. Instead, we should tell users to go to the next step and request a release.
See in red the steps to remove and in yellow the link to remove:

FYI @praveenkuttappan I went ahead and made the changes so now it looks like:

We can also remove Generate SDK step itself. Right?

@maririos Generate SDK for non .NET looks like below now.

We can also remove Generate SDK step itself. Right?

We could streamline the process by only including the .NET language. However, we should inform users that SDKs for other languages will be generated and released during the Release SDK step. We should not use the phrase "generated automatically" because that is inaccurate. There is a member of the management plane team (vendor) responsible for generating and releasing the management plane SDKs.


Rotating a point on a ellipse

Description of bug / unexpected behavior

When creating the typical representation of the ellipse with a rotating point and segments to the focuses, the animation do not finish over the horizontal axis.
Expected behavior

The point must rotate TAU radians and finish its trajectory on the horizontal axis.
How to reproduce the issue

Code for reproducing the problem
class elipse(Scene):
    def construct(self):

        elipse = Ellipse(width=5.6, height=4.0, color=BLUE)
        foco1 = Dot().move_to(1.96*LEFT)
        foco2 = Dot().move_to(1.96*RIGHT)

        t = ValueTracker(0)

        dot = always_redraw(lambda :   Dot(elipse.point_at_angle(t.get_value())))
        line_L = always_redraw(lambda : Line(start = foco2.get_center(), end = dot.get_center(), color = RED, buff = 0.1))
        Line_R =  always_redraw(lambda : Line(start = foco1.get_center(), end = dot.get_center(), color = GREEN, buff = 0.1))

        self.add(elipse)
        self.add(dot,foco1,foco2)
        self.play(Create(Line_R), Create(line_L))
        self.play(t.animate.set_value(TAU),run_time=5, rate_func=linear)

Additional comments

As ad_chaos suggested me in discord, the problem is solved by adding self.wait() at the end of the code.

Yes, for some reason the last frame of the animation is not processed if it's the last animation in the scene. Adding self.wait() resolves this problem.
I think you need to close this issue.


changes required for making page responsive

This looks a little tight to me

Cards look a bit strange when you change the height of the window vertically, perhaps these cards need to have a fixed height?

For accessibility, we would want to avoid having 2 scrollbars - we'd just want the vertical one on the right.

I believe the cause of this is the timeline picture, that seems to be a fixed size, so doesn't reduce in width when the window is shrunk. This should be an easy fix while we have the timeline as a picture - set the width to 100% for the img tag of the timeline (preferably in the stylesheet). This means that the image will be 100% of the width, not over, no matter the size of the window. It will adjust accordingly.


Name stays when character deleted from library

Steps to reproduce:

Select a user created character from the library, the name appears in the top bar.
Long press the same character to bring up the red 'x'
Touch the red 'x' to remove the character from the library - the name stays in the top bar until another character is selected.

Donald can help if he has capacity.


python312Packages.dvc-render: 1.0.1 -> 1.0.2

Diff: https://github.com/iterative/dvc-render/compare/refs/tags/1.0.1...1.0.2
Changelog: https://github.com/iterative/dvc-render/releases/tag/1.0.2
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 303235 run on x86_64-linux 1

14 packages built:

dvc (python311Packages.dvc)
dvc-with-remotes
dvc-with-remotes.dist
dvc.dist (python311Packages.dvc.dist)
python311Packages.dvc-gdrive
python311Packages.dvc-gdrive.dist
python311Packages.dvc-hdfs
python311Packages.dvc-hdfs.dist
python311Packages.dvc-render
python311Packages.dvc-render.dist
python311Packages.dvclive
python311Packages.dvclive.dist
python312Packages.dvc-render
python312Packages.dvc-render.dist


Tutorials webpage

Hi, there are multiple examples available in the repository (in notebooks or scripts) but they are not available online as a webpage in documentation.
Can I send some PRs so that these examples are explained thoroughly and displayed in separate webpage like famous "keras examples" page.
Thanks!

Hi Asapanna
@avital FYI
Thanks for reaching out! We're always very interested in improving on our examples, so let's try to figure out together how to best do this.
Note that we already have a (very simple) page in our RTD: https://flax.readthedocs.io/en/latest/examples.html
When you refer to "Keras examples" I assume you mean this page: https://keras.io/examples/
It seems they have all their examples as Colab notebooks. We believe that this is  great format for some easy examples to get started (like our Annotated MNIST), but for larger examples (e.g. our examples/wmt) we think it's actually better to have the code in multiple files that can be edited with text editors / IDEs - both because Colab does not scale very well with large cells, and because for serious ML you'll want to have this in separate modules, and refactoring a Colab into a module is an error-prone process.
Because we love Colabs, we already added some notebooks to examples that allow you to explore the code and data, and even modify it and store it on your Drive - all without any installation required. Check out the MNIST example Colab and the Imagenet example Colab.
But this is just me talking about what we already have, so I'd be curious to hear your ideas how to improve it!
Best, Andreas

Note that we already have a (very simple) page in our RTD: https://flax.readthedocs.io/en/latest/examples.html

Sure, yes. I saw the page. But can we have more examples. Especially targeted for beginners so that experienced as well and new researchers / practitioners can start training models with ease. The idea is to have least entry barrier as possible.

When you refer to "Keras examples" I assume you mean this page: https://keras.io/examples/

Yes !!

for larger examples (e.g. our examples/wmt) we think it's actually better to have the code in multiple files that can be edited with text editors / IDEs - both because Colab does not scale very well with large cells, and because for serious ML you'll want to have this in separate modules, and refactoring a Colab into a module is an error-prone process.

I think the examples should NOT be focused on covering as many "use-cases" as possible like keras-examples-page. And we do NOT need to implement .
Instead, in my humble opinion, the falx examples page should be centered around these two ideas - (which would help both experienced and beginner researchers/practitioners to pick up flax with ease)

Need to implement small examples which cover the Flax API as much as possible. We cover it in breadth but not in depth.
Whenever a new feature is out in flax we build a simple model using that feature(s). This would help others to replicate novel ideas by looking at examples.

Thanks for the additional explanations.
IIUC then you propose to mainly rework our examples page, leaving the full-fledged examples ("depth") as is, but adding more small notebooks to the examples ("breadth"), similar to our "How do I ...?" series, but shorter and more self-contained.
Let's maybe first sketch a short list of such examples in this issue, and then we can discuss which ones to implement and how (e.g. it will be important that the examples are automatically tested so we notice when something becomes stale).

Let's maybe first sketch a short list of such examples in this issue

You read my mind :) I am going through the already implemented examples. Once I go through all, will draft the list.
Please note: I am talking about examples in google colab only. So that we can also convert them to webpages on the go.
These are the categories that I am thinking to cover. Please let me know if there are any categories that we can add/remove.

Modelling: Modelling (different types) architectures in flax for computer vison and NLP
Training: how models in different ways. In TPU/GPU/CPU. Using different types of training loops or schedulers as in "how to section".  (Adding hyperparameter tuning is relevant here?)
New flax features: Examples based on new flax features or new research papers
and ..?

Hi @INF800, any updates on this issue?
Note I recently added a HOWTOs tracking issue #2116 where you could also add ideas.

Closing this then!


perf(buffer)!: apply SSO technique to text buffer in buffer::Cell

Description
ratatui allocates one String buffer for each buffer::Cell. char is not sufficient since one character can consist of multiple code points. However, in almost all cases the cells have a single code point. It means that ratatui allocates many small heap memories (~16 Bytes/cell) only for the rare cases.
Allocation on heap is slower than allocation on stack so moving these allocations to stack can improve performance of allocating buffer::Buffer.
To archive this, a small string optimization (SSO) is available in general (which is adopted by C++'s std::string). Fortunately, compact_str crate is a solid implementation of SSO. It provides a drop in replacement of standard String so we can use it with very small changes (as diffs of this PR show).
The CompactString has the same size as String and has 24 bytes inline storage. By using it for text buffer of buffer::Cell, the symbol character can be stored on stack in almost all cases.
One problem is that symbol field of Cell struct is no longer a standard String object. Now it is a CompactString but it would have small impact on users' code base since users usually modifies the buffer cells through widgets.
Measurement summary

Total number of small (16Bytes) heap allocations was reduced from 288895 to 8178
Total size of small (16Bytes) heap allocations was reduced from 4.41MiB to 127.78KiB
Performance of initial rendering was 1.77x faster
Performance of rendering on each tick was 1.14x faster

Memory profiling
I used xctrace (general purpose profiler bundled in Xcode) to measure memory profiling. I modified the demo example to use TextBackend and stops after 100 tick iterations to measure the impact on this change precisely.

Diff applied to demo example
diff --git a/examples/demo/dummy.rs b/examples/demo/dummy.rs
new file mode 100644
index 0000000..b733aca
--- /dev/null
+++ b/examples/demo/dummy.rs
@@ -0,0 +1,31 @@
+use std::{error::Error, io};
+
+use ratatui::backend::TestBackend;
+use ratatui::prelude::*;
+
+use crate::{app::App, ui};
+
+pub fn run(enhanced_graphics: bool) -> Result<(), Box<dyn Error>> {
+    let backend = TestBackend::new(318, 82);
+    let mut terminal = Terminal::new(backend)?;
+
+    // create app and run it
+    let app = App::new("Test Demo", enhanced_graphics);
+    let res = run_app(&mut terminal, app);
+
+    if let Err(err) = res {
+        println!("{err:?}");
+        // } else {
+        //     println!("{:?}", terminal.backend().buffer().content);
+    }
+
+    Ok(())
+}
+
+fn run_app<B: Backend>(terminal: &mut Terminal<B>, mut app: App) -> io::Result<()> {
+    for _ in 0..100 {
+        terminal.draw(|f| ui::draw(f, &mut app))?;
+        app.on_tick();
+    }
+    Ok(())
+}
diff --git a/examples/demo/main.rs b/examples/demo/main.rs
index c2f742b..a236d18 100644
--- a/examples/demo/main.rs
+++ b/examples/demo/main.rs
@@ -1,10 +1,11 @@
-use std::{error::Error, time::Duration};
+use std::{env, error::Error, time::Duration};

 use argh::FromArgs;

 mod app;
 #[cfg(feature = "crossterm")]
 mod crossterm;
+mod dummy;
 #[cfg(feature = "termion")]
 mod termion;
 #[cfg(feature = "termwiz")]
@@ -25,12 +26,19 @@ struct Cli {

 fn main() -> Result<(), Box<dyn Error>> {
     let cli: Cli = argh::from_env();
-    let tick_rate = Duration::from_millis(cli.tick_rate);
-    #[cfg(feature = "crossterm")]
-    crate::crossterm::run(tick_rate, cli.enhanced_graphics)?;
-    #[cfg(feature = "termion")]
-    crate::termion::run(tick_rate, cli.enhanced_graphics)?;
-    #[cfg(feature = "termwiz")]
-    crate::termwiz::run(tick_rate, cli.enhanced_graphics)?;
+    if env::var("DO_NOT_USE_DUMMY_BACKEND")
+        .map(|v| !v.is_empty())
+        .unwrap_or(false)
+    {
+        let tick_rate = Duration::from_millis(cli.tick_rate);
+        #[cfg(feature = "crossterm")]
+        crate::crossterm::run(tick_rate, cli.enhanced_graphics)?;
+        #[cfg(feature = "termion")]
+        crate::termion::run(tick_rate, cli.enhanced_graphics)?;
+        #[cfg(feature = "termwiz")]
+        crate::termwiz::run(tick_rate, cli.enhanced_graphics)?;
+    } else {
+        crate::dummy::run(cli.enhanced_graphics)?;
+    }
     Ok(())
 }

The measured results of 16Bytes heap allocations are as follows:

main branch
sso branch

Total bytes
4.41MiB
127.78KiB

# of allocs
288895
8178

Screenshot

Initial rendering performance
I measured E2E launch time (initial rendering) performance using hyperfine.
I removed the for loop to measure a single tick iteration where screen is rendered once.

Diff applied to demo example
diff --git a/examples/demo/dummy.rs b/examples/demo/dummy.rs
index b733aca..6b2a4f7 100644
--- a/examples/demo/dummy.rs
+++ b/examples/demo/dummy.rs
@@ -23,9 +23,7 @@ pub fn run(enhanced_graphics: bool) -> Result<(), Box<dyn Error>> {
 }

 fn run_app<B: Backend>(terminal: &mut Terminal<B>, mut app: App) -> io::Result<()> {
-    for _ in 0..100 {
-        terminal.draw(|f| ui::draw(f, &mut app))?;
-        app.on_tick();
-    }
+    terminal.draw(|f| ui::draw(f, &mut app))?;
+    app.on_tick();
     Ok(())
 }

The result on my machine was as follows:
$ hyperfine --shell=none ./demo_main ./demo_sso
Benchmark 1: ./demo_main
  Time (mean ± σ):       8.8 ms ±   0.4 ms    [User: 6.4 ms, System: 1.5 ms]
  Range (min … max):     8.4 ms …  11.4 ms    316 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 2: ./demo_sso
  Time (mean ± σ):       5.0 ms ±   0.1 ms    [User: 3.0 ms, System: 1.1 ms]
  Range (min … max):     4.7 ms …   5.8 ms    581 runs

Summary
  ./demo_sso ran
    1.77 ± 0.09 times faster than ./demo_main

100 tick iterations performance
In addition, I measured performance of 100 tick iterations with hyperfine. And it turned out that this branch is 1.14x faster than main branch.
$ hyperfine ./demo_main ./demo_sso
Benchmark 1: ./demo_main
  Time (mean ± σ):      85.2 ms ±   2.5 ms    [User: 81.8 ms, System: 2.1 ms]
  Range (min … max):    82.2 ms …  94.9 ms    34 runs

Benchmark 2: ./demo_sso
  Time (mean ± σ):      74.9 ms ±   0.6 ms    [User: 72.2 ms, System: 1.5 ms]
  Range (min … max):    73.3 ms …  76.8 ms    38 runs

Summary
  ./demo_sso ran
    1.14 ± 0.03 times faster than ./demo_main

I noticed performance regression that a benchmark with 100 tick iterations got ~10% slower.
After some investigation with CPU sampling profiler, I found that calling clear() then pushing new string/char is slightly slower with CompactString (I guess it checks its length to determine to use inline storage or spill to heap memory).
I fixed it at c5ecda6 and confirmed the fix with hyperfine:
$ hyperfine ./demo_before ./demo_after
Benchmark 1: ./demo_before
  Time (mean ± σ):      95.2 ms ±   2.0 ms    [User: 92.1 ms, System: 1.8 ms]
  Range (min … max):    92.7 ms … 101.2 ms    30 runs

Benchmark 2: ./demo_after
  Time (mean ± σ):      75.3 ms ±   0.9 ms    [User: 72.5 ms, System: 1.6 ms]
  Range (min … max):    73.6 ms …  77.9 ms    38 runs

Summary
  ./demo_after ran
    1.26 ± 0.03 times faster than ./demo_before

I can reproduce the improvements on:
tui-term-main vs tui-term-sso:

Main:

Sso:

Note: The benchmarks are all done with a grid of 80:24, so I am likely looking at a minimum of performance gained. Also even tough I use buffer::Cell directly, I did not need to change any code.

Thanks for doing tests on this - this was missing from the previous approaches.
Some questions:

How does this compare to the time spent rendering in the actual terminal?
I'd suggest using something like https://github.com/ratatui-org/ratatui/pull/583 as a test for this (it changes every cell and the background / foreground every frame, so in theory will ensure that we see the effect on the worst performing app.
We could perhaps simplify to create a random full buffer every frame instead.
It would be a good idea to tests ~10K cells (200x50) as this tends to be the size of a 16in Macbook with some default font sizes (as well as 80x24 for a min and 255x255 for a max)
Is it worth testing the other string types?
What are your thoughts on the absolute magnitude of the change (the relative 14% change obscures that this is 0.1ms - what sorts of scenarios is that worth worrying about in?
Would it be worth making the Cell fields private to avoid exposing this internal detail?

Some previous similar work (tagging for their insights / interest):

https://github.com/ratatui-org/ratatui/issues/356 by @jamespharaoh
https://github.com/ratatui-org/ratatui/pull/231 by @Philipp-M
https://github.com/fdehau/tui-rs/pull/644 by @rhysd :)

@a-kenji Thank you for your trial. It is very helpful to try this branch by practical external widgets. I'm glad to hear that benchmarks on your side were improved.
@joshka Thank you for the feedback.

How does this compare to the time spent rendering in the actual terminal?

No backend is modified. So the difference is caused only by accessing to the content (e.g. crossterm).
Theoretically it only adds one branch per accessing one cell, since CompactString needs to check if it's using inline storage or heap buffer. The branch only adds a few instructions and we can ignore the impact.
Practically, I'll try to measure the actual impact with real backend. I'll post the result later.

I'd suggest using something like https://github.com/ratatui-org/ratatui/pull/583 as a test for this (it changes every cell and the background / foreground every frame, so in theory will ensure that we see the effect on the worst performing app.

OK, I'll measure performance with colors_rgb and TestBackend later.

We could perhaps simplify to create a random full buffer every frame instead.

Do you mean measuring creating Buffer instance alone? It's possible but I feel initial rendering measurement is sufficient for this.

It would be a good idea to tests ~10K cells (200x50) as this tends to be the size of a 16in Macbook with some default font sizes (as well as 80x24 for a min and 255x255 for a max)

OK, I'll try 200x50 and 255x255. 80x24 was already measured by @a-kenji's benchmark and I feel that's sufficient.

Is it worth testing the other string types?

I considered the following crates:

smol_str : MSRV policy does not fit to this crate. And the author said that ecow is the better implementation.
ecow : This is a good alternative. However, in terms of the following points, I selected compact_str

Its inline storage size is smaller than compact_str
It internally manages its buffer with reference counter for O(1) clone. This is effective when cloning string happens frequently but it is not the case of this crate's usage. And increasing/decreasing atomic reference counter has additional cost.

If you have other alternative, I'll take a look.

What are your thoughts on the absolute magnitude of the change (the relative 14% change obscures that this is 0.1ms - what sorts of scenarios is that worth worrying about in?

I'm not sure the fact that rendering on each tick is 1.14x faster has real impact on user's use case. Rendering happens per some intervals (e.g. 100ms) so program has enough budget for rendering each frames with current main branch. CPU usage of program may slightly go down tough.
The most important point in regards to performance is initial rendering since a program usually tries to render the first paint and it is CPU intensive. The absolute values of my benchmark with only initial rendering are 5.0ms (sso) vs 8.8ms (main). I feel 3.8ms is large enough to be considered.

Would it be worth making the Cell fields private to avoid exposing this internal detail?

Well, that depends on maintainers. I have no strong opinion. CompactString's API is almost compatible with String. Users of buffer::Cell (I guess there would be only a few direct users though) can use the field almost the same as String. .into() method automatically converts string types into CompactString too.
In general, exposing struct fields can cause an issue like this. If I were a maintainer of this crate, I would add a builder for Cell and add accessor methods instead of exposing fields.

I tried colors_rgb example with TestBackend. I applied the following patch to the example.

Diff
diff --git a/examples/colors_rgb.rs b/examples/colors_rgb.rs
index b15c859..d28e395 100644
--- a/examples/colors_rgb.rs
+++ b/examples/colors_rgb.rs
@@ -17,36 +17,36 @@ use palette::{
     convert::{FromColorUnclamped, IntoColorUnclamped},
     Okhsv, Srgb,
 };
-use ratatui::{prelude::*, widgets::*};
+use ratatui::{backend::TestBackend, prelude::*, widgets::*};

 type Result<T> = std::result::Result<T, Box<dyn Error>>;

 fn main() -> Result<()> {
-    install_panic_hook();
+    // install_panic_hook();
     App::new()?.run()
 }

 struct App {
-    terminal: Terminal<CrosstermBackend<Stdout>>,
+    terminal: Terminal<TestBackend>,
     should_quit: bool,
 }

 impl App {
     pub fn new() -> Result<Self> {
         Ok(Self {
-            terminal: Terminal::new(CrosstermBackend::new(stdout()))?,
+            terminal: Terminal::new(TestBackend::new(200, 50))?,
             should_quit: false,
         })
     }

     pub fn run(mut self) -> Result<()> {
-        init_terminal()?;
-        self.terminal.clear()?;
-        while !self.should_quit {
+        // init_terminal()?;
+        // self.terminal.clear()?;
+        for _ in 0..100 {
             self.draw()?;
-            self.handle_events()?;
+            // self.handle_events()?;
         }
-        restore_terminal()?;
+        // restore_terminal()?;
         Ok(())
     }

@@ -71,7 +71,7 @@ impl App {

 impl Drop for App {
     fn drop(&mut self) {
-        let _ = restore_terminal();
+        // let _ = restore_terminal();
     }
 }

I compiled the example on both sso and main branches with --release and measured performance with hyperfine:
$ hyperfine ./colors_rgb_main ./colors_rgb_sso
Benchmark 1: ./colors_rgb_main
  Time (mean ± σ):     292.0 ms ±   7.5 ms    [User: 289.3 ms, System: 1.5 ms]
  Range (min … max):   283.9 ms … 309.7 ms    10 runs

Benchmark 2: ./colors_rgb_sso
  Time (mean ± σ):     286.4 ms ±   1.2 ms    [User: 283.8 ms, System: 1.3 ms]
  Range (min … max):   284.8 ms … 288.3 ms    10 runs

Summary
  ./colors_rgb_sso ran
    1.02 ± 0.03 times faster than ./colors_rgb_main

Even in the worst case, sso branch is faster than main branch.

I applied #583 patch to both sso and main branches and monitored the FPS with 200x50 terminal size.

sso → about 21.50 fps
main → about 20.00 fps

sso
main

I measured performance with hyperfine in 200x50, 80x24, and 255x255:
200x50
$ hyperfine ./demo_main_200_50 ./demo_sso_200_50
Benchmark 1: ./demo_main_200_50
  Time (mean ± σ):      38.6 ms ±   1.5 ms    [User: 36.3 ms, System: 1.2 ms]
  Range (min … max):    37.0 ms …  46.5 ms    71 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 2: ./demo_sso_200_50
  Time (mean ± σ):      34.7 ms ±   0.7 ms    [User: 32.7 ms, System: 0.9 ms]
  Range (min … max):    33.7 ms …  39.1 ms    79 runs

Summary
  ./demo_sso_200_50 ran
    1.11 ± 0.05 times faster than ./demo_main_200_50

80x24
$ hyperfine ./demo_main_80_24 ./demo_sso_80_24
Benchmark 1: ./demo_main_80_24
  Time (mean ± σ):      12.2 ms ±   0.6 ms    [User: 10.6 ms, System: 0.8 ms]
  Range (min … max):    11.5 ms …  15.3 ms    198 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 2: ./demo_sso_80_24
  Time (mean ± σ):      11.5 ms ±   0.3 ms    [User: 10.0 ms, System: 0.7 ms]
  Range (min … max):    11.0 ms …  12.6 ms    200 runs

Summary
  ./demo_sso_80_24 ran
    1.06 ± 0.06 times faster than ./demo_main_80_24

255x255
$ hyperfine ./demo_main_255_255 ./demo_sso_255_255
Benchmark 1: ./demo_main_255_255
  Time (mean ± σ):     193.1 ms ±   6.9 ms    [User: 187.3 ms, System: 4.2 ms]
  Range (min … max):   187.0 ms … 212.2 ms    15 runs

Benchmark 2: ./demo_sso_255_255
  Time (mean ± σ):     174.9 ms ±   6.2 ms    [User: 170.2 ms, System: 3.3 ms]
  Range (min … max):   169.7 ms … 196.1 ms    17 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Summary
  ./demo_sso_255_255 ran
    1.10 ± 0.06 times faster than ./demo_main_255_255

If you have other alternative, I'll take a look.

I don't have specific crates in mind for this - more just curious as there's a few options mentioned in the previous PRs.

Do you mean measuring creating Buffer instance alone? It's possible but I feel initial rendering measurement is sufficient for this.

I meant that it could be worth investing in a small test application for this. The simplest test case without any extra processing that would mess things up would be to render a full screen of symbols filled with random characters to the buffer.

The most important point in regards to performance is initial rendering since a program usually tries to render the first paint and it is CPU intensive. The absolute values of my benchmark with only initial rendering are 5.0ms (sso) vs 8.8ms (main). I feel 3.8ms is large enough to be considered.

My main concern is that this one time gain is outside of the realm of a human being able to see any difference, and I'm wondering if there's a reasonable use case where that speedup will matter for some application.

And this branch is not only for CPU performance but also for memory usage. Heap memory usage by this branch is obviously smaller than main branch since each cell string contains its content inline instead of allocating extra heap memory. Even if this branch has no CPU performance benefit, it is worth consideration in terms of memory consumption.

Yep, but the memory concern is pretty small too - most computers these days have at least 8GB RAM, so 4MB is 0.05% or put another way, I'd need 20 copies of an application running to make a 1% difference in memory usage.

In general, exposing struct fields can cause an issue like this. If I were a maintainer of this crate, I would add a builder for Cell and add accessor methods instead of exposing fields. Then fields of Cell can be modified/added without breaking change. (Without #[non_exhaustive], adding new field to the struct is a breaking change.)

Yeah, I think if I were designing many of the types from scratch I'd create explicit builders (and just use one of the derive builder crates) rather than having the types themselves be builders. The set_symbol() is kind of a builder style setter though, as it returns the cell. This means if we do change this, there's a fairly simple workaround for code that gets broken by this being made private. That said it's worth taking the deprecate and then change pattern for it so that people at least have a chance to see a change that helps drive fixes.

I want to make sure you hear that doing the perf tests here is really important and useful to see, and I value that work a lot. At the same time, I also want to highlight that the results seem to suggest that changing this doesn't seem worth it - at least for now. I'm willing to be wrong on this and change my mind however. But I'd like to consider a few more things first.
Rather than changing the low level data structure, I wonder if there's some better data structure / algorithm than our current one for handling this. When tuning an operation for performance that happens a lot, there's two approaches that attack the same problem: 1. reduce the cost of the individual operation, 2. reduce the count of the operations. The second approach is usually much more important than the first (here N = ~10-20K / frame) . Here are some possible ideas to investigate:

Is there some way to store each Span as the full span in the cell rather than as individual symbols? This would effectively be a zero allocation approach. I suspect there's a lot of strings that are one style and take up the entire row that could be stored just as is. We'd need some sort of z-index to ensure that things which are written over the top of each other are processed in the right order to make this work (e.g. the title of a block is written over the top of the borders etc.)
I suspect a lot of UIs (e.g. the normal / average case) have screens that are mostly blank - what if we stored each Cells as Option<Cell> instead of Cell - do we get a perf increase from the lowered number of allocations?
I wonder if there's a simple way we can just use Cow slices of the spans instead of allocating a String for each cell? The buffer would have to gain a lifetime for this, but that might be the right thing to change.
Is there some way that instead of clearing the buffer, we could start from a buffer that is already the same as the previous buffer, and instead re-allocating strings that are the same as the previous buffer, we just use the existing strings (this would effectively move the diff code to happen during render() instead of at the end, and perhaps would mean that we keep track of a render mask for each cell or something like that.
If we're breaking compatibility on this, then it's worth considering what else we should fix in the same place. My big want is to make get() & get_mut() return Option rather than panicking for index out of bounds.
Is there a way to deliver this behind a feature flag?
Are there any other ideas that could work?

Also, is there some way to make the performance tests app and commands part of the code for the PR, to make it easy to repro this rather than applying patches - perhaps by building it into the benchmarks we have currently?
I also noticed https://nikolaivazquez.com/blog/divan/ a little while ago which might be a good replacement for criterion if we want to invest more in benchmarking.

I'm sorry for the delayed response. I needed to handle some issues in tui-textarea.

I don't have specific crates in mind for this - more just curious as there's a few options mentioned in the previous PRs.

OK. Then I considered all options you kindly listed in the previous comment. I didn't mention smartstring, but its features are almost identical to compact_str and I selected compact_str because it is well-maintained.

I meant that it could be worth investing in a small test application for this. The simplest test case without any extra processing that would mess things up would be to render a full screen of symbols filled with random characters to the buffer.

Thanks for the additional explanation. I think I could get the point. And I don't think demo example is small in terms of measuring the rendering performance. It contains many widgets and the graph is dynamically updated on each tick.

My main concern is that this one time gain is outside of the realm of a human being able to see any difference, and I'm wondering if there's a reasonable use case where that speedup will matter for some application.

Well, I think speedup matters for all applications in my opinion. Even if a single optimization is small, gathering many small optimizations can cause speedup which people can notice.
We don't see any trade offs. So I'm honestly not understanding why it is a concern that the speed gain is small.

That said it's worth taking the deprecate and then change pattern for it so that people at least have a chance to see a change that helps drive fixes.

I agree with this idea.

I want to make sure you hear that doing the perf tests here is really important and useful to see

That's not possible for me until you elaborate on the criteria of really important and useful.

Rather than changing the low level data structure, I wonder if there's some better data structure / algorithm than our current one for handling this.

I don't have time to do such big changes.

Honestly I was disappointed by your response. I measured performance as much variations as possible. And there is no down side actually observed. And you just say 'I have a concern' without any concrete examples.

Yeah I agree, I think if it doesn't introduce a massive dependency chain, or not well maintained or written dependencies (which is not the case here), it's a net-benefit. I think the benches shown here are significant enough and it doesn't introduce a lot of new code that needs to be maintained.
Finding an algorithmically more optimal solution is orthogonal to this change and can still be done after this.

We don't see any trade offs. So I'm honestly not understanding why it is a concern that the speed gain is small.

My apologies for not properly explaining the actual concern (If I understand you right, you took offense that just focused on the size of the speed increase)
Let's examine the tradeoff:

Upside: 0.18ms per frame speedup here based on the 255x255 rendering of the demo ((193.1 - 174.9)ms/100). This would be a fairly usual case (except for full screen animation apps like colors_rgb). For comparison, 60fps is 16.67ms per frame.
Downside: every application / library that manually sets the Cell::symbol field using a String has to be updated. I don't know how large this number is (https://github.com/search?q=ratatui+".symbol+%3D+"&type=code suggests it's small, but there are ~193 repos that use tui-logger - I'm guessing a bunch of those are still on tui-rs not ratatui though, 🤷♂). This is just the public github repos though too, so there might be other places.
To me it seems difficult to argue that breaking user's code is worth such an imperceptibly small performance increase.

The main concern I have with this is that when we change the public interface, we make users have to change their code, and potentially break downstream applications at the installation point (i.e. our user's user's are the ones that can't install the app). Adding CompactString to the public API, expecting our users to update and then later expecting them to update again if we find a better approach that is not compatible with the CompactString approach is annoying. I have a tendency to try to be conservative when asking other people to change their code (and prefer to do it once rather than twice when possible).
Assuming we make the symbol field private (or perhaps pub(crate), then there's less concern as CompactString becomes an internal detail instead of a public API issue. I think one way to do this would be to mark the field as deprecated and make a release to help communicate this to downstream users prior to changing it, and also adding a getter for the field so that people can migrate to that before the change happens. Making the field private potentially breaks anyone using the serde serialization too, so there's downsides to this approach that are also "interesting"
Anyway, I hope that's helped clarify why this isn't as simple as just changing something small here, and if I've misunderstood your perspective at all, please do continue to advocate your position to help me see things from your perspective.

I agree with this idea very much. How about the following timeline?

Add deprecation message on Cell::symbol field
Add a new default constructor and getter for symbol field
Release ratatui v0.24.1 (or v0.24.2 if v0.24.1 doesn't fit)
Hide the symbol field
Release the change at v0.25.0

Sounds good to me. Let's do it! :)

Everything past here is more about my curiosity about than it is about the problem

There is a larger memory consumption benefit

What sort of systems would this change be a problem for? The smallest EC2 instance has 512MB, and I can't recall a laptop I've owned since perhaps 2010 that's had less than 16GB.

You said that 5ms is too small for meaningful optimization, but on my machine vim command starts in 11ms and I would be happy if it is reduced to 6ms since I start vim ~20 times every day.

I stand by that - perf isn't an absolute thing. Implementing perf changes when there isn't a reasonable situation where they can be detected is a failure to heed Knuth's observation regarding optimization:

"Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

I'd be curious to see if you can actually reliably detect this (I certainly could not). Want to test it out?
Some silly test code
use std::{env::args, thread::sleep, time::Duration};

fn main() {
    if rand::random() {
        println!("Slowed by 5ms - did you notice?");
        sleep(Duration::from_millis(5));
    }
    let args = args().skip(1).collect::<Vec<String>>();
    let err = exec::Command::new("vim").args(&args).exec();
    eprintln!("vim: {err}");
}

I created a PR for tui-logger. As you can see, the change is really small and authors can easily adopt them. Since ratatui has not released v1 yet, it is easy to assume some breaking changes in v0.x release for users as other breaking changes ratatui has added.

I understand that the change itself is fairly small and that it's easy to think this means that breaking is trivial. That's not where the issue gets felt though. A lot of smaller apps have installation instructions that are cargo install foo (they often miss the advised --locked). Every user that runs that command in a downstream application now fails if we break an API. We've seen something similar happen when we introduce breaking changes for our own examples (developers copy code from our git repo and expected it to work with the most recent release). E.g. removing the generic parameter from Frame recently had this problem (and there was another issue in some other breaking changes).

While I am not sure if there is a good way to test for it, I suspect the large differential in allocations here might be able to switch a core from one p state to another in low load scenarios.

Sounds good to me. Let's do it! :)

I'll make a PR for the step 1. and 2.

What sort of systems would this change be a problem for?

Well, I cannot come up with. Maybe some systems which are almost running out memory and causing thrashing?

Implementing perf changes when there isn't a reasonable situation where they can be detected is a failure to heed Knuth's observation regarding optimization

Yes, I understand your point and I agree with Knuth's rules for optimization. It's the matter of trade off (complexity v.s. benefit).
The point is that your stance is that small optimizations which people cannot perceive are premature optimization. And I don't stand for this because sum of optimization gain can be meaningful. For my vim example, I can save 50ms * 20 * 365 = 365s per year and I'd appreciate it. And this PR does not bring much complexity in its internal structure (CompactString just put string bytes inline instead of allocated heap memory). So the Knuth's concern about the introducing complexity is not so applying to this case IMO.

I created a PR: #624

I rebased onto the latest main branch to resolve conflicts and force-pushed the commit since a merge commit is not allowed by lint.

453bc2d made Cell::symbol field private.

This should be merged only after releasing 0.24.1

I rebased onto the latest main branch for fixing the conflict with #626. 3aa7f79 hid the Cell::symbol field and removed the deprecation warning.

I suspect this will go out in v0.26.0 not v0.25.0 as we've got some breaking changes in main that prevent us shipping the next version as v0.24.1.

Sure. So,

Release 0.25 with the deprecation message
Release 0.25.1, 0.25.2, ... (if necessary)
Release 0.26 with this change

Am I correct?

Yep

This can be merged after a rebase right?

I will rebase tomorrow.

I rebased this branch onto the latest main branch.

I'm writing a blog post about benchmarking the I/O streams (block buffered, line buffered, etc.) using Ratatui and I will reference this PR there. However, I just tried it and I'm getting a slightly less FPS on this branch. I'm not sure what is happening yet but I will read through the comments in here and try to figure it out. Maybe it is an initial rendering issue.
Here is the unpolished code that I'm using to benchmark (press Space to toggle between streams):

io_bench.rs
use std::{
    fmt,
    fs::File,
    io::{self, BufWriter, LineWriter, Write},
    os::fd::{FromRawFd, OwnedFd},
    time::{Duration, Instant},
};

use anyhow::Result;
use crossterm::{
    event::{self, Event, KeyCode, KeyEventKind},
    terminal::{disable_raw_mode, enable_raw_mode, EnterAlternateScreen, LeaveAlternateScreen},
    ExecutableCommand,
};
use enum_iterator::{last, next_cycle, Sequence};
use lazy_static::lazy_static;
use palette::{convert::FromColorUnclamped, Hsv, Srgb};
use ratatui::{prelude::*, widgets::*};

lazy_static! {
    static ref RAW_STDOUT_FD: OwnedFd = unsafe { OwnedFd::from_raw_fd(1) };
}

#[derive(Copy, Clone, Debug, Default, Sequence)]
enum IoStream {
    #[default]
    Stdout,
    LineBufferedStdout,
    BlockBufferedStdout,
    Stderr,
    LineBufferedStderr,
    BlockBufferedStderr,
}

impl fmt::Display for IoStream {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(
            f,
            "{}",
            match self {
                IoStream::Stdout => "stdout (unbuffered)",
                IoStream::LineBufferedStdout => "stdout (line buffered)",
                IoStream::BlockBufferedStdout => "stdout (block buffered)",
                IoStream::Stderr => "stderr (unbuffered)",
                IoStream::LineBufferedStderr => "stderr (line buffered)",
                IoStream::BlockBufferedStderr => "stderr (block buffered)",
            }
        )
    }
}

impl IoStream {
    fn as_stream(&self) -> Result<Box<dyn Write>> {
        Ok(match self {
            IoStream::Stdout => Box::new(File::from(RAW_STDOUT_FD.try_clone()?)),
            IoStream::LineBufferedStdout => Box::new(io::stdout()),
            IoStream::BlockBufferedStdout => Box::new(BufWriter::new(io::stdout())),
            IoStream::Stderr => Box::new(io::stderr()),
            IoStream::LineBufferedStderr => Box::new(LineWriter::new(io::stderr())),
            IoStream::BlockBufferedStderr => Box::new(BufWriter::new(io::stderr())),
        })
    }
}

#[derive(Debug)]
struct Fps {
    frame_count: usize,
    last_instant: Instant,
    fps: Option<f32>,
}

impl Default for Fps {
    fn default() -> Self {
        Self {
            frame_count: 0,
            last_instant: Instant::now(),
            fps: None,
        }
    }
}

impl Fps {
    fn tick(&mut self) {
        self.frame_count += 1;
        let elapsed = self.last_instant.elapsed();
        // update the fps every second, but only if we've rendered at least 2 frames (to avoid
        // noise in the fps calculation)
        if elapsed > Duration::from_secs(1) && self.frame_count > 2 {
            self.fps = Some(self.frame_count as f32 / elapsed.as_secs_f32());
            self.frame_count = 0;
            self.last_instant = Instant::now();
        }
    }
}

struct FpsWidget<'a> {
    fps: &'a Fps,
}

impl<'a> Widget for FpsWidget<'a> {
    fn render(self, area: Rect, buf: &mut Buffer) {
        if let Some(fps) = self.fps.fps {
            let text = format!("{:.1} fps", fps);
            Paragraph::new(text).render(area, buf);
        }
    }
}

struct RgbColorsWidget<'a> {
    /// The colors to render - should be double the height of the area
    colors: &'a Vec<Vec<Color>>,
    /// the number of elapsed frames that have passed - used to animate the colors
    frame_count: usize,
}

impl Widget for RgbColorsWidget<'_> {
    fn render(self, area: Rect, buf: &mut Buffer) {
        let colors = self.colors;
        for (xi, x) in (area.left()..area.right()).enumerate() {
            // animate the colors by shifting the x index by the frame number
            let xi = (xi + self.frame_count) % (area.width as usize);
            for (yi, y) in (area.top()..area.bottom()).enumerate() {
                let fg = colors[yi * 2][xi];
                let bg = colors[yi * 2 + 1][xi];
                buf.get_mut(x, y).set_char('▀').set_fg(fg).set_bg(bg);
            }
        }
    }
}

struct AppWidget<'a> {
    title: Paragraph<'a>,
    fps_widget: FpsWidget<'a>,
    rgb_colors_widget: RgbColorsWidget<'a>,
}

impl<'a> AppWidget<'a> {
    fn new(app: &'a App) -> Self {
        let title = Paragraph::new(vec![Line::styled(
            app.current_stream.to_string(),
            Style::new().bold(),
        )])
        .alignment(Alignment::Center);
        Self {
            title,
            fps_widget: FpsWidget { fps: &app.fps },
            rgb_colors_widget: RgbColorsWidget {
                colors: &app.colors,
                frame_count: app.frame_count,
            },
        }
    }
}

impl Widget for AppWidget<'_> {
    fn render(self, area: Rect, buf: &mut Buffer) {
        let main_layout = Layout::default()
            .direction(Direction::Vertical)
            .constraints([Constraint::Length(1), Constraint::Min(0)])
            .split(area);
        let title_layout = Layout::default()
            .direction(Direction::Horizontal)
            .constraints([Constraint::Min(0), Constraint::Length(8)])
            .split(main_layout[0]);

        self.title.render(title_layout[0], buf);
        self.fps_widget.render(title_layout[1], buf);
        self.rgb_colors_widget.render(main_layout[1], buf);
    }
}

#[derive(Debug, Default)]
struct App {
    should_quit: bool,
    switch_stream: bool,
    current_stream: IoStream,
    // a 2d vec of the colors to render, calculated when the size changes as this is expensive
    // to calculate every frame
    colors: Vec<Vec<Color>>,
    last_size: Rect,
    fps: Fps,
    frame_count: usize,
}

impl App {
    pub fn run(io_stream: IoStream) -> Result<bool> {
        let mut terminal = init_terminal(io_stream.as_stream()?)?;
        let mut app = Self::default();
        app.current_stream = io_stream;

        while !app.should_quit && !app.switch_stream {
            app.tick();

            terminal.draw(|frame| {
                let size = frame.size();
                app.setup_colors(size);
                frame.render_widget(AppWidget::new(&app), size);
            })?;
            app.handle_events()?;
        }
        restore_terminal(io_stream.as_stream()?)?;
        Ok(app.should_quit)
    }

    fn tick(&mut self) {
        self.frame_count += 1;
        self.fps.tick();
    }

    fn handle_events(&mut self) -> Result<()> {
        if event::poll(Duration::from_secs_f32(1.0 / 60.0))? {
            if let Event::Key(key) = event::read()? {
                if key.kind == KeyEventKind::Press && key.code == KeyCode::Char('q') {
                    self.should_quit = true;
                }
                if key.kind == KeyEventKind::Press && key.code == KeyCode::Char(' ') {
                    self.switch_stream = true;
                };
            }
        }
        Ok(())
    }

    fn setup_colors(&mut self, size: Rect) {
        // only update the colors if the size has changed since the last time we rendered
        if self.last_size.width == size.width && self.last_size.height == size.height {
            return;
        }
        self.last_size = size;
        let Rect { width, height, .. } = size;
        // double the height because each screen row has two rows of half block pixels
        let height = height * 2;
        self.colors.clear();

        use rand::Rng;
        let mut rng = rand::thread_rng();
        for y in 0..height {
            let mut row = Vec::new();
            let randomness_factor = (height - y) as f32 / height as f32; // More randomness towards the bottom
            for _ in 0..width {
                let base_value = randomness_factor * ((height - y) as f32 / height as f32);
                let random_offset: f32 = rng.gen_range(-0.1..0.1); // Adjust the range as needed
                let value = base_value + random_offset;

                // Clamp the value to ensure it stays within the valid range [0.0, 1.0]
                let value = value.max(0.0).min(1.0);

                let color = Hsv::new(0.0, 0.0, value); // Set hue to 0 for grayscale
                let color = Srgb::<f32>::from_color_unclamped(color);
                let color: Srgb<u8> = color.into_format();
                let color = Color::Rgb(color.red, color.green, color.blue);
                row.push(color);
            }
            self.colors.push(row);
        }
    }
}

fn init_terminal<W>(mut stream: W) -> Result<Terminal<CrosstermBackend<W>>>
where
    W: Write,
{
    enable_raw_mode()?;
    stream.execute(EnterAlternateScreen)?;
    let mut terminal = Terminal::new(CrosstermBackend::new(stream))?;
    terminal.clear()?;
    terminal.hide_cursor()?;
    Ok(terminal)
}

fn restore_terminal<W>(mut stream: W) -> Result<()>
where
    W: Write,
{
    disable_raw_mode()?;
    stream.execute(LeaveAlternateScreen)?;
    Ok(())
}

fn main() -> Result<()> {
    let mut io_stream = last::<IoStream>().unwrap_or_default();
    loop {
        io_stream = next_cycle(&io_stream).unwrap_or_default();
        if App::run(io_stream)? {
            break;
        }
    }
    Ok(())
}

@orhun

However, I just tried it and I'm getting a slightly less FPS on this branch.

That's interesting. Since terminal rendering is a complicated task, I guess its performance is depending on the terminal implementation and CPU arch (I'm using iTerm2 and older Intel Mac). If I could understand how to run your benchmark, I would be happy to run it on my side.
However, please note that the point of this PR is an improvement of memory usage. CPU usage is just a bonus.

However, please note that the point of this PR is an improvement of memory usage. CPU usage improvement I've seen is just a bonus.

Oh, got it. Fair point.

If I could understand how to run your benchmark, I would be happy to run it on my side.

I'm actually just running the code that I shared and compare the FPS's (not the best approach for benchmarking but it kinda works)
Without this change:

Details

With this change:

Details

With a second thought, there isn't much difference. But it would be nice to see the result on your side as well.

Rebased against main (Buffer moved to buffer/buffer.rs and Cell moved to buffer/cell.rs)

Merged. Thanks for the PR.

@rhysd my blog post (previously mentioned) is out now if you want to give it a read: https://blog.orhun.dev/stdout-vs-stderr/
Thanks for your contribution!


Estimated MaxCLL changes based on window size

Thanks for making this great piece of software! I use it all the time to help figure out what's going on in HDR space.
For some reason, the Estimated MaxCLL calculation changes pretty drastically based on the window size. I've noticed it report 300 nits darker if the window is smaller in images that are 600 nits peak.
Is this because it composites the current frame and grabs the brightest pixel rather than looking at the full-res image?

Sorry for the delay in responding.
Unfortunately, I don't have an explanation for your reported behavior. The MaxCLL calculation DOES do a downscale, but this is based on the source/image's dimensions and is intended to bound the cost of the calculation for very large image sizes. Right now it caps the larger image dimension to the lesser of 0.5 and 2048 pixels.
The window size handler explicitly is supposed to not recalculate MaxCLL.
Do you mind sharing more repro info - is there a particular image that shows the change, certain specific window pixel sizes that trigger this (even a guestimate), also your GPU, driver and display info (a dxdiag would be sufficient).
Thanks!


readme: change readme for a test pr.

readme: change readme for a test pr.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Adding ability to supply region with a particular step and adding new…

… steps for ECR and Kinesis.  Adding another step for s3 upload with KMS

First of all, sorry for the delay. GitHub notifications seem to be broken.
Thank you for your effort on this topic. Imho there are too many changes in one PR which makes review and comments on specific features.

I like the addition for the AWS credentials but the formatting is broken on this changes.
I think the docker pull/push belongs to the docker plugin and not the AWS plugin.
I will have a look at the Docker Login as it seems to be complex and brings additional dependencies.
Instead of the region flag, "withAWS" should be used. This prevents too complex options on the steps.

So the current state will not make it into the plugin, but if you like you can split it up and we can discuss specific parts.


Fix import in README

The example code in the README requires the full grope module to be imported.

Thank you again! :)


Backdrop + modal height issue

When the modal header text wraps (2+ lines) and viewport height is limited, modal breaks. This is due to the calculated height not including the extra lines of header text and the parent element flex positioning.

Possible solutions:

At a viewport minimum height, truncate text

Set overflow of all content to auto (scroll)

It looks like whole modal scroll is the best solution here, I know there was a larger conversation around this earlier though so want to confirm with @kybaker @mcarrano @andresgalante

Modal scroll for the smaller height would be consistent with the decision to scroll within the modal for taller modals. +1 for the first solution proposed.

@mattnolting I also like the first solution (to truncate the title). This allows us to scroll the body of the modal, only, which was the desired outcome. In actuality, the need to truncate the title should be rare as I would say it's bad design practice to have a dialog title that is so long that it needs to wrap.


[Feature Request] Add support for LD2450

Category

[x] Hardware / PCB
[ ] Software / ESPHome

Whats your idea?
LD2450 seems to be the new kid on the block, its a new mmWave sensor with the ability to track up to 3 people at once. Screek just released a new product the 2A which utilizes this new sensor and is using esphome with it.

Nice features. I've ordered one and will take a look :)


Problems compiling GlassBR_SRS.tex

Can we address the problems with compiling GlassBR_SRS.tex correctly?  From the default output of Drasil, I had to take the following steps to get correct compilation:

Copy the files into the same folder as the tex file (as mentioned in issue #214 ).  To get the figures to fit, I also had to adjust the width of the figures using [width=0.5\textwidth] within each of the includegraphics commands.

Change the double subscripts in equations from entries like $x_z_1$ to entries like $x_{z_1}$.  This occurred in the table of symbols (page 5 of the pdf, around line 200 for the tex file).

Change text entries like  X_z_1 to  X_z_1.  The original generated tex for this will compile, but it doesn't shown an underscore where one is expected, but a dot.

Interestingly the symbol $x_{z_1}$ only appears in the table of symbols.  It does not seem to appear anywhere else in the document.

1 and 2 will be addressed shortly. I don't quite understand what you mean in 3. Also, your last note makes perfect sense as its one of the issues (#396) that needs to be corrected.

Thanks for the clarification Dan.  I had forgot about the changes to the symbol map to be used for both the code generation and the SRS.  Item 3 from my original issue is about the LaTeX text.  In the table of symbols the Description field has the LaTeX code "X_z_1."  The intention is almost assuredly to print X followed by an underscore followed by z followed by an underscore followed by 1.  LaTeX however does not interpret the text this way.  I expected the compile to fail, since LaTeX uses the underscore in math mode and the text in question is not in math mode.  It does compile though, just showing weird dots, instead of underscores.  A description of how to typeset an underscore is given at:
http://www.tex.ac.uk/FAQ-underscore.html
My solution has always been simpler, instead of _, I use _ in the tex code.  In the GlassBR example, this change works fine.

I just realized you were explaining that you were escaping the underscores (I had to look at your comment source, since markdown uses backslash as an escape as well). What I saw was "instead of _, I use _", but what your source shows is "instead of , I use \"

Oops.  I should have previewed my document before commenting.  😄

Thanks @szymczdm.  I can see that you have changed the path to the figures and that you have a datafiles folder with a GlassBR sub-folder.  However, where I have been building the srs from is not the correct relative path.  I have been running the glassbr executable from within the .stack-work folder.  Should I be running something different?  I see a dist folder in the code directory, but it didn't seem to be updated when I ran make.  (The generated glassbr SRS doesn't use the new path.)
I'm sure it is a trivial thing to sort out.  I think all I need to know is where to find the glassbr executable that I should be running to create the tex file that I can then run make on.

After you run make there should be a "build" folder with the up-to-date examples within. That should have the correct relative path.

Yep, that works.  😄


MWPW-136317 Dropzone enabled on L1 verbs (plus compress) except rotate

Dropzone enabled for the following L1 verbs plus compress (exclude rotate), rest of the verbs will have it only for the first visit.
Resolves https://jira.corp.adobe.com/browse/MWPW-136317
Before:
https://stage--dc--adobecom.hlx.live/acrobat/online/sign-pdf?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/request-signature?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/pdf-editor?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/delete-pdf-pages?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/compress-pdf?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/rearrange-pdf?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/split-pdf?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/add-pages-to-pdf?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/extract-pdf-pages?martech=off
https://stage--dc--adobecom.hlx.live/acrobat/online/crop-pdf?martech=off
After:
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/sign-pdf?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/request-signature?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/pdf-editor?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/delete-pdf-pages?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/compress-pdf?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/rearrange-pdf?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/split-pdf?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/add-pages-to-pdf?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/extract-pdf-pages?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/crop-pdf?martech=off
https://puru-mwpw-136317--dc--puruadobe.hlx.live/acrobat/online/add-pdf-page-numbers?martech=off

Codecov Report

Merging #381 (80b3eb2) into stage (0d0c604) will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##            stage     #381      +/-   ##
==========================================
+ Coverage   93.30%   93.34%   +0.04%
==========================================
  Files          25       25
  Lines        2927     2930       +3
==========================================
+ Hits         2731     2735       +4
+ Misses        196      195       -1

Files Changed
Coverage Δ

.../blocks/dc-converter-widget/dc-converter-widget.js
77.06% <100.00%> (+0.16%)
:arrow_up:

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Formatted only my code changes so that it's easy to review.

@puruadobe @Blainegunn is there anything blocking here? Can we get this on stage?


preview large mdoc

texi2mdoc -I/doc wget.texi >wget.mdoc
pod2mdoc big.pod >big.mdoc
open wget.mdoc in ManDrake - no errors in editors, but in preview nothing.
for smallest mdoc - all fine.
tixi2mdoc, pod2mdoc from http://mandoc.bsd.lv/
wget.mdoc.zip

Fixed in master.


Hooded Lantern Nested Effects

Could it be possible to add nested effects for the two ways the Hooded Lantern could be used, fully opened or partially opened?
This module has been a game changer for myself and my players!

Sure thing, I'll add it

So thinking about this more, I don't know that it makes sense. An unhooded lantern is just a regular lantern, and there is already an effect for that.


jsons:// notification results in empty body

Describe the bug
JSON body with jsons:// not working. No additional keys appears to be getting added in the body.
Version
Exact version in the top right area: 0.45.5
To Reproduce
Steps to reproduce the behavior:

Go to settings, Notifications
Use a jsons url (this is a test url from RequestBin):

jsons://eo9r1d6nv9dr6um.m.pipedream.net

Add a JSON body:

{
"watchtitle": "{{watch_title}}"
}

Send test notification
The following body arrives:

{"version":"1.0","title":"ChangeDetection.io Notification - https://changedetection.server/settings#notifications","message":"https://changedetection.server/settings#notifications had a change.\n---\n\n---","attachments":[],"type":"info"}

In the debug logs, it shows:

2023/11/01 19:13:10,000 - SENDING - [{"title": "ChangeDetection.io Notification - https://changedetection.server/settings#notifications", "body": "https://changedetection.server/settings#notifications had a change.\n---\n\n---", "url": "jsons://eo9r1d6nv9dr6um.m.pipedream.net?avatar_url=https://raw.githubusercontent.com/dgtlmoon/changedetection.io/master/changedetectionio/static/images/avatar-256x256.png", "body_format": "text"}]

Expected behavior
I expected the body to look like:
{
  "version": "1.0",
  "title": "ChangeDetection.io Notification - https://changedetection.rekord.cloud/settings#notifications",
  "message": "https://changedetection.rekord.cloud/settings#notifications had a change.\n---\n\n---",
  "attachments": [],
  "type": "info",
  "watchtitle": "My Watch Title"
}

heya
is the body actually empty, or is it just missing   "watchtitle": "My Watch Title" ?

and did you try both as 'test notification' and a real notification? any difference?

There is a body sent (step 4), but it's missing the watchTitle key.
Just tried a real notification and it does add the expected keys, so it seems to be only in the test notifications.
Didn't realize that would be different, solved then.
Thanks!

Dont feel bad https://github.com/dgtlmoon/changedetection.io/issues/1898
its sort a lose-end, it doesnt store the second-most-recent snapshot yet, so it cant rebuild the 'test notification' correctly, But we're working on that

No worries, thanks for the fast reply! Great work :)


cuDNN error

Running inference with the pre-trained model on a VM (Tesla K80) gives RuntimeError: cuDNN error: CUDNN_STATUS_ARCH_MISMATCH. I set up the env with your requirements. Do you have any hints on what might go wrong here?

Could you please provide your complete pytorch environment, like cuda, cuDNN, pytorch version? I guess your incorrect cuDNN version causes this error.

Thx for getting back. Issue is solved. You were right, setting cuda=11.8 did the trick.


Prep next phase of research based on last phase Conclusions

Done when :

Identify the next focus for fable
Define more clearly research questions for  "what are barriers to adoption for DS in GC space "
Get draft on privacy/ consent forms for intended interview with PS before Sarah is on mat leave 🤞

Draft round 2 testing-of Fable goals
Collect data (locate patterns) on accessibility and usability insights from multiple single assistive technology users
Uncover barriers in your experience and understand their impact on assistive technology users
Observe distinctions between the first round of testing, and the second
Uncover what is bias and repetition, versus individual feedback
Learn new user experience/feedback on new components applied in second round of testing
What methodologies will be used?
Self guided guided tasks : 1 request (screen magnification)
Compatibility testing : 1 request (2 alt navigation, 1 screen reader, 2 screen magnification)
Prototype : 3 request (1 screen magnification, 1 alternative navigation)
Focus on documentation/DS webpage navigation, testing new components.
*different from first round, apply similar structure with new content (find patterns on usability), as repetition is necessary to collect more accurate data on user experiences on the assistive techs applied
In the same doc, describe the high priority components we want to tackle
This will be based on your discussion with dev/designers (narrowing in on which new components to test, and what medium level priorities we want to focus on this round
Objectives : Round 2 with Fable testing is meant to re-test and find the patterns on assistive technologies that has been tested in the first round of testing. Purpose of repetition is to compare distinction between different users experience with our product, and collect more data. Testing more on new tools, and components applied on the webpage, along with testing the accessibility of our webpage navigation, and learn what needs to be iterated on.


Support a new widthType property.

The widthType supports normal and fullscreen.
normal restricts the nav bar left/middle/right to fit a max-width of 1230px.
fullscreen sets the max-width to be '100%'
Note that there is existing support for a css variable that sets the maxWidth of this same element.
max-width: var(--d2l-navigation-immersive-content-max-width, 100%);
It is possible for there to be a clash if the user sets both the css variable and sets this widthType property. If the user does not set either, the max-width will be 100%

I'm a bit confused... since the variable exists, can your component/app not set that?  Or are you thinking we should keep the value of 1230px constrained to this repo as much as possible?
If so, I still think we should set the variable instead of overriding it... maybe a Boolean property that reflects to attribute instead, and hooks into css selectors?

Basically we don't want a million places all knowing about the 1230 value -- we want to be able to change the width of our pages in the future (as we've done many times in the past) without going into tons of different places.
To resolve it, I suggested to Vincent that we introduce a width-type to our page templates (we just have one right now) and to the nav components that can either be normal or fulllscreen. Normal would restrict it to the 1230px value, but we can change that in the future easily.
@wongvincent, I think there's actually only 1 usage of the variable, and unsurprisingly it's setting it to 1230. So we could clean all of this up and just get rid of it altogether. :)
I also think the code solution is even easier... you could do it with as simple CSS selector:
.d2l-navigation-immersive-container {
    /* existing CSS */
    max-width: 100%;
}
:host([width-type="normal"]) .d2l-navigation-immersive-container {
    max-width: 1230px;
}

You'll need to make sure the attribute is reflected though for that to work.

Ok cool, discussed more with Dave during standup and also @geurts.  I'm good with cleaning up the css variable altogether, and Jeff is going to update the design.d2l immersive nav docs to remove the comment about being able to set any width.
Looks like it's also used in folio-app (https://search.d2l.dev/xref/GitHub/Brightspace/folio-app/src/components/nav-bar/nav-bar.js?r=908b16f9#11), but it's an FRA so we could fix that after merge if needed.

Merged with [increment minor], but it didn't create a new release? 😕


[zh] translate document releasing docs

What type of PR is this?
/kind documentation

What this PR does / why we need it:
translate document releasing docs

i18n/zh/docusaurus-plugin-content-docs/current/developers/document-releasing.md

Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:

@wlq1212 Thanks for your contribution, please sign all commit at first.

/lgtm

Thanks @wlq1212 and welcome to Karmada community~


AzOps invoke failure results in ADO task timing out

Describe the bug
Using the AzOps Accelerator push pipeline.
A failure in the AzOps invoke as part of the Push pipeline causes the pipeline task to halt but not fail
Steps to reproduce

Remove your SPN's management group role assignment
Run a AzOps Push in Azure DevOps
Watch the deploy fail, and then subsequently hang the agent for an hour

Screenshots

Adding more evidence


Remove jQuery AJAX from the comment edit history

Removed all jQuery AJAX calls and replaced with our fetch wrapper
Tested the comment edit history list, diff, and delete functionality and it works as before

Demo using fetch instead of jQuery AJAX

rm jQuery is better to have in asap :)


CI support golangci-lint

Some common errors can be reduced by adding linter, eg: #582
linter-tool: golangci-lint https://golangci-lint.run/

#1068 added it. I have to say, golangci-lint is an amazing tool. Thanks a lot for your advice.


added onRenderFail

Related to https://github.com/prebid/Prebid.js/pull/3656
Also removed unnecessary versions of the file

PR timed out and no response. Will re-open this docs PR if code PR is re-opened and completed.


erase_if for absl:: containers should probably return the number of erased elements

Describe the bug
C++20 standard describes a function template size_type erase_if(unordered_set&, Pred) that erases elements based on a predicate and returns the number of erased elements: https://en.cppreference.com/w/cpp/container/unordered_set/erase_if .
A similar function template is available for std::unordered_map.
Abseil provides erase_if overloads for {flat_hash,node_hash,btree}_{set,map}. However, these overloads return void.
I realize that absl containers should not be considered as drop-in replacement for STL containers. However, they are designed to have a very similar interface, and I think in this case bringing the interface closer would only help with adoption of absl containers in existing codebases.
Looking at the code, it seems like this can be implemented in a generic way that doesn't depend on internal implementation of the containers. As an example, we could remember the size of the container in the beginning of erase_if and subtract the new size from it after removal.
This would however technically be an API break since someone could do compile-time logic based on return value of erase_if.
Steps to reproduce the bug
https://godbolt.org/z/7qTbhdWPe
What version of Abseil are you using?
trunk on godbolt.org
What operating system and version are you using
Irrelevant
What compiler and version are you using?
gcc 10.3
What build system are you using?
godbolt.org
Additional context

Thanks for letting us know. I'm going to gather some real-world data about the impact of the possible API break.

I've made this change. It should hopefully end up on GitHub tomorrow.

Re: ترويج https://www.patreon.com/_kataaaphhttps://t.co › DnrGObHyU2 خطابة
#سعودية #مسيار للتوصل<PHONE_NUMBER> - t.co / Twithttps://
www.patreon.com/_kataaaphter [ref:00DA0000000K0A8.5004w00002HIlo9:ref]
في الثلاثاء، ٣٠ نوفمبر ٢٠٢١ ١٠:٤٣ م Derek Mauro @.***>
كتب:

I've made this change. It should hopefully end up on GitHub tomorrow.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/abseil/abseil-cpp/issues/1065#issuecomment-982959772,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOFUR2ZW7ZM6PQKIUUMSVNDUOUSMRANCNFSM5IOFRPRA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


There is no red border on the new added property box for 'Add/Edit Entity' dialog

Storage Explorer Version: 1.33.0-dev (96)
Build Number: 20240110.12
Branch: main
Platform/OS: Windows 10/Linux Ubuntu 22.04/MacOS Sonoma 14.2(Apple M1 Pro)
Architecture: x64/x64/x64
How Found: From running test cases
Regression From: Previous release (1.32.1)
Steps to Reproduce

Expand one storage account -> Tables.
Create a table -> Click ''Add'.
Click 'Add Property'.
Check whether there is a red border on the new added property box.

Expected Experience
There is a red border on the new added property box.

Actual Experience
There is no red border on the new added property box.

Verified this issue on the main build 20240113.2. Reproduce. Will verify it on a newer build.

The change has been merged but not verified (a package update is required). You can wait till this issue is closed before trying to verify yourself.

Verified this issue on the main build 20240120.3. Fixed.


NetCDF resource ncdump command doesn't work

After a file is uploaded into HydroShare, the system should run "ncdump" command to generate the metadata header information text file. Currently, the backup python code is used to generate the text file.  Need to recover the ncdump command call.

the system should run "ncdump"

@gantian127 try adding this to the Dockerfile and verify that it does what you want on a rebuild.
RUN apt-get update && apt-get install -y netcdf-bin

@mjstealey Thanks for commenting on this issue. I will try to do it.

@mjstealey I just tested the code and it works to call the ncdump command. Do you want me to add this line of code in the Docker file and merge it to develop branch or you want to update it directly in the hs_docker_base image? Thanks

Do you want me to add this line of code in the Docker file and merge it to develop branch

Yes, please do.
When it's time to release 1.6.10 I'll migrate your code, as well as any other new Dockerfile code, into the base image. But until that time it should be in the Dockerfile so that anyone who pulls the develop branch has it.


Install Error: Extension Error: The validator 'default' must be a function.

Source: branch: master

It's already fixed in #8523, take a pull from the master


Mac OS Brew build INSTALL DESTINATION "/usr/local" permissions error

When doing a Brew build - https://github.com/vfxpro99/usd-build-club/wiki/USD-on-macOS#building-usd-on-macos-using-brew
I get a /usr/local permissions error, see below...
=== BUILD AGGREGATE TARGET install OF PROJECT usd WITH CONFIGURATION Release ===
Check dependencies
Write auxiliary files
write-file ~/Projects/stage/build/usd.build/Release/install.build/Script-B1C0B8A4926A49DCA9E02A73.sh
chmod 0755 ~/Projects/stage/build/usd.build/Release/install.build/Script-B1C0B8A4926A49DCA9E02A73.sh
PhaseScriptExecution CMake\ PostBuild\ Rules ~/Projects/stage/build/usd.build/Release/install.build/Script-B1C0B8A4926A49DCA9E02A73.sh
cd ~/Projects/USD
/bin/sh -c ~/Projects/stage/build/usd.build/Release/install.build/Script-B1C0B8A4926A49DCA9E02A73.sh
/usr/local/Cellar/cmake/3.7.2/bin/cmake -DBUILD_TYPE=Release -P cmake_install.cmake
-- Install configuration: "Release"
-- Installing: /usr/local/lib/python/pxr/init.py
-- Installing: /usr/local/share/usd/plugins/plugInfo.json
CMake Error at pxr/cmake_install.cmake:39 (file):
file cannot create directory: /usr/local/plugin/usd.  Maybe need
administrative privileges.
Call Stack (most recent call first):
cmake_install.cmake:32 (include)
make: *** [install_buildpart_0] Error 1
Command /bin/sh failed with exit code 2
Looks like it is related to references to /usr/local in the ~/Projects/stage/build/pxr/cmake_install.cmake file:
if("${CMAKE_INSTALL_COMPONENT}" STREQUAL "Unspecified" OR NOT CMAKE_INSTALL_COMPONENT)
list(APPEND CMAKE_ABSOLUTE_DESTINATION_FILES
"/usr/local/pxrConfig.cmake")
if(CMAKE_WARN_ON_ABSOLUTE_INSTALL_DESTINATION)
message(WARNING "ABSOLUTE path INSTALL DESTINATION : ${CMAKE_ABSOLUTE_DESTINATION_FILES}")
endif()
if(CMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION)
message(FATAL_ERROR "ABSOLUTE path INSTALL DESTINATION forbidden (by caller): ${CMAKE_ABSOLUTE_DESTINATION_FILES}")
endif()
file(INSTALL DESTINATION "/usr/local" TYPE FILE FILES "~/Projects/stage/build/pxrConfig.cmake")
endif()

I got around this in a pretty questionable way, by chowning /usr/local and suppressing System Integrity Protection. It's probably a good idea to modify the brew build to build to a non /usr/local location. Any thoughts?

In the build-macos-brew.sh I added the line "-DCMAKE_INSTALL_PREFIX=${ROOT}/INSTALL " and that seems to give me a successful build...   See below:
cmake -G "Xcode" ../../USD
*-DCMAKE_INSTALL_PREFIX=${ROOT}/INSTALL *
-DOPENSUBDIV_ROOT_DIR=${ROOT}/homebrew
-DCMAKE_PREFIX_PATH=${ROOT}/homebrew
-DCMAKE_INSTALL_PATH=${ROOT}/homebrew
-DPYTHON_EXECUTABLE=${ROOT}/homebrew/bin/python
-DPYTHON_INCLUDE_DIR=${ROOT}/homebrew/Frameworks/Python.framework/Versions/2.7/include/python2.7
-DPYTHON_LIBRARY=${ROOT}/homebrew/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
-DPYTHON_LIBRARIES=${ROOT}/homebrew/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib

setting the install prefix is the right approach.


don't set the loading state to false if forceFetch is true

Issue: #63

Coverage increased (+0.3%) to 98.208% when pulling a49c96533b49551444ceefc42c668cf564a1d703 on forceFetch-loading-state into 69921f6f021e1825503a450ea8fd2865fb469c19 on master.

Coverage increased (+0.01%) to 97.906% when pulling afd90426ee6676b0529d0a5f23a82938a683e64d on forceFetch-loading-state into 44710e9cfecdc9da6e3fbfb82ee277a41452132f on master.


fix(): Reconciles port numbers in the ovpn client deployment

Description

Fixes #309
How Has This Been Tested?
Tested it manually using two single node kind clusters, will write test cases soon using envtest and update the PR
Checklist:

[x] The title of the PR states what changed and the related issues number (used for the release note).
[ ] Does this PR requires documentation updates?
[ ] I've updated documentation as required by this PR.
[x] I have ran go fmt
[ ] I have updated the helm chart as required by this PR.
[ ] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[ ] I have tested it for all user roles.
[ ] I have added all the required unit test cases.
[ ] I have verified the E2E test cases with new code changes.
[ ] I have added all the required E2E test cases.

Does this PR introduce a breaking change?

one issue notices the commits are unverified (No user is associated with the committer email). We require every contributor to certify that they are legally permitted to contribute to our project. A contributor expresses this by consciously signing their commits,
please follow this link
https://github.com/kubeslice/worker-operator/blob/master/CONTRIBUTING.md#contributor-compliance-with-developer-certificate-of-origin-dco

Thanks for reviewing @Rahul-D78

@kon3m can we put PR to ready to review.

@narmidm i have found an issue with this fix where in some cases distinct ports do not get assigned to the client deployments or to put in another way both the clients are connected to the same nodePort which should not happen. Thats the reason i have converted the PR to draft. Now i have updated the PR adding the fix for that as well, could you please review it?

@kon3m , can you rebase with the latest master. it has some fixes of E2E pipeline. After That I will trigger the E2E again.

@narmidm done, i have just rebased with master and pushed.

@narmidm could you please trigger the E2E pipeline?

report link  'https://kubeslice.github.io/e2e-allure-reports/Kind-worker-operator-2024-03-21T08:01:56-master-444/index.html'

report link  'https://kubeslice.github.io/e2e-allure-reports/Kind-worker-operator-2024-03-27T07:14:18-master-445/index.html'

report link  'https://kubeslice.github.io/e2e-allure-reports/Kind-worker-operator-2024-03-29T13:01:53-master-446/index.html'

Opened a new clean PR


Close the connection but not the client

What is the purpose of the change
Close the connection but not the client
optimize #13546
Brief changelog
Verifying this change

Checklist

[x] Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Add some description to dubbo-website project if you are requesting to add a feature.
[ ] GitHub Actions works fine on your own branch.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (1bdbfd8) 70.37% compared to head (e22b021) 70.35%.
Report is 1 commits behind head on 3.2.

Additional details and impacted files
@@            Coverage Diff             @@
##              3.2   #13600      +/-   ##
==========================================
- Coverage   70.37%   70.35%   -0.02%
==========================================
  Files        1606     1606
  Lines       69997    69999       +2
  Branches    10098    10098
==========================================
- Hits        49260    49250      -10
- Misses      16111    16115       +4
- Partials     4626     4634       +8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Implement UKF

Implement UKF, UK smoother, unit tests, pendulum demo.

Hi,
I see that you have been implementing UKF, EKF etc as fully different methods.
I have done some work of coding this in a more general fashion here: https://github.com/EEA-sensors/sqrt-parallel-smoothers/tree/main/parsmooth/linearization
It follows the "generalised statistical linear regression" formulation. See this for example: https://scholar.google.fi/citations?view_op=view_citation&hl=sv&user=q0rtB0EAAAAJ&citation_for_view=q0rtB0EAAAAJ:IjCSPb-OGe4C
Doing it in a similar fashion would IMO reduce boilerplate and allow for more precise testing than with fully independent implementations. WDYT?

Hi Adrien! Thank you for your comment,
That's a great idea; we've an nlgssm folder that contains models and tests common to all non-linear Gaussian ssms, which has been ever-growing precisely because of the commonality that you mentioned.
Factoring out all but the linearization method is a great idea, @murphyk WDYT?

I like the simplification, both in code and in concepts! Great suggestion!

Hi @AdrienCorenflos . I totally agree. I have actually rewritten my book chapter on SSM inference to use this more abstract formulation, and I think we should refactor the code to follow suit, since it is much more modular and elegant.
Here is the description of Kalman filtering in my new notation:

Here is the corresponding version for EKF

and UKF

and finally the method you mentioned, which works with exponential family likelihoods using generalized statistical linear regression:

I will upload a new version of my full book shortly, which has all the details ;)

I just uploaded the latest version of the book to https://github.com/probml/pml2-book/releases/tag/2022-07-13.
Please see sec 8.3-8.8 - I would love feedback.

I think we can close this now that we've merged Peter's code.