Port to LVGL v7.0

List of necessary changes:

[x] Update lvgl submodule.
[x] Update lvgl example submodule.
[x] Update lvgl configuration file.

Update lvgl submodule.

It's in dev-7.0 branch

Update lvgl example submodule.

It's in rework-7 branch

Update lvgl configuration file.

It'd be best to create a new one based on lv_conf_templ.h in dev-7.0.
Could you make a build test to see if there are any serious issues?

I will do the tests here: https://github.com/C47D/lv_port_esp32_v7
Once we get it working I will update this repo

First teaser, still some work to do i think

What is the size and resolution of the display and LV_DPI?

Size of the screen is 320*240, LV_DPI is 100.

The demo recognizes if it's a small, medium, large or extra large display and sets the layouts accordingly. This display should have been recognized as small.
What is the size of the display (in inches). We can calculate the real DPI from it.
Besides, please modify LV_DISP_SMALL_LIMIT to 25 in lv_conf.h.

The LCD size is 3.2" (inches), ok I will modify LV_DISP_SMALL_LIMIT to 25 and upload a pic.

So the actual DPI is: `sqrt(320^2 + 240^2) / 3.2 = 125.
And width of the display is 320/125 = 2.56".
If you set to LV_DISP_SMALL_LIMIT to 25 lvgl will consider > 2.5" display as medium-sized and create this ugly layout.  So please set it to 30 (3.0") instead of 25.

Much better

Awesome, thank you!
I'll recalculate the default size limits.
You can enable LV_USE_PERF_MONITOR in lv_conf.h to see the current FPS and CPU usage.

I'm getting 33FPS, with CPU from 18% to 28% and peaks of 34%.

@C47D I'm curious; what FPS do you get at the moment when you scroll past the gauges in the Values tab? I'd like to compare with what I see on STM32F7 (I get 14-18 FPS while scrolling; it increases after that).

Hi @embeddedt, this particular display doesn't have a touch controller, I will try to setup another display to measure the FPS values you've requested.

@C47D I've modified the demo in test/no-tp branch. It changes between the tabs automatically and shows the gauges on the second tab.

Hi, I took a video of the demo using the test/no-tp branch.
https://youtu.be/BJv-vr03RsM

Thank you!
There is still huge FPS drop when the gauge is fully redrawn. At least refreshing the needle only is not that bad.
I have a few questions:

What is the size of the display buffer?
Do you use 2 buffers with DMA in flush_cb?
Have you enabled -Os or -O3 optimization?

Hi, I'm not at home right now I will update my reply with the data you requested.
I had an issue with the demo, the demo stopped working after some minutes and the screen is stuck, I have tried to replicate the issue but it stopped at different times. I will try to add the log functions to lvgl so I can know when it stops.

the demo stopped working after some minutes and the screen is stuck

I also saw this happen once on my STM32F7, but I didn't have time to investigate further. It was a few days ago.

Added some printf debugging points to see where the display is getting stuck and I can't replicate the issue, the demo has been running for almost an hour, but noticed that the lv_tick_task is still running.
There's also a new issue on the lvgl repo which uses the same dev board as me and doesn't report the issue Dev-7.0 performance experiments

I also saw the freeze but I thought it's a bug in my hacky display driver. I've already debuged that it stops on a while(vdb->flushing); I still don't know if lv_disp_flush_ready is not called or it's called  but as it is called from an interrupt (for all of us) it might mess up things somehow. I mean some assembly level you-never-find-it bug.
I suspect the latest because while(vdb->flushing); reads a  bit field which can be quite complicated on assembly level.

I suggest running the lv_demo_stress() which makes much more drawing.

I call lv_disp_flush_ready directly from the flush_cb function, so it can't be entirely an interrupt-related issue.

I call lv_disp_flush_ready directly from the flush_cb function, so it can't be entirely an interrupt-related issue.

Ah, good to know. I thought you are using the DMA based flush_cb.

I call lv_disp_flush_ready from a callback when the SPI finished transferring the data via DMA.

@kisvegabor Do you know if it's possible to get a backtrace when the application is stuck? I'm searching some tutorials to debug the esp32 chip with gdb.

If you have debug symbols available and can attach to the device with GDB after it's already running, you should be able to interrupt execution, type bt, and get a decently usable backtrace.

It seems @embeddedt knows it much better than me :slightly_frowning_face:

I added some debug code and will run the stress demo all night, and hopefully it'll freeze.

I think I found the issue but the picture is not perfect yet.
These are the interesting parts of lv_disp_buf_t. So there are 4 bits next to each other. When going to the next part to refresh lvgl might write the last 3 fields. So what I suspect is:

When e.g. last_area is set, first the whole bitfield is read to a register
An interrupt comes and set flushing = 0
In the register read in 1) the last_area is set but in the register,flushing is still 1 because it was cleared in the "real" variable.
The register is written back and it overwrites the flushing bit.

So it's a typical Read-Modify-Write issue.
It all makes sense but @embeddedt said he doesn't use interrupt in the flush_cb.
So @embeddedt, are you sure you weren't using an interrupt based driver when it froze for you?
I've pushed a trivial fix. Let's see if it helps.

Thanks for the explanation @kisvegabor, I've set the tools to be able to debug the demo-application (I'm still using the demo-widgets app), I'm running it with gdb so I can backtrace when the application gets stuck.
The "problem" I see is that I we don't know how much time it will take to the application to stall, so how much time should we run the application with the fix to be sure it worked as expected?

are you sure you weren't using an interrupt based driver when it froze for you?

100% sure. I don't have access to interrupts in the context that LittlevGL runs in. My setup is a bit unique so it's quite possible that the issue isn't within LittlevGL (although I didn't experience this with 6.1 or earlier snapshots of 7.0).

how much time should we run the application with the fix to be sure it worked as expected?

We have several weeks before release, so I'm pretty sure one of us will run into the issue if it's still present. I'd say let it run for a few hours and if nothing goes wrong, we can consider it's fixed for now.

@kisvegabor I've fixed an error on the lv_examples repo that I found when trying to run the stress demo, I sent a pull request to that repo.
Is this the expected behavior of the demo? https://youtu.be/7goD6lRqTLc

I think that's what it's supposed to look like - the idea is that it randomly creates and moves a bunch of objects around to try and trigger weird bugs like this one.
I've had lv_demo_widgets running for a few hours now - it hasn't crashed. I think I can safely say the freeze is gone for me.

I also haven't hit the bug again, I will leave the demo running overnight.
You guys know lvgl better than I do, is there any particular reason why flushing and flushing_last are ints instead of uint32_t?

@kisvegabor I've fixed an error on the lv_examples repo that I found when trying to run the stress demo, I sent a pull request to that repo.
Is this the expected behavior of the demo? https://youtu.be/7goD6lRqTLc

Thank you for the PR. Yes, it should look something like this. You can increase TIME_STEP in lv_demo_stress.c (e.g. to 200) to see more.
It was running for me for more than 8 hours. So it really seems to be solved. :tada:
Okay, let's turn back to the original topic: v7 on ESP.
@C47D seemingly it's running well on ESP. It could be faster though... On STM32F7, in general, I measured higher FPS with v7 compared to v6, but on ESP for a 320x240 TFT, I'd expect higher FPS.
Could you answer these questions, please, to exclude some possibilities?

@C47D

I do have debug symbols available, but it's a pain to setup the ESP32 tools on Windows, I was using the WSL but can't run openocd there, so i have to setup the toolchain in Windows directly :/.

actually you  can or use https://visualgdb.com/

Hi @kisvegabor,

What is the size of the display buffer?
For this particular display is LV_HOR_RES_MAX * 40, so 320 * 40 bytes, 12.5KBytes.

Do you use 2 buffers with DMA in flush_cb?
This is the flush_cb of the display driver available on my dev kit:
https://github.com/littlevgl/lv_port_esp32/blob/d124fe22a99580c47b7e45f449faf028390d4353/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c#L150-L177

But we configure with 2 buffers the display buffer:
https://github.com/littlevgl/lv_port_esp32/blob/d124fe22a99580c47b7e45f449faf028390d4353/main/main.c#L71-L74

Have you enabled -Os or -O3 optimization?

No, the default project is being compiled without optimizations and debug symbol generation enabled, here are the possible optimization levels available in esp-idf.

What is the speed of the SPI?

The speed of the SPI is 40MHz for this particular driver.

Can you try it with -Og and see what happens? That should already be an improvement over no optimization.

@embeddedt the project is being compiled with - 0g, I can try the other options tho if you want me to.

@C47D
Just a minor correction: one buffer has 320402 byte = 25 kB (because there is 2 bytes/pixel)
Could you try -O2 and -Os too?

I usually don't use -Og so tested how it compares to others on my STM32F7 dev board:
-O0: 7 FPS
-Og: 18 FPS
-O2: 25 FPS
-Os: 24 FPS

So hopefully you also will see ~50% performance boost.

@C47D
Did you do anything do add the -O2 setting? Mine shows only debug amd release.

@barbiani I'm using the master branch of the esp-idf, maybe is because of that.

@kisvegabor this is the demo stress compiled with -02
https://youtu.be/lTgXsJjYHk8

actually you can or use https://visualgdb.com/

Thanks for the information @barbiani, I ended up running openocd on the Windows prompt and gdb on WSL, but I had to change the drivers for the FTDI chip on windows to be able to connect openocd to it.
I debugged the application and it ran for about 5 hours with no issues, so i think the patch @kisvegabor did worked.

While writing a touch driver I have found that lvgl crashes with touches
outside of the screen. Meaning coordinates bigger than x and y resolutions.
On Sat, Apr 25, 2020, 18:52 Carlos Diaz<EMAIL_ADDRESS>wrote:

actually you can or use https://visualgdb.com/
Thanks for the information @barbiani https://github.com/barbiani, I
ended up running openocd on the Windows prompt and gdb on WSL, but I had to
change the drivers for the FTDI chip on windows to be able to connect
openocd to it.
I debugged the application and it ran for about 5 hours with no issues, so
i think the patch @kisvegabor https://github.com/kisvegabor did worked.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-619445592,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJDE4GCIGJK25NZL7WTRONLRBANCNFSM4MNR4RYA
.

I debugged the application and it ran for about 5 hours with no issues, so i think the patch @kisvegabor did worked.

Awesome! :slightly_smiling_face:

While writing a touch driver I have found that lvgl crashes with touches
outside of the screen. Meaning coordinates bigger than x and y resolutions.

I'll check it!

While writing a touch driver I have found that lvgl crashes with touches
outside of the screen. Meaning coordinates bigger than x and y resolutions.

I'll check it!

Should work now.

@C47D

this is the demo stress compiled with -02

It seems to me it's quite the same FPS as -Og.
Sending 320 * 240 * 16bit with 40 MHz takes 30 ms (33 FPS) so it should not be a limiting factor (as it happens in parallel with rendering). However, a 200 MHz MCU should be much faster with a 320x240 TFT. It'd be awesome to see how does it look on an oscilloscope to send an empty screen to the driver (CLK would be enough). I know it needs more effort to do, so I'm just writing it, in case you or @barbiani  has time, interest an oscilloscope to measure it. :slightly_smiling_face:

@kisvegabor, sorry if i say something silly but, aren't we limited by the spi clock and not the cpu clock?
I don't have an oscilloscope at home to test it :/, I'm going to the office until next Wednesday so i can test it until then, seems like i need a better logic analyzer :)
Regards

sorry if i say something silly but, aren't we limited by the spi clock and not the cpu clock?

From the quick calculation above, we can see that it takes only 30 ms the send a full screen image. So, it limits the FPS to 33. If data sending to the display happens in the background (using DMA) the MCU is free to deal with rendering. If rendering takes < 30 ms we are still limited by the SPI clock speed, but the limit is still at 33 FPS. So is we see 16 FPS it means rendering took 60 ms. Which is way too much for a so small display and a 160 (or 240?) MHz MCU.
Could you try the widgets demo again with -O2? The stress demo could be misleading because it's not a realistic load.

Thanks for the explanation @kisvegabor, I'm compiling the widgets demo with -O2, will upload the video asap.

Here's the video for demo widgets compiled with -O2 https://youtu.be/6FWhe9Ovd6k

@C47D Thank you for the video. It still looks quite the same FPS as with -Og. :(
I'm looking forward to seeing the oscilloscope measurement on Wednesday! :)

@C47D
I've added some minor fixes to lv_demo_bechmark.
Could you try it and share a photo/video about the summary screen at the end?

I ran lv_demo_benchmark on STM32F746 (without the GPU). This looks like a really great tool. Here's the results I'm seeing. For brevity I'm only including the ones that either stuttered visually or have poor FPS numbers.

Small/medium text - 18 FPS
Shadows -  ~15 FPS when small, ~11 FPS when an offset is added, <7 FPS when they're large. They seem to be very taxing.
Image (A)RGB rotate/zoom anti-aliased - ~11 FPS (regardless of opacity or alpha)
Medium/large compressed text - 8/4 FPS, respectively
Substr. shadow - 9 FPS
Substr. text - 15 FPS

I should have mentioned before that all of my tests thus far without GPU are done with a custom driver that does direct copies to the framebuffer in flush_cb, and not with the official TFT driver, which uses standard DMA in flush_cb. I only use the official TFT driver to test with the GPU.
Results with the official TFT driver (GPU enabled):

"Opa speed" increased from 79% to 91%.
All uncompressed text now renders at ~11 FPS regardless of size.
Circle borders now render at 16 FPS (they weren't on the "Slow but common" list before).
Circles without opacity render at 10 FPS, but with opacity they drop to 13 FPS.
Shadows now render at 8 FPS when small, 5 FPS when an offset is added, and 2 FPS when they're large. :confused: It seems that the GPU does a worse job with this (2-3x the speed without GPU).
Image RGB renders at a surprising 173 FPS. Is this possible?
Rotated images render at 15-20 FPS, adding anti-aliasing cuts that in half.
Compressed text renders at 8 FPS when small, 6 FPS when medium, and 2 FPS when large.
Substr. shadow - 4 FPS

@embeddedt
It'd be interesting to measure with speed/gpu branch and enable LV_USE_GPU_STM32_DMA2D to use the builtin GPU support.
I've added LV_SHADOW_CACHE_SIZE to lv_conf.h to speed up shadow drawing. You should set it to 10000.
In general, it's strange that I see 2-3 times better FPS on STM32F769 which is quite the same MCU.

Still disabled caching? (It'd explain the slowness)
What is the optimization level? (-O3 for me)
What is the display buffer size?
Display buffers are in internal RAM?
Are you using the latest lvgl and lv_examples versions?

@C47D

The ESP dev board there is no external RAM, right?
I saw that there is some kind of flash caching in ESP but still don't know what it is really good for. Do you know anything about it?
I read this doc and it seems it can cause some speed up if we'd place the performance-critical functions to IRAM.

It'd be interesting to measure with speed/gpu branch and enable LV_USE_GPU_STM32_DMA2D to use the builtin GPU support.

Will do.

Still disabled caching? (It'd explain the slowness)

Yes; that's probably why it's so slow (although I would hope that it can perform reasonably even without the cache).

What is the optimization level?

I'm using -Og at the moment.

Are you using the latest lvgl and lv_examples versions?

Always. :slightly_smiling_face:

What is the display buffer size? Display buffers are in internal RAM?

Thanks for checking that. I had forgotten to increase the size in the demo project, so it's currently at about 1/27th of the display size. We're probably losing a lot of performance to all the style calculations.
I'll increase the buffer size and optimization level, change LV_SHADOW_CACHE_SIZE, and report back with the results from speed/gpu.

@kisvegabor @embeddedt how often should I call lv_tick_inc? Right now it's called every 10ms.

In general, it shouldn't matter too much, however it's better to keep below 5 ms for higher precision.
In this case, when lvgl needs to measure 10-100 ms (the rendering time) the 10 ms tick resolution can cause some measurement error. E.g. if the real rendering time is 45 ms (22.2  FPS), lvgl can measure i as 40 ms (25 FPS) or 50 ms (20 FPS).

@embeddedt
I forget t answer it: "Image RGB renders at a surprising 173 FPS. Is this possible?"
Yes, it's possible because the benchmark measures the rendering time and calculates the FPS from it. So if something was rendered in 10 ms, it will be displayed as 100 FPS. Although, the real FPS is limited by LV_DISP_REFR_INTERVAL. The performance monitor (right bottom corner) takes this limit into account, but the benchmark doesn't.

I was also updating the esp32 port to dev-7.0 and today I added an adc touch driver to my wrover kit here
Just bought a cheap touch panel and hooked up to some gpios. Works quite well and has stable readings.

@barbiani I didn't knew that was possible, it's nice to know is worked quite well.

Here are two measurements of the MOSI line.
Visuals tab of lv_demo_widgets with LV_INDEV_DEF_READ_PERIOD and LV_DISP_DEF_REFR_PERIOD set to 15. So 66FPS.
Compiler flag -Os
TFT SPI at 60Mhz
Cores at 240Mhz
Flash SPI at 80Mhz
Gauge animating in the middle of the page:

Gauge animating and scrolling:

Does every horizontal division is 20ms?

Does every horizontal division is 20ms?

Correct.

Thank you @barbiani!
I suppose there were ~10 FPS (100 ms/frame) while scrolling.
There is one thing I don't understand. One "burst" on the trace is ~ 10 ms wide. And there is 20 ms between the start of the bursts. If you also have a 320x240 display with 40 lines of buffer (1/6 display) it makes sense because  20 ms x 5 = 100 ms (x5 instead of x6 because the top is not refreshed). So one burst should be a sending of an 1/6 part.
But why does it take so long to send 1/6 display?
320x40 =  12,800 pixels -> 204,800 bits. To send it on 60MHz should be only 3.4 ms.
I've just found this post which says that code placed in SPI flash can be read with 16 MB/s throughput (4 M instruction/sec I guess). If I understand it correctly it's very very slow. I know there is caching but realistically every new drawing function needs to be reloaded when called. It seems it'd help to move critical function into IRAM. If you agree I can a define to lv_conf.h the prefix these functions.

Thank you @barbiani!
I suppose there were ~10 FPS (100 ms/frame) while scrolling.

It shows 16 to 20 fps scrolling the animated gauge.

There is one thing I don't understand. One "burst" on the trace is ~ 10 ms wide. And there is 20 ms between the start of the bursts. If you also have a 320x240 display with 40 lines of buffer (1/6 display) it makes sense because 20 ms x 5 = 100 ms (x5 instead of x6 because the top is not refreshed). So one burst should be a sending of an 1/6 part.
But why does it take so long to send 1/6 display?
320x40 = 12,800 pixels -> 204,800 bits. To send it on 60MHz should be only 3.4 ms.

Here is a better view of the clock line. Shows 2 buffer updates taking roughly 10ms.

I've just found this post which says that code placed in SPI flash can be read with 16 MB/s throughput (4 M instruction/sec I guess). If I understand it correctly it's very very slow. I know there is caching but realistically every new drawing function needs to be reloaded when called. It seems it'd help to move critical function into IRAM. If you agree I can a define to lv_conf.h the prefix these functions.

I have two SPI options set in the sdk:
[*] Place transmitting functions of SPI master into IRAM
-*- Place SPI master ISR function into IRAM

I do not think that going further is worth the trouble. The performance is good. Unless the SPI driver is doing busy waits I say we should leave as it is.

@barbiani
Wow, that's really impressive!
The 10 ms update time is measured on the scrolling gauge?
I'm really curious about the performance impact of placing critical function to IRAM so I added  LV_ATTRIBUTE_FAST_MEM to lv_conf.h. Could you try the speed with #define LV_ATTRIBUTE_FAST_MEM IRAM_ATTR?

@barbiani  Can you try with this changes, polling disp_spi_is_busy is done inside disp_spi_send_data already so i think it's not necessary to poll for it in the ili9341_send_cmd function.
I don't know if it's possible to send everything in one transaction (setting /CS low, sending all the commands and data and setting /CS high again) we would need to check the driver datasheet for it.
Another way to improve it is acquire the spi bus at the beginning of the ili9341_flush function and release it when we are done sending the colors, see Bus Acquiring
diff --git a/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c b/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c
index 10847d0..7a8db1e 100644
--- a/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c
+++ b/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c
@@ -128,15 +128,18 @@ void ili9341_init(void)

 void ili9341_flush(lv_disp_drv_t * drv, const lv_area_t * area, lv_color_t * color_map)
 {
-       uint8_t data[4];
+       uint8_t data[] = {
+               (area->x1 >> 8) & 0xFF,
+               area->x1 & 0xFF,
+               (area->x2 >> 8) & 0xFF,
+               area->x2 & 0xFF
+       };
+
+       uint32_t size = lv_area_get_width(area) * lv_area_get_height(area);

        /*Column addresses*/
        ili9341_send_cmd(0x2A);
-       data[0] = (area->x1 >> 8) & 0xFF;
-       data[1] = area->x1 & 0xFF;
-       data[2] = (area->x2 >> 8) & 0xFF;
-       data[3] = area->x2 & 0xFF;
-       ili9341_send_data(data, 4);
+       ili9341_send_data(data, sizeof data);

        /*Page addresses*/
        ili9341_send_cmd(0x2B);
@@ -144,14 +147,11 @@ void ili9341_flush(lv_disp_drv_t * drv, const lv_area_t * area, lv_color_t * col
        data[1] = area->y1 & 0xFF;
        data[2] = (area->y2 >> 8) & 0xFF;
        data[3] = area->y2 & 0xFF;
-       ili9341_send_data(data, 4);
+       ili9341_send_data(data, sizeof data);

        /*Memory write*/
        ili9341_send_cmd(0x2C);

-
-       uint32_t size = lv_area_get_width(area) * lv_area_get_height(area);
-
        ili9341_send_color((void*)color_map, size * 2);
 }

@@ -178,21 +178,18 @@ void ili9341_enable_backlight(bool backlight)

 static void ili9341_send_cmd(uint8_t cmd)
 {
-         while(disp_spi_is_busy()) {}
          gpio_set_level(ILI9341_DC, 0);         /*Command mode*/
          disp_spi_send_data(&cmd, 1);
 }

 static void ili9341_send_data(void * data, uint16_t length)
 {
-         while(disp_spi_is_busy()) {}
          gpio_set_level(ILI9341_DC, 1);         /*Data mode*/
          disp_spi_send_data(data, length);
 }

 static void ili9341_send_color(void * data, uint16_t length)
 {
-               while(disp_spi_is_busy()) {}
     gpio_set_level(ILI9341_DC, 1);   /*Data mode*/
     disp_spi_send_colors(data, length);
 }

@barbiani
Wow, that's really impressive!
The 10 ms update time is measured on the scrolling gauge?

I'm really curious about the performance impact of placing critical function to IRAM so I added LV_ATTRIBUTE_FAST_MEM to lv_conf.h. Could you try the speed with #define LV_ATTRIBUTE_FAST_MEM IRAM_ATTR?

I can not tell anymore. Need better measurement setup as a possible speed up is small to see.
Getting 33 fps scrolling the gauge (~15ms to send the two buffers).
Average under 1mS to update the needle.

@C47D

@barbiani Can you try with this changes, polling disp_spi_is_busy is done inside disp_spi_send_data already so i think it's not necessary to poll for it in the ili9341_send_cmd function.

I was thinking that we could avoid going through the queues during the buffer setup code. 100us (for just a few bytes) per transfer is lost in these sections.

Yep, we can avoid them, can you add the code for that on disp_spi_send_data?

The new widgets slide show demo averages at 25fps. I think that there is a problem with the animations if the cpu usage is 100%. If you leave the demo running, the pages will eventually get stuck at the bottom.
I can see that the page is not moving up/down anymore. Screen looks static, but the spi bus is really busy. Very different than what I see with no page animation.
Defining MASK_AREA_DEBUG makes it very clear. The tab page is not moving but its contents are being redrawn.

Can you take a video of it?
Does the time it takes to get stuck consistent?
This also happen when there's no automatic slide of the tabs?
This issue happened before, and there was a fix for it (13dd42fd0bb61fc65a6d317e093436e76be59967) but you need to make sure both lvgl and lv_examples submodules are updated.
If the submodules are not updated you can try:
Go to components/lv_examples/lv_examples, then do git fetch and then git checkout --track origin/rework.
Go to components/lvgl/lvgl, then do git fetch and then git checkout --track origin/dev-7.0

I believe it should be origin/rework-7 not origin/rework.

I think the last cherry picked commit on lv_example was merged into rework.

@barbiani

I can not tell anymore. Need better measurement setup as a possible improvement is too small to see.
Getting 33 fps scrolling the gauge (~15ms to send the two buffers).
Average under 1mS to update the needle.

I video would be really great. If you set LV_DISP_DEF_REFR_PERIOD 10 the max FPS will be limited at 100, so will the changes better.
FreeRTOS's tick should be also set to a lower value (2-3 ms?),or move lv_tick inc() to a 1 ms timer. It will give more precision on FPS measurement.
@C47D

I think the last cherry picked commit on lv_example was merged into rework.

It seems it's in rework-7: https://github.com/littlevgl/lv_examples/commits/rework-7

Yep, it is rework-7, sorry about the confussion, i'm at work and have to finish a prototype tomorrow 😄

It seems that the animation tries to go past the page bottom... it can not, but still wants to refresh it.
1 and 2

Thanks, @barbiani.
I've fixed it in lv_examples

@barbiani I just found out this: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/spi_master.html#transactions-with-data-not-exceeding-32-bits
Transactions with Data Not Exceeding 32 Bits

When the transaction data size is equal to or less than 32 bits, it will be sub-optimal to allocate a buffer for the data. The data can be directly stored in the transaction struct instead. For transmitted data, it can be achieved by using the tx_data member and setting the SPI_TRANS_USE_TXDATA flag on the transmission. For received data, use rx_data and set SPI_TRANS_USE_RXDATA. In both cases, do not touch the tx_buffer or rx_buffer members, because they use the same memory locations as tx_data and rx_data.

I did that optimization, also incorporated your changes from your PR and some refactoring I need to be able to use the SD card on my board, into a new branch named dev-7.0, I'm using the Wrover Kit v4.1 board to test it, and also a ESP32 board with the SSD1306 to test monochrome displays.
Can you test it?
Here's a video of the current state of that branch:

@C47D  You can change
https://github.com/littlevgl/lv_port_esp32/blob/5d07cda76dbe5e6136c97b8fc65e70ee56e42eb4/components/lvgl_esp32_drivers/lvgl_tft/disp_spi.c#L119 to spi_device_polling_transmit(spi, &t); to halve the flush_cb setup time.
Are you using the sd card with the wrover kit?

Sure, I will do that change.
And yes I'm using the Wrover with SD card. I had to remove a couple of resistors to make it work.

Just pushed the commit, but I'm having the same performance as before, how are you configuring the ESP32?

All maxed out.
240 MHz core clock
80MHz qio spi flash
-O2 compiler flag
Spi master functions in IRAM
On Mon, May 4, 2020, 02:31 Carlos Diaz<EMAIL_ADDRESS>wrote:

Just pushed the commit, but I'm having the same performance as before, how
are you configuring the ESP32?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-623265068,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJAVY6VFRNAA5GPUHCTRPZHLPANCNFSM4MNR4RYA
.

Do you also use the LV_ATTRIBUTE_FAST_MEM?

I do!
On Mon, May 4, 2020, 04:43 Gabor Kiss-Vamosi<EMAIL_ADDRESS>wrote:

Do you also use the LV_ATTRIBUTE_FAST_MEM?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-623309061,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJCIZ2K77IWLZBWGKVLRPZW2LANCNFSM4MNR4RYA
.

Then all seems good, at least as far as I can judge. We are limited to 30 FPS by the SPI and we saw 25-30 FPS. I think it was usually 52 because of the 10 ms resolution of lv_tick. Let's see if it works like this for @C47D too.
If you have time and interest one more thing can be tried out. Set monitor_cb in the driver, printf the rendering time  in it, and make flush_cb empty (but call the lv_disp_flush_ready). This way we could see the pure rendering time. A 2-3 ms resolution for tick would be also great for the measurement. I saw that it can be easily set in menuconfig.
It's really not an important thing, just out of curiosity. :slightly_smiling_face:

I will try it. What I noticed was that the perf monitor cpu usage does not
update during the demo widgets animation. Only when it changes direction.
On Mon, May 4, 2020, 12:25 Gabor Kiss-Vamosi<EMAIL_ADDRESS>wrote:

Then all seems good, at least as far as I can judge. We are limited to 30
FPS by the SPI and we saw 25-30 FPS. I think it was usually 52 because of
the 10 ms resolution of lv_tick. Let's see if it works like this for @C47D
https://github.com/C47D too.
If you have time and interest one more thing can be tried out. Set
monitor_cb in the driver, printf the rendering time in it, and make
flush_cb empty (but call the lv_disp_flush_ready). This way we could see
the pure rendering time. A 2-3 ms resolution for tick would be also great
for the measurement. I saw that it can be easily set in menuconfig.
It's really not an important thing, just out of curiosity. 🙂
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-623530989,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJDV672DIRQSUPVFXBTRP3M6RANCNFSM4MNR4RYA
.

I tried to replicate your configuration:

All maxed out.
240 MHz core clock
80MHz qio spi flash
-O2 compiler flag
Spi master functions in IRAM

But I can't get the same results, and I get some artifacts/glitches on the display.
I will be able to continue the investigation until tonight (8 hours from now). What I saw earlier is that if the demo doesn't switch tabs I get 6% CPU usage and 66FPS.
There are some other tasks I would like to work on:

[ ] Indev, I haven't tested any touch drivers.
[ ] Multiple devices (touch and display controllers) on the same bus, I2C should be easier than SPI.
[ ] Cleanup and comment the code on lvgl_esp32_drivers directory, so is easier for users to add support for their needs.
[ ] Whatever fix we found on the SPI data transfers.

@C47D
Use .flags = SPI_DEVICE_NO_DUMMY|SPI_DEVICE_HALFDUPLEX

@barbiani Would it be possible to upload your project somewhere?

@kisvegabor It is in my fork

I am not sure why yet, but I gained 18fps (from 25 to 43) overall moving guiTask to CPU0.

Thanks for the heads up, will update #110 with this, I've been fixing some bugs on the master branch, will add those as well.

From https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/general-notes.html ...The duty of enabling cache for APP CPU is passed on to the application...

Does it mean we were running uncached so far?

I've found that the core 0 cache is enabled in the bootloader and core 1 is
enabled in the app compiled with the sdk. Right now I have no idea why it
is so much faster in core 0. 43fps scrolling the gauge page vs 25 before.
On Fri, May 8, 2020, 03:15 Gabor Kiss-Vamosi<EMAIL_ADDRESS>wrote:

From
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/general-notes.html
...The duty of enabling cache for APP CPU is passed on to the application...
Does it mean we were running uncached so far?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-625652898,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJEJ2LNGNQKFXGWLLC3RQOPPPANCNFSM4MNR4RYA
.

@barbiani Is your repo fork updated?

It is not. The way I am going with the testing will break the compatability
with the other tft controllers.
On Fri, May 8, 2020, 03:36 Carlos Diaz<EMAIL_ADDRESS>wrote:

@barbiani https://github.com/barbiani Is your repo fork updated?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-625660477,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJHYX6GYHBJFGX3PDADRQOSARANCNFSM4MNR4RYA
.

Could it be only measurement error somehow? It might be worth a try to see toggling a pin around lv_task_handler as you did in your previous measurement.

Hello, I just tried the dev-7.0 branch with my ILI9488. I had some problems with the default frequency set to 64MHz by the f6113afe568b000602a52acb9f17c65261f753c6 commit. I have to say, I'm on a breadboard, so this could be the issue here, and I won't get any PCBs for my project until late this summer. From what I have tested, 64MHz seems just above the "sweet spot" for this display driver (data sheet says max. 40MHz, but I reached 60MHz with no artifacts), since the screen contents are intelligible but heavily damaged. Could it be possible to have a custom default frequency for this driver set to 40~60MHz and maybe, in the future, having the ability to set a custom one from menuconfig? I have never used kconfig but looking at it, it doesn't seem that difficult, so I could at least try to add this small feature tomorrow or in the next few days.

It looks really helpful, thanks for the PR, hopefully more to come :)

60MHz is actually 40... and 64 is 80. The esp32 does not have fractional
clock multipliers so the last jump is from 40 to 80.
On Fri, May 8, 2020, 21:39 Carlos Diaz<EMAIL_ADDRESS>wrote:

It looks really helpful, thanks for the PR, hopefully more to come :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/lv_port_esp32/issues/101#issuecomment-626076338,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJCPP5Q6TYEM7PZCPK3RQSQ27ANCNFSM4MNR4RYA
.

@C47D
Have you ever tried running guiTask in the PRO_CPU? I can not understand why it is much faster than APP_CPU. This is true if wifi/bt are disabled.

No, I haven't. Did you tried to?

I did. About twice the FPS.
On Sun, May 24, 2020 at 6:09 PM Carlos Diaz<EMAIL_ADDRESS>wrote:

No, I haven't. Did you tried to?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lvgl/lv_port_esp32/issues/101#issuecomment-633300391,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJBXAECWZDGLWS3INYLRTGEIPANCNFSM4MNR4RYA
.

Do you think we should add a comment about that on the source code? I'm about to merge the dev-7.0 PR into master, I'm finishing some arrangements in the README

I would not add the note until someone figures it out. The cores should be the same.

I would guess that there's another task running on the APP_CPU that we're unaware of.

I've found these topics
https://www.esp32.com/viewtopic.php?t=9315
https://www.esp32.com/viewtopic.php?t=8558

It has something to do with running lvgl in a task. This approach gives the same result as running lvgl in core 0 but without the drawbacks of sharing the core with the comms stack.
Do expect 2x the performance over the code from this repository.

So instead of running a periodic timer and increase the LVGL tick on the callback of it you are increasing it at the FreeRTOS tick rate?

I am running lvgl in a timer callback (that is called in a core 1 task) and
lv tick in an interrupt. There is no semaphore.
On Mon, May 25, 2020, 12:57 Carlos Diaz<EMAIL_ADDRESS>wrote:

So instead of running a periodic timer and increase the LVGL tick on the
callback of it you are increasing it at the FreeRTOS tick rate?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lvgl/lv_port_esp32/issues/101#issuecomment-633633357,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYLTJHMLRD63IC42WPZHQDRTKIP3ANCNFSM4MNR4RYA
.

Maybe #136 is related?

@barbiani Could you review it? I don't have access to my pc to test it right now

@C47D
Have you ever tried running guiTask in the PRO_CPU? I can not understand why it is much faster than APP_CPU. This is true if wifi/bt are disabled.

Could it be connected to this?

Timer callbacks are called from a task running on the PRO CPU.

I think @barbiani meant the task that runs LVGL, which is attached to APP CPU, not the CPU of the tick timer.

I think @barbiani meant the task that runs LVGL, which is attached to APP CPU, not the CPU of the tick timer.

Yes, I understand it. If a timer callback runs on the different (in regard to the LVGL task) core it might have speed penalty (I didn't dig into the lv_tick_inc() func so don't know how expensive it is). If the LVGL task started on the same core as a timer callback then performance is higher than if it started on an another core, so it is just a guess.

What I am experiencing is that I get better performance when lv_task_handler() is scheduled on core 0.

For this demo is OK to pin the lvgl task to CPU 0, or don't even create a new task for it. Once the users try to enable WiFi or BLE into their applications they would need to pin the lvgl to CPU 1.
Is this right?

Closed with #110 being merged into master.  Further performance improvements can be discussed in new issues.

A small optimization to prevent uneeded context switches.
uint32_t lv_delay;
while (1) {
	if (xSemaphoreTake(xGuiSemaphore, (TickType_t)10) == pdTRUE) {
		lv_delay = lv_task_handler();
		xSemaphoreGive(xGuiSemaphore);
	}
	vTaskDelay(lv_delay);
}

Nice to see that LVGL feature being used, thanks for the tip @barbiani :)

I'd add lv_delay = 10; at the beginning of the loop body. That way if the semaphore doesn't get taken the thread will always sleep for 10 ms and try again.

Good point. The idea was posted. I have it as static.
Could increase the sem take timeout... but anyway, it sleeps exactly the amount needed by lv_task_handler();


Confusing directions on the "Expressions" page in the github actions documentation

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
It's about this article: https://docs.github.com/en/actions/learn-github-actions/expressions

What changes are you suggesting?
This section of the documentation says clearly that GitHub automatically evaluates the IF conditional as an expression so "you may omit the expression syntax ${{ }}.
However right after that direction it shows an example with expression syntax inside an IF.
Additional information
No response

@fsgreco Thank you for opening an issue and linking it to your PR! ✨
We appreciate the context and screenshot, too 👍


integration with other framework

Hello,
Can you tell me please json format and post/get format so I can write other file manager controller.
Thanks.

Hello,
You don't need Laravel Framework to integrate in your project, check this documentation: http://php-filemanager.rhcloud.com -> Installation of Filemanager for PHP.

Updated documentation https://github.com/guillermomartinez/filemanager-ui/wiki

If you want to create another connector for another language you have to guide the original connector filemanager-php

Thanks, I will try and I will send you the result :)


issue with authorization token is not valid at the current time

Repro steps:

Action: cosmosDB.createDocDBDatabase
Error type: Forbidden
Error Message: The authorization token is not valid at the current time. Please create another token and retry (token start time: Tue, 02 Oct 2018 12:00:46 GMT, token expiry time: Tue, 02 Oct 2018 12:15:46 GMT, current server time: Tue, 02 Oct 2018 11:54:00 GMT).
ActivityId: f5644cc2-b10e-4406-ae18-aa7f01446e0c, Microsoft.Azure.Documents.Common/<IP_ADDRESS>
Version: 0.8.0
OS: win32

@dharmiksv Does this happen consistently?  If so, can you provide more info for us to attempt to recreate it?  Thx.


chore: Update .github/workflows/audit.yaml in artichoke/nightly

Managed by Terraform.
Contents
---
name: Audit
"on":
  push:
    branches:
      - trunk
  pull_request:
    branches:
      - trunk
  schedule:
    - cron: "0 0 * * TUE"
jobs:
  ruby:
    name: Audit Ruby Dependencies
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v2

      - name: Install Ruby toolchain
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: ".ruby-version"
          bundler-cache: true

      - name: bundler-audit
        run: bundle exec bundle-audit check --update

Fix bug from prior rollout in #50.


 does not show documentation

Just FYI:
In the completion popup menu <c-n> shows lazy-doc documentation in the info popup, but if you go back by typing <c-p> the documentation does not show up again.
Looks like a Vim bug. I filed a report.
https://github.com/vim/vim/issues/14204
You can close this issue as appropriate.

Should be fixed by https://github.com/vim/vim/pull/14231

Thanks.  Is this issue fixed now by your changes to Vim?

Yes, it is fixed.


Make one grid line wider

Is your feature request related to a problem? Please describe.
I cannot easily see where is the 0 grid line in a given chart.
Describe the solution you'd like
It would be great if the 0 grid line could somehow be made more wider than the others, or to be able to configure a different color for it, etc. This way it could be easily seen for example if the temperature went below zero.

For example here, the 0 grid line is a bit thicker than the others:

You can add an annotation to the chart to distinguish the 0 line
annotations: {
  yaxis: [
    {
      y: 0,
      borderColor: '#333'
    }
  ]
}


Research solutions to workaround xrpl-py/issues/632

See https://github.com/XRPLF/xrpl-py/issues/632
Clio only supports validated ledgers and even provides a separate document describing ledger command functionality.
However, if there is a mix of rippled and clio nodes on s1/s2 then clients can have difficulties parsing the response as it may be different for rippled and clio. Let's try to find a solution to this (even if it's fixing response parsing on the library side).

Clio can use the same structure of rippled response to make sure these front lib can parse both without crash.

We can add "closed" but that would change our existing API. I'm not sure there is another way though.

@godexsoft there will be change in API when rippled version 2.0 is released. Since we will need to release clio with matching API changes, this seems like a good opportunity to unify the interface between the two.


fix: consider atime == mtime to be a non-change

Changes the file listener to treat atime === mtime as a non-change.
Currently, if these are equal, we fire a CHANGE event which causes bugs on windows (as windows emits two change events when you first write).

nevermind, looks like it breaks ubuntu now and fixes windows... 🤦♂
a problem for another day

saw you merged the v4 branch @paulmillr , awesome 🎉
ill try pick this one up tomorrow. if we can fix this bug we're pretty much ready to go i think
i could do with help on it though, especially given ill be out travelling from weds so my time will drop for a bit


Add back uneven sharding

Previous PR (https://github.com/tenstorrent/tt-metal/pull/7658) had to be reverted as it regressed performance in RN50.  Unfortunately it was pushed in while a bug had disabled RN50  and when that bug was enabled and RN50 was added back we saw it regress.
This PR applies a fix for that case:
Models Perf Pipeline (Currently Running): https://github.com/tenstorrent/tt-metal/actions/runs/8827547642

Working CI run: https://github.com/tenstorrent/tt-metal/actions/runs/8830583160/job/24244611643


Added no-text option

Added an option (-x or --notext) that prints no additional information, just the moon art, even if the height of the moon art is less than 27 lines

Should we activate this option in wttr.in/Moon also? What do you think? (say: wttr.in/Moon?x)

Sure, if that isn't too much work it would be nice. Changing the size of the moon with something like wttr.in/Moon?n32 would be nice as well.


experiment to see if assignment is broken.

this is a nonsense pr and should be ignored

Hello!  I am a robot who works on Magic Modules PRs.
I have detected that you are nathan doing a test of things.  Thanks for your contribution!  This problem is your fault in the first place.

Hello!  I am a robot who works on Magic Modules PRs.
I have detected that you are nathan doing a test of things.  Thanks for your contribution!  This problem is your fault in the first place.

Hello!  I am a robot who works on Magic Modules PRs.
I have detected that you are nathan doing a test of things.  Thanks for your contribution!  This problem is your fault in the first place.


Regenerated Recordings

[ ] The purpose of this PR is explained in this or a referenced issue.
[ ] The PR does not update generated files.

These files are managed by the codegen framework at Azure/autorest.go.

[ ] Tests are included and/or updated for code changes.
[ ] Updates to CHANGELOG.md are included.
[ ] MIT license headers are included in each file.

Pending PRs to be merged :-
https://github.com/Azure/azure-sdk-for-go/pull/16989
https://github.com/Azure/azure-sdk-for-go/pull/16978


Sidecar support

Is there any plan to officially supporting sidecar files?
There are multiple type of sidecar files: XMP, exif, etc. These files live next to the original media file and should have the same format as the data in a jpg itself. It is a good way to attach metadata to e.g: videos.
Here is how exiftool deals with it: https://exiftool.org/metafiles.html
Here is an other lib for xmp sidecar reading that has some test cases: https://github.com/PhilipF5/xmp-sidecar/tree/master/test
I also generated some:
I used exiftool with:
exiftool.exe -xmp -b IMG_5910.jpg > IMG_5910.xmp
exiftool.exe -exif-b IMG_5910.jpg > IMG_5910.exif
The output:
IMG_5910.jpg

IMG_5910.exif: IMG_5910.exif.txt
IMG_5910.xmp: IMG_5910.xmp.txt
I dropped the .exif to https://mutiny.cz/exifr/ and seems to work without an issue. .xmp unfortunately not.
Context of the feature request:
We are currently replacing exif processing (https://github.com/bpatrik/pigallery2/issues/277) for pigallery2. Your nice project looks really promising: https://github.com/bpatrik/pigallery2/issues/277#issuecomment-836948216. We would like a lib that can support sidecars too https://github.com/bpatrik/pigallery2/issues/156. Unfortunately, our only option is exiftool currently: https://github.com/bpatrik/pigallery2/pull/294#discussion_r629922316

Hello. I coincidentally wanted to implement this for a long time. It is both simple and difficult :D
Let me elaborate.
The parsers for XMP, ICC, etc... are already implemented and could be easily applied to any sidecar file. But...
Reading image files is easy because the format is deterministic. Each format starts with a signature (magic bytes). Then there's a structure of segments, each beginning with another signature - denoting what's wrapped inside that segment.
With sidecars, there's no such metadata I could use for determining the format and thus picking the correct parser. Sidecar just starts with the raw data right away. I can only try each parser, wrap it in a try/catch and wait until it fails to try another.
This is absolutely valid solution, but since exifr prides itself with performance, I wouldn't want to add this (in my eyes) inefficient code to parse().
I've been thinking and prototyping for the past two days and came up with function parseSidecar(file[, options][, formatName]). Or something similar. What do you think? Would this work for you?
It'd be just for now, until some better solution materializes. But let's be honest, there's nothing more permanent than a temporary solution :D

@MikeKovarik, why not just relying on the file name extension, e.g. .xmp or .exif ?

@MikeKovarik thank for the quick response!
From my perspective I would also prefer the best possible performance. (We put lately quite som effort to compare multiple exif readers: https://github.com/bpatrik/pigallery2/issues/277#issuecomment-840515653 ).
Options I see here is:

Try/catch everything (not the fastest)
Can read the first few bites a guess the best reader (eg.: if it starts with < then its probably an xmp). There is a chance that you make a bad assumption as this approach would not be a standard way of inferring the data structure. -> this can cause confusion -> lot of github tickets :)
I also think, using extension is a good option. If the data is not parseable based on the extension it should fail. I would keep the format an input option, so users (developers) can force a parser. It would be fast and no magic involved.

You can also take a look at https://exiftool.org/metafiles.html and try to understand how that works on different extensions. The only thing is that exiftool is really slow. During that time it can try all possible parsers twice.

Hello. I've been writing up proper response that I'll post later. In meantime. I gave shot to the "try infer from filename if type isn't explicitly" way with try/catch fallback. But since I'm about to leave train, there's no time for chit chat. I wanted to at least give you the work in progress code if you'd like to take a look at it before I speak my peace :D. So far my weekend seems busy.
For now, you can take a look at
https://github.com/MikeKovarik/exifr/blob/master/debug/sidecar.mjs
and
https://github.com/MikeKovarik/exifr/commit/20f36ae6d6810401ace7af681fd26a3a198b4fca

I had a look.
I think (without testing) the majority of the time is spent on reading the file and not parsing the result.
So doing some try-catches should not have too high cost. I would do some quick benchmarks on some general photos. (generate all possible type of sidecars for like 100 photos with exiftool and parse them with and without the try-catch trick)
I would still call out (or just simple not support) the try-catch trick. Inferring the datatype has extra cost. I think user usually knows the data they want to feed your lib, so providing the type should be free for them.
Unrelated: I can recommend using .editorconfig and lint (my tslint) in your project. It made my code way more coherent and caught some bad practices, anti-patterns. (like use const if possible, etc.). It is easy to set up and modern IDE-s can auto format the code to match the linting expectations. (Please take it with a grain of salt, I'm not a web developer in my dayjob)

Is the xmp parser function public, so that could be used in the meantime for parsing xmp sidecars, while the exifr.sidecar gets to a release?

Hello again after a long time. I'm sorry for the delayed response. I'm was looking for a new job and the multi-step hiring process with coding homeworks was putting strain on me.
I'm not so fond of solely relying on file the extension because it isn't bulletproof.
Even though the .txt at the end of IMG_5910.xmp.txt was added github. Someone else can not knowingly download the file and run with it. Or there may not be any file extension at all. I know these are edge cases but ideally I'd like there to be as little friction as possible. And like you mentioned, issues can arise from edge cases and missing out overly complex docs.
That's why there's something i liked about try/catching. Though absolutely not ideal. Performance: it's not so bad anymore. Thanks to async/await syntax it has been reworked in V8 some two years ago. It now runs at the same speed as .then/.catch. But of course, there's always some time spent feeding the data into parser while waiting for it to crash.
But I'd like to keep it as a last resort. As is in the last commit i presented. You can specify the type explicitly, or it will be infered from the filename extension (though the condition on line 13 should also contain && segType === undefined). Then I'll be adding some rudimentary XMP check at line 22 - "does the first few bytes include <?x" or something like that. And if all else fails, then there will be the try/catch fallback.
TLDR: it can be very efficient and you can avoid guessing.
Regarding the last comment: it's in master but not yet published. I thought you'd want to clone master and play with it, but I didn't release it as it wasn't the final implementation.
If the proposal is fine by you then there's not much more work to be done and I could release it soon.
Regarding linting: thanks for the note. I'm thinking about it from time to time. Bit i have a mixed feelings about it from other projects. It can be pain to setup the way I'd like it to work :D perhaps some day

Hello @bpatrik . I forgot to mention that 7.1.0 with sidecar support was released last week.

That is excellent news! Thank you! (sorry I also got carried away a bit lately, I hope you get the position, that you were wishing for :) )


insert-topic script

What
Insert-topic mongo script
How to review
Check script code is ok
Run script in mongo shell as prescribed in the readme and check if it inserts correctly.
Who can review
Anyone

Reviewed untested.


Enable cloudwatch logging for lambda deployment tf

Adds support for cloudwatch logging for the lambda functions
Fixes handling of bool variable in headers for the python lambda function
Fixes the lambda extn layer handling to trigger an update on change of extension code
Refactors the lambda_deployment module to have clear boundaries (should only handle deployment of a provided zip package)

:broken_heart: Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-11-21T06:09:22.932+0000

Duration: 6 min 2 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
101

Skipped
1

Total
102

Steps errors
Expand to view the steps failures

make build

Took 0 min 3 sec . View more details here
Description: make build

:robot: GitHub comments
Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

/test

/test

/test


Missing syscalls events when multi-event name in rule condition

Describe the bug
I want to test if falco can capture the fchownat syscall in my enviroment, here is the thing, If I set the condition
condition: container.id != host and evt.type in (fchownat) and k8s.pod.name=nginx-pod-b965fcc45-nk5xx
falco indeed can shows
{"hostname":"falco-99tt5","priority":"Notice","rule":"The program is run in a container","source":"syscall","time":"2023-01-11T07:26:36.764431585Z", "output_fields": {"container.id":"1b93d2852905","evt.args":"","evt.time":1673421996764431585,"evt.type":"fchownat","fd.directory":null,"fd.name":null,"k8s.ns.name":"demo","k8s.pod.name":"nginx-pod-b965fcc45-nk5xx","proc.cmdline":"chown root file1","proc.name":"chown","proc.pname":"bash","proc.vmsize":12684,"syscall.type":"fchownat","user.name":"<NA>"}} {"hostname":"falco-99tt5","priority":"Notice","rule":"The program is run in a container","source":"syscall","time":"2023-01-11T07:26:36.764473224Z", "output_fields": {"container.id":"1b93d2852905","evt.args":"","evt.time":1673421996764473224,"evt.type":"fchownat","fd.directory":null,"fd.name":null,"k8s.ns.name":"demo","k8s.pod.name":"nginx-pod-b965fcc45-nk5xx","proc.cmdline":"chown root file1","proc.name":"chown","proc.pname":"bash","proc.vmsize":12684,"syscall.type":"fchownat","user.name":"<NA>"}}
however if I change the condition into
condition: container.id != host and evt.type in (fchownat, stat, access) and k8s.pod.name=nginx-pod-b965fcc45-nk5xx
then only access  event was captured in falco logs
{"hostname":"falco-99tt5","priority":"Notice","rule":"The program is run in a container","source":"syscall","time":"2023-01-11T07:36:14.076586416Z", "output_fields": {"container.id":"1b93d2852905","evt.args":"mode=0(F_OK) ","evt.time":1673422574076586416,"evt.type":"access","fd.directory":null,"fd.name":null,"k8s.ns.name":"demo","k8s.pod.name":"nginx-pod-b965fcc45-nk5xx","proc.cmdline":"chown root file1","proc.name":"chown","proc.pname":"bash","proc.vmsize":364,"syscall.type":"access","user.name":"<NA>"}} {"hostname":"falco-99tt5","priority":"Notice","rule":"The program is run in a container","source":"syscall","time":"2023-01-11T07:36:14.076594852Z", "output_fields": {"container.id":"1b93d2852905","evt.args":"res=-2(ENOENT) name=/etc/ld.so.nohwcap ","evt.time":1673422574076594852,"evt.type":"access","fd.directory":null,"fd.name":null,"k8s.ns.name":"demo","k8s.pod.name":"nginx-pod-b965fcc45-nk5xx","proc.cmdline":"chown root file1","proc.name":"chown","proc.pname":"bash","proc.vmsize":364,"syscall.type":"access","user.name":"<NA>"}} {"hostname":"falco-99tt5","priority":"Notice","rule":"The program is run in a container","source":"syscall","time":"2023-01-11T07:36:14.076619050Z", "output_fields": {"container.id":"1b93d2852905","evt.args":"mode=4(R_OK) ","evt.time":1673422574076619050,"evt.type":"access","fd.directory":null,"fd.name":null,"k8s.ns.name":"demo","k8s.pod.name":"nginx-pod-b965fcc45-nk5xx","proc.cmdline":"chown root file1","proc.name":"chown","proc.pname":"bash","proc.vmsize":372,"syscall.type":"access","user.name":"<NA>"}}

How to reproduce it

install the falco helm
set the custon falco rule as condition: container.id != host and evt.type in (fchownat) and k8s.pod.name=nginx-pod-b965fcc45-nk5xx where the k8s.pod.name is the name of pod to execute chown command
enter the nginx pod and execute chown root file1
check the falco logs and record it as log_previous
change the falco custom rule into condition: container.id != host and evt.type in (fchownat, stat, access) and k8s.pod.name=nginx-pod-b965fcc45-nk5xx
enter the nginx pod again and execute chown nobody file1
check the falco logs and record it as log_multi_name
compare the two log files

Expected behaviour
log_multi_name would contains the syscall events of fchownat just as log_previous would do

Screenshots
this is the previous one

this is the multi_name log

Environment

Falco version:
Falco version: 0.33.1
Libs version:  0.9.2
Plugin API:    2.0.0
Driver:
API version:    2.0.0
Schema version: 2.0.0
Default driver: 3.0.1+driver

System info:

Cloud provider or hardware configuration:
OS: rke

Kernel:
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Installation method:

Helm falco
Additional context
I already try to use -A to get general syscall events, and if I don't specify the evt.names, fchownat can't show up as well

Hi! Thanks for reporting this issue!
I honestly think this is rather important, let's give it a try for 0.10.1!
/milestone 0.10.1

Mmh running sinsp-example directly works:
./libsinsp/examples/sinsp-example -f "evt.type in (fchownat, stat, access)" -o "%evt.type %syscall.type"

...
{"evt.type":"access","syscall.type":"access"}
{"evt.type":"access","syscall.type":"access"}
{"evt.type":"fchownat","syscall.type":"fchownat"}
{"evt.type":"fchownat","syscall.type":"fchownat"}
...

This gets fixed by: https://github.com/falcosecurity/falco/pull/2347
I don't think we will make it for 0.34.0 though :(

Since it is a Falco issue, i'll move this to Falco!


Typo causing an error in splrep

Hello everybody!
So, basically, line 512 of the file fitpack.py in scipy.interpolate is:
warnings.warn(RuntimeWarning(+_iermess[ier][0]))

The '+' is causing a TypeError:
TypeError: bad operand type for unary +: 'str'

I think it is a typo and the '+' should be removed:
warnings.warn(RuntimeWarning(_iermess[ier][0]))

Indeed. Do you care to submit a fix?

Fixed in gh-5261


[PB-2386] - Allow to edit phone and email in workspace

Ticket

https://inxt.atlassian.net/browse/PB-2386

Updates

Allow owner to edit the phone and email in the account workspace section

PR related

web: https://github.com/internxt/drive-web/pull/1356
wip: https://github.com/internxt/drive-server-wip/pull/430

This has been desestimated. The change email can be done on b2c and the phone number is not shown in any place nor we need it.


Switch to golang 1.19

What type of PR is this?
/kind cleanup
What this PR does / why we need it:
Switching to the latest go release.
Which issue(s) this PR fixes:

None
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

Updated golang to 1.19.

@saschagrunert PTAL https://github.com/kubernetes/test-infra/pull/27888

/retest

So, let's check if a rebase works for this one.

/lgtm


Compiler removes nested if statements

I have this if statement in a critical event in my bot which is used to lockdown posting tweets to a specific channel and tagging the bot in a message.
if (
	(message.channelId !== "1081589817680347186" &&
		!message.content.includes("<@1100024221339504733>")) ||
	message.author.bot
) return;

However, one of the server members pinged my bot and it posted his message on our twitter which is a huge problem for a 4.6K followers account which is used to publish information about a game. After looking at my and the compiled code I noticed that the compiler is the problem because the if in the resulting .mjs file is this
if (message.channelId !== "1081589817680347186" && !message.content.includes("<@1100024221339504733>") || message.author.bot) {
	return;
}

I've tried to reproduce the issue and can confirm this happens. Seems like an issue with the tsup transpiler. In the mean time I recommend breaking down your statements maybe into their own variables. Thanks for bringing this issue up!


Surreal() constructor not working without url

In earlier versions the Surreal constructor did not need any arguments. This allowed to declare a const like
export const DB = new Surreal();
This is not working anymore.

Fixed by #105


feat: Add ignoreUnknownField support in JsonWriter

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[ ] Make sure to open an issue as a bug/issue before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
[ ] Ensure the tests and linter pass
[ ] Code coverage does not decrease (if any source code was changed)
[ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕

We should revisit the test case for schema update while introducing this feat (related to this comment).


Panel closing unexpectedly when clicking inside and dragging and releasing outside

If the user clicks down within the panel, drags outside, and then releases, the panel closes. This causes UI friction when trying to highlight some text within a panel for example.
https://github.com/EightfoldAI/octuple/assets/77997364/b3b3feee-7e21-4ff9-a672-a631231a8b77

Good catch!


ability for enabling offboard antenna on CM4 based miners (Syncrobit)

CM4 has a dtparam called ant2 which enables an offboard antenna (instead of an onboard) on WiFi CM4s.
This is possible on Syncrobit miners to enable an offboard antenna by adding a dtparam of dtparam=ant2
https://www.jeffgeerling.com/blog/2022/enable-external-antenna-connector-on-raspberry-pi-compute-module-4
I think we can do this using the balena supervisor (or probably using device env variables) to enable the offboard antenna.
I do not think that can be done at runtime - as it is needed to be in the config.txt and parsed at boot.

this can be done using env var. config.txt vars begin with BALENA_HOST_CONFIG_
https://docs.balena.io/learn/manage/configuration/#adding-custom-configuration
We might need to ensure that we don't overwrite any exiting values.

Yes just not sure if we can do that from the device supervisor?

Closed by #567


createUserToken not working in rules on Auth0

I am trying to execute the following code in rules on Auth0
function (user, context, callback) {
  // TODO: implement your rule
  // createUserToken

  const stream = require('getstream');
  const client = stream.connect('YOUR_API_KEY', 'API_KEY_SECRET');
  const userToken = client.createUserToken(user.user_id);
  context.idToken['stream']=userToken;
  return callback(null, user, context);
}

But I am getting the following error: ERROR: client.createUserToken is not a function
This is happening on the test data itself.
gz#4501

Which version are you using ?
If you're on 5.0.0, you need to update the line such that `const stream = require('getstream').default;
5.0.1 should work without changes.

I was able to solve it using
  const stream = require('getstream@4.2.2');


Linting the code and setting up Eslint

I will set up Eslint in project and lint the code and make the code prettier

@gauravdas014 please assign me this issue

/assign

@gauravdas014 please assign me so that I can start working. I have experience in linting and eslint

@shashankmotruri please assign me this

@subhangi2731 Sorry for the late reply. You can work on the issue.


Specified public directory 'functions/dist' does not exist, can't deploy hosting to site

I'm submitting a...

[x] Bug report
[ ] Feature request
[ ] Documentation issue or request

In package

[ ] @ng-toolkit/init
[ ] @ng-toolkit/serverless
[x ] @ng-toolkit/universal
[ ] @ng-toolkit/pwa
[ ] @ng-toolkit/firebug

Current behavior

Behavior works, rendering is correctly working on view source.
When trying to deploy to firebase hosting I get: Specified public directory 'functions/dist' does not exist, can't deploy hosting to site
Expected behavior

The application would deploy to firebase hosting.
Environment

Angular version: 6.2.9

- Node version: 8
- Platform:

Others:

Hello @hmonarch
I do not have any knowledge regarding the topic but maybe a quick StackOverflow post could help you to find out what's going on.
Cheers!


Support reading RLE-encoded Boolean values in Parquet scanner【Easy】

Requirement Description
Per this discussion on the Parquet mailing list, RLE will become a valid encoding for the boolean type in parquet. We should add support for reading this.
https://mail-archives.apache.org/mod_mbox/parquet-dev/201712.mbox/%3CCAJPUwMDbGgkS1WmN8OvuuA%3DQ%2BXd%2BOwLn2XZAu7CNGF1sMVZMJg%40mail.gmail.com%3E
Mentor
@DorianZheng
Registration procedure
Step 1. Add a comment when you sign up. The comment can be a brief self-introduction and reason for signing up, including what you do, what you're good at, what you're interested in and your technology stack, and how familiar you are with StarRocks. Example:

I pick! My name is Johnny. I am from StarRocks and I have been working on database kernel development for 1 year. I am interested in OLAP, good at C++ programming, and have been using StarRocks for 6 months. I chose to solve this issue because I am familiar with execution engine.

Step 2. On your WeChat, add the StarRocks community assistant "StarRocks-1" as a WeChat friend and leave your GitHub ID. The assistant will invite you to a WeChat group based on the issue you picked.
Step 3. The mentor will communicate with all the participants who picked this issue and assign the task to the best match.
Before you start

Read the StarRocks GitHub workflow carefully, which provides the configurations of BE and FE and describes how to submit a pull request.
To learn more about StarRocks, read StarRocks documentation.

Pull request requirements

After you complete code development, upload the code to the StarRocks code repo as a pull request (PR).

PR title format: [SR GeekCamp] + PR title
Labels ：[Geek Camp 2022/feature]

After you submit your PR, comment on the status of your PR submission as requested under the Pinned Issue [StarRocks geek Camp 2022] page.

Comment format: "I have finished Issue #" + PR link id

I pick! My name is WuChang. I have been working on bigdata programming for 1 year. I am interested in OLAP, good at Java programming, and have been using StarRocks for 6 months. I chose to solve this issue because I am familiar with execution engine.

I pick! My name is eric, I have been learning on database kernel development for 1 year. I am interested in OLAP, good at C++ programming. I think it is a good chance to join the StarRocks community and read the code.


Demo is broken

Hi, just wanted to let you know the demo linked in the README is broken due to a CORS error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://s3.amazonaws.com/neocortex-js/examples-data/mnist_mlp_sample_data.json.gz. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). Status code: 404.

Bucket doesn't exists.


Add AKS setup instruction

documentation about how to create AKS for NIM deployment.

There are still a few typos/confusing verbiages I pointed out. Let's fix those real quick before we merge these docs through.


ConstructPortalEvent not firing for Nether Portals

I am currently running

SpongeForge version: spongeforge-1.12.2-2655-7.1.0-BETA-3095
SpongeVanilla version: spongevanilla-1.12.2-7.1.0-BETA-47

Issue Description
As the title says, ConstructPortalEvent is not firing during the creation of nether portals. Tested on both spongeforge and spongevanilla.
Test code below:
@Listener
public void onPortalCreate(ConstructPortalEvent event) {
    logger.info("Fired!");
}

Duplicate of https://github.com/SpongePowered/SpongeCommon/issues/1559


[Feature] Volume Shortcuts and k3d-managed volumes

Changes
Added

Volumemount-Shortcuts

Use some destination shortcuts with the --volume/-v flag that k3d automatically expands

k3s-storage -> /var/lib/rancher/k3s/storage
k3s-manifests -> /var/lib/rancher/k3s/server/manifests
k3s-manifests-custom -> /var/lib/rancher/k3s/server/manifests/custom (not K3s default: this is just some sub-directory inside the auto-deploy manifests directory which will also be parsed)
k3s-containerd -> /var/lib/rancher/k3s/agent/etc/containerd/config.toml (use with caution, K3s generates this file!)
k3s-containerd-tmpl -> /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl (used by K3s to generate the real config above)
k3s-registry-config -> /etc/rancher/k3s/registries.yaml (or just use --registry-config)

k3d-managed volumes

non-existing named volumes starting with a k3d- prefix will now be created and managed by k3d

Links
fixes #905

Need actual examples.  Tried this and got error:
(base) ~ k3d cluster create --servers 2 -v k3d-data:k3s-storage foobar
WARN[0000] No node filter specified
FATA[0000] Failed Cluster Configuration Validation:  failed to validate volume mount 'k3d-data:k3s-storage': volume mount destination doesn't appear to be an absolute path: 'k3s-storage' in 'k3d-data:k3s-storage'
(base) ~

Fix provided, see https://github.com/k3d-io/k3d/issues/905#issuecomment-1043157081


rewrite implements

httparty incompatible changes(reported in #19) make upload fail(I noticed #19 now).
And httmultiparty(https://github.com/jwagener-soundcloud/httmultiparty) is unmaintained.
I rewrite implements by using faraday gem.
And I changed interfaces of Gyazo::Client using keyword args. This contains incompatible changes.
So I bump up major version.

LGTM


Add timezone support

Currently, timezone returned as part of parsed Date objects is local timezone of the client. This timezone can vary from user to user, even on the same computer.
One of the following should be implemented,

Server's timezone would be detected and used by the Date parser
Alternatively, server's timezone should be configurable in ClientConfig

Completed in https://github.com/SUSE/saltstack-netapi-client-java/commit/a3fb6de5c3bf9f0cb122e5d0444138d007889366 . Closing


docs: Add OIDC section

Signed-off-by: Carlos Esteban Lopez<EMAIL_ADDRESS>Hey, I just made a Pull Request!

:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

@Rugvip this is intended to address the OIDC part of #10303

Not sure why the Microsite Verify check pipeline is failing...

@luchillo17,
I am working through oidc integration and the docs on that is scattered across discord, PRs, deprecation docs, and buried within the main repo code. I really hope this gets merged!

Ah the diff was just to highlight all the changes. I think everything present in the diff probably needs to be in docs, but not the diff as such.
I think OIDC is a bit of an implementation detail, speaking of azureOIDC as a name. Especially keeping the API ref in mind, as that's often internal API where you just want to auth towards the upstream service. Granted, a lot of times this will only be used for sign-in, but even then we might as well encourage that naming pattern tbh.
Given that just azure is has a lot of other meanings and already has it's own implementation, I'm thinking it's best we make up our own auth service that we use throughout. Typically I'd use acme for that as a pretend organization, so acmeAuthApiRef and 'acme' as the ID

Thank you for contributing during Hacktoberfest 🍂! You can claim the Backstage Hacktoberfest Holopin at https://bck.st/hacktoberfest-holopin 🙌🏻

I have tried what @Rugvip mentioned, after the implementation, I can login to the portal, but when I refresh it, it redirects to the login page. I have observed that I am not getting a refresh token after logging in (myprovider-refresh-token). So whenever I refresh the page I get 401 Unauthorized.
Request URL: http://localhost:7007/api/auth/myprovider/refresh?optional&env=development

@myplaybooks Probably better if you file a bug, I put in the docs what worked for me at the time, it may have changed since.


display custom link on device table (Discussion #4549)

As ...
Ozzie - Operator
I want ...
Details in the discussion with screenshots from netbox UI
Ability to select custom links in list / table view for related objects.
So that ...
Example: I created an SSH link for devices, I'd like to be able to click that link in the table view to be able to quickly open ssh sessions to multiple devices without going into the device detail view
I know this is done when...
I use netbox - sorry, not sure beyond that
Optional - Feature groups this request pertains to.

[ ] Automation
[ ] Circuits
[X] DCIM
[X] IPAM
[ ] Misc (including Data Sources)
[X] Organization
[ ] Plugins (and other Extensibility)
[ ] Security (Secrets, etc)
[ ] Image Management
[ ] UI/UX
[ ] Documentation
[ ] Other (not directly a platform feature)

Database Changes
Unsure - I don't think there would be any as it's already displayed on detail pages, etc.
External Dependencies
no new libs needed that I know of

Any chance to bump this into next version?

Also looking for similar functionality. Using Nautbot for console connections, and would be nice to be able to a click a link that opens a SSH session to the console server/port from the console connections page.


Set default behavior to create shapefile

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I don't include the flag to create shapefile and essentially lose all my inference.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Make default behavior to export shapefile.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Don't forget to include the flag.
Additional context
Add any other context or screenshots about the feature request here.

This option is going to be removed entirely. In the grand scheme of things, this requires very little overhead and offers the ability to run expensive inference with minimal output.


Documentation: Add Build and Run instructions in README

What do you want to see?
Add Build and Run instructions in the README file, to help first-time contributors understand how to run the project on their computers.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

can you assign this to me?


Automated cherry pick of #9931

Cherry pick of #9931 on release-6.5.
/cc  @devinbinnie
NONE

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.


Keep diffuse intact so that shader generation for generateNormals works

Fix for a bug reported on the forum: https://groups.google.com/forum/#!topic/cesium-dev/b3iU1vlmaxo

obj2gltf outputs KHR_materials_common which gltf-pipeline can generate shaders for
If the model has no normals it sets the emission color instead of the diffuse color. This is needed for the constant lighting equation
But if --generateNormals is set, gltf-pipeline will recreate the shaders again but this time expects the diffuse to exist.
So instead this checks if normals will be regenerated before swapping the emission and diffuse.

I consider this just a bandaid for the time being and is more of a gltf-pipeline problem. Once gltf 2.0 is in full force this also won't be an issue.

Thanks @lilleyse!  I confirmed this bug was in the last official release, so I updated changes.
We should do a release as soon as #85 is in.


IGNITE-17745 Fix race condition in CMG Manager

https://issues.apache.org/jira/browse/IGNITE-17745

LGTM


Release 1.1.0

Pending https://github.com/weaveworks/scope/pull/2026

@davkal @foot @ekimekim @bowenli @jpellizzari @tomwilkie @jml @abuehrle @pidster @errordeveloper @rade Please chip in by testing this release branch in the scenarios below and tick off what you tested afterwards (including your name at the end). Have a look at the CHANGELOG to prioritize your testing.
We need to release this ASAP (for the start of Re:Invent today).
To make things  easier to test I have pushed the image weaveworks/scope:release-1.1-5319df7
Thanks!

[ ] Start a 2 node cluster and leave a UI connected, watch for performance regression
[ ] Run a Scope cluster on ECS
[ ] Run Scope on https://github.com/microservices-demo/microservices-demo/ (Will require some of the changes from https://github.com/microservices-demo/microservices-demo/pull/437 to work )
[ ] Run Scope on Docker For Mac
[ ] Start and stop Scope in standalone mode

[ ] Linux
[ ] docker-machine

[ ] Connect probes from release candidate to Weave Cloud

[ ] Local scope in probe-only mode connects to Weave Cloud and renders correctly

[ ] dev
[ ] prod

[ ] Controls appear for containers and function correctly

[ ] dev
[ ] prod

[ ] No new errors in logging or telemetry

[ ] dev
[ ] prod

Things to think about

Kubernetes Integration
CPU and memory usage

Rebased and updated CHANGELOG to include https://github.com/weaveworks/scope/pull/2043

OK, given the time pressure and the relatively amount of changes in this release I am satisfied with the testing. I am releasing.


[SPARK-29126][PYSPARK][DOC] Pandas Cogroup udf usage guide

This PR adds some extra documentation for the new Cogrouped map Pandas udfs.  Specifically:

Updated the usage guide for the new COGROUPED_MAP Pandas udfs added in https://github.com/apache/spark/pull/24981
Updated the docstring for pandas_udf to include the COGROUPED_MAP type as suggested by @HyukjinKwon in https://github.com/apache/spark/pull/25939

Test build #112022 has started for PR 26110 at commit 0ecba8a.

Test build #112022 has finished for PR 26110 at commit 0ecba8a.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112022/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17029/
Test PASSed.

Test build #112025 has started for PR 26110 at commit 1802cbd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17032/
Test PASSed.

Test build #112025 has finished for PR 26110 at commit 1802cbd.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112025/
Test FAILed.

Test build #112029 has started for PR 26110 at commit da4f00b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17036/
Test PASSed.

Test build #112029 has finished for PR 26110 at commit da4f00b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112029/
Test PASSed.

retest this please

From a cursory look, seems fine. cc @icexelloss, @BryanCutler, @viirya

Test build #112538 has started for PR 26110 at commit da4f00b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17502/
Test PASSed.

Test build #112538 has finished for PR 26110 at commit da4f00b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112538/
Test PASSed.

Merged build finished. Test PASSed.

Test build #112944 has started for PR 26110 at commit f7b9b80.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17866/
Test PASSed.

Test build #112939 has finished for PR 26110 at commit 81713b7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112939/
Test PASSed.

Test build #112944 has finished for PR 26110 at commit f7b9b80.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112944/
Test PASSed.

Merged to master.


Wrong path name when installing packages on Git Bash for Windows(zsh shell)

When attempting to install packages via antidote in Git Bash for Windows, it fails to install packages due to path errors (seems to work fine in WSL). It would be nice to be able to use Antidote in Git Bash

Git Bash is, by default, Bash. I hate to ask, but can you confirm you are you running Zsh from bash before running antidote, or are you just hoping antidote will work in straight git bash? Antidote is, and will always be, a Zsh-only plugin manager, so I just need to confirm.

Yes, I even mentioned it in the title of the issue. I have used oh-my-zsh in git bash (ash shell) but it is brutally slow.
When I echo the $SHELL variable, it says bash but I have it set to open zsh upon load.

Yes, I even mentioned it in the title of the issue. I have used oh-my-zsh in git bash (zsh shell) but it is brutally slow.
When I echo the $SHELL variable, it says bash but I have it set to open zsh upon load.

You can ask for solutions on this GitHub Gist (maybe):
https://gist.github.com/fworks/af4c896c9de47d827d4caa6fd7154b6b
But if you are unable to be running Zsh before antidote load is executed obviously it will fail. And it maybe fail anyway because it's a weird setup using Git BASH (his name says everything...) but trying to run  Zsh in it. Also Windows may be interfering with it because git bash is not a perfect recreation of a Linux shell and FS. Stuff done for Zsh won't work as expected, I think this is obvious, isn't it?
In my opinion this is not a problem that the antidote developer (@mattmc3) should help troubleshooting because the used setup is not the pretended to be.
And as you said, you could always use WSL that is (by far) the best solution (and a native) to run Linux stuff on Windows. And you can do everything you do with Git Bash with WSL but in a more efficient way and without almost any problem because in the back it is a Linux system.
That being said I hope you could get this to work as you wish to.
I'll recommend you to use WSL or (like me) a dual-boot set up with an entire Linux distro paired with your Windows. And you will be able to do whatever you want with the PC 😉 I think is the best setup today taking into count that the price of the SSD are so low and they are big enough to carry two SO.
PD: I'm not part of the development team I'm just a random user of this awesome plugin manager. And my answer is not pretended to be mean, I'm only saying the things as-is (or almost how I think they are).

I'm open to see if we can get things working in git bash. I just don't have a Windows machine handy at the moment, so it'll take me a bit to get a test scenario set up to see if I can reproduce this. If so, and it's something that can be addressed within antidote, I'll consider coding a fix.

Without any promises I am able to use windows and have some experience with shell programming.
I am on finals but I am pretty sure this is an easy fix.
I'll see if I can fix the issue and submit a pr :)

One question to speed things up. Per my understanding it seems like the path used when antidote generates zsh_plugins.zsh is $ANTIDOTE_HOME?
It would be helpful to know this information, as I can just make it point to the right location when on Mingw64

This is the function that determines the home: https://github.com/mattmc3/antidote/blob/main/functions/antidote-home

Submited a PR!

@mattmc3 I just tried to set it up again and am getting the same issue:

@benwilburn can you try using my fork and see if that works?

I'm also having the same issue as @benwilburn when trying to use antidote in zsh invoked from git bash. If I add the __fix_windows_path function to functions/__antidote_get_cachedir and use it like this:
  local result
  if [[ "${OSTYPE}" == darwin* ]]; then
    result=$HOME/Library/Caches
  elif [[ "${OSTYPE}" == (cygwin|msys)* ]]; then
    result=$(__fix_windows_path ${LOCALAPPDATA:-$LocalAppData}) # <--- Note the change for the path
  elif [[ -n "$XDG_CACHE_HOME" ]]; then
    result=$XDG_CACHE_HOME
  else
    result=$HOME/.cache
  fi

Then the issue is fixed and antidote works correctly. Could the fix by @DiegoMagdaleno be merged in?

I can't merge that in without a good way to test it into the future, so what I'm really held up by is making sure there's a CI workflow to verify this works. I don't run Windows.

There may be a way to have the GitHub runners handle this, since they do come with git bash installed by default. @mattmc3, if I figure out a way to hook it into a Windows runner, would you be open to a PR?

Yes, definitely.

I can take full responsability for maintaining all of the Windows related bugs.

If v1.9.2 doesn't fix it, please re-open. I was able to properly test on an old Windows clunker, so I'm reasonably confident I can support this without CI working for Windows yet. No reason to hold up the feature. Thanks for the cygpath implementation @marovira.


Invalid e-mail

Bug Report
Describe the bug
Can't register with my e-mail address<EMAIL_ADDRESS>(google)
Steps To Reproduce
Try<EMAIL_ADDRESS>Screenshots

Desktop

macOS
FF Developer Edition

hm, weird. I added you to the list manually if that's ok! Not really sure why Mailchimp decided to flag it.

I thought it was a front-end issue. Thanks!

The message displayed in the banner is whatever Mailchimp replied to the request with. So in this instance, it sent back the result as an error, and the message as what you see above. 🙂


Imagery Discussions

Tasks associated with discussions for future imagery products.

Sprint 2

[ ] Analyze current discover usage

@brigsz , @gregbunce , @jacobdadams please provide any relevant closing notes/comments and close this FY24 Q4 objective.

this feature has been migrated to FY25/Q1 https://github.com/agrc/planning-queue/issues/425


Feature Request: Redirect callback

Synopsis
It would be great to be able to pass a prop to <Redirect /> that would allow a callback to be fired when the redirect occurs.

Suggested prop name and arguments
onRedirect which would return the following information (willPerformRedirect: boolean, to: string | object, push: boolean).

willPerformRedirect is the result of locationsAreEqual(prevProps.to, location)
to is the redirect to prop passed back to the callback
push is the redirect push prop passed back to the callback

Motivation
I have a scenario where I am rendering a series of links that update some urlParameters that in turn renders a scene. You can imagine this series of links like 'tabs' within a route. Some of these tabs are access controlled by a list of permissions i am managing about a particularly user. As such, I generally link to the route with a urlParameter of 'default', which my tabs recognise as an indicator that they should determine if the user can see at least one of the tabs, and redirect to that ('default' is then replaced with 'general', for example). It would be great to be able to determine, when this "redirect to default" action actually takes place, so that i can perform some subsequent actions without having to take snapshots of the passed in computedMatch and do comparisons.
I am sure there are other use cases, where being able to be notified of a redirect occurring would be useful, this is just my immediate experience.
Comments/Criticisms/Other work arounds welcome :)

I'm not sure I'm understanding your use case here. It sounds like your rendering only works correctly when coming in from this redirect, which in turn sounds like an imperative style. In general, your components shouldn't care how they came to be, just with whatever the current props are. That's a declarative style, which jives with how React works and makes for good, sustainable code.
At the very least, whenever you determine you're going to render a Redirect, that would be the point at which you can signal out some sort of preparatory action to your system that one is about to occur. No callback API needed.

@timdorr apologies if my use case isnt explained correctly. Trying to distill it down without specifics is difficult. Assume for the moment a different use case.
if that callback were to want to update a state value on the component that was rendering the Redirect when it performs its redirect, it is my understanding that calling the setState directly in the render method is something of an anti-pattern/frowned upon. However, providing it as part of a callback, for when a subsequent action occurs is performed is okay. If this still an anti-pattern?
Similarly this could be useful when a Redirect is rendered as a child of a Switch for the purposes of logging attempts to access old routes, i think?


date time for submissions should be saved

By default the version deployed from master doesn't save the date/time of each submission. This field should be saved by default.

The live instance now has an automatic created_at column; please note any preexisting columns have the datestamp from when the column was added (today).


Cannot run with libstd MIR on macOS due to stack guards

Executing any Rust program on macOS (and I assume anything but Linux) leads to the following crash:
[ skade miri ] cargo run --bin miri -- --sysroot ~/.xargo/HOST tests/run-pass/hello.rs
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/miri --sysroot /Users/skade/.xargo/HOST tests/run-pass/hello.rs`
WARN:miri::terminator: ignoring C ABI call: pthread_self
WARN:miri::terminator: ignoring C ABI call: pthread_get_stackaddr_np
WARN:miri::terminator: ignoring C ABI call: pthread_self
WARN:miri::terminator: ignoring C ABI call: pthread_get_stacksize_np
error: can't call C ABI function: mmap
   --> /Users/skade/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/sys/unix/thread.rs:283:26
    |
283 |               let result = mmap(stackaddr, psize, PROT_NONE,
    |  __________________________^
284 | |                               MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, 0);
    | |________________________________________________________________________^
    |
note: inside call to std::sys::imp::thread::guard::init
   --> /Users/skade/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/rt.rs:44:26
    |
44  |         let main_guard = sys::thread::guard::init();
    |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to std::rt::lang_start
   --> /Users/skade/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/rt.rs:32:1
    |
32  | / fn lang_start(main: fn(), argc: isize, argv: *const *const u8) -> isize {
33  | |     use panic;
34  | |     use sys;
35  | |     use sys_common;
...   |
72  | |     }
73  | | }
    | |_^

error: aborting due to previous error

thread 'main' panicked at 'Box<Any>', src/librustc_errors/lib.rs:526:8
note: Run with `RUST_BACKTRACE=1` for a backtrace.

I assume anything but Linux

Certainly not Windows as that's covered by Travis.
But it seems that non-Linux unices still use mmap. Seems we have to revert https://github.com/solson/miri/pull/239. :(

Certainly not Windows as that's covered by Travis.

Ah, sorry 😆 . I had quick glance through the relevant code and from "not linux" followed "everything else". The filename unix.rs should have been a tell that there was some context :).

We could add a mac Travis in order to prevent such things from happening.

@RalfJung I recently switched off macOS, so I can't test ;). Thanks!


Can not hide "Please keep this tab open" web mode box on ChatGPT site

Please have a way to hide this permenantly

👌

v2.3.0

Hi, can I disable this "keep this tab open" banner completely. very annoying. appears every time chatgpt website is opened. I don't use the web version of chatgpt in this plugin at all. Even if I do, I only need to be reminded one time.

@duhd1993 I will disable this banner next version when you choose the non-web mode


Support appending to the supplied Dockerfile in devtools-golang

The Golang devtools support the building of an OCI either from a
devtool-supplied Dockerfile, or when customizations are required, a
user-supplied Dockerfile.
To avoid unnecessary duplication of the supplied Dockerfile in the case
where only a few additional Dockerfile commands are needed, add support
for modifying it via an append.

This might become brittle, if extensions are tightly tied to the current implementation of the devtools dockerimage. If the devtools base image wants to change user, then services might start to break. If the only use-case we have for now is to copy&chown resources to be shipped, would it be more safe to include a whitelist of directories that will be copied and chowned if they exist?

This might become brittle, if extensions are tightly tied to the current implementation of the devtools dockerimage. If the devtools base image wants to change user, then services might start to break.

Agree, and it would most likely end up being abused.

If the only use-case we have for now is to copy&chown resources to be shipped, would it be more safe to include a whitelist of directories that will be copied and chowned if they exist?

Had a look at a few projects and an opinionated copy should cover most use cases. There are some Go projects with Dockerfiles that need a little more but it should be possible to rework them to use ConfigMaps instead.
I'll rework this then.

Replacing with PR https://github.com/coopnorge/engineering-docker-images/pull/1531