[DNM] STM32 SPI DMA bug fixes and enhancements

This PR has several bug fixes and enhancements for the STM32 SPI driver when using DMA

When using GPIO CS the CS select was enabled after the DMA started the transfer, resulting in the first few bits being transferred while CS was still disabled.

when doing RX or TX only the driver hung and never could finish the transfer

The driver used a busy wait loop to wait for DMA to finish, using a lot of CPU for large (20ms+ transfers). Now the DMA ready is signaled with a semaphore.

LGTM as long as you have been able to test that the first bits are correctly transmitted.

It looks good on my scope, this is a SPI OLED display, hence the big transfer. (which also explains why I need DMA);

@aurel32 are you fine with latest status ?

@erwango @aurel32  I would like to try with the RX and TX semaphore first, but won't get to it until Friday.

@lochej some how github doesn't let me request a re-review by you, but could you take a look at the semaphore code?

@aurel32 could you take a look at the sem code ?

Question: Is DMA working on L053 too? If yes, can someone help with UART-1 and SPI-1 configs?

Question: Is DMA working on L053 too? If yes, can someone help with UART-1 and SPI-1 configs?

Sorry @nandojve I don't have L053 hardware, so I can not test that.

Question: Is DMA working on L053 too? If yes, can someone help with UART-1 and SPI-1 configs?

I could test on my nucleo_l073rz with a correct test setup and DT configuration if you need help.

I have the feeling there is something wrong with the STM32 DMA in general, somehow it looks like channel and streams are mixed up in some places, the Zephyr's MACRO-FU is not really helpful in understanding the code.
When I enable logging it endlessly complains about "Unknown interrupt happened". In the dma code there seems to be some code parts that can pass the channel nr to the isr (that code seems unused) but when i add it I can print out the channel nr in the DMA isr and it complains about channel 4 (which I don't use), there is a stream 4 though. So maybe the naming is wrong, and it the variable should be named stream?
All very confusing, and DT docu isn't really helping either :-/
@lochej what is your DT setup ?

I have the feeling there is something wrong with the STM32 DMA in general, somehow it looks like channel and streams are mixed up in some places, the Zephyr's MACRO-FU is not really helpful in understanding the code.
When I enable logging it endlessly complains about "Unknown interrupt happened". In the dma code there seems to be some code parts that can pass the channel nr to the isr (that code seems unused) but when i add it I can print out the channel nr in the DMA isr and it complains about channel 4 (which I don't use), there is a stream 4 though. So maybe the naming is wrong, and it the variable should be named stream?
All very confusing, and DT docu isn't really helping either :-/

From looking at the bindibgs for DMA-STM32, streams and channel are determined from dmav1 or dmav2. (i.e., dmav1's stream number is dmav2's channel number).
Dma bindings

@lochej what is your DT setup ?

Don't have any yet, but I could build one to test this out.

@lowlander Thanks for the code changes, code wise it looks all good. I gave it a try for the communication between an SX1276 module and a STM32L433 and it doesn't work. But it didn't work before this PR either. I'll try to debug that in the next days.

@lowlander Thanks for the code changes, code wise it looks all good. I gave it a try for the communication between an SX1276 module and a STM32L433 and it doesn't work. But it didn't work before this PR either. I'll try to debug that in the next days.

@aurel32 any idea what goes wrong? Is there any difference between the L and F series when it comes to DMA/SPI ?

Dma bindings

@lochej the documentation and naming doesn't make it easier, from that doc;
  1. channel: the dma stream from 0 in V1 or 1 in V2 to <dma-requests>
  2. slot: DMA periph request ID in DMA_V2 or DMAMUX, dma request in V1

So the first number is the channel (but really it is the stream) and the second the "slot" is the ID (what the ST docu calls the channel).
Also from that doc
For the client part, example for stm32f411 DMA V1 on DMA2 instance
    Tx using stream 5 on channel 3 (stream 2 on channel 2 is also possible)
    Rx using stream 2 on channel 3 (stream 0 on channel 3 is also possible)
    spi1 {
     dmas = <&dma2 5 3 0x28440 0x03>,
           <&dma2 2 3 0x28480 0x03>;
     dma-names = "tx", "rx";
     };

So the channel is the stream and the slot is the channel, not confusing at all.
And when I look at the dma_callback(struct device* dev, void* arg, uint32_t channel, int status) in the spi driver, that is the stream not the channel, but it is called channel.
So channel, slot, id, stream, and request all seem to be used at random, this makes it very hard to understand the code.
But it also means my DT should be correct. :-)

Question: Is DMA working on L053 too? If yes, can someone help with UART-1 and SPI-1 configs?

I could test on my nucleo_l073rz with a correct test setup and DT configuration if you need help.

I tried this, without success. It builds, and board run but at first spi communication try it hangs:
nucleo_l053r8
config SPI_STM32_DMA
default y
depends on SPI
config SPI_STM32_INTERRUPT
default n
depends on SPI
config SPI_STM32_USE_HW_SS
default n
depends on SPI
&spi1 {
       /* Not sure if below is right too */
	dmas = <&dma1 3 1 0x20240 0x04>,
	       <&dma1 2 1 0x20480 0x04>;
	dma-names = "tx", "rx";
};
&dma1 {
	status = "okay";
};

So channel, slot, id, stream, and request all seem to be used at random, this makes it very hard to understand the code.

What mess ^^!!

@nandojve AN2548 shows that stm32L series are using DMA V2.
Make sure your DT uses V2 definition. On the other hand, the DMA code seems to be designed for V1 with stream definitions. I don't know if information can help but DMA handling might need to be refactored to differ based on CPU type (STM32L series, STM32F4 series...).
STM32F0/F1/F3/G0Lx DMA description V2

@lowlander @nandojve AN2548 shows that stm32L series are using DMA V2.
Make sure your DT uses V2 definition. On the other hand, the DMA code seems to be designed for V1 with stream definitions. I don't know if information can help but DMA handling might need to be refactored to differ based on CPU type (STM32L series, STM32F4 series...).
STM32F0/F1/F3/G0Lx DMA description V2

@lochej I am really considering rewriting the whole STM32 DMA driver, There are several things I dislike about it, like the use of kmalloc, integer flags shared between isr and other code, weird function pointer setup, weird IRQ handling.

@nandojve could you give the update a try (in combination with #27723 ) , noticed that when logging is on the highest level it hung by me too, that seems to be a race condition. I changed the init order a bit and make sure both RX and TX DMA are enabled (if there is no buffer it uses the dummy buffer, so they should always be enabled)

Hi @lowlander, not sure if you had time to look at NVIC from L0x. It seems that channels are sharing interrupt lines.

IRQ
Acronym
Desc

9
DMA1_Channel1
DMA1 Channel1 global interrupt

10
DMA1_Channel[3:2]
DMA1 Channel2 and 3 interrupts

11
DMA1_Channel[7:4]
DMA1 Channel4 to 7 interrupts

I got below error after enable DMA with #27263 and #27723:
[147/150] Generating isr_tables.c
FAILED: zephyr/isr_tables.c
cd /home/gfbudke/zephyros/mainline/zephyr/build/zephyr && /home/gfbudke/zephyr-sdk-0.11.3/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --input-target=elf32-littlearm --output-target=binary --only-section=.intList /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr_prebuilt.elf isrList.bin && /usr/bin/python3.7 /home/gfbudke/zephyros/mainline/zephyr/arch/common/gen_isr_tables.py --output-source isr_tables.c --kernel /home/gfbudke/zephyros/mainline/zephyr/build/zephyr/zephyr_prebuilt.elf --intlist isrList.bin   --sw-isr-table --vector-table
gen_isr_tables.py: error: multiple registrations at table_index 11 for irq 11 (0xb)
Existing handler 0x80073cf, new handler 0x80073c5
Has IRQ_CONNECT or IRQ_DIRECT_CONNECT accidentally been invoked on the same irq multiple times?

I believe it is related to https://github.com/zephyrproject-rtos/zephyr/pull/27723/commits/a0cfe27b84a8f0858f7631c64bf6f82e7b2e45d3. I do prefer your changes but we need some alternative for devices with shared lines.
My guess is L0 devices will share DMA int
@erwango  ^^ ?

@nandojve first of all thanks for testing, and yes it seems L0 has only 3 IRQ's for the 7 channels, I will update the code to deal with that, since it is known at compile time it should be doable.  And at the end of the week I also should have hardware to test it myself (although this compile error I could/should have found without hardware).

@aurel32 could you also give it a retry in combination with #27723 to see if it works on a L4 ?

Sorry I have been busy doing another setup using a nucleo l432kc board and an sx1276 instead of the final board where things are a bit more difficult to debug. I am also addressing the sx1276 directly instead of using the Zephyr driver. Therefore I haven't tested #27723 yet.

@lowlander Thanks for the code changes, code wise it looks all good. I gave it a try for the communication between an SX1276 module and a STM32L433 and it doesn't work. But it didn't work before this PR either. I'll try to debug that in the next days.

@aurel32 any idea what goes wrong? Is there any difference between the L and F series when it comes to DMA/SPI ?
update/edit: I noticed the F4 is V1 and L4 V2 of the driver, and the channel offsets are different, maybe that is where things go wrong, maybe you have the wrong channel/stream setup in your DT ?

I have progress a bit on that. Initially without this PR, I had this kind of error:
[00:00:00.206,000] <wrn> dma_stm32: dest_buffer address is null.
[00:00:00.206,000] <err> dma_stm32: dma stream 1 is busy.
[00:00:00.206,000] <err> main: TX failed: -16

The 3 first commits of this PR make things to work. However the last one (drivers: spi_ll_stm32: use semaphores to prevent busy wait loop) breaks things again. It breaks when spi_write() is called with a transaction of 2 buffers, while it works fine with 1) spi_write()and a single buffer or 2) spi_transceive() and multiple buffers.
I haven't debugged further yet.

Just looking at the code, it seems that before we waited for either of the two transfers to stop:
static bool spi_stm32_dma_transfer_ongoing(struct spi_stm32_data *data)
{
       return ((data->dma_tx.transfer_complete != true)
               && (data->dma_rx.transfer_complete != true));
}

/* once SPI is no more busy, wait for DMA transfer end */
while (spi_stm32_dma_transfer_ongoing(data) == 1) {
}

That way they could be reloaded independently if they do not have the same length. With your change we expect both transfers to finish at the same time.

@lowlander I wonder in which conditions the third commit "drivers: spi_ll_stm32: make sure DMA is done sending" commit is necessary and if there is another way to achieve the same result.
Without that commit, it seems that all the work that is done in the for (; ;) loop of the transceive_dma function can be done directly in the DMA interrupt, as there are all short functions without busy loop. The semaphore can then be used to signal that all the buffers have been RXed or TXed. That would actually make sense to run the SPI transfer as fast as possible without waiting between buffers depending on the priority of the thread calling spi_write/read/transceive.

@aurel32 From the logic there should not be any difference from the old driver, that waited for the busy to go away (which pretty much means TX is ready) and than looped on spi_stm32_dma_transfer_ongoing  which is the same as waiting for the two semaphores in a row. So I am not sure I understand why it works without the last commit and doesn't work with the last commit.
One of the big problems I had with the old driver was that a RX only did not work because it seems it needs a TX to generate the clock signal. This also means that TX-len may not be smaller than RX-len, else the RX DMA will never finish. On the other hand the RX-len can be smaller for a single transfer, but when having multiple buffers part of the RX will be lost.
I think your right that the better way to handle all corner cases is to reload the DMA from the ISR, because there it is known if it is RX or TX that is done.  But I have the feeling that will be quite some work.

@aurel32 From the logic there should not be any difference from the old driver, that waited for the busy to go away (which pretty much means TX is ready) and than looped on spi_stm32_dma_transfer_ongoing which is the same as waiting for the two semaphores in a row.

No it is not the same, as with spi_stm32_dma_transfer_ongoing the loop stops as soon as RX or TX or both transfers are complete. With the semaphore, the code waits for TX and RX transfers to finish.

So I am not sure I understand why it works without the last commit and doesn't work with the last commit.

One of the big problems I had with the old driver was that a RX only did not work because it seems it needs a TX to generate the clock signal. This also means that TX-len may not be smaller than RX-len, else the RX DMA will never finish. On the other hand the RX-len can be smaller for a single transfer, but when having multiple buffers part of the RX will be lost.

Yes, I agree that the total TX and RX len should be the same, however I think the current implementation allows the buffers to be split differently. For example for a 5-byte transfer, TX could use 2 buffers of 2 and 3 bytes and RX a single buffer of 5 bytes.

I think your right that the better way to handle all corner cases is to reload the DMA from the ISR, because there it is known if it is RX or TX that is done. But I have the feeling that will be quite some work.

i'll give it a try.

No it is not the same, as with spi_stm32_dma_transfer_ongoing the loop stops as soon as RX or TX or both transfers are complete. With the semaphore, the code waits for TX and RX transfers to finish.

Negative boolean logic gets me every time, combined with a name "ongoing" that looks at RX and TX, I misunderstood thinking it was RX and TX ready instead of RX or TX ready :-/
PS: are you really going to give the IRQ stuff a try, not that we both write something.

PS: are you really going to give the IRQ stuff a try, not that we both write something.

I gave it a first try and it works well in my test case: https://github.com/aurel32/zephyr/commits/stm32-spi-dma
The code still use boolean flags instead of semaphores, it still has to be converted the same way you already did with your version. I am also not sure about the condition to check for the end of the transfer, i have chosen to test for the SPI being busy as before and the TX empty flag like in your "drivers: spi_ll_stm32: make sure DMA is done sending" commit. Note that this is done only at the end of all the whole transfer instead of each buffer, so I expect also some performance improvements when multiple buffers are used.
PS: I won't work on this before tomorrow night CEST, feel free to cherry-pick the commits and continue the work on the semaphores.

PS: I won't work on this before tomorrow night CEST, feel free to cherry-pick the commits and continue the work on the semaphores.

OK I will give it a try tonight

@aurel32 I merged your commits into this PR, probably have to clean things up a bit before upstreaming, but for testing it is OK I think.  It seems to work with my usecase (RX-only, TX-only, and RX+TX, but always just 1 buffer).  I do have the bad feeling it will break when RX and TX are not the same length, where or we have to fill things up with dummy buffers, or some check at startup will need to catch that and give an error.

@aurel32 I merged your commits into this PR, probably have to clean things up a bit before upstreaming, but for testing it is
OK I think. It seems to work with my usecase (RX-only, TX-only, and RX+TX, but always just 1 buffer).

Thanks for merging. There is still one issue I have there with your version of the code. You replaced the
while (LL_SPI_IsActiveFlag_BSY(spi) == 1 || LL_SPI_IsActiveFlag_TXE(spi) == 0) {
by
while (LL_SPI_IsActiveFlag_TXE(spi) == 0) {
The busy part comes from the original loop. Without it, things are not working as expected here. I do not have a scope at hand, but I guess when stopping the SPI too early, the SS pin is not released correctly (I am using CONFIG_SPI_STM32_USE_HW_SS=y). I guess it can also be done using 2 different busy loops.
The bad news however is that the sx1276 driver doesn't work with this PR (but as I said wasn't working before). I am going to debug that.

I do have the bad feeling it will break when RX and TX are not the same length, where or we have to fill things up with dummy buffers, or some check at startup will need to catch that and give an error.

I agree there is an issue there, but I think it was already the case before that PR. I also thing that not all SPI drivers (even without DMA) handle that correctly, so it's probably just a question of adding a check.

@aurel32 I merged your commits into this PR, probably have to clean things up a bit before upstreaming, but for testing it is
OK I think. It seems to work with my usecase (RX-only, TX-only, and RX+TX, but always just 1 buffer).

Thanks for merging. There is still one issue I have there with your version of the code. You replaced the
while (LL_SPI_IsActiveFlag_BSY(spi) == 1 || LL_SPI_IsActiveFlag_TXE(spi) == 0) {
by
while (LL_SPI_IsActiveFlag_TXE(spi) == 0) {
The busy part comes from the original loop. Without it, things are not working as expected here. I do not have a scope at hand, but I guess when stopping the SPI too early, the SS pin is not released correctly (I am using CONFIG_SPI_STM32_USE_HW_SS=y). I guess it can also be done using 2 different busy loops.

I never could get CONFIG_SPI_STM32_USE_HW_SS to work, maybe that was my problem, I will add both again.

The bad news however is that the sx1276 driver doesn't work with this PR (but as I said wasn't working before). I am going to debug that.

Do other SPI device work? With other words do you think  it is the SX1276 or your CPU ?

I do have the bad feeling it will break when RX and TX are not the same length, where or we have to fill things up with dummy buffers, or some check at startup will need to catch that and give an error.

I agree there is an issue there, but I think it was already the case before that PR. I also thing that not all SPI drivers (even without DMA) handle that correctly, so it's probably just a question of adding a check.

Easiest way out is probably a check at the start of the transaction to see if tx-len == rx-len when both are > 0 (when rx or tx  is 0 it should be ok, and when both are 0 the transaction makes no sense)

The bad news however is that the sx1276 driver doesn't work with this PR (but as I said wasn't working before). I am going to debug that.

Do other SPI device work? With other words do you think it is the SX1276 or your CPU ?

SPI with DMA works fine when I directly use the spi_read/write/transceive functions. However on the same hardware the Lora driver only works without the DMA. I therefore think it's purely a software issue, there is probably some minor difference.

The bad news however is that the sx1276 driver doesn't work with this PR (but as I said wasn't working before). I am going to debug that.

Do other SPI device work? With other words do you think it is the SX1276 or your CPU ?

SPI with DMA works fine when I directly use the spi_read/write/transceive functions. However on the same hardware the Lora driver only works without the DMA. I therefore think it's purely a software issue, there is probably some minor difference.

The lora driver seems to do a buffer set with two buffer with RX/TX (same buffer so same size) and a TX only, For the TX only case the dummy RX probably should be the total length of the TX len (same for the other way around) I will check if that is the case later and also fix the BSY waiting.

@aurel32 re-added the bsy wait, and changed the length calculation in case only a RX or TX buffer is used, now the dummy DMA transfer should be just as long as the total of the real transfer (not just the size of the first buffer)

So after debugging, it seems that the two main issues are:

data->dma_tx.dma_cfg.head_block->dest_address and data->dma_rx.dma_cfg.head_block->source_address are only valid for the first dma transfer. Replacing them by (uint32_t)LL_SPI_DMA_GetRegAddr(cfg->spi) fixes the biggest part of the issue.
Not fully sure, but it sees that we need to call LL_SPI_DisableDMAReq_TX() and LL_SPI_DisableDMAReq_RX() just before calling LL_SPI_Disable() in transceive_dma().

With that the Lora driver seem to work fine, although I only tested it very lightly. I get many <err> dma_stm32: Unknown interrupt happened. errors though.
Enough for today, I'll look at that again tomorrow.

So after debugging, it seems that the two main issues are:

data->dma_tx.dma_cfg.head_block->dest_address and data->dma_rx.dma_cfg.head_block->source_address are only valid for the first dma transfer. Replacing them by (uint32_t)LL_SPI_DMA_GetRegAddr(cfg->spi) fixes the biggest part of the issue.
Not fully sure, but it sees that we need to call LL_SPI_DisableDMAReq_TX() and LL_SPI_DisableDMAReq_RX() just before calling LL_SPI_Disable() in transceive_dma().

Could't we just call LL_SPI_DisableDMAReq_TX and LL_SPI_DisableDMAReq_RX in the ISR just before giving the semaphore ?

So after debugging, it seems that the two main issues are:

data->dma_tx.dma_cfg.head_block->dest_address and data->dma_rx.dma_cfg.head_block->source_address are only valid for the first dma transfer. Replacing them by (uint32_t)LL_SPI_DMA_GetRegAddr(cfg->spi) fixes the biggest part of the issue.

data->dma_tx.dma_cfg.head_block gets assigned a pointer to a stack based variable, that looks very wrong.

Could't we just call LL_SPI_DisableDMAReq_TX and LL_SPI_DisableDMAReq_RX in the ISR just before giving the semaphore ?

Yep, that's a good idea.

So after debugging, it seems that the two main issues are:

data->dma_tx.dma_cfg.head_block->dest_address and data->dma_rx.dma_cfg.head_block->source_address are only valid for the first dma transfer. Replacing them by (uint32_t)LL_SPI_DMA_GetRegAddr(cfg->spi) fixes the biggest part of the issue.

data->dma_tx.dma_cfg.head_block gets assigned a pointer to a stack based variable, that looks very wrong.

I used the value that was assigned to  periph_addr in the old code, but that's indeed plainly wrong.

update: I pushed some changes to (hopefully) fix that.

Thanks. It seems to works fine. I still have to do more testing, but this PR looks in a good shape now.

Thanks. It seems to works fine. I still have to do more testing, but this PR looks in a good shape now.

I have the feeling there is still some sort of race condition that prevents the transfer from finishing. Last night my hardware got a watchdog reset and my monitoring software said it was due to a task doing SPI transfers not responding in time.
Apart from finding (which won't be easy, because it ran for atleast 5 hours before it happened) and fixing this I think we need to not K_FOREVER wait on the semaphores, but give them a timeout for when things go really wrong, better fail one transaction than hang the system.

@nandojve  I now have a L0 board with a SPI-FeRAM shield and managed to get it working, could you give this PR in combination with  #27723 a try ?
PS: the dts settings I used are;
&spi1 {
	status = "okay";

	dmas = <&dma1 3 1 0x28440 0x04>,
	       <&dma1 2 1 0x28480 0x04>;

	dma-names = "tx", "rx";

	cs-gpios = <&gpiob 6 GPIO_ACTIVE_LOW>;
};

@lochej  could you also give this in combination with  #27723 a try ?

With the recent changes from this PR, the SPI driver works perfectly with DMA on an L433 with a SX1276 chip and the corresponding LoRa driver. I get <err> dma_stm32: Unknown interrupt happened DMA ISR. messages, but I have tracked that to a bug in the DMA ISR routine fixed by PR #27723 (and this is not a regression). When using both PRs, all works perfectly without any error message.
The only think that might be missing, is that according to the documentation, the DMA channel must not be configured once enabled. In turns it means that the DMA channel must be disabled at the end of a transfer. It might also bring some power saving, especially for the Lx series. This can be done simply with this patch:
        /* end of the transfer : all buffers sent/received */
        LL_SPI_Disable(spi);

+       /* Disable DMA channels */
+       dma_stop(data->dev_dma_rx, data->dma_rx.channel);
+       dma_stop(data->dev_dma_tx, data->dma_tx.channel);
+
        spi_stm32_complete(data, spi, ret);

        spi_context_release(&data->ctx, ret);

@nandojve I now have a L0 board with a SPI-FeRAM shield and managed to get it working, could you give this PR in combination with #27723 a try ?
PS: the dts settings I used are;
&spi1 {
	status = "okay";

	dmas = <&dma1 3 1 0x28440 0x04>,
	       <&dma1 2 1 0x28480 0x04>;

	dma-names = "tx", "rx";

	cs-gpios = <&gpiob 6 GPIO_ACTIVE_LOW>;
};

Hi @lowlander , I'm using AT25 EEPROM driver with an ST M95640 compatible SPI device.
Using same configuration that you shared I got below:
uart:~$ eeprom read M95640 1024 8
Reading 8 bytes from EEPROM, offset 1024...
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_stm32_configure: Installed config 0x20000ad4: freq 4000000Hz (div = 8), mode 0/0/0, slave 0
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: tx_bufs 0x200010b0 - rx_bufs 0x200010c0 - 1
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: current_tx 0x200010a8 (1), current_rx 0x200010b8 (1), tx buf/len 0x200010a0/2, rx buf/len 0x200010a4/2
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: tx_bufs 0x200010f8 - rx_bufs 0x20001100 - 1
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: current_tx 0x200010f0 (1), current_rx 0x20001108 (2), tx buf/len 0x200010ec/3, rx buf/len 0x00000000/3
[00:00:01.599,000] <dbg> spi_ll_stm32.spi_context_update_rx: rx buf/len 0x20001160/8
[00:00:02.598,000] <err> os: ***** HARD FAULT *****
[00:00:02.598,000] <err> os: r0/a1:  0x20000fe0  r1/a2:  0x40020000  r2/a3:  0x00000001
[00:00:02.598,000] <err> os: r3/a4:  0x00000001 r12/ip:  0x00000000 r14/lr:  0x00000000
[00:00:02.598,000] <err> os:  xpsr:  0x00000000
[00:00:02.598,000] <err> os: Faulting instruction address (r15/pc): 0x00003e74
[00:00:02.598,000] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
[00:00:02.598,000] <err> os: Current thread: 0x20000468 (logging)
[00:00:02.599,000] <err> os: Halting system

When I try use my S2LP native Zephyr driver I got this:
D: Initialize S2-LP Transceiver
D: SDN GPIO configured on GPIOA:8
D: IRQ GPIO configured on GPIOA:0 connected at DIO0
D: GPIO configured
D: SPI GPIO CS configured on GPIOB:6
D: SPI configured
D: Installed config 0x200000d0: freq 4000000Hz (div = 8), mode 0/0/0, slave 1
D: tx_bufs 0x20001290 - rx_bufs 0x200012a0 - 1
D: current_tx 0x20001288 (1), current_rx 0x20001298 (1), tx buf/len 0x20001280/2, rx buf/len 0x20001284/2
D: tx_bufs 0x20001280 - rx_bufs 0x20001288 - 1
D: current_tx 0x20001278 (1), current_rx 0x20001290 (2), tx buf/len 0x20001270/2, rx buf/len 0x20001274/2
D: rx buf/len 0x200012b4/2

Hi @lowlander , I'm using AT25 EEPROM driver with an ST M95640 compatible SPI device.
Using same configuration that you shared I got below:
uart:~$ eeprom read M95640 1024 8
Reading 8 bytes from EEPROM, offset 1024...
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_stm32_configure: Installed config 0x20000ad4: freq 4000000Hz (div = 8), mode 0/0/0, slave 0
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: tx_bufs 0x200010b0 - rx_bufs 0x200010c0 - 1
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: current_tx 0x200010a8 (1), current_rx 0x200010b8 (1), tx buf/len 0x200010a0/2, rx buf/len 0x200010a4/2
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: tx_bufs 0x200010f8 - rx_bufs 0x20001100 - 1
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: current_tx 0x200010f0 (1), current_rx 0x20001108 (2), tx buf/len 0x200010ec/3, rx buf/len 0x00000000/3
[00:00:01.599,000] <dbg> spi_ll_stm32.spi_context_update_rx: rx buf/len 0x20001160/8
[00:00:02.598,000] <err> os: ***** HARD FAULT *****
[00:00:02.598,000] <err> os: r0/a1:  0x20000fe0  r1/a2:  0x40020000  r2/a3:  0x00000001
[00:00:02.598,000] <err> os: r3/a4:  0x00000001 r12/ip:  0x00000000 r14/lr:  0x00000000
[00:00:02.598,000] <err> os:  xpsr:  0x00000000
[00:00:02.598,000] <err> os: Faulting instruction address (r15/pc): 0x00003e74
[00:00:02.598,000] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
[00:00:02.598,000] <err> os: Current thread: 0x20000468 (logging)
[00:00:02.599,000] <err> os: Halting system

That looks like some other problem, why would the logging thread crash? Can you try with assert and logging disabled ?

When I try use my S2LP native Zephyr driver I got this:
D: Initialize S2-LP Transceiver
D: SDN GPIO configured on GPIOA:8
D: IRQ GPIO configured on GPIOA:0 connected at DIO0
D: GPIO configured
D: SPI GPIO CS configured on GPIOB:6
D: SPI configured
D: Installed config 0x200000d0: freq 4000000Hz (div = 8), mode 0/0/0, slave 1
D: tx_bufs 0x20001290 - rx_bufs 0x200012a0 - 1
D: current_tx 0x20001288 (1), current_rx 0x20001298 (1), tx buf/len 0x20001280/2, rx buf/len 0x20001284/2
D: tx_bufs 0x20001280 - rx_bufs 0x20001288 - 1
D: current_tx 0x20001278 (1), current_rx 0x20001290 (2), tx buf/len 0x20001270/2, rx buf/len 0x20001274/2
D: rx buf/len 0x200012b4/2

Could you share your software with me? I have a STEBAL-FKI868V2 kit here now, so I could try to reproduce your problem.

The only think that might be missing, is that according to the documentation, the DMA channel must not be configured once enabled. In turns it means that the DMA channel must be disabled at the end of a transfer. It might also bring some power saving, especially for the Lx series. This can be done simply with this patch:
        /* end of the transfer : all buffers sent/received */
        LL_SPI_Disable(spi);

+       /* Disable DMA channels */
+       dma_stop(data->dev_dma_rx, data->dma_rx.channel);
+       dma_stop(data->dev_dma_tx, data->dma_tx.channel);
+
        spi_stm32_complete(data, spi, ret);

        spi_context_release(&data->ctx, ret);

Will add that and a check to see if total-tx-len and total-rx-len are the same (when both of them are != 0)
I would also like to reproduce @nandojve problem, to see it that exposes any fundamental design errors.

Hi @lowlander , I'm using AT25 EEPROM driver with an ST M95640 compatible SPI device.
Using same configuration that you shared I got below:
uart:~$ eeprom read M95640 1024 8
Reading 8 bytes from EEPROM, offset 1024...
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_stm32_configure: Installed config 0x20000ad4: freq 4000000Hz (div = 8), mode 0/0/0, slave 0
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: tx_bufs 0x200010b0 - rx_bufs 0x200010c0 - 1
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: current_tx 0x200010a8 (1), current_rx 0x200010b8 (1), tx buf/len 0x200010a0/2, rx buf/len 0x200010a4/2
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: tx_bufs 0x200010f8 - rx_bufs 0x20001100 - 1
[00:00:01.598,000] <dbg> spi_ll_stm32.spi_context_buffers_setup: current_tx 0x200010f0 (1), current_rx 0x20001108 (2), tx buf/len 0x200010ec/3, rx buf/len 0x00000000/3
[00:00:01.599,000] <dbg> spi_ll_stm32.spi_context_update_rx: rx buf/len 0x20001160/8
[00:00:02.598,000] <err> os: ***** HARD FAULT *****
[00:00:02.598,000] <err> os: r0/a1:  0x20000fe0  r1/a2:  0x40020000  r2/a3:  0x00000001
[00:00:02.598,000] <err> os: r3/a4:  0x00000001 r12/ip:  0x00000000 r14/lr:  0x00000000
[00:00:02.598,000] <err> os:  xpsr:  0x00000000
[00:00:02.598,000] <err> os: Faulting instruction address (r15/pc): 0x00003e74
[00:00:02.598,000] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
[00:00:02.598,000] <err> os: Current thread: 0x20000468 (logging)
[00:00:02.599,000] <err> os: Halting system

That looks like some other problem, why would the logging thread crash? Can you try with assert and logging disabled ?

When I try use my S2LP native Zephyr driver I got this:
D: Initialize S2-LP Transceiver
D: SDN GPIO configured on GPIOA:8
D: IRQ GPIO configured on GPIOA:0 connected at DIO0
D: GPIO configured
D: SPI GPIO CS configured on GPIOB:6
D: SPI configured
D: Installed config 0x200000d0: freq 4000000Hz (div = 8), mode 0/0/0, slave 1
D: tx_bufs 0x20001290 - rx_bufs 0x200012a0 - 1
D: current_tx 0x20001288 (1), current_rx 0x20001298 (1), tx buf/len 0x20001280/2, rx buf/len 0x20001284/2
D: tx_bufs 0x20001280 - rx_bufs 0x20001288 - 1
D: current_tx 0x20001278 (1), current_rx 0x20001290 (2), tx buf/len 0x20001270/2, rx buf/len 0x20001274/2
D: rx buf/len 0x200012b4/2

Could you share your software with me? I have a STEBAL-FKI868V2 kit here now, so I could try to reproduce your problem.

Sure, https://github.com/nandojve/zephyr/tree/ht-micron_s2-lp
S2LP have a BUG. If you try use another device on SPI bus you need put S2LP on shutdown otherwise you will see only zeros on the bus. STEVAL-FKI868V2 need be reworked to use both S2LP and EEPROM on regular Arduino pins, need move 3 resistors I think. LED gpio is the same SCK clock, be carefully. L053 have internal EEPROM, make sure you are using AT25 driver.
To use S2LP you need build samples/drivers/sigfox/send

with this change, can 2 DMA instances be enabled with a loop transfer transfer (tests/drivers/dma/loop_transfer) on the 1st one.

@nandojve could you try with disabling logging CONFIG_LOG=n and disabling asserts CONFIG_ASSERT=n ? I think I have found a race condition, where the reloading of the RX buffer isn't fast enough so it misses bytes at the start, and because of that never finishes because it still wants to receive the set number of bytes.

@aurel32 could you try with logging and assert enabled and/or higher bus speed (I use 5MHz, when I reduce it to 500kHz it works fine again) There seems the be a timing problem when using multiple buffers in transceive mode. For example reading from SPI FRAM, there are two buffers one 3 byte command buffer and the buffer that should get the read data. When the TX is done with the 3 bytes, it will get the IRQ and load the next buffer, so than it directly starts sending again (and with that receiving) but the RX done IRQ comes after that and so the loading of the next RX buffer is not yet done. If there now is the slightest delay between the TX-done and RX-done IRQ (for example the 50 micro seconds for assert checking) we loose bytes (and with that lock up). And this was with 5MHz SPI, if that would run at 50MHz there is simply no way to be on time.
So I think we have to move the reload out of the ISR again and make sure the rx and tx buffer-sets are always exactly the same count and  size. Or make sure the buffers given to the DMA are the same size, so we have to split up the buffers to always match, if the TX-set has 2 buffers of 2 and 4 bytes and the rx set 2 buffers of 3 bytes, we will have to split them in 3 buffers of 2, 1, and 3 bytes, so the RX and TX DMA are always finished after the same number of bytes, and so the reloading happens at the same time (when both RX and TX are disabled).

@aurel32 could you try with logging and assert enabled and/or higher bus speed (I use 5MHz, when I reduce it to 500kHz it works fine again) There seems the be a timing problem when using multiple buffers in transceive mode. For example reading from SPI FRAM, there are two buffers one 3 byte command buffer and the buffer that should get the read data. When the TX is done with the 3 bytes, it will get the IRQ and load the next buffer, so than it directly starts sending again (and with that receiving) but the RX done IRQ comes after that and so the loading of the next RX buffer is not yet done. If there now is the slightest delay between the TX-done and RX-done IRQ (for example the 50 micro seconds for assert checking) we loose bytes (and with that lock up). And this was with 5MHz SPI, if that would run at 50MHz there is simply no way to be on time.

I have tried that, but I have not been able to reproduce the issue. My SPI runs at 10MHz which is the maximum supported by the SX1276. OTOH, it's on a L4 CPU, I guess there is way more power than on your L0 system.

So I think we have to move the reload out of the ISR again and make sure the rx and tx buffer-sets are always exactly the same count and size. Or make sure the buffers given to the DMA are the same size, so we have to split up the buffers to always match, if the TX-set has 2 buffers of 2 and 4 bytes and the rx set 2 buffers of 3 bytes, we will have to split them in 3 buffers of 2, 1, and 3 bytes, so the RX and TX DMA are always finished after the same number of bytes, and so the reloading happens at the same time (when both RX and TX are disabled).

I have seen that you made that change. It still works fine for me. I guess in a future we can use the same principle that you use in the ISR to avoid a context change, which is probably more expensive than reloading the DMA. But it's probably better to do that in another PR to avoid waiting indefinitely for it to be ready.
That part looks good to me, it just lacks the dma_stop at the end of the transfer. I have more comments on the second commit concerning the cache handling. Maybe it's also better to do that in a different PR?

I have seen that you made that change. It still works fine for me. I guess in a future we can use the same principle that you use in the ISR to avoid a context change, which is probably more expensive than reloading the DMA. But it's probably better to do that in another PR to avoid waiting indefinitely for it to be ready.

Not might be possible to do it all in the ISR, might get a bit tricky because the next transfer can only be started after both RX and TX are done. For the moment I think it would be a good thing to first at least have something that works.

That part looks good to me, it just lacks the dma_stop at the end of the transfer.

I will put that back in, that is no problem.

I have more comments on the second commit concerning the cache handling. Maybe it's also better to do that in a different PR?

I will split that out to a new PR, for someone else to finish, F7 is not one of my platforms I need and I already spend way to mush time on this. And I have the feeling cache handling is still not that well supported in Zephyr and could become a lengthy discussion.

Moved the F7 cache handling code to its own PR #27911

@lochej @FRASTM @erwango @cybertale @aurel32 Now that the DMA changes are in master this PR should be ready to be merged if there are no more change requests. @cybertale could you take a look at the small DMA code change, that I only noticed when measuring the SPI signal with a scope. I think the change is OK, but a second look by someone else never hurts.

@lowlander I'm tagging this PR with V2.4.0 milestone.
V2.4.0 merge window is closing today, but we'll still be able to merge bug fixes as long as the bug relates to a filled issue (with a "fixes #foobar" reference).
So, if not merged today, an issue will need to be filled.

@lowlander I'm tagging this PR with V2.4.0 milestone.
V2.4.0 merge window is closing today, but we'll still be able to merge bug fixes as long as the bug relates to a filled issue (with a "fixes #foobar" reference).
So, if not merged today, an issue will need to be filled.

@erwango there might actually already be some issues that could have been fixed by this PR, might have to go trough SPI related issues that are out standing to see if the reporters can test with this PR


Access Analytics on Events

This story is quite large and covers too many things, let's re-write it later to cover either:

As an organizer, I want to generate reports on event participation, such as total steps walked, active participants, and team rankings.
As an organizer, I want to access analytics on user engagement to improve future events.

Part of Epic 4 of #16
User Story Statement:
As an FSF event organizer, I want to view the previous events, so that I can see a history of user activity over the past 2 years.
Background
Why do we want it? We want this feature because FSF event organizers need a way to see the performance of a previous event so that they can make better decisions based on their past experience.
Who is this for? This is for event organizers and not users.
Do we have the data to support it? Yes, we have verified that this feature is vital to FSF organizers.
Solution/Requirements
Functional Requirements:

User should be able to view a list of previous events

User should be able to view user activity for all previous events

User should be able to filter events by year

Non-functional Requirements

Data for an event should be stored in the database indefinitely unless there is a reason for deletion
Back-office should be able to display data for an event within 1 second after clicking on it
The data attached to a previous event should never be changeable

Design Resources
Figma Design Board
Dependencies
Are we waiting on other user stories or tasks to be completed?  Yes, we have some other user stories to complete.   The homepage still needs to be created so that the event organizer can click on "events."
Are other user stories or tasks waiting on this issue to be completed?  Yes, this user story is blocking one other user story.  "As an event organizer, I want to be able to import past events, so that I can easily start a new event without retyping the same information." This story requires that the "View Previous Events" feature is available.
List of tasks and subtasks

[ ]  Initialize backend https://github.com/OregonStateUniversity/osu-fsf/issues/15
[ ] Creation of events  https://github.com/OregonStateUniversity/osu-fsf/issues/21
[ ] Initialize database
[ ] Connect database to back office
[ ]  Initialize webpage
[ ]  Add "events" button to homepage
[ ]  Save event data after event creation
[ ]  Collect user activity data and store in database
[ ]  Create events page
[ ]  Display event data when event clicked on

Estimate
I estimate each sub-task to take a week.  Some will take longer than others, but the completion of all of these tasks will be near the last half of the project's completion.
Open Questions

What user data will we store?
Will events be in the database permanently?
Will users see past events?

Acceptance Criteria

Given a user clicks on "events", the list of previous events will be displayed
Given a user clicks on an event, the event data will be displayed
Given a user creates an event, the event information is saved in database

Definition of Done

[ ] All code passes linting checks
[ ] All code passes unit/integration tests
[ ] Fulfills all functional and non-functional requirements
[ ] Covers all acceptance criteria
[ ] All subtasks listed are completed
[ ] User should be able to view previous events, click on events, and see data for that event

check


[KEYCLOAK-8984] Warning in Gatekeeper on invalid state

Co-authored-by: Frode Ingebrigtsen<EMAIL_ADDRESS>

I have a question on this one. I had implemented the final landing page using encoded state, to accomodate the way it used to work.
I understand that now, setting the state for further checks by the client at the end of the sequence of redirection is no more possible.
Further, to control the final landing page, we must now use a special "request_uri" cookie...
Am I assuming correctly?

Another remark: I cannot find the JIRA 8984 so it is difficult to grasp the background of this change.
The closest I could find is KEYCLOAK-8856, but no mention there of the strategy adopted with this PR.
Could someone shed some light on this change?

Not sure I completely understand your setup. Did you use a modified Gatekeeper, or did you actually use the state from the protected application? (I assume it's from the cookie then).
The behavior before this pull request was to pass the original URL in state (base64 encoded):
https://github.com/keycloak/keycloak-gatekeeper/pull/433
This was probably not a good solution since it basically removes the entropy from the state parameter. The pull request replaced the value of state with a uuid, but failed to replace state with another mechanism to communicate the original URL to the callback after authentication. This pull request fixes that by using another cookie (request_uri) to communicate the original URL. The protected application should hopefully not need to read this cookie, as Gatekeeper should return with a redirect to this URL as response to the authentication callback.
I filed issue 8984 because I noticed a warning message in the logs, and flagged in as an "Security Issue" since I was unsure if it affected validation of the state parameter. It didn't, but that's why the issue is not visible.

I used the unmodified master from just before this PR was merged, and passed an explicit state from the client (a JS application, using an iframe to manage the authN). Indeed,
I understood that this was the way to retrieve a custom state after login (this allowed me to pass some parameter etc to close the iframe at the end of the handshake).
So now I had to adapt this setting and replace the encoded state query param by a cookie. This works now, with my client setting the cookie when calling /oauth/authorize.
I am just a bit wary that we break such a functionality without much announcement. I did not inspect much a PR named "fix warning xxx" and only realized it broke stuff, so this took me by surprise.

@fredbi everything on master branch is considered under development. For stable releases, please download the binaries from keycloak.org. We don't provide any announcements for changes into the codebase, instead, we use Jiras and keycloak-dev to keep track of everything.
Anyways, if there's something you believe we should do better, feel free to bring this up on the mailing list.


chore: add script to apply sync fixes

This PR automates the manual fixes which need to be applied to sync PRs into aztec-packages.

Benchmark results
No metrics with a significant change found.

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,676
1,540
687
739
766

proof_construction_time_sha256_30_ms
11,674
3,133
1,399
1,433
1,461

proof_construction_time_sha256_100_ms
44,581
12,040
5,657
6,163
5,487

proof_construction_time_poseidon_hash_ms
236
80.0
59.0
89.0
150

proof_construction_time_poseidon_hash_30_ms
5,715
1,565
720
779
795

proof_construction_time_poseidon_hash_100_ms
11,313
3,119
1,396
1,409
1,448

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
64 txs

l1_rollup_calldata_size_in_bytes
1,412
1,412
1,412

l1_rollup_calldata_gas
9,452
9,464
9,464

l1_rollup_execution_gas
616,093
616,105
616,105

l2_block_processing_time_in_ms
1,313
4,889
9,676

l2_block_building_time_in_ms
46,693 (+2%)
183,291 (+1%)
368,068 (+1%)

l2_block_rollup_simulation_time_in_ms
46,520 (+2%)
182,633 (+1%)
366,768 (+1%)

l2_block_public_tx_process_time_in_ms
25,384 (+3%)
104,029
214,174 (+1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
9,451 (-1%)
14,476

node_database_size_in_bytes
14,463,056
21,352,528

pxe_database_size_in_bytes
18,071
29,868

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
simulation_time_in_ms
witness_generation_time_in_ms
proving_time_in_ms
input_size_in_bytes
output_size_in_bytes
proof_size_in_bytes
num_public_inputs
size_in_gates

private-kernel-init
134 (+1%)
499
12,298 (+2%)
20,630
64,614
89,536
2,731
524,288

private-kernel-inner
410 (+1%)
1,015 (+1%)
39,515
92,318
64,614
89,536
2,731
2,097,152

private-kernel-tail
594 (+2%)
3,049 (+1%)
36,920 (+1%)
96,541
77,732
11,648
297
2,097,152

base-parity
6.58
1,172 (-9%)
2,895 (+1%)
128
64.0
2,208
2.00
131,072

root-parity
50.4 (+1%)
48.2 (+1%)
44,726 (+2%)
27,084
64.0
2,720
18.0
2,097,152

base-rollup
749 (+4%)
2,323
74,732
119,734
756
3,648
47.0
4,194,304

root-rollup
112
63.4 (-1%)
19,601 (+1%)
25,297
620
3,456
41.0
1,048,576

public-kernel-app-logic
546 (+2%)
3,132 (-3%)
48,130
105,253
86,550
116,768
3,582
2,097,152

public-kernel-tail
1,139 (+2%)
23,845
155,477 (+1%)
401,002
7,646
11,648
297
8,388,608

private-kernel-reset-small
602 (+2%)
2,061 (-7%)
45,747
120,733
64,614
89,536
2,731
2,097,152

merge-rollup
28.7 (-1%)
N/A
N/A
16,534
756
N/A
N/A
N/A

public-kernel-setup
647 (-2%)
N/A
N/A
105,253
86,550
N/A
N/A
N/A

public-kernel-teardown
545 (-3%)
N/A
N/A
105,253
86,550
N/A
N/A
N/A

private-kernel-tail-to-public
N/A
8,527
91,191
N/A
N/A
116,768
3,582
4,194,304

Stats on running time collected for app circuits

Function
input_size_in_bytes
output_size_in_bytes
witness_generation_time_in_ms
proof_size_in_bytes
proving_time_in_ms
size_in_gates
num_public_inputs

ContractClassRegisterer:register
1,344
9,944
469 (-1%)
N/A
N/A
N/A
N/A

ContractInstanceDeployer:deploy
1,408
9,944
42.2
N/A
N/A
N/A
N/A

MultiCallEntrypoint:entrypoint
1,920
9,944
1,459
N/A
N/A
N/A
N/A

SchnorrAccount:constructor
1,312
9,944
994
N/A
N/A
N/A
N/A

SchnorrAccount:entrypoint
2,304
9,944
2,149 (+1%)
16,768
49,436 (+1%)
2,097,152
457

Token:privately_mint_private_note
1,280
9,944
1,138
N/A
N/A
N/A
N/A

Token:transfer
1,376
9,944
4,072
16,768
46,378
2,097,152
457

Benchmarking:create_note
1,312
9,944
960
N/A
N/A
N/A
N/A

FPC:fee_entrypoint_public
1,344
9,944
226
N/A
N/A
N/A
N/A

SchnorrAccount:spend_private_authwit
1,280
9,944
77.7 (-7%)
N/A
N/A
N/A
N/A

Token:unshield
1,376
9,944
3,236 (-1%)
N/A
N/A
N/A
N/A

FPC:fee_entrypoint_private
1,376
9,944
4,023
N/A
N/A
N/A
N/A

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
512 leaves
1024 leaves
2048 leaves
4096 leaves
32 leaves

batch_insert_into_append_only_tree_16_depth_ms
10.5
17.0
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.7
31.8
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.609
0.521
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
48.7 (-1%)
76.6
249
475 (-1%)
940
1,862
N/A

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
543
1,055
2,079
4,127
N/A

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.497 (-1%)
0.472
0.451
0.443 (-1%)
0.445
0.443
N/A

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
58.3
113 (-1%)
359 (+1%)
697 (-1%)
1,400
2,785
N/A

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
106
208
692
1,363
2,707
5,395
N/A

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.504
0.506 (-1%)
0.487 (+1%)
0.479 (-1%)
0.483
0.483
N/A

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
62.8 (-1%)

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
107

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0.556 (-1%)

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
84,050
665,267

Transaction size based on fee payment method
| Metric |  |
| - |  |

@ludamad Looks like this has broken inbound syncs due to not being able to find the fixup-sync.sh script.
I'm a little confused on this as we checkout the full aztec-packages repo and all the paths in the relevant step are relative to the repo root so I'm not sure why this wouldn't be picked up. Could you take a quick look to see if I've done something dumb?
https://github.com/AztecProtocol/aztec-packages/actions/runs/9290648812/job/25567342096


Set current timestamp based on scheduler start

Trello: https://trello.com/c/wgBKh6pz
The current measurement time used to be based on the current time, which could cause timestamps to diverge by 1 minute. The current measurement time is now based on when the scheduler started.

tested with 12 BT addresses (collecting 0 responses from 12 BT addresses takes more than one minute). Using a timestamp aligned with start of measurement task now works


Travis CI Matrix

This patch provides a new Travis CI Matrix for building and
testing the source tree. This also includes support for
automatically submitting to Coverity Scan.
Signed-off-by: “Rian<EMAIL_ADDRESS>

Coverage remained the same at 100.0% when pulling 8adf3755a0f26af8bd6e44485bf6db63a0ede481 on rianquinn:travisci_matrix into cbba7208941fc02bd895fecb4ad49acd83483078 on Bareflank:master.

Coverage remained the same at 100.0% when pulling 8adf3755a0f26af8bd6e44485bf6db63a0ede481 on rianquinn:travisci_matrix into cbba7208941fc02bd895fecb4ad49acd83483078 on Bareflank:master.

Coverage remained the same at 100.0% when pulling 8adf3755a0f26af8bd6e44485bf6db63a0ede481 on rianquinn:travisci_matrix into cbba7208941fc02bd895fecb4ad49acd83483078 on Bareflank:master.

Coverage remained the same at 100.0% when pulling 8adf3755a0f26af8bd6e44485bf6db63a0ede481 on rianquinn:travisci_matrix into cbba7208941fc02bd895fecb4ad49acd83483078 on Bareflank:master.


[FLINK-9366] DistributedCache works with Distributed File System

What is the purpose of the change
Allows to use DistributeCache to cache files from cluster internal DFS (reverted behaviour).
Brief change log
(for example:)

Local files (based on uri schema) are distributed via BlobServer
Files in DFS are cached from DFS

Verifying this change

Added test for distributing files through DFS: org.apache.flink.hdfstests.DistributedCacheDfsTest
local files shipping covered by previous tests

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving):no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

Could you have a look @zentol @aljoscha ?

Looks good functionality wise, had some minor comments.


javascript/build contains IML.React.*, should be SHAP.React.*

The build folder shouldn't be committed. If it needs to be committed, then the package name should be correct.
The visualization would require window.SHAP and not window.IML.

@rishacha I don't see it here https://github.com/shap/shap/blob/master/javascript/build/index.html
@thatlittleboy Maybe a candidate for closing?

@detrin Thanks.
Yea we are using window.SHAP in the repo now, don't see any instances of window.IML.

Now 1.4k more to go


Revise StringNames and NodePaths in C#.

Describe the project you are working on
A 3D MMORPG with lots of moving parts and dependencies, each of which needs to be very performant.
Describe the problem or limitation you are having in your project
StringNames and NodePaths, which were clearly designed for GDScript, cause garbage collection performance spikes [1, 2] in C#. Since they're used intermittently and are implicitly casted from strings, they can be easy to miss. Functions that use them are often called every frame (e.g. Input.IsActionPressed(StringName Action)), making these issues very insidious. Even the Godot and Redot Docs overlook this.
There are several workarounds:

Store StringNames in variables.

static readonly StringName Position = "position";
// ...
Get(Position);

This solution is cumbersome and messy when you have a lot of StringNames.

Use the source-generated cached StringNames.

Get(Node3D.PropertyName.Position);

This solution only works for properties and methods of built-in classes. It doesn't work for shader parameters, GDScript interop, and the various methods that use them (for example Input and Tween and MultiplayerSynchronizer).

Use caching functions

Get(StringName("position"));

This is my personal solution since it's the cleanest. (See code). However, it pretty much invalidates the whole purpose of StringNames and NodePaths which is to improve performance.
Describe the feature / enhancement and how it helps to overcome the problem or limitation
The ideal solution would be one of the following:

Remove StringNames and NodePaths, since they don't add much value.
This would be ideal, and clearly works in other engines like Unity. However, it could be a lot of work and would be a breaking change.

Remove StringNames and NodePaths from the C# API.
This would also be ideal, since they are such an issue in C# specifically.  However, it would also be a breaking change.

Use caching in the implicit casts to StringName/NodePath.
This would be a simple patch that would also solve the issue. StringNames and NodePaths could still be cached as variables if you needed the performance boost (but realistically 99% of games don't need it since it's a micro-optimisation).

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
Assuming the 3rd solution is chosen (to use caching in the implicit casts) here are my caching functions:
global using static RecycleExtensions;

using ZiggyCreatures.Caching.Fusion;

public static class RecycleExtensions {
    private static readonly FusionCache StringNameCache = new(new FusionCacheOptions());
    private static readonly FusionCache NodePathCache = new(new FusionCacheOptions());

    /// <summary>
    /// Converts the string to a StringName, recycling if possible.
    /// </summary>
    public static StringName StringName(string String) {
        return StringNameCache.GetOrSet(String, _ => (StringName)String);
    }
    /// <summary>
    /// Converts the string to a NodePath, recycling if possible.
    /// </summary>
    public static NodePath NodePath(string String) {
        return NodePathCache.GetOrSet(String, _ => (NodePath)String);
    }
}

The StringName implicit cast currently looks like this:
public static implicit operator StringName(string from)
{
    return new StringName(from);
}

Instead, it could be changed to this (replace FusionCache with your cache of choice):
using ZiggyCreatures.Caching.Fusion;

private static readonly FusionCache StringNameCache = new(new FusionCacheOptions());

public static implicit operator StringName(string from)
{
    return StringNameCache.GetOrSet(from, _ => new StringName(from))!;
}

If this enhancement will not be used often, can it be worked around with a few lines of script?
The enhancement should be regularly used. Please see my discussion in the Godot Engine repository to see the performance difference. The workarounds are simple but can be easy to miss and are overcomplicated.
Is there a reason why this should be core and not an add-on in the asset library?
It is not possible for this to be an add-on, since you can't monkey-patch in C#.

Please see my discussions in the Godot and Redot repository for further reading.

Remove StringNames and NodePaths, since they don't add much value.

This won't happen, it violates compatibility.

Remove StringNames and NodePaths from the C# API.

This neither can happen, it both violates compatibility and violates binding functionality.


Confusion about the quantitative results of the NPS dataset

Why maskrcnn achieved the same AP as Dogfight with a lower precision. look forward to your help.

Hi @mouyihan,
Thanks for your question.
As you can see from the scores, MaskRCNN has good number of true positives indicated by recall. False positives are responsible for low precision and as a result precision moved up and down.
While calculating average precision, we use interpolation and calculate the area under the curve. We are using (standard) all point interpolation (https://github.com/mwaseema/Drone-Detection/issues/13#issuecomment-1366042109), most of those lower precision points didn't affect the overall AP due to interpolation.
I hope this answers your question.
Regards,
Waseem


Bookie tests failing; need to fix installation for Travis CI

@miso-belica pointed out that the tests for Bookie are failing. That's part of the issue. The other part is that Bookie is not currently able to build because certain packages have aged out and are no longer available. I don't remember the specifics at the moment so I can't elaborate more without re-trying the build process. What I remember though is I got stuck while trying to modernize the packages with later versions.

@craigmaloney Can you give me some info about breadability/bookie? My story is that I use breadability in https://github.com/miso-belica/sumy and did some changes into in the past. It's quite a while :)
From my point of view I guess we could remove support for old Py2.6 if possible or maybe even Py3.3. But I am not sure about it. I can live with it in sumy.
Also I would refactor tests to run using pytest and I would remove unittest2 package then if possible. I can see there is some old branch port-pytest but it seems there is still old "good" unittest. I can help with refactor to pytest if you are interested and you (or anybody) can do me a review. And also fix the 2 failing tests with it.
What do you think?

I was speaking on the project as a whole. I don't think the current batch of tests can be run on breadability by itself.
I hate to say the word "fork" but I'm wondering if it's time to fork off breadability on its own until if / when Bookie can be modernized. Are you currently maintaining a branch?

Hmm, sorry, but I don't use bookie and I don't even know what exactly it is. I always though some kind of Readability app fork? But I don't see how breadability fork help here. If problem is somewhere else in bookie stack I think it should be fixed there. I believe breadability can be fixed right here.
I can help with porting to pytest and fix the failing tests if dropping py2.6 support is not option and someone will do me a review, but I don't think I can help with other bookie packages. If you can write how breadability is blocking some other package to upgrade maybe I can fix also that issue, but that's all.

breadability was/is an open source bookmark management app and breadabilty
was a python port of the open version of the readability JS code long ago
so that we could load the web page content for searching when you
bookmarked something.
Definitely don't worry about bookie with breadability updates and I'm happy
to help test/review any changes that make it work. No need for py2 support
or the like. I'd started to port all my own stuff to pytest and am a fan so
any changes there are welcome as well.
On Tue, Mar 27, 2018 at 1:41 PM Mišo Belica<EMAIL_ADDRESS>wrote:

Hmm, sorry, but I don't use bookie and I don't even know what exactly it
is. I always though some kind of Readability app fork? But I don't see how
breadability fork help here. If problem is somewhere else in bookie stack I
think it should be fixed there. I believe breadability can be fixed right
here.
I can help with porting to pytest and fix the failing tests if dropping
py2.6 support is not option and someone will do me a review, but I don't
think I can help with other bookie packages. If you can write how
breadability is blocking some other package to upgrade maybe I can fix also
that issue, but that's all.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bookieio/breadability/issues/34#issuecomment-376612302,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEoi6_JzsLJRvqiJ38xOPHptNnLWyJhks5tinmygaJpZM4S3T2D
.


Build failure: hash mismatch in qemu-8.2.3

$ nix-prefetch-url https://download.qemu.org/qemu-8.2.3.tar.xz
path is '/nix/store/hv1fyagi0acym5xlws4gjmj54qz4nvl3-qemu-8.2.3.tar.xz'
0r4raqm2rcaswivgp0ryl8r9m0pns5mqgi01fqqmb540csrpyx6w
$ nix hash to-sri --type sha256 0r4raqm2rcaswivgp0ryl8r9m0pns5mqgi01fqqmb540csrpyx6w
sha256-3HR/s2aAlFUxdgHEh2vR9oKaMqI+g/t25FqxLCpWmWQ=

The hash in nixpkgs/master right now is:
  version = "8.2.3";

  src = fetchurl {
    url = "https://download.qemu.org/qemu-${finalAttrs.version}.tar.xz";
    hash = "sha256-d1sRjKpjZiCnr0saFWRFoaKA9a1Ss7y7F/jilkhB21g=";
  };

I had both tarballs kicking around, so I checked the differences:
diff -rc good/qemu-8.2.3/roms/seabios/.version sus/qemu-8.2.3/roms/seabios/.version
*** good/qemu-8.2.3/roms/seabios/.version       2024-04-24 09:09:59.000000000 -0700
--- sus/qemu-8.2.3/roms/seabios/.version        2024-04-24 13:15:10.000000000 -0700
***************
*** 1 ****
! rel-1.16.3-0-ga6ed6b70
--- 1 ----
! rel-1.16.3-0-ga6ed6b7
Only in good/qemu-8.2.3/subprojects: dtc

@Misterio77
A tarball with a hash matching what is in master is at https://download.qemu.org/old/qemu-8.2.3.tar.xz
seems like some trouble with the release.

That's... super weird. download.qemu.org says the tarballs last modified date was April 25th:
qemu-8.2.3-reupload.tar.bz2                        06-May-2024 13:59           156774533
qemu-8.2.3-reupload.tar.bz2.sig                    06-May-2024 13:59                 310
qemu-8.2.3-reupload.tar.xz                         06-May-2024 13:59           129917944
qemu-8.2.3-reupload.tar.xz.sig                     06-May-2024 14:00                 310
qemu-8.2.3.tar.bz2                                 25-Apr-2024 02:18           156774533
qemu-8.2.3.tar.bz2.sig                             25-Apr-2024 02:18                 310
qemu-8.2.3.tar.xz                                  25-Apr-2024 02:18           129917944
qemu-8.2.3.tar.xz.sig                              25-Apr-2024 02:18                 310

Maybe we should use the gitlab tarball instead?

Okay, found some context: https://mail.gnu.org/archive/html/qemu-stable/2024-05/msg00025.html

Thanks!
Not sure this is worth fixing or just wait for https://github.com/NixOS/nixpkgs/pull/298097 to land.

It's going to be a while before we can get 9.0.0 due to the release freeze, so this is definitely worth fixing now.


rbac: use authenticated principal filter state

Signed-off-by: Kuat Yessenov<EMAIL_ADDRESS>Please provide a description of this PR:

@kyessenov: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test release-notes

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test release-notes

lint_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test release-notes

lint_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test lint

gencheck_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test release-notes

lint_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test lint

gencheck_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test gencheck

integ-pilot-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-pilot-istiodremote

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test release-notes

lint_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test lint

gencheck_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test gencheck

integ-pilot-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-pilot-istiodremote

integ-security-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-security-istiodremote

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test release-notes

lint_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test lint

gencheck_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test gencheck

integ-pilot-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-pilot-istiodremote

integ-security-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-security-istiodremote

integ-security_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-security

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test gencheck

integ-pilot-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-pilot-istiodremote

integ-security-istiodremote_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-security-istiodremote

integ-security_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-security

release-notes_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test release-notes

lint_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security_istio
361b4191f23bfec8025af7e8250714f359d2d6f8
link
true
/test integ-security

release-notes_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test release-notes

lint_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test lint

integ-security-istiodremote_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security-istiodremote

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test release-notes

lint_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test lint

integ-security-istiodremote_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security-istiodremote

integ-security_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test lint

integ-security-istiodremote_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security-istiodremote

integ-security_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security

release-notes_istio
8ba85c7861d9465ad9c2e13c030a9227f8ddc152
link
true
/test release-notes

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-istiodremote_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security-istiodremote

integ-security_istio
c22c3c4273f38c95abc323c809d6c1b3c5e9fefd
link
true
/test integ-security

release-notes_istio
8ba85c7861d9465ad9c2e13c030a9227f8ddc152
link
true
/test release-notes

lint_istio
8ba85c7861d9465ad9c2e13c030a9227f8ddc152
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
66588f89894b9ebbb6b57f131d98da13d5f5506f
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-multicluster_istio
cc4be13564aa7230c29ce270183792a39b2aa65a
link
true
/test integ-security-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest


add my session about GraphQL

Thanks! I will edit it a bit with a session below and just link that from the schedule like the others.


fix: resolve problems flagged by go vet and staticcheck

Closes: https://github.com/alec-pinson/ip-whitelister/issues/58
Closes: https://github.com/alec-pinson/ip-whitelister/issues/59
Changes:

Fixes typos.
resolve staticcheck issues.
Add go vet to github workflow

Thank you for fixing these


Handle Visual Studio 2003 and earlier

This should resolve https://github.com/leethomason/tinyxml2/issues/347 Please don't merge until it is confirmed.

Happy to integrate this in, once the correct fix is in place. I think the oldest VC I have is 2010.

I just pushed the branch "2003": https://github.com/leethomason/tinyxml2/tree/2003 I didn't see anything that needed to be brought over from this PR, but I'm not sure. Please check!

Closing PR - moved to branch.


Support erase headers option for multiple files

Like a tail command -q option.

-q Suppresses printing of headers when multiple files are being examined.


TST: also run flake8 on Python 2.7

Closes #54.

Oops, duplicate of https://github.com/pydata/pandas-gbq/pull/54

Codecov Report

Merging #55 into master will decrease coverage by 44.87%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master      #55       +/-   ##
===========================================
- Coverage   74.18%   29.31%   -44.88%
===========================================
  Files           4        4
  Lines        1511     1511
===========================================
- Hits         1121      443      -678
- Misses        390     1068      +678

Impacted Files
Coverage Δ

pandas_gbq/gbq.py
16.08% <0%> (-63.14%)
:arrow_down:

pandas_gbq/tests/test_gbq.py
31.88% <0%> (-49.94%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cec8c86...a884dc1. Read the comment docs.


Example KQL plugin that use Sentinel and Defender

An example of a mixed-use KQL plugin that integrates Defender and Sentinel skills, featuring multiple configuration examples and detailed comments to guide users.

Closing this pull req. as I just realized the "ExamplePrompts:" includes a trailing 's' and should be "ExamplePrompt:" I'll resubmit in a few minutes.


tf.data.Dataset.shard is doing split in different way as assumed by OS2S

tf data shard is doing "sequential" split, i.e. dataset k will get every i % num_workers == k element and OpenSeq2Seq assumes internally that data will be split "globally", meaning that all data is divided into equal sequential parts and the first one goes to the first dataset, second to the second and so on. This will break multi-GPU inference when tf.data.Dataset.shard approach is used. We should make OS2S either to have both options supported or to switch to tf.data version, since that's what we expect users to do when they create new datasets.

Fixed in dev0.4 branch


[FEATURE] Consider Visvalingam-Whyatt algorithm for simplification

Looking at the Planetiler codebase, it seems that line simplification is currently done with the Douglas-Peucker algorithm. As an alternative, have you considered the Visvalingam-Whyatt algorithm, which may give better-looking results? Tilemaker seems to be using Visvalingam-Whyatt through a C++ library originally made for Tippecanoe inspired by a Go implementation, but there’s also a Java function in JTS. See also here and here for interactive explanations of the algorithm, and the original paper.

Yes! That would be a great addition. The one issue with JTS library is that it should use a priority queue but instead does a brute force approach to find the next smallest triangle to remove, and probably has some other inefficiencies. It wouldn't be hard to implement an efficient version in planetiler though. That would also give us flexibility to use the effective area weighting function used on https://mapshaper.org
Beyond that it's mostly just a question of what the API should look like to switch between DP and VW. Could we specify "strategy=retain effective areas" and somehow transform the DP tolerance to a VW effective area limit?
See more discussion at https://github.com/onthegomap/planetiler/discussions/534 and https://github.com/onthegomap/planetiler/discussions/708

Added Visvalingam Whyatt simplification option in #1109. You can opt-into it on a feature by using feature.setSimplifyMethod(SimplifyMethod.RETAIN_EFFECTIVE_AREAS) or feature.setSimplifyMethod(SimplifyMethod.RETAIN_WEIGHTED_EFFECTIVE_AREAS).  By default "tolerance" gets squared and used as the minimum area threshold, so you need to tune tolerance up a bit in order to end up with similar output size as DP simplification.  The weighted version sharp corners as well.
I'll tackle the GeometryPipeline in a followup PR.


ignore shebang at line=0 common in node-scripts

this simple-patch will ignore the first-line shebang common in many command-line-tool nodejs scripts.  intended to make jslint easier-to-use for nodejs-developers
live web-demo of this patch available at: https://kaizhu256.github.io/JSLint/branch.ignore_shebang/index.html
screenshot of jslint ignoring first-line shebang, but properly linting the second-line

Thanks


Update mathlib to 4.12.0-rc1

@fpvandoorn I can now report that the Nat.reducePow bug that was causing PANICs in builds of this repo has been fixed.

Thanks! Next time please update with lake update -Kenv=dev or scripts/update_mathlib.sh.


Upgrade HydroShare to use iRODS version 4.2

Upgrade HydroShare to use iRODS version 4.2

Branch 1818-irods-4.2.0 created as starting point for this work.

MUST run using irods/use-local-irods.sh script in order to enable locally federated iRODS v.4.2.0 Docker containers.

Example using the hydrodev VM
$ cd /home/hydro/hydroshare/
$ git checkout 1818-irods-4.2.0
$ cd irods/
$ ./use-local-irods.sh
$ cd ../
$ ./hsctl rebuild --db

Closing this since we'll revisit upgrading HydroShare 4.2.x and not 4.2 at this time due to #2140.


Eye catchy UI/UX.

In the initial phase, I started this as a simple and logical project but at that time not any other services were available like this so I think it is better to create this a usable product for many.
Make your contribution to UI/UX.

@kuanoni Added Split screen view with split-adjustable #2

#2  @


Add AuthenticationProperties to HandleRequestResult and RemoteFailureContext

#1188 Requires https://github.com/aspnet/HttpAbstractions/pull/889
User state for the login is flown via auth properties, but these aren't available in the OnRemoteFailure event. To add them we need to flow them via the AuthResult.
If this works out then we may use it to flow additional data for #1178. The difference is that the data in #1178 isn't a consistent data structure across handlers.
/cc: @brockallen

@Tratcher,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

More for @anitchanana than for me to review.

bump
I've filled out the tests and addressed some of the ordering issues.

Updated


Small patch fixes

Pull Request Checklist

[x] Will your PR merge into the dev branch?
Exceptions will be made for urgent bugfixes.
[X] Have you forked from dev?
If not, please rebase your PR onto the most recent dev tip.
[x] Does your PR title succinctly describe the changes?
Explain to a new user by completing the sentence: 'This PR will: ...'
[x] Fill out the template below.

Description of the Change
This pull requests make three small changes:

only outputs multiconformer_model.pdb if --debug flag is on. This file is not needed and should not be used except for by developers.
updates readme/help on qFit protein to not use -z electron as that is currently not workoing
add normalize occupancy at beginning of post refine script to patch issue in qFit protein  (not summing occupancies to one).

dev changed in the meantime, so rebase & force-push was required to fix merge conflict


Run overall command does not take into account Test Root Folder

The used GdUnit4 version:
4.1.4
The used Godot version:
4.1.1.stable
Describe the bug
The run overall command does not take into account changes to the "Test Root Folder" option. I believe this stems from this line, which I assume should include GdUnitSettings.test_root_folder() rather than just "test".

Thanks for reporting this bug.