Pi0 does not exec kernel.img when using -d

It boots to the  "rainbow screen of life" but then goes no further.  From what I read that indicates kernel.img is not launching.
Files are requested and loaded, changes to config.txt do cause changes in what is loaded (as should be expected).  Everything appears to fully load, and no errors in my best configuration (eg failed to transfer control, "Failed : 0x..." - which I have seen in some of my failed tests mentioned below).
A list of what I have tried so far
I have tried two different Pi0s (non-W) to ensure one was not defective.
I have tried every commit with no success.  Every one including the first few that did not even get to stage 2 on a Pi0.
I have tried both stock Jessie boot files as well as custom kernels and a bare metal helloworld.
I have tried with and without a usb hub (the hub does get in the way a bit).
I have tried some older start.elf/bootcode.bin files (got 8 blinks indicating they were too old).
I wrote my own bootcode.bin that just blits data to the serial port.  That works.  That is the only thing I can get to work.  This however does not init the arm, etc so its of limited functionality.
I am running out of ideas for things I can try to rule out anything on my end, with my gear, etc.  I am starting to think the problem must be in start.elf not properly jumping to where it should jump to.

Usbboot works perfectly for me on a Model A, a Zero 1.2 and a Zero-W, but has only ever worked once on my Zero 1.3 - I cannot begin to imagine how that could happen.
The "Failed:" message is not important, I see that when rpiboot is working normally.
You should use the latest bootcode/start/fixup files in the firmware master branch, they work perfectly well for usbboot.
The only thing I can suggest is that you keep buying more Pi Zeros until you find one that works, but you should check that your boot partition will run from sdcard.

I dont think "keep buying until you get one that works" is a viable solution to the problem.
The boot partition does work when its booted from an SD card.
I have used the latest firmware from the github rpi/firmware repo.

The board that fails for me has serial-number 56c76327. I cannot do anything to help resolve this issue until we have a start.elf that gives debug info over the serial port.

No resolution or comment in a week? :/

I presume that normal Raspbian on SD card boots fine? @ghollingworth Any thoughts?

yes it does.  That same image I did (on desktop)
sudo kpartx -a jessie.img
sudo mount /dev/mapper/loop0p1 /mnt
rpiboot -d /mnt
and variations of using a working image as a base for my boot dir.

Yeah, working on it...
On Tue, Mar 21, 2017 at 1:37 PM Nom-DePlume<EMAIL_ADDRESS>wrote:

yes it does. That same image I did (on desktop)
sudo kpartx -a jessie.img
sudo mount /dev/mapper/loop0p1 /mnt
rpiboot -d /mnt
and variations of using a working image as a base for my boot dir.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/raspberrypi/usbboot/issues/13#issuecomment-288080133,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB9CHOOFagrIkDw1L8Ed3PpS9bOh1-zdks5rn9KigaJpZM4Mazoi
.

--
Director of Software Engineering, Raspberry Pi (Trading) Limited
t:  +44 (0) 1223 322633
m: +44 (0) 7450 946289
e<EMAIL_ADDRESS>w: www.raspberrypi.org

@Nom-DePlume
Where did you get the information to write bootcode.bin?   I was just searching the web on the boot process of the RPI. I stumbled on a few github trees, but they weren't particularly usefull.
I was actually thinking of doing things the other way around as something that could be interesting.  Use the standard bootcode.bin and rewrite a custom second stage loader.

You know this is a silly question, but I was wondering something.  Why in the world does start.elf run in GPU mode.   Why can't bootcode.bin jump to arm, load a new loader on the arm.  Then if necessary possibly load the existing gpu based .elf file to get the rest of the gpu loaded.
I've always wondered why the boot process is as backward as it is.  My guess is historical reasons.
I've been looking at some code in Linux, and lots of drivers load firmware fragments on demand as needed.   Say like nvidea.  Some of the GPU code doesn't actually get loaded until the gpu is used in accelerated mode.

Correct, historical reasons.  The original SoC in the Pi was the 2835 which was, in effect, a Videocore4 with a small ARM core bolted on. The Videocore starts up running some internal first stage boot code, reading from the SD card and downloading its firmware, (bootcode.bin) and running it. Then bootcode.bin then starts the ARM core(s) and loads and runs start.elf. At least, that's what I remember.

Yes, that's always seemed backwards.  Sounds like bootcode.bin really needs to jump to something in arm mode.  Possibly in a mode where many of the advanced features are disabled.
Then is some other gpu specific code gets loaded later, it unlocks the features.

Most people in linux seem to load all the secret stuff later. After the kernel has loaded.

No...
bootcode.bin runs on the VC4 processor (running on the L2 cache only) it then sets up the SDRAM and loads the main firmware code onto the VPU (start.elf)
start.elf then initialises and kicks off the ARM
If we implemented all the ARM starting code in bootcode.bin then it wouldn't fit into the L2 cache.  Also bootcode.bin doesn't support reading from directories so you'd also have to implement that as well.
In the end yes it is largely historical but we couldn't do it any other way without completely re-writing everything

What if a mini start.elf where available that just what's absolutely needed.

I'm actually interested in bootloader and if a mini start.elf where available, I think you would have lots of bare metal people interested in pounding on it. Like I said, you could document the thing and people could party on it.  The idea would be that most of the advanced features are disabled.

BTW, what's this loader.bin file I keep reading about on the web. It seems to have existing at one time or another.
I'm really interesting in maybe writing a bootloader for the RPI, but I don't want to deal with any GPU specific code. Just ARM.  That way I could even share it out with people and such.

BTW, one more question. If bootcode.bin doesn't support reading from directories how does all the netboot does work.  That sounds kind of complex and all.
Anyway, longer term I was toying with ideas on how you guys could get away from the FAT file system which is ancient and everybody hates.  I've been thinking about creating a boot filesystem on some flash chips, but the information to do it doesn't seem to be documented.   Would it be possible to get some documentation for that?
Ideally, I would still like to have some way to jump into ARM mode with as little of the gpu enabled as possible.

Basically the idea would be to use a cheap microcontroller say like a PIC chip that that emulates one of the support boot modes.  The idea would be to keep the data in the secure store by making sure that it can only be updated with a new copy that has a valid signature.  I see you have several supported boot modes.  USB is one option, but I would rather not go that route since RPI typically only have 1 USB port.
I guess I could start with usb device mode and test on the new RPI 0 I just bought by haven't used yet.   But like I said, device boot mode doesn't seem to be documented as far as I can tell.

OK, I'm about to quit.
You know when ARM64 started, it started with someone writting a stub file that got shared out.  I actually had no idea how that stub worked, but it was enough to get people started.   The idea was that this stub was catted onto a new boot mode.  In this case arm 64.
What if a stub file was created that just contained enough GPU code to finish the job that bootcode.bin couldn't do itself.  The idea is that this + custom code gets loaded at a fixed address.   The head of the image contains the some secret stuff to get things going which then jumps to a fixed offset in the image.  Preferable in an ARM mode of some kind.

We can call the the file bootcode.bin.  But to make it you have to cat a special file onto the start of the new boot loader that just has enough GPU code to jump into arm mode at a fixed address.  So people like me that don't want to know anything about GPU code could finish the job.  The idea would be that the stub code is only what essential to starting up a single ARM cpu core.
@Nom-DePlume
Does you bootcode.bin work?  How much would it take to just get it to the point that it could unreset the ARMs to run at a fixed address.  What way I could possibly finish took over from where you started.   The idea would be to take your image and cat it with my ARM specific stuff.

What if a stub file was created that just contained enough GPU code to finish the job that bootcode.bin couldn't do itself.

That's not a stub - it's a start.elf replacement. If one has the tools and knowledge to write "enough GPU code to finish the job that bootcode.bin couldn't do itself" then you have basically replaced the firmware. Writing a replacement bootloader is more tricky (see Gordon's comment about the L2 cache usage), but significantly smaller.

I might be interested in attempting that.  A replacement bootcode.bin that just contains enough stuff to get the arm going at a fixed address.  The arm would do the rest based on info I can get from the web and such.
I think eventually you guys would want to make as much of the GPU as optional as possible, and perhaps running as much as the boot code on the arm as possible actually makes sense long term.

Sorry, I ment a replacement bootcode.bin that is built with a GPU stub at the front.

You could even call the GPU code at the front, "Magic Numbers".

So yes, the idea would be that the "Magic Numbers" just contains enough stuff to boot strap the ARM at a particular offset in the image.  All the rest of the code would be written on the ARM including loading any other files from whatever filesystem or where it would need to be loaded from?
@Nom-DePlume
What would it take to get your bootcode.bin to be in the form of magic numbers that just initializes enough of the RPI to get the ARM to unreset at a fixed offset?

Here is what I'm thinking long term here.
I take @Nom-DePlume version bootcode.bin which is a fixed size that fits in the L2 cache.  This L2 cache code jumps on the ARM to a fixed offset that need not be in the L2 cache.  This ARM is really just itself a stub to a stripped down Linux kernel that I compile with a custom config.txt.   This Custom Linux kernel has all the open source drivers and file systems to run a real boot system.
This preboot system would then be able to load the legacy components from the network or whatever using standard Linux code.  Which would include the existing bootcode.bin, start.elf, etc.
Like I'm not trying to replace the existing firmware. But I think a very complex bootloader could be make this way of which all of it would be open source code.

Basically, just create a bootcode.bin that's a single really large file that's:
bootcode.bin(GPU only)+armstub+linux-kernel+othergoo  which all gets called bootcode.bin.
That would essentually turn everything into a single big file that gets loaded from wherever.  The only thing that wouldn't need to be open sourced would be the GPU stub at the front.  That way all the networking code for the bootloader would be running on the ARM from open source code.

Are any existing options available that would allow me to prototype something like this on my own?
It seems like the only piece I'm missing here is the stripped down firmware files(whatever this would be bootcode.bin/start.elf,etc)  that I would cat onto the front of this whole system(which of course would not need to have any network code at all). Just enough to essentually jump into a standard Linux kernel image. In fact, I may not even need to write any code at all.  Just a bunch of config files.

@Electron752 there are Videocore IV assembler tools that generate code.  I forget where I got mine but everything has to be done with GPU asm because the arm part is not yet loaded, the MMU isnt initialized either so you are working with real addresses, etc.  This makes it harder to accomplish great meaningful programs because you have to do some of the heavy lifting yourself.
As mentioned above you have to limit the size of bootcode.bin to the size of L2 cache.  Basically if you were to write a totally open boot loader it would have to do a lot of things the same way.  There are some real world constraints on why it is the way it is.
The microcode in the SoC gets the ball rolling by knowing a few things like how to read FAT file systems and do USB booting and that is what loads bootcode.bin into the L2 cache and then just executes it arbitrarily.  it is not an elf file or anything else that way which means you kinda have to write it in assembler.
You really need to understand the VC IV system in order to go down this road with anything meaningful (my blit data to the uart is not really meaningful, I also rely on the default boot clock speed for 115.2k serial/uart and other things).  It is more of a hello world type program.

@Electron752 I have read the last few posts you wrote now, and basically you are talking about doing exactly what the RPi boot process is.
The SoC bootrom (hardcoded microcode in the chip) loads bootcode.bin into L2 cache and JMPs to its start and executes from there.
bootcode.bin then (usually) loads the elf files and executes them. This process brings up the arm part, initializes the mmu, etc.
Lastly it will load the linux kernel.  This is slightly more complicated because it will set ATAGS and some other things but meh.  It then executes the kernel and from there it is linux or freebsd or whatever the kernel happens to be.
If you just want to replace the kernel you may want to look into RPi bare metal as for the most part that is what that group is doing.  This is also easier because there are FAR more tools for arm development than there are for VC IV development.  There is a lot more information on it as well.  A config.txt change is all that is required to load your arm program instead of the linux kernel.  But you have to write your own schedulers and threading code (or be single threaded).  You have to do a lot of the heavy lifting yourself, which may not be that difficult depending on the task.  Linux does RTOS (realtime operating system) although I do not know if the RTOS extensions  have been fully ported to the RPi (it should be) so really there is a lot available to a great many people.
Stand on the shoulders of giants and all that.  I do not see a compelling need to replace the boot loader stuff.  I understand the political side of wanting it all open fighting against the corporate side that  requires certain things to not be open but I do not see that as a compelling enough reason personally.  Few people care that their desktop UEFI bios isnt open (some really do  though) and the boot loader here is kinda analogous to that.

Hi,
I don't think this has anything to do with open and closed. Or corporate vs. free.  That's a completely orthogonal issue in my opinion.
I was just thinking of were this could all be going and such.  Would anything stop someone who wanted to do some heavy lifting from say just recompiling u-boot or linux with the VC compiler?
Essentially have an arch/vc directory in the Linux tree?
Ideally, I could image in the way distance future one could have actual devices drivers in such compiled with the VC compiler and have a Linux version that has somethings compiled for VC and some for ARM.

If it were possible (I'm not sure it is), I am not sure what benefit having uboot running on the Videocore would give you over running it on the ARM (There is a u-boot that people have got running on PI). In fact I suspect it would be next to useless, with no USB, no ethernet etc support. Might be possible to add those with a lot of effort and insider knowledge (I'm not able to comment as I don't know enough), but again, this already works on the ARM side, so why bother.

Although the VPU is still unbeatable with specialised vector code, for standard scalar operations a single ARM core is quicker, so the loader should do as little as possible before starting the ARMs.

@Electron752 You can already do that.  The boot loader stuff is like the BIOS, once you get to the point of the linux kernel launching you can run any flavor of linux.  You may have to build the kernel to apply some patches that may not be in your distribution of choice but generally speaking any arm binary distribution should work more or less out of the box.
There is no reason to play with the bootloader stuff to accomplish that.
That is why I thought it was to have an "open BIOS" for the RPi hardware, because that is the only thing that would be gained.
Config.txt (which is loaded before the kernel is launched) can specify which kernel and some other options are loaded up at boot time.  Arch is more of a function of the "partition 1" (or 5 in noobs) data than boot or "partition 0" data.

Actually, I think u-boot is horrible.  The fact that it's a hacked on old version is Linux is really, really bad.
@pelwell : Ok so recompiling the whole OS doesn't make sense then since the ARM is typlically faster.
@Nom-DePlume : I'm not really sure what the whole point of this is actually!
I think the boot process isn't that great.  I think alot of the boot process could be improved, and I think u-boot which is the main alternative is really bad.
Ideally, the RPI should work just like NVIDIA, AMD, Intel, etc.  They all load firmware blobs into who knows where on demand, and I would think ideally the RPI should work that way too.
What started me down this loader path was that it appears it's impossible with the current design of the firmware to actually create an open firmware without redoing the whole thing including writing some kind of loader.

I was thinking, is I wanted to write a new loader yet keep compatibility with existing stuff.  I would assume there is not reason this new loader couldn't have a selectable option that keeps compatibility with existing stuff, correct?
I could image some kind of menu system like grub has where one of the options is the existing system.  Essentially all that would be required would be to load the standard bootcode.bin at the usual address and reset the VPU, correct?

@Electron752  The boot process is kinda immutable in the overall design.  The chip  itself has a bootrom, microcode that may not be changed and is programmed in at the time the chips are made.  That is what loads a file called "firmware.bin" which its name is also hardcoded.    This is done by Broadcom.
Firmware.bin starts out on the GPU side of things, and has to initialize ram, bring up the ARM core, and do some other  general set up tasks.  This is required of any process that loads because none of that is done for us by the chip itself.  So yes, the linux kernel could take over this job but logically it makes sense to not do this as part of the kernel.  The code to do this would never be accepted upstream.
The difference between the way Intel or AMD x86 chips work is that they start out enabled.  They then bring up the video second (the BIOS may bring it to a default state).  This is more of the standard way things are done but with the Broadcom Soc used in RPi its backwards.
U-boot is not the only way to do things, and I think what you are getting at is you would like to see the boot loader do PXE or whatever else itself?  That could be done in a "bare metal" application that would then load and switch to the kernel loaded.  No need for u-boot but it would replace the u-boot part with something else and operate in a grand scheme in the same way (load and replace itself).  Again, there is no reason to change the bootloader to do this as it gives you the tools to do exactly this out of the box.
The point about mentioning bare metal in my last post was to give you a pointer that may be more suitable.  This thread was really about how the USBBOOT was not working with a RPi 0 1.3 and not about the larger boot process.  The point about talking about the disassembler was shameless self promotion :)  In part cause I want a job somewhere that pays more than the 0 I am currently earning.

OK, I've move this discussion to the bare metal.
I think what's being missed here is not only do I want to replace the loader, but I also want to make an open firmware alternative available.  Unfortunately, that isn't really possible by adding onto the existing system because the usually firmware is already loaded at that point.  The only real option at that point is "kill" the running firmware that's already been loaded, and that's not a good design in my opinion.

Part of the firmware is on the chip and owned by Broadcom.  The bootcode.bin, start.elf, etc uses Broadcom APIs afaik (thus the LICENSE.Broadcom file that has shipped with RPi distros since 2012 or so).  That part cant be changed without doing what Compaq did in the 1980s (and it cost them $1M of 1980s money, back when $1M was a whole lot of money).   One group/person has to reverse everything and write a spec.  0 coding done by that group.  Another group which should be "clean" meaning they have 0 history of reversing anything, 0 history with the RPi boot process, etc would code the clean boot loader from the spec and only the spec and not ask for help or google anything (Compaq sequestered their developers while writing the IBM PC Bios clone).  That is the only way it survived court challenges.
Because of HDMI keys and other stuff that is in the bootrom you can get into DMCA issues (in the US) even if you are just doing interoperability (one of the exceptions to the DMCA).
So its a gamble on a less than trivial process, and for what benefit when you can write your own "kernel" that just loads from however you want it to load.  I mean how many people reflash their desktop bios to have a totally open boot loader?  Especially when UEFI can be particularly nasty when it comes to malware (separate memory, processor, etc on many systems).

OK, I think I understand what your saying.
It sound like what I want to really do is cause some of the components of the SOC that the RPI is based on to run a custom program chosen from the ARM.  It sounds like in this case this would specifically be the VPU.  The only issue is that the custom program wouldn't ever end.  I'm not sure if the custom program would be linked in with the kernel image, or if it's loaded from an separate file.
Do you know if something like that is possible?

I am not quite clear on what you are asking so I will offer more information in the hopes of the shotgun approach hitting your question.
The SoC microcode does the initial boot.  That just loads bootcode.bin.  Once it loads that file it hands off execution and basically stops running at least as a bootloader.  Bootcode.bin loads start.elf and executes that (this is when the rainbow screen appears).  At that point that is no longer running.  Start.elf goes through its load up and initialization process and eventually it loads the linux kernel (or whatever is specified in config.txt).  It then executes that and basically stops running.
While the SoC side does not ever completely 100% stop,  thus the mailbox functionality, the components related to the boot process do go away for all intents and purposes once the kernel is executed.  The linux kernel runs on the arm side while the other code lives in the vpu side.  Mailboxes are how you bridge that gap.
When you use mailboxes to communicate between the two sides of the chip there is code that does handle that which is not open but neither is the firmware inside of your monitor :)
The kernel or whatever program you choose to run would not be linked to the SoC code.  Mailboxes are used to communicate between the two (kinda like RPC calls).  The kernel is loaded into memory and run - it must be free standing meaning it cannot rely on code from the system itself.  It can rely on certain locations for things such as where the GPIO Function Select stuff is, or how to set the clock speed or whatever.  These are defined in part in the Broadcom datasheet as well as the open release they did for some of the VPU stuff.  There are also people that figured undocumented stuff out, wrote it down and that led to others being able to more easily learn it.
This is really not that different from knowing that a uart port on an x86 box was at port 0x3f8 (if memory serves).  That was documented and set up by the bios to map it so that developers could have a sane, somewhat stable platform to code for.  If memory addresses, ports, etc changed with each system it would ultimately mean software that works on one box would not work on another.
So if you wanted to write a custom boot loader program to replace u-boot or whatever you could.  Specify that ifile n config.txt and it will be executed (on the arm side).  I believe the only 3 files you will need in addition are config.txt, start.elf and bootcode.bin.
Your program can then have drivers for whatever hardware you want (ethernet, uart, etc).  That can fetch the OS kernel from whatever source it can talk to (local storage, tftp/bootpc, pxi, etc).  Using a similar process once you have loaded the new kernel you can just jump over to it and start executing and free the memory your bootloader used.

Yes, I think we have two different issue here that have gotten mixed up. The boot progress and the custom open firmware.  I think my ultimate goal here is simply to use as little of the existing stuff that gets loaded in the boot process as possible.  And it sounds like as long as mbox isn't used, almost none of the existing stuff would be used.
Part of my goal here is to get the VC4 open source driver to work on the RPI3 even though the firmware is calling into the expander.  I want to access the expander from the arm using the open source driver for it.  I can't do that if some of the firmware is still running.
I want my kernel to be able to do power management, thermal, clock, etc.  Most of which the information is all available.  But I don't want to use any of the existing stuff that got loaded during the boot process.

You wont have a meaningful system without using mailboxes.
https://github.com/raspberrypi/firmware/wiki/Mailboxes
Mailboxes facilitate communication between the ARM and the VideoCore. This page lists the available mailboxes/channels. Each mailbox is an 8-deep FIFO of 32-bit words, which can be read (popped)/written (pushed) by the ARM and VC. Only mailbox 0's status can trigger interrupts on the ARM, so MB 0 is always for communication from VC to ARM and MB 1 is for ARM to VC. The ARM should never write MB 0 or read MB 1.

Well, I guess a really simple way to accomplish what I'm looking for is to be able to set a custom handler for the unused mailbox. In this case 7.  That handler could do things for my kernel from the VC side that can't or couldn't easily be done from the ARM side.
I'm still not sure how this would stop the expander access though.

well if you really really wanted to you could replace bootcode.bin and have it bring up the arm and run threads on it or something.  But you should look at the instruction set for the VPU, I find it kinda small because it is not intended to work quite the same way as the arm side.
Here is where I got my list of opcodes for my disassembler so you can take a gander.  His 2013 disassembler works but it does not do all of the code analysis that I want which is what prompted me to write my own :)  Almost done with it too, but only 3 hours sleep today so I probably wont touch it again until tomorrow.
http://hermanhermitage.github.io/videocore-disjs/dis.html

Oh you also may have issues with the broadcom license, read it - its included in the boot partition of every raspbian distro since 2012.

That's the whole point.  I don't really want to replace bootcode.bin.  I really don't want to go in that direction, it's too difficult.
Perhaps I can simply overwrite the VPU interrupt vectors. I noticed some open source code that appears to be doing that.
Sorry you haven't gotten much sleep.

@Nom-DePlume
OK, I understand now what I need to do now. Essentially I need to completely reinitialize the entire system from the ARM.
Do you know offhand if during my initialization sequence if it's possible to initialize the VPU clock to be off?  Or perhaps in my power management initialization initialize the VPU to be off?
The idea would be to keep the VPU off and thus stop any expander access period, or possibly only power it back on after swaping out it's code from under it. Kind of like what your saying with the monitor.  Power down the monitor without actually assuming that it's functional.

You will probably get more answers on the RPi forums than  here.   This is for the usbboot software only, and this thread for the specific issue I raised.  https://www.raspberrypi.org/forums
There may even be a thread about doing this, I havent looked.

So the original topic was about usbboot.  I just happen to have recently bought a PI 0-w.  Is their any way I can help?
I've heard of some other companies support printing debug information on JTAG if you have the correct debugging hardware.  Any chance the soc supports something like that?  I've read someone the GPU actually does have a JTAG port on the board somewhere.

I do not have a Pi0w so I cant know for sure if it has the same pinouts as the Pi0 but I made a list of the pinouts here https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=175504

OK thanks.  It might take me awhile before I can do anything meaningful since I'm not used to dealing with those kinds of connectors.  I might have to find or make an adapter somewhere.

Please find another home for this chat.

I decided to give it another try with the latest firmware as of 2 days ago.  This too failed in exactly the same way as previously.
One thing I may have forgotten to mention is that when start.elf loads the activity light flashes green then immediately goes out.  That and the rainbow screen are the only indications I get that it gets as far as start.elf.
I also tried with:

an empty config.txt
removing all DTB files from the boot dir
removing fixup.dat (just because)
removing cmdline.txt

I have not tried all 16 permutations of those files but I could if its likely to help figure this out.
If there is something I should try then let me know and I will try it.

I have also tried all those things, except that I removed the kernel instead of cmdline.txt - start.elf does not get far enough to notice that the kernel is missing.

This being about usbboot and all, is it possible to get the RPI of any kind to boot from a CD of some kind? Or even a flash ram stick to run an installer of some kind?
What would it take to be able to get just enough information to write a replacement start.elf? I have access to online resources and such, I would simply just need @Nom-DePlume's jump address or I could even start from bootcode.bin.  I noticed more information is available on the VC4 then I expected.

Same problem with a Zero W:
al@al-desktop:~/rpi-ramdisk/multistrap-image$ sudo rpiboot -d boot/
Waiting for BCM2835/6/7
Sending bootcode.bin
Successful read 4 bytes
Waiting for BCM2835/6/7
Second stage boot server
File read: config.txt
File read: start.elf
File read: fixup.dat
File read: config.txt
File read: config.txt
File read: cmdline.txt
File read: kernel.img
File read: initrd
File read: bcm2708-rpi-0-w.dtb
File read: config.txt
File read: overlays/pi3-disable-bt.dtbo
File read: overlays/i2c-rtc.dtbo
Failed control transfer
Second stage boot server done

Followed by silence on the serial port.
Same boot files work fine on a SD card.

Have the same problem as @ali1234 with my Zero W and no issues with Zero.
I noticed that it boots OK  if you unplug the usb after the message "Second stage boot server" (you need to have USB PWR plugged in obviously)

Resolved with https://github.com/raspberrypi/usbboot/pull/17 - this also fixes another issue where Linux 4.11 might fail to boot (from rpiboot) with a longer dtb.

Tested, #17 fixes this both on the normally working Pi0 and the other one that wont do USB gadget if booted from SD.
Thanks.

Ok, actually I've found and fixed the bug with the bootcode that causes
this, it was getting stuck waiting for another transfer from the host.  I'm
assuming resetting the usb bus will also trigger the same behaviour though.
Will check in the code at some time anyway because there seems to be
another issue with this still not working with all pi zero hardware
Gordon

Finally got this working properly, usbboot has been updated but you'll need to wait for a new version of start.elf
Should be checked in soon


Make it possible to pass a path of GoPro Fusion images

Currently have to pass single image, make it possible to pass directory of images.
Should validate directory has front and back images.

@karamvirup expected format like this: https://github.com/trek-view/fusion2sphere#examples-macos


Python modules in sandbox

I am trying out Codapi as a documentation tool for our RESTful API – I like it very much, congrats on this project and its implementation!
:relaxed:
Is there any information available which Python modules are available in the sandbox when using the WASI engine?
I tried to import requests and urllib3, but both give me the same “ModuleNotFoundError”…
Is there a way to provide a Python sandbox demonstrating an HTTP request in Codapi?

Hi Toby, thanks for trying Codapi! There are only stdlib modules available in the Python WASI sandbox. Also, the network doesn't work, so no HTTP requests at all.
You can set up your own Codapi server with a Python sandbox and enable networking there if you'd like.

I see. Would that involve adding a sandbox and then adding the desired modules in requirements.txt? Or how would I enable networking?
Are there any security implications of enabling networking in Codapi?

That would involve adding a sandbox, installing all the necessary packages (like requests) and enabling external networking in Docker. With all the possible consequences of enabling networking in Docker.

OK, makes sense.
Wouldn’t it be safer to provide (one of) the HTTP libraries in the WASI engine, then? I read that urllib3 is available in pyodide – but I know too little about web assembly to make anything out of this information…
Is there a way to load e.g. the requests module in a <script> element, similar to how runno.js and wasi.js are loaded?

Network requests may work with Pyodide, but I haven't tried it (and Pyodide is not yet available as a Codapi engine).

Is there a way to load e.g. the requests module in a

OK, thanks for the support.
I added a Python sandbox following the documentation, added the requests module in requirements.txt, and enabled networking by setting the corresponding property in configs/config.json. Works great!

Glad to hear everything went smoothly! I'll probably try adding Pyodide later and let you know if it becomes a viable option.


corrected to clinical

Summary
corrected "client" to "clinical" #500
Thanks for contributing to Terra.
@cerner/terra

@PayalSawant Appreciate the work. However, let's follow the guidelines mentioned here to submit a PR: https://engineering.cerner.com/terra-ui/#/contributing/terra-ui/git-workflow


update log4j.properties file name

This PR fixs #4
Flink 1.12 updated the log4j configuration file name. If we continue to use the log4j.properties here. The kubernetes logs will not be accessible.
Updating the name here will fix the issue.
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/native_kubernetes/#logging

Maybe we also need to bump the Flink version to 1.12. Right?

Maybe we also need to bump the Flink version to 1.12. Right?

Yeah, I agree. Thanks for reminding.


Python API

Hi, is it possible to add Python API?

Hey @igrinis, a python API is not currently on our road-map.
However, since the core functionality exists under src/maluuba (minus the nodejs and C# bindings) The idea would be to expose that with python bindings to get a convenient python API.


Remote desktop can default to 1:1 ratio

Problem description
I hope that when remote desktop starts, the size of the picbox can be adjusted to match the remote resolution.
Proposal
int screenWidth = Screen.PrimaryScreen.Bounds.Width;
int screenHeight = Screen.PrimaryScreen.Bounds.Height;
if(screenWidth > d.Resolution.Width && screenHeight > d.Resolution.Height)
this.ClientSize = new Size(d.Resolution.Width, d.Resolution.Height);
Additional context
No response

yes pls, i have the issue when testing that only a part of their screen is shown

make it a pull request


Forward references from front matter.md blow something up

My markdown had a line like

Mitigation: blah blah blah ()*

Causes some sort of python failure, producing the following:

/usr/lib/python2.7/re.py(155)sub()
154     a replacement string to be used."""
--> 155     return _compile(pattern, flags).sub(repl, string, count)
156

ipdb>
Note that this gets dumped into the output file, not enabling debugging.

I'm pretty sure I fixed this


Use stderr for logging in prod

Using stderr in prod by default is a good idea (we made the change in the official recipe a while ago -- https://github.com/symfony/recipes/blob/master/symfony/monolog-bundle/3.7/config/packages/monolog.yaml#L51) as it makes it work out of the box with anything compatible with Docker, including Platform.sh.

Thanks Fabien.


An question about accuracy of proto reimplementation

Why did the proto accuracy increase from 84% to 89%？how is it different from the previous proto？

We change the original cased word embedding to uncased word embedding.


Issues

Goo app but have three problems.

The search bar is messed up. Every artist song i search is randomized, and I have to change it every time by sorting it. Please make the sorting options permanent.
No option to ignore mediastore arts it shows incorrect albumarts of the songs in an album.
Please add the option to use the default notification bar because I am using android 14.

Thanks for listening

Please make the sorting options permanent.
will do this

No option to ignore mediastore arts it shows incorrect albumarts of the songs in an album.
this is the only way the app is able to fetch album art. so do not want to remove/update it.

Please add the option to use the default notification bar because I am using android 14.
I am trying to use the new notification media style, but haven't been able to work it out


Display User Quiz Attempts and Scores on Dashboard Cards

Is your feature request related to a problem? Please describe.
Currently, our dashboard does not provide users with immediate feedback on the number of attempts and scores they have achieved for each quiz type.
Describe the solution you'd like
I propose to enhance the user interface by displaying the number of quiz attempts and the corresponding scores directly on the cards of each quiz type on the dashboard. This feature will allow users to quickly gauge their progress and performance across different quiz categories at a glance.
Describe alternatives you've considered
An alternative could be to create a separate progress tracking page where users can see their detailed quiz history. However, this would still require additional navigation and might not be as immediately impactful as having the information directly on the dashboard.
Additional context
Integrating this feature would significantly improve user engagement and provide a more rewarding and motivating experience by allowing users to easily monitor their improvement and achievements.

solved in branch


2.1.0-rc2: cleos net peers does not work

$ cleos net peers
generates:
Error 3200006: invalid http request
Error Details:
Unable to parse valid input from POST body
no parameter should be given

This works fine in 2.0.

Seems we can close this issue now that the code is committed for 2.1.0-rc3

Seems we can close this issue now that the code is committed for 2.1.0-rc3


feat: add module readme generator

This allows doodba private modules to benefit from having a generated readme.
The template is pretty similar to the one in https://github.com/OCA/maintainer-tools, but it removes all OCA-specific stuff, making it more suitable for doodba.
@moduon MT-956 https://github.com/OCA/maintainer-tools/pull/557

The CI failure is not related to this PR

Not sure what can happen. Can you give me any idea?

I opened https://github.com/Tecnativa/doodba-copier-template/issues/374 to keep both issues focused 😊

I see that #375 went ✅  so maybe it was just a temporary failure. I rebased; let's see...

Coool, it's green now!


How to compare ScriptObject with other ScriptObject?

I just want to make subscribe/unsubscribe methods
so:
const a = () => { }

subscribe(a) // internally doing: subscribers.Add(scriptObject)
unsubscribe(a) // subscribers.Remove(scriptObject)

but comparer always return false

HI @CrosRoad95,
It's often best to let the script engine deal with script objects. In this case, you can use JavaScript's Object.is method:
private bool Equals(ScriptObject func1, ScriptObject func2) {
	return func1.Engine.Equals(func2.Engine) && func1.Engine.Script.Object.@is(func1, func2);
}

You can see this technique in action here.
Good luck!

Hi again,
In the next ClearScript release, script objects will implement identity comparison.
Thanks!

Version 7.3.5 added support for script object identity comparison.


chore(operator): add openebs 2.11.1 cStor operator

This PR adds following changes:

Adds 2.11.1 RC1 cStor operator YAML

Special notes for your reviewer:

Didn't update the openebs.io/version label value on NDM-Operator.

Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[ ] Title of the PR starts with chart name (e.g. [stable/openebs])

Signed-off-by: mittachaitu<EMAIL_ADDRESS>

Overridden by #251


[Transaction] Standalone start workService and transaction cannot open at the same time.

Describe the bug
Standalone start workService and transaction cannot open at the same time.
To Reproduce
Steps to reproduce the behavior:

change standalone.conf transactionCoordinatorEnabled=true

start standalone. will not use transaction

change standalone.conf transactionCoordinatorEnabled=true

start standalone -nfw. can use transaction

Expected behavior
can use workService and transaction at the same time.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: mac

Additional context
Add any other context about the problem here.

Are you sure? Is it a recent regression? Do you have some stacktra c'è of an error?

It seems ok

I think __transaction_buffer_snapshot  can mean that transaction is ok.


Add HistoryArchiveAdapter to work with historyarchive package with test

Link to ingestion design doc
Link to detailed structure of history archive files produced by Stellar-Core
Link to examples of stellar-archivist usage
Link to integration with other services doc
Expand the design to implement HistoryArchiveAdapter that works with Archive in the historyarchive project.
Add a test that validates a bucket using HistoryArchiveAdapter from a file in the filesystem using a filesystem backend.

[x] implement HistoryArchiveAdapter
[x] test with file system or S3 adapter
[x] little tool to calculate stats for archives to prove usage. Tx/s over 5 min period.

This is the red portion in the image below:

pending review comment items:

[ ] StateReadCloser: https://github.com/stellar/go/pull/1216#discussion_r284794230

Here are some notes from our deep dive into history archive ledgers and buckets yesterday

GetState API should return a ledgerHeader object along with a StateReader.

StateReader.Read() should return “Dead”, “Alive”, or “Init” in addition to the xdr.LedgerEntry. Init only supported from protocol 11 onwards.

Buckets to be read in descending order, in the order of 9.snap, 9.curr, 8.snap, ..., 1.curr

Second implementation after MemoryStateReader can be a 18-way-merge reader which is similar to the merge step in merge sort, and outputs only a single entry per key which is always an “Init” entry. This should be a different API from StateReader to differentiate, since it would not have the incremental bucket-specific LedgerEntry values (ie no Alive and Dead entries).

TxSet, results, SCP do not need to be ingested right now since that will be used in the next stage when we have LedgerCloseData.

Testing — Unit Test that creates a bucket programmatically and verifies that the values read out from StateReader are correct. That is, it validates the read order and that the HistoryArchiveAdapter produces a StateReader that goes over all the required data from the HistoryArchive checkpoint file.

to move the conversation from email to here, copying @bartekn 's comments on above:

GetState API should return a ledgerHeader object along with a StateReader.

For the checkpoint ledger or for all the ledgers between previous and current checkpoint? This is data in ledger folder, correct?

StateReader.Read() should return “Dead”, “Alive”, or “Init” in addition to the xdr.LedgerEntry. Init only supported from protocol 11 onwards.

Buckets to be read in descending order, in the order of 9.snap, 9.curr, 8.snap, ..., 1.curr

Second implementation after MemoryStateReader can be a 18-way-merge reader which is similar to the merge step in merge sort, and outputs only a single entry per key which is always an “Init” entry. This should be a different API from StateReader to differentiate, since it would not have the incremental bucket-specific LedgerEntry values (ie no Alive and Dead entries).

I was thinking to keep StateReaders simple and always return raw data from buckets. Then if user doesn't need raw data they can use Processor provided by us that merges all the changes and returns flat data to the next item on the pipeline.
However, I'm wondering if there's any use case (other than tests) when users need raw bucket data instead of final state ("Init" entries only).

For the checkpoint ledger or for all the ledgers between previous and current checkpoint? This is data in ledger folder, correct?

Yes

I was thinking to keep StateReaders simple and always return raw data from buckets. Then if user doesn't need raw data they can use Processor provided by us that merges all the changes and returns flat data to the next item on the pipeline.
However, I'm wondering if there's any use case (other than tests) when users need raw bucket data instead of final state ("Init" entries only).

That's a good point. I think the question boils down to how much of the bucket architecture we want to expose to the end user of HistoryArchiveAdapter.
I'm of the opinion that we should abstract away the details of the bucket architecture. We can have a single StateReader API that outputs dead, alive, init entries. This API will also work for the 18-way-merge implementation which will only output init entries. If we want to expose more details of the archives in the future then it may require a different API and will probably have an entirely different use case too.
@MonsieurNicolas @tomquisel @ire-and-curses thoughts?

I think it's fine either way:
for the first version having a non optimal stream of changes coming from buckets is fine (ie, having entries getting created only to be updated or deleted in a different bucket).
I agree that we should try to avoid bleeding certain details of the bucket list deep into the pipeline.
For this to work, I think that we should try to expose this data with the same schema than the other "diffs":

buckets contain things as BucketEntry
later stellar-core exposes meta with streams of LedgerEntryChange.

STATE was added to help Horizon ingest data (before we only provided new state, but Horizon in some cases needs the "previous state"), so the meta is now generated as a pair (old "STATE", new state) (for updates/deletes) and creates are just creates (there is no previous state)

The mapping looks like this:

BucketEntry
LedgerEntryChange

LIVE
CREATED or UPDATED

DEAD
REMOVED

Starting with protocol 11, BucketEntry (with INIT) has a better mapping:

BucketEntry
LedgerEntryChange

INIT
CREATED

LIVE
UPDATED

DEAD
REMOVED

Now that I write this, it seems that trying to make the pre-protocol 11 work is going to be challenging.
You're going to need

either a database of sorts as to emit the proper meta (you need the "previous" value), and it's the wrong place to keep state (we should only keep state in processors that manage "views" of the ledger).
or implement something like 18-way merge (that provides the "previous value" for each entry)

The alternative (18-way merge) makes this problem irrelevant as what is going to be emitted will be "pure": only "CREATED" entries (no need to remember what the old value was), in addition to being more efficient.

ok, I can update the implementation to be an 18-way-merge implementation.
I'd want to merge what I have so far into the master branch since it gives a reasonable output and has basic testing too.
I can then work on the 18-way merge as an improvement off the new master, modifying my current implementation.
@MonsieurNicolas  does this sound good?

yes, sounds good!
On Tue, May 7, 2019 at 1:53 PM Nikhil Saraf<EMAIL_ADDRESS>wrote:

ok, I can update the implementation to be an 18-way-merge implementation.
I'd want to merge what I have so far into the master branch since it gives
a reasonable output and has basic testing too.
I can then work on the 18-way merge as an improvement off the new master,
modifying my current implementation
https://github.com/stellar/go/pull/1216.
@MonsieurNicolas https://github.com/MonsieurNicolas does this sound
good?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stellar/go/issues/1134#issuecomment-490250986, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABAEYA7KK6TKNOUHBYU45L3PUHT3XANCNFSM4HE7MG5A
.

Closing, as I believe this is complete as off this commit: https://github.com/stellar/go/commit/bd2c1a55262bdfee0542ee5d333d0a92a6003df6
If not, then please feel free to reopen


Spider: Allegheny County Finance and Development Commission

Spider Name:
alle_finance_dev
Website:
http://www.county.allegheny.pa.us/economic-development/authorities/meetings-reports/fdc/meetings.aspx
Scraping Notes:
Currently, the Finance and Development Commission (FDC) meetings include the meetings for all of the following organizations:
Higher Education Building Authority (HEBA)
Hospital Development Authority (HDA)
Industrial Development Authority (IDA)
Residential Finance Authority (RFA)

I'm working on this

Bad SSL on www.county.allegheny.pa.us domain
(http://www.county.allegheny.pa.us/economic-development/authorities/meetings-reports/fdc/meetings.aspx)
Use
https://alleghenycounty.us/economic-development/authorities/meetings-reports/fdc/meetings.aspx

I will work on this one.

Thanks @YujiShen ! Let us know if you have any issues!

Closed by #101


nvcompDecompressIsTypeSensitive declared with no implementation

nvcomp.h declares the nvcompDecompressIsTypeSensitive function but it is not defined anywhere.

This was removed in 2.0.1.


missing builds since 3.22.1

your Releases no longer include binaries after 3.22.1 and that version is the last one listed for downloading on your website (https://gnosis-safe.io/).
please fix this, that release is ~8 months old at this point.

We delivered a new desktop version based on v3.33.2
https://github.com/safe-global/safe-react/releases/tag/v3.33.2%2Bdesktop

@chaserene should be linked now 👍


support window and MonadOfNoReturn

What is the support window for streaming? I would like to drop support for versions of base prior to Applicative-Monad, and change all instances of 'return' in the code to 'pure'.

What's the point?

Also, there are some places where a 'Monad' constraint could be relaxed into an 'Applicative'

Be careful. We generally aim to make Stream a fast simulation of FreeT. Some redundant constraints are intentional for the sake of compatibility.

'return' is a blemish

It is, but we can wait as long as we want to remove it. I generally think support windows should be determined mostly by CPP burden, and the burden here is very low. Once it gets high enough to have a good reason to exclude 7.8, we can surely pull out other compat warts.

Be careful. We generally aim to make Stream a fast simulation of FreeT. Some redundant constraints are intentional for the sake of compatibility.

It is, but we can wait as long as we want to remove it. I generally think support windows should be determined mostly by CPP burden, and the burden here is very low. Once it gets high enough to have a good reason to exclude 7.8, we can surely pull out other compat warts.

Alright, I think I agree now. Closing.


QA failure at the end of 41_working_with_pipelines.adoc

While testing the 41_pipelines process i ran into an issue. My accident model pipeline @failed. It looks like the logs saying that pip was not able to install some requirements.  @diego-torres or @strangiato perhaps we can take a look tomorrow if i have trouble figuring it out with a fresh brain.
accident-detection-mg228-2376030189-full.log
accident-detection-mg228-2376030189-main-full.log

looks like it was unable to connect to this url to download the model:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: /rhods-public.s3.amazonaws.com/sample-data/accident-data/accident-sample.zip (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f29eba5d910>: Failed to resolve 'https' ([Errno -2] Name or service not known)"))
probably something to do with @gutseb connectivity?
@asheet , @thinko , can any of you guys have a chance to see if this is reproduced in your environments?
@gutseb , please try again.

got the same issue. I am able to reproduce.

It was the connection url that was making it fail. Fixed the url, now it's failing on the train model


Update pubspec.yaml

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 21.03%. Comparing base (8936c2f) to head (4787d62).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #447   +/-   ##
========================================
  Coverage    21.03%   21.03%
========================================
  Files          296      296
  Lines         9196     9196
========================================
  Hits          1934     1934
  Misses        7262     7262

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


HL-65: Application step 2 UI

Description :sparkles:
-Application step 2 UI implemented
-Common components refactored
Issues :bug:
Testing :alembic:
Screenshots :camera_flash:
Additional notes :spiral_notepad:
-Styled dropdown not done
-Validation and error handling not done
-Some issues with grid layout of the step 1 and 2 fields ( check abstract)
-Translations of the tooltips are missing

Codecov Report

Merging #150 (ed58721) into develop (a073f7b) will decrease coverage by 28.07%.
The diff coverage is 21.25%.

@@             Coverage Diff              @@
##           develop     #150       +/-   ##
============================================
- Coverage    71.18%   43.11%   -28.08%
============================================
  Files          260      101      -159
  Lines         5269     1619     -3650
  Branches       353      270       -83
============================================
- Hits          3751      698     -3053
+ Misses        1455      894      -561
+ Partials        63       27       -36

Impacted Files
Coverage Δ

...s/forms/application/step1/ApplicationFormStep1.tsx
0.00% <0.00%> (ø)

...orms/application/step1/companyInfo/CompanyInfo.tsx
82.05% <ø> (ø)

...forms/application/step1/useApplicationFormStep1.ts
0.00% <ø> (ø)

...s/forms/application/step2/ApplicationFormStep2.tsx
0.00% <0.00%> (ø)

...forms/application/step2/useApplicationFormStep2.ts
0.00% <0.00%> (ø)

...omponents/applications/forms/application/styled.ts
0.00% <0.00%> (ø)

frontend/benefit/applicant/src/constants.ts
100.00% <100.00%> (ø)

backend/benefit/companies/admin.py

backend/shared/shared/oidc/models.py

...nd/kesaseteli/companies/tests/data/company_data.py

... and 154 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a073f7b...ed58721. Read the comment docs.


Adding ability for deep object property lookup (_.get)

This is simular to that of lodash and i believe there is a big use for this functionality to be added to the core. One of the biggest benefits of this is that you no longer have to check for undefined up the object chain like so
Before this.object && this.object.one && this.object.one.two
Now _.get(this.object, 'one.two')
I found most of this on another post I cannot remember who did it but this is very useful
_.mixin({
  get: function (obj, key) {
      var type = typeof key;

    if (type == 'string' || type == "number") {
      key = ("" + key).replace(/\[(.*?)\]/,/\[(.*?)\]/, function (m, key) { //handle case where [1] may occur
        return '.' + key.replace(/["']/g,/["']/g, ""); //strip quotes
      }).split('.');
    }
    for (var i = 0, l = key.length; i < l; i++) {
      if (typeof obj !== 'undefined' && _.has(obj, key[i])) obj = obj[key[i]];
      else return undefined;
    }
    return obj;
  }
});

This has been proposed and rejected a few times (search for "string programming" or "deep path" and you'll find #2268, #1169, #1266 and others). Also check out underscore-contrib.


chore(blog): release notes for 2.1 release

See also https://github.com/asyncapi/spec/issues/536

@fmvilas there are 2 TODOs for me but these are cosmetics, you can review rest to speed up before release, no need to wait for final content I think


What are the parameters?

Great idea here Dave! Working on a Siri Shortcut that will make this even easier to use.
What were the parameters that rendered the best outputs?

Default parameters


1.23 - filter dropdown menus don't work on iOS

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
I know all attention is on 2.0.0 now, so it's ok if this is not applicable anymore...
Just wanted to note that the new status/tag filter dropdown menus are non-functional on iOS (Safari) as far as I can tell. I cannot tap/click/activate them at all.
👟 Reproduction steps
open kuma on mobile Safari
try to click e.g. the Status filter bubble.
👀 Expected behavior
menu opens
😓 Actual Behavior
nothing happens
🐻 Uptime-Kuma Version
1.23.0
💻 Operating System and Arch
ubuntu 20 / Docker
🌐 Browser
Mobile Safari - iOS 16
🐋 Docker Version
No response
🟩 NodeJS Version
No response
📝 Relevant log output
No response

Can confirm the same for ios 16.6 with mobile firefox (which technically is safari, too)

What do you mean exactly by don't work?
Do you by any chance mean https://github.com/louislam/uptime-kuma/pull/3607?
Is there something written in the log? (if you/somebody else has a mac)

I forgot to comment on this:

I know all attention is on 2.0.0 now, so it's ok if this is not applicable anymore...

There still will be a patch-release as always (see the 1.23.1-milestone), before https://github.com/louislam/uptime-kuma/pull/2720 can be merged into master
=> work shifts to the ´2.0.0´-milestone 😉
PRs from those with Apple devices welcome. I don't know if this is testable any other way.

Yes, I just update the changelog a bit:

Update: The last minor version (1.23.x) means that there won't be any new features introduced in v1. However, bug fixes will still be provided. You can expect to see 1.23.1, 1.23.2 and so on.

What do you mean exactly by don't work? (what button can't you click?)

These:

Do you by any chance mean https://github.com/louislam/uptime-kuma/pull/3607?

No - not related to that issue

Is there something written in the log?

You mean the log of the uptime kuma instance? No. Mobile browser log? Maybe- I don't know how to access/view that. Maybe I have to browse it while my phone is in some kind of debug mode? I'll check.

Can reproduce on BrowserStack's iPhone device.

using ios (BrowserStack):
the console  => no relevant errors.
macos ventura - firefox 115:

Seems to be a "feature" of safari ios: https://itnext.io/fixing-focus-for-safari-b5916fef1064

I have no idea but I searched a little and found a simple playground on MDN, I added some logging to it and a button, seems to work on my phone...
https://developer.mozilla.org/en-US/play?id=tDaotaNvwKc33Vt8CDIuJb4fHLbgP57V1C4Lq2rpXUqN09I7fORDXtCdduhIKjW%2BnV32mnArodGEGyJW

Idk, about your example. The following does not work:
https://play.vuejs.org/#eNqNkk1PAjEQhv/KpBcgQThwI2D8CCZ6EKPerAdYBijutk07Rcxm/7uz7YIfIehptzPP++btTEtxaW1vG1AMxchnTlkCjxTsudSqsMYRlA6XFSydKaDFYEtqqTOjPYFxC3QwBgbaL68dqUf9ZMFiPhAWNp8R8glgVDvyZ6G2cLE0WfBKj6WIFj0b/LrdaqpnjLQ6UjSYCXSM4/IejMZsPQ9ERgN9WGRFOh26ADe1DBgoEJQGFje6fkJTvj7X019j90fYOen/hd2Dx+P9Ctd0Y6Af6dIUp3Hwyg+hLJstVFU9/sPIRVeQ5y0t1aq38UbzestaKUVmCqtydFNLircoBZukDFLM8ty838UauYDdfT1bY/Z2pL7xu7omxYNDj26LUhx6NHMrpNSePN3jjv8PzcIsQs70ieYjepOHOmPCroJecOxvXEx7Gx+p0qtnP9kRar+/VB20JqvIS8FP9/rE1b/iDnqDqJO6EtUnuoQLmw==

I monkeyed around with the playground with some help from chatGPT. This version works on my iPhone, not sure if it can be adapted to Kuma
source

<template>

    <button type="button">
      Focus on me in div
    </button>

  <button @focus="order.push('focusin-btn')" @blur="order.push('focusout-btn')" type="button">
    Focus on me in button
  </button>

  Order is: {{ order }}
</template>

That is pretty trippy: The above button works on IOS and the below button works on chrome-linux

Should we add both v-on's to the div and the button?
(That sounds like a horrible plan, but I don't have better ideas)
as in this code?

macos ventura - firefox 115 (I don't have macos-safari avalibale anymore. If somebody can verify safari as well, that would be appreciated)

@CommanderStorm
I can confirm on MacOS Safari v16.5.2 that those dropdown menu filters are not working for me either.
https://github.com/louislam/uptime-kuma/assets/97180817/202d75f5-6f56-41e7-9eef-2d6c7cf959b9

The reason @focusin and @focusout are on the div element is that we want the drop-down to be kept open when the user clicks on the drop-down items. In this case the toggle button itself may lose focus, but the div is still in focus.
As for how to fix this bug, I have no idea. I'll learn to use browserstack when I get some free time soon.


[Bug]: Throw NPE during restarting AMS

What happened?
If SQL session had closed when execute multiple SQL operations in one transaction, the process id may only persistence in databse table_runtime, but not in others(table_optimizing_process, task_runtime). The status of optimzing will block in RUNNING, so we have to clear them in the database and restart AMS. but AMS will not load any task based on the process id that exists only in the table meta, so an exception will occur.
Affects Versions
master
What engines are you seeing the problem on?
AMS
How to reproduce

only persistence process id in table_runtime and update optimzing status as Running
restart AMS

Relevant log output
No response
Anything else
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

If SQL session had closed when execute multiple SQL operations in one transaction, the process id may only persistence in databse table_runtime, but not in others(table_optimizing_process, task_runtime)

I think we should use Transaction to avoid the non-atomic operation, rather than fixing abnormal states after starting AMS.

I think we should use Transaction to avoid the non-atomic operation, rather than fixing abnormal states after starting AMS.

I agree, but sometimes the state of the optimizating process goes wrong due to unexpected errors and we have to clean up the records in the database and restart AMS. On the other hand, I think it would be nice to enhance the robustness of AMS

A more specific example of "unexpected errors" would help us better understand the necessity of doing so.

A more specific example of "unexpected errors" would help us better understand the necessity of doing so.

I'd like to keep transaction first, but the system doesn't always run as we expect. I fount some errors make the state of process not correct or have to clean up and restart:

Only update table_runtime when start executing process, table_optimizing_process and table_runtime should be update or insert but it was not.
Those tables that are hard to rewrite block other pending tables. So I tried to stop the optimization by cleaning up the records. I know this is not a good solution and should be fixed by closing the process with self-optimizing disabled.
These issues need to be fixed by other PRs, the current PR just improve the robustness of the system and prevent other users from having this problem with another unfixed version

I think it's necessary to do more checks on recovered metadata during initialization.
However, I'm more concerned about the problem of inconsistent metadata. We should try to reproduce it, identify the cause of the problem, and fix it.


Adding new account then click "add and open" opens the 'default' storage account

I had 2 existing storage accounts setup. I then added another storage account name and key. Then i clicked "add & open" and it opened on the 'default' account (the account that was listed i think or the one i last opened during a session).
I would expect it to open the account I just added.

@cicorias thanks for reporting. I can't seem to reproduce the behavior (have plenty of accounts, add a new one, click 'Save + Open ' and it opens the recently added account). Which version are you running, the last release, source, or live build?

release build. i'll try with dev perhaps it's been fixed and close this now...


Add "unzip" package and fix sed on restarts

Added the unzip package for use in pre.sh scripts that download plugins for CounterStrikeSharp e.g.
if [[ ! -d ${STEAMAPPDIR}/game/csgo/addons/counterstrikesharp/plugins/BasicAdmin ]]; then
    wget -q https://github.com/Hackmastr/css-basic-admin/releases/download/6c6034a/css-BasicAdmin-6c6034a.zip -O css-basicadmin.zip && unzip -q css-basicadmin.zip -d ${STEAMAPPDIR}/game/csgo/
fi

As well as changing the sed command so if you restart the server with a different value (say you change hostname in docker-compose.yml) it will update the server.cfg file correctly

Thanks for your contribution!

I added the sed changes for if you mount your own config (-v ./server.cfg:/etc/server.cfg) so it rewrites any options you then give like CS2_SERVERNAME, since someone might have a config file that won't contain {{SERVER_HOSTNAME}} and so on

I added the sed changes for if you mount your own config (-v ./server.cfg:/etc/server.cfg) so it rewrites any options you then give like CS2_SERVERNAME, since someone might have a config file that won't contain {{SERVER_HOSTNAME}} and so on

That makes sense.
The reason I opted for Jinja2 style variable templating is that I'd like to add full Jinja2 template support at some point in the future. That would see /etc/server.cfg change to become /etc/server.cfg.j2 which would then be passed through the templating engine.
I'll get the Zip change merged in for you (thanks!) but I'd prefer to keep the templating style as it is.


Add connector type to Micrometer tags

Expose more information about metrics to allow more complex filtering
Closes #72

Thanks @pruivo


Save PNG from rs-capture.cpp

Required Info

Camera Model
{ D415 }

Firmware Version
<IP_ADDRESS>

Operating System & Version
Windows 10

Kernel Version (Linux Only)
N/A

Platform
PC

SDK Version
2.0

Language
C++

Segment

Issue Description
I am trying to alter "rs-capture.cpp" to also save a PNG file of the depth information.  I am trying to modify the code from "rs-save-to-disk.cpp" to achieve this, but I am either getting errors, or if-statements are being skipped and I am unsure why.
Here is the current version of the code I have:
`// Capture Example demonstrates how to
// capture depth and color video streams and render them to the screen
int main(int argc, char* argv[]) try
{
// Declare depth colorizer for pretty visualization of depth data
rs2::colorizer color_map;
auto depth_units = rs2::context().query_devices().front()
    .query_sensors().front().get_option(RS2_OPTION_DEPTH_UNITS);

// Declare RealSense pipeline, encapsulating the actual device and sensors
rs2::pipeline pipe;
// Start streaming with default recommended configuration
pipe.start();

// Capture 30 frames to give autoexposure, etc. a chance to settle
for (auto i = 0; i < 30; ++i) pipe.wait_for_frames();

/*
// Wait for the next set of frames from the camera. Now that autoexposure, etc.
// has settled, we will write these to disk
for (auto&& frame : pipe.wait_for_frames())
{
// We can only save video frames as pngs, so we skip the rest
if (auto vf = frame.asrs2::video_frame())
{
auto stream = frame.get_profile().stream_type();
// Use the colorizer to get an rgb image for the depth stream
if (vf.isrs2::depth_frame()) {
vf = color_map.process(frame);
}
        // Write images to disk
        std::stringstream png_file;
        png_file << "CMD-" << vf.get_profile().stream_name() << ".png";
        stbi_write_png(png_file.str().c_str(), vf.get_width(), vf.get_height(),
            vf.get_bytes_per_pixel(), vf.get_data(), vf.get_stride_in_bytes());
        std::cout << "Saved " << png_file.str() << std::endl;

    }
}

*/
rs2::frame data = pipe.wait_for_frames(); // Wait for next set of frames from the camera
rs2::frame depth = data.as<rs2::frameset>().get_depth_frame(); // Find and colorize the depth data
rs2::frame color = data.as<rs2::frameset>().get_color_frame(); // Find the color data
save_frame_depth_data("depth", depth, depth_units);

auto vf = data.as<rs2::video_frame>();
// Write images to disk
std::stringstream png_file;
png_file << "CMD-" << vf.get_profile().stream_name() << ".png";
stbi_write_png(png_file.str().c_str(), vf.get_width(), vf.get_height(),
    vf.get_bytes_per_pixel(), vf.get_data(), vf.get_stride_in_bytes());
std::cout << "Saved " << png_file.str() << std::endl;

return EXIT_SUCCESS;

}`
ERROR:
Exception thrown at 0x00007FFCED429709 in capture.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000DA8D94E5B0. Exception thrown at 0x00007FFCED429709 in capture.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000. Exception thrown at 0x00007FFCED429709 in capture.exe: Microsoft C++ exception: rs2::error at memory location 0x000000DA8D94E670.
Can anybody help me understand this better?

There is a possible long way round of getting the result you are aiming for (saving a PNG with rs-capture code).
First, use an adaptation of rs-capture to save the data in .bin binary file format.
https://github.com/IntelRealSense/librealsense/issues/1485#issuecomment-380851655
Secondly, convert the .bin file into a .png
https://github.com/IntelRealSense/librealsense/issues/4740#issuecomment-529819400

Hi @CarlSeeTwo  Do you require further assistance please, or can this case be closed?   Thanks!

I suppose it can be closed.  I never really figured out some of the C++ calls, and why creating an rvalue object wouldn't allow me to call some of the functions as when I create an Ivalue object, but I have pivoted away from this regardless and am trying something different.
Thank you for your help.

Thanks so much for the update!


Chessboard mounting

Thank you for your great work.
for chessboard detection from point cloud, does the chessboard need to be mounted on a stand or can a person hold it ? would that affect the detection of the chessboard ?

It is beneficial to use a stand. The reasons for that:

The reflected points from the person may cause wrong segmentation results: the points from the person and the chessboard may be clustered to one segment.
If the chessboard is not standstill in the hand of the person, the camera and the lidar should record the chessboard in the same time (synchronised).

Thank you. I figured that your first point is my problem.  I couldn't perform the measurement again so here is how I solved it ( for anyone facing similar problem in the future) :

run the 3 scripts normally (img_corner_est, pcd_corner_est, LM_opt)
based on the results you get after running LM_opt script, find the odd images which show that there is a corner between the person and the chessboard and filter them out along with their point cloud from the original image,pointcloud pairs you had in the folders img and pcd
rename the images and the point clouds and run the 3 scripts again.

Note: Make sure that the intrinsic parameters ('instrinsic_para'  in the config file) are accurate.
I attached an example of an odd image

@AwsKhalil  Could you tell me your enviroment of running ILCC. I cann't run the pcd_corners_est.detect_pcd_corners(). And I cann't solve it. Thank you very much!


Vulnerability in 1.19.x

VSCode Version: 1.19, 1.19.1, 1.19.2
OS Version: Linux, Mac, Windows

I've found a vulnerability in version 1.19 and above which allows attackers to compromise systems where vscode is running.
Could a maintainer contact me at<EMAIL_ADDRESS>for further details?

Emmmm, I think I found this, too. Is it the port 9333?

@ChiChou the fix for the above mentioned issue went into 1.19.3. Can you contact me at egamma at microsoft dot com and provide more details.

Hi @egamma , have you received the mail?

@ChiChou yes, I´ve received the mail and replied to it on the same day.

Sorry, Gmail marked it as junk.


aschaeffer/nonstaticfilepaths

This will replace the hardcoded filepaths with cubescript svars
It furthermore (bundled together with the pull-request in the data-repository "a_teammate/nonstaticsync" brings the code and the data repo back together.

ready for merge,
testers may download https://github.com/inexor-game/data/tree/a_teammate/nonstaticsync as data repository

tested, seems to work
:+1:

:+1:
Tested as well, sane.


Add inter segment tests for text search and fix bug for Lucene query parser creation

Add inter segment tests for text search and fix bug for Lucene query parser creation. Parser should be instantiated per query (search expression). Also added text search tests for the SQL path

Ran the profiler again locally to compare the numbers to previous run (when the doc id cache optimizations were implemented https://github.com/apache/incubator-pinot/pull/5199). No difference.


baremetal: remove default listen address

这个 PR 实现什么功能/修复什么问题:
remove default listen address
是否需要 backport 到之前的 release 分支:
release/2.10.0

/cc @Zexi @swordqiu

Codecov Report

Merging #1330 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #1330      +/-   ##
=========================================
- Coverage    7.85%   7.84%   -0.02%
=========================================
  Files         372     372
  Lines       58386   58386
=========================================
- Hits         4588    4581       -7
- Misses      53401   53405       +4
- Partials      397     400       +3

Flag
Coverage Δ

#aFlag
7.84% <ø> (-0.02%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/util/seclib2/seclib.go
91.48% <0%> (-4.26%)
:arrow_down:

pkg/appsrv/response.go
66.15% <0%> (-3.08%)
:arrow_down:

pkg/util/rbacutils/rbac.go
73.83% <0%> (-0.94%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7010a75...7d0fafa. Read the comment docs.

/lgtm
/approve


[fr] Support to-many associations replace without neither id or key

For example if Root has many Items it should be possible to save a following DTO (all items should be replaced).
I think it is a good idea to introduce new association mode for such behaviour.
Root {
  id
  items {
    // simple properties, but no id, no key
  }
}

Please pull the branch save-command-rafactor and use AssociatedSaveMode.VIOLENTLY_REPLACE


added article about import assertions

Added article about import assertions that might be available in the next build

The general information section is a bit misleading, as the file contents aren't actually checked to see if they match the provided import assertion, as they are loaded from local files we can be sure that the content is not malicious.
So the import assertion actually only hints at what the actual content type of the the import is.
Also, it would be better if there is an example for all available import assertion types and not only for JSON.

Thank you


Where are saved files stored?

I've launched the SDK, created a file and saved it but I can't find it anywhere in my file system.
I've launched the SDK with:
./iosdk_mac start

currently iosdk only launches docker images for editor and whisk,  so no files are saved.
next step is to introduce iosdk init <folder>  to create a new template to be modified and then iosdk start <folder> will allow to edit the files in the folder in your local folder. The init is actually already implemented but not yet the template.

In version 0.3 there is now the init command


chore/build repo

chore: build latest source

I don't know if this is the correct way to do it. But since you don't have CI set up I ran npm run build, then npm run package and checked in the files. I haven't tried bumping the version in package.json and I'm guessing since this isn't to be pushed to npm you'll just make a Github release on the repo?

Yeah, That is what I am doing I am just bumping up the release version.


[Tailwind] tokengate/Error

ℹ What is the context for these changes?

Makes use of Tailwind in @shopify-tokengate/Error component.
I had to modify shared/Text to add a record for colors. The reason behind this is that without this, Tailwind was not picking up the dynamic className for sbc-text-${color}. This led to sbc-text-critical not being defined or compiled.  There are two approaches to resolving this, and I chose one that resulted in more code up front, but less compiled code.

Safelisting classes

I did not choose this path because it resulted in additional classes being compiled during the build process. When safelisting with a pattern for the text-colors, I received classes for all of the opacity ranges of the text colors.

Explicitly defining the color classes that will be used (and should be compiled).

🕹 Demonstration

🎩 How can this be tophatted?

📖  Storybook
✅ Checklist

[ ] Tested on mobile
[ ] Tested on multiple browsers
[ ] Tested for accessibility
[ ] Includes unit tests
[ ] Updated relevant documentation for the changes (if necessary)

Update -- rebased to factor in bottom padding on the card. Previous commit would have had 8px additional on the bottom of the card.


Allow to configure maxIdleTime property for HttpClient

Since 0.9.0.RELEASE of reactor netty there is a possibility
to configure the pooled connection max idle time.
This commit gives the possibility to set it via spring configuration.

@Ziemowit Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@Ziemowit Thank you for signing the Contributor License Agreement!


Radios with more information

There are use cases to show additional descriptive information with a radio button.

It is likely that this would also be helpful as an option for checkboxes, but I do not have use cases for this as of writing.
Notes/screenshots:
https://docs.google.com/document/d/13NkTdfpH7ZD7gBsmhYgjlux9K2mKa5s9t5JNmFV_KpI/edit?usp=sharing

@mcoker can you take a look at this? Is there a way to create this using existing components or do we need to create a custom radio or check component that includes a block of descriptive text?

@mcarrano currently you can build something similar using the content component's <small> element, which is used for "side comments, such as small print." You can read more about that here - https://www.w3.org/TR/2011/WD-html5-author-20110809/the-small-element.html
And here is a demo - https://codepen.io/mcoker/pen/yLLLYor
As for whether we want to build something to support this layout with radios/checks, we could build support for this in the radio and check components themselves, or this could be some sort of description text associated with a Form component's Form Group, and used with any sort of form group in a form.

Hi all!
Here are my design proposals for solving the use cases listed in @katierik requirement docs.
https://marvelapp.com/50eh328/screen/64407412 (first page has all three use cases, next page has spacing/specs called out)
@mcarrano I know there have been previous requests for tiles and their hover/selected states. I included design proposals in the mock above. Do you think this should be a separate component/issue?
cc: @mcoker @mcarrano

@lboehling these designs looks great. @mcoker do you see any issues? Also, how would we implement this? Do you see it as part of the radio and check components or some separate element that is added to the form?
Regarding the hover/select states for cards, there is another issue open for that here: https://github.com/patternfly/patternfly-next/issues/2470  Can you add those mocks to this issue? Thanks.

The stacked descriptions look like something that would be built into the radio component itself. The description would be placed in the layout underneath the radio label.
The horizontal descriptions are a bit different. It seems like there is an overall layout (in the designs) that these radio components are a part of, and the radio/label/description are positioned by that layout. In that case, the description would not be a part of the radio component itself, and instead text that is adjacent to a radio element. In Josh's mockup, the description in the horizontal/tabular radios is black. If we can use black text, I think that's as simple as using a radio in a table, and simply putting the description in the adjacent cell (as normal text). Would that be acceptable? If not, the radio descriptions in @lboehling's design looks like it would be "helper text" in a horizontal form layout.
As a side note, would we also add this for the check component? They have the same layout currently.

I think this would be good to add to both checkbox and radio components.

I agree @mcoker The radio buttons with horizontal descriptions can be achieved with a layout the includes radios (or checkboxes) and text in a two column table. Therefore, I think that I would limit this issue to provide the vertically stacked description as part of the component. That was what I saw as the original ask. Does that make sense?

@mcarrano sounds good to me!

@mcoker @lboehling can we consider this done? I revisited the designs and it seems like the first solution, Radios with stacked descriptions, addresses this issue. If you feel like you have everything you need, @mcoker , then I can open a new issue for core implementation and add to the prioritized backlog.

@mcarrano Yep, go ahead and open an issue. Thanks @mcarrano and @lboehling

I've opened a new core issue here: https://github.com/patternfly/patternfly-next/issues/2526
@lboehling can you just upload your original artwork to Sketch Cloud and add a link for future reference? Thanks.

@mcarrano added to the first comment in this issue!

Thanks @lboehling . Will close this one out.


BUG: scipy.sparse.linalg.spsolve: fix memory error caused from overflowing signed 32-bit int input to doubleCalloc

BUG: For big linear systems AX = B such as when the right hand side B is of size 46679 x 46680 containing doubles, spsolve(A, B) gets stuck in a runtime memory error that looks like below:
Python(8835,0x10eafadc0) malloc: can't allocate region
:*** mach_vm_map(size=18446744065245585408, flags: 100) failed (error code=3)
Python(8835,0x10eafadc0) malloc: *** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
  File "sparse_solve.py", line 14, in <module>
    x = spsolve(A, B)
  File "/Users/liviofetahu/sesco-llc/matricez/env/lib/python3.8/site-packages/scipy-1.8.0.dev0+1921.c30fefc-py3.8-macosx-10.15-x86_64.egg/scipy/sparse/linalg/dsolve/linsolve.py", line 204, in spsolve
    x, info = _superlu.gssv(N, A.nnz, A.data, A.indices, A.indptr,
RuntimeError: SUPERLU_MALLOC failed for buf in doubleCalloc()
 at line 693 in file scipy/sparse/linalg/dsolve/SuperLU/SRC/dmemory.c

which is caused by the Python spsolve function making a call to x, info = _superlu.gssv(N, A.nnz, A.data, A.indices, A.indptr, b, flag, options=options) where the _superlu module is implemented in C and exposed to Python through the low level Python C Extension Modules API.
Py_gssv in C calls gssv (inner-ly), and gssv in turn calls dgstrs, with this last one calling doubleCalloc(n * nrhs) where int n = L->nrow and int nrhs = B->ncol. Finally, doubleCalloc makes a call to the SUPERLU_MALLOC macro
buf = (double*) SUPERLU_MALLOC((size_t)n * sizeof(double));
if( !buf ) {
	ABORT("SUPERLU_MALLOC failed for buf in doubleCalloc()\n");
}

with the macro being defined as the void* superlu_python_module_malloc(size_t size) function that sets some member mem_ptr = malloc(size); and returns nullptr if malloc returns nullptr (meaning that the system wasn’t able to return a valid pointer to usable contiguous memory because it couldn’t allocate it), so buf in doubleCalloc is a null pointer and doubleCalloc aborts with the failure message explained above -- all this due to (L->nrow * B->ncol) exceeding the maximum value of a signed 32-bit int.
Fixes #14984

The linked issue is https://github.com/scipy/scipy/issues/14984

Who are some of the code owners that would be best to request a pull request code review from @tylerjereddy ?

@liviofetahu We are basically pulling the changes to SuperLU from upstream https://github.com/xiaoyeli/superlu They should be fixed there to avoid divergence between copies.

Hmm we also have the patch file https://github.com/scipy/scipy/blob/master/scipy/sparse/linalg/dsolve/SuperLU/scipychanges.patch
I'm not sure as sure as I was writing my previous comment.

Hmm we also have the patch file https://github.com/scipy/scipy/blob/master/scipy/sparse/linalg/dsolve/SuperLU/scipychanges.patch
I'm not sure as sure as I was writing my previous comment.

I took a look at the patch file @ilayn and it seems like this bug is not fixed there.

@liviofetahu thanks for this fix. To make the request of @ilayn more clear, can you please:

Add these changes to scipychanges.patch, so that in case we update to a new SuperLU version this fix does not silently disappear.
Submit these changes also as a PR to https://github.com/xiaoyeli/superlu

Thanks @liviofetahu! Upstream PR was already merged, so no need anymore to include in our patch file. I rebased to fix the merge conflicts.

Thanks @liviofetahu! Upstream PR was already merged, so no need anymore to include in our patch file. I rebased to fix the merge conflicts.

Thanks for taking care of this @rgommers -- I forgot to mention that I forgot to include my changes in scipychanges.patch but I'm glad to hear from you that there is no need anymore to include the changes in the patch file.

There was also a follow up commit upstream to fix this bug for other precisions, but rather than sync just that patch, we should just update all of SuperLU in one go.

There was also a follow up commit upstream to fix this bug for other precisions, but rather than sync just that patch, we should just update all of SuperLU in one go.

That sounds good -- now it shouldn't be hard to put these changes into the other SuperLU source files that handle the other types of precision.

CI failures are unrelated, in it goes. Thanks @liviofetahu!

Sounds good @rgommers -- working on the fixes for the other source files in the upstream SuperLU repo that are responsible for the various kinds of precision.

You mean you'll submit a new PR here to sync https://github.com/xiaoyeli/superlu/commit/afe15f3e8ea0b5e3b13d8aaa01377ce70165c385? That is the fix for the other precisions, unless I'm missing something.

You mean you'll submit a new PR here to sync xiaoyeli/superlu@afe15f3? That is the fix for the other precisions, unless I'm missing something.

Yes, that was exactly my plan


SDK 35 support

Cannot run tests with SDK 35. Error: Package targetSdkVersion=35 > maxSdkVersion=34
Possible solution:
Add new api level

Thanks. I think you got an answer here so let me close this issue
https://github.com/robolectric/robolectric/issues/9203


I want access to database

I'm working on a project "quran Inflections" and I cloned your website for this purpose
now I need access to the database
@mmahalwy

Salam @ehabdayoub I'd suggest adding yourself to our slack account.


[SQL] Add Support for 'varchar2'

Varchar changes to red but varchar2 doesn't

Could you make a PR for this?

Is this in the SQL standard, Postgres, MySQL or both?

It is Oracle.
Oracle suggests the use of varchar2 over varchar.

I'd add it as just a built_in to the basic SQL then.
https://github.com/highlightjs/highlight.js/blob/master/src/languages/sql.js

Believe this was just merged.

https://github.com/highlightjs/highlight.js/pull/2163


Consider whether or not to add an option not to write parquet and json

Img2dataset currently always write json stats files and parquet metadata files.
It seems like the right thing to do to me
But maybe there are some cases when it's not good
Let's try to think if that's the case

Another option might be to save those in a _stats or _metadata subdir


fix doc strings for csr_array and friends

Fix doc strings for csr_array and friends and sparray

I have two independent verifications that the docs are fixed with this PR, so I'll merge now!


Prevent creating unnecessary momentum variables.

Closes #52.

@n2cholas I was looking at this in prep for a tweak to RMSProp that includes a trace, instead of the conditional and multiples chain defs, what about
def chain(*args: optax.GradientTransformation) -> optax.GradientTransformation:
    """Applies a list of chainable update transformations.

    Given a sequence of chainable transforms, `chain` returns an `init_fn`
    that constructs a `state` by concatenating the states of the individual
    transforms, and returns an `update_fn` which chains the update transformations
    feeding the appropriate state to each.

    Args:
      *args: a sequence of chainable (init_fn, update_fn) tuples.

    Returns:
      A single (init_fn, update_fn) tuple.
    """

    init_fns, update_fns = zip(*filter(None, args))

    def init_fn(params):
        return [fn(params) for fn in init_fns]

    def update_fn(updates, state, params=None):
        if len(update_fns) != len(state):
            raise ValueError('The number of updates and states has to be the same in '
                             'chain! Make sure you have called init first!')

        new_state = []
        for s, fn in zip(state, update_fns):
            updates, new_s = fn(updates, s, params)
            new_state.append(new_s)
        return updates, new_state

    return optax.GradientTransformation(init_fn, update_fn)

def rmsprop(
        learning_rate: ScalarOrSchedule,
        decay: float = 0.9,
        momentum: float = 0.,
        eps: float = 1e-8,
        centered: bool = False,
        nesterov: bool = False,
        initial_scale: float = 0.) -> optax.GradientTransformation:

    if centered:
        return chain(
            scale_by_stddev(decay=decay, eps=eps, initial_scale=initial_scale),
            scale_by_learning_rate(learning_rate),
            optax.trace(decay=momentum, nesterov=nesterov) if momentum > 0 else None
        )
    return chain(
        scale_by_rms(decay=decay, eps=eps, initial_scale=initial_scale),
        scale_by_learning_rate(learning_rate),
        optax.trace(decay=momentum, nesterov=nesterov) if momentum > 0 else None
    )

@rwightman good point, multiple chains are a bit ugly, but I'm not sure about adding the None filter to chain (seems a bit overfit to this usecase). Perhaps instead we could use:
optax.trace(decay=momentum, nesterov=nesterov) if momentum > 0 else optax.identity()

The identity is a no-op, and this keeps chain clean. What do you think?

@n2cholas I didn't realize there was already an identity, yeah that works too.

Great! For this PR, I'll keep the branch, since when momentum=0 you don't need to use chain at all and can just return scale or scale_by_schedule. But I agree that it'll keep the code clean for the RMSProp case.


[REVIEW] HighLevelGraph: now returns iterators

This PR makes HighLevelGraph.items(), .keys(), .values() return iterators, which speed up the calls and reduce memory use.
Notice, dict in python3 returns dict_items, dict_keys, dict_values, which support len() but apparently it is not possible to create them at the Python level cleanly thus I don't think it is worth the extra complexity to implement our own.

[x] Tests added / passed
[x] Passes black dask / flake8 dask

This seems reasonable to me.  I don't think this breaks anything but it would good to check in with other libraries which rely on HighLevelGraph such as dask-cudf and dask-ml .
cc @rjzamora @TomAugspurger

I'm curious, what is the background here?  What motivated this change?

^ this would be useful information for any PR like this, by the way.

No objections from me either way though.  This change seems innocuous

The motivation is speeding up the calls and reduce memory use :P
I noticed that HighLevelGraph is used as a regular dict including calls to items(), keys(), values(), which, in the old version, are expensive operations:
def items(self):
    def items(self):
        items = []
        return toolz.unique(
        seen = set()
            toolz.concat([layer.items() for layer in self.layers.values()]),
        for d in self.layers.values():
            key=operator.itemgetter(0),
            for key in d:
        )
                if key not in seen:
                    seen.add(key)
                    items.append((key, d[key]))
        return items

I noticed that HighLevelGraph is used as a regular dict including calls to items(), keys(), values(), which, in the old version, are expensive operations:

Did you determine this through benchmarking an issue, or did you just notice these methods weren't returning iterators? Mapping items, keys, and values in python aren't just iterators - they also support __contains__, and multiple traversals. While the iterator might satisfy our internal use, it also might break others code as the interface has changed. It would be nice to see some benchmark validating that this change improves some real example in a measurable way.

The motivation is speeding up the calls and reduce memory use :P

I take it that the context here is highly private then?  How much of an impact did this have?  In what kind of workload?  These sorts of details help maintainers understand if this change is important and people should spend volunteer time on it, or if we should reject/ignore it to spend time on other more important topics.
Given that it's coming from you specifically, I'm almost certain that it's important, but it would still be generally appreciated if you can provide more context when you suggest changes like this.  I personally would appreciate increased transparency.

Fair enough, sorry for that.
Honestly, I haven't given this PR much thought. I noticed that we are constructing these new lists, which is not super expensive, but in a code path where we are discussing the use of isinstance(x, tuple) vs istask(x), it seemed worth optimizing.
However, I must have confused myself because I thought this PR had an real impact on a workflow I am exploring but running in a clean environment I cannot reproduce it.
To avoid being a complete wast of your time, I have clean up the code and made HighLevelGraph conform 100% with collections.abc.Mapping by returning KeysView, ItemsView, and ValuesView.

Thanks for the PR @madsbk!
Apologies for coming late into this discussion. In #4918 we intentionally overloaded the values method because the default Mapping.values implementation is ~10x slower for HighLeveLGraphs with many layers. In particular, this leads to a noticeable performance decrease for large collections with the distributed scheduler as distributed.futures_of uses HighLevelGraph.values().
I should have included a code comment about this in #4918 as this additional context is definitely not clear from looking at the code. I can add a comment as well as a benchmark to dask-benchmarks so we'll be able to track the performance of HighLevelGraph.values(), etc.
I'm in favor of keeping the existing implementation in order to avoid a performance regression

I stand corrected, thanks @jrbourbeau I will close this issue :)
Sorry for the noise.

Thanks for the comment @jrbourbeau, totally forgot about that PR.


add README instructions

any usage instructions would be very handy

Is this library supposed to be used as a plugin for TC or a plugin for Newman?  Any usage instructions would be fantastic.

Agree.
Can't get it work from newman command line..
-r=cli,teamcity
-r=cli,newman-reporter-teamcity
Is it possible at al?

Usage from code is pretty strightforward (haven't checked it out): https://github.com/postmanlabs/newman#using-newman-as-a-nodejs-module
Opened an issue for newman to clarify their position on subject with calling thirdparty reporters from CLI or just support popular CI https://github.com/postmanlabs/newman/issues/910

I tried to use it from code and couldn't get it to work.  Not much documentation on how to do that with a custom plugin from newman.   Anyway, a good solution is to just use the junit reporter and in your teamcity build click Build Features and select XML report processing and then point to the report folder you setup in the newman config (newman/ by default).

To run from command line when newman is installed globally, reporter must be installed globally too
npm i -g newman-reporter-teamcity
newman team improved their documentation https://github.com/postmanlabs/newman/pull/912

#5 should be good enough to get folks going.  Thanks for writing this plugin!


fix: Audit fixes

Audit fixes

Also merging this, to get us into the snap store.
@souhaillM @boyuan-chen @mmontour1306 please check the PR if you got some times. Thank you.