MQTT OTA csum error in reboot after "OTA sucess"

Hi,
with latest commits of homie-esp8266 I get the following result when updating my ESP8266-07s using homie-ota:
✴ OTA available (version 1.0.11)
OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded

Receiving OTA firmware (353600/353600)...
✔ OTA success
Triggering OTA_SUCCESSFUL event...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 51)...

Device is idle
↻ Rebooting...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 375)...
��� MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...
ets Jan  8 2013,rst cause:1, boot mode:(3,6)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3de0c112
@cp:0
ld
ets Jan  8 2013,rst cause:4, boot mode:(3,6)
wdt reset
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x1d
csum 0x1d
csum err
Flashing same binary using serial works.
Any ideas?
Thanks,
Alex

I am experiencing something similar.  I can get the OTA to upload to the device, and it apparently succeeds, but the device fails to reboot.
If I flash the same image via serial, it works.
I am building with PlatformIO, if that makes a difference.  I have the latest code as of today from the develop branch and the latest versions of the dependencies, from their respective repositories.
Here's my output, for reference.  [...snip...] is where I took out long sequences of similar messages:
`** Booting into normal mode **
{} Stored configuration
  • Hardware device ID: d1008cef
  • Device ID: firelight1
  • Boot mode: normal
  • Name: FireLight
  • Wi-Fi:
    ◦ SSID: clough
    ◦ Password not shown
  • MQTT:
    ◦ Host: mqtt
    ◦ Port: 1883
    ◦ Base topic: devices/
    ◦ Auth? no
  • OTA:
    ◦ Enabled? yes
↕ Attempting to connect to Wi-Fi...

✔ Wi-Fi connected
Triggering WIFI_CONNECTED event...
↕ Attempting to connect to MQTT...
Sending initial information...
✔ MQTT ready
Triggering MQTT_CONNECTED event...
Calling setup function...
Sending statistics...
  • Wi-Fi signal quality: 100%
  • Uptime: 1s
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 1)...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 2)...

[...snip...]

Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 21)...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 22)...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 23)...
✴ OTA available (version 1.0.10)
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 24)...

OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (1055/348720)...
Receiving OTA firmware (2150/348720)...
Receiving OTA firmware (3245/348720)...

[...snip...]

Receiving OTA firmware (344885/348720)...
Receiving OTA firmware (345980/348720)...
Receiving OTA firmware (347075/348720)...
Receiving OTA firmware (348170/348720)...
Receiving OTA firmware (348720/348720)...
✔ OTA success
Triggering OTA_SUCCESSFUL event...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 25)...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 26)...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 27)...

[...snip...]

Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 341)...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 342)...
Device is idle
↻ Rebooting...
✖ MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...

 ets Jan  8 2013,rst cause:2, boot mode:(3,7)

load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3ffe84e8
@cp:0
ld
e:
 ets Jan  8 2013,rst cause:3, boot mode:(3,7)

ets_main.c`

Oh, I'm using the latest homie-ota python code, and I'm using mosquitto as my mqtt broker, if any of those things matter.

Just to confirm the issue - I experienced the same,  but with some minor differences / additions:

not using platform.io
first try worked flawlessly,  second and third rendered device irresponsible (stuck in reboot)
I must have been lucky with the first try.

@mrpace2 mind taking a look? #208 seems related

Since we responded with 200 we did commit the update to flash and the arduino/esp8266 Updater code told us that the update was successfully written to flash. With #208, he got a 400. That is, we found that something was wrong with the OTA payload or the MD5.
The one commonality between #204 and #208 is that in both cases we do reboot. According to https://github.com/esp8266/Arduino/issues/1017, even the latest arduino/esp8266 still has issues with restart after OTA. Could this be our problem here? @clough42, @Gulaschcowboy can you please check if it helps to reset or power cycle your device after serial flashing but before trying OTA update?

@marvinroger I wonder about the Homie output in between ↻ Rebooting... and ets Jan  8 2013,rst cause:.... Right after we print ↻ Rebooting... we flush the serial port and we call ESP.restart(). Shouldn't we see the boot message before anything else from Homie? Could it be possible that it restarted again during startup?
Also, varying reset causes in the logs above (1=normal boot, 4=watchdog, 2=reset pin, 3=software reset). See https://github.com/esp8266/Arduino/blob/master/doc/boards.md#boot-messages-and-modes

@mrpace2 the serial output after the ESP.restart() is not alarming (see https://github.com/esp8266/Arduino/blob/master/cores/esp8266/Esp.cpp#L140), the reboot is not immediate.
They both use homie-ota, which, correct me if I am wrong, does not send an MD5. So the flash appears as successful, but the Updater does not actually check if the firmware is valid. @clough42 said

If I reset the device, I get only the off-baud garbage string and the device doesn't come up

So the flashed firmware is definitely wrong. Which, along with #208, makes me think the b64 decoding is broken.

@clough42 @Gulaschcowboy can you provide your firmwares binaries?

Can you test again with the latest git rev?

Hi Marvin,
tested right now with latest homie-ota and latest homie rev. Result is similar:
✴ OTA available (version 1.0.2)
OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (985/350673)...

Receiving OTA firmware (349825/350673)...
Receiving OTA firmware (350672/350672)...
✔ OTA success
Triggering OTA_SUCCESSFUL event...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 50)...

Device is idle
↻ Rebooting...
✖ MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...
ets Jan  8 2013,rst cause:2, boot mode:(3,6)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3de0c112
@cp:0
ld
ets Jan  8 2013,rst cause:4, boot mode:(3,6)
wdt reset
load 0x4010f000, len 1384, room 16
tail 8
chksum 0xd6
csum 0xd6
csum err
ets_main.c

✔ OTA success
...
v3de0c112
@cp:0
ld

indicates that

the Homie OTA update was successful,
the restart was successful, and that
eboot successfully copied the OTA update from the flash update region to the "hot" sketch area.

It crashed afterwards:
ets Jan 8 2013,rst cause:4, boot mode:(3,6)

is a watchdog reset. It seems to have happened before Homie's first log output. Maybe the firmware was published and saved correctly, but the firmware binary itself was broken? Does the same BIN file work when you upload it over the serial port?
If you share your BIN and BASE64 files, I could try them out on one of my setups. I don't have an ESP-07, though.
Another thing you could try are using MD5 checksums. Basically, you checksum the BIN file and Homie verifies the checksum right before committing the received OTA update to flash. This would help ruling out any issues with

base64-encoding,
MQTT publishing or
base64-decoding inside Homie.

Thanks for sending the files. I will try them out in a bit. I can test on Nodemcu.
Homie automatically detects is a firmware blob is binary or base64, based on the first firmware byte which is always 0xE9. It did detect base64 in your case.
Recent homie-ota does support base64. It was added a few weeks ago. A few minutes ago I created a new homie-ota PR adding support for MD5 checksums for the new OTA sequence (see #220).

Here's the last binary I attempted to use:
firelight.zip
This is a zip file, containing the .bin file.  It's compiled for a WeMos D1 mini, which has a 4M flash, with the default PlatformIO configuration.  I'm actually using an ESP12F module, but the specs are the same.
I am attempting to flash with Homie-OTA.
The source code is here:  https://github.com/clough42/homie-firelight/blob/master/src/main.cpp

@clough42, thanks, will test shortly.
@Gulaschcowboy, can you also provide mqtt and serial log files and the base64 file you were uploading?

@clough42 Running the latest Homie code, I uploaded the bin file you provided (a) using my little script (attached) and (b) using the latest homie-ota. I can upload your bin file without problems. The board restarts and launches your firmware. See the attached logs.
Please try again:

recompile your app to use the latest homie-esp8266
download the BIN file once over serial
hard-reset or power cycle your board
run OTA either using the latest homie-ota or something along the lines of the bash script that I attached.

Please report results. If it still fails, please provide logs.
firelight-tests.zip

@Gulaschcowboy Same thing here. It works fine on my setup. See the attached logs, this time generated using your firmware blob. When you're back home, please try again (same as above). Have a safe trip!
homie-temp-tests.zip

Hi @mrpace2

check
check
check
via homie-ota

Result:
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 32)...
✴ OTA available (checksum 745b6573bfdaafbee0805739b779f63d)
��� OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (985/350880)...
Receiving OTA firmware (2005/350880)...

receiving OTA firmware (350880/350880)...

Question: What should be the setting for homie-ota.ini:
OTA_FIRMWARE_BASE64     =
true or false?
I tried both (of course including restart of homie-ota and re-upload of firmware.bin)
Same result.
What else should I provide to help?

Hi @mrpace2,
tried both ways that you described in your attached zip file. Both ways (homie-ota and your script) end in:
Receiving OTA firmware (350845/350880)...
Receiving OTA firmware (350880/350880)...
Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x4024a39b depc=0x00000000
ctx: sys
sp: 3ffffad0 end: 3fffffb0 offset: 01a0

stack>>>
3ffffc70:  3ffef460 3fff05d4 3ffffcac 40219ad6
3ffffc80:  4024a39b 3ffef460 3ffffcb0 4010068c
3ffffc90:  4024a3ea 000002c1 00000007 4020c179
3ffffca0:  00072380 3ffef460 0000019d 3fff34c4
3ffffcb0:  0000000f 00000000 00000000 402199a8
3ffffcc0:  00072380 3fff05d4 3ffef460 4020c6d4
3ffffcd0:  00000000 00000000 00000000 00000000
3ffffce0:  3fff36c4 0000000f 0000000d 00000000
3ffffcf0:  3ffeddc8 4010322a 4021eec3 00000000
3ffffd00:  ffffffff 00000000 3ffe9d51 00000000
3ffffd10:  4021ef16 3ffed320 3fff1fa4 00000000
3ffffd20:  40000000 00072352 0000002e 3fff2762
3ffffd30:  00000023 00072380 00000020 00000000
3ffffd40:  3ffeab72 3fff31e4 3ffffda0 00000000
3ffffd50:  3ffeab72 3fff31dc 00000000 4021b9d4
3ffffd60:  00072380 4010322a 00000000 4020babf
3ffffd70:  40000000 3fff31cc 00000000 4021bcd2
3ffffd80:  3fffc278 40103080 3fffc200 40210014
3ffffd90:  00072380 40103080 3fffc200 00000002
3ffffda0:  3fff330c 00000030 4020ba9c 4021b9ac
3ffffdb0:  40000000 3ffeadb6 0000030a 003c3dda
3ffffdc0:  3fff31dc 3fff275c 00000000 00000001
3ffffdd0:  ffffffff 3fffc6fc 000000d6 00000159
3ffffde0:  3ffffe50 0000002e 3fff35dc 4021bb6d
3ffffdf0:  0000002e 00072352 00072380 00000000
3ffffe00:  4022495e 3fff1fa4 3ffeb1e5 40210fd5
3ffffe10:  0000002e 00072352 00072380 00000000
3ffffe20:  3ffeab72 00000550 3fff00fc 4020fe09
3ffffe30:  3fff0570 000002f1 000002f1 00000080
3ffffe40:  00000000 00000006 3fff372c 4023ec98
3ffffe50:  00000000 00000000 00000000 40107378
3ffffe60:  3fff0255 00000006 3fff372c 4023ecd8
3ffffe70:  3fff2364 3fff1cc4 3fff53c4 3fff0150
3ffffe80:  00000001 3fff3644 3fff00fc 4020fe3c
3ffffe90:  00000001 3fff3644 3fff00fc 402115f0
3ffffea0:  3fff3234 3fff3238 3fff3234 4023c806
3ffffeb0:  3fff1ea4 3fff1eb0 00000006 3fff1cb8
3ffffec0:  3fff3234 3fff1cc0 3fff1cbc 40211643
3ffffed0:  3fff3234 3fff1cc0 3fff1cbc 402404ec
3ffffee0:  00000550 3e2ba8c0 00000010 00000550
3ffffef0:  00000010 00000000 3ffeb194 40107368
3fffff00:  00000000 00000000 3ffeb1ad 3fff1ea8
3fffff10:  3ffeab4a 3fff1eb0 3fff3644 4023e9f1
3fffff20:  002c4288 3fff2364 3fff2364 3ffee7b0
3fffff30:  00000000 3fff3644 0000001c 3fff2364
3fffff40:  3ffeab3c 00000000 3fff3644 4023dded
3fffff50:  2a358f3e 00000001 4010522e 3ffeddc8
3fffff60:  3ffed320 40101bd5 00000000 00000005
3fffff70:  00000002 40101967 00000002 00000000
3fffff80:  401019a2 00000002 00000000 3fff23ec
3fffff90:  3fffdc80 00000000 3fff3644 40232863
3fffffa0:  40000f49 3fffdab0 3fffdab0 40000f49
<<<stack<<<

ets Jan  8 2013,rst cause:2, boot mode:(3,6)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3de0c112
~ld
After that ESP restarts normal (at least this is better than before, as it doesn't produce a csum error anmore)
Attached you find a log taken with:
mosquitto_sub -h localhost  -t homie/test1/# -u  -P  -v > test.log
test.zip

base64 (=default, if not set):
OTA_FIRMWARE_BASE64 = True

binary:
OTA_FIRMWARE_BASE64 = False

I'm afraid I can't help much with your exception. You could try locating the exact code position where it crashes by looking up addresses contained in the exception dump from a map file or a listing generated using xtensa-objdump. I have never done that myself for ESP, but I know that other people have, so it is possible.
Or add extra debug logs around BootNormal.cpp:L413-L424 to see how far it gets before it dies.
Just guessing... Are you sure your power supply is clean? Maybe there's another board you could try?
Or, start over from a simple Homie sketch. Something like:
#include <Homie.h>

void setup() {
    Serial.begin(115200);
    Homie.setup();
}

void loop() {
    Homie.loop();
}

Hi @mrpace2
I guess you misinterpreted the logs. After your last commits it doesn't crash anymore after firmware is copied to "hot" flash area, instead it crashes before. So at least the ESP is not rendered unusable anymore. It crashes, resets and starts old firmware. That was different before todays rev.
So my non programmer-impression is: you/ @marvinroger   changed something today, that causes a slightly different behaviour/issue than before.
About OTA in general: OTA worked for me before the base64-related changes in homie-esp8266 about ~3 weeks ago.
About power supply/ESP type: I have tested with 2 node MCUs and a ESP07 that I have with me right now. (yes, I always travel with ESPs :-) )
Node MCUs are powered by built-in USB, 07 is powered by an external power supply and a linear voltage regulator with a beefy capacitor built in. So it is very unlikely to be a power supply or device issue.
Also 2 Sonoff relays at home have same problem since above mentioned b64 changes. (Couldn't test them with today's rev). Sketch is almost bare minimal, actually it is the example provided by Marvin. (sonoff with button)
All ESPs run stable  in normal operation.
About xtensa-objdump: Don't know if I'm capable of doing it or being able to interpret results...
About adding code in BootNormal.cpp:  That's also beyond my skills I guess
:-(
Thanks a lot
Alex

@Gulaschcowboy I did notice that the behavior changed for you and I did notice in your log that your app rebooted with the old md5. The location where I suggested to add debug output is before the reboot, for a reason ;-)
How can I duplicate your problem here? As I said, it worked fine when I tried your previous firmware a few hours ago. Do you have a repo I could fork? Would you be open to sharing your latest binary again so I can try that one out?
What toolchain are you using?
Thanks for your help.

I think I got over the problem now:
a) updated Arduino to 1.6.12
b) switched to OTA_FIRMWARE_BASE64 False (binary)
Loaded ESP via Arduino and USB once
Can now repeatedly upgrade or downgrade versions
c) switched back to OTA_FIRMWARE_BASE64 True (BASE64)
✴ OTA available (checksum d5f35e99e9df82360a91879400820e98)
↕ OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (1056/350736)...
<snip>
Receiving OTA firmware (350736/350736)...

Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x4024a353 depc=0x00000000

ctx: sys
sp: 3ffffad0 end: 3fffffb0 offset: 01a0

>>>stack>>>
3ffffc70:  3ffef5e0 3fff070c 3ffffcac 40219aee
3ffffc80:  4024a353 3ffef5e0 3ffffcb0 4010068c

see jpmens/homie-ota#44

@ckrey Thanks for your help. So, still something wrong with base64... ☹ Maybe the issue only happens with Arduino-built firmware? (I'm using PlatformIo). @Gulaschcowboy what is your build environment?
I'm just reading up on ESP8266 exceptions. Exception 3 is a LoadStoreErrorCause which seems to be what data aborts are with ARM or page faults with x86 (roughly). epc1 and excvaddr both contain addresses. Presumably, one is the location of the opcode where it happened and the other one is the address that was accessed. With all the exceptions reported here, epc1 seems to be 0x4000bf64 and excvaddr varies around 0x4024a3XX. excvaddr could be the opcode location.
Can you please generate a map file matching your firmware and look up addresses 0x4000bf64 and 0x4024a353 in the map file? In order to generate a map file, please add link option
 -Wl,-Map,firmware.map

to your LDFLAGS. I don't know how this is done in Arduino. In PlatformIo, you would add
build_flags = -Wl,-Map,firmware.map

to your platformio.ini file. The next linker run will then generate a file firmware.map which contains mappings of symbolic names to addresses. You may also post your map file here.

@mrpace2 Yes, Arduino IDE 1.6.12, The sketch doesn't seem to matter. As said, I even have the problem with the Marvins Sonoff button sketch.
Build flags: I will have to try tonight.

To decode the stack trace easily:

Install Arduino IDE 1.6.11 (there's an issue with the EspException plugin wih the 1.6.12)
Install https://github.com/me-no-dev/EspExceptionDecoder

Please paste the decoded stack trace, not the one with the addresses as we cannot do anything without the binaries. 😉

To give it a try, I installed Arduino 1.6.12, arduino/esp8266 and Homie/develop + dependencies on my machine. Same thing. It just doesn't want to crash. It works fine with base64 or binary and with homie-ota or manual publishing (my little ota-update.sh script).
I attached annotated serial and mqtt logs showing 4 updates:

binary update using ota-update script (version 1.0.2 to 1.0.3)
base64 update using ota-update script (version 1.0.3 to 1.0.4)
binary update using latest homie-ota (version 1.0.4 to 1.0.3)
base64 update using latest homie-ota (version 1.0.3 to 1.0.4)

All updates worked as expected. None of them raised an exception.
Thanks to @ckrey for fixing the read-boolean bug in homie-ota. I am using the latest version that includes your fix.
I also attached the bare-minimum sketch that I was testing with (I changed the Homie_setFirmware version number in between tests), as well as my Arduino build log, in case somebody wants to compare library version numbers. For the libraries that did not come with Arduino, I downloaded the latest ZIP files from github.
arduino-tests.zip

Following my latest post about EspException: you can use Arduino 1.6.12 with the following EspException zip : https://github.com/me-no-dev/EspExceptionDecoder/files/594809/EspExceptionDecoder-1.0.4.zip

Hi @marvinroger @mrpace2 ,
attached my decoded stacktrace. Hope it helps.
Thanks a lot,
Alex
stacktrace.txt

@Gulaschcowboy thanks, did you paste the stack trace into the decoder from Exception (3): to <<<stack<<<?

no, just from >>>>stack to <<<<stack. attached full paste:
stacktrace.txt

@Gulaschcowboy can you try, just like @ckrey, to set OTA_FIRMWARE_BASE64 to False in homie-ota? Just to confirm your issue also happens with base64 only?

@marvinroger Tried that before, tried again now. result:
✴ OTA available (checksum 471f6229335cbb136aed58187dfc5c89)
Receiving OTA firmware but not requested, skipping...
Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x40249e7f depc=0x00000000
BTW, the stack traces before were "created" using @mrpace2 's script

@Gulaschcowboy thanks for providing the stack trace.
@marvinroger I am looking into it. It crashes while exiting _publishOtaStatus(400, PSTR("NOT_REQUESTED")); called from BootNormal.cpp:299

@marvinroger @mrpace2 I don't know, if it help or confuses more:

I compiled now with platformio, uploaded via serial to NodeMCU board -> connects, so far so good
I uploaded same firmware.bin (with base64 set to false) to homie-OTA, did OTA -> success
pressed OTA upload again with same firmware -> same crash as with arduino created binary:

Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 25)...
✴ OTA available (checksum 5e619432e444f54f608378e074298435)
Receiving OTA firmware but not requested, skipping...
Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x4024b247 depc=0x00000000
So my non-developer guess/impression: We have 2 issues:

OTA which should be executed, because firmware is different: crash in arduino, but not in pio
OTA which is refused due to same binary: crash in arduino AND in pio

@Gulaschcowboy When you generated the stack trace, $implementation/ota/firmware was rejected because the new firmware's checksum matched the running firmware. You probably received a 304 for the $implementation/ota/checksum message before. Homie was about to send you a 400 NOT_REQUESTED when it crashed inside String:concat(char) which was called from BootNormal::_publishOtaStatus.
Sorry, I'm afraid I don't fully understand what you were trying to say in your last message. But, anyway, one way or another, the function that crashes your system will end up being called, either to reject OTA firmware or to report success or failure at the end of an OTA update. And it doesn't make a difference here if the firmware blob is binary or base64-encoded or whether you used homie-ota or something like my script.
I committed an attempt to workaround the issue to my own homie-esp8266 fork on Github. Would you mind trying out this workaround on your system? For this test, please replace your current homie-esp8266 library with https://github.com/mrpace2/homie-esp8266/archive/patch-trying-to-fix-issue-204.zip in your Arduino, or use https://github.com/mrpace2/homie-esp8266.git@fcfdc66 with your PlatformIo library configuration.
Please report the result. If it still fails, please send another stack trace. A matching log (serial or mqtt) would be greatly appreciated.
Thanks again for your help.

@mrpace2 What I tried to say is: I see different behaviors, when I use different steps. Somehow we have two different issues:
Tests:
ENV is always like this:

NodeMCU clone 4m/1m
Homie-OTA set to base64 = False
Homie and Homie-OTA always latest REV

Test1 with binary compiled with arduino:

OTA when it should do OTA because of different hash -> crash
OTA when it shouldn't do OTA -> geting a 304 -> crash

Test 2 with binary compiled with PIO:

OTA when it should do OTA because of different hash -> it works
OTA when it shouldn't do OTA -> getting a 304 -> crash

Did you get my point now?
Will do your suggested tests now.
Thank you very much!

@mrpace2 I tried with your library now, results:
Compiled with Arduino, b64 disabled, your patched homie lib:

OTA with version1.0.0 over serially flashed 1.0.0 -> crash
OTA with version 1.0.1 over serially flashed 1.0.0 -> it works (why, don't ask me, see my post before)
OTA with version 1.0.1 over OTA-flashed 1.0.1 -> crash

See logs, traces an decoded traces below.
stack-arduino-v100tov100.txt
stack-arduino-v100tov101.txt
stack-arduino-v101tov101.txt

Thanks.
Regarding (1) and (3), is not the 304 response to $implementation/ota/checksum that crashes. It's the 400 response to the next message ($implementation/ota/firmware) that crashes. FYI, homie-ota "blindly" sends firmware no matter what homie-esp8266 responds to the checksum.
What I am trying to say is that all these crashes are most likely caused by the same bug that triggers somewhere inside String::concat called from _publishOtaStatus because _publishOtaStatus is called no matter what the result of the OTA update is. I find it strange that _publishOtaStatus sometimes works and sometimes doesn't. My first attempt to fix the issue obviously didn't help. Will have to come up with another solution.

I am not able to reproduce this behavior, so it will be hard to debug.
@Gulaschcowboy can you please comment these lines and try again with a scenario that normally fails?

@marvinroger Do you have experience reading those stack traces? The stack traces seems to interleave two execution contexts? For example, in stack-arduino-v101tov101.txt roughly every other stack frame seems to come from some unrelated activity:
[?]     0x40249e87: sleep_reset_analog_rtcreg_8266 at ?? line ?
[Homie] 0x40219cef: String::concat(__FlashStringHelper const*) at /home/atze/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/WString.cpp line 519
[?]     0x40249e87: sleep_reset_analog_rtcreg_8266 at ?? line ?
[?]     0x40249e87: sleep_reset_analog_rtcreg_8266 at ?? line ?
[Homie] 0x4020bbb8: HomieInternals::BootNormal::_publishOtaStatus(int, char const*) at /home/atze/Dokumente/arduino/libraries/homie-esp8266/src/Homie/Boot/BootNormal.cpp line 52
[?]     0x40218f3c: Print::println() at /home/atze/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/Print.cpp line 64
[Homie] 0x4020c3a0: HomieInternals::BootNormal::_onMqttMessage(char*, char*, AsyncMqttClientMessageProperties, unsigned int, unsigned int, unsigned int) at /home/atze/Dokumente/arduino/libraries/homie-esp8266/src/Homie/Boot/BootNormal.cpp line 299
[?]     0x4021c21c: Print::write(unsigned char const*, unsigned int) at /home/atze/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266/Print.cpp line 64
[?]     0x40225f48: ieee80211_output_pbuf at ?? line ?

Because the other context is also calling String, I am thinking that maybe something in String in non-reentrant? I'm thinking about replacing String inside _publishOtaStatus with a pre-allocated character buffer, something like
_otaStatusMessage = std::unique_ptr<char[]>(new char[longestOtaStatusMessage]);

What do you think?

(1)(3) crashes directly
(2) works when b64 is set to false
(4) same as (2) but b64 set to true and it crashes
So it may be the same function as you say, but it behaves differently in all mentioned steps.
At least it is consistently bad with base64 :-(

@marvinroger you mean doing
/*
if (info) {
payload += ' ';
payload += info;
}
*/
?

@mrpace2 not really, I was not able to find some viable docs about the stack traces. I was also thinking about replacing the String with a char array. You don't even need a unique_ptr, just instantiate a char array in the stack in _publishOtaStatus

@Gulaschcowboy yes

Or, even more radicale:
uint16_t BootNormal::_publishOtaStatus(int status, const char* info) {
  /*
  String payload(status);
  if (info) {
    payload += F(" ");
    payload += info;
  }
  return Interface::get().getMqttClient().publish(_prefixMqttTopic(PSTR("/$implementation/ota/status")), 1, true, payload.c_str());
  */
  return 0;
}

uint16_t BootNormal::_publishOtaStatus_P(int status, PGM_P info) {
  // return _publishOtaStatus(status, String(info).c_str());
  return 0;
}

@marvinroger I'll work on that.

@marvinroger @mrpace2
Commenting out these lines works!
base64 set to false:

hash to same hash -> just gets ignored
hash to different hash -> updates, reboots

base64 set to true:

hash to same hash -> just gets ignored
hash to different hash -> updates - crash

@Gulaschcowboy Thx

@Gulaschcowboy we're not done, when b64 is set to true, you say it still crashes? Can you give us a stack trace again?
And no one's supporting the other one, it's just teamwork. Thanks a lot @mrpace2 for your great efforts, once again!

@marvinroger @mrpace2
Let's crash it baby...
Arduino, base64 with hash to different hash, crash after OTA:
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 32)...
✴ OTA available (checksum afba76c6a59fe5b57ae32c7808e06851)
��� OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (985/350418)...
...
Receiving OTA firmware (350416/350416)...
Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x4024a209 depc=0x00000000
ctx: sys
sp: 3ffffad0 end: 3fffffb0 offset: 01a0
...
base64-arduino-v100-v101.txt

@Gulaschcowboy Try again after #225 has been merged.

Before merging #225 blindly, without understanding what happens, let's try another thing. I've seen somewhere sleep_reset_analog_rtcreg_8266 are triggered when there's not enough power. And until now, every weird issues like this one was due to power issues.
@Gulaschcowboy can you try to power your ESP8266 from an external power supply (don't even plug it into your computer), and make it crash again? You won't have Serial feedback so try to power on a LED or something to check if OTA was successful.
I understand that it worked before, so it might not look like a power issue, but as more code gets added, the ESP8266 has more to do and there's indeed a moment where power might become an issue. Maybe this problem has come! And maybe the Linux kernel does not handle power on USB like Windows or something...
I would love you to tell me that this was the problem! 😄

@marvinroger @mrpace2
About probable power issue: I could reliably reproduce the crashes independent from the way of powering the ESP.
Tested:

USB via onboard voltage regulator/serial connected to laptop
USB via onboard voltage regulator/serial connected to external power supply (USB 5V / 2A)
Circumventing the onboard linear voltage regulator by powering directly with 3,3V (plus a beefy stabilizing capacitor across Vcc/GND)

As all powering setups crashes with the old code I don't expect the power to be responsible.

Chiming in, got here from #224, so getting the exception 3 after:
Receiving OTA firmware but not requested, skipping...

Everything on breadboard here, powered by an MB-102, being itself powered by a pretty huge 12V power supply (able to withstand 5A @ 12V). Got a 10µF between across Vcc/GND too.
Last change I made was moving from 1.5.0 to head of develop branch. Tried again with 1.5.0 it works fine, so I would agree wint @Gulaschcowboy that is probably isn't related to power supply.
Not right now, but I could give a shot at some kind of git bisect on Homie see if it can give us any pointers as when it appeared.

Sorry I didn't read everything, so my suggestion deesn't make much sense. Still I'm pretty sure it doesn't come from the power supply :)

I just tested with the latest homie-ota, and it worked!
Note, I got the same error as @bleader when running on WIndows 10 with Python 2.7.12:
Receiving OTA firmware but not requested, skipping...

I tried it again on Linux (Ubuntu 14.04.4 LTS with Python 2.7.6) and it worked on the first try.

I just discovered that my Windows git repo was a few days out of date.  After updating, It's still not working for me.  Now it's starting the update, but only sending about 7K instead of the 300K+ of the image.  Not sure what's happening, but it is working on Ubuntu, so I'm happy.

You made me realize my repo wasn't up to date, the last update being to actually really read the OTA_FIRMWARE_BASE64... which may explains why it didn't change a thing when I tried to disable it.
So, now that I did update, OTA works with OTA_FIRMWARE_BASE64 disabled, but not if I enable it. Which makes it less likely to be due to homie-ota only.
@clough42 Are you sure both your configurations have it disabled?

@bleader Ahh...good catch.  It's explicitly disabled on the Ubuntu machine (which works) and the line in the ini file is commented out on the WIndows machine, which doesn't.  If it defaults to enabled, then that's the difference.  I'll toggle that and test when I get a chance.

Finally getting around to testing this. Environment: platformio on macOS 10.12.1, Homie 2.0, latest commit 49eca384a4bc13fee34728ac40f5702701e384b4 and homie-ota (also latest :-)
$ md5 firmwares/H2MuM-2.0.4.bin
MD5 (firmwares/H2MuM-2.0.4.bin) = 6f90474f8ecd7abe87564dcddec9e01d

(Base64-encoded firmware here)
When OTA is trigerred (B64-enabled), the result looks promising until the stack trace:
✴ OTA available (checksum 6f90474f8ecd7abe87564dcddec9e01d)
↕ OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (1053/350529)...
Receiving OTA firmware (2148/350529)...
[...]
Receiving OTA firmware (350200/350529)...
Receiving OTA firmware (350528/350528)...

Exception (3):
epc1=0x4000bf64 epc2=0x00000000 epc3=0x00000000 excvaddr=0x4024b633 depc=0x00000000

ctx: sys
sp: 3ffffad0 end: 3fffffb0 offset: 01a0

>>>stack>>>
3ffffc70:  3fff0df8 3ffef5f4 3ffffcac 40202102
3ffffc80:  4024b633 3fff0df8 3ffffcb0 40100688

If I configure home-ota to not use base64, I see the following expected messages, and the update (sometimes) works:
devices/18fe34d40655/$implementation/ota/status 206 348426/350528
devices/18fe34d40655/$implementation/ota/status 206 349886/350528
devices/18fe34d40655/$implementation/ota/status 206 350528/350528
devices/18fe34d40655/$implementation/ota/status 200

What happens when, when not using base64, it (often) does not work? Does it crash? Does it report something?

By the way, is anybody using NodeMCU v2s experiencing these issues? I only have these, and I don't have any problems with the latest Git, b64 or not.

Hi, I have that issue with NodeMCU boards (clones) as well with esp-07s

These ones?

yes, i guess so. Clones from LoLin.

Well, I don't experience a single issue on official NodeMCU boards, so I guess the problem is you are using clones. I am not sure if the issue actually comes from the hardware or from esp8266/Arduino, but I am pretty sure it does not come from homie-esp8266. Everything being rocking stable on official NodeMCUs...

I understand your point, but my point is: Same issue on NodeMCU clone as well as on ESP07 modules, always with base64. Would it be helpful to give you access to my environment/modules?

I understand your point too, but I would need to test on real hardware. I'll try to buy some.

I'm in the process of trying OTA on  D1 Mini clones of this kind https://www.aliexpress.com/item/D1-mini-Mini-NodeMcu-4M-bytes-Lua-WIFI-Internet-of-Things-development-board-based-ESP8266-by/32635160765.html
I'm using this Python "server" https://github.com/jpmens/homie-ota
My code can be accessed from https://github.com/amayii0/Homie-DS18B20
As this is the first time ever i'm doing such thing, don't expect a quick reply... :)
(spoiler : it currently just crashes after OTA / reboot)

Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 27)...
Temperature: 19.00 °C
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 28)...
✴ OTA available (checksum 1c47b3d8886fb2730d7e53b24acdd43c)
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 29)...
↕ OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (1056/4167)...
Receiving OTA firmware (2151/4167)...
Receiving OTA firmware (3246/4167)...
Receiving OTA firmware (4166/4166)...
✔ OTA succeeded
Triggering OTA_SUCCESSFUL event...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 30)...
Device is idle
↻ Rebooting...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 31)...
✖ MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...
ets Jan  8 2013,rst cause:2, boot mode:(3,6)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3ffe84d0
@cp:0
ld
s$␀lܟ<␀�l�<␃␌␄␌�␄l�␄c|��␃�␓�;�c␌�c��g'�lng���␌b␜x��l;l{l���␃��r�Ó␃x�o�␘␃␌␄�␌d␌��␌␄␌#␄g�|␃l�␌␄�c��'o�␀d��$␃�␛␒no␄$␃␏␃g{���o␌␄b␄�␏$␏r��g␄␌c␄�␇l�sĜ���␌��d␂��'�␃{$␀l��<␀�l�|␃␌␄␌�␄l�␌c|��␃�␓�r�c�␄c��gn�dog���␌c␜x
�l;l{lx�o�␘␃␌␄�␌d␌��␌␄␌c␄g�|␃l�␌␄�c��'o�␀d��$␃�␛␒'o␌$␃␎␃gs���o␌␄b␄�␎$␏;��g␄␌c␄�␏l�sĜ���␌��d␂��g�␃sd␀$��|␀�l�|␃␌␌␄�␄$�␄c|��␃�␛�{�c�␌c��og�l'o���␌#␜p�${ls$p�g�␐␃␄␄�␄l␄Ĝ␄␌␌c␌'�|␃$�␄␄�b��og�␀l��l␃�␒␛og␄l ␃␇␃n;��
�g␄␌c␄�␇l␇s��n␌␄c␌�␇d�{�����␄��$`␃��o�␃

Your firmware file is bad, it just cannot be 4k in size, it is usually
around 300k.
Le 18 févr. 2017 1:01 AM, "Amayii"<EMAIL_ADDRESS>a écrit :
I'm in the process of trying OTA on D1 Mini clones of this kind
https://www.aliexpress.com/item/D1-mini-Mini-NodeMcu-4M-
bytes-Lua-WIFI-Internet-of-Things-development-board-
based-ESP8266-by/32635160765.html
I'm using this Python "server" https://github.com/jpmens/homie-ota
My code can be accessed from https://github.com/amayii0/Homie-DS18B20
As this is the first time ever i'm doing such thing, don't expect a quick
reply... :)
(spoiler : it currently just crashes after OTA / reboot)
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 27)...
Temperature: 19.00 °C
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 28)...
✴ OTA available (checksum 1c47b3d8886fb2730d7e53b24acdd43c)
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 29)...
↕ OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (1056/4167)...
Receiving OTA firmware (2151/4167)...
Receiving OTA firmware (3246/4167)...
Receiving OTA firmware (4166/4166)...
✔ OTA succeeded
Triggering OTA_SUCCESSFUL event...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 30)...
Device is idle
↻ Rebooting...
Triggering MQTT_PACKET_ACKNOWLEDGED event (packetId 31)...
✖ MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...
ets Jan 8 2013,rst cause:2, boot mode:(3,6)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3ffe84d0
@cp https://github.com/cp:0
ld
s$␀lܟ<␀�l�<␃␌␄␌�␄l�␄c|��␃�␓�;�c␌�c��g'�lng���␌b␜x��l;l{l���␃��r�Ó
␃x�o�␘␃␌␄�␌d␌��␌␄␌#␄g�|␃l�␌␄�c��'o�␀d��$␃�␛␒no␄$␃␏␃g{�
��o␌␄b␄�␏$␏r��g␄␌c␄�␇l�sĜ���␌��d␂��'�␃{$␀l��<␀�l�|␃␌␄␌�␄l�␌
c|��␃�␓�r�c�␄c��gn�dog���␌c␜x
�l;l{lx�o�␘␃␌␄�␌d␌��␌␄␌c␄g�|␃l�␌␄�c��'o�␀d��$␃�␛␒'o␌$␃␎␃gs�
��o␌␄b␄�␎$␏;��g␄␌c␄�␏l�sĜ���␌��d␂��g�␃sd␀$��|␀�l�|␃␌␌␄�␄$�␄
c|��␃�␛�{�c�␌c��og�l'o���␌#␜p�${ls$p�g�␐␃␄␄�␄l␄Ĝ␄␌␌c␌'�|␃$�␄␄�b��og�␀l��l␃�␒␛og␄l
␃␇␃n;��
�g␄␌c␄�␇l␇s��n␌␄c␌�␇d�{�����␄��$`␃��o�␃
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/marvinroger/homie-esp8266/issues/204#issuecomment-280799770,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA8eNalZQlrYX4uVgoKgPJA1KPmCJK6Mks5rdjTVgaJpZM4KkyL5
.

This is where it gets interesting. My firmwares are not 4k:
352.592 for D1Mini-DS18B20-<IP_ADDRESS>.bin
352.592 for D1Mini-DS18B20-<IP_ADDRESS>.bin

Does it happen all the time? Can you post a mosquitto_sub log of all the messages sent?

@amayii0 what mqtt server are you using? I believe I ran into similar issue when max payload sized on my mqtt was limited. I had more success with iot.eclipse.org

MQTT broker IIRC is moquette as built in PiDome.
https://bitbucket.org/pidome/pidome-mqttbroker/src/b14dd502b16e178c866288512ca10deab6c1d825/src/main/java/org/pidome/plugins/mqtt/pidomeMQTTBroker/PluginBrokerRunner.java?at=default&fileviewer=file-view-default

@marvinroger ill check that later. Maybe also switch to mousquitto for debug

@marvinroger it seems payload gets limited to 5.556 bytes. This would rather be related to the broker I'm afraid.
pi@garagepi ~ $ mosquitto_sub -h <IP_ADDRESS> -p 1883 -t homie/5ccf7fd3b7d5/#
D1Mini-DS18B20
true
60
17.37
2452e21eb82983a708b60968b10d4daa
temperature
3
100
true
esp8266
2.0.0
unit,degrees
{"wifi":{"ssid":"dd-iot"},"mqtt":{"host":"<IP_ADDRESS>","port":1883,"auth":false},"name":"Ds18b20","ota":{"enabled":true}}
5C:CF:7F:D3:B7:D5
Ds18b20
2.0.0
<IP_ADDRESS>
<IP_ADDRESS>
200
c
1c47b3d8886fb2730d7e53b24acdd43c
[encoded data, 5556 characters]
I've dumped that to a file using MQTTFX and same story. The homie-OTA is running from W7 host, looks like I can run that from the rpi instead.

@amayii0 that's indeed related to the broker... We cannot do anything on our side, unfortunately.

@timpur After cleaning the libraries folder as requested in #477 OTA works for me.

Nice, recommend platform IO using vscode this will fix your lib dependency issues in the future. But does have its anoyences ....


Texture

Hey, do you know why my textures don't want to be put on? It does it to me and to several of my friends:
$this->initComponent("diamond_hoe", $creative);

@JustTimmm have you found any solution? Textures are also not displayed in my case.

So have you all solved it? I also have this problem

So have you all solved it? I also have this problem

@Atara6431 I've downloaded the latest version of the plugin from github, looks like the poggit contains the old version

@and7ey which one?last updated five months ago on github?

@Atara6431 yes

My custom item hotbar textures aren't working either!

My custom item textures aren't working either, the icon is always empty!

same bro

My custom item textures aren't working either, the icon is always empty!

same bro

Check out my pull request, you'll have to edit Customies tho.

This is fixed


Trooper suggestion: improve entering wallet button

Add two button press or combo of buttons to enter wallet from Tetris

Reduced time to press the button to 1/2 sec
https://github.com/arcbtc/bowser-bitcoin-hardware-wallet/commit/d509ed882925541c0ffb85f6271fb0087ac2a004#diff-2ebfbbc910d4be8839608fce4dddfe8b


Cannot reset password for an EPerson created in Admin UI

Describe the bug
If you create a new EPerson via the Admin UI (http://localhost:4000/admin/access-control/epeople), then it's impossible to set or reset a password for that user.
To Reproduce
Steps to reproduce the behavior:

Login as an Admin
Create a new EPerson (Visit "Access Control" -> "People" -> "Add EPerson" -> fill out data -> "Submit")
Notice that the form doesn't let you set an initial password
Now, logout
Attempt to reset the new User's password by clicking "Have you forgotten your password?" and entering the user email address.  (NOTE: If you are running DSpace locally, you may need to set mail.server.disabled=true so that you can see the text of the generated email in your dspace.log file.)
Click on or visit the generated forgot email page (http://localhost:4000//forgot/[randomuuid])
Attempt to set a password.  You'll receive a 400 error that says: Attempting to replace a non-existent value (netID).

The problem appears to be in this method: https://github.com/DSpace/DSpace/blob/main/dspace-server-webapp/src/main/java/org/dspace/app/rest/repository/patch/operation/EPersonPasswordReplaceOperation.java#L99
That method requires that a password exists before it lets you reset your password.  (It also throws an incorrect exception that mentions "netID" which has nothing to do with the actual error.)
Expected behavior
There must be a way to set a user's password after creating them from the Admin UI.
One of the following MUST be possible:

Either the Admin should be allowed to initialize a password when creating the user account
OR, a random password should be initialized for the user account (and shown to the admin or emailed to the user)
OR, it should be possible to reset a password even if a password was not already set.

My slight preference would be for option 3, as it's in line with DSpace 6 (and it should require no changes to the Angular UI).  I'm not against the other options if there's a clear reason why we do not want a password to be reset when it's empty.

The problem is that the request is a replace, e.g. [{"op":"replace","path":"/password","value":"******"}], but you can't replace what doesn't exist
The potential solutions to approach this:

Send an add PATCH: the problem is that Angular can't know whether the value exists (whether it should be add or replace), and add is not implemented
Accept a replace even if no value exists. This doesn't impose any technical issues, but contradicts the concept of a replace. I'm assuming this will be used since it's the easiest solution.

Estimate includes tests

@benbosman : Sounds good overall.  After looking at your options & rereading our own contract docs on PATCH, it sounds like the correct behavior here is to always do an "add".  According to our docs for "add"

Please note that according to the JSON Patch specification RFC6902 a subsequent add operation on an already set attribute have the effect to replace the previous value.

So, it sounds like we might want to go with your first solution & send an "add" no matter what.  If the value exists, then "add" should replace it.  Does that change the estimate here?  I'm assuming it also would require minor changes to the Angular UI to ensure an "add" is sent instead of a "replace".

That's indeed a better solution. This would add the changes:

Angular should send an add instead of replace: small change
REST should support an add of the password (and possibly also still support replace, or only add): larger change
When add is used, no value has to be present in REST
All ITs should be adjusted to use add


Show par contract

Will require wrapping one of dds's many par calculation functions.

Probably DealerPar and ConvertToDealerTextFormat.
This will require a way to input the dealer and vulnerability.


Why does implicit wait not take effect?

I used command to change UI in my project. For example:
<ListView
 ...
 ItemsSource="{x:Bind ViewModel.ExpandedLevels}">
 <ListView.ItemContainerTransitions>
  <TransitionCollection />
 </ListView.ItemContainerTransitions>

 <ItemsControl.ItemsPanel>
  <ItemsPanelTemplate>
   <VirtualizingStackPanel>
   ...
   </VirtualizingStackPanel>
  </ItemsPanelTemplate>
 </ItemsControl.ItemsPanel>
 <ItemsControl.ItemTemplate>
 ...
 </ItemsControl.ItemTemplate>
</ListView>

public ICommand SelectItemCommand { get; }

I wanted to use implicit wait to solve the display delay problem. But did not take effect.
I think it may be because the implicit wait may only take effect when the Page control switches.
Or does it take effect every time when finding elements?

it take effect every time when finding element

@kram95731 How are you currently implementing your wait?

@kram95731 this approach worked for me (C#). The key is to include it with the instantiation of your driver session. Best of luck
session = new WindowsDriver<WindowsElement>(new Uri(WindowsApplicationDriverUrl), appCapabilities, TimeSpan.FromMinutes(2));

it take effect every time when finding element

I get it.Thanks a lot.

@kram95731 this approach worked for me (C#). The key is to include it with the instantiation of your driver session. Best of luck
session = new WindowsDriver(new Uri(WindowsApplicationDriverUrl), appCapabilities, TimeSpan.FromMinutes(2));

Thanks for your suggestion. I have found the cause of problem. The element was found but the number is wrong. For example:
HTTP/1.1 200 OK
Content-Length: 102
Content-Type: application/json

{"sessionId":"3B1ABD33-B1F1-4CFE-85C1-7284A8F00A4D","status":0,"value":[{"ELEMENT":"42.462094.4.96"}]}

WinAppDriver returned one element.
Driver.FindElementsByClassName("GridView").Single(list => list.GetAttribute("AutomationId").Equals(...))

But in my code, I try to find more than one element. So the error happened.
WinAppDriver did find the element. So it returned. However at that time, there should be two elements or more.
Thank you all the same.


Migration error for multiple databases: django.db.utils.ProgrammingError: (1146, "Table 'db_2.django_content_type' doesn't exist")

Describe the problem
Hi. We have a django site that uses 4 databases. I'm trying to install Constance on our Django site and am getting an error regarding the django_content_type table in one of the non-default databases.
We've written a DB router to route selected apps to specific databases. Non-specified apps route to default. Because Django, we have a script to run the migrations for all the databases.
python manage.py migrate --noinput
python manage.py migrate --noinput --database=db_2
python manage.py migrate --noinput --database=db_3
python manage.py migrate --noinput --database=db_4

Running the script gives the following error when it runs the db_2 command:
Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying database.0001_initial... OK
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/sholleman/src/.virtualenvs/sma/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line

[...]

Migration error: django.db.utils.ProgrammingError: (1146, "Table 'db_2_staging.django_content_type' doesn't exist")

We considered creating a  django_content_type tables each database, but that feels wrong. Removing the two instances of .using(using) from create_perm() in apps.py stops the error, but breaks one of your tests, test_post_migrate_signal_creates_content_type_and_permission_in_secondary_database. I think the error is due to a disconnect between the migration and the post-migrate signal. The migration looks to our custom db router and doesn't do anything to db_2. But the create_perm() in apps.py is ignoring our db router. Do you have any suggestions for fixing this?
Steps to reproduce

Have a multi-database site with a special DB mapping router to route model calls to the correct database. Only the default database has the table, django_content_type.
Install Constance with database backend.
Run migrations for non-default database. e.g., python manage.py migrate --noinput --database=ib_files
Error

System configuration

Django version: 1.8.15
Python version: 2.7.12

First, I'm sorry you have 4 databases. I hope you understand that a out-of-the-box tool like constance doesn't often target this type of system when it's developed.
I would argue we could fix this by making sure we pick up the correct router in the code that creates the permissions. Since I'm not sure how custom your routers are, I'd suggest to implement a signal handler override that hard codes your router into the call of create_perm and see what the passed using parameter contains. Then decide to either just override the whole constance app config to abide by your router or find a different way. Using the using() queryset method is definitely the official way to go about it, so I'm concerned this showed up for you at all. Either way, please report back.

Fixed in #258

@camilonova Awesome. Thank you!

0
I have to face same issue and there are a couple of approaches, but the one I think is the most probable one.
Maybe you are loading views or queries to database but you haven´t granted enough time for Django to migrate the models to DB. That's why the "table doesn't exist".
Make sure you use this sort of initialization in you view's code:
Class RegisterForm(forms.Form):
def init(self, *args, **kwargs):
super(RegisterForm, self).init(*args, **kwargs)
A second approach is you clean previous migrations, delete the database and start over the migration process.


Dependency on jackson libraries with security vulnerabilities

I have seen that you're depending on jackson libraries
<jackson.version>2.9.8</jackson.version>

with security vulnerabilities:
CVE-2019-12086 More information
moderate severity
Vulnerable versions: >= 2.0.0, < 2.9.9
Patched version: 2.9.9
A Polymorphic Typing issue was discovered in FasterXML jackson-databind 2.x before 2.9.9. When Default Typing is enabled (either globally or for a specific property) for an externally exposed JSON endpoint, the service has the mysql-connector-java jar (8.0.14 or earlier) in the classpath, and an attacker can host a crafted MySQL server reachable by the victim, an attacker can send a crafted JSON message that allows them to read arbitrary local files on the server. This occurs because of missing com.mysql.cj.jdbc.admin.MiniAdmin validation.

Are you planning to update to 2.9.9 or higher version?

this is now updated


Update buffer constructor

Hey!
new Buffer() constructor has been deprecated in Node >10, and we're getting warnings in the console while using node-opcua which is relying on this library.
I've followed https://nodejs.org/fr/docs/guides/buffer-constructor-deprecation/ to update the code.
All test passing with Node 0.8 and Node 12.13, no deprecation warnings present.

I have only just realised that the library is deprecated.. sorry


EKF not fully reliable

Yesterday we have done some hover tests where we wanted to tune mc_pos_control for altitude and position. We flew with master 0fed23a and we chery-picked EKF fix https://github.com/PX4/Firmware/pull/2968.
The fix itself works as before it, our LPOS_Z was totally different then baro signal, and it was causing way too much CPU load. Now the load dropped down to ~25% for us, and EKF performance is much better, but still not as good as before pulling the https://github.com/PX4/Firmware/pull/2815#event-413847903.
Note: We had quite bad GPS signal yesterday, but it was pretty much the same for all the tests.
The main issues are sudden jumps/spikes in the position setpoint, and the fact that position setpoint rises although throttle stick is in or below the deadzone - basically we had a situation that aircraft was rising a couple of meters up while throttle stick was at zero (http://logs.uaventure.com/view/ekETmDV6hEsFNhENV7CAkA! - unfortunately logmuncher does not display rc inputs - I think we have some naming mismatch), ergo estimator does not work properly.
Below are two snaps showing the spikes and lpos not fully following baro:

(http://logs.uaventure.com/view/WPiHd9TEXGvyGoSjZgidkh)
Then we switched to the branch with old version of EKF and its performance was far more reliable (pilot finally had control in altitude control), although not fully perfect due to bad gps signal. But the difference has been significant.
(http://logs.uaventure.com/view/gNoRJNsCHftMQ6pAJcFgn7)

I'm not too confident about a cherry-picked state. The changes on the system level were profound and you might have ended up with a bad merge. Could you do a bench test with current master vs. your stable state and let me know what you find?

Hi @LorenzMeier, sorry for a bit late response, frankly we had quite a lot of troubles dealing with the new compilation method and including our own modules. Anyway, we've just done indoor string test with master bb2e5b7, and we did not observe improvement as there are still weird spikes in altitude estimation. We'll probably do outdoor test with gps signal in next hour or so, but I do not think the behaviour will change for better.
With all the changes in estimators you've done, are there actually maybe any params that we should look into which change could improve performance of ekf?
(http://logs.uaventure.com/view/KH4HkCrG3Rem7qJDv4ryT3)

It looks like the filter is resetting quite often. Could you attach the MAVLink message log file from the SD card as well?
What is immediately visible is that the Z variation spikes correlate to changes in airspeed.

Message log:
DISARMED by RC
[cmd] arming state: STANDBY
Flight mode: ACRO
baro healthy
No geofence set
Flight mode: MANUAL
settings autosaved
ARMED by RC
[cmd] arming state: ARMED
[sdlog2] log dir: /fs/microsd/log/sess001
[sdlog2] starting: log001.px4log
Flight mode: ACRO
TAKEOFF DETECTED
DISARMED by RC
[cmd] arming state: STANDBY
LANDING DETECTED
SENSOR FAILSAFE! RETURN TO LAND IMMEDIATELY
[sdlog2] logging stopped
settings autosaved
Flight mode: MANUAL
ARMED by RC
[cmd] arming state: ARMED
[sdlog2] log dir: /fs/microsd/log/sess002
[sdlog2] starting: log001.px4log
Flight mode: ACRO
TAKEOFF DETECTED
DISARMED by RC
[cmd] arming state: STANDBY
LANDING DETECTED
[sdlog2] logging stopped
settings autosaved
About the airspeed - we do not fuse it in hover. We always have observed weird spikes when in hover (even before estimator changed), but we do not use it.

@SimonWilks @AndreasAntener A wing-wing flight with master would be much appreciated. I'm not sure if the EKF has an issue per se or its a VTOL thing.
@mswingtra You are very clearly "using" the airspeed, since it correlates with your Z spikes. I just need to work out if that's an issue with the filter or the fact that you are using it outside its design space.

In the previous dataset airspeed spikes seem to be correlated, but spikes in LPOS_Z occur more often.
Bench-test with creating airspeed spikes by tapping and blowing to pitot. Airspeed spikes do not correlate to spikes in LPOS_Z in this dataset. Note: data is re-scaled (aircraft was all the time on ground, max. airspeed reached almost 45 m/s):

And here is dataset when we flew in hover with old EKF, there are spikes in airspeed but they're again not correlated.

The estimator diverges only in states 6 and 9 (the local position and velocity in world-z direction).
I just did a diff between our working estimator before the update and the current one after update and want to ask if the changed way of including the accel measurements could cause the problem? Newly we use _sensor_combined.accelerometer_integral_m_s signals to propagate the states, see
ekf_att_pos_estimator_main.cpp:line1262

@LorenzMeier sorry, the wingwing has been stripped down and retired for now.  Don't have any other wing at the moment that can handle ground bounce ;)

@LorenzMeier @mswingtra @SimonWilks Scheduling a test flight for this tomorrow morning.

We have not been able to reproduce this after our fixes. Please re-open with new logs if it persists on master (and please merge / rebase, cherry-picking is hopeless).


Strapi Studio drop down list doesn't show all options

Hey guys, just testing Strapi and Strapi Studio.
I've created Pokemon and Poketypes so far.
A Pokemon can have many Poketypes and each Poketype can have belong to many Pokemons, so a Many-To-Many relationship.
I created all the Poketypes first such as Grass, Fire, Water, Electric etc.
Then when I went to create the first Pokemon called Bulbasaur (http://www.pokemon.com/us/pokedex/bulbasaur) who is Grass and Poison type, I could not pick Grass and Poison from the relation drop down list:

As you can see from the two screenshots above, Grass and Poison are not in the list.
The work around is for me to choose a few of the ones that are shown in the list until Grass and Poison show up. I choose Grass and Poison, then I delete the ones that I temporarily selected earlier.
Hopefully this hasn't been asked before, if it has, please mark duplicate and delete/close this issue.

Hey @chewedon, sorry for the lack of communication this last few weeks-- but this was for a good reason. I have some interesting news regarding this issue.
We decided to split the Strapi stack in two different projects. Here is the reason:

Strapi is and always will be a Node.js framework-- nothing more.
The built-in features such as the Studio, the admin panel and the built-in APIs are not part of the framework as we call it. They are management features. Those features will be in another project will release soon.

The Strapi framework and the utils are for two different kind of users. This is why we decided to split the current ecosystem in two different projects. Don't worry: the built-in APIs and admin panel will still be open-source, even for the new project.
The Strapi v1.x.x will no longer be supported due to those changes and we'll encourage everyone to migrate to the v2. The migration process will be quick and smooth, easy as possible.
We'll soon write a blog post about our choices and why we unfortunately can't afford to support the v1 anymore, like we expected.
Since your issue concerns the admin panel and not the framework, I decided to close it aiming to keep the issue tracker lean and focus on the framework only.
Also, your issue has been fixed in this new project, thanks a lot for your suggestion. :-)


function.html: add simple navigation

Sorely needed.

Doesn't look all that great... how about a list?

I like having it be horizontal because there are a lot of items and a long vertical list is rather unpleasant.
In any case, nobody ever likes my styling. But these pages desperately need navigation, nobody ever adds it, so I suggest pulling it and someone who is more popular better at styling can then style it.

Heh, OK.

thanks

Actually, is the navigation at the top even needed? With the LNAME2 changes by themselves, you can get a link to the section by clicking the section title.

https://github.com/D-Programming-Language/dlang.org/pull/1145

Actually, is the navigation at the top even needed? With the  LNAME2  changes by themselves, you can get a link to the section by clicking the section title.

I don't know what that means. There was no existing navigation.

I don't know what that means. There was no existing navigation.

This PR changes two things:

Each heading is now an anchor, and thanks to some JS is also clickable. You can get the link to a section simply by clicking the heading.

It adds the "navigation" blob at the top.

What I meant is that 2) isn't that necessary because of 1).


Doesn't handle .nii file

Running the segmentation on an uncompressed T2_brain.nii file gives the following error:
Cifstream::Open: Cannot open file T2/T2_brain.nii.gz
Error: convert-image command returned non-zero exit status 1

Converting the .nii to a compressed .nii.gz format works normally. An unimportant error check to add in perhaps!

Thank you for spotting this :) !
This is now fixed in #21


Make opentdk available in Maven repository

To allow developers to integrate the OpenTDK modules into their Java Maven projects as pom.xml entry, the JARs and dependencies have to be available in the Maven Central Repository https://mvnrepository.com/. When using GitHub, the main branch of the selected repository can be used as deployment branch.
Therefore it is necessary to clarify the following points:

What are the requirements for the main branch ?
How can the Maven Repository be registered ?
How does the deployment process in GitHub look like ?

Good overview to keep all required stepts in mind: https://dzone.com/articles/how-to-publish-artifacts-to-maven-central

Requirements

Be owner of the domain that should be registered. If there is no ownership, a .github domain can be used
A pom.xml in the main branch with all necessary information
A GPG key pair that gets integrated into GitHub to allow the communication with the Maven Repository
Application via a JIRA ticket at Sonatype
DNS domain record with the name of the Sonatype Ticket OSSRH-XXXXX to proove the ownership

Remaining steps

Deploy into a staging repository and not the snapshot repository of Sonatype
Edit the JIRA ticket to inform Sonatype that the Repo is ready for final migration to Maven Central
Test the OpenTDK integration in a pom.xml of a random external project


k8s  deploy ldap  Multi master replication fail

My env LDAP_REPLICATION = ture but the dubug output does not add replication
Replication does not open

ww


BNs for student features in MGDistiller

I notice that there are BNs for student features in MGDistiller. Why need additional batch normalizations? What if calculating distillation loss by student features directly ?

If without BN, the final results will be a little poor than those with BN, especially on the small dataset.
So, I add it to all the experiments in default to produce stable improvements.


Move Asciidoc extensions into the project

Moved from https://github.com/gradle/dotorg-docs/
This change makes it easier to author improvements to documentation and snippet generation/styling directly inside the Build Tool project.

@bot-gradle test this

I've triggered the following builds for you:

PullRequestFeedback build

I was under the impression that GE is also using asciidoctor for their build. Do they not use these extensions?

I was under the impression that GE is also using asciidoc for their docs. Do they not use these extensions?

We should ask @britter just in case?

We're not using this as far as I can tell.

@bot-gradle test and merge

I've triggered a build for you.


Device model

Put the device model and the computational model on a similar basis from an object-orientated point of view.

Coverage increased (+1.07%) to 93.963% when pulling 83df8ef4f6615572b7b966522be70adc048c2e22 on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.9%) to 94.751% when pulling 43c5a6af013ef1efb6ec35f2efc91dd46cdb105a on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.2%) to 94.057% when pulling d702b7d4703feb615d627fe4de08c659f76f16ff on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.2%) to 94.132% when pulling 21dcc2d296622f5c152c2a60916eea48bdba73bc on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.2%) to 94.072% when pulling 21dcc2d296622f5c152c2a60916eea48bdba73bc on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.2%) to 94.072% when pulling f70a071637119428f264c4caa861262f2bdaeebe on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.2%) to 94.072% when pulling f70a071637119428f264c4caa861262f2bdaeebe on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.

Coverage increased (+1.2%) to 94.072% when pulling f70a071637119428f264c4caa861262f2bdaeebe on device-model into bbe533390e5c8d720917cdea45754161952ca8a5 on master.


The Homebridge would be halted sometime after update to 1.0.2

Thanks for the new patch. I found an issue after update to 1.0.2 with the log:
[3/30/2017, 9:25:22 AM] [Broadlink RM Pro 3 MR] getSwitchState: 0
[3/30/2017, 9:25:22 AM] [Broadlink RM Pro 3 LR] getSwitchState: 0
[3/30/2017, 9:25:22 AM] [Broadlink RM Pro 3 MR] getCurrentTemperature (24.1)
[3/30/2017, 9:25:22 AM] [Broadlink RM Pro 3 LR] getCurrentTemperature (22.7)
events.js:326
throw new TypeError('"listener" argument must be a function');
^
TypeError: "listener" argument must be a function
at device.removeListener (events.js:326:15)
at Timeout.removeTemperatureListenerTimer.setTimeout [as _onTimeout] (/usr/local/lib/node_modules/homebridge-broadlink-rm/accessories/aircon.js:265:14)
at ontimeout (timers.js:380:14)
at tryOnTimeout (timers.js:244:5)
at Timer.listOnTimeout (timers.js:214:5)
and have to re-run sometime.

Looks like one of the dependencies didn't get updated. Try this:
npm r broadlinkjs -g
npm i homebridge-broadlink-rm -g

Thanks Luke, I shall try.
Furthermore, could I disable the thermometer for RM mini device as could not get the actual temperature? Please advise.

What happens when it can't get the temperature? It logs a timeout error, right?

It always shows 12C, and sometimes timeout. I am not sure that was caused by the RM mini.

I assume the temperature isn't 12C? It's odd that any temperature would be returned.
I've just pushed 1.0.3 with a new psuedoDeviceTemperature property - check the README for info and let me know how it goes.


Javascript _html_dom

Description
Describe more about the issue.
Doc Domain

[ ] Html
[ ] CSS
[x] JavaScript
[ ] Bootstrap

Are you contributing under any open-source program ?

HSOC'22

@girija0707 can u please assign me this issue under HSOC'22.


[KeyVault Admin] Add and implement core-http-compat

This package has already migrated to Core V2 (core-client and core-rest-pipeline). However, this introduced many braking changes. In order to decrease the impact, core-http-compat was created.
We need to integrate this dependency into the package.
More information about the migration to Core V2 can be found in https://github.com/Azure/azure-sdk-for-js/issues/15594

A new version of Key Vault admin has been released without core-http-compat


Modernize Dependencies v2

Incrementally update dependencies to see what is causing the docker image creation problem.

Pull Request Test Coverage Report for Build fbb2ac0a-e93a-4027-9d03-ff9b54608557

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 89.449%

Totals

Change from base Build 0c970699-4b59-472e-8a2f-0f20996ccf94:
0.0%

Covered Lines:
3632

Relevant Lines:
3990

💛  - Coveralls

Will never merge. Closing.


Allow changing between two PRs on the same branch

As part of our workflow we'll open two PRs from the same branch into different target branches eg:

feature/branch -> main
feature/branch -> QA

This will create two entries in the 'Github' tab

currently it is impossible to switch 'focus' between these requests without checking out a different branch first. Pressing 'checkout' on the 'unfocussed' PR does not do anything - presumably as you're already on that branch.
Ideally we could switch between multiple PRs on the same branch a bit more easily (eg the 'checkout' button would switch which PR you are currently focussed on).

Duplicate of https://github.com/microsoft/vscode-pull-request-github/issues/2853


Where are deps installed / downloaded to?

Hi. I tried to find the path where deps are installed or downloaded to, but no luck. Can you point me in the right direction?

The deps

Do you mean the tools that go-plus and related packaged download to make use of? All that is run is go get -u <package path>, so that would imply the source is at $GOPATH/src/<package path> and the binaries are at $GOPATH/bin.

Thanks. Okay, I see. The plugin actually uses the go tool in the background, so it's as if I had go get xxx myself.


Update the documentation to show that all PRs opened for GSSOC should point to the develop branch

Describe the bug ✍
Currently most PRs that are opened are also pointed to the main branch and that's not what the maintainers want for GSSOC
Provide step by step information reproduce the bug 📄
Include in the documentation that all PRs should point to the `develop1 branch. And also when creating new branches that they want to work on, it should be while they're on the develop branch. This would make it easier for the maintainers when reviewing and merging PRs
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hi @sdas1311
This is the resaon why. You'd have to checkout to develop before you can see the latest merged changes

Hi @sdas1311 This is the resaon why. You'd have to checkout to develop before you can see the latest merged changes

My task is to : Include in the main branch documentation that all PRs should point to the `develop1 branch ?

Yes

I created a PR for this issue, can you please review it?

Develop branch and also when create a new branch on their fork, they should be checked out to develop and not main. So they can get the latest changes

I created a PR for this issue, can you please review it?

hi! could you please assign me this issue.

Develop branch and also when create a new branch on their fork, they should be checked out to develop and not main. So they can get the latest changes

hi @Lovelyfin00 , I created a PR for this issue 2 days ago, can you please review it?


Version dropdown: should the "version_match" match the version or the name of the version?

See https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/configuring.html#configure-switcher-version-match for the explanation of the feature.
For the configuration of the version dropdown, you provide a json like
[
    {
        "name": "v2.1 (stable)",
        "version": "2.1"
    },
...
]

and so when specifying the switcher['version_match'] in conf.py, there are two potential things that it could match: the "name" or the "version" from the json file.
Currently, it tries to match the version:
https://github.com/pydata/pydata-sphinx-theme/blob/ef7d5a28f0b3354b2c71cc7bd041f607e315eb99/src/pydata_sphinx_theme/theme/pydata_sphinx_theme/_templates/version-switcher.html#L68-L75
While starting to use the version dropdown for the pandas docs, I actually implemented this to match the "name" (https://github.com/pandas-dev/pandas/pull/45370). The reason for this is because our "version" needs to follow the url scheme, and for historical reasons this is a bit complicated (so in our versions.json, the "name" entry is simpler)
Now, our case might of course be a bit specific.
But so that did me wonder in general which of the two would in practice be the most useful to try to match?
(cc @drammock)

ha, good catch. I'm actually not sure if this is working or not for MNE-Python because I didn't backport the theme's switcher to our stable docs, but I suspect that if I did, it wouldn't work. In fact for us it would probably still not work if we matched against entry.name.  We currently do a little magic in conf.py so that the match succeeds for the dev version:
release = mne.__version__  # The full version, including alpha/beta/rc tags.
version = '.'.join(release.split('.')[:2])  # The short X.Y version.
# (we need something like ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ for "stable" too!)
switcher_version_match = 'dev' if release.endswith('dev0') else version
html_theme_options = {
    # (other entries omitted for brevity)
    'switcher': {
        'json_url': 'https://mne.tools/dev/_static/versions.json',
        'url_template': 'https://mne.tools/{version}/',
        'version_match': switcher_version_match,
    }
}

I don't know if that would be enough to fix things in the pandas case?  Because I'm not familiar with the URL versioning scheme you folks use.  MNE-Python uses this:

version
URL
dropdown version
dropdown name

1.0
dev
dev
1.0 (devel)

0.24
stable
stable
0.24 (stable)

0.23
0.23
0.23
0.23

0.22
0.22
0.22
0.22

0.21
0.21
0.21
0.21

...
...
...
...

which is... messy.

to say something a bit more coherent about this:
The problem that I think has been revealed here is that the version value of a given entry in versions.json needs to do two things: (1) match against a version_match value from conf.py when the docs are built, and (2) be the correct form to insert into a template URL to make it valid.  If you wanted to match against entry.name instead, then you'd need a way to generate the "display name" of the version within conf.py and also hard-code it in versions.json --- not impossible but seems like extra effort?  (you need variables release and version in conf.py for sphinx anyway, why not use one of them?)
That said, maybe these two things could be decoupled.  If each versions.json entry had a new key (url_string?) that is what gets inserted into the template URL, that might make things a lot easier / more flexible for projects where the URL scheme is messy for historical reasons.  This new url_string could be optional (like name, it could default to the same value as version if undefined).

So for pandas, that table looks something like:

version
URL (after pandas.pydata/org/)
dropdown "version"
dropdown "name"

1.5.0.dev
docs/dev
docs/dev
1.5 (dev)

1.4.0
docs
docs
1.4.0 (stable)

1.3.5
pandas-docs/version/1.3
pandas-docs/version/1.3
1.3.5

1.2.5
pandas-docs/version/1.2
pandas-docs/version/1.2
1.2.5

1.1.5
pandas-docs/version/1.1
pandas-docs/version/1.1
1.1.5

...
...
...
...

Currently, I use this long dropdown "version" that includes a larger part of the url, to overcome the messy url scheme (for historical reasons). Now, the idea of being able to override the "url_template" (https://github.com/pydata/pydata-sphinx-theme/issues/573), could maybe also used here to simplify the "version", and make it easier to match that version.
And because of that complexity in the dropdown "version", in our case it's actually simpler to construct the dropdown "name" in our conf.py. But this problem will of course not be something generic.

Something else that I noticed with the "matching" of the version, is that the documentation needs to be built specifically for a specific version.
Especially for the "stable" docs, for the matching to work, those docs need to be specifically be "built as stable docs". Assume your last release is 1.3, and you built the docs for 1.3, then the dropdown version might be "stable". But if you then have a next release, you can simply copy those docs to a /1.3/ folder, as you also need to update the "version_match" in the conf.py and rebuilt those docs.
This might be obvious, but eg in the case of pandas, our "stable" docs are actually a symlink to a specific version, and on release we update the symlink to point to the correct docs. With such an approach, this "version matching" will also never work.

It's actually also not working on our own demo docs, but I think this has two reasons:

we don't do a good (consistent) job of bumping our "dev" version in __init__.py, which is currently still hardcoded as "0.7.1" instead of being something like "0.7.1+.." or "0.8.0dev0" (at the moment be just bump that version when doing a release, in the past we had the habit to also bump it to a dev version after each release)
we also haven't updated our switcher.json to include the latest release or a dev version (we should update our release instructions to update this file in addition to the version itself)

I think since the beginning of the year the documentation of the version switcher is now well explained. Is it still an issue ?

This issue is on pause for several months and it's working on the latest pandas documentation version and on our site as well.
The initial question is answered here :
https://pydata-sphinx-theme.readthedocs.io/en/latest/user_guide/version-dropdown.html#add-a-json-file-to-define-your-switcher-s-versions
We are matching on version as name is an optional parameter.


[Enhancement] Method may fail to close stream and reliance on default encoding [PinpointConfiguration]

Search before asking

[X] I had searched in the issues and found no similar issues.

Enhancement Request

located at:
org/apache/eventmesh/trace/pinpoint/config/PinpointConfiguration.java line 117
analysis and explanation:

The method creates an IO stream object, does not assign it to any fields, pass it to other methods that might close it, or return it, and does not appear to close the stream on all paths out of the method.  This may result in a file descriptor leak.  It is generally a good idea to use a finally block to ensure that streams are closed.
Found a call to a method which will perform a byte to String (or String to byte) conversion, and will assume that the default platform encoding is suitable. This will cause the application behaviour to vary between platforms. Use an alternative API and specify a charset name or Charset object explicitly.

Describe the solution you'd like
reference:

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Hi, I want to fix this bug. @Alonexc


fix: use pytest instead of deprecated setup.py command

Coverage: 81.066%. Remained the same when pulling daa4bd007457e8d7325f0dd0902353c5302f5072 on fix/setuppy_doc into bb0e29deabe08b389a798bd22568a96da07fe5e8 on master.


Switch from failure to anyhow

This moves us back to std::error::Error compatibility and helps build
times since it doesn't rely on an older version of syn

https://travis-ci.org/rustwasm/walrus/jobs/596675888#L861-L865
error[E0599]: no method named `iter_chain` found for type `anyhow::error::Error` in the current scope
 --> examples/round-trip.rs:8:20
  |
8 |         for c in e.iter_chain() {
  |                    ^^^^^^^^^^
error: aborting due to previous error

Need to update the examples too.


Efficient Handling of Post Processing of Model Scores

During the model evaluation phase, we utilize the postprocessing module.

01_scores_wandb_new.py: In this module we utilize the models to calculate cross-modal scores, across the text and audio

General idea is we compute score for an (audio,caption) for all the text captions in that particular split, likewise, we also calculate scores for a caption, taken in pair with all the audios
For ex. If train split has 7678 audios and each has 5 captions, then we will have about 23034 captions, which will be paired with 1 audio. Vice-versa, 1 caption will be paired with 7678 audios
These pairs will be used to evaluate audio2txt scores of R1, R5, R10 and mAP10

02_xmodel_data_split.py: In this module we split the audio2text (fid2items) and text2audio (tid2items) pairs into separate databases, for efficient handling in score calculations
03_retrieval_split_wandb_new.py: We utilize this to calculate the metrics for txt2audio and audio2text using the fid2items.db and tid2items.db

The real issue arises, when we create 2 dbs for fid2items and tid2items in
https://github.com/Yuvraj-Dhepe/dcase2023-audio-retrieval/blob/00602e761d8e81f7c7654520b44f8f8b6ed83d91/postprocessing/02_xmodel_data_split.py#L61-L87

The reason for inefficiency is we need to keep the all_tid2items variable in memory, while whole processing is not completed.
Batching is not feasible, because we have to collect all fids for a single tid, and in batching it wouldn't work, cause the value for tid's will overwrite the old batches contents.
I tried a simple version, where I write tid_2_items where batch size is 1, so writing to tid_stream tid by tid, this gets very bad in terms of time consumption, like around 12 hrs just to process the tids (N I am talking about only 1x times the original data, I have to go to 5x times)

https://github.com/Yuvraj-Dhepe/dcase2023-audio-retrieval/blob/00602e761d8e81f7c7654520b44f8f8b6ed83d91/postprocessing/02_xmodel_data_split_batching.py#L70-L92

Even if I do it batching, like collecting some tids it still is quite slow

https://github.com/Yuvraj-Dhepe/dcase2023-audio-retrieval/blob/3751491e4e9666e64af1cefee6cbaacf601f4792/postprocessing/02_xmodel_data_split_batching_2.py#L107-L131

Hence I would like to know how to handle this scenario in a better manner, so I can efficiently calculate the metrics, from the model scores. Currently, I only face the memory limitation to process the train set's model scores (where train set is 3x or more than the original train set) to get the evaluation metrics.

Validation Dataset X Model Scores. One can play with the dataset, and the scripts mentioned above to efficiently process them and get approximately the following scores using the retrieval files:
Text to Audio

Split
R1
R5
R10
mAP

eval
0.132440191
0.371674641
0.500287081
0.232714665

val
0.13645933
0.363444976
0.50277512
0.235242196

train
0.13850333
0.350872623
0.468276709
0.229974988

Audio to Text

Split
R1
R5
R10
mAP

eval
0.025645933
0.119043062
0.191578947
0.098888661

val
0.028133971
0.124210526
0.204210526
0.104337055

train
0.063454024
0.266788226
0.382026569
0.159521473

Solution is to not run evaluation only with eval and valid sets 😄

Solution is to not run evaluation only with eval and valid sets 😄


[SWARM-1921] Making KC SecurityContext availability not depending on the use of Secured

Windows tests look good


in silico PCR error

Hi there, when I try to run the in silico PCR on a merged file containing sequences from BOLD, I get the following error
|            Function | Extract amplicons through in silico PCR
|         Import data | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:01
|  Transform to fasta | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
|       In silico PCR | ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   1% 0:00:02 0:00:00
Traceback (most recent call last):
  File "/Users/saraswaminathan/reference_database_creator/crabs", line 844, in <module>
    crabs()
  File "/Users/saraswaminathan/miniconda3/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/saraswaminathan/miniconda3/lib/python3.12/site-packages/rich_click/rich_command.py", line 152, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/saraswaminathan/miniconda3/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/saraswaminathan/miniconda3/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/saraswaminathan/reference_database_creator/crabs", line 611, in crabs
    trimmed_seqs, untrimmed_seqs = cutadapt(console, columns, adapter, temp_input_path, fasta_dict, mismatch_, overlap, threads_)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/saraswaminathan/reference_database_creator/function/crabs_functions.py", line 1094, in cutadapt
    if seq != fasta_dict[header]:
              ~~~~~~~~~~^^^^^^^^
KeyError: '>crabs_11695_Mogoplistidae\tMogoplistidae\t1301073\tEukaryota\tArthropoda\tInsecta\tOrthoptera\tMogoplistidae\tNA\tNA\tCCTCTATFatal error: Uncaught

 HTTP_Request2_MessageException: Request timed out due to default_socket_timeout php.ini setting in /usr/share/pear/HTTP/Request2/SocketWrapper.php

The last section of that says "KeyError" goes on for a really long time and is unintelligible so I cut it off after a few lines just so you could see. I tried the in-silico-PCR command with the primer sequences in your example and some other ones as well, and I got the same error both times.
I am running python 3.12.4 with cutadapt 4.9 on an M2 macbook pro running Ventura 13.2.1

Hello @saraswaminathan,
Could you please provide the exact code you have been using thus far? I've tried to recreate the error by running the following lines, but didn't receive any errors. FYI, the primers I used are the standard COI primers for metabarcoding. The KeyError you are receiving indicates that there is something wrong with the sequence headers, so maybe the file is wrongly parsed. Any additional info would be greatly appreciated.
crabs --download-bold --taxon Mogoplistidae --output bold_mogoplistidae.fasta
crabs --download-taxonomy
crabs --import --import-format bold --names names.dmp --nodes nodes.dmp --acc2tax nucl_gb.accession2taxid --input bold_mogoplistidae.fasta --output crabs_bold.txt --ranks 'superkingdom;phylum;class;order;family;genus;species'
crabs --in-silico-pcr --input crabs_bold.txt --output crabs_bold_insilico.txt --forward GGWACWGGWTGAACWGTWTAYCCYCC --reverse TAIACYTCIGGRTGICCRAARAAYCA

Best wishes,
Gert-Jan

Hi Gert-Jan,
My original workflow involved downloading hundreds of small fasta files, importing and merging all of them, and then conducting the in-silico PCR. I am now downloading fewer, larger fasta files, and have had no issue with the in-silico PCR since making that change.


Limit transposition of narrow-N to only CPUMaterializeEncoding

The comments added in this PR explain why this is wanted on CPU, and not wanted on GPU.
This fixes e2e correctness issues that we were experiencing on GPU data tiling, as transposition of narrow-N was not fully implemented there.

wait, why does it have submodule changes?

Oh, good catch, thanks! Side effect of doing too many things during integrate week.

I want to shuffle some code about encodings, so I'm going to land the PR.


Allow searching for word definitions/translation from PowerToys Run

Description of the new feature / enhancement
When typing a single word or concept - provide its definition as one of the results
Can be translation of a word in a foreign language, definition of a word, etc
Scenario when this would be used?
Whenever there is a word you want to look up
Supporting information
Similar to what is available in the Mac search - where typing a word gives you it's definition in addition to other search results

/dup https://github.com/microsoft/PowerToys/issues/9095


handle return val of fwrite() to pacify compiler warn

Getting compiler warnings on gcc 4.4.7:
+ make tini-static
Scanning dependencies of target tini-static
[100%] Building C object CMakeFiles/tini-static.dir/src/tini.c.o
cc1: warnings being treated as errors
/go/tini/src/tini.c: In function 'print_license':
/go/tini/src/tini.c:211: error: ignoring return value of 'fwrite', declared with attribute warn_unused_result
make[3]: *** [CMakeFiles/tini-static.dir/src/tini.c.o] Error 1
make[2]: *** [CMakeFiles/tini-static.dir/all] Error 2
make[1]: *** [CMakeFiles/tini-static.dir/rule] Error 2
make: *** [tini-static] Error 2

See https://github.com/docker/docker/issues/28273 for background.

/cc @crosbymichael

@crosbymichael fyi i've checked for return value instead of just ignoring the return val in accordance with man page

Hey @andrewhsu, thanks a lot for bringing this up!
I'd like to avoid having to many exit() points in Tini, so do you think we could just skip that part and do nothing in the error branch? What about:
void print_license(FILE* const file) {
	if(LICENSE_len > fwrite(LICENSE, sizeof(char), LICENSE_len, file)) {
		// Don't handle this error for now, since parse_args won't care
		// about the return value. We do need to check it to compile with
		// older glibc, though.
		// See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25509
		// See: http://sourceware.org/bugzilla/show_bug.cgi?id=11959
	}
}

(I also think it's worth adding a comment around just so we don't accidentally break this for you later!)
This appears to compile fine for me, but please let me know if it does with your older gcc / libc (I do believe the issue you're running into is due to glibc, not gcc: http://sourceware.org/bugzilla/show_bug.cgi?id=11959)
To summarize:

Thanks again for this PR!
Let me know if the proposed change fix looks good for you, and if it does, please amend the PR and I'll merge.
Please also feel free to add yourself to the contributors list (update the README template under tpl then rebuild to update the actual README — I'm happy to do this for you if needed).

To be clear: in the longer term I agree Tini should exit with 1 here if fwrite fails, but this should be handled by parse_args, not print_license, as it's a problem in other places as well (print_usage also ignores return values for fprintf). This should not hold up this fix you though, hence why I suggest we just ignore the error for now!
Thanks!

Update with the comment

Thanks! I'll merge this now.


I am unable to install pcf dev. cf cli in the last version but the installer says is too old.

This is what happens:
localhost:Downloads jorge$ cf update-cli
You are using the latest version of cf cli (v6.40.1)
localhost:Downloads jorge$ cf --version
cf versión 6.40.1+85d04488a.2018-10-31
localhost:Downloads jorge$ ./pcfdev-v0.30.0+PCF1.11.0-osx
Your cf CLI version is too old. Please install the latest cf CLI.
localhost:Downloads jorge$

How is it possible?

I'm facing this problem, too:
`$ ./pcfdev-v0.30.0+PCF1.11.0-linux
Your cf CLI version is too old. Please install the latest cf CLI.
$ cf update-cli
You are using the latest version of cf cli (v6.40.1)`

@dcmoeller @llyorshch - You are using the legacy version of PCF Dev, I would recommend using our new version - https://network.pivotal.io/products/pcfdev
If you run into system or memory constraints - I would recommend the OSS version - CF Dev - https://github.com/cloudfoundry-incubator/cfdev only.  It has a smaller footprint and requires less resources.

This does not work for me:
$ cf dev start -f pcfdev-pas.v.<IP_ADDRESS>-build.0.101.0.iso
'dev' ist kein registrierter Befehl. Siehe 'cf help -a'

$ cf install-plugin -r CF-Community "cfdev"
CF-Community nach Plug-in cfdev durchsuchen...
Das angeforderte Plug-in verfügt über keine Binärdatei für Ihre Plattform.
FEHLGESCHLAGEN

$ cf install-plugin cfdev
CF-Community nach Plug-in cfdev durchsuchen...
Das angeforderte Plug-in verfügt über keine Binärdatei für Ihre Plattform.
FEHLGESCHLAGEN

Cited from https://github.com/cloudfoundry-incubator/cfdev:

Please note CF Dev only supports macOS at this time

@dcmoeller - That is slightly out of date - ive updated it - as we support MacOS and Windows 10 at this time.  Linux support is on our roadmap.

@llyorshch @dcmoeller could you try installing 0.30.0 via cf install-plugin instead of by executing it directly. Like this:
cf install-plugin ./pcfdev-v0.30.0+PCF1.11.0-linux

@sclevine OK, thank you, it seems to work now:

Downloading VM ...

:-)


Update repos.js

hi @Rahban1, kindly make your PR in the main branch, currently it is in dev branch


Is there a multiset implementation

redisson libray has types listmultimap, setmultimap, maps, and sets.  Is there an implementation for a multiset that is avaiable?

There is implementation of multiset


Load dynamic Google Client ID

Hi,
I am trying to load dynamicly Google Client ID.
Right now GoogleSigninProvider is injected in .config of app and GoogleSigninProvider.init() method is the only place the it can be called. I have separate service that provides me a Client ID. What is the process to call that service and get the Client ID value in .config block so I can call init() with that value.
Do you maybe know the solution?
Best regards

Also, is there a way to call GoogleSigninProvider.init() or likewise function somewhere else rather than in .config block?

@sharkee89 there is a setClientId method on the GoogleSignin provider. The init method sets some options for the service to use, but I'm unsure if it's required to be called. I would try dynamically getting your Client ID and then calling GoogleSignin.setClientId with that value. Could you let me know if that works?

@rtucker88 Hi, as far as I can see, setClientId method is not accessible via GoogleSignin, it is a private method accessible only via GoogleSigninProvider so I still have a problem because in my .config block my dynamicly generated clientId is not available so I need some method to set client id via GoogleSignin.
In my .run block when I try to call GoogleSignin.setClientID, I get: Uncaught TypeError: GoogleSignin.setClientId is not a function.
I am not fan of rewriting the module so is there something that can be done?

Hi @sharkee89, I see the issue now. I have an idea of how to fix this. I'm going to try to get to it after work today.


all demos fail to run on linux

triangle: .../vulkan-demos.git/base/vulkantools.cpp:236: VkShaderModule_T* vkTools::loadShader(const char*, VkDevice, VkShaderStageFlagBits): Assertion `size > 0' failed.
Aborted (core dumped)
running the intel driver on linux

This assertion is raised if the shader files could not be found. Be sure to put the binaries into the bin subfolder (see readme.md for details) and make sure they have access to the shaders in data/shaders.
I tested all examples on Linux and they run fine.

OK I missed that.. so it's not "meant" to be packaged as an system-wide installable thing? At least I don't know what to change
and my own build makes many binaries segfault, but that could be a driver bug too

No, there is no systemwide install for Linux.


Added gImages search

THIS TIME IT SHOULD ACTUALLY WORK
bloody useless Github for Windows not updating for local changes.

fixed

there :P

Thanks :)

Actually I probably screwed up, I don't think JSON supports trailing commas like that. I'll fix it.


Initial Pull request for logstash-filter-language

Here is the pull request discussed in logstash issue #4859.
Thank you!

@suyograo - Anything else that I need to do with this submission?

LGTM. Thanks for contributing!


the exit() problem when using cpp_neighbors.batch_query() function

Hello, HuguesTHOMAS!
When I try to use the function cpp_neighbors.batch_query() function in some other scenes, I find a failure case that when the code goes into this function, the code will cause exit() without any warning information or error message (I don't know whether should I use other IDE to Debug the code in C++ instead of Pycharm). This situation will happens after several loops (In every loop I will call this function once)，I checked the number of points (around 60000)，It wasn't the biggest number (around 150000) compared with previous loops. I also checked whether there exists nan , inf, overflow or underflow in the points data, the answer is not.
I want to know that if there exists any using limitation of this function or there are some factors that I didn't noticed.

Hi @cppcute-pm,
The cpp functions is extremely hard to debug. The python debug mode wont work with them. You need to place cout in the cpp file to follow the code execution by printing stuff. Also you need to recompile the wrappers every time you change the cpp file. I had issues in the past and it took me quite some time to find the errors. Some ideas of possible mistakes:

The neighbor indices are negative or higher than the number of points
The batch lenghts are not correct

Good luck

Thanks a lot for your reply!
I'll check my usecase carefully.