Unable to flash Tekin SP22

Hello,
I try to flash a Tekin SP22 WiFi switch / power meter. I use the master branch after #279 has been merged to it, executing on a Raspberry Pi.
Commit is d9211e0f17641b4af46336c69fd067d0742684ec. Raspbian Buster
I run ./start_flash.sh after a fresh reboot, however, the vtrust-flash wifi network does not appear. After stopping and restarting start_flash.sh it appears and I can connect to it.
I put the switch into flashing mode by a long press on the button, it starts to blink red-blue.
About two seconds after I press Enter to continue flashing, the switch goes back to normal mode (stops blinking):
Starting pairing procedure in screen
RTNETLINK answers: File exists
~/tuya-convert
Waiting for the upgraded device to appear
If this does not work have a look at the '*.log'-files in the 'scripts' subfolder!
.... dots keep appearing

and that's it until I cancel the flashing after a couple of minutes.
The log files:
smarthack-mqtt.log
smarthack-psk.log
smarthack-smartconfig.log
smarthack-web.log
smarthack-wifi.log
Thanks your any advice!

Looks like it did flash the intermediate firmware. Try unplugging the device and plugging it back in while start_flash is still running (the vtrust-flash AP is online). Then try to curl http://<IP_ADDRESS>.

Ok, I made some headway. Don't know exactly why, it seems that my problem was, that on first attempt after the Raspberry reboot the vtrust-flash network was not created. Before my second attempt, I did not execute stop_flash.sh before, which somehow made it not working. But now:
IoT-device is online with ip <IP_ADDRESS>
Fetching firmware backup
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  4 1024k    4 49640    0     0  14413      0  0:01:12  0:00:03  0:01:09 14413
curl: Saved to filename 'firmware-71d2e4.bin'
curl: (18) transfer closed with 998936 bytes remaining to read
======================================================
Getting Info from IoT-device
VTRUST-FLASH 1.1
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://<IP_ADDRESS>/backup
ChipID: 71d2e4
MAC: 84:0D:8E:71:D2:E4
BootVersion: 7
BootMode: normal
FlashMode: 1M DOUT @ 40MHz
FlashChipId: 144051
FlashChipRealSize: 1024K
Active Userspace: user2 0x81000

======================================================
Please make sure to note the correct SPI flash mode!
Installing an alternative firmware with the wrong flash mode will leave the ESP unable to boot!

Next steps:
1. To go back to the orginal software
   # curl http://<IP_ADDRESS>/undo

2. Be sure the conversion software runs in user2
   # curl http://<IP_ADDRESS>/flash2

3. Flash a third party firmware to the device
BE SURE THE FIRMWARE FITS THE DEVICE AND USES THE CORRECT FLASH MODE!
MAXIMUM SIZE IS 512KB
put or link it to ./files/thirdparty.bin
A basic build of Sonoff-Tasmota v6.5.0 is already included in this repository.
   # curl http://<IP_ADDRESS>/flash3
Alternatively let the device download and flash a file via HTTP:
   # curl http://<IP_ADDRESS>/flash3?url=http://<IP_ADDRESS>/files/thirdparty.bin

HAVE FUN!

So I continue:
root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>/flash2
Device is already booting from userspace 2 (0x81000)
root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>/flash3
Device should flash http://<IP_ADDRESS>/files/thirdparty.bin and restart

Which right now is the current state. On my mobile phone, the vtrust-flash network is still there, but no sonoff-xxx and it has been like that for a couple of minutes. 5 minutes at least, was I too impatient?
I can't reach the device as <IP_ADDRESS> either. Un-Plugging hasn't changed anything either. Right now, it seems to be bricked...
smarthack-mqtt.log
smarthack-psk.log
smarthack-smartconfig.log
smarthack-web.log
smarthack-wifi.log
Any suggestions? Thanks!

root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>/flash3
Device should flash http://<IP_ADDRESS>/files/thirdparty.bin and restart

Do you see thirdparty.bin being requested in your smarthack-web.log?

Which right now is the current state. On my mobile phone, the vtrust-flash network is still there, but no sonoff-xxx and it has been like that for a couple of minutes. 5 minutes at least, was I too impatient?
I can't reach the device as <IP_ADDRESS> either. Un-Plugging hasn't changed anything either. Right now, it seems to be bricked...

Definitely not bricked, there's hope!

Ok, I now have a vtrust-recovery SSID, so it seems it successfully flashed the intermediate firmware. Some other comments suggest to restart the flashing (while mobile phone is connected to vtrust-flash or vtrust-recovery?)

No you do not need to keep your phone connected after the smartconfig process. If you see vtrust-recovery this tells you for sure that the device is running the intermediate firmware (not bricked) but you'll need to restart the device (unplug and replug) so that it connects to vtrust-flash before being able to do anything else.

Ok, next edit. With the correct sequence of plugging/unplugging and start_flash I was able to get so far again, that I can curl stuff and tried to initiate the flashing again by calling flash3, just like above. Giving it much more time to complete (~30 minutes), I can't still see no sonoff-xxx wifi. Unplugging-plugging the switch make me end up again with the vtrust-recovery network.

As I said, you can't do much with vtrust-recovery, it just lets you know that it still works but can't connect to vtrust-flash.

Trying to initiate flashing by calling
# curl "http://<IP_ADDRESS>/flash3?url=http://<IP_ADDRESS>/files/thirdparty.bin"
Device should flash http://<IP_ADDRESS>/files/thirdparty.bin and restart

instead of just flash3 gives the same result.

curl "http://<IP_ADDRESS>/flash3" is the exact same as curl "http://<IP_ADDRESS>/flash3?url=http://<IP_ADDRESS>/files/thirdparty.bin", it is preferred that you do the first to avoid typos.
How long did you wait between commands?

How should I proceed from here?

Were you able to get able to get a firmware backup? Please post it if you have one.

root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>/flash3
Device should flash http://<IP_ADDRESS>/files/thirdparty.bin and restart

Do you see thirdparty.bin being requested in your smarthack-web.log?

Yes:
[I 191004 19:02:09 web:2246] 200 GET /files/thirdparty.bin (<IP_ADDRESS>) 5179.36ms

appeared right after calling flash3 (not more than 5 seconds later).

curl "http://<IP_ADDRESS>/flash3" is the exact same as curl "http://<IP_ADDRESS>/flash3?url=http://<IP_ADDRESS>/files/thirdparty.bin", it is preferred that you do the first to avoid typos.
How long did you wait between commands?

Between which commands?
Between calling flash3 and unplugging without having found a sonoff-xxx wifi: ~30 minutes

How should I proceed from here?

Were you able to get able to get a firmware backup? Please post it if you have one.

firmware-71d2e4.bin.log (please remove .log extension)
Thanks a lot!

Unfortunately it looks like that firmware backup did not complete. I won't be able to run this to try reproducing your issue. Let's see if anyone else can comment on their experience with the SP22.

I bought a pair of switches, the other one I left unflashed so far. After several tries I was able to connect it:
# curl http://<IP_ADDRESS>
VTRUST-FLASH 1.1
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://<IP_ADDRESS>/backup
ChipID: 70de99
MAC: 84:0D:8E:70:DE:99
BootVersion: 7
BootMode: normal
FlashMode: 1M DOUT @ 40MHz
FlashChipId: 144051
FlashChipRealSize: 1024K
Active Userspace: user2 0x81000

The firmware from the other, untouched switch, info above: firmware-70de99.bin.log
The firmware from the flashed switch which loads the intermediate firmware and creates the vtrust-recovery network: firmware-71d2e4.bin.log
Thanks!

Hi, everybody,
I ordered 4 smart sockets type Teckin SP22 from Amazon. The last 2 days I spent flashing these things. But it was only a trial.
I come to the point where the socket should connect to generic Wifi Access Point. But nothing happens except a lot of dots while the shellscript is running and the LED at the socket flashes blue fast and then stops sometime.
In the logfile I can see that my phone has connected to the AP, but the socket doesn't seem to want to connect to the AP.
I read now that there is a new firmware from Tuya which prevents it from flashing. It seems that I already have such a firmware on it. I can exclude an update from the Internet.
I also read that the problem seems to have been fixed with the latest version of Tuya-Convert. If that's true, I can't confirm that.
Is there anything else I can try? My version of Tuya-Convert is from yesterday evening. It runs on a Raspberry Pi with Kali Linux installed.
Thank you
Joe
German:
Hi zusammen,
ich habe mir 4 smarte Steckdosen von Typ Teckin SP22  über Amazon bestellt.  Es blieb aber nur beim Versuch.
 Allerdings passiert nichts außer jeder Menge Punkte während das Shellscript läuft und das bei der Steckdose die LED schnell blau blinkt und dann irgendwann aufhört.

Ich habe jetzt gelesen das es wohl eine neue Firmware seitens Tuya gibt, welche das flashen verhindert. Wie es scheint, habe ich eine solche Firmware bereits drauf. Ein Update aus dem Internet kann ich auschliessen.
Außerdem habe ich gelesen das das Problem scheinbar mit der neuesten Version von Tuya-Convert behoben worden ist. Wenn das stimmt, kann ich das so nicht bestätigen.
Gibt es vielleicht noch etwas was ich ausprobieren kann? Mein Version von Tuya-Convert ist von gestern Abend. Sie läuft auf einem Raspberry Pi mit Kali Linux installiert.
Vielen Dank
Joe

@floli thanks for the firmware. Did you experience any issues getting flashed?
@Sentinel-sys the new firmware patch has been solved in the latest version of tuya-convert. The device MAC address should show up in the wifi log if the smartconfig procedure succeeded. Can you post your logs?

@kueblc thank you for your reply. Here are the latest logfiles
Wifi Log:
Backing up NetworkManager.cfg... Restarting NetworkManager... Backing up /etc/dnsmasq.conf... Writing dnsmasq config file... Creating new /etc/dnsmasq.conf... Writing hostapd config file... Configuring AP interface... Applying iptables rules... Starting DNSMASQ server... Starting AP on wlan1 in screen terminal... Configuration file: /etc/hostapd/hostapd.conf Using interface wlan1 with hwaddr ca:e0:e0:23:ed:6d and ssid "vtrust-flash" wlan1: interface state UNINITIALIZED->ENABLED wlan1: AP-ENABLED  wlan1: STA 18:1d:ea:30:68:53 IEEE 802.11: authenticated wlan1: STA 18:1d:ea:30:68:53 IEEE 802.11: associated (aid 1) wlan1: AP-STA-CONNECTED 18:1d:ea:30:68:53 wlan1: STA 18:1d:ea:30:68:53 RADIUS: starting accounting session 24E81877E1D8D1B1 wlan1: STA 18:1d:ea:30:68:53 WPA: pairwise key handshake completed (RSN) wlan1: AP-STA-DISCONNECTED 18:1d:ea:30:68:53 wlan1: STA 18:1d:ea:30:68:53 IEEE 802.11: authenticated wlan1: STA 18:1d:ea:30:68:53 IEEE 802.11: associated (aid 1) wlan1: STA 18:1d:ea:30:68:53 IEEE 802.11: authenticated wlan1: STA 18:1d:ea:30:68:53 IEEE 802.11: associated (aid 1) wlan1: AP-STA-CONNECTED 18:1d:ea:30:68:53 wlan1: STA 18:1d:ea:30:68:53 RADIUS: starting accounting session AE6B82E83D64C67E wlan1: STA 18:1d:ea:30:68:53 WPA: pairwise key handshake completed (RSN) wlan1: AP-STA-DISCONNECTED 18:1d:ea:30:68:53 wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: associated (aid 1) wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: associated (aid 1) wlan1: AP-STA-CONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b RADIUS: starting accounting session DDBB85EC4CB0F547 wlan1: STA d8:1c:79:7f:58:5b WPA: pairwise key handshake completed (RSN) wlan1: AP-STA-DISCONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: disassociated due to inactivity wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE) wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: did not acknowledge authentication response wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: associated (aid 1) wlan1: AP-STA-CONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b RADIUS: starting accounting session 5E4124C3BF49C151 wlan1: STA d8:1c:79:7f:58:5b WPA: pairwise key handshake completed (RSN) wlan1: AP-STA-DISCONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: associated (aid 1) wlan1: AP-STA-CONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b RADIUS: starting accounting session 5E4124C3BF49C151 wlan1: STA d8:1c:79:7f:58:5b WPA: pairwise key handshake completed (RSN) wlan1: AP-STA-DISCONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: associated (aid 1) wlan1: AP-STA-CONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b RADIUS: starting accounting session 5E4124C3BF49C151 wlan1: STA d8:1c:79:7f:58:5b WPA: pairwise key handshake completed (RSN) wlan1: AP-STA-DISCONNECTED d8:1c:79:7f:58:5b wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: authenticated wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: disassociated due to inactivity wlan1: STA d8:1c:79:7f:58:5b IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)
Web Log:
    tornado.ioloop.IOLoop.current().start()   File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 148, in start     self.asyncio_loop.run_forever()   File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever     self._run_once()   File "/usr/lib/python3.6/asyncio/base_events.py", line 1415, in _run_once     event_list = self._selector.select(timeout)   File "/usr/lib/python3.6/selectors.py", line 445, in select     fd_event_list = self._epoll.poll(timeout, max_ev) KeyboardInterrupt
Do you need also the other logs?
Thank you
Joe

I see stations 18:1d:ea:30:68:53 and d8:1c:79:7f:58:5b in your wifi log, is one the device you are trying to flash?
I do not see any requests in the web log. Are there connections in the psk log?
You can create a network capture with tcpdump -i wlan0 -w capture.pcap to see what's happening after the device connects.
By the way, you can post your logs with triple backtick as in ``` Log here ```

@kueblc
First MAC address is my laptop and second MAC address is my mobile phone. So no smart plug listed there.
I just checked the PSK Log and I didn´t see any connection there.
What I´m trying to do today is, setup a freshly installed Raspberry Pi with Raspbian and try it from there. Then I provide you all the logfiles and pcap files you asked for.
Thank you
Joe

@floli thanks for the firmware. Did you experience any issues getting flashed?

Haven't tried yet to flash this switch, still fear it ends up like the other in the intermediate. Will give it a try later today and update here.

Okay here we go.
New Raspian Buster on a Raspberry 3B+. Latest Updates all installed and newest Tuya Convert loaded from Github.
I followed the intructions and ended with a lot of dots and nothing in the logfiles:
Wifi Log:
Starting AP on wlan0 in screen terminal...
Configuration file: /etc/hostapd/hostapd.conf
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:77:09:4e and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: STA d8:1c:79:7f:58:5b IEEE 802.11: associated
wlan0: AP-STA-CONNECTED d8:1c:79:7f:58:5b
wlan0: STA d8:1c:79:7f:58:5b RADIUS: starting accounting session 3507B510F6385066
wlan0: STA d8:1c:79:7f:58:5b WPA: pairwise key handshake completed (RSN)

The MAC address you see here ist my smartphone...
Web Log:
Listening on port 80```

PSK Log: This Logfile is empty

Smartconfig Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-smartconfig.log
Put Device in Learn Mode! Sending SmartConfig Packets now
Sending SSID                  vtrust-flash
Sending wifiPassword          flashmeifyoucan
SmartConfig in progress
..........
SmartConfig complete.```

MQTT Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-mqtt.log
1570354055: mosquitto version 1.5.7 starting
1570354055: Using default config.
1570354055: Opening ipv4 listen socket on port 1883.
1570354055: Opening ipv6 listen socket on port 1883.```

This time I forgot to do the tcp dump.

I tryed a second time with tcp dump enabled and this is what happend.

Wifi Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-wifi.log
Configuration file: /etc/hostapd/hostapd.conf
wlan0: Could not connect to kernel driver
Using interface wlan0 with hwaddr b8:27:eb:77:09:4e and ssid "vtrust-flash"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: STA d8:1c:79:7f:58:5b IEEE 802.11: disassociated
wlan0: STA d8:1c:79:7f:58:5b IEEE 802.11: associated
wlan0: AP-STA-CONNECTED d8:1c:79:7f:58:5b
wlan0: STA d8:1c:79:7f:58:5b RADIUS: starting accounting session 8DC11DE6BA5B20A8
wlan0: STA d8:1c:79:7f:58:5b WPA: pairwise key handshake completed (RSN)
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: associated
wlan0: AP-STA-CONNECTED 68:57:2d:42:1f:2b
wlan0: STA 68:57:2d:42:1f:2b RADIUS: starting accounting session 228DCA223FAE9FD0
wlan0: STA 68:57:2d:42:1f:2b WPA: pairwise key handshake completed (RSN)
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: disassociated
wlan0: AP-STA-DISCONNECTED 68:57:2d:42:1f:2b
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: disassociated
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: disassociated
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: associated
wlan0: AP-STA-CONNECTED 68:57:2d:42:1f:2b
wlan0: STA 68:57:2d:42:1f:2b RADIUS: starting accounting session 1CC280B2F01030E4
wlan0: STA 68:57:2d:42:1f:2b WPA: pairwise key handshake completed (RSN)
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: disassociated
wlan0: AP-STA-DISCONNECTED 68:57:2d:42:1f:2b
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: disassociated
wlan0: STA 68:57:2d:42:1f:2b IEEE 802.11: associated
wlan0: AP-STA-CONNECTED 68:57:2d:42:1f:2b
wlan0: STA 68:57:2d:42:1f:2b RADIUS: starting accounting session 7F32420F9D7386D6
wlan0: STA 68:57:2d:42:1f:2b WPA: pairwise key handshake completed (RSN)```

PSK Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-psk.log
PSK: f2927a4be0bd70ee89d024bdcd458043db2c584cf85e02a6aa7aafc493854e51
new client on port 443 from <IP_ADDRESS>:49160
ID: 0142416f68626d643661473931494652319b51d46c0bf4cfcf3f0a24897e7d206e5fa3f5f7d606638aab49cb8c9b41c6edd1
PSK: f2927a4be0bd70ee89d024bdcd458043db2c584cf85e02a6aa7aafc493854e51
new client on port 443 from <IP_ADDRESS>:49161
ID: 0142416f68626d643661473931494652319b51d46c0bf4cfcf3f0a24897e7d206e5fa3f5f7d606638aab49cb8c9b41c6edd1
PSK: f2927a4be0bd70ee89d024bdcd458043db2c584cf85e02a6aa7aafc493854e51
new client on port 443 from <IP_ADDRESS>:49162
ID: 0142416f68626d643661473931494652319b51d46c0bf4cfcf3f0a24897e7d206e5fa3f5f7d606638aab49cb8c9b41c6edd1
PSK: f2927a4be0bd70ee89d024bdcd458043db2c584cf85e02a6aa7aafc493854e51```

Web Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-web.log
User-Agent: TUYA_IOT_SDK
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 37

payload F6936110184CFDD22BF16399424E4507
WARNING: it appears this device does not use an ESP82xx and therefore cannot install ESP based firmware
Answer tuya.device.dynamic.config.get
reply {"result":"fn8oBaEAaJe3I+enO7OkYdd0LkD9WHnY6k8vmvctrJa+UccVve925bpwMk0dth5Y9F2VBmqNqTqJcWuiX4YubCuqzfAcOcp7VWpR2SX69N+7mevrGF2WUpAllBDW/Q5l","t":1570354863,"sign":"9b00fd63bd3edcfb"}
[I 191006 10:41:03 web:2246] 200 POST /d.json?a=tuya.device.config.get&et=1&other={"token":"00000000","region":"US"}&t=30&uuid=b9ebdf74d0308df1&v=4.1&sign=8b34b2e906d43bf02ae2d20242d09637 (<IP_ADDRESS>) 55.68ms```

Smartconfig Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-smartconfig.log
Sending wifiPassword          flashmeifyoucan
SmartConfig in progress
..........
SmartConfig complete.
Put Device in Learn Mode! Sending SmartConfig Packets now
Sending SSID                  vtrust-flash
Sending wifiPassword          flashmeifyoucan
SmartConfig in progress
..........
SmartConf```

MQTT Log:

```pi@raspberrypi:~/tuya-convert/scripts $ tail -f smarthack-mqtt.log
1570354055: mosquitto version 1.5.7 starting
1570354055: Using default config.
1570354055: Opening ipv4 listen socket on port 1883.
1570354055: Opening ipv6 listen socket on port 1883.
^C1570354787: mosquitto version 1.5.7 terminating
1570354794: mosquitto version 1.5.7 starting
1570354794: Using default config.
1570354794: Opening ipv4 listen socket on port 1883.
1570354794: Opening ipv6 listen socket on port 1883.```

How can I upload the TCP dump???

It seems this time I found a device but this is not an ESP32xx device... At the device itself is written "TECKIN SP22"

Thank you

Joe

@floli the devices are still stuck in the intermediate firmware? Worst case scenario, they are not bricked, you can connect to vtrust-recovery and issue curl http://<IP_ADDRESS>/undo.
@Sentinel-sys sorry to say that if they are not ESP82xx, there's nothing we can do!

@kueblc
But is this really true? I can´t imagine that this is a totally different kind of product...?

@floli the devices are still stuck in the intermediate firmware? Worst case scenario, they are not bricked, you can connect to vtrust-recovery and issue curl http://<IP_ADDRESS>/undo.

I did that and it was confirmed with Rebooting into userspace 1. But that's all that changed, the switch still creates the vtrust-recovery network after reboot and does not react to button presses.
I can still connect to it using the standard procedure with start_flash.sh.
root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>
VTRUST-FLASH 1.1
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://<IP_ADDRESS>/backup
ChipID: 71d2e4
MAC: 84:0D:8E:71:D2:E4
BootVersion: 7
BootMode: normal
FlashMode: 1M DOUT @ 40MHz
FlashChipId: 144051
FlashChipRealSize: 1024K
Active Userspace: user2 0x81000

root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>/undo
Rebooting into userspace 1

root@marduk ~/tuya-convert (git)-[master] # curl http://<IP_ADDRESS>
[...]
Active Userspace: user2 0x81000

(the active userspace has not been changed)
Initiating the flash leaves me at the same intermediate state:
 # curl http://<IP_ADDRESS>/flash3
Device should flash http://<IP_ADDRESS>/files/thirdparty.bin and restart

The other, identical, switch I left untouched so far, connected only one to extract the firmware.
Thanks!

So I´m a little bit further. I just cut one of this things in half and had a lock at the PCB. It seems that there is an TYWE2S Chip inside. And this leeds me to this page here:
https://github.com/arendst/Sonoff-Tasmota/issues/3950
Maybe there is a way to implement this into Tuya-Convert, so you didn´t need to open these things. It´s quite hard to come in, dough... :-)
Thank you
Joe

@Sentinel-sys are you sure it's a TYWE2S and not a WR3? They have started using them in the Teckin smart plugs, even with older boards silk screened with TYWE2S. See #269
@floli I haven't had the chance to look at your firmware backups. Please follow up with me if I haven't gotten back to you in the next few days.

@kueblc
I'm afraid I don't really know much about the individual chips. However, on the board next to the corresponding chip is "TYWE2S". I got the plugs from "Gosund" now, at least they still have screws on the backside.
But I can't see any differences in the chips at the moment.
I will take a closer look at it tonight and compare it. I will also try to flash the "Gosund" parts...
Wish me luck
Ich kenne mich leider nicht so wirklich mit den einzelnen Chips aus. Allerdings steht auf der Platine neben dem entsprechenden Chip "TYWE2S".
Allerdings kann ich aktuell keine Unterschiede bei den Chips feststellen.
Ich werde mir das heute Abend noch einmal genauer ansehen und vergleichen.
Wünscht mit Glück
Gruß
Johannes
https://nextcloud.die-dormanns.de/index.php/f/97997
https://nextcloud.die-dormanns.de/index.php/f/98000
https://nextcloud.die-dormanns.de/index.php/f/97987

I agree with guys that says that new chip is being used by tekin SP22.
Yesterday, I tried to flash them with no luck. Here you are my logs.
==> smarthack-web.log <==
User-Agent: TUYA_IOT_SDK
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 37

payload 593360F8F03D17F6534F121DD6E2090C
**WARNING: it appears this device does not use an ESP82xx and therefore cannot install ESP based firmware**
Answer tuya.device.dynamic.config.get
reply {"result":"fn8oBaEAaJe3I+enO7OkYdd0LkD9WHnY6k8vmvctrJatKtfL+0oMHlKkGrYoGl499F2VBmqNqTqJcWuiX4YubPyC9VcBmtkwJpq3luO0uBm7mevrGF2WUpAllBDW/Q5l","t":1570554898,"sign":"fca7aa269ae4d49c"}
[I 191008 18:14:58 web:2246] 200 POST /d.json?a=tuya.device.config.get&et=1&other={"token":"00000000","region":"US"}&t=30&uuid=ab712cfef6edfffd&v=4.1&sign=2da154d988bc1829e67a0b37a5015ada (<IP_ADDRESS>) 57.42ms

I have added bold letters to warning message. It seems they are using a different chip. Will it be supported?

Hi,
I was able to flash 4 SP22 sockets (purchased some weeks ago from ebay) with the newest master. I have opened one up and the label is saying TYWE2S.
Host system was an old laptop with Ubuntu 18.04. Had to
sudo systemctl disable systemd-resolved
sudo systemctl stop systemd-resolved
to resolve the port 53 issue.
The script is messing with some files in the /etc. Backup the whole folder before starting to be able to get everything back to normal (dns etc.).
Thanks for your effort so far to make this work.

@jjdiazgarcia no support is planned for non-ESP82xx chips, as that would require a completely different firmware. I'm not aware of any custom firmware solutions for the Realtek based WR3.
@blissarts these issues should be resolved in the development branch, thanks for reporting.

@kueblc ok. Then, I will send back plugs. Thanks so much.

@floli I haven't had the chance to look at your firmware backups. Please follow up with me if I haven't gotten back to you in the next few days.

Hey @kueblc ,
were you able to take a look at the firmwares?
Best Thanks!

Thanks for following up with me @floli. I just had a look, but they are not complete. A successful backup should be exactly 1MB. How did you create the backups?

I uploaded two firmwares.
The first one is from an untouched switch, which I haven't tried to flash: firmware-70de99.bin.log
The firmware from the flashed switch which loads the intermediate firmware and creates the vtrust-recovery network: firmware-71d2e4.bin.log.
** old uploads above **
Both firmware files were created from the start_flash.sh, right after connecting to the switches.
I fetched the firmware from the semi-flashed (intermediate firmware creating the vtrust-recovery network) again:
IoT-device is online with ip <IP_ADDRESS>
Fetching firmware backup
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  3 1024k    3 32120    0     0  12935      0  0:01:21  0:00:02  0:01:19 12935
curl: Saved to filename 'firmware-71d2e4.bin'
curl: (18) transfer closed with 1016456 bytes remaining to read
======================================================
Getting Info from IoT-device
VTRUST-FLASH 1.1
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://<IP_ADDRESS>/backup
ChipID: 71d2e4
MAC: 84:0D:8E:71:D2:E4
BootVersion: 7
BootMode: normal
FlashMode: 1M DOUT @ 40MHz
FlashChipId: 144051
FlashChipRealSize: 1024K
Active Userspace: user2 0x81000

firmware-71d2e4.bin.log (filesize is only 32K).
From the other, unflashed device I wasn't able to get another firmware images. Connecting to it seems to be a game of luck, without much luck on my side. As described above, it goes back from pairing mode to normal when Waiting for the upgraded device to appear.

Hey,
with the most recent version of tuya convert I was able to successfully flash both devices.
Thanks for your help!

Great! Thanks for following up!

Hello,
for me, even the latest version of tuya convert reports that no ESP82xx is contained in my SP22 (shipped yesterday from Amazon (Fanglishi EU).
Greetings,
Hendrik

Hello,
same for me. Two Teckin SP22 arrived yesterday and I get for bot the error, that no ESP82xx is contained.

Yes @henfri and @blohm19, tuya-convert will report this for your benefit, so that you understand why the flash is not successful and so that you do not attempt to flash the device by serial.
Do not expect this to change with any updates to tuya-convert as of course tuya-convert can not modify your device's hardware.

I know.
We are trying to help others by building some statistics.
Tuya-convert is working Well. Just the Message about the wrong hardware could bei written in the Main Screen instead of being somewhat hidden in the logs.

OK great just making sure. Thanks for the feedback!

Yes, thank you. I think your clarification is useful


Formula doesn't update properly in some cases

https://bossanova.uk/jexcel/v3/examples/spreadsheet-formulas
If you click in Total column row 1 and change the formula to =B1*C2 the total at the bottom updates.  However, if you click in Total column row 1 again and use the quick copy anchor (the block box in the bottom right) and drag it down to fill D1:D8, the total at the bottom does not update even though the values above do.

During the drag and pasting the values the formula is not updating.
but if you click on formula cell in edit view and press enter it will take the new calculations.

A lot of improvements on the PRO distribution in regards to that. CE has some improvements and eventually will get all.
https://jexcel.net


[Android] Temporary disable the unified media pipeline by default

Chromium start use desktop media pipeline on Android from version 52,
see the background: http://crbug.com/507834 and
https://docs.google.com/document/d/1QB4RVu0zT54Bys7dUHU6QeR3DyjOs-wk-MN9tjEzquQ/edit?usp=sharing
Currently, we still can use this patch to disable unified media pipeline,
that means use WebMediaPlayerAndroid instead of WebMediaPlayer_Impl, and
not use FFmpeg for demuxing and audio decoding.
However, this is just a temporary workaground, need to continue working
on find the final solution before Chromium make unified media pipeline
as the only code path.
BUG=XWALK-7231

@rakuco please help to merge


Update helpers in helper-overview.md [skip ci]

Image and Security helper not exists.

Yes. Better to mention that it's available as extension but leave it.

Of course ;)

@pana1990 can you fix it in a pull request?

@samdark done!

Thanks!


The Cheat

I don't really expect you to use this, but I was wondering how easy it was to pwn the other players.
It would be interesting to see a bot being able to defend itself from this attack.
Also, I ran out of time, but it's trivial to take control of the other bots and use them to destroy each other.

Yeah, I think the only way around this would be to freeze objects and use immutable functions instead.

Or copy each method during initialize, and hook into Object#method_added() to install them

I predict this one will win in a landslide.

This could be more fun if was renamed (as to not give away it's powers), acted like a normal bot until it had low health, and then used its emp when your opponents thought they were about to win.

Seems a lot like https://github.com/davekinkead/battlebots/pull/5 :P

it does...


Support for loading Open3D triangle meshes, materials and pointclouds into pygfx

This pull request (PR) adds support for loading Open3D geometries and materials into pygfx, as discussed in #641. Open3D provides a comprehensive suite of tools for loading, processing, and saving 3D geometry data. This PR introduces compatibility features similar to the existing integration for Trimesh.

The implementation supports loading triangle meshes, materials, and point clouds from Open3D objects. Additionally, I have added two new examples to the gallery to demonstrate these capabilities.
Key Features and Methods:

pygfx.geometries._compat

geometry_from_open3d(x) - Is exposed through pygfx.geometries and calls one of the other two methods.
geometry_from_open3d_triangle_mesh(x)
geometry_from_open3d_point_cloud(x)

pygfx.materials._compat

texture_from_numpy_array(image_array, dim=2, **kwargs)
material_from_open3d(x) - Is exposed through pygfx.materials

pygfx.utils.load

meshes_from_open3d(mesh_model, materials=True)

There is currently an issue, that open3d is only built for python 3.11 and the gallery is built on 3.12. How should this be fixed? Removing it from the gallery completely?

@almarklein  I have updated code as requested.
In my opinion I would leave it at version 3.12 at the moment. Open3D currently doesn't have any great examples that need to be built anyway, and the 3.12 version should be released soon. But if you want to downgrade the CI Python version, I can of course do it.

In my opinion I would leave it at version 3.12 at the moment.

Ok, let's do that, and enable the example when Open3D works on 3.12

@Korijn I just wanted to check in and see if you've had a chance to look at the code yet?


Make sessions never expire with $config['sess_expiration'] = -1

According to the documentation:
sess_expiration > 0 => "The number of SECONDS you want the session to last."
sess_expiration = 0 => "Setting to 0 (zero) means expire when the browser is closed."
Can we please have a -1 configuration, for sessions that never expire?
Thank you.

The answer is simple - no. This is a really bad idea and we won't be encouraging this type of thing.
Although you're free to increase the sessioExpiration time to any number you need.

Why is it a bad idea? Frameworks shouldn't constraint design choices.

Why is it a bad idea? Frameworks shouldn't constraint design choices.

But at the same time, the framework shouldn't encourage using bad/insecure design choices. That's why it won't be implemented into the core. You can read more about session management recommendations here.
Nothing stops you from setting sessionExpiration to 1 year or so... Also you can implement "remember me" functionality that will live "forever" if you want. You can do all these things if you want and we can't stop you.

You have no idea about the system we are creating, requirements, audience, etc.

A never expiring session is a really bad idea, in terms of security. You are practically inviting hackers to exploit your web app by using session-based attacks.
This is not a silly design choice. What you are proposing into the core is rather insecure. Users would not want to use a framework that is promoting vulnerabilities within itself.

I agree with you that I have no idea about the system you're creating, its requirements, etc. but even if the idea of the never expiring session may fit your project's needs it certainly doesn't fit the framework's core. Sorry.


[FEATURE] Status view: Open file in different editor

I'm trying to use Sublime + GitSavvy as a git client, while using VSCode for editing (support is much better for certain languages). One of the few drawbacks I've found, is when opening files from the status view. I would like to open them in VSCode instead of Sublime.
Is is a possibility to make files open in a different editor, or simply make files open with the default file handler instead of opening them in a buffer?
Thanks!

Everything is possible but with limited resources this is sure pretty low-priority. High effort, low impact. One UX comes to mind how would you switch this behavior: now I work in Sublime, now I work with GitSavvy as a companion.  And don't "they" say: Ah, git integration is sooo good in code, and it's open source.

@kaste I would say this would be an opt-in setting. Is GitSavvy open to include this feature, given a PR?

@franciscolourenco No. I dont want to polute GitSavvy with code like open on vs, open in x, open in y, etc. What I sugguest is that we build a small package which can do this.
I use sublime most of the time. From time to time I need to open a file in VScode because it is better with typescript. Therefor I build https://github.com/stoivo/sublime-open-in-vscode a small plugin which open current file in VScode. When it gets more mature we will release it.  I would accept a PR the which whould expand the selection to the full path and open that file is VScode. Please open a PR there and we can contine the develpment there.
I will close this issue and PR

It doesn't have to be specific to vscode. It could be to open file in the default application.
Implementing it in stoivo/sublime-open-in-vscode would mean one couldn't use the o shortcut from GitSavvy right?
In any case I've implemented this in my fork, scratches my itch, even though opening and closing files is not nearly as quick. Still missing Sublime+GitSavvy for that reason.
Thanks for the reply.

We could implement a command simial to git savvys GsStatusOpenFileCommand and overwrite the o key binding with
    {
        "keys": ["o"],
        "command": "vs_open_from_gitsavvy_status",
        "context": [
            { "key": "setting.command_mode", "operator": "equal", "operand": false },
            { "key": "setting.git_savvy.status_view", "operator": "equal", "operand": true },
            { "key": "selector", "operator": "equal", "operand": "git-savvy.status meta.git-savvy.status.section meta.git-savvy.status.file" }
        ]
    },

That's exactly what @franciscolourenco should have done. Not a fork but an add-on!

But we don't maintain it.

@kaste thank you, @stoivo was pretty clear, no need for a repeater.


init Additional property init is not allowed

Trying to deploy a stack with Docker 17.06.2-ce
A service is having 'init:' property set:
version: "3.1"
services:
  dummy:
    image: hello-world
    init: true

Getting an error:
$ docker stack deploy -c dummy.yml dummy
init Additional property init is not allowed

The 'init:' option had been introduced in version 2.2 of 'docker-compose' format. Any clue would be greatly appreciated.

Is the init: option supported by docker stack deploy?  Does your example file work with docker-compose?
I found init: listed under https://docs.docker.com/compose/compose-file/compose-file-v2/#init (in the 2.2 file format reference), but not under the v3 reference, so perhaps it was removed in v3? (likely due to not being supported for services under Swarm Mode -- see https://github.com/moby/moby/issues/25303, where it appears https://github.com/docker/swarmkit/pull/2350 was merged in swarmkit, but still pending for CLI in https://github.com/docker/cli/issues/51 and/or https://github.com/moby/moby/issues/34529)

Additionally, in the future, it'd be better to post questions like this in the Docker Community Forums, the Docker Community Slack, or on Stack Overflow.  Thanks!

Compose file support was implemented in https://github.com/docker/cli/pull/1129, and completes a long chain of PR's to implement this feature :tada:


New rule: disallowUnusedParams

Ref #1771

Interesting so we're adding escope

If we're adding this, also remove unused from jshintrc please

If we're adding this

We discussed this couple times in gitter, it seems we were generally liking the idea. Plus it is pretty useful to have is as autofix.

also remove unused from jshintrc please

unused also catches the vars, whereas this rule works only for params


Dependency true/punycode is abandoned

PHP Version
irrelevant
Shopware Version
any 6.x
Expected behaviour
Should not be dependent on an abandoned piece of software.
Actual behaviour
Depends on true/punycode
See: https://packagist.org/packages/true/punycode
How to reproduce
Just call
composer update --dry-run
in your project root.
And when it tells
Package true/punycode is abandoned, you should avoid using it. No replacement was suggested.
than you see that warning, that I am talking about.

Thanks for pointing out the issue, sadly we can not drop the dependencies in a minor release, but we will remove those dependencies in the next major version.
I created a official issue tracker ticket for that and will close this issue for now.
https://issues.shopware.com/issues/NEXT-23972

I see why you do not want to drop them in a minor release.
But why did you close this issue before solving it and while NEXT-23972 is still open?

As we internally work with another issue tracker and not github the status on the official ticket is more accurate. There is no workflow to automatically link and resolve github issues, when we resolve the internal ticket. Therefore i closed both of the issues for now, so that they don't get stale.
Closing as completed is not quite right, but the only other option github offers is closing as not completed which is even worse in this case as it is actually planned.


ensure zap workflows work

resolves #152 and related to #143
cause of record count anomaly issue

there appeared to be duplicates in the raw Project and Project BBL data pulled from the CRM (e.g. Project ID 2023K0228 and BBL<PHONE_NUMBER>)
these duplicates were caused by the appending of CRM records to existing SQL tables

each dataset is downloaded as multiple json files
during the combine stage of the Runner, all downloaded json files are written to a sql table using if_exists="append"
all sql queries in this data product use a DB schema arg passed to the Runner so that data is isolated by build environment in a persistent database edm-zap

examples of DB schemas are weekly_export_schedule_main, weekly_export_workflow_dispatch_zap_runnable, test_data_actual_pr_147

we weren't deleting the table if it exists before writing the first json file to sql

changes

add ZAP product to centralized test workflow
rename and fix secrets in ZAP workflows
ensure an existing sql table is deleted before ingesting new CRM data
remove a redundant INNER JOIN from an early intermittent dcp_projectbbls table (it's used to limit BBL records to publicly visible projects, doesn't have to happen twice)
add an assertion to an integration test to ensure a single project and a BBL associated with it have the expected number of rows in their respective tables after Runner.combine()

run notes

most recent CRM export run on this branch: link
The file sizes from the 8/15 build are very close to those from the valid 7/17 build and have the following row counts:

Projects file: 32,428
Project BBLs file: 67,289

dev notes

inspecting integration test results helped identify the problem. the number for rows for a single project was increasing after each test (e.g. 5 rows became 6 rows)
ZAP devcontainer/image requires google cloud services so I held off on switching to our standard one. but we don't use the bigquery tables at all so I commented out all uses of it and we can revisit this

Oops - meant to ask for this in this PR, but would you mind consolidating requirements files, or at least bumping zap's requirements so that dependabot is happy?


Live preview not working

I was quite happy to see live previews be added a few days back, but after only a day-or-two they stopped working for me. I've tried disabling-and-re-enabling them, but no dice. I'm not getting any error codes in the "Start stable-diffusion" cell, so I'm unsure what to do.
Deleting my "sd" folder and recreating it might fix the issue, but I'd rather save that option as a last resort.

you don't need to delete the sd folder anymore, the issue is maybe with gradio server being slow, try the localtunnel


Use shift() when reducing lines to preserve lines up to cursor

The previous behavior of a resize to a smaller height kept the start of the buffer and tacked on a prompt line.  This fix allows the text up to the existing prompt to be preserved.

Hi @blink1073, thanks for opening this.
Attempted to try this and I fell into the following issue, which was not present before:

created an empty xterm front-end
created a Docker container with a bash shell that has a multi-line bash prompt
attached the xterm front-end to the docker container through a websocket
resized the back-end pty to match the size of the front-end visual element
the whole multi-line bash prompt disappeared

Could you please try a similar case in jupyter-js-ui to see if this happens there as well?

Hi @parisk, I don't have the infrastructure set up to use xterm at the moment.  We have been using term.js in Jupyter and I hand-modified these lines in node_modules to verify that it worked.

Okay @blink1073. Please let me make a couple of tests more, to see where the bug is and I will get back to you within the next hours most probably.

I was able to swap xterm for term.js in my application and verify that this fix works.

Thanks @blink1073 for letting me know. I still got the issue faced. Please let me do some more tests, to find out what might be going wrong. Seems quite strange.

Do you have any public branch of your application that I could clone to test out myself, in order to find out  if something goes wrong with my system?

This setup requires Python installed with the Jupyter Notebook.  The easiest way to get there is using Anaconda: https://www.continuum.io/downloads
git clone https://github.com/blink1073/jupyter-js-ui
cd jupyter-js-ui
git checkout xterm
npm install
npm run build
npm run build:examples
cd examples
# edit node_modules/xterm/src/xterm.js
cd terminal
npm run build
python main.py

I've been running history and then resizing the window vertically to verify.

@blink1073 I followed your instructions step-by-step and the iPython notebook does not seem to use xterm.js as its terminal.
Could you please point out if am I doing something wrong and how to fix it? Providing logs and screenshots below in order to be as helpful as possible.
Note
The file examples/terminal/build/bundle.js includes Xterm in it. It just seems that examples/terminal never loads for some reason.

Screenshots
Occupied ports (nothing listens on 8765)

Non-accessible at localhost:8765

Notebook home page

Terminal is not xterm.js

@blink1073 nevermind. Got it working. On the issue right now

Everything seems to be working great 😊 .
Thanks for your contribution!

Sweet, my pleasure.


Regarding open sourcing the CODE

When will you release the code?

Lolz! What is the purpose of this repository???

bump

bump

Sorry for the delay. The code is now released.


No metrics on authorization failure

I observed some random outages of our metrics for all proxmox nodes during incidents with only a single affected node. The log is showing this authorization failure triggered by a request done in the collector.py to get the VM metrics. So I think it happens if a node is reported as online but isn't correctly reachable or has other problems. This should be handled by a try/except logic to allow all other metrics to get correctly collected. I'll add a pull request for that.
Dec 21 00:00:14 node5 pve_exporter[476752]: Exception thrown while rendering view
Dec 21 00:00:14 node5 pve_exporter[476752]: Traceback (most recent call last):
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/http.py", line 101, in view
Dec 21 00:00:14 node5 pve_exporter[476752]:     return self._views[endpoint](**params)
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/http.py", line 53, in on_pve
Dec 21 00:00:14 node5 pve_exporter[476752]:     output = collect_pve(self._config[module], target)
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/collector.py", line 301, in collect_pve
Dec 21 00:00:14 node5 pve_exporter[476752]:     return generate_latest(registry)
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/prometheus_client/exposition.py", line 90, in generate_latest
Dec 21 00:00:14 node5 pve_exporter[476752]:     for metric in registry.collect():
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/prometheus_client/registry.py", line 75, in collect
Dec 21 00:00:14 node5 pve_exporter[476752]:     for metric in collector.collect():
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/collector.py", line 272, in collect
Dec 21 00:00:14 node5 pve_exporter[476752]:     for vmdata in self._pve.nodes(node['node']).qemu.get():
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/proxmoxer/core.py", line 84, in get
Dec 21 00:00:14 node5 pve_exporter[476752]:     return self(args)._request("GET", params=params)
Dec 21 00:00:14 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/proxmoxer/core.py", line 79, in _request
Dec 21 00:00:14 node5 pve_exporter[476752]:     resp.content))
Dec 21 00:00:14 node5 pve_exporter[476752]: proxmoxer.core.ResourceException: 401 Unauthorized: b''

Thanks for the report. Can you tell whether the exception is crashing pve_exporter? I.e., are there any sings that subsequent scrapes are executed with another pid?

From what I can see there is no crash, only the exception and the fact that no metrics are rendered. But looking for that I found another exception in the same part of the code:
Dec 21 10:39:26 node5 pve_exporter[476752]: Exception thrown while rendering view
Dec 21 10:39:26 node5 pve_exporter[476752]: Traceback (most recent call last):
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/http.py", line 101, in
Dec 21 10:39:26 node5 pve_exporter[476752]:     return self._views[endpoint](**params)
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/http.py", line 53, in o
Dec 21 10:39:26 node5 pve_exporter[476752]:     output = collect_pve(self._config[module], target)
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/collector.py", line 301
Dec 21 10:39:26 node5 pve_exporter[476752]:     return generate_latest(registry)
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/prometheus_client/exposition.py", li
Dec 21 10:39:26 node5 pve_exporter[476752]:     for metric in registry.collect():
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/prometheus_client/registry.py", line
Dec 21 10:39:26 node5 pve_exporter[476752]:     for metric in collector.collect():
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/pve_exporter/collector.py", line 272
Dec 21 10:39:26 node5 pve_exporter[476752]:     for vmdata in self._pve.nodes(node['node']).qemu.get():
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/proxmoxer/core.py", line 84, in get
Dec 21 10:39:26 node5 pve_exporter[476752]:     return self(args)._request("GET", params=params)
Dec 21 10:39:26 node5 pve_exporter[476752]:   File "/usr/local/lib/python3.7/dist-packages/proxmoxer/core.py", line 78, in _req
Dec 21 10:39:26 node5 pve_exporter[476752]:     raise ResourceException("{0} {1}: {2}".format(resp.status_code, httplib.respons
Dec 21 10:39:26 node5 pve_exporter[476752]: KeyError: 595

It seems to me that the proxmoxer library might be a bit out of date. At least after commit 4b2f7167 those status codes shouldn't result in such an exception. Would it be possible to update proxmoxer on the affected hosts?

Well you are right, I though I had updated it a while ago but somehow I didn't and was stuck with the minor version before that fix^^
But if I read the mentioned commit right, it only fixes the second error not the first one, right?

They also fixed some authentication related issues in the last couple of releases, thus it might help with the first one as well.
If it doesn't, then I'd need a bit more info on how the scraping is setup in your case. Namely:

Do you run pve_exporter on every proxmox node?
Do you scrape every node? Or just a subset? Or even just one node in order to gather the metrics?
Do you have any relevant relabel_configs and/or metric_relabel_configs in place in the prometheus configuration?

I think we have to wait than if this happens again. This will take some time because the problem only occurs rarely as the cluster is very stable most of the time. But I can still answer your question.
Currently we run the exporter on every node but only one node is scraped (I think I will change this in the future). And currently there is no relabeling config in place.

Ok, also see the comments in #54 for the relabeling thing. I need to document this in the wiki, at some point.

Ok...moving the discussion from PR 56 over here. The problem is that if a host behind the node, where the pve-exporter gets its data from (api node), fails the api query you use in the affected code could fail for that host. This has the effect that the whole scrape fails for all nodes even if all other nodes, including the api node, are still available. To prevent this we need to catch the exception. Then we can still get api data for all other nodes while the problematic node could fail not affecting the other requests.

Thanks for the explanation and I apologize that I was a bit slow on the uptake. I believe that this is the same issue as in #30 but I apparently failed to fix that completely. I tried to address this in #31 by checking whether a node is available before trying to access the lxc/qemu configurations. But apparently this is not enough :roll_eyes: .
I am suspecting the following failure mode which the previous fix did not take into account: If the status of a node changes from online to offline mid-loop, then that will break the whole scrape as reported in #30 and as you observed as well. Considering that there are performance issues with that piece of code as reported in #58, the chances for this kind of failures are certainly there.
I've published v2.1.0 earlier today with new command line flags. It is now possible to run pve_exporter with the --no-collector.config flag which will deactivate the buggy collector. The only metric this collector contributes is pve_onboot_status. Thus its usefulness is limited anyway. Given all those weird issues with that piece of code, I am considering to deactivate it by default in a future release.
That said, if you like to roll another PR, I'd be willing to include one if it replaces the incomplete fix from #31. I.e, just replace if node["status"] == "online": with try: and then catch the exception after the second loop (lxc). Instead of swallowing it, log it the same way as in http.py.
Thanks again for reporting the issue and also thanks for insisting.

I will look if I find the time to update my PR next week. But only to be sure, the issue I observe is not limited to only one scrape while the status changes, it can persist for hours.

it can persist for hours.

Interesting. Out of curiosity: When that happens, can you still use the PVE web interface on the api node to see/edit all the quemu/lxc guests on other nodes?
Also when the scrape succeeds. How long does it take? Easiest way to check that (at least on linux and mac) is to scrape manually (e.g. using curl) and measure using the time tool: time curl http://pve-node.example.org:9221/pve

Opened #63 crediting @leahoswald for the fix. Do you think that works for you?

Looks great, thanks!

Rolled this into 2.1.1. Thanks for the report and sorry again it took so long for me to understand the problem.


Correctly annotate python warnings

it looks like build is failing because tests need to be updated:
        with pytest.warns(UserWarning, match=err):
>           OldExplanationDashboard(explanation, model, dataset=x_test, true_y=y_test)
E           Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) was emitted. The list of emitted warnings is: [DeprecationWarning('ExplanationDashboard in interpret-community package is deprecated and removed.Please use the ExplanationDashboard from raiwidgets package instead.',)].

it looks like build is failing because tests need to be updated:
        with pytest.warns(UserWarning, match=err):
>           OldExplanationDashboard(explanation, model, dataset=x_test, true_y=y_test)
E           Failed: DID NOT WARN. No warnings of type (<class 'UserWarning'>,) was emitted. The list of emitted warnings is: [DeprecationWarning('ExplanationDashboard in interpret-community package is deprecated and removed.Please use the ExplanationDashboard from raiwidgets package instead.',)].

Done.


fix layout of right-to-left in homepage and tables

The layout of Home page had problem showing right-to-left texts so i fixed it.
Also tables in posts that are right-to-left had the same issue causing show them incorrectly.

Thank you!

Thank you!


When generate model using goctl, the numeric type is treated as a bigint

Describe the bug
A field of type numeric in the PostgreSQL database is generated as int64. And the numeric database type is treated as a bigint type in goctl.yaml
To Reproduce

The sql is
create table demo (
  id bigint generated always as identity primary key,
  max_position decimal(20, 4) not null
);

The genereated model struct is
Demo struct {
  Id               int64          `db:"id"`
  MaxPosition      int64          `db:"max_position"`
}

The goctl.yaml is
model:
  types_map:
     bigint:
       null_type: sql.NullInt64
       type: int64
       unsigned_type: uint64
     decimal:
       null_type: decimal.NullDecimal
       pkg: github.com/shopspring/decimal
       type: decimal.Decimal

Expected behavior
The type of MaxPosition is decimal.Decimal
Environments (please complete the following information):

go-zero version 1.7.2
goctl version 1.7.2

I have reproduced it here and it can be generated normally. Please make sure your goctl version is greater than or equal to 1.6.5

Thank you, I'll try this feature later


Adds support for concatted input of both title and subreddit

This will add support for concatenated input to the neural network. If this feature is toggled on then the output of the LSTM for the titles will be combined with the subreddit linked to that title.
NOTE:This requires that the datasets contain the subreddits on the format:
title,subreddit,users

The feature is toggled on/off using the config use_concat_input.
Currently the subreddit is just added as its unique ID, not as a one-hot vector. This could easily be changed in the future though.

This would solve issue #13


Web Service?

This is an amazing tool, thank you for building and publishing it!  Do you by chance know if anyone is hosting a web service that utilizes this tool to allow users to paste a url once and generate archives across all of the supported archive service providers in one go?  That would be amazing.  If now, I may be able interested in building such a tool.  Let me know what you think.

@arderyp A web service that uses this module would be great, however, I would be concerned about archives blocking many requests from a single source.

Good point.  If I look into it, I'll be sure to let you know. Feel free to close this issue.
Sent from a phone

On Mar 30, 2017, at 12:47 PM, Mat Kelly<EMAIL_ADDRESS>wrote:
@arderyp A web service that uses this module would be great, however, I would be concerned about archives blocking many requests from a single source.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


[Exception] Add initial DeltaCatError support

This PR add initial DeltaCat exception support.
With this support:
DeltaCat exception can be easily identified by error code going forward.
Retryable DeltaCat error can be identified at task level and job level.

⚠ Performance Alert ⚠
Possible performance regression was detected for benchmark. Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
Benchmark suite 	Current: 8529248 	Previous: d77a208 	Ratio
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[1-incremental-pkstr-sknone-norcf_V1] 0.5710365395077868 iter/sec (stddev: 0) 	2.2521535582165826 iter/sec (stddev: 0) 	3.94
This comment was automatically generated by workflow using github-action-benchmark.

Can you look at this performance drop as well?

⚠ Performance Alert ⚠
Possible performance regression was detected for benchmark. Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
Benchmark suite 	Current: 8529248 	Previous: d77a208 	Ratio
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[1-incremental-pkstr-sknone-norcf_V1] 0.5710365395077868 iter/sec (stddev: 0) 	2.2521535582165826 iter/sec (stddev: 0) 	3.94
This comment was automatically generated by workflow using github-action-benchmark.

Can you look at this performance drop as well?

The performance alert is gone after the address comment commit. Not consistent.


Update stringtable.xml

Revise some Chinese translations to make the meaning more accurate

Thank you for the contribution! Merging now!


refactor(ekf_localizer): fix tf2 listener

Description
In the ekf_localizer node,  there is a strange way of writing for tf_listener , so I fixed it.
Related links
Parent Issue:

Link

How was this PR tested?
Lsim works
Notes for reviewers
None.
Interface changes
None.

Effects on system behavior
None.

@SakodaShintaro If you're concerned about it, should we implement it using canTransform?


[Bugs] Should return after gin response

https://github.com/free5gc/pcf/blob/9dabfef9d73e8239fe9e7783f4085ff2a6726927/internal/sbi/processor/ampolicy.go#L173-L179

@HanHongChen FYI!
Fix it in your next PR.


feat: add idle data client wrapper

This commit adds support for an idle client timeout to address the issue of dropped connections in lambda environments.

oh, looks like you may have to fix the auth token support before we can merge this though.


Investigate and add Land Management and Farming in Australia, 2016-17

Please investigate and add: http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/4627.02016-17?OpenDocument
Unfortunately is not as an API on http://stat.data.abs.gov.au/
just land management practices survey up to 2013-14

I've followed the same approach as #239, so we get multiple levels of hierarchy.


[NEW ICON] DBT

Skill Name
DBT
Why?
It is popular in Data Engineering.
Reference Image

I created a new project since this project is no longer updating & the author left it, dbt is available skills-icons
https://skills-icons.vercel.app/api/icons?i=dbt


Modified git tag option in dbt deps

resolves #10381

Problem
in option git config --local column.ui always, dbt deps failed to install packages.
details on link
Solution
Use git tag --no-column instead of git tag --list to prevent the issue described in https://github.com/dbt-labs/dbt-core/issues/10305
Checklist

[x] I have read the contributing guide and understand what's expected of me
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
[ ] This PR includes type annotations for new and modified functions

Do i need to add unit test? (unit test for git deps already existed, but git cmd test is not existed)
I don't add git cmd unit test for dbt, because I think it was not necessary for external tool.

I've written unit test code. anyone review this?


Transparent ordering

Some executions always need to precede others, transparent ordering allows for this to be baked into the state modules themselves.
For instance, a cloud network system needs to always precede compute.

https://github.com/saltstack/idem/commit/fd95901e41c59e639576ed5d948c2897885732af
WOOOOOOOOOOOOOOOOOOOOOOT!


Using prometheus component with new auth system generates lots of warnings

Home Assistant release with the issue:

0.73.1
Last working Home Assistant release (if known):
Operating environment (Hass.io/Docker/Windows/etc.):

Docker
Component/platform:

https://home-assistant.io/components/prometheus
Description of problem:
I'm trying out the new auth system preview as described in https://developers.home-assistant.io/blog/2018/07/02/trying-new-auth.html . I'm using prometheus component, which provides the /api/prometheus endpoint.
Prometheus polls this periodically, based scrape_interval in your config. After enabling the new auth with legacy api password, I'm getting a message every minute:
home-assistant_1  | 2018-07-12 02:46:12 WARNING (MainThread) [homeassistant.components.http.auth] Please use access_token instead api_password.
home-assistant_1  | 2018-07-12 02:46:12 INFO (MainThread) [homeassistant.components.http.view] Serving /api/prometheus to <IP_ADDRESS> (auth: True)

I'm using the rcommended config from the component documentation.
  - job_name: 'hass'
    scrape_interval: 60s
    metrics_path: /api/prometheus
    params:
      api_password: ['so secret']
    scheme: http
    static_configs:
      - targets: ['localhost:8123']

I tried changing the param from api_password to access_token as suggested, but that just results in 403 unauthorized. Maybe the warning needs to be tweaked a bit more specific?
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
# Example configuration.yaml entry
homeassistant:
  auth_providers:
   - type: homeassistant
   - type: legacy_api_password
http:
  api_password: so secret
  use_x_forwarded_for: true
  trusted_proxies: <IP_ADDRESS>
auth:

Additional information:

Not an issue. That warning is designed to remind us what component need to change. See #15376

Warning text may be misleading, the change is not change a param, the authorization part of code need to change to use oauth 2 protocol.

I never use this component, but after quickly browse the Prometheus document, I believe following part are relevant
I believe you use one of following config before
# Optional HTTP URL parameters.
params:
  [ <string>: [<string>, ...] ]

# Sets the `Authorization` header on every scrape request with the
# configured username and password.
# password and password_file are mutually exclusive.
basic_auth:
  [ username: <string> ]
  [ password: <secret> ]
  [ password_file: <string> ]

For new auth system, you need use one of this setup
# Sets the `Authorization` header on every scrape request with
# the configured bearer token. It is mutually exclusive with `bearer_token_file`.
[ bearer_token: <secret> ]

# Sets the `Authorization` header on every scrape request with the bearer token
# read from the configured file. It is mutually exclusive with `bearer_token`.
[ bearer_token_file: /path/to/bearer/token/file ]

To get bearer token, you need to first create client_id/client_secret, then get refresh_token, then use refresh_token get access_token (that is the term we used in HA, Prometheus call it bearer_token)
Since access_token will be expired every 30 minutes, it is not practical to change your Prometheus config every 30 minutes, so, we have a task #15195 to generate a long-live access_token for such scenario.
As always, PR is welcome.

I'm not sure how useful this warning is in its current form, especially to an end user.
Even for a developer, it's possible this message gets logged without the request completing (ie invalid password) when it displays what URL was requested, and so it's impossible to know what component generated the warning.
I can just disable the auth system to avoid having all the warnings

You can turn off the warning
logger:
  logs:
    homeassistant.components.http.auth: error

Add path to the warning message sounds good.


tsconfig: allow imports from src/

As discussed here:

https://github.com/electron-react-boilerplate/electron-react-boilerplate/issues/2737

This allows for using the tsconfig import paths everywhere.

@Shuhala review plz


Extraneous  tag in final code when plugin is enabled

I implemented this plugin on my current Eleventy project and I've realised that the final html output gets an extra <html> tag, completely ignoring the one already present in my code, like so:
<!DOCTYPE html><html>
<html lang="en">

<!-- content goes here -->

</html><!-- my own -->
</html><!-- the plugin's -->

I'm running @11ty/eleventy ^0.11.0 and eleventy-plugin-images-responsiver ^1.5.3. I've detected the issue by turning all other plugins and imports inside my .eleventy.js file. I can reproduce the issue regardless of what is present in my images-responsiver config. Removing the doctype and <html> tag entirely from my templates means they are generated again, minus my personal attributes, in the final output.
I've since moved on to a custom shortcode component, but the relevant files are referred to in this commit: https://github.com/fabiomrbarbosa/fabiomrbarbosa.com/commit/caae0df2191181036c0fbaf9ebb80af31292f68f
Thanks in advance for your attention.

@fabiomrbarbosa this should be fixed in latest release

Was also experiencing this, can confirm that update to v1.8.2 fixed it.

@dirtystylus thanks!


[Feature Request] Control TV through CEC

It would be great if it was possible to not just control XMBC through Kore, but also the rest of the TV setup - eg. changing channel and source on the TV which can be done through HDMI-CEC

So you're suggesting adding IR features to kore?
What about devices without IR blaster?

Initially I was thinking about just taking advantage of the features already bundled through the cec-client lib bundled with most raspberry installations such as Kodi and OSMC, like turning the TV on/off, changing channel, adjusting the volume and so on.

Kore communicates with Kodi via the JSON-RPC API and i believe that it doesn't expose any hdmi-cec functions. @Montellese can you confirm this?

That is correct although it has been requested before.

I might be scope creep from your guys perspective ... because essentially what is the essence of what I'm asking is a single app for controlling both Kodi and my TV (because I use Kodi on my raspberry) - which makes sense from a user perspective, but not necessarily from a Kodi Remote perspective - and, yes, sounds like it requires an interface expansion in the JSON API. I would however love to have such a feature. Doesn't it sound awesome :)


Error Creating Listeners on Machines that does not support IPv6 Family

https://github.com/envoyproxy/envoy/commit/01d2e16e973a76cc05f5e9e5c7a74f73498bc6c2
Description:
After the PR https://github.com/envoyproxy/envoy/pull/3851 (commit https://github.com/envoyproxy/envoy/commit/01d2e16e973a76cc05f5e9e5c7a74f73498bc6c2) we are not able to create Listeners from ADS on machines where IPv6 family is not enabled.
The error we get is
07:09:56 2018/07/19 14:09:56 Envoy STDERR: [2018-07-19 14:09:56.284][26][warning][upstream] source/common/config/grpc_mux_impl.cc:223] gRPC config for type.googleapis.com/envoy.api.v2.Listener update rejected: Error adding/updating listener testListener: IPv6 addresses are not supported on this machine: [::]:0
Repro steps:
Try to create Listener with ADS(any xDS) with the following Listener configuration. There is nothing special in the Listener configuration. Also please note that we are not using the new FilterMatch.
listener := v2.Listener{
		Name: "testListener",
		Address: core.Address{
			Address: &core.Address_SocketAddress{
				SocketAddress: &core.SocketAddress{
					Protocol: core.TCP,
					Address:  localhost,
					PortSpecifier: &core.SocketAddress_PortValue{
						PortValue: uint32(6666),
					},
				},
			},
		},
		FilterChains: []listener.FilterChain{{
			Filters: []listener.Filter{{
				Name:   util.HTTPConnectionManager,
				Config: managerStruct,
			}},
		}},
	}

It is because of these lines specifically where we are trying to create CidrRange for IPv6 addresses

@PiotrSikora fyi..


DependencyTelemetry object hold reference to HttpWebRequest keeping connections open in case of transmission errors

List of NuGet packages and version that you are using:

    <PackageReference Include="Microsoft.ApplicationInsights" Version="2.14.0" />
    <PackageReference Include="Microsoft.ApplicationInsights.DependencyCollector" Version="2.14.0" />
    <PackageReference Include="Microsoft.ApplicationInsights.WindowsServer" Version="2.14.0" />
    <PackageReference Include="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel" Version="2.14.0" />

Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file):
net461

Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.):
Windows (application is running on Service Fabric)

Describe the bug

We use ServerTelemetryChannel which is meant to hold telemetry items in memory/disk in case there's connectivity issues with the DC endpoint.

In such case (connectivity issue) that happened in two of my VMs, we suspect that keeping so many DependencyTelemetry items in memory caused cascading effects on the machine by keeping connections open, reaching to ~30K connections

The connections were left open because each DependencyTelemetry holds a reference to a System.Net.Connection object:

The connection objects seem to point to the actual dependency I was calling (in this case Azure Storage) and not the DC endpoint (Breeze):

Another symptom we had, is that % GC Time was skyrocketing from 5% to 40% steady, could be related to the amount of objects kept in

To Reproduce
There is no simple repro for this, the ask is to avoid holding reference to the HttpWebRequest object if possible based on its impact, when retrained for long time in memory.

DependencyTelemetryModules stores the raw objects inside the telemetry for a reason. Its cleared when the DependencyTelemetry is Track ed. The behavior you are seeing should have no relation to the particular telemetry channel being used.
I need to investigate more on this...
From the screenshot shared, the DependencyObjects are kept in memcache, by design. If the repro occured when you had network issues, then its possible that the MemCache stored a lot of open HttpClient calls.. It'd be cleared automatically when the call ends (which could take long time if the call itself is slow), or when 100secs expires..
I will check more.

Why is it important to dispose the connection after the DependencyTelemetry is tracked? Theoretically it is possible to get all the information needed by observing the request+response, without holding a reference.
Maybe you can hold a weak reference instead? which will allow the GC to collect the Connection objects?

I didn't mean that the SDK should dispose, I meant that the SDK mustn't hold Connection obejects from being collected by GC.
At the moment this is the case, due to holding a reference to the Connection.
Since so many connections were not collected and disposed due to being referenced (by the DependencyTelemetry) I believe my application could not recover (for many hours), until I restarted the nodes.

mm..not sure of this yet..
SDK held on to the DepenencyTelemetry (which holds HttpRequest object containing Connection I believe), because the call did not finish. So if the httpcall was not finished, would GC be able to reclaim it anyway?

Think of a situation where the DC endpoint is unavailable for 30m, hence the ServerTelemetryChannel would keep the DependencyTelemetry items in memory.
Because of that, references to many HttpWebRequest objects are kept, which hold the GC from collecting them (and the Connection objects as a consequence) for long time, even though the dependency request itself to the storage account have ended (with success, failure or timeout).

There is no connection between ServerTelemetryChannel and this. No telemetry is collected from SDK's call to Ingestion endpoint.

Update:
Between the time httpclient makes start callback and end callback, depenencytelemetry (with HttpWebRequest inside it) is held in cache. The above issue is occuring because HttpClient .NET framework is missing to the fire the end callback in certain situations like client side timeouts/dns/etc, which cause Application Insights to keep the dependencytelemetry item in MemoryCache.
This is a known issue. There is automatic cleanup after 100 sec.
There are no fixes planned in .NET Framework to address this (see below issue). This is not an issue in .NET Core. Closing as no action for the SDK.
Runtime issue link: https://github.com/dotnet/runtime/pull/33732


Windows build

So I have a build script (powershell) that does the following:

Builds windows binaries
Adds entries in the registry fore firefox per this doc.
Copies the binaries / manifest to the proper location.

Something is wrong though. I am getting this error when clicking the padlock icon:
Error parsing native host manifest C:\Users\qbit\AppData\Local\browserpass\com.dannyvankooten.browserpass.json: JSON.parse: unexpected character at line 1 column 1 of the JSON data  NativeMessaging.jsm:104
This extension does not have permission to use native application com.dannyvankooten.browserpass (or the application is not installed)

There isn't much information beyond that - the AppData\Local path is being pulled from the registry, and it contains valid json:
PS C:\Users\qbit\go\src\github.com\dannyvankooten\browserpass> cat C:\Users\qbit\AppData\Local\browserpass\com.dannyvankooten.browserpass.json
{
  "name": "com.dannyvankooten.browserpass",
  "description": "Browserpass binary for the Firefox extension",
  "path": "C:\\Users\\qbit\\AppData\\Local\\browserpass\\browserpass-windows64.exe",
  "type": "stdio",
  "allowed_extensions": [
<EMAIL_ADDRESS>   ]
}
PS C:\Users\qbit\go\src\github.com\dannyvankooten\browserpass>

I have verified that the executable works.
I am not well versed in windows stuff - so I could be doing something totally stupid - if more experienced windows users have any input - I would love to hear it! I have tested with the in-store firefox extension as well as a built locally (below script pukes a zip into release that can be used as well) - both fail with the same invalid json error.
Here is the build script:
$app = 'com.dannyvankooten.browserpass'
$browsers = "firefox","chrome"
#$arches = "windows:64.exe","linux:64","darwin:x64","openbsd:64","freebsd:64","netbsd:64"
$arches = "windows:64.exe"
$releaseFiles = ("*-host.json", "install.*", "README.md", "LICENSE")

$dirpath = Join-Path -Path $env:localappdata -ChildPath 'browserpass'
$jsonpath = Join-Path -Path $dirpath -ChildPath "$app.json"

& cp chrome\host.json chrome-host.json
& cp firefox\host.json firefox-host.json
& cp chrome\policy.json chrome-policy.json

# build our binaries
foreach ($arch in $arches) {
	$parts = $arch.Split(':')
	$os = $parts[0]
	$name = $parts[1]

	Write-host  "Building browserpass for: $os$name"
	& env GOOS=$os GOARCH=amd64 go build -o browserpass-$os$name ./cmd/browserpass
}

$browserify = ".\node_modules\.bin\browserify"

& $browserify chrome/script.browserify.js -o chrome/script.js
& $browserify chrome/inject.browserify.js -o chrome/inject.js

& cp chrome\*.html firefox\
& cp chrome\*.css firefox\
& cp chrome\*.js firefox\
& cp chrome\*.png firefox\
& cp chrome\*.svg firefox\

new-item -type Directory -Path release -force
Compress-Archive -Path firefox/* -Update -DestinationPath release\firefox.zip
Compress-Archive -Path chrome/* -Update -DestinationPath release\chrome.zip

foreach ($arch in $arches) {
	$parts = $arch.Split(':')
	$os = $parts[0]
	$name = $parts[1]

	Write-host  "Building release for: $os$name"
	$rlist = $releaseFiles += "browserpass-$os$name"
	Compress-Archive -Path $rlist -Update -DestinationPath "release\browserpass-$os$name.zip"
}

# Make our local directory
new-item -type Directory -Path $dirpath -force

# copy our bin to local directory
& cp browserpass-windows64.exe $dirpath

# copy the native messaging manifest
$ffile = gc firefox-host.json
$ffile -replace '%%replace%%', ((Join-Path -Path $dirpath -ChildPath 'browserpass-windows64.exe' | ConvertTo-json) -replace '^"|"$', "") > $jsonpath

# Add our registry values
New-Item -Path "hkcu:\Software\Mozilla\NativeMessagingHosts\$app" -force
New-ItemProperty -Path "hkcu:\Software\Mozilla\NativeMessagingHosts\$app" -Name '(Default)' -Value $jsonpath -force

Hmm... JSON.parse: unexpected character at line 1 column 1 - since the JSON does look valid, I would first check that you have Windows line endings, and also that the file encoding is UTF-8 without BOM.
/cc @chrboe and @aventrax, you might be interested in this, and maybe you also have some ideas on how to get this approach to work?

Ha, it was totally a windows line endings thing! Sweet! I have functional browserpass on windows :D

Google says this should work Get-Content Unix.txt | Out-File Dos.txt 🙂
Will you make a PR once you get your script completely working? I know many people would love to have a support for Windows!

PR added, closing this one.


It is error if I add chinese dialogue

In Corpus/reddit_cleaned_part, It is error if I add chinese dialogue。
report error as follows:
InvalidArgumentError (see above for traceback): assertion failed: [All values in memory_sequence_length must greater than zero.]

你好，你怎么解决的


Failed ECS deployment feedback

Would it be possible to return specific feedback from ECS in the case that the task failed to launch (i.e. could not find image, etc)?
When running a service update that fails, the logs will return Service failed to deploy (image attached).

Having this directly in the Buildkite logs would be very handy as it may negate needing to go into the AWS console & dig around as to why the deployment failed.

@huewood that is quite an interesting idea... unfortunately, I don't think it would be possible for two reasons.
1- there are a lot of reasons why a deployment may fail. Off the top of my head I can think of: wrong image (or lack of permissions), healthchecks - both in the container or target group -, and rollback configurations in the deployment. So the places to look at are very environment-dependent and normally outside of ECS itself (for example, in my experience, you need to read task logs to debug healthcheck failures)
2- even if we had a complete list of things to look at, getting all that information will mean giving the agent a whole lot of permissions to be able to gather the information which may be a deal-breaker in a lot of situations


Calling getNode() on the ref of an Animated component is no longer necessary

Getting the following warning on RN 0.62.

FlatList: Calling getNode() on the ref of an Animated component is no longer necessary. You can now directly use the ref instead. This method will be removed in a future release.

You can use animatedComponent instead: https://github.com/iyegoroff/react-native-collapsible-header-views/blob/36ce3ca2e3938274acb3fa7641205ce6647e39a1/src/with-collapsible-header.tsx#L245-L247

Is there a code sample I can look at which describes the correct usage here? Not entirely sure by looking at the documentation how to implement this.

https://github.com/iyegoroff/react-native-collapsible-header-views/blob/master/CollapsibleHeaderViewsExample/screens/WrappedInstanceMethods.tsx

Apologies, but I'm not understanding based on that sample. My implementation looks like this:
<CollapsibleHeaderFlatList
    ref={scrollRef}
    data={hikes}
    extraData={hikes}
    CollapsibleHeaderComponent={<FeedHeader city={city} />}
    headerHeight={35}
    showsVerticalScrollIndicator={false}
    keyExtractor={(item) => item.key}
    renderItem={this.renderItem}
    refreshControl={refreshControl}
    onEndReached={onEndReached}
    ListFooterComponent={this.renderFooter}
    headerContainerBackgroundColor={theme.colors.rootBackground}
    disableHeaderSnap
    bounces
/>

I didn't get what do you want to achieve - just use animatedComponent method instead of getNode.

I'm not calling getNode() anywhere though, so I'm confused as to how I should implement animatedComponent(). Upgrading to RN 0.62 started throwing the warning whenever a  is loaded.

If you are not calling getNode, then it is not related to rnchv. getNode is not called from the module.

Oh you're totally right, this is a react-navigation issue. Thanks for your help and for the great package!


The notice about path_helper on OS X is incorrect

This:

If you have administrator privileges, you must fix an Apple-introduced problem in Mac OS X 10.5 Leopard by executing the following command, or BASH and Zsh will have the wrong PATH when executed non-interactively.
sudo chmod ugo-x /usr/libexec/path_helper

Is incorrect. Users shouldn't disable path_helper.
The problem is OS X is that it comes with /etc/zshenv that uses path_helper. However, /etc/zshenv gets sourced for every zsh shell, even non-interactive. To put a stop to that, simply:
sudo mv /etc/{zshenv,zprofile}

We are now in 2017, and running El Capitan, so this is probably not quite the right thread. But Googling 'prevent zprofile messing my path' lands me here and I can see that the comments sound relevant.
Basically, I set PATH in .zshenv, as zsh intended.  The fact that /etc/zprofile is sourced after .zshenv messes up my path: it calls path_helper -s, which sneakily reorders my PATH in such a way that /usr/local/lib and other system dirs are placed before the ones in which I carefully override certain commands.  For example, I use Conda to manage Python and its modules. On all my other systems, which python yields the desired $CONDA_HOME/bin/python. On this mac, it is /usr/bin/python, which is not at all what I want.
So, I'd say this path_helper thing is quite intrusive.

Anyway, for anybody who'll land here too, here is a workaround:
in .zshenv: before customizing PATH:
# MacOS workaround sneaky path_helper in /etc/zprofile
if [ -x /usr/libexec/path_helper ]; then
  eval `/usr/libexec/path_helper -s`
fi

At the end of .zshenv:
# MacOS workaround sneaky path_helper in /etc/zprofile
export GOODPATH=$PATH

Create a .zprofile (I don't usually have one):
# MacOS workaround sneaky path_helper in /etc/zprofile
export PATH=$GOODPATH

Now everything works as intended.

@pdemarti, if we consider unix shell initialization, then we may come to conclusion that /etc/zprofile is not called in some cases. At least in the case when we

Executing a command remotely with ssh

So, I suppose the correct solution would be not to include
if [ -x /usr/libexec/path_helper ]; then   eval /usr/libexec/path_helper -s fi
in .zshenv, but to save PATH at the end of .zshenv
export GOODPATH=$PATH, as you did.
Then, you just need to set PATH, such that it would be
PATH=$GOODPATH:<PATH excluding those paths that are already in GOODPATH>


Check for missing translations

Is there a tool to check if all requested translation keys in an application are defined in all translation files? I want to add this to our build pipeline so we don't miss any translations. It would also be great if unused keys in a translation file could be tracked, so we can cleanup unused translations.

I think you will have to use a custom function on
I18n.missingTranslation to report back missing translation keys
I don't think there is a tool for unused translations
There is one for Ruby but I am not sure how it works nor whether it's accurate:
https://github.com/glebm/i18n-tasks

While using the missingTranslation function is great for reporting back on runtime, I would rather find missing translations beforehand. I've looked at i18n-task but didn't really understand how use it with a javascript project.

I think that would be source code analyse level task and I doubt it's a simple task
Not to mention including such function in this library

It would indeed be a code analyse task. For bigger applications it's hard to track if a translation is missing for a certain language or all languages.


Tin Plate/RF Plug

Unless I missed something, none of the machines are craftable without another mod that adds ore dictionaried plates. The Rolling Machine like all other machines required RF plugs which are made with the small batteries which are made with the Tin Plates. But the small plate presser does not work on tin. Seems to be a bit of an issue in the progression here.

This is an oversight on my part, going to add a block so it can be processed in the small plate presser.  Scheduled for 0.3.2.


Is it possible to add a ros wrapper?

HI @jbehley , thanks a lot for sharing this great work! I tested it using KITTI. The visualization of point cloud  result is good.
Then I tried to write a ros warper for SuMa, so as to test it on real robots/cars with a 3D lidar. Just like what is used in the qt wrapper /visualizer/VisualizerWindow.cpp and implementation /visualizer/visualizer.cpp, I create a class SuMaRosWarper with a member of std::shared_ptr<SurfelMapping> fusion_,  then created a std::shared_ptr<SurfelMapping> pointer to initialize it. However, a segment fault was thrown in the construction of SurfelMapping. I backtraced the code and found the error occurred when called glGenBuffers(1, &id_);:

BUT I can run your qt version smoothly. What's wrong with this? any suggestion?

Since SuMa uses OpenGL for computation, you always need a opengl context in the same (!) thread. Updates and everything must also happen in this thread.
With ROS you have to take into account that nodes might not run on the same thread.
See for instance glow::X11OffscreenContext for a headless/windowless context.
But with this you still need a X server running. A true X free context you could get with EGL.

Thanks a lot!

Hi, @jbehley  Do you think it's possible to use ros rviz to create a opengl context for SuMa?

If you manage to run it in the same thread, then it could work. But I never
tried this. So no guarantees.

Am 27.09.2019 um 16:43 schrieb LiShuaixin<EMAIL_ADDRESS>Hi, @jbehley Do you think it's possible to use ros rviz to create a opengl context for SuMa?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@LiShuaixin Were you able to use rviz with suma?


Use Different Mirrors for CentOS

Motivation

We have integrated Driverkit in our infrastructure for building kernel modules that we use in an agent we develop. As input to Driverkit, we've started using Kernel Crawler to automatically discover kernels for us to build with Driverkit. However, we have network policies in our infrastructure that block domains originating from China.
Feature

The CentOS mirrors used in Kernel Crawler are currently hardcoded to China-based domains. Can we swap those to mirrors hosted elsewhere? I think an ideal future might be allowing users to pass in their own mirrors in the event of air-gapped networks...
Alternatives

The alternative would be to allow a user to pass in what mirror they would like to have Kernel Crawler use, but that change may be large for now.
Additional context

N/A

@EXONER4TED: There is not a label identifying the kind of this issue.
Please specify it either using /kind <group> or manually from the side menu.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/kind feature


Adds providential flag to support multiple GPUs UNW-175

Adds --gpus flag
No effect currently, will focus on backend implementation now

UNW-175

We already have this!

See https://github.com/unweave/cli/pull/13


metadata AbstractObj no longer implements Obj

AbstractObj is useful ancestor keeping most of metadata objects, including those which can't be considered
as first class citizens (they are nested into another object and it is not possible to manipulate them directly)
closes #188

Coverage remained the same at 81.49% when pulling ef70d42d43c0daa867c993ef6e9ba5a6e5283305 on jmi.obj into 9f05055d08d5b59a17495e8bd4650dc65993a671 on master.

Coverage remained the same at 81.49% when pulling b70fb17e6b9cc5731982672e774075588369e838 on jmi.obj into 9f05055d08d5b59a17495e8bd4650dc65993a671 on master.


Feature/user docs

I updated the README.md file with a getting started guide.
Closes #16

@MartinX3 I added a simple README. You can give feedback, if you want :-)

:tada: This PR is included in version 1.5.0-rc.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Feature/clean up cards

👋 Thanks for creating a pull request! 🚀 Checkout the storybook we created for it here.


More styling for the register page, pulling in new variables

What's this PR do?

More styling to the registration page so it nearly matches the mock
Pulling in the new query params

How should this be reviewed?

Checklist
- [ ] Documentation added for changed endpoints. (ref)
- [ ] Tests added for new features/bug fixes.
@DFurnes What's the best way to test the session stuff locally?

Ah, should probably remove the code related to the password confirmation as well, one sec

@DFurnes What's the best way to test the session stuff locally?

Make a test case! If you visit the /authorize endpoint with your additional query parameters, you should be able to follow the redirect and then assert the login page looks as expected.

@DFurnes Updated this PR so everything should be working & have test coverage, see the latest commit:
https://github.com/DoSomething/northstar/pull/613/commits/7c492cb96834cf8c6991d3001629504af5cf0583

@deadlybutter Can you update the documentation for the authorize endpoint with the new title, callToAction, and coverPhoto query parameters as well?

👍 👍

@DFurnes Hows that?


Update Terraform terraform-aws-modules/ec2-instance/aws to 4.1.1

This PR contains the following updates:

Package
Type
Update
Change

terraform-aws-modules/ec2-instance/aws (source)
module
patch
4.1.0 -> 4.1.1

Release Notes

terraform-aws-modules/terraform-aws-ec2-instance
v4.1.1
Compare Source

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, click this checkbox.

This PR has been generated by Renovate Bot.

@balena-ci I self-certify!


Add AWS ECS exporter to the list of exporters on docs

This AWS exporter takes metrics from ECS API

Thanks!
That's certainly one of the best tested exporters I've seen. Watch that you only use the _total suffix on counters, not gauges.

I've updated the exporter fixing those gauges. Thank you!


tm asset file sizes

pipeline-spread-geometries.json (very large) and existing-pipeline.json (large) are taking up alot of space in the bundle. Try adding a minimizing step with mapshaper.

fixed with 70998f5


Functionality to check if a language is supported or not.

santali changes are there in this PR. please remove it.
make sure always you create branch from the master. (unless you need some changes from another branch)


SA - Hi,  can you please add stacked Bar chart. Curretly, you are proving the m[...]

Sender<EMAIL_ADDRESS>Comment: Hi,
can you please add stacked Bar chart. Curretly, you are proving the multiple bars iwth different color. We need one bar should divided into the different colors.
Thanks & regards
Durga
Open #19 in Usersnap Dashboard

Download original image
Browser: Chrome 65 (Windows 10)
Referer: http://www.amexio.org/api/v4.1/index.html#/simple-grid
Screen size: 1920 x 1080 Browser size: 1858 x 1014
Powered by usersnap.com

Bar chart has stacked bar option, we will send small example of stack bar implementation to you.

Stacked Bar Chart example:http://amexio.org/api/v4.1/index.html#/bar-chart-demo


No helper coverage in simplecov 0.11.0

Hi, after update to simplecov 0.11.0 all my helpers are shown with 0% coverage, even so I have specs for them running. If I manually add a require 'my_helper' to my_helper_spec.rb, I get coverage information again.
Is this supposed to be handled that way now or is it a bug?

Actually after some investigation I found out, that coverage was not executed also in older versions due to some misconfiguration in my app (SimpleCov.start was called too late). So it actually helped me to find a bug. So not an issue.

The system works! cc @hugopeixoto