[Support] PiVPN+Wireguard Internet access doesn't work while on Pihole DNS

In raising this issue, I confirm the following:
{please fill the checkboxes, e.g: [X]}

[X] I have read and understood the contributors guide.
[X] The issue I am reporting can be replicated.
[X] The issue I am reporting can be is directly related to the pivpn installer script.
[X] The issue I am reporting isn't a duplicate (see FAQs, closed issues, and open issues).

Issue
I have a fresh install of PiVPN with Wireguard, on top of an also-freshly installed Pi-hole DNS server. At first, I thought that clients connected to the Wireguard server had no connectivity, but based on the FAQ/another issue, I loaded the VPN connection on my phone over mobile data AND changed the DNS server to <IP_ADDRESS>, which caused the VPN to work (it does not work without the edited DNS server).
Presumably there is some bad interaction happening with PiVPN and Pi-hole, but it's not clear to me where. Pi-hole seems to work fine in isolation, and PiVPN works when the client is totally off of my home LAN.
Have you searched for similar issues and solutions?
(yes/no / which issues?)

https://github.com/pivpn/pivpn/issues/1162
https://github.com/pivpn/pivpn/issues/1190
https://github.com/pivpn/pivpn/issues/1192 (tried setting Pi-hole DNS to listen on eth0 only, didn't help)
https://github.com/pivpn/pivpn/issues/1195
Console output of      curl -L install.pivpn.dev | bash
::: Update option selected.
::: The updating functionality for PiVPN scripts is temporarily disabled
::: To keep the VPN (and the system) up to date, use 'apt update' and 'apt upgrade'

Console output of      pivpn add or pivpn add nopass
::: Client Keys generated
::: Client config generated
::: Updated server config
::: Updated hosts file for Pi-hole
::: WireGuard reloaded
======================================================================
::: Done! foo.conf successfully created!
::: foo.conf was copied to /home/pi/configs for easy transfer.
::: Please use this profile only on one device and create additional
::: profiles for other devices. You can also use pivpn -qr
::: to generate a QR Code you can scan with the mobile app.
======================================================================

Console output of      pivpn debug
::: Generating Debug Output
::::		PiVPN debug		::::
=============================================
::::		Latest commit		::::
commit 13f0fe7cbdcdb31537b3fd0e2eb34652e886cc1b
Author: 4s3ti<EMAIL_ADDRESS>Date:   Wed Dec 9 19:22:29 2020 +0100

    ProBot Stale

    Added probot integration to marke topics as inactives and automatically
    close them.
    read .github/stale.yml for more details.
=============================================
::::	   Installation settings    	::::
PLAT=Raspbian
OSCN=buster
USING_UFW=0
IPv4dev=eth0
dhcpReserv=
IPv4addr=<IP_ADDRESS>/24
IPv4gw=<IP_ADDRESS>
install_user=pi
install_home=/home/pi
VPN=wireguard
pivpnPORT=51820
pivpnDNS1=<IP_ADDRESS>
pivpnDNS2=
pivpnHOST=REDACTED
INPUT_CHAIN_EDITED=0
FORWARD_CHAIN_EDITED=0
pivpnPROTO=udp
pivpnDEV=wg0
pivpnNET=<IP_ADDRESS>
subnetClass=24
UNATTUPG=1
INSTALLED_PACKAGES=()
=============================================
::::  Server configuration shown below   ::::
[Interface]
PrivateKey = server_priv
Address = <IP_ADDRESS>/24
ListenPort = 51820
### begin nixpad ###
[Peer]
PublicKey = nixpad_pub
PresharedKey = nixpad_psk
AllowedIPs = <IP_ADDRESS>/32
### end nixpad ###
### begin mat-iphone ###
[Peer]
PublicKey = mat-iphone_pub
PresharedKey = mat-iphone_psk
AllowedIPs = <IP_ADDRESS>/32
### end mat-iphone ###
### begin foo ###
[Peer]
PublicKey = foo_pub
PresharedKey = foo_psk
AllowedIPs = <IP_ADDRESS>/32
### end foo ###
=============================================
::::  Client configuration shown below   ::::
[Interface]
PrivateKey = nixpad_priv
Address = <IP_ADDRESS>/24
DNS = <IP_ADDRESS>

[Peer]
PublicKey = server_pub
PresharedKey = nixpad_psk
Endpoint = REDACTED:51820
AllowedIPs = <IP_ADDRESS>/0, ::0/0
=============================================
:::: 	Recursive list of files in	::::
::::[4m/etc/wireguard shown below	::::
/etc/wireguard:
configs
keys
wg0.conf

/etc/wireguard/configs:
clients.txt
foo.conf
mat-iphone.conf
nixpad.conf

/etc/wireguard/keys:
foo_priv
foo_psk
foo_pub
mat-iphone_priv
mat-iphone_psk
mat-iphone_pub
nixpad_priv
nixpad_psk
nixpad_pub
server_priv
server_pub
=============================================
::::		Self check		::::
:: [OK] IP forwarding is enabled
:: [OK] Iptables MASQUERADE rule set
:: [OK] WireGuard is running
:: [OK] WireGuard is enabled (it will automatically start on reboot)
:: [OK] WireGuard is listening on port 51820/udp
=============================================
:::: Having trouble connecting? Take a look at the FAQ:
:::: https://github.com/pivpn/pivpn/wiki/FAQ
=============================================
:::: WARNING: This script should have automatically masked sensitive       ::::
:::: information, however, still make sure that PrivateKey, PublicKey      ::::
:::: and PresharedKey are masked before reporting an issue. An example key ::::
:::: that you should NOT see in this log looks like this:                  ::::
:::: YIAoJVsdIeyvXfGGDDadHh6AxsMRymZTnnzZoAb9cxRe                          ::::
=============================================
::::		Debug complete		::::
:::
::: Debug output completed above.
::: Copy saved to /tmp/debug.log
:::

Have you taken any steps towards solving your issue?
tcpdump output:
root@raspberrypi:/home/pi# tcpdump -n -i eth0 udp port 51820
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:21:26.243211 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:26.243302 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:26.243351 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 148
09:21:26.247161 IP <IP_ADDRESS>.51820 > NNN.NNN.N.NNN.55582: UDP, length 92
09:21:26.299773 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 32
09:21:28.127688 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:28.127791 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:31.663584 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:31.664506 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:31.664577 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:31.664616 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:31.664659 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:31.664695 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:31.665040 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:31.665086 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:32.147132 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:32.151099 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:33.755726 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:33.755822 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:33.755863 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:33.755900 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:33.755936 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:33.759598 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:33.759671 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:33.759710 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:35.098313 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:35.137306 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.206470 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.211437 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.813448 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.817155 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.817221 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.817258 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.830122 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:37.830186 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:37.830226 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:37.830262 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:38.332967 IP <IP_ADDRESS>.51820 > NNN.NNN.N.NNN.55582: UDP, length 32
09:21:40.287544 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:40.287637 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:41.396678 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:41.396770 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:43.147451 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:43.148255 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:45.203591 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:45.203696 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:46.175396 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:46.175502 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:46.176151 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:46.176210 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:46.176248 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:46.176284 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:46.179570 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:46.179640 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:49.347726 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:49.347810 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:49.688036 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:49.688114 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:21:50.493039 IP <IP_ADDRESS>.51820 > NNN.NNN.N.NNN.55582: UDP, length 32
09:21:56.389398 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:56.389509 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:57.567752 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:21:57.568575 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:22:05.755012 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:22:05.757489 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 112
09:22:06.492994 IP <IP_ADDRESS>.51820 > NNN.NNN.N.NNN.55582: UDP, length 32
09:22:14.238450 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:22:14.238571 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:22:24.413027 IP <IP_ADDRESS>.51820 > NNN.NNN.N.NNN.55582: UDP, length 32
09:22:29.611402 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:22:29.611518 IP NNN.NNN.N.NNN.55582 > <IP_ADDRESS>.51820: UDP, length 96
09:22:39.772984 IP <IP_ADDRESS>.51820 > NNN.NNN.N.NNN.55582: UDP, length 32
^C
72 packets captured
72 packets received by filter
0 packets dropped by kernel
root@raspberrypi:/home/pi# exit
pi@raspberrypi:~ $ sudo iptables -vnL
Chain INPUT (policy ACCEPT 27765 packets, 3014K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 23961 packets, 3959K bytes)
 pkts bytes target     prot opt in     out     source               destination
pi@raspberrypi:~ $ sudo iptables -t nat -vnL
Chain PREROUTING (policy ACCEPT 3121 packets, 287K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 3018 packets, 239K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 3299 packets, 233K bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  all  --  *      eth0    <IP_ADDRESS>/24          <IP_ADDRESS>/0            /* wireguard-nat-rule */

Chain OUTPUT (policy ACCEPT 3299 packets, 233K bytes)
 pkts bytes target     prot opt in     out     source               destination

Unsure if there are recommendations particular to this situation. I tried pointing the DNS of the client VPN profile to <IP_ADDRESS>, but it seems like having the Pi-hole managing DNS for my LAN means no local traffic will work on the VPN.

and iptables:
iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT

iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P POSTROUTING ACCEPT
-P OUTPUT ACCEPT
-A POSTROUTING -s <IP_ADDRESS>/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE

I have the same set up working, using the test branch.  My config seems very close to yours, except for the local network addresses.  Did you select the pi-hole DNS when you set up the pivpn?

@shelleycat485 yes, I did tell PiVPN’s installer to use the Pi-hole DNS. I wonder if there could be something off about my Pi-hole or router settings that could be interfering

Have a look: https://github.com/pivpn/pivpn/issues/1192#issuecomment-740537159

@orazioedoardo Hmm, I tried running pihole restartdns both before and after starting the VPN connection, but still no dice. Again, it works when the phone DNS is set to <IP_ADDRESS>, but not to either the PiVPN-local address (10.x.x.x) or the LAN-local address (192.x.x.x). Which Pihole log would I see Ignoring query from non-local network in? I'm suspicious my issue is different, but I want to verify.

The log is inside /var/log/pihole.log. By the way do you have pihole set to “listen on all interfaces” in the pihole dashboard?

@orazioedoardo presently it is set to "listen only on eth0", but I will do a quick double check on the other settings to make sure that doesn't change anything. Toggling that setting has been one of the steps I've followed to try and resolve this. None of the logs in /var/log mention the Ignoring query from non-local network bit, either.

It should be listen on all interfaces

@orazioedoardo is that the unconditional “listen on all interfaces” which has a warning about keeping your Pi behind a router? or the one which listens on all interfaces to devices one hop away?

listens on all interfaces to devices one hop away Is enough (which is what pivpn should do when choosing to use pihole by the way) has

Confirmed that even on that Pi-hole setting, I experience the same issue. I tried restarting both the DNS server and the entire Pi after saving the DNS settings.
Name            Remote IP                Virtual IP      Bytes Received      Bytes Sent      Last Seen
nixpad          XX.XXX.XXX.XX:XXXXX      <IP_ADDRESS>        444B                276B            (not yet)

Curiously, when I connect to the VPN and check pivpn clients, despite not having any internet access, I still get some "bytes received" and "bytes sent", but the VPN has apparently not "seen" the client.

I still get some "bytes received" and "bytes sent", but the VPN has apparently not "seen" the client.

That's very weird, what does sudo wg show wg0 say? Do you have latest handshake in there? Do you see any DNS requests from <IP_ADDRESS> in the Pi-hole log at /var/log/pihole.log?

@orazioedoardo the wg show output gives the same byte counts. I was informed this is most likely the VPN handshake, which actually makes sense because the counts change a small amount every time I dis/reconnect the VPN client connection. However, the wg show output does NOT say latest handshake:
interface: wg0
  public key: server-pub
  private key: (hidden)
  listening port: 51820

peer: client-pub
  preshared key: (hidden)
  endpoint: xx.xxx.xxx.xx:ppppp
  allowed ips: <IP_ADDRESS>/32
  transfer: 888 B received, 552 B

As far as I can tell, the only mentions of 10.6.0.anything in /var/log/** is related to the setup of the VPN service:
...
/var/log/syslog:Dec 24 03:37:36 raspberrypi wg-quick[438]: [#] ip -4 address add <IP_ADDRESS>/24 dev wg0
...
/var/log/daemon.log:Dec 24 03:37:36 raspberrypi wg-quick[438]: [#] ip -4 address add <IP_ADDRESS>/24 dev wg0
...

Nothing in any of the pihole related logs.

There should a "latest hadnshake" field. It's like the handshake is failing. Can you inspect the log in the wireguard mobile app for any clue?

No replies in a while, closing...

No replies in a while, closing...

I think #1195 is related to this.
Looks like no one solved the problem. I fell back on slower OpenVpn until any further solution.

I think #1195 is related to this.
Looks like no one solved the problem. I fell back on slower OpenVpn until any further solution.


mjml2txt

When writing a newsletter one often embed both HTML and text format in the email.
It would be great if mjml could also generate the plain text part from the same markup using a mjml2txt() function

I would also like a feature like you described.
What i've tried so far:
npm install -g html-to-text
then bash
cat emailmarketing.html | html-to-text > emailmarketing.txt
Does funny things with image src links and line breaks on a mjml generated template.
Thanks

The only immediate solution we can see is indeed to rely on html-to-text, which as @dominiquedutra said does funny things from a HTML template so is not an ideal solution.
It's also probably not the role of MJML to generate the text from a HTML template as there are libs for that. If you see an efficient way to do it though, feel free to reopen!


Add ability to manually bypass failed tasks

 Issue by thieman
Friday Jul 19, 2013 at 15:35 GMT
Originally opened as https://github.com/thieman/dagobah/issues/22

Use case: I just had a task fail, so I went into IPython and screwed around in the interpreter until I fixed the bug (which occured half-way through a processing task) and let the task complete in my interpreter. Now, I want Dagobah to continue the rest of the failed job without having to run the entire task that I just completed manually.

 Comment by rclough
Wednesday May 07, 2014 at 20:18 GMT

This would be a cool feature, here's 2 ideas I have for it:

A "start from step " option that starts any workflow from a given step
A "mark step as completed" option which might be cleaner. Ie after manual intervention, you mark the step as successful and the dependency chain continues as normal.

1 has additional use cases beyond failures, ie you have bad data from one point on so there's no explicit failure, but you dont want to run the whole job again. 2 is probably the simplest fix simply for manually correcting errors like in your use case.

 Comment by thieman
Wednesday May 07, 2014 at 21:50 GMT

I think we'd want to do a slight variant on 2. Instead of marking as completed, we should add a fourth task state representing a skipped job. Job states currently include waiting, running, and failed. These states are used to determine what actions are possible at any given time: https://github.com/thieman/dagobah/blob/master/dagobah/core/components.py#L49
I think the clarity of adding a fourth state would be worth the extra effort. It would be weird to get a job completed email that told you Task 7 was completed when in fact you skipped it.
You should be able to manually mark a task as skipped in the UI and then retry the job. Skipped tasks should be counted as "completed" for the purposes of graph traversal and job completion.


Increased timeout for SignatureTests

Increased timeout for SignatureTests. There has been reports of timeouts on some platforms, based on stack trace, test is not stuck on blocking call (such as reading from /dev/random), but is doing some BigInteger arithmetic (BigInteger arithmetic being slow on some platforms?). See:
https://github.com/adoptium/aqa-tests/issues/4336#issuecomment-1512952403

Probably caused by absence of intrinsics for some platforms, see:
https://github.com/adoptium/aqa-tests/issues/4336#issuecomment-1540956287

Maske sense.I can see it in our infra from time to tim on ppc64le:
timed out (timeout set to 120000ms, elapsed time including timeout handling was 120543ms).
480>>120 => ok  to merge. ty!

thanks


Adding Samigo RESTful APIs

We need to add supports for samigo RESTful APIs so that data could be retrieved and migrated.

Here are the APIs I believe will be sufficient for migration.
The APIs will help to get

a list of all tests (published and unpublished) for a site
info for a specific test (published and unpublished)
export test data in qti format for a test
need super admin permit (mostly)

77f3dcba24cd0b227c
8b4db2da581daec62

Here are the ones that we added and should be working now for retrieving data with basic security check:
direct/sam_unpub/site/<site id>.json

/direct/sam_unpub/<assessmentId>.json

/direct/sam_pub/site/<site id>.json

/direct/sam_pub/<publishedAssessmentId>.json

/direct/sam_export_qti/<assessmentId>.json

Done. Deployed.


Union type restricting to last assigned value in a hash

Not sure if this is supposed to happen (and I can't have union typed generic classes in hashes)
class Blah(T)
  def add(v: T)
  end
end

blahs = {} of Int32 => Blah(String | Int32)
blahs[0] = Blah(String).new
blahs[1] = Blah(Int32).new

blahs[0].add("Hello")
blahs[1].add(0)

Error in line 10: no overload matches 'Blah(Int32)#add' with types String
Overloads are:
 - Blah(Int32)#add(v : Int32)

example on play

Well, there are some things going.

You know that the blah at 0 is of String. But the type inference can't distinguish between blah at 0 and blah at 1. They need to be have assigned the same static type (at least a union).

Actually I would expect that to compile. I wouldn't expect that Foo(T) <: Foo(T | U) for any generic class Foo. That is a Foo(T) can be used as a Foo(T | U). @asterite is there any logic for this right now for the hashes values? Because http://play.crystal-lang.org/#/r/6ba shows a reduced example that does not work.

http://play.crystal-lang.org/#/r/6az declares the hash as Int32 => Blah(String) | Blah(Int32) which seems proper, but yet suffers from the same limitation detailed in 1).
What is the use case for a hash where the values might respond to one overload or another? There might be another way to code that?

Thankyou for the response!
My use-case is a bit convoluted, I'm working on an event system based on Xorg So I create a set of handlers using the Xorg event constants as keys the actual code here.
with the macros its a bit hard to follow but essentially unwraps to
@handlers[Constants::CreateNotify] = EventHandler(LibX11Events::XCreateWindowEvent).new(->(ev: Lib
X11Events::XEvent) { ev.xcreatewindow })
@handlers[Constants::UnmapNotify] = EventHandler(LibX11Events::XUnmapEvent).new(->(ev: LibX11Events::XEvent) { ev.xunmap })

The event object the Xlib uses is a C union and has a differing object based on the event fired. The initial plan was to register the set of handlers in the hash with a (generic)event class and a "retriever" block to pull the right object out of the union. (Not sure how much sense this makes!)
Essentially
WindowCreated = 0
KeyPressed = 1

@handler[WindowCreated] = MyEvent(NewWindow).new { |native_event| native_event.window}
@handler[KeyPressed] = MyEvent(NewKeyPress).new ( |native_event| native_event.keypressed }

@handler[WindowCreated].add -> (window: NewWindow) {
   puts "Woo a window!"
}

.. event loop ..
  @handler[event.type].fire(event.native_event_union)

I guess I should rethink my approach :)

If you are dealing with external libs with it's own way to handle unions/dispatch you will most likely reach a point where a cast will be needed. It will be a bit of code that might explode in runtime instead of compile time.
As long as those casts do not reach the user every time I thinks is fine.
There might need to be hidden cast to handle the callback if you want to store them all together.
If all the events type are known, you might split them by types in different ivars and have a proper type.
Or by usage of macros you can wrap the @handler[event.type] in a way that returns casted versions of events based on the event type arg used at the handler registration.
(I'm just thinking out loud...)

Ahh having separate ivars sounds like the 'correct' way just harder to manage, the macro/casted versions approach sounds a lot nicer to deal with!
thank you for your help

@bcardiff Maybe that shouldn't compile. Well, it doesn't compile, but you get an error in a different place. I tried to do a trivial thing to fix it but it seems it's not that simple.
@aktowns I think for this case, at least for now, you would need to store the handlers as a Hash(Int32, {Void*, Void*}), where {Void*, Void*} is the representation of a proc. Then you would define on_... handlers that accept blocks with restrictions (based on the event type). Then when invoking a handler you'd do handler = @handlers[...]; Proc(ParticularEventObject).new(*handler) to recreate the proc, but there you will have to have a big case for every event type and recreate the correct Proc type.
Eventually you'd be able to just say Hash(Int32, Proc) and that would mean the above, but you'd still have to do the casts.
(but this is just one solution I can think right now)

@aktowns so is the issue resolved for you? @asterite do you want to open a new issue about the thing that shouldn't compile?

I'll close this. Most of the issue is related to variance and covariance: #1297.


Route handler return value

Where in the source code does choo handle the return value of route handler functions (the hyperx html)?  I see where where the router sets the handler, but I don't see where choo takes that return value and renders it.

Its in Choo#_prerender, right here: https://github.com/choojs/choo/blob/master/index.js#L259

Thanks.  FWIW, I think the router auto-rendering is an anti-pattern.  Everywhere else, one does the rendering manually (see beginner's guide).  By returning htmlx in route functions, there is no direct way to attach logic to elements after they are appended.  I use MutationObserver to listen for renders, and then make a callback out of the app emitter in my templates.  Am I doing it wrong?

Having automatic rendering on navigation I think is very much the responsibility of a framework such as choo. For reacting to updates and routing on the component level, you'd probably want to use nanocomponent (which comes with choo). It has lifecycle hooks for load/unload behavior among some other niceties. See the documentation here: https://github.com/choojs/choo#components


fix: SecurityException's HTTP status code

Description
Follow-up  #6228

fix SecurityException's HTTP status code

How to Test
diff --git a/app/Config/Filters.php b/app/Config/Filters.php
index 7b70c4fb3..06e0b9941 100644
--- a/app/Config/Filters.php
+++ b/app/Config/Filters.php
@@ -30,7 +30,7 @@ class Filters extends BaseConfig
     public array $globals = [
         'before' => [
             // 'honeypot',
-            // 'csrf',
+            'csrf',
             // 'invalidchars',
         ],
         'after' => [
diff --git a/app/Config/Routes.php b/app/Config/Routes.php
index 71b7a86de..b41442a41 100644
--- a/app/Config/Routes.php
+++ b/app/Config/Routes.php
@@ -30,6 +30,7 @@ $routes->set404Override();
 // We get a performance increase by specifying the default
 // route since we don't have to scan directories.
 $routes->get('/', 'Home::index');
+$routes->post('/', 'Home::index');

 /*
  * --------------------------------------------------------------------
diff --git a/app/Config/Security.php b/app/Config/Security.php
index 1474404be..c4cdf9b0e 100644
--- a/app/Config/Security.php
+++ b/app/Config/Security.php
@@ -80,7 +80,7 @@ class Security extends BaseConfig
      *
      * Redirect to previous page with error on failure.
      */
-    public bool $redirect = true;
+    public bool $redirect = false;

     /**
      * --------------------------------------------------------------------------
diff --git a/app/Controllers/Home.php b/app/Controllers/Home.php
index 7f867e95f..dd8f1ea72 100644
--- a/app/Controllers/Home.php
+++ b/app/Controllers/Home.php
@@ -6,6 +6,6 @@ class Home extends BaseController
 {
     public function index()
     {
-        return view('welcome_message');
+        return '<form method="post" action="/"><input type="submit" value="Submit"></form>';
     }
 }

Before:

After:

Checklist:

[x] Securely signed commits
[ ] Component(s) with PHPDoc blocks, only if necessary or adds value
[ ] Unit testing, with >80% coverage
[ ] User guide updated
[x] Conforms to style guide

Yes, I forgot to implement HTTPExceptionInterface, so the HTTP status code was 500, not 403.


fix: sheet content vertical overflow

Same as https://github.com/sek-consulting/solid-ui/pull/109, but for Sheet component which suffers from exactly the same problem. Description follows:
If the sheet content grew too large, the portion overflowing the viewport would be unreachable as there was no way to scroll to it. This can easily happen on mobile with keyboard open.
The solution I chose is to limit the height to viewport height and let the browser handle the overflow by showing a scrollbar.

Thanks!


Keep connection status in sync after disconnect

Once the disconnect command is called, this allows calling the
constructor again to connect to a different database or refresh the
connection of a long running process.
In particular I noticed this problem when using the database connection with a dependency injector and supervisor as a long running process.

Hi David, thanks for contributing.
Calling a constructor multiple times sounds like a recipe for disaster, but it makes sense to clear $connected property after disconnection so I'm happy to merge this, thanks :+1:

Thanks for accepting my pull request. I agree it sounds like disaster to call the constructor multiple times but I need some way to change databases depending on parameters sent to the supervisor job. I didn't want to use the select_db as this binds the solution to only mysql. Perhaps adding a method to alter the connection information would be safer... adding a requirement that the method can only be used when the connection is disconnected.


Added check context manager

Addded the pybryt.check context manager for checking against reference implementations in assignment notebooks.
Closes #38

Codecov Report

Merging #49 (b18b536) into main (41b2a48) will increase coverage by 0.35%.
The diff coverage is 99.23%.

@@            Coverage Diff             @@
##             main      #49      +/-   ##
==========================================
+ Coverage   94.81%   95.16%   +0.35%
==========================================
  Files          21       22       +1
  Lines        1079     1179     +100
==========================================
+ Hits         1023     1122      +99
- Misses         56       57       +1

Impacted Files
Coverage Δ

pybryt/annotations/relation.py
100.00% <ø> (ø)

pybryt/complexities.py
100.00% <ø> (ø)

pybryt/reference.py
99.24% <97.91%> (-0.76%)
:arrow_down:

pybryt/__init__.py
100.00% <100.00%> (ø)

pybryt/execution/__init__.py
100.00% <100.00%> (ø)

pybryt/execution/tracing.py
100.00% <100.00%> (ø)

pybryt/integrations/otter.py
75.80% <100.00%> (+0.19%)
:arrow_up:

pybryt/invariants.py
100.00% <100.00%> (ø)

pybryt/student.py
98.07% <100.00%> (+2.24%)
:arrow_up:

pybryt/utils.py
100.00% <100.00%> (ø)

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41b2a48...b18b536. Read the comment docs.


Not being able to set the into argument of @separate

Suppose I have a DataFrame column named :embeddings. Each row in this column contains an array of floats. Since the function unnest_wider() isn't available yet, I'm considering an alternative approach. Since, unnest_wider() is not available yet, I thought I could first concatenate the array elements into a single string. After that, I planned to use the @separate macro to split the string, effectively emulating the unnest_wider() function.
However, I've encountered an issue. When I try to specify the into argument with new_names, I receive an error: "ERROR: LoadError: MethodError: no method matching iterate(::QuoteNode)". I've reviewed the macro's source code and didn't find any apparent issues. Interestingly, if I omit the new_names argument (similar to R's separate() function where it auto-generates column names), I get another error: "ERROR: LoadError: MethodError: no method matching var" @separate"(::LineNumberNode, ::Module, ::Symbol, ::Symbol, ::String)".
new_names = ["x$(i)" for i in 1:1536];
println("sd")
horror_movies_tidier_ =
    @chain horror_movies_tidier begin
        transform(_, :embeddings => ByRow(x -> join(x, "_")) => :embeddings)
    end
@chain horror_movies_tidier_ begin
    @separate(embeddings, new_names, "_")
end

We'll take a look. I think what's happening here is that @separate() is assuming that new_names refers to the actual name of the column to be created rather than a container of column names.
Normally, we would handle this with interpolation but @separate(), as currently implemented, doesn't support interpolation yet.
This should be fixable. Stay tuned.

Looking forward to the fix!

I think I have fixed this. I'll put in the PR soon.

Fixed in #40 and merged.
From your example, the following syntax should work now:
@chain horror_movies_tidier_ begin
    @separate(embeddings, !!new_names, "_")
end

Prefixing a !! to new_names tells TidierData.jl to look in the parent environment for new_names because the default assumption in TidierData.jl is that all "bare" symbols refer to column names.

Thanks to @drizk1 for all the hard work on this.


Stuck on last command

I entered this command:
mi-fastboot oem-unlock "UNLOCK_TOKEN"
and stuck, nothing happens

Previously I had requested to get an unlock token and I had to wait 6 days.
After 6 days I opened Termux:
cd termux-miunlock
./get_token.sh --region=global --product=PRODUCT --token=TOKEN DATA
and get the unlock token, then I used the last command and got stuck like in thn screenshot

Try mi-fastboot stage token.bin && mi-fastboot oem unlock

Try echo "UNLOCK_TOKEN" | xxd -r -p > token.bin then mi-fastboot stage token.bin && mi-fastboot oem unlock

I've tried it and it's still stuck


Update to current upstream

Hey! Many thanks for your efforts maintaining this project, really appreciated!
Is there a chance that this project will update to the current upstream?
BR

Hey! Any chance you could give an update on this topic?

Hi @bbiallowons! Sorry for the late reply. I became a father recently and have been adjusting to a new set of priorities! I have time now and will get right on it. Will link the PR here when it's ready.

PR is here. Currently a WIP as some tests are failing.

PR merged. Will release the package in the next couple of days.

The most recent release has been updated to the current upstream, thanks!

Sry for the late reply, thank you!


Building messages

Description
Huge node modules and puppeteer download when building. Also there is no index.js in the resulting dist.zip archive.
Expected behavior
dist.zip to be small
Actual behavior
dist.zip weighs about 6.5mo
Steps to reproduce the bug

make build

Package versions

terraform-aws-cognito-auth: latest

System information

OS: MacOS

You’re free to PR to improve on bundle size 😊 currently short on time.

Sorry, I might have been unclear, I was merely surprised the zip provided in the repo is ~150k, whereas mine goes up to 6.5mo. Any idea why? The node_modules I have is much larger than yours.

I can confirm that the size of dist.zip is now 5.6mb. However, I do not have the time to debug why. When I run ncdu on the zip, it seems like core-js and some other unnecessary dependencies have been pulled in. The direct dependencies haven't changed:
  "dependencies": {
    "mime": "^2.4.4",
    "mimemessage": "^1.0.5",
    "mustache": "^4.0.1",
    "quoted-printable": "^1.0.1"
  }

For this reason, I assume that one of the transitive dependencies got updated, which leads to an increase in size. Happy to merge PRs.

I will have a look and PR if I find something ;)


Add a disable check updates option

I think that an option to disable update checking should be added, this is useful for mods that use old versions of kade engine or mods that use a pre-compiled version of kade engine.

I think that an option to disable update checking should be added, this is useful for mods that use old versions of kade engine or mods that use a pre-compiled version of kade engine.

they just delete the outdatedsubstate file lmao

or remove the singular line of code that switches the state lmao


Remove Individual Data Type Caps in Per-shard Buffering for Remote Write

This PR consolidates the sampleBuffer and exemplarBuffer buffers into the pendingData buffer, removing the individual cap for each respective data type in favor of having a cap for all data points combined.
Fixes #8788

Thanks for this. @cstyan would you have some time to review it?
@LeviHarrison did you run any benchmarks on this?

Not yet, I probably should.

I should be able to review next week, end of this week is a holiday here. Benchmarks with memory stats would be good :)

I set up a benchmark using Grafana's TNS demo, so plenty of exemplars. I took this screenshot after it had been running for about 10 minutes.

After about 20 minutes:

Speaking of benchmarking... https://github.com/prometheus/test-infra/issues/249

// TODO(cstyan): Casting this many times smells, also we could get index out of bounds issues here.
I removed this comment from where maxExemplars was assigned, is it still valid?

@cstyan I'll take a look at expanding the benchmarks. I think that will probably end up as a separate PR thought.

We could also go the approach of:
var reset func()
if !s.qm.sendExemplars {
  reset = func() {
    pendingData[nPending].Samples = pendingData[nPending].Samples[:0]
  }
} else {
  reset = func() {
    pendingData[nPending].Samples = pendingData[nPending].Samples[:0]
    pendingData[nPending].Exemplars = pendingData[nPending].Exemplars[:0]
  }
}

might be cleaner, maybe even better for performance, IDK

:+1: lgtm

Thanks for all your time reviewing!


Make sure solc is present to test packages

Evidence this works: https://office.runtimeverification.com/jenkins/blue/organizations/jenkins/KEVM/detail/PR-1176/1/pipeline


SourceMap Support

Sourcemaps are currently not supported meaning that stack traces will be from transpiled JS sources. This is not ideal for debugging purposes.
The parcel bundler has implemented support for source maps in a rust package. Its not available on crates.io but the logic of deserializing a sourcemap is not very complicated.
https://github.com/parcel-bundler/source-map/tree/master/parcel_sourcemap
Here is explanation of sourcemaps:
https://www.bugsnag.com/blog/source-maps/
If this could be implemented without pulling in serde json dependency it would be ideal since the QuickJS engine already has a build in json parser

Can't sourcemaps be handled at the bundler level? What does llrt need to do?

It can! It would just be convenient. But ideally, source maps should stay of the deployment artifacts in Lambda as they add size that needs to pulled in to the execution environment, which adds time. Users can get the original source by using offline approach instead like https://www.npmjs.com/package/source-map-cli


fix sso clog config

Coverage decreased (-0.05%) to 48.481% when pulling 09b5a0e6cf235052b4ab6f473bcce63e8ed4dca7 on nobodyiam:fix-sso-clog into 8fb16b6aa026dc895733f50348e72fb096045003 on ctripcorp:master.


Audit for load fault and azure faults

Audit for load fault and azure faults

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65b20229d280792b8f0c9279--harness-developer.netlify.app


add generic item slot

Type

What types of changes does your code introduce?
incorporate a generic slot for item

Put an x in the boxes that apply_

[ ] Fix
[x] Feature

Checklist

Put an x in the boxes that apply._

[x] Title as described
[ ] Add unit test by vitest if necessary

Thank you for this suggestion. As mentioned in #254, I would like to see something like this in this project. But I think the corresponding column name of the item should be passed as well.

Thank you for this suggestion. As mentioned in #254, I would like to see something like this in this project. But I think the corresponding column name of the item should be passed as well.

In this way, if I have a column with the 'general' id, it will be applied to all columns.
<template #item-general>
...some code...

I consider that it is better that the item be left without the -general.
Thanks,
Juan


Name in Visual Studio does not match the name in Docs

Name in Visual Studio does not match the name in Docs
in Visual Studio is "Randomized base address" and not "use address space layout redomization"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 777b381a-baf8-f9a9-970e-5fef4157f79d
Version Independent ID: 392e117d-61e3-b17e-36cc-8fe738ef3e42
Content: /DYNAMICBASE (Use address space layout randomization)
Content Source: docs/build/reference/dynamicbase-use-address-space-layout-randomization.md
Product: visual-cpp
Technology: cpp-tools
GitHub Login: @corob-msft
Microsoft Alias: corob

@orenweil
The phrase in parentheses is descriptive, not the name of the project property in the IDE. You'll find that's true for almost all of the compiler and linker options. We identify the name of the property used to set this option in Visual Studio down in the To set this linker option in Visual Studio section. That's why I'm disinclined to change the description here. But I do appreciate that you've taken the time to make the suggestion. Thanks!


[add] form_db spec

Coverage decreased (-4.6%) to 79.97% when pulling 7f64f5272f0ed64c1c85810d6e7239b450b4fe22 on pr-form-db-spec into 43d1dbc367d1de1ecd8ab43b5caef558e75e7bf3 on master.


Missing qt package under Linux

The current version of OpenIGTLinkIO doesn't build under Linux when GUI support in enabled. More specifically, CMake fails to generate the makefile because the target Qt5::X11Extras is not found.
Qt5::X11Extras is a dependency of VTK when VTK is built with Qt GUI support. For some reason, VTK lists the package as a private dependency (see the code in vtk repos here ), which means you need to make sure the target Qt5::X11Extras is explicitly defined in any projects that uses this module of VTK.
What I don't understand is that the Qt5::X11Extras target needs to be defined in the base directory of OpenIGTLinkIO (and not just the GUI directory) otherwise cmake produces an error when generating the makefile. This pull request makes sure the target is defined in the root directory and I added a comment to make it obvious it is a workaround. The cleaner solution I think would be for VTK to define the package as a public dependency.

Any thoughts about this PR. Should I just merge it?

Hmm, what is the error when Qt5::X11Extras is only defined in the GUI directory?

I do not get a configure error looking for x11extras.
Using: Ubuntu 16.04, qt5.5.1, vtk 9.0.0
Testing now with Slicer's vtk

After enabling IGTLIO_GUI in Slicer extension nightly build:
No configure error with Ubuntu 16.04, Slicer nightly's VTK, qt5.9.
Build error though!

Can you post your configure log with the error and your CMakeCache.txt?

This is the output of CMake when I generate makefile:
CMake Error at GUI/CMakeLists.txt:161 (add_library):
Target "igtlioGUI" links to target "Qt5::X11Extras" but the target was not
found.  Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
CMake Error at Examples/qIgtlClient/CMakeLists.txt:31 (add_executable):
Target "igtlioQtClient" links to target "Qt5::X11Extras" but the target was
not found.  Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?
CMake Error at GUI/CMakeLists.txt:161 (add_library):
Target "igtlioGUI" links to target "Qt5::X11Extras" but the target was not
found.  Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
CMake Error at Examples/qIgtlClient/CMakeLists.txt:31 (add_executable):
Target "igtlioQtClient" links to target "Qt5::X11Extras" but the target was
not found.  Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?
Generating done

I have the problem on both Ubuntu 16.04 and 18.04 with vtk-7.1.1 and old OpenGL backend. I have vtkGUISupportQt enabled, but not vtkGUISupportQtOpenGL.

Also, I use Qt 5.9.5 on 18.04 and 5.5.1 on 16.04

Here are the cache and the log:
CMakeCache.txt
CMakeOutput.log

The problem seems to come from vtk. In vtkQt, you have those lines in the CMakeLists.txt
set(QT_LIBRARIES_PRIVATE ${Qt5X11Extras_LIBRARIES})
...
vtk_module_link_libraries(${vtk-module} LINK_PRIVATE ${QT_LIBRARIES_PRIVATE})

So the library is linking with Qt5X11Extras, but it is not telling its clients to do so.


Not saving logs

Hello, i'm using the .deb package for the deploy and with this application.yml (on /etc/shinyproxy/) it doesnt log container logs nor shinyproxy logs
proxy:
  title: GDM Analytics Team
  logo-url: https://i.imgur.com/xFU0biq.png
  favicon-path: /etc/shinyproxy/favicon.ico
  landing-page: /
  template-path: ./templates/1col
  #template-path:./templates/modified_navbar
  port: 3839
  authentication: none
  admin-groups: GDM
  container-log-path: /etc/shinyproxy/logs/containers
  # Example: 'simple' authentication configuration
  users:
  - name: guest
    password: guest
    groups: user
  - name: jclavijo
    password: gdm7030
    groups: GDM
  - name: mmoine
    password: gdm7030
    groups: GDM
  - name: lmendonca
    password: gdm7030
    groups: GDM
  # Docker configuration
  docker:
    cert-path: /home/none
    url: http://localhost:2375
    port-range-start: 20000
  specs:
  - id: 7030_vis_results
    display-name: 70/30 Results Visualization APP
    description: App to visualize and make selections in the RETEST 70/30 experiments
    container-cmd: ["R", "-e", "shiny::runApp('/root/BIBD_GS/src/r/shiny/app.R')"]
    container-image: 7030
    access-groups: GDM, user

logging:
  file:
    name: /etc/shinyproxy/logs/shinyproxy/shinyproxy.log

On the other hand, when I use the java -jar command, it works just fine with the same application.yml file
Any thoughts?

Hi @francescocamussoni
The deb package of ShinyProxy creates the /etc/shinyproxxy directory. This directory is owned by the root user, but can be read by the shinyproxy user. However, the shinypoxy user cannot write into that directory. Therefore the logs are not saved.  ShinyProxy logs an error message on startup about this:
java.nio.file.AccessDeniedException: /etc/shinyproxy/logs

You can fix this using:
sudo mkdir /etc/shinyproxy/logs
sudo chown -R shinyproxy:shinyproxy /etc/shinyproxy/logs

Hi , this issue was opened a long time ago and I the answer above should do the trick. Therefore, I'm going to close this issue. If you are still facing an issue, please re-open the issue or open a new one.


Use (non-Material) Blueprints to pass metadata to Materials

After #698, Cesium for Unreal has the ability to pass feature metadata into Materials as textures 🎉. While the "Auto Fill" button on the CesiumEncodedMetadataComponent only discovers numeric properties, even string properties can be coerced to numbers (i.e.: parsed) and then used for shading.
However, it's not currently possible to do more complicated transformations of properties to prepare them for use in Materials. For example, Cesium OSM Buildings has a cesium#color property with string values like "rgb(100, 65, 170)". There's no way to parse this string and write an actual color into the generated texture. I hacked up a way to do this in https://github.com/CesiumGS/cesium-unreal/tree/metadata-colors, but I don't think we should ship it in this form.
In CesiumJS, 3D Tiles Styling allows the use of JavaScript-like expressions to style. Implementing this in Unreal at runtime is unlikely, but we could offer an in-editor option to turn a style description into a Material.
Perhaps the more Unreal-y way to do it, though, is to let the user write a Blueprint that takes feature metadata as input and returns values to be stored in a texture, which can then be accessed from a Material. Conceptually this is really straightforward, and a proof of concept of this would probably be quick to implement. But it raises some questions:

Where do the Blueprints live? On the encoded metadata component?
Will executing a Blueprint function per feature be too slow?
Can we run a Blueprint function in a worker thread, so we don't have to block the main thread while we're processing metadata for shading?

Related to #771.

We considered potentially incorporating this into the metadata overhaul, but we probably won't implement it due to the time constraints and performance concerns. But to quote @kring,

Even just the ability to write and register encoding functions in C++, and then select which one of them to use in the CesiumEncodedMetadataComponent UI, would be huge.

Inspired by this idea, I'd like to rework the encoded metadata component UI as so:

Display the property's actual type, regardless of whether or not it is supported.
Have some option that displays a method of coercion? Like an enum dropdown that says "None, Uint8, Float." We can even throw in "Color" based on the metadata-color branch, handling rgb values as well as hexcode colors.

This way, it more accurately reflects the data that is available, even if it's incompatible with the GPU. And if the user chooses a coercion method that isn't compatible with that type, maybe we can make use of templates to indicate that it can't be coerced?
That's what I aim to implement for now. This probably won't close the issue, should this implementation make it in, but it's hopefully a good start.


[React] Add option to hide personal wallet switch to React

Problem solved
Short description of the bug fixed or feature added
Changes made

[x] Public API changes: list the public API changes made if any
[ ] Internal API changes: explain the internal logic changes

How to test

[ ] Automated tests: link to unit test file
[x] Manual tests: step by step instructions on how to test

Thanks @kien-ngo  - LGTM 🚀

@kien-ngo can you add the same for React Native? I can run it if you don't have the RN setup but it should be pretty similar

@iketw copy that


ModuleNotFoundError: No module named 'unagi.unagi'

Hello, I'm trying to test out whether I correctly setup the repo. I have installed the required modules, including unagi, but I still always get the ModuleNotFoundError: No module named 'unagi.unagi' when I run 'unagi +experiment=test dataflow.data_dir=LOCAL_PATH_TO_DOWNLOAD_CIFAR'
How can I solve it?

Hi, thank you for your interest!
This looks like an install error - can you list all the steps/commands you used to instal it?

Hello Dan, I used the command that you posted:
git clone https://github.com/HazyResearch/thanos-code.git
cd thanos-code
export PATH="<path to conda dir>/anaconda3/bin:$PATH", where I replaced path to conda dir as the path where I installed miniconda.
conda create -n unagi_env python=3.8
conda activate unagi_env
make dev

Did the install succeed? If you run Python from within unagi_env can you import unagi?

Did the install succeed? If you run Python from within unagi_env can you import unagi?

Hello Dan. I can import unagi under unagi_env, but the weird things is that there's no module named unagi.unagi even after download.
 File "/home/jz24694/miniconda3/bin/unagi", line 9, in <module>     from unagi.unagi import main as unagi_main ModuleNotFoundError: No module named 'unagi.unagi'

From within the unagi_env can you run these commands in Python and paste the output:
import unagi
print(unagi.__file__)
import unagi.unagi
from unagi.unagi import main

My output is the following:
Type "help", "copyright", "credits" or "license" for more information.
>>> import unagi
>>> print(unagi.__file__)
/home/jz24694/miniconda3/envs/unagi_env/lib/python3.8/site-packages/unagi/__init__.py
>>> import unagi.unagi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'unagi.unagi'
>>> from unagi.unagi import main
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'unagi.unagi'

Thanks for replying!

This is what I see when I run these commands:
>>> import unagi
>>> print(unagi.__file__)
None
>>> print (unagi.unagi.__file__)
/PATH_TO_THANOS_CODE/thanos-code/unagi/unagi.py

Do you have another package installed called unagi? There may be a namespace conflict.

Hello, I re-installed everything again, and now this is my output of running these commands under thanos-code folder:
>>> import unagi
>>> print(unagi.__file__)
None
>>> print(unagi.unagi.__file__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'unagi' has no attribute 'unagi'
>>>

Also, I get the following output when running unagi +experiment=test dataflow.data_dir=LOCAL_PATH_TO_DOWNLOAD_CIFAR

That is the expected behavior. The test script only runs for a few epochs.

Thanks! Now I can do training. One extra question: If I have multiple GPUs, how can I assign one of them to do training?


How do I use URP?

Hey, I've read through the docs, but I am unable to setup URP, I just can't find where to add the rendering feature, and there is no Dear Im Gui component.. any help would be greatly appreciated (maybe post screenshots). I am using Unity 2019.

Hello there. It seems in 2019 the directive HAS_URP is not satisfied because we use version 10+.
I'm fixing and will update very soon.
Thanks for your feedback.

https://github.com/psydack/uimgui/commit/68dfafdf75a40b0dd0b0d2db062d1b7212399c61 fix this. Could you use the version 4.0.1 and see if you can following the instructions below?
I made a blank project so it can be slightly different from yours
1- find your renderer and add RenderImGui component on it:

2- add U ImGui Component

3- change Renderer Type to procedural (just because it's 2019)
4- drag you RenderImGui Component created on your renderer

5- add your shader and style (the UImGui package has two default that you can use: Packages/UImGui/Resources/)

6- (optional) add Show Demo Window Script in your scene

7- (optional) the URP shaders are not working very well with FXAA, so if you're getting an blurry image, you can change to other:

If it fixed your problem, could you close this issue please?
Thanks again.

Yes, this worked. Thanks a lot! :)

Noting for anyone trying to do this with URP 17 / Unity 6, the steps are different:

Add the RenderImGui comonent and Rendering Feature as stated above
In the inspector for your Universal Renderer Data asset (the thing to which you just added the Rendering Feature), click the 3 dot menu at the top right (next to the lock icon) and change it from "Normal" to "Debug"
Now you should see a "Renderer Features" dropdown, expand that, then right click on the RenderImGui renderer feature and select "Copy"
Now go back to the RenderImGui component, right click on the Render Feature input and select "Paste"


Use DPP runner instead of API loader

DPP Runner (https://github.com/akariv/dpp-runner) is a small utility library for running a single pipeline/source spec in a temporary workspace and as a separate thread.
You can submit these specs to it, and it will call a callback function with status updates.
The modifications in this PR:

[ ] Remove the status callback from os conductor (the one that API loader used to call), and repurposes the code to serve as the dpp-runner callback.
[ ] Modifies the package/upload handler, to create a fiscal source-spec from the provided datapackage and use that spec as an input to dpp-runner
[ ] Modifications to the Dockerfile to allow all that goodness to actually work in a single container.

(there are still a couple of tests failing, will take care of them)

@brew fixed CR, tests and Docker.

Nice! A joy to get rid of more env vars :D


[QUESTION] Styling mapped row

Prerequisites

[ x ] Have read and understood: https://github.com/Maatwebsite/Laravel-Excel/blob/3.1/.github/SUPPORT.md
[ x ] Checked if your Laravel Excel version is still supported: https://docs.laravel-excel.com/3.1/getting-started/support.html#supported-versions
[ x ] Checked that your question isn't already asked before.
[ x ] Filled in the entire issue template

Versions

PHP version: 7.4.4
Laravel version: 6.18.13
Package version: ^3.1

Description
Hello, I am wondering if it is possible to style a mapped row using this package.
I have read in the docs that you could use events to access the underlaying Phpspreadsheet instance and then call styling functions.
However, it does not seem possible to me to access the PhpSpreadsheet instance from the WithMapping::map function. What I would like to achieve is to either bold or add border on certain mapped row to indicate that these row are related.
I.E:
row 1
row 2 (bordered)
row 3 (bordered)
row 4
Here is where I'd like to apply the border/bold

public function map($ticket) : array
    {
        if ($ticket->hasManyRecalls()) {
            return $this->getMultipleRecallsMap($ticket);
        }

        $recall = $ticket->recalls->first();
        return [
            $ticket->ID,
            $ticket->OpeningDate,
            $recall->Date,
            $recall->recaller->fullname ?? '',
            $recall->internalNote ? $recall->internalNote->getFormattedText() : ''
        ];
    }

private function getMultipleRecallsMap($ticket) { // Apply styling within this function
        $map = $ticket->recalls->map(function($recall) use ($ticket) {
            return [
                $ticket->ID,
                $ticket->OpeningDate,
                $recall->Date,
                $recall->recaller->fullname ?? '',
                $recall->internalNote ? $recall->internalNote->getFormattedText() : ''
            ];
        })->toArray();

        return [[], ...$map, []];
    }

Thank's for your time!

That’s not possible, sorry

I've put this as a proposal. Conditional stying based on cell value might be something that will be included in a next release.

Will be part of 4.x


Add ability to close terminals

Fixes #114.  I removed Close Document until we get proper focus-request messages.

I'm not sure this is actually what we want. Let's hold off (leave open) until we can talk to Brian in person at the dev meeting next week.

Closing in favor of the solution proposed in https://github.com/jupyter/jupyter-js-plugins/issues/141


Element::closest

https://developer.mozilla.org/en-US/docs/Web/API/Element/closest
The Element.closest() method returns the closest ancestor of the current element (or the current element itself) which matches the selectors given in parameter. If there isn't such an ancestor, it returns null.

Thanks Álvaro for your contributions!

I wonder if a neater way to check for the "closest" element would be to iterate up the parentNode in combination with the matches() function? What do you think @aguimaraes ?


Issues with Kwikset 914 (nodes going dead, jamming up entire

Is your problem within Home Assistant (Core or Z-Wave JS Integration)?
NO, my problem is NOT within Home Assistant or the ZWave JS integration
Is your problem within ZWaveJS2MQTT?
NO, my problem is NOT within ZWaveJS2MQTT
Checklist

[X] I have checked the troubleshooting section and my problem is not described there.

[X] I have read the changelog and my problem was not mentioned there.

Describe the bug
Hello!
I have three Kwikset 914 deadbolts that I am having trouble with.  I have a fairly active 131-node Zwave network currently powered by ZwaveJS 9.0.2 & ZwaveJS2MQTT 6.7.1 on a Raspberry Pi 4B, installed by npm.  I am using an SiLabs UZB-7 (with 7.17.2 firmware) as the controller.  I currently have HomeAssistant 2022.4.5 talking to this setup.
What I am observing is that sending lock/unlock commands to the three deadbolts results in a variety of intermittent behaviors -- none of these necessarily happen "every time", but all happen very frequently:

slow response (often as long as 10 seconds or more) to lock & unlock commands
incorrect status after slow lock & unlock commands (ie, deadbolt may lock/unlock after a multisecond delay, but the status will not change to locked/unlocked)
deadbolt node going "dead" (and requiring a ping to come back alive)
commands to other Zwave network effectively freezes up for a period of time when the above happens -- commands to other devices

This probably goes without saying, but the behavior tends to compound when trying to interact with multiple locks in a row -- for example, issuing a lock command to all three deadbolts at bedtime will typically jam up other Zwave commands for quite some time.  I've made this behavior a little less painful by introducing a short delay between each lock command (3 seconds in the log I'm going to attach).
PS: Last thought I thought might be worth sharing: Even though the three deadbolts are the same model, ZwaveJS detects one of them as something different (although all three report the same v4.10 firmware).  I have observed these same results even after re-interviews, and exclusion/re-inclusion cycles:

Node 12 detected as Kwikset 914/Door Lock
Node 13 detected as Kwikset 914/Door Lock
Node 15 detected as GED1800/SmartCode 10 Touchpad Electronic Deadbolt

PPS: I do have the capability to capture Zniffer logs if needed & requested.
Device information
Manufacturer: Kwikset
Model name: 914
Node ID in your network: 12, 13, 15
How are you using node-zwave-js?

[ ] zwavejs2mqtt Docker image (latest)
[ ] zwavejs2mqtt Docker image (dev)
[ ] zwavejs2mqtt Docker manually built (please specify branches)
[ ] ioBroker.zwave2 adapter (please specify version)
[ ] HomeAssistant zwave_js integration  (please specify version)
[ ] pkg
[ ] node-red-contrib-zwave-js (please specify version, double click node to find out)
[ ] Manually built from GitHub (please specify branch)
[X] Other (please describe)

Which branches or versions?
version: node-zwave-js 9.0.2, zwavejs2mqtt 6.7.1
Did you change anything?
yes (please describe)
If yes, what did you change?
For the past few months I considered this setup to be a sandbox as I got to know HomeAssistant and ZwaveJS a bit better, but I got comfortable enough with it that I migrated everything from another Zwave controller/platform in the past few weeks (growing me to the 131 nodes I have at the moment).  I have anecdotally observed this problem has gotten more severe as I moved more and more Zwave devices over to this setup.
Did this work before?
Yes (please describe)
If yes, where did it work?
See previous comment about recent changes.
Attach Driver Logfile
zwavejs_2022-04-18.log.gz
A great example of this behavior in action would be around 2022-04-19T03:16:09.817Z.
Note that the ping sent to Node 013 at 2022-04-18T23:28:54.155Z is automated; when one of the three deadbolt nodes go "dead" I have my setup wait 15 seconds and then send it a ping to bring it back to life.

I'm almost certain this isn't caused by the locks, but by excessive reporting which causes your controller to stop sending anything. Essentially https://github.com/zwave-js/node-zwave-js/issues/3906 but much less severe.
Z-Wave JS doesn't handle a jammed controller well yet, but you should be able to get on top of this by turning down on the meter and multilevel sensor reporting. In your log I see >7000 electric meter reports, >14000 lux sensor reports, and almost 5000 motion sensor reports, many of which are repeated.
This sometimes happens multiple times per second, which is definitely too much for a stable network. I suggest the following:
Check each meter and multilevel sensor report setting, if you actually need it. E.g. do you automate on the illuminance? If not, turn it off.
Turn the reporting interval waaaaaaay up and use reporting on (significant) change where possible. E.g. a bunch of these measurements are the multisensor reporting that the lux level is still 0. Or half of Node 097's reports are "I am still using 1.6W".
Decide where you need which granularity and precision. Maybe the kWh measurement changing from 1.696 -> 1.697 -> 1.698 -> ... -> 1.705 could be lowered to reporting every 0.01 kWh, or even 0.1 kWh. These might not be many reports per device, but over a large network this adds up quickly. Especially if you have bursts where you move through the house, 5 motion sensors fire, turn on lights and 15 power meters start reporting every few seconds.
If you need periodic reports as a backup, I suggest not going below once per hour per device.

That makes sense given it has gotten worse as I've moved the various sensors over to this platform.  I'll work on cranking down all of the reporting.
I also have quite a few Zsticks, maybe I'll move the locks and three strategically placed repeaters over to a seconds ZwaveJS2MQTT instance.
Thanks.

Oh, any thoughts about that third lock showing up as the wrong device/model?  Would you be interested in the debug results of a re-interview of it?

Its very likely identifying itself using different IDs which are used by resellers. At least the Google results look similar.

Thanks again for looking at this with me.  Here's what I've done since yesterday:
I went through the log I originally attached, and indeed there was roughly a metric ton more sensor reports than I expected -- specifically, I found 14111 Illuminance reports, nearly 100% of these from my 32 Aeotec Trisensors.  This blew my mind since I was using 0 lux for my Aeotec Trisensors's "Light Intensity Change Threshold" ([41-112-0-22]) which effectively disabled threshold reporting for Illuminance.  I expected the Trisensor to then only report Illuminance based on the timer set in "Automatic Reporting Interval: Light Sensor ([41-112-0-24]) which I had at 14400 seconds.  In practice this appears to not be the case, as log review shows that the Trisensors were reporting Illuminance every 3 minutes regardless (even when the value stayed the same).  Across 32 Aeotec Trisensors, that worked out to an average of an Illuminance report every 5 seconds; yikes!
So with that in mind, I cranked the Light Intensity Change Threshold to 10000 lux and and Automatic Reporting Internal: Light Sensor to 14400.  I don't really use Air temperature either, so I cranked Temperature Change Threshold to 50 and Timed Temperature Report Interval to 14400.  I also disabled threshold power and energy threshold reports from the 50 Inovelli Red Series switches/dimmers and increased the power and energy reporting interval to 14400.
That has quieted down the mesh significantly; I now sometimes go a few minutes without seeing a single report from a sensor.
But back to the lock issue, I'm still seeing the original symptoms: huge delays in response to lock/unlock, sometimes the locks never respond to lock/unlock, sometimes when the lock does finally act it doesn't report back and takes a Refresh Values to get the truth.
I find that if I am very slow and deliberate -- ie, if I wait for a lock/unlock command to complete & report back before doing moving on to the next lock -- it works.  But if I overlap the lock/unlock commands at all (or issue other Zwave commands before the lock has completed + reported back) it almost always runs in to the trouble symptoms.
I'm attaching a new log from this morning with the quieter mesh only covering 1 hour.  The locks are nodes 12, 13, and 15. (Side note: I also see two Trisensors still reporting Illuminance every 3 minutes in here, I'll go wake them up now).  Does this new log still indicate the 700 Series ATF issues?
If you need me to repeat with more specific timestamps of when I issued commands vs. what happens + when, or a Zniffer capture to go along with a ZwaveJS log, I'm happy to do so.  I appreciate you trying to help me sort this out.
TRIMMED_zwavejs_2022-04-21.log

The controller is still jammed once for roughly a second or two while the onslaught of commands from the network is coming. I think there is still another underlying issue, but this is roughly what happens:

Driver commands Lock A
This is done, Driver moves on to Lock B
Meanwhile Lock A wants to reports its status, requests a nonce
Driver is still waiting for Lock B, cannot respond yet
Lock A gets impatient, request a nonce again
This command gets re-transmitted a few times by the network, probably due to lost ACKs
Driver is done with Lock B, moves on
Now Lock B wants to report its status too
...
boom 🤯

They don't support Security S2, do they? This would also help a ton.

What worries me a bit are the route speeds used. Out of 213 transmission reports,

12 are 100 kbit/s
63 are 40 kbit/s
136 (!) are 9.6 kbit/s, which is more or less the fallback speed if everything else fails

The fallback speed seems to be used for every communication with the locks, which could be caused by either:

The controller, which might be affected by interference. Make sure to follow the advice under General troubleshooting - this can work wonders
The repeater nodes. All traffic to the locks seems to go through nodes 26 and 27
The locks themselves

zwavejs2mqtt has a tool (similar to the one in PC Controller) to diagnose the link between a node and the controller and between two nodes. After you've made sure of a good controller placement and healed, I'd use that to figure out where the link goes bad:

26, 27 -> 1
12, 13, 15 -> 26
12, 13, 15 -> 27

Make sure to initiate the check on the locks towards the other nodes, as FLiRS devices as check targets would introduce incorrect delays.

@AlCalzone, will review your advice about device routing in the next day or two.  Thank you.  I do use a USB extension cable to mount the Zstick in a position that it has direct line of sight to 2 of the locks, so I'm surprised all of the strange routing is even necessary.
I did move everything to an Aeotec Zstick Gen5+ and there was definitely improvement: the lock/unlock commands work reliabily now.  The feedback from the lock that it is locked/unlocked is occasionally still spotty, but I'll focus on the bits you suggested before reporting back with more detail.
To your S0 vs S2 question: Is there an S2-supporting deadbolt you know is reliable?  I'm not married to these Kwikset 914s.  I bought them because they were Zwave Plus assuming that meant S2, but of course that turned out to be an incorrect assumption.

To your S0 vs S2 question: Is there an S2-supporting deadbolt you know is reliable? I'm not married to these Kwikset 914s. I bought them because they were Zwave Plus assuming that meant S2, but of course that turned out to be an incorrect assumption.

Looks like Kwikset has a newer model called the Home Connect 620 that does include S2:

https://products.z-wavealliance.org/regions/2/categories/6/products

I am going to order one of these today, and if they work well I'll replace the 914s with these guys.

I have 3x HomeConnect 620 zwave700 models working just fine. Make sure you get one of the new 700 series ones, if you can - can be tricky sometimes as the model number is the same - HomeConnect 620.