How to use a PIR sensor for motion detection instead of basic motion features (external/internal)

Hello,
I want to use RCWI for birds survey in my garden. In the scene are some trees with leaves, and when there is some wind that makes leaves moving, Motion detection starts and creates videos.
I'd like to use a PIR sensor in orer to detect motion only for living subjects like birds.
A PIR sensor is cheap and connects on the GPIO, but how to use it in RPI_Cam_Web_Interface instead of the standard motion detection methods ?
Thanks in advance if you can help me on that project.
Pierre

There are two aspects here; sensing events from the PIR, and triggering activity in RCWI.
RCWI can be fully controlled by sending commands into its control pipe (FIFO) at /var/www/html/FIFO This is documented in the wiki and you can try things out by just using the echo command.
E.g.
echo 'ca 1 10' > /var/www/html/FIFO
will capture a 10 second video
So if you have a program that detects motion events from the PIR then that event can send in commands to the FIFO to do whatever you want.
Probably the easiest is to use a python program to detect when the GPIO the PIR is connected to activates and then the program can send commands to the FIFO.
In python something like this should send that command
fifo = open("/var/www/html/FIFO", "a")
fifo.write("ca 1 10")
fifo.close()
There is also a slightly more sophisticated method which is to use the FIFO1 control pipe, This is monitored by the scheduler to detect motion events and then do whatever the scheduler has been configured to do. Writing a '1' to FIFO1 signals a motion start event. Writing a '0' signals a motion stop event. The advantage of doing it this way is that the commands that get sent to the camera are easily changed in the scheduler web interface rather tan being coded directly in your python program. You probably want to get the first method going first before trying this.

Thanks for these infos. The echo command works perfectly well on my Raspberry config.
I should receive tomorrow my PIR sensor and I'll connect it o the GPIO pins.
What I need to know is if I have to configure it in Raspbian or if the sole connection o GPIO is enough to use it.
If it is active, how can I set up RCWI in order that motion detection could use it ? Do I have to create and run a python script each time I power on the Raspberry or is RCWI able to use the motion detection with PIR only with its own motion detection settings ?
When I'll have the PIR used for motion detection, my project will be ready for full use !

Attaching a PIR to a GPIO will do nothing without a program or script to detect its signal and do something with it.
The program or script can be in any language that you like. I suggested python as that is probably the easiest to use for this. It should only be a few lines long. There are lots of examples of this on the web you can start from. If you don't have much experience with this then start from one of the examples that just prints out text messages when motion is detected so you can see how to do that and then add in the bit of python I suggested to send the command into the FIFO.
The RCWI allows use of external motion either by accepting commands as described or by sending in external motion start/stop signals into FIFO1 as described in the second part of my reply.
Note that if you have a python program to handle the PIR and trigger the camera activity then you can easily arrange for it to start automatically every time the raspberry boots so that you would not need to do that manually. Again there is lots of information on how to do this. One method is just add a command to the /etc/rc.local file which already has the commands to start up the RCWI software.

Hi Robert,
Yes I have python scripts that use PIR for test and display messages like "motion detected".
But how to use the PIR or the script in order to get the info in FIFO, FIFO1 or FIFO0  ? Which settings to check in camera or motion detection or schedule in order that RCWI use the PIR instead of detection of changes in landscape ?
I should get my PIR today and I'll try all that

You don't really need to change anything in the RCWI set up as it listens to and obeys any instructions coming into FIFO or FIFO1 all the time. You do not need to activate motion detection. That is for the built in video motion detection.
Instead of the instruction to print "motion detected" in the script you need to replace that with the lines of python I showed to send a command into one of the FIFOs.
The RCWI include actions for both motion started and motion finished which is intended for motion detection like the built in video method which waits until the scene is still before signalling a motion stop. The PIR fires when it sees motion and its signal stays on for a fixed period of time before resetting to wait for the next motion. So it is probably easier to base the action on just the motion detected (start) and arrange that to capture a video of a fixed length whenever the PIR fires. The capture command (ca) allows for this by having a second optional parameter for the number of seconds of a fixed length recording. So "ca 1 10" will start a video recording and it will automatically terminate after 10 seconds.
Method 1 - Direct command Python
Include following in the script when motion detected
fifo = open("/var/www/html/FIFO", "a")
fifo.write("ca 1 10")
fifo.close()
Method 2 - Via scheduler set up
In scheduler set up put ca 1 10 in the motion start field; save settings. You don't need anything in motion stop field
Include following in the script when motion detected. Doing it this way allows you to change the command and the length of the video from the web interface and leave the script alone.
fifo = open("/var/www/html/FIFO1", "a")
fifo.write("1")
fifo.close()
If you want to test method 2 manually then you can just use the echo command
echo '1' > /var/www/html/FIFO1
which will simulate a motion start event.
Integration of the script
Test the script manually first just by running it. When it is OK then you can start it automatically when the raspberry starts up. It is very important you put an & at the end of the line as this means the scripts runs in the background and allows other things to continue. If you leave it out the boot sequence will stall. There are many methods to start a script automatically. See
https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/
Maybe use the /etc/rc.local method as that is already there. You can just use a simple
sudo python myPIRscript.py &

Hi,
I've received my PIR. I made test with a python script and that works fine.
But I tried to include your lines for fifo , and the script returns an error :
(the addition of the 3 lines with fifo are the only change from the original scrpt wich works...)
OSError :  [Error 29] Illegal seek

Here is my python script :
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14 # Assign GPIO14 pin 8 to PIR
GPIO.setup(PIR_PIN, GPIO.IN) # Setup GPIO pin PIR as input
print('Sensor initializing . . .')
time.sleep(60) # Give sensor time to start-up, 60 seconds
print('Active')
def pir(pin):
print('Motion Detected!')
GPIO.add_event_detect(14, GPIO.FALLING, callback=pir, bouncetime=100)
print('[Press Ctrl + C to end program!]')
fifo=open("/var/www/html/FIFO","a")
fifo.write("ca 1 10")
fifo.close
try:
while True:
time.sleep(0.001)
except KeyboardInterrupt:
print('\nScript ended')
finally:
GPIO.cleanup()
Also I don't know where to place the 3 lines fifo in this script. I tried at several paces, but always the same error occurs.

That is because you have the send command stuff inside a loop while the PIR signal is active.
You need to arrange it so it only sends once when  the PIR signal goes active and then waits until the PIR signal becomes inactive before then starting again. So you will need a loop waiting it to go inactive inside the loop.

Something like this for the text version which should only print the message once per motion trigger
`import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14
GPIO.setup(PIR_PIN, GPIO.IN)
print('Starting up the PIR Module (click on STOP to exit)')
time.sleep(1)
print ('Ready')
while True:
if GPIO.input(PIR_PIN):
print('Motion Detected')
time.sleep(1)
while GPIO.input(PIR_PIN):
pass
`

Here is my script which works but records continuously.
Please could you help me and arrange it in order that it wait for another event to restart recording ?
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14
GPIO.setup(PIR_PIN, GPIO.IN)
print('Starting up the PIR Module (click on STOP to exit)')
time.sleep(1)
print ('Ready')
try:
while True:
if GPIO.input(PIR_PIN):
print('Motion Detected')
fifo = open("/var/www/html/FIFO1", "w")
fifo.write("ca 1 10")
fifo.close
time.sleep(1)
except KeyboardInterrupt:
print('\nScript ended')
finally:
GPIO.cleanup()

Here is my script.
Could you help me to arrange it in order that it works like you say ?
Many thanks
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14
GPIO.setup(PIR_PIN, GPIO.IN)
print('Starting up the PIR Module (click on STOP to exit)')
time.sleep(1)
print ('Ready')
try:
while True:
if GPIO.input(PIR_PIN):
print('Motion Detected')
fifo = open("/var/www/html/FIFO1", "w")
fifo.write("ca 1 10")
fifo.close
time.sleep(1)
except KeyboardInterrupt:
print('\nScript ended')
finally:
GPIO.cleanup()

Have you  tried the version with the inner loop waiting for the PIR to go low?

I tried your last script (ht one which ends with "pass").
But it returns errors and nothing works.
<i don't see in it th fifo instructions. Perhaps the script was not enterely displayed in your comment...

You need to get indenting right in python. If this gives errors then tell me what  they are
import RPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14
GPIO.setup(PIR_PIN, GPIO.IN)

print('Starting up the PIR Module (click on STOP to exit)')
time.sleep(1)
print ('Ready')

try:
  while True:
    if GPIO.input(PIR_PIN):
      print('Motion Detected')
      time.sleep(1)
      while GPIO.input(PIR_PIN):
        pass
except KeyboardInterrupt:
  print('\nScript ended')
finally:
  GPIO.cleanup()

I made the script just printing out the text. Once that works then replace the print with the lines to send the fifo commannd

Here is a version with the fifo in. I removed all text printing which you don't want in a program running in the background and moved the delays around a bit. Note you send direct commands like ca 1 10 to FIFO. If you want to use the FIfo1 scheduler version then you need to send '1' not ca 1 10 as the scheduler determines what command to use.
import RPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14
GPIO.setup(PIR_PIN, GPIO.IN)

time.sleep(1)

try:
  while True:
    if GPIO.input(PIR_PIN):
      fifo = open("/var/www/html/FIFO", "w")
      fifo.write("ca 1 10")
      fifo.close
      while GPIO.input(PIR_PIN):
        time.sleep(1)
    time.sleep(1)
except KeyboardInterrupt:
  pass
finally:
  GPIO.cleanup()		  ```

Hi,
Sorry... This last script doesn't work.  RCWI don't record videos....

I have just tried the script but had to simulate the PIR trigger and it recorded a video for me OK
Try one with some print statements so you can see what is going on.
import RPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14
GPIO.setup(PIR_PIN, GPIO.IN)

time.sleep(1)
print('Starting up the PIR Module (click on STOP to exit)')

try:
  while True:
    if GPIO.input(PIR_PIN):
      print('Motion Trigger started')
      fifo = open("/var/www/html/FIFO", "w")
      fifo.write("ca 1 10")
      fifo.close
      while GPIO.input(PIR_PIN):
        time.sleep(1)
      print('Motion Trigger ended')
    time.sleep(1)
except KeyboardInterrupt:
  pass
finally:
  GPIO.cleanup()

Hi Robert,
That's very strange. The script displays correctly the statements, but recording videos is not correct !  The video is recorder when I stop the script !
When script is run, I see all the statements, (many ones), but no video are recorded. It's when I Stop the script that a video is recorded !
What would be good were to record videos when PIR detects a motion. And not when script stops. Also, it should record further videos if new motions are detected.
Once again, many thanks for your support. I hope that this discussion will lead to a good solution and help also other guys...

I missed the brackets off the close call which means the write was just buffered and not flushed out
use fifo.close()
The script should then work as intended and do a recording for each pir trigger

Here is the script from the distributor of the PIR. It is very close to yours, but the method for checking detection seems different. Perhaps is that dur to the behaviour of the PIR which have High and Low signal level...
You can see that they dont use "GPIO.input(PIR_PIN)" but this one :
GPIO.add_event_detect(14, GPIO.FALLING, callback=pir, bouncetime=100)
Here is the full script
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
PIR_PIN = 14 # Assign GPIO14 pin 8 to PIR
GPIO.setup(PIR_PIN, GPIO.IN) # Setup GPIO pin PIR as input
print('Sensor initializing . . .')
time.sleep(60) # Give sensor time to start-up, 60 seconds
print('Active')
def pir(pin):
print('Motion Detected!')
GPIO.add_event_detect(14, GPIO.FALLING, callback=pir, bouncetime=100)
print('[Press Ctrl + C to end program!]')
try:
while True:
time.sleep(0.001)
except KeyboardInterrupt:
print('\nScript ended')
finally:
GPIO.cleanup()

The PIR sensors I have used in the past go active high when movement is detected, stay high for a time period and then go active low and wait for the next movement.
Can you include a link for your sensor?
From the suggested script it seems to go active low when movement is detected and uses the GPIO.FALLING event to signal motion detection.
What you can try is to use that script and put the 3 lines to send the command into the same place as where it says print(Motion Detected) with the same level of indent.
The other thing to be aware of is that some PIR sensors can false trigger if they are too close to sources of wifi and can then continuously trigger even without movement. If you get continuous Motion Detected messages without real motion then try to move the sensor away from the Raspberry.

Hi,
I attach the pdf which explains the behaviour and the use of my PIR.
I tried both FALLING and RISING options but both makes continuoumy recording...
I also tried to use if/else in order to check/see the value of GPIO.Input. All that works but always continuously ecord video !
I've seen in the document that there is a composnent of the PI which increases or decrease the sensibility. Perhaps do I'll try to act on it.
Currently, I make my tests with the raspberry and PIR just near from me. Even if the PIR "looks" at the opposite of me, perhaps do it detects something.
HC-SR501 PIR Sensor_EN.pdf

in addition to previous comment.
Hereunder is the def pir I wrote for testing the behaviour of the PIR.  I used BOTH instead of FALLING in order to handled each level.
That works, but always continuous recording.
Perhaps could it be necessary to define a waiting time or instruction to stay at LOW level before restarting motion detection ?

def pir(pin):
if GPIO.input(14):
print('Rising edge detected on 14')
print('Motion Detected!')
fifo = open("/var/www/html/FIFO", "w")
fifo.write("ca 1 10")
fifo.close()
print('ca 1 10 envoye a FIFO')
else:
print('Falling edge detected on 14')
GPIO.add_event_detect(14, GPIO.BOTH, callback=pir, bouncetime=100)

The HC-S501 PIR sensor is known to continuously trip if there is rf signals around such as wifi. Being near to the Pi can cause problems.
You can try shielding it with some grounded metal foil.
I used to try to use that sensor but it was always tricky near other electronics.
For PIR applications I now use sensors based on AM312 which are much better. Their electronics is in the same shielded case as the sensor itself and they are pretty immune to rf interference problems.
The AM312 and the HC-S501 are both active high on detecting motion so the original script should be fine.

Good...
Do you have a link for purchase ? (I prefer on amazon...)

If you search for am312 pir sensor on Amazon or eBay you will find lots of sellers. I usually use eBay for things like this as they can be faster. Sometimes items like this on Amazon come from overseas suppliers and take a while to come.
I don't have any experience of using PIR sensors with small birds. PIR sensors all work by detecting changes in the thermal radiation pattern. As birds are much smaller then you probably need the sensor to be reasonably close to the target. The AM312 would be helpful here as it is a lot smaller giving you more flexibility in positioning.

Hi,
I've ordered one at Amazon france and it should be delivered on next monday.
I'll try all th scripts we disccused, and I'll see if that works better.
For my birds survey, I just have to detect at 0,5 m.  That should be good enough...
I've also seen another PIR from Parallax. But I don't know if it is close to the am312 or close to the HC-5501 !

As far as I can see the Parallax seems to be similar to HC-5501. The chip used looks similar and it is not shielded so it may well have same interference issues.
Hopefully the AM312 will work for your application.
The other thing worth mentioning is that when using the in-built motion detection. It is possible to apply an electronic mask to limit the area of the image where motion detection occurs. This can be helpful in some cases in eliminating triggers from motion outside of the areas of interest. Details on how to do this are in the wiki.

I've tried to use the system outdoor, far from my wifi box. Same continuously recording occurs...
Then the sole wifi possibles is the one of the PI 3B.... Do you think that this could be the cause of the issue ?
I look fordward to receive the am312 tomorrow...

It is the raspberry wifi that will cause retriggering. If you can put the senssor on extension leads and get it distant from the raspberry that might help, but the am312 is a better bet.

Yes I could put it on extension, but for my proect, it will always be near from the raspberry.... Then the better thing is to try the am312.
Other solution : I could use a older raspberry without wifi, and just connect a wifi key when needded.

Hi, I'd like to test a raspberry pi model which is without wifi, but has an ethernet rj45 port.
If you have one you don't use now, could you sell it to me ?
Which is the best model ?

You don't need a different raspberry model. You can just disable wifi (and bluetooth).
See for example
https://www.animmouse.com/p/how-to-disable-wifi-and-bluetooth-on-raspberry-pi/

Hello Robert
I've received the am312 and connected it on my raspi. That works well !!! Just as intended.   Motion detected---> REcording 10s video and stop until next motion detection.
We've define ca 1 10 , that records 10s videos. Is it possible to recod video as long as a subject is in front of PIR ? (in one video, not in continuous independant videos like with RHC-5501)

If you want the camera to record whilst the PIR signal is active and to stop when it is inactive then

Change the command from ca 1 10 to ca 1  This will just turn the camera on.

repeat the 3 lines that send the command in the part after it has waited for the PIR to go low (where motion ended is printed) but make the command ca 0  which will stop the recording.

If you are using the version that uses the edge triggered then you will need the one that detects BOTH edges and add the 3 lines to turn the camera off in the part where it prints falling edge detected.
However, you may find that you get several short videos as any still period of more than a few seconds will stop the camera. in the first type of script not using the pir edge method, you could set a minimum time for a recording by putting a sleep after recording starts before it starts looking for PIR inactive.

Hi,
I've contacted the distributor of thePIR and explained the issue. They told me to decrease sensibility and time output. But that was not satisfying. They'll send me a new PIR in replacement in case of the one is defective.
I'll keep yu in touch when I'll test the new one.
Now, I need to automatically initialise the PIR when I power the raspberry.  Can I use the script which works well aithe the am312 ?  How to rn it automatically a stating time ?
Thanks for all !
Pierre

One of the easiest ways to start the python program automatically at boot is to add an entry to the /etc/rc.local file
e.g. sudo nano /etc/rc.local
cursor down to just before the end and add
sudo python /path/to/pythonfile &
e.g.
sudo python /home/pi/pirTrigger.py &
save the file and next time the raspberry is booted the script will be started
It is very important you put the space & on the end of the line as this ensures the script runs in the background.

It shouldn't matter whether it is before or after as it is starting a program independently of the camera stuff.
If putting it before causes an error in RCWI starting up then it sounds like something is wrong with how you entered the line. Make sure you entered the path correctly including the case of the letters
You can check whether the script is running by using the command
ps lww -C python
If it is running then you should see a process with the command line

Hi,
Some feedback after many tries and settings.
I got a replacement PIR sensor, but it had the same issues of unattended motion detection. I spent lot of time to set up the 2 components which set up Timeout and sensitivity.  I no have the PIR working rather correctly. I still have unwanted detections, but they are due to lighting changes with sun or clouds. At 80% I get good detection, even with small birds lke robin !


etcd snapshot controller thrashes on etcdsnapshotfile management when server is run with --disable-agent

The docs do mention that it is not supported to run an agentless server with embedded etcd:
https://docs.k3s.io/advanced#running-agentless-servers-experimental

When started with the --disable-agent flag, servers do not run the kubelet, container runtime, or CNI. They do not register a Node resource in the cluster, and will not appear in kubectl get nodes output. Because they do not host a kubelet, they cannot run pods or be managed by operators that rely on enumerating cluster nodes, including the embedded etcd controller and the system upgrade controller.

Despite this warning, users have managed to use it successfully by working around the previously minor limitations. The addition of ETCDSnapshotFile support, which expects to find a Node for the host running etcd, has caused a significant regression for these users. The snapshot controller appears to get stuck in a loop, continuously creating and deleting ETCDSnapshotFile resources.
To reproduce this, run: k3s server --cluster-init --etcd-snapshot-schedule-cron='* * * * *' --disable-agent
INFO[0066] Saving etcd snapshot to /var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-k3s-server-1-1711174624
{"level":"info","ts":"2024-03-23T06:17:04.231021Z","caller":"snapshot/v3_snapshot.go:65","msg":"created temporary db file","path":"/var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-k3s-server-1-1711174624.part"}
{"level":"info","ts":"2024-03-23T06:17:04.232826Z","logger":"client","caller":"v3@v3.5.9-k3s1/maintenance.go:212","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2024-03-23T06:17:04.232856Z","caller":"snapshot/v3_snapshot.go:73","msg":"fetching snapshot","endpoint":"https://<IP_ADDRESS>:2379"}
{"level":"info","ts":"2024-03-23T06:17:04.233081Z","caller":"v3rpc/maintenance.go:126","msg":"sending database snapshot to client","total-bytes":1294336,"size":"1.3 MB"}
{"level":"info","ts":"2024-03-23T06:17:04.238648Z","caller":"v3rpc/maintenance.go:166","msg":"sending database sha256 checksum to client","total-bytes":1294336,"checksum-size":32}
{"level":"info","ts":"2024-03-23T06:17:04.238685Z","caller":"v3rpc/maintenance.go:175","msg":"successfully sent database snapshot to client","total-bytes":1294336,"size":"1.3 MB","took":"now"}
{"level":"info","ts":"2024-03-23T06:17:04.23878Z","logger":"client","caller":"v3@v3.5.9-k3s1/maintenance.go:220","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2024-03-23T06:17:04.244638Z","caller":"snapshot/v3_snapshot.go:88","msg":"fetched snapshot","endpoint":"https://<IP_ADDRESS>:2379","size":"1.3 MB","took":"now"}
{"level":"info","ts":"2024-03-23T06:17:04.244714Z","caller":"snapshot/v3_snapshot.go:97","msg":"saved","path":"/var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-k3s-server-1-1711174624"}
INFO[0066] Applying snapshot retention=5 to local snapshots with prefix etcd-snapshot in /var/lib/rancher/k3s/server/db/snapshots
INFO[0066] Reconciling ETCDSnapshotFile resources
I0323 06:17:04.248724      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174624-46ef6e" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174624 saved on k3s-server-1"
INFO[0066] Reconciling ETCDSnapshotFile resources
INFO[0066] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174566
INFO[0066] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174566
I0323 06:17:05.345400      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174566-ac0972" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174566 saved on k3s-server-1"
INFO[0068] Reconciliation of ETCDSnapshotFile resources complete
ERRO[0068] Failed to take scheduled snapshot: nodes "k3s-server-1" not found
ERRO[0069] Failed to delete ETCDSnapshotFile for non-etcd node k3s-server-1: etcdsnapshotfiles.k3s.cattle.io "local-etcd-snapshot-k3s-server-1-1711174566-ac0972" not found
ERRO[0069] Failed to delete ETCDSnapshotFile for non-etcd node k3s-server-1: etcdsnapshotfiles.k3s.cattle.io "local-etcd-snapshot-k3s-server-1-1711174624-46ef6e" not found
INFO[0069] Reconciliation of ETCDSnapshotFile resources complete
ERRO[0069] Failed to record snapshots for cluster: nodes "k3s-server-1" not found
INFO[0070] Reconciling ETCDSnapshotFile resources
INFO[0070] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174566
INFO[0070] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174624
I0323 06:17:08.347487      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174566-ac0972" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174566 saved on k3s-server-1"
I0323 06:17:08.745261      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174624-46ef6e" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174624 saved on k3s-server-1"
INFO[0071] Reconciliation of ETCDSnapshotFile resources complete
ERRO[0071] Failed to record snapshots for cluster: nodes "k3s-server-1" not found
INFO[0072] Reconciling ETCDSnapshotFile resources
INFO[0072] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174624
INFO[0072] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174566
I0323 06:17:10.545603      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174624-46ef6e" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174624 saved on k3s-server-1"
I0323 06:17:11.145664      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174566-ac0972" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174566 saved on k3s-server-1"
INFO[0073] Reconciliation of ETCDSnapshotFile resources complete
ERRO[0073] Failed to record snapshots for cluster: nodes "k3s-server-1" not found
INFO[0074] Reconciling ETCDSnapshotFile resources
INFO[0074] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174566
INFO[0074] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174624
I0323 06:17:12.945348      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174566-ac0972" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174566 saved on k3s-server-1"
I0323 06:17:13.545189      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174624-46ef6e" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174624 saved on k3s-server-1"
INFO[0076] Reconciliation of ETCDSnapshotFile resources complete
ERRO[0076] Failed to record snapshots for cluster: nodes "k3s-server-1" not found
INFO[0077] Reconciling ETCDSnapshotFile resources
INFO[0077] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174566
INFO[0077] Creating ETCDSnapshotFile for etcd-snapshot-k3s-server-1-1711174624
I0323 06:17:15.348402      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174566-ac0972" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174566 saved on k3s-server-1"
I0323 06:17:15.945665      54 event.go:376] "Event occurred" object="local-etcd-snapshot-k3s-server-1-1711174624-46ef6e" fieldPath="" kind="ETCDSnapshotFile" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ETCDSnapshotCreated" message="Snapshot etcd-snapshot-k3s-server-1-1711174624 saved on k3s-server-1"
INFO[0078] Reconciliation of ETCDSnapshotFile resources complete
ERRO[0078] Failed to record snapshots for cluster: nodes "k3s-server-1" not found

This goes on pretty much endlessly, constantly creating and deleting resources, and spamming the server with events.
We should probably disable management of ETCDSnapshotFile resources when the server is running without an agent, or maybe just if we can't find a Node for the current host.

Validated on master branch with commit 3f906bee79cb32072bea061cbbf4e38142343c66
Environment Details

Infrastructure

[x] Cloud
[ ] Hosted

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"

$ uname -m
x86_64

Cluster Configuration:

HA: 3 server/ 1 agent

Config.yaml:

token: xxxx
cluster-init: true
disable-agent: true
write-kubeconfig-mode: "0644"
node-external-ip: <IP_ADDRESS>
node-label:
- k3s-upgrade=server
etcd-snapshot-retention: 2
etcd-snapshot-schedule-cron: "* * * * *"
etcd-s3: true
etcd-s3-access-key: xxxx
etcd-s3-secret-key: xxxx
etcd-s3-bucket: xxxx
etcd-s3-folder: xxxx
etcd-s3-region: xxxx

Testing Steps

Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

Install k3s

curl -sfL https://get.k3s.io | sudo INSTALL_K3S_COMMIT='3f906bee79cb32072bea061cbbf4e38142343c66' sh -s - server

Check the journal logs for reconciliation error messages:

$ sudo journalctl -xeu k3s | grep 'Failed to reconcile snapshot ConfigMap: no nodes have reconciled ETCDSnapshotFile resources, requeuing'
$ sudo journalctl -xeu k3s | grep error | grep snapshot

Replication Results:

k3s version used for replication:

$ k3s -v
k3s version v1.29.3+k3s1 (8aecc26b)
go version go1.21.8

We can see several error messages on reconciliation requeuing:
 $ sudo journalctl -xeu k3s | grep 'Failed to reconcile snapshot ConfigMap: no nodes have reconciled ETCDSnapshotFile resources, requeuing'
Apr 11 21:56:25 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:25Z" level=debug msg="Failed to reconcile snapshot ConfigMap: no nodes have reconciled ETCDSnapshotFile resources, requeuing"
Apr 11 21:56:56 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:56Z" level=debug msg="Failed to reconcile snapshot ConfigMap: no nodes have reconciled ETCDSnapshotFile resources, requeuing"
Apr 11 21:57:28 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:57:28Z" level=debug msg="Failed to reconcile snapshot ConfigMap: no nodes have reconciled ETCDSnapshotFile resources, requeuing"

 $ sudo journalctl -xeu k3s | grep error | grep snapshot
Apr 11 21:56:04 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:04Z" level=error msg="error syncing 'local-etcd-snapshot-ip-172-31-16-97-1712872505-428a6c': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-etcd-snapshot-ip-172-31-16-97-1712872505-428a6c\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-etcd-snapshot-ip-172-31-16-97-1712872505-428a6c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: be05a51b-714d-4a39-a8e6-571917f60f8f, UID in object meta: , requeuing"
Apr 11 21:56:04 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:04Z" level=error msg="Failed to record snapshots for cluster: nodes \"ip-172-31-16-97\" not found"
Apr 11 21:56:06 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:06Z" level=error msg="error syncing 'local-etcd-snapshot-ip-172-31-16-97-1712872562-4d0b13': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-etcd-snapshot-ip-172-31-16-97-1712872562-4d0b13\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-etcd-snapshot-ip-172-31-16-97-1712872562-4d0b13, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8e45277b-8e84-4e8b-ac44-5984e7d340a9, UID in object meta: , requeuing"
Apr 11 21:56:07 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:07Z" level=error msg="error syncing 'local-on-demand-ip-172-31-16-97-1712872526-9574ea': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-on-demand-ip-172-31-16-97-1712872526-9574ea\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-on-demand-ip-172-31-16-97-1712872526-9574ea, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 66960485-96e8-4068-ad73-295a95a48b8d, UID in object meta: , requeuing"
Apr 11 21:56:07 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:07Z" level=error msg="error syncing 'local-on-demand-ip-172-31-16-97-1712872518-9651cf': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-on-demand-ip-172-31-16-97-1712872518-9651cf\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-on-demand-ip-172-31-16-97-1712872518-9651cf, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 85ce3d94-aae9-400d-9598-3a1dad6f2c23, UID in object meta: , requeuing"
Apr 11 21:56:08 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:08Z" level=error msg="error syncing 'local-on-demand-ip-172-31-16-97-1712872534-fe3ecf': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-on-demand-ip-172-31-16-97-1712872534-fe3ecf\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-on-demand-ip-172-31-16-97-1712872534-fe3ecf, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b5a8ef96-195f-44fd-9c22-7ac9d5624e6a, UID in object meta: , requeuing"
Apr 11 21:56:08 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:08Z" level=error msg="error syncing 'local-etcd-snapshot-ip-172-31-16-97-1712872505-428a6c': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-etcd-snapshot-ip-172-31-16-97-1712872505-428a6c\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-etcd-snapshot-ip-172-31-16-97-1712872505-428a6c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: d8816b7c-55bf-44d1-a6bd-7d2738a635d7, UID in object meta: , requeuing"
Apr 11 21:56:09 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:09Z" level=error msg="error syncing 'local-on-demand-ip-172-31-16-97-1712872526-9574ea': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-on-demand-ip-172-31-16-97-1712872526-9574ea\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-on-demand-ip-172-31-16-97-1712872526-9574ea, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: af610bcb-f848-4552-b4d3-f769871b5648, UID in object meta: , requeuing"
Apr 11 21:56:10 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:10Z" level=error msg="error syncing 'local-on-demand-ip-172-31-16-97-1712872518-9651cf': handler managed-etcd-snapshots-controller: Operation cannot be fulfilled on etcdsnapshotfiles.k3s.cattle.io \"local-on-demand-ip-172-31-16-97-1712872518-9651cf\": StorageError: invalid object, Code: 4, Key: /registry/k3s.cattle.io/etcdsnapshotfiles/local-on-demand-ip-172-31-16-97-1712872518-9651cf, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 258dd772-7245-4fbf-ab23-faf18ca557c8, UID in object meta: , requeuing"
Apr 11 21:56:10 ip-172-31-16-97 k3s[8451]: time="2024-04-11T21:56:10Z" level=error msg="Failed to take scheduled snapshot: nodes \"ip-172-31-16-97\" not found"

Validation Results:

k3s version used for validation:

$ k3s -v
k3s version v1.29.3+k3s-3f906bee (3f906bee)
go version go1.21.8

We do not see the repeat error messages on reque / reconciliation here:
 $ sudo journalctl -xeu k3s | grep 'Failed to reconcile snapshot ConfigMap: no nodes have reconciled ETCDSnapshotFile resources, requeuing'

 $ sudo journalctl -xeu k3s | grep error | grep snapshot
Apr 11 22:21:27 ip-172-31-18-168 k3s[24196]: time="2024-04-11T22:21:27Z" level=debug msg="Error encountered attempting to retrieve extra metadata from k3s-etcd-snapshot-extra-metadata ConfigMap, error: configmaps \"k3s-etcd-snapshot-extra-metadata\" not found"


Fix dynamic SubAccess of zero-length vectors

Type of change: bugfix
API impact: none
Backend code-generation impact: none
Desired merge strategy: squash
Summary: as described in #230, RemoveAccesses currently produces broken code when encountering (legal) out-of-bounds dynamic accesses to zero-length vectors. This PR adds a new pass ZeroLengthVecs that handles these cases, both as sources of data and as sinks of assignments.
Since this pass is Dependency-native, I would welcome any input on @seldridge on the dependency / LoweringCompilersSpec stuff :)
Currently, the new pass itself declares what I believe is the minimal set of prereqs. It gets run as part of the High -> Mid compiler, and it is a prereq for RemoveAccesses.
Contributor Checklist

[x] Did you add Scaladoc to every public function/method?
[x] Did you add at least one test demonstrating the PR?
[x] Did you delete any extraneous printlns/debugging code?
[x] Did you specify the type of improvement?
[x] Did you state the API impact?
[x] Did you specify the code generation impact?
[x] Did you request a desired merge strategy?

Reviewer Checklist (only modified by reviewer)

[ ] Did you add the appropriate labels?
[ ] Did you mark the proper milestone (1.2.x, 1.3.0, 1.4.0) ?
[ ] Did you review?
[ ] Did you mark as Please Merge?

@albert-magyar: What's going on with LoweringCompilersSpec is that if you add a transform, you then need to increase all the "line numbers" for the transform order patches. To describe what you're doing the following diff works:
diff --git a/src/test/scala/firrtlTests/LoweringCompilersSpec.scala b/src/test/scala/firrtlTests/LoweringCompilersSpec.scala
index 2fa69903..648e45cd 100644
--- a/src/test/scala/firrtlTests/LoweringCompilersSpec.scala
+++ b/src/test/scala/firrtlTests/LoweringCompilersSpec.scala
@@ -61,6 +61,7 @@ class LoweringCompilersSpec extends FlatSpec with Matchers {
       passes.PullMuxes,
       passes.ReplaceAccesses,
       passes.ExpandConnects,
+      passes.ZeroLengthVecs,
       passes.RemoveAccesses,
       passes.Uniquify,
       passes.ExpandWhens,
@@ -156,18 +157,17 @@ class LoweringCompilersSpec extends FlatSpec with Matchers {
   it should "replicate the old order" in {
     val tm = new TransformManager(Forms.MidForm, Forms.Deduped)
     val patches = Seq(
-      Add(3, Seq(Dependency(firrtl.passes.ZeroLengthVecs))),
-      Add(5, Seq(Dependency(firrtl.passes.ResolveKinds),
+      Add(6, Seq(Dependency(firrtl.passes.ResolveKinds),
                  Dependency(firrtl.passes.InferTypes))),
-      Del(6),
       Del(7),
-      Add(6, Seq(Dependency[firrtl.passes.ExpandWhensAndCheck])),
-      Del(10),
+      Del(8),
+      Add(7, Seq(Dependency[firrtl.passes.ExpandWhensAndCheck])),
       Del(11),
       Del(12),
-      Add(11, Seq(Dependency(firrtl.passes.ResolveFlows),
+      Del(13),
+      Add(12, Seq(Dependency(firrtl.passes.ResolveFlows),
                   Dependency[firrtl.passes.InferWidths])),
-      Del(13)
+      Del(14)
     )
     compare(legacyTransforms(new HighFirrtlToMiddleFirrtl), tm, patches)
   }

Or: I didn't write this "diff" with any fuzz in mind...
I'm a-okay with using a minimal set of prerequisites. I would like to do that for all transforms as part of a 1.4 activity. The current dependencies are, admittedly, over-restrictive as a safe means of replicating the old order.


[test-framework] Remove adding duplicate workload to echo workloads on update

Fixes #30880.
#30675 introduced an issue to echo Restart where we'd continually add a new workload for the same pod on every pod state change, resulting in 8-9 workloads rather than the expected 1. Removing the line below fixes the issue for my case but I may not understand the intention behind adding the old workload back in
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[x] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[x] Does not have any changes that may affect Istio users.

Nathan and I discussed and this isn't the right fix, closing in favor of separate PR


Deployment of service

We need to deploy the service in any cloud (Azure, ...). It would be great if we have a test stage and a production stage and GitHub Actions are used to deploy the master automatically in the test stage and tags by hand to production.
@gdams do we have such Actions already in use somewhere? @johnoliver how is the API deployed?

@gdams since we have the first working version ready it would make sense to find at a home for the service :) For development the service is running on OpenLiberty but we can use any Microprofile compatible runtime for production. I would like to have it ready and integrated together with all other new website services in Q1 2022. How can we proceed here?

Looks like Heroku provides a good tooling: https://github.com/adoptium/documentation-services/issues/53

blocked by https://github.com/OpenLiberty/ci.maven/issues/1327


Split trust location flag for input and for pileup data - 1.0.14 version

1.0.14 version of #6752 + #6774
I'll start running testbed tests with it now. If everything goes Ok, then we have to patch all the production agents during the cmsweb production deployment.

You can ignore the python 2.7 stuff on the branches for now since it's probably complaining about stuff that exists in the trunk, not introduced by the commit.

@ticoann can be merged, please. I'll ask for a new tag later today/tomorrow, once we get other PRs validated for 1.0.14


Allow to unset the current-context (-u?)

It would be nice if we could select current-context to empty, so unset the current-context, as for safety reasons, that would be similar (in effects) to logging off. This way, you can prevent commands being executed agains a cluster if any of the automation you're running does accidentally try to do something.
My workflow is similar to:

going to work on something (login), so do kubectx my-context
do my work
finished working on a cluster (logout), so do kubectx -off (OR SIMILAR)

Using kubectx -d <CTX> does delete the context, but not unsets the current-context, so still has a value to a non existing context. What I'm requesting is the opposite, retain the context, but unset the value from current-context.

Does kubectx “” (empty string as arg) work?

kubectx ""
error: empty context names are not allowed

@jorgemoralespou hmm looks like a this is a kubectl limitation. We primarily use kubectl to interact with kubeconfig, and never edit it directly (for a lot of reasons).
kubectl config set current-context ""
error: you cannot use set to unset a property

kubectl config use-context ""
error: empty context names are not allowed

It seems like kubectl can't do this. :(
On the other hand, given ~100k users we have and we're receiving this request for the first time, I'm inclined to think this is rather an exotic request. You can achieve what you want by setting something like KUBECONFIG=/dev/null in your shell.

@ahmetb the command you're looking for is kubectl config unset current-context

I actually just came here looking for the same thing. It could also be useful to have a flag to specify the fact that you want to unset the current context, maybe something like -u:
kubectx -u

Thanks, yeah I am currently still -1 for adding a feature for this to kubectx, need more evidence that this is a real user need (especially since kubectl can already do it).

Thanks, yeah I am currently still -1 for adding a feature for this to kubectx, need more evidence that this is a real user need (especially since kubectl can already do it).

That's fair. I have just gotten used to manipulating my context through kubectx since it makes it much easier.

kubectl can already do it

kubectl can already do config use-context too, I'm here for the shell sugar!

Totally agree. I'm really surprised this has not come up before, as leaving the context set is almost as being always logged into the cluster. Any thing that we do might be executed on the current context/cluster we have set.
I understand that many users don't work with multiple clusters, and that when you have PS1 indicating you the context you have selected is really obvious you want it set/unset when working/not working on stuff on the cluster.

I think simply, most users don't want to unset kubectl, because it leaves kubectl malfunctional. Most people dealing with multiple kubeconfig files can simply unset KUBECONFIG and get the same behavior. This is probably why it didn't come up before.

Yeah that is spot on. Leaving kubectl (and other client-go tools which understand kubeconfig) malfunctioning is precisely my use case. For those of us not using multiple config files unsetting the current-context is the next best thing.

After some thinking, I think I'm open to this.
Let's pursue this in #187 (it seems like we need to add integration tests and polish messages, readme etc).

Just found out that this feature exists, just for the record, I really like it because I work with multiple clusters and multiple customers and it is mind easing to unset context after I finish a task in one environment so when I need to do something somewhere else I have to explicitly set the context back, so I can not run something on a different cluster (context) accidentally, so thank you.
It would be even better to have kubectx (and kubens too) be shell session aware.


chore: Update CODEOWNERS file

Which problem is this pull request solving?
Replacing @netlify/compute with @netlify/runtime-pod-compute
List other issues or pull requests related to this problem
This relates to https://github.com/netlify/pod-compute/issues/89

[x] I have read the contribution guidelines.
[x] The status checks are successful (continuous integration). Those can be seen below.

A picture of a cute animal (not mandatory but encouraged)

@eduardoboucas can I merge this? :) I don't know what the guidelines are for this repo

Go for it! We also use Kodiak on most repos, so you could add the automerge label and the PR will get merged automatically when all the checks pass.


Feature Request: Integrate Trivy Vulnerability Scanner into sbomqs`

Description:
sbomqs currently lacks a vulnerability scanner, which is essential for identifying potential security risks in the software components. Integrating a Trivy vulnerability scanner can enhance the security assessment capabilities of sbomqs.
currently we scan sbomqs images manually using trivy and found one vulnerability as mentioned in this issue #298
Importance of Trivy Scan:
Trivy is a comprehensive and easy-to-use vulnerability scanner for container images, file systems, and Git repositories, suitable for detecting vulnerabilities, misconfigurations, and secrets.
Solution
Integrate the Trivy vulnerability scanner into sbomqs to automatically scan the SBOM for vulnerabilities in the components.
https://github.com/marketplace/actions/aqua-security-trivy

@riteshnoronha , should we integrate this ?


Bump Wasmtime to 13.0.0

This is an automated pull request from CI which indicates
that the next release-12.0.0 branch has been created and
the main branch is getting its version number bumped from 12.0.0 to
13.0.0.
Maintainers should take a moment to review the release
notes for 12.0.0, and if any changes are necessary send
PRs to the main branch to update RELEASES.md and then backport
these PRs to the release branch.
Maintainers should also review that aarch64-apple-darwin builds
are passing via embark's CI.
Another automated PR will be made in roughly 2 weeks time when for
the actual release itself.
If any issues arise on the main branch before the release is made
then the issue should first be fixed on main and then backported
to the release-12.0.0 branch.

I manually triggered this to verify that https://github.com/bytecodealliance/wasmtime/pull/6687 works. If CI passes in full I'll clean up this branch and close this.

Hurray green CI!


Fix getPrimaryKeys() to work with single unnamed primary key

Fixes #394

LGTM. Thanks for the fix!


Add support WebP images

https://developers.google.com/speed/webp/
WebP is natively supported in Google Chrome and the Opera browser.
Todo: based on header on the request

Accept:image/webp
Accept:image/webp,image/*,*/*;q=0.8


TX/RX backlight without timeout

It would be great to have TX/RX backlight without timeout or with separate timeout.
E.g. I have 5 seconds timeout which is good for battery saving when interacting with the radio, but during TX/RX the backlight switches off after 5 seconds, which is quite annoying and I e.g. cannot read the s-meter. Workaround is to set higher timeout.

Or maybe the backlight counter could start after the TX/RX condition is cleared (if the TX+RX backlight is enabled).

I have side button 2 set to do nothing on short press. I know it's not what you're looking for but at least it lets you wake up the backlight without doing anything else.


Disconnect in names between feature group and overview-alpha

In the table the gene names are different on the list vs the overview page.

the name is coming from the Figfam and not the plfam, but we had to go to production to see that.

Verified.


Update installation.md

Note the importance of a correct hostname

I added something similar already.
https://merlinz01.github.io/redpepper/installation.html

It doesn't work to just update the hostname.  You have to regenerate all the certs, and it's really confusing.  That's why it needs to be at the top of the instructions.

I have updated both of the affected scripts so that future users will be asked to verify the hostname.


[image_picker] Using image_picker getVideo method getted is jpg file

Steps to Reproduce

install image_picker 0.6.7 version
using getVideo source is gallery
print then value.path

The bug only Android

Logs

I/hwaps   (23361): JNI_OnLoad
W/IInputConnectionWrapper(23361): reportFullscreenMode on inexistent InputConnection
W/IInputConnectionWrapper(23361): finishComposingText on inactive InputConnection
I/flutter (23361): /data/user/0/com.bytegem.snsmax/cache/image_picker2121108544.jpg
I/OpenGLRenderer(23361): Initialized EGL, version 1.4

[✓] Flutter (Channel stable, v1.17.1, on Mac OS X 10.15.5 19F96, locale
    zh-Hans-CN)
    • Flutter version 1.17.1 at /Users/seven/Desktop/work/flutter
    • Framework revision f7a6a7906b (3 weeks ago), 2020-05-12 18:39:00 -0700
    • Engine revision 6bc433c6b6
    • Dart version 2.8.2

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /Users/seven/Library/Android/sdk
    • Platform android-29, build-tools 29.0.3
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.5)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.5, Build version 11E608c
    • CocoaPods version 1.9.1

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 46.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.45.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.11.0

[!] Proxy Configuration
    • HTTP_PROXY is set
    ! NO_PROXY is not set

[✓] Connected device (1 available)
    • HUAWEI CAZ AL10 • XPUDU16C08014500 • android-arm64 • Android 7.0 (API 24)

my codes:
final picker = ImagePicker();
                picker.getVideo(source: ImageSource.gallery, ).then((value) {
                  print(value.path);
                });

It is normal to use the camera for video recording, and the path obtained after selecting the video from the album is the jpg suffix

Hi @medz
follow up on https://github.com/flutter/flutter/issues/52419,
I'm closing the current one as duplicate.
If you disagree please write in the comments and I will reopen it.


ci: reduce number of e2e running concurrently but without increasing total time for PR checks

Description
This pull request changes the following:

update flow-pull-request-checks.yaml and flow-build-application.yaml

make e2e-mirror-node-tests run after e2e-tests
make e2e-relay-tests run after e2e-mirror-node-tests

Related Issues

Closes #366

:tada: This PR is included in version 0.27.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Bug in loading quarterly and yearly time series data

Description
It looks like recent changes (https://github.com/awslabs/gluon-ts/pull/253) introduced a bug in loading yearly, quarterly data. The loaded train and test datasets have monthly frequency instead of yearly ("12M") or quarterly ("3M") frequency.
This seems to be the culprit: https://github.com/awslabs/gluon-ts/blob/master/src/gluonts/dataset/common.py#L330
I actually came across this while doing evaluations for yearly data. Model trained with DeepAREstimator failed during evaluation with the following error. It shows the frequency here correctly in the error message since the estimator gets it from the metadata but the actual train and test datasets have wrong frequency.
AssertionError: Cannot extract prediction target since the index of forecast is outside the index of target
Index of forecast: DatetimeIndex(['1752-07-31', '1753-07-31', '1754-07-31', '1755-07-31',
               '1756-07-31', '1757-07-31'],
              dtype='datetime64[ns]', freq='12M')
 Index of target: DatetimeIndex(['1749-12-31', '1750-12-31', '1751-12-31', '1752-12-31',
               '1753-12-31', '1754-12-31', '1755-12-31', '1756-12-31',
               '1757-12-31', '1758-12-31', '1759-12-31', '1760-12-31',
               '1761-12-31', '1762-12-31', '1763-12-31', '1764-12-31',
               '1765-12-31', '1766-12-31', '1767-12-31', '1768-12-31',
               '1769-12-31', '1770-12-31', '1771-12-31', '1772-12-31',
               '1773-12-31', '1774-12-31', '1775-12-31', '1776-12-31',
               '1777-12-31', '1778-12-31', '1779-12-31', '1780-12-31',
               '1781-12-31', '1782-12-31', '1783-12-31', '1784-12-31',
               '1785-12-31'],
              dtype='datetime64[ns]', freq='12M')

To Reproduce
In [1]: import gluonts

In [2]: from gluonts.dataset.repository.datasets import get_dataset

In [3]: dataset = get_dataset("m4_yearly", regenerate=False)

In [4]: next(iter(dataset.test))['start']
Out[4]: Timestamp('1749-12-31 00:00:00', freq='M')

In [5]: dataset.metadata.freq
Out[5]: '12M'

In [6]: dataset = get_dataset("m4_quarterly", regenerate=False)

In [7]: next(iter(dataset.test))['start']
Out[7]: Timestamp('1749-12-31 00:00:00', freq='M')

Since start field had wrong frequency Forecast start date was wrongly set here by shifting 31 months instead of  instead of 31 years in case of m4_yearly data. This explains why the forecast index started at 1752-07-31 as shown in the error message instead of 1780-12-31.

It's not clear why ProcessStartField.process needs to have all that logic: we could use some documentation explaining why simply doing
return pd.Timestamp(string, freq=freq)

is not sufficient

AFAIK, we do this processing, to align all timestamps within the same period.
E.g. all timestamps in August 2019 should to 2019-08-31:
In [32]: to_offset("M").rollback(pd.Timestamp.now())
Out[32]: Timestamp('2019-08-31 12:57:48.139117')

As we can see, rollback doesn't affect the time information, that's why we call replace on it to reset these values:
In [34]: _32.replace(hour=0, minute=0, second=0, microsecond=0, nanosecond=0)
Out[34]: Timestamp('2019-08-31 00:00:00')

That being said, I think 12M still only aligns the data on a monthly bases and not a yearly one:
In [48]: to_offset("12M").rollback(pd.Timestamp.now())
Out[48]: Timestamp('2019-08-31 13:09:39.763214')

vs
In [49]: to_offset("Y").rollback(pd.Timestamp.now())
Out[49]: Timestamp('2018-12-31 13:09:59.602807')

It's not clear why ProcessStartField.process needs to have all that logic: we could use some documentation explaining why simply doing
return pd.Timestamp(string, freq=freq)

is not sufficient

I've tried running the tests with your suggestion and the only tests failing are the ones, which explicitly test for the alignment. Maybe @vafl can tell us more, why we need to align the timestamps?

The main reason for aligning the time stamps are the time dependent features. These are quite expensive to compute and so we cache the feature values for all time points in a feature matrix:
https://github.com/awslabs/gluon-ts/blob/4d044a2dcd9f3f5282e72a5324db12a6b7b30c50/src/gluonts/transform.py#L840
When we add time features in the transformation, we do a lookup of the start time point and then just index into the feature matrix to return the corresponding part. This cache would not work if time points are not aligned with a regular grid.
Maybe we can improve the alignment logic taking into account the multiple?


Automatic Cell Size with UITextView

New issue checklist

[ ] I have reviewed the README and documentation
[ ] I have searched existing issues and this is not a duplicate

General information

IGListKit version: 2.1.0
iOS version(s): 9.0 - 10.0
CocoaPods/Carthage version: 1.1.1
Xcode version: 8.2.1
Devices/Simulators affected: iPhone 7
Reproducible in the demo project? (Yes/No): No
Related issues: --

I am using IGListKit extensively in my entire iOS App. What I would want to achieve is a Text View inside a CollectionCell and have it auto size, by the dynamic text contained in that Text View. I saw in the Instagram app, that you have this feature built in, are you using IGListKit in the Edit Porfile Screen. If yes I would really appreciate, how you pulled it off.
Thank in advance.

@dennisgec we aren't using IGListKit to size our editable cells yet, this is a good question though! We probably need to get #459 working for this to work really well since reloading the cell will recycle the view.
Going to get a PR up for that very soon!

@rnystrom Good to know, thanks already for getting back that quickly.

Closing to track at #516


[WIP] Registration Updates: Provide visible labels or instructions for all input controls.

Description
The form fields on the Register page do not have on-screen labels. Placeholder text is used as on-screen labels. When users start writing, the placeholder text disappears. People with cognitive disabilities will benefit from on-screen labels to enter the proper input.
Notes
To simplify textfield design, created a custom textField with desired look to avoid overhead of managing a separate imageView for background and padding.

RegistrationTextField.swift

Objective-C

OEXRegistrationFieldWrapperView.h
OEXRegistrationFieldWrapperView.m
OEXRegistrationFormTextField.h
OEXRegistrationFormTextField.m
OEXRegistrationFieldEmailView.h
OEXRegistrationFieldEmailView.m
OEXRegistrationFieldPasswordView.h
OEXRegistrationFieldPasswordView.m
OEXRegistrationFieldTextAreaView.h
OEXRegistrationFieldTextAreaView.m

These views were on same pattern. So to simplify the code i have converted it into Swift and merged similar code in same class.
Swift

RegistrationFormFieldView.swift

Objective-C

OEXRegistrationFieldEmailController.h
OEXRegistrationFieldEmailController.m
OEXRegistrationFieldTextController.h
OEXRegistrationFieldTextController.m
OEXRegistrationFieldPasswordController.h
OEXRegistrationFieldPasswordController.m
OEXRegistrationFieldTextAreaController.h
OEXRegistrationFieldTextAreaController.m

These Controllers were on same pattern. So to simplify the code i have converted it into Swift and merged similar code in same class.
Swift

RegistrationFieldController.swift

How to test this PR

Goto Register Screen.
Verify Design - Input control having on-screen labels (* for required fields).
Verify Input validations.

@zeshanarifarbisoft It seems like people often overlook the information in the comment the bot added to the pull request.  What could we do to improve it?

Test cases are missing, need to run the gradle command and generate the new screenshot test cases.

@zeshanarifarbisoft It's important to respond to comments and questions on the pull request.  Did you see the paragraphs the bot added at the very beginning?

@salman2013 please review PR, suggested changes has been made.

@nedbat
I have added as much information as possible in PR description, I am working on edX with Arbisoft mobile team. i ll definitely let you know if i have suggestions to improve information added by bot to catch attention.

@zeshanarifarbisoft The bot said:

We can't start reviewing your pull request until you've submitted a signed contributor agreement or indicated your institutional affiliation.

Can you respond to that?

@zeshanarifarbisoft Please look why the screenshot test is updating for "Resources" Tab

@nedbat
i have sent signed individual Contributor Agreement at<EMAIL_ADDRESS>

@salman2013
requested changes has been updated and pushed, you can review it again.

@saeedbashir you can review this now as i am done with review. Thanks

@zeshanarifarbisoft Thanks, we've received your contributor agreement.

@saeedbashir Requested changes are addressed. you can review again.

@zeshanarifarbisoft 🎉 Your pull request was merged!
Please take a moment to answer a two question survey so we can improve your experience in the future.


Fix runtime apis with default parameter change

Purpose
Fixes #36559
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Closing since new Pr sent https://github.com/ballerina-platform/ballerina-lang/pull/36836


Issue with converting local realm to flexible sync realm

What happened?
So i have this app that uses local config as I want to offer a fully offline first approach in my app. Now I am adding flexible sync config on the basis whether user wants to login so all that works fine. But the problem happens when the data created under local config does not copy to flexible sync so I looked into MongoDB docs but found this.
Following the way of JS/TS code throws me the realm exception as:
RealmException (RealmException: Error code: 2007 . Message: Realm cannot be converted to a flexible sync realm unless flexible sync is already enabled)

Because switching from local to flexible would be very difficult for users and it will make flexible sync useless for them since no data was saved unless backed up externally which can only be opened in realm studio or a local config based app with same schema
Repro steps

Initiate local realm
add flexible sync
login to a user
start copyTo
throws exception

Version
3.13.8
What Atlas Services are you using?
Both Atlas Device Sync and Atlas App Services
What type of application is this?
Flutter Application
Client OS and version
Android 13/Windows 11
Code snippets
  Realm getSyncedRealm(App app) {
    // if (app.currentUser != null) {
    currentUser = app.currentUser;
    final Configuration syncConfig = Configuration.flexibleSync(
      app.currentUser!,
      realmSchemas,
    );
    final localConfig = Configuration.local(
      realmSchemas,
      schemaVersion: 11, // increase this number after updating any model
    );
    final localRealm = Realm.open(localConfig);
    localRealm.then((realm) => realm.writeCopy(syncConfig));

    notifyListeners();

    return Realm(syncConfig);
  }

Stacktrace of the exception/crash you're getting
No response
Relevant log output
No response

Perhaps its best to keep your local and device sync realm db's separate, each with their own schema - then you could make a function to copy each db document programatically from the local to device sync. The JS/TS Realm SDKs far before the Dart/Flutter SDK, so there is occasionally functionality that is missing (yet to come) in the Dart/Flutter SDK. Hope this helps.

Perhaps its best to keep your local and device sync realm db's separate, each with their own schema

Then it defeats the purpose of my app. The realm files are already stored in different locations for both local and flexible sync.
Even tried to fetch all documents written in the local configuration and tried running the write > addAll but it then throws:
cannot add object to realm because the object is managed by another realm instance

cannot add object to realm because the object is managed by another realm instance
-> If you try deep copying each document (i.e. create new instances of each document) before you write > addAll

If you try deep copying each document (i.e. create new instances of each document) before you write > addAll

Can you give some sample code?

If you try deep copying each document (i.e. create new instances of each document) before you write > addAll

Can you give some sample code?

Something like the below, FYI ive not spun this up, a rough quick draft but should give you the right idea. You'll obviously have to realm generate _CatOne and _CatTwo so CatOne and CatTwo can be utilised.
/// Cat One Model
@RealmModel()
@MapTo("CatOne")
class _CatOne {

  /// The id (unique identifier)
  @PrimaryKey()
  late ObjectId id;

  /// Cat Name
  late String name;
}

/// Cat Two Model
@RealmModel()
@MapTo("CatTwo")
class _CatTwo {

  /// The id (unique identifier)
  @PrimaryKey()
  late ObjectId id;

  /// Cat Name
  late String name;
}

/// Deep Copy Cate One List to Cat Two List
List<CatTwo> deepCopyCatOneListToCatTwoList({
  required final List<CatOne> catOneList,
}) {
  List<CatTwo> catTwoList = <CatTwo>[];

  // Process all Cat One Items
  for (int x = 0; x < catOneList.length; x++) {
    final CatOne catOne = catOneList[x];

    // Create Cat 2 Object
    catTwoList.add(
      CatTwo(
          catOne.id, // ideally a new id here and reference cat_one_id as second key
          catOne.name
      ),
    );
  }

  return catTwoList;
}

// Then after this write all catTwoList to your 2nd Realm local DB in a write async block

So i did it in the following way.
I retrieved all the instances using .all() and assigned to x then from the x, I created a new list(y) and added each instance from x to y and then did realm.write -> realm.add(y.e).
Could I have done it better? maybe not creating a redundant list y and directly writing from x?

Sounds good

Why it got closed? The actual solution does not work. The one that worked is merely a workaround.

Because this is not something that can be done in the flutter SDK. We need it to be supported by the Core database first, which is tracked in https://github.com/realm/realm-core/issues/5711 (I can see that the issues didn't get linked correctly on Github).


Issue parsing matching data in subtree filter for instance-identifier

I am trying to parse data using matching subtree filter passed but getting issue in getting a match for instance-identifier.
I am using latest version (1.1.36) of libyang but the result is same. Though code used for Netopeer2 server is previous version of internal implemented filter APIs (my project limitation is not to use sysrepo library).
Attached are my code files for reference. Request you to please provide your inputs.
The issue is not observed when the instance-identifier value doesn't have a container at the top and matching data gets printed:
$ cat stream.yang
module stream {
namespace "urn:test:params:xml:ns:yang:stream";
prefix stream;

    list subject {
      key "name";
      leaf name {
        type string;
        description
          "The name of subject.";
      }
    }
}

Filter and Data is:
char *inputStr = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
  <rpc xmlns=\"urn:ietf:params:xml:ns:netconf:base:1.0\" message-id=\"1\">
    <get>
      <filter type=\"subtree\">
        <student:student-data xmlns:student=\"urn:test:params:xml:ns:yang:student\">
          <student:resource xmlns:stream=\"urn:test:params:xml:ns:yang:stream\">/stream:subject[name='medical']</student:resource>
        </student:student-data>
      </filter>
    </get>
  </rpc>";

char *data = "
  <student-data xmlns=\"urn:test:params:xml:ns:yang:student\">
    <id>7</id>
    <resource xmlns:stream=\"urn:test:params:xml:ns:yang:stream\">/stream:subject[stream:name='medical']</resource>
  </student-data>";

But when container is present then there is blank data after matching:
onl-fogwall-112:code (master %=)]$ cat stream.yang
module stream {
namespace "urn:test:params:xml:ns:yang:stream";
prefix stream;
  container roles {
    list subject {
      key "name";
      leaf name {
        type string;
        description
          "The name of subject.";
      }
    }
  }
}

Filter and Data is:
char *inputStr = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
  <rpc xmlns=\"urn:ietf:params:xml:ns:netconf:base:1.0\" message-id=\"1\">
    <get>
      <filter type=\"subtree\">
        <student:student-data xmlns:student=\"urn:test:params:xml:ns:yang:student\">
          <student:resource xmlns:stream=\"urn:test:params:xml:ns:yang:stream\">/stream:roles/stream:subject[name='medical']</student:resource>
        </student:student-data>
      </filter>
    </get>
  </rpc>";

char *data = "
  <student-data xmlns=\"urn:test:params:xml:ns:yang:student\">
    <id>7</id>
    <resource xmlns:stream=\"urn:test:params:xml:ns:yang:stream\">/stream:roles/stream:subject[stream:name='medical']</resource>
  </student-data>";

Note: Also observed, when root container is not present in instance-identifier then matching data is printed twice.
Code having matching data output.zip
Code having no data output.zip

Hi,
so, the 2 duplicate instances should be fixed in the referenced commit. However, the second case, with no match, will not be fixed (now). The problem is actually a really complex one, concerning instance-identifiers and how to represent them in XPath. I can only suggest you change
/stream:roles/stream:subject[name='medical']

to
/stream:roles/subject[name='medical']

and it is going to work.
We are working on a new version of libyang where all the types are implemented in a much more robust way and it would work there. But it is almost impossible to cover all the use-cases in the current libyang.
Regards,
Michal

@michalvasko : Thank you very much for the quick fix, yes both the provided solutions are working for me.
Can you provide a tentative date by which the robust solution as mentioned above will be available to users.

Hi,
we are hoping to have some working version by the end of the year.
Regards,
Michal

Thanks Michal, closing this issue.


[build] Add CodeQL Security Scan workflow

Summary, imperative, start upper case, don't end with a period
Add a workflow to run CodeQL security scan every Monday,Wednesday,Friday evening.
How was this PR tested?
Tested in my fork: https://github.com/huangminchn/venice/actions/workflows/codeql.yml
Does this PR introduce any user-facing changes?

[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.

What does this CodelQL scan do? From Google search looks like it is similar spot bug?

@huangminchn  Hey Min, this PR hasn't been inactive for a while so let me close it so our review queue could be cleaner. Please re-open it when you have the bandwidth working on it again.


Calling reset_to_usb_boot does not properly reset the device

A call to reset_to_usb_boot somehow locks up the device, but it will not properly reboot and show up on USB. Maybe some preparation is required, but looking at the pico-sdk I could not find anything special.
https://github.com/JeremyGrosser/rp2040_hal/blob/02c3d00151ebe62777ffe9a525d36d7f9473aa2b/src/drivers/rp-rom.ads#L154

The function should be marked as "No_Return", but this does not fix the issue.

I am able to reproduce this issue. reset_to_usb_boot does work if called before RP.Clock.Initialize, but not after.
I think the _usb_boot and subsequent _usb_clock_setup procedures in the bootrom are expecting the clocks to be in a specific state before entry and we don't have an easy way to get back to the default state after RP.Clock.Initialize

@JeremyGrosser thanks for the hint. pico-sdk does not disable the ROSC. So if we remove line 127 in rp-clock.adb or if ROSC is re-enabled right before the call to reset_to_usb_boot the issue is fixed.
https://github.com/JeremyGrosser/rp2040_hal/blob/b629518ee7b44d14a246b533a0eda6469fe67dd8/src/drivers/rp-clock.adb#L127


Add Node to drupal web image.

root@577fd2d4bc81:/app# node -v
v6.9.1
root@577fd2d4bc81:/app# npm -v
3.10.8

Resolves https://github.com/continuouspipe/drupal8-lightning/issues/19

This is ok to merge (but I don't have access)


Heroku client internal error (Did not recognize your adapter specification)

Am having some trouble with heroku pg:psql:
>heroku pg:psql -a APPNAME
 !    Heroku client internal error.
 !    Search for help at: https://help.heroku.com
 !    Or report a bug at: https://github.com/heroku/heroku/issues/new

    Error:       Did not recognize your adapter specification (126: The specified module could not be found.   - C:/Program Files (x86)/Heroku/vendor/gems/json-
1.7.7/lib/json/ext/parser.so). (MultiJson::AdapterError)
    Command:     heroku pg:psql -a boxgreen-prod
    Plugins:     heroku-repo
    Version:     heroku/toolbelt/3.43.3 (i386-mingw32) ruby/2.1.7
    Error ID:    75a70242fbf54243b395565dfecfc5c2

>heroku version
heroku/toolbelt/3.43.3 (i386-mingw32) ruby/2.1.7
heroku-cli/5.2.20-9d094b0 (windows-386) go1.6.2
=== Installed Plugins
heroku-repo

Tried heroku update and downloading the latest toolbelt from the website and reinstalling, both of which did not help.

Did this ever get resolved? I'm having a similar error message.

Nope, moved on Windows so it's not an issue any more.

I am getting a similar issue... any suggestions?

MacBook-Pro:node-js-getting-started jameseisenlohr$ heroku pg:psql
!    Heroku client internal error.
!    Search for help at: https://help.heroku.com
!    Or report a bug at: https://github.com/heroku/heroku/issues/new
Error:       Did not recognize your adapter specification (dlopen(/usr/local/heroku/vendor/gems/json-1.8.2/lib/json/ext/parser.bundle, 9): Library not loaded: /usr/local/opt/gmp/lib/libgmp.10.dylib

Referenced from: /usr/local/heroku/vendor/gems/json-1.8.2/lib/json/ext/parser.bundle
Reason: image not found - /usr/local/heroku/vendor/gems/json-1.8.2/lib/json/ext/parser.bundle). (MultiJson::AdapterError)
Command:     heroku pg:psql
Version:     heroku-toolbelt/3.43.15 (x86_64-darwin10.8.0) ruby/1.9.3
Error ID:    1b06d6ce1e7f4c9b95cbc211de17366a
More information in /Users/jameseisenlohr/.heroku/error.log


Version website with Mike

Changes introduced with this PR
Introduce full website versioning with mike as recommended by mkdocs docs. While we do copy old version reference docs in the reference guide mike gives some distinct advantages, namely:

We can version all aspects of the site including getting started, installation guides etc
We do not have to rebuild, curate or make sure the old documentation is buildable/looks good with new mkdocs versions, mike does not rebuild old versions of the docs and simply lets them sit in the gh-pages branch.

This commit also updates the github actions workflow to automatically push the main branch to the upcoming version and any v* branches to the respective versions.
I'll merge in a weeks time unless someone stops me for a review.

By contributing to this repository, I agree to the contribution guidelines.

Indeed, that's to be handled next. One case that's currently broken is that mike doesn't add redirects for old links at the root at all AFAIK. You're running a reverse proxy for containerssh.io right? Can you add a redirect of containerssh.io/foo/bar -> containerssh.io/latest/foo?
Where foo != v* or upcoming or latest (the tags we're currently using)?

Actually I might have fixed it by copying the redirect files from latest to the root.

https://containerssh.github.io/getting-started/docker/ -> https://containerssh.github.io/v0.4/getting-started/docker/ works

For some reason  https://containerssh.github.io/getting-started/kubernetes/ -> https://containerssh.github.io/v0.4/getting-started/kubernetes/ doesn't though, weird. It's the same code.
In any case, currently containerssh.io 404s I assume that's some cache?

In any case, currently containerssh.io 404s I assume that's some cache?

To use a custom domain you need a CNAME file in the root directory of your gh-pages branch.
Currently it seems you moved the CNAME file in the v0.4-subdirectory:
https://github.com/ContainerSSH/containerssh.github.io/blob/gh-pages/v0.4/CNAME

Ah, that explains it thanks. I was under the impression that @janosdebugs was running a reverse proxy for the domain and its cache/configuration was causing the 404. Will fix it asap.


 Do you think you could come up with a creative design for this as top priority for webpack.js.org?

Sure I can start working on that.

Add a notice: "this as an alpha page, not complete, go away, follow our blog for announcements [here]"
Make sure it's visible on all browsers.
It's a much more effective way to mitigate this than to frantically design the page and try to respond to everyone on twitter

Also: once the page is done, reset all counters and then open up to the public

Yes. I agree. I'm submitting some basic content on a branch, and then merging to develop for a deploy.

This can be the baseline for @skipjack to work off of.

@TheLarkInn sorry for the delay on this... so I think @NejcZdovc has been working on this (see #444) so I don't want to overwrite any of his work. As mentioned in #430 I'm fairly busy this month but after I PR some current work I have locally I'll do whatever I can to help push this forward.
@NejcZdovc maybe you want to close #444 and move that discussion over here?

@skipjack my issue is more focused on functionality of voting then on design. I am sadly not a designer, so I can't really help on design part, but I can help implement it if you do a quick mock.
We can discuss my suggestion here as well, but I think this are two different things. On the other hand I agree that my issue should be done before this issue can be done.
If you guys like my suggestion I can easily start implementing it.

see also #447

I saw it, and I think that I can work off this PR

@skipjack my issue is more focused on functionality of voting then on design. I am sadly not a designer, so I can't really help on design part, but I can help implement it if you do a quick mock.

Lost track of this one, now that #474 is pretty much done I can try to help out more with the voting page. I'll start messing around with the design soon and open a PR.

Ok finally got a few rough drafts together for this... interested in some feedback before I go too much further. I made three states, one for what the page would look like logged out, one logged in as a normal user, and one as an admin.
Logged Out

Logged in as a Normal User

Logged in as an Admin

I kept it as single page interface for now, although down the road it might make sense to break it down into more pages. If people think this is heading in the right direction I'll add a comp for what the admin settings menu could look like and one for creating a new topic.
cc @sokra @TheLarkInn @NejcZdovc @pksjce (as I know you all have worked on the voting section)
cc @jhnns @pastelsky (as I figure you may have some design feedback)
P.S. I will push this to the design repo, even though it's deprecated, in case anyone wants to take a look at the raw PSD.

@pksjce I love it :+1:

Wow that's impressive looking. How does this look from a mobile standpoint?
Also a fun tiny low hanging fruit feature we could open to the community is to make the little logos next to the influence could be tiny colored webpack logos. Overall it looks awesome.

Okay other thoughts. As much as I like showing each of the participants I worry about page loads for all the images aka extra requests. Maybe we can show the top 5 influencers along with a "total count" of people who have voted for that feature.

How does this look from a mobile standpoint?

Wanted to get some feedback first on the desktop version, I'll add a mobile comp soon.

Also a fun tiny low hanging fruit feature we could open to the community is to make the little logos next to the influence could be tiny colored webpack logos.

Yea that could be cool, I did steal the shape for those icons from the logo.

Maybe we can show the top 5 influencers along with a "total count" of people who have voted for that feature.

Sounds good, figured we'd have to limit it a bit.
I'll come back with some updates soon and any other feedback/ideas in the meantime I'm all ears.

I think the concern from the beginning has been that the CTA area ie. the vote button is very tiny. I could not think of a way to rectify it at all :(

@pksjce

I think the concern from the beginning has been that the CTA area ie. the vote button is very tiny. I could not think of a way to rectify it at all :(

Yeah I see what you mean, and I was thinking we should definitely at least bump them up on mobile/smaller screens. Although maybe there is a better approach like using buttons or something too.

What do the little gear icons lead to?

Those would be to a settings menu only available for admins. The options would include archive, unarchive, lock, unlock, and edit. I'll share a comp for those as well. I'm going to give this until the weekend before digging back into it, so if anybody else has other feedback please let me know before then...
Also, here were some of the things I found for inspiration on Dribbble, maybe they'll spark some other ideas too:
Voting UI
Voteraction
Voting Results
Mobile Comp

Maybe we can rename the participants to sponsors and allow everyone with high vote in golden influence to be visible in this list.

@pksjce as for the admin menu, something like this:

@TheLarkInn working on a mobile version of the comp now. Should have something to share soon. Once I have that ready I think we should clean up some of the voting components, port them to another repo that can be maintained and managed separately from this one, and implement the new design. Anyone interested in maintaining that repo? @NejcZdovc @pksjce would either or both of you be?
I'd be happy to help with cleaning up and porting (I have a little bit of an obsession with refactoring/code cleanup 😆 ), but from there I probably won't have much time to tackle issues, new features etc. as there's plenty of issues that still need to be tackled here.

@skipjack this voting would still be part of webpack.org.js or could be used separately?

@NejcZdovc we'd still integrate it as a section of this site, but it would have it's own /src directory, issue management, config, etc. This would...

Take some weight off of this repo both in terms of code and issues.
Allow us to have separate maintainers.
Give us flexibility on where and how we deploy it (either here or elsewhere).
Give it space to breath, add new features to, fix issues in isolation, etc.
Allow us to version and release it in a more targeted way.

It could be consumed via an npm package or git submodule, and I think @bebraw and I would both be willing to help with the consumption and integration into this repo. There would definitely be some things to figure out but it basically already functions as a micro SPA in this site, via the <Interactive> component, and is already fairly isolated with it's own set of components and styling all under /components/vote.
There's definitely a little lifting to extract and consume it initially but I think in the long term the pros definitely outweigh the cons. It'll be nice to get started on too, as I think it'll be the first of a few things mentioned in #301 that will help keep this repo more targeted.

@skipjack ok let's do it :smiley:

Maybe set up webpack-contrib/voting-app and then consume as a package? I can do the repo if you can agree on the name.

I would go a as a npm package, yes

I'm good with that name 👍.

Done. Here. You should have admin level rights to it.

@NejcZdovc @pksjce I'll ping you once the designs are done and we can working on porting it. @bebraw any ideas on integration? Should we just have one wrapper component that imports the styling and top-level component, and renders the component in an antwar <Interactive> tag?

@skipjack ql, let me know when I can help

@skipjack Wrapper sounds like a good starting point, yeah.

Finally got a mobile comp together:

It's not perfect but I think this is a good starting point. I'm working on porting things to the new repository now, I'll create an issue for the redesign there and move over any existing vote issues from this repository.
@pksjce @NejcZdovc I created a vote-team and added you both. As soon as we finish the port I think we'd be happy for you two to take over as the core maintainers (plus anyone else from the core team like @sokra or @TheLarkInn who are interested). Then @bebraw and I will help with re-integration into this site via an npm package.

@skipjack so we should start porting now or will you do some more things in this repo?

I have it almost ready, I'll push where I'm at in the next hour or so.


Layout tab is still opening

Ver <IP_ADDRESS>.
Layout tab is still opening after fresh start. Is there any way how to save
setting and keep this tab closed?

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 27 Jan 2014 at 9:40

Closing this issue but all hope is not lost! We still want to help.
See: https://github.com/jimradford/superputty/wiki/Legacy-Imported-Issues on how to help us to help you!


Generalize process handling bit

Moves stellar-bin options into the LocalProcess class. Eventually I suspect we'll want docker modes too. Will depends on how the test plan itself shapes up.

:+1:


Maybe worth a look - new Python to JavaScript compiler

Advertised as:

Small, highly readable JavaScript code
Almost no runtime overhead w.r.t. JavaScript
Implementation of a large part of Python 3.5 including multiple inheritance, properties, **args and *args, comprehensions, bound function pointers, hierarchical modules etc.
Seamless and elegant integration with any JS library

http://www.transcrypt.org/

Thanks @pya  Yes, I was recently made aware and already reached out (see their issue nr. 1). Transcript indeed has very similar goals, follows a very similar approach and has very similar (planned) features. From what I can tell, the features that it has/will have that pyscript has not are multiple inheritance, bound functions and **kwargs. I think having at least bound functions is probably a good idea...


serverIPAddress in entries

HAR 1.2 — http://www.softwareishard.com/blog/har-12-spec/ — has schema support for serverIPAddress which is defined as:

serverIPAddress [string, optional] (new in 1.2) - IP address of the server that was connected (result of DNS resolution)

This seems like it might not be too hard to implement so I might take a stab at it if there are no objections.

The feature sounds fine to me, pull requests are welcome!

Cool. I'll try to get a PR done in the next week or two.
Also, not sure if I linked to the R package I made for Splash: https://github.com/hrbrmstr/splashr
Splash is by far the best javascript-enabled scraper I've used. Serious kudos to you and the folks who made and maintain it.

Awesome! What do you think about mentioning your project in Splash docs? I'm not sure what's a proper place for it, but for now we can add it to README, similar to scrapy-splash.

Any update about this feature?

That would be awesome, I have absolutely no idea where to start to implement that and as this issue is pretty old, I want to make sure you're not already implementing it before digging into it.
Do you have any update?
FYI, I use splash in this tool: https://github.com/CIRCL/lookyloo / https://lookyloo.circl.lu/ and having the IP address of each domain would allow me to gather more information about the potential maliciousness of a domain.
And while we are at it, crazy request: would it be possible to pass a DNS server to the docker image?


pull request for e2e test at 12/15/2016 23:26:24

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Support loading auth config for repos with no scheme in their URL

Support loading auth configuration for repos when their serverAddress
is specified with no scheme component but their entry in the configuration
file includes a scheme component.
Older versions of Docker write authentication tokens to the configuration
file including their scheme ("https://") but newer onces do not. Thus
in order to allow people to use the newer format (looking up auth for
a repo that isn't prefixed with scheme) with an older configuration file
we try prefixing the serverAddress with "http://" in some cases.
Fixes #804

Codecov Report

Merging #910 into master will increase coverage by 0.01%.
The diff coverage is 55.55%.

@@             Coverage Diff              @@
##             master     #910      +/-   ##
============================================
+ Coverage     66.04%   66.06%   +0.01%
- Complexity      769      770       +1
============================================
  Files           171      171
  Lines          3240     3247       +7
  Branches        369      371       +2
============================================
+ Hits           2140     2145       +5
- Misses          940      942       +2
  Partials        160      160

@mattnworb @caipre How does this look?


XAML rendering on UWP and Xamarin Forms

Hello, we're using Mapsui, with MbTiles on UWP and Xamarin Forms. We're very pleased thus far. Great job. We're looking to convert our static map pin (.png file) into a piece of XAML which is specific to a number/name of the Object being represented on the map. Example: if we were representing people, rather than representing a static, common .png avatar for each person on the map, we'd like to generate a new XAML TextBlock/Border indicating the name of each person, "Bob", "Jim", etc..
What's the best way to go about this? Thank you in advance folks!

Perhaps you could use a combination of LabelStyle and SymbolStyle with a SVG or polygon.

@charlenni Thanks for your work on the Xamarin Forms nuget package. Do you know of any examples I could dig through for inspiration?

Thanks to you guys. I'll work on publishing an example once this is up and running. Great work. @pauldendulk and the rest of the contributors. Thanks again.