Cleanup: inline index and reuse size vars in random strategy

Signed-off-by: Chanwit Kaewkasi<EMAIL_ADDRESS>

Coverage increased (+0.02%) to 39.74% when pulling fd643ad07909fd5c918eeca68d50791614f73d6d on chanwit:cleanup-random into 558b1e3c529069bc8a278354eed6c5160b387a6c on docker:master.

LGTM


Unify metrics

Closes #299. Unify metrics in one submodule. Note there are a lot of changed files from metrics being moved from utils into loss.metric.

Add Metric baseclass that all metrics should inherit from
All metrics can perform preprocessing (complex abs, norm) and postprocessing (training, reduce)
Metrics can be used either as metrics, or as losses (e.g. replacing torch.nn.MSELoss())
Unit tests for metrics
Add L1L2 metric

Checks to be done before submitting your PR

[x] python3 -m pytest tests/ runs successfully.
[x] black . runs successfully.
[x] make html runs successfully (in the docs/ directory).
[x] Updated docstrings related to the changes (as applicable).
[x] Added an entry to the CHANGELOG.rst.

Thanks for your review @tachella . Re. your comments:

I have removed Metric inheriting from Loss, and now Metric expects (x_net, x=None, y=None)
All metrics can be imported from deepinv.loss.metric, deepinv.metric, or deepinv.loss, so we will have backwards compatibility. For the old functions in deepinv.utils.metric, I have kept them and added deprecation errors.
lower_better is an attribute that should be defined in all implementations of metrics, this is what they do in pyiqa too. When lower_better=True (most of the metrics), it doesn't affect anything. However when lower_better=False, as is the case in SSIM, the inverse is taken when train_loss=True is passed. I have made this clearer in the docs + docstrings.


Add Clojure support for .boot files

Boot is a Clojure build tool that uses the .boot extension for Clojure code run in the Boot environment.  It would rock if .boot files were Clojure highlighted.

Fixed in https://github.com/github/linguist/pull/2254 and https://github.com/github/linguist/pull/2300 https://github.com/github/linguist/pull/2300

cant you add a option to say in second line something like #[lang=clojure]
to override fileextension to use proper lang syntax? its common with lint...
+1

Yep, you can do something very similar to that. Check out the overrides we support: https://github.com/github/linguist/blob/master/README.md


 Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

ℹ Artifact update notice
File name: go.mod
In order to perform the update(s) described in the table above, Renovate ran the go get command, which resulted in the following additional change(s):

20 additional dependency was updated
The go directive was updated for compatibility reasons

Details:

Package
Change

go
1.19 -> 1.22.4

github.com/emicklei/go-restful/v3
v3.10.1 -> v3.11.0

github.com/go-logr/logr
v1.2.3 -> v1.4.1

github.com/golang/protobuf
v1.5.2 -> v1.5.4

github.com/google/go-cmp
v0.5.9 -> v0.6.0

golang.org/x/crypto
v0.6.0 -> v0.21.0

golang.org/x/mod
v0.8.0 -> v0.15.0

golang.org/x/net
v0.7.0 -> v0.23.0

golang.org/x/oauth2
v0.5.0 -> v0.10.0

golang.org/x/sync
v0.1.0 -> v0.6.0

golang.org/x/sys
v0.5.0 -> v0.18.0

golang.org/x/term
v0.5.0 -> v0.18.0

golang.org/x/text
v0.7.0 -> v0.14.0

golang.org/x/tools
v0.6.0 -> v0.18.0

google.golang.org/protobuf
v1.28.1 -> v1.33.0

k8s.io/api
v0.26.1 -> v0.30.2

k8s.io/apimachinery
v0.26.1 -> v0.30.2

k8s.io/klog/v2
v2.90.0 -> v2.120.1

k8s.io/kube-openapi
v0.0.0-20230217203603-ff9a8e8fa21d -> v0.0.0-20240228011516-70dd3763d340

k8s.io/utils
v0.0.0-20230220204549-a5ecb0141aa5 -> v0.0.0-20230726121419-3b25d923346b

sigs.k8s.io/structured-merge-diff/v4
v4.2.3 -> v4.4.1


mo.ui.slider: full width option

Description
Add an option to render the slider as full width.
Suggested solution
add a full_width keyword argument to mo.ui.slider, defaulting to False
Alternative
No response
Additional context
No response

Hi can I work on this, I am new to github, but I was interested in this repo. Also, is there anything that I should know while reproducing the environment, or is the ReadMe good enough to follow?
Thanks!
Amogh


player stops responding after a while

player doesn't re-authenticate the token, so the player stops responding. Disabling the plugin, then re-enabling and re-logging temporarily fixes the issue.
Steps to reproduce: leave player running for over an hour

I personally can't reproduce this, been playing for hours without any issues.
I might however add a way to re-login to combat this, if anyone else seeing this issue wants to help fix it please give it a go.


build:list command fails with no helpful information

Build/Submit details page URL
No response
Summary
We have a bitbucket CI/CD pipeline step that uses the output of eas build:list --json --non-interactive to determine next steps.
The message we see is Error: build:list command failed.
We then ran a few commands to see more information about the installed eas-cli and confirm it is installed and configured correctly. I will paste below the commands and corresponding outputs.
Command
#!/bin/sh
type -a eas

Output
/usr/local/bin/eas

Command
eas --version

Output
eas-cli/3.18.2 linux-x64 node-v16.18.1

Command
eas whoami

Output
Accounts:
 - [redacted] (Role: Developer)

Command
eas diagnostics

Output
 EAS CLI 3.18.2 environment info:
    System:
      OS: Linux 5.15 Alpine Linux
      Shell: 1.35.0 - /bin/ash
    Binaries:
      Node: 16.18.1 - /usr/local/bin/node
      Yarn: 1.22.19 - /usr/local/bin/yarn
      npm: 8.19.2 - /usr/local/bin/npm
    Utilities:
      Git: 2.38.5 - /usr/bin/git
    npmPackages:
      expo: ^48.0.0 => 48.0.10
      expo-dev-client: ~2.2.1 => 2.2.1
      expo-updates: ~0.16.4 => 0.16.4
      react: 18.2.0 => 18.2.0
      react-dom: 18.2.0 => 18.2.0
      react-native: 0.71.8 => 0.71.8
      react-native-web: ~0.18.10 => 0.18.12
    Project workflow: managed

any advice?
Managed or bare?
Managed
Environment
expo-env-info 1.0.5 environment info:
System:
OS: Linux 5.15 Alpine Linux
Shell: 1.35.0 - /bin/ash
Binaries:
Node: 16.18.1 - /usr/local/bin/node
Yarn: 1.22.19 - /usr/local/bin/yarn
npm: 8.19.2 - /usr/local/bin/npm
npmPackages:
expo: ^48.0.0 => 48.0.10
react: 18.2.0 => 18.2.0
react-dom: 18.2.0 => 18.2.0
react-native: 0.71.8 => 0.71.8
react-native-web: ~0.18.10 => 0.18.12
Expo Workflow: managed
looks like expo doctor failed here:
/opt/atlassian/pipelines/agent/build/node_modules/expo-doctor/build/index.js:2
2023-08-03T16:20:11.532986651Z stderr P (()=>{var __webpack_modules__={47548:(e,r,n)=>{"use strict";Object.defineProperty(r,"__esModule",...

Error output
No response
Reproducible demo or steps to reproduce from a blank project
running
eas build:list --json --non-interactive
in eas-cli/3.18.2 linux-x64 node-v16.18.1

Hi, could you try to run the same command with the EXPO_DEBUG=1 flag?


Support Pipes in the LbEndpoint's Address

The address for an LbEndpoint may be a Pipe (Unix Domain Socket) instead of a SocketAddress. (See https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/core/address.proto.html). It would be very helpful to support Pipes.

@nickman Merged code looks slightly different. Since your changes were breaking compile. But in essence the code does what you wanted to do. can you verify this. Would like to close this issue once you confirm

Closing this since there is only a minor change from your PR @nickman


Completely fresh install brings up an error, and config fails

I used both a fresh install of your config directly from git, and my version updated to your upstream, and in both cases the config fails with the following error showing up on entering nvim.
E5113: Error while calling lua chunk: Vim:E185: Cannot find color scheme 'carbon'
stack traceback:
        [C]: at 0x560dbbd8e970
        [C]: in function 'require'
        ...pot/home/dawn/git/shaunsinghnyoom.nvim/fnl/core/init.lua:2: in main chunk
        [C]: in function 'require'
        ...m/hotpot/home/dawn/git/shaunsinghnyoom.nvim/fnl/init.lua:26: in main chunk
        [C]: in function 'require'
        /home/dawn/git/shaunsinghnyoom.nvim/init.lua:47: in main chunk```

oh, the issue is nyoom tries to load carbon but can only compile carbon on exit, so it just errors out forever. The latest commit should fix it


Possible issue with wave function on PI 4 8GB bullseye 64bit

Hello,
I suspect an issue with pigpio V79 on PI 4 8GB running raspbian bullseye 64bit
From same python code using function "wave_send_once", I have correct output ( taken from piscope on PI B+ ) on a PI B+:

From PI 4, signal is slighly different ( taken from piscope on same PI B+ with a wire linking PI 4 GPIO 5 to PI B+ GPIO 6 ) :

Are you aware of any issue or some specific parameters to set to have wave function working on a PI 4 ?
Thank you

There are unresolved issues with late silicon stepping on pi4 models - specifically with hardware pwm peripheral (#517).  I'm not aware of any issues with waves.  However, the pwm is used to pace the timing on the waves.  Hmmm.
Can you run the pigpio tests?  ./x_pigpio.c for instance

There are unresolved issues with late silicon stepping on pi4 models - specifically with hardware pwm peripheral (#517). I'm not aware of any issues with waves. However, the pwm is used to pace the timing on the waves. Hmmm.
Can you run the pigpio tests? ./x_pigpio.c for instance

Hello,
I ran test and it seems ok :
`# ./x_pigpio
Testing pigpio C I/F
pigpio version 79.
Hardware revision 13644053.
Mode/PUD/read/write tests.

TEST  4.1  PASS (notify open/begin: 0)
TEST  4.2  PASS (notify pause: 0)
TEST  4.3  PASS (notify close: 0)
TEST  4.4  PASS (sequence numbers ok: 1)
TEST  4.5  PASS (gpio toggled ok: 1)
TEST  4.6  PASS (number of notifications: 80)
Waveforms & serial read/write tests.

TEST  7.1  PASS (set watchdog on count: 39)
TEST  7.2  PASS (set watchdog off count: 0)
Bank read/write tests.
TEST  8.1  PASS (read bank 1: 0)
TEST  8.2  PASS (read bank 1: 33554432)
TEST  8.3  PASS (clear bank 1: 0)
TEST  8.4  PASS (set bank 1: 1)
TEST  8.5  PASS (read bank 2: 0)
TEST  8.6  PASS (clear bank 2: 0)
TEST  8.7  PASS (NOT APPLICABLE: 0)
TEST  8.8  PASS (set bank 2: 0)
TEST  8.9  PASS (NOT APPLICABLE: 0)
Script store/run/status/stop/delete tests.
TEST  9.1  PASS (store/run script: 100)
TEST  9.2  PASS (run script/script status: 201)
TEST  9.3  PASS (run/stop script/script status: 110)
TEST  9.4  PASS (delete script: 0)`

For sanity, can you toggle the gpio on Pi4 using another method to verify a reasonable output on piscope?   From Pi4 terminal for example:
pigs pwm 5 128

should produce a square wave on gpio 5.

I tested with following command issued on pi4 ( gpio 6 of pi4 connected to gpio 6 of pi 2 running piscope ) :
pigs pwm 6 128
Here is screenshot of piscope. The other strange behaviour is that after issuing command, gpio 6 send signal forever until pi reboot.

Here is also the vcd file containing the entire trace :
pigs-pwm-sample.zip

Thanks for the vcd file.
The low pulses are generally 3 to 5 us wide but sometimes are groups of small pulses lasting for 45 us.  Obviously, this is wrong.  Not only is the duty cycle wrong but there should never be multiple transitions within a pwm cycle.  (Sometimes they are missing entirely).  Is it possible there is an output drive issue with the gpio?
The frequency of the pulses (or pulse groups) is around 800 Hz which seems plausible for a pwm cycle.
On my Pi 4 4GB bullseye 64-bit I'm able to observe a square wave of 800 Hz.  Initially, the frequency was 4000 but I was able to set it to 800 with pigs pfs 27 800.  I'm using gpio 27.

pigpio.zip
My captured vcd file for reference.

Thanks for the vcd file.
The low pulses are generally 3 to 5 us wide but sometimes are groups of small pulses lasting for 45 us. Obviously, this is wrong. Not only is the duty cycle wrong but there should never be multiple transitions within a pwm cycle. (Sometimes they are missing entirely). Is it possible there is an output drive issue with the gpio?
The frequency of the pulses (or pulse groups) is around 800 Hz which seems plausible for a pwm cycle.
On my Pi 4 4GB bullseye 64-bit I'm able to observe a square wave of 800 Hz. Initially, the frequency was 4000 but I was able to set it to 800 with pigs pfs 27 800. I'm using gpio 27.

All is possible but since pi is new and argon one fan seems correctly controlled, I don't think that GPIO are broken.
I did several other tests without being able to get a correct output.
I also tried following commands :
# pigs write 6 0
# pigs write 6 1
# pigs write 6 0
The output is attached and seems inverted
sample-1-0-only.zip
Regarding GPIO 27 : i can mesure some signal on it ( from pi 2 GPIO 27 ) without issuing any command :
gpio27-after-reboot.zip

Regarding GPIO 27 : i can mesure some signal on it ( from pi 2 GPIO 27 ) without issuing any command

What mode is gpio27?  pigs mg 27 issued from Pi 2

pigs mg 27

Without anything plug on it, command is returning 0 ( it is a fresh raspbian install used only for piscope )

Regarding the 'sample-1-0-only' vcd, it appears that a transition is missing.
$enddefinitions $end
#0
...
0G
...
#5
1G

I'm unsure, but could be an artifact of piscope.  In any case, the gpio seems to be driving the pin correctly.

Mode 0 indicates the gpioi is in 'input' mode.  By default the gpio should have a weak pull down so it should remain low (0).  That said, if you touch it with your finger it could pick up noise.

Please tell me more about what is running on the Pi4 8GB:

Desktop 64-bit raspberry pi os?
Any additional software?
Do you have audio enabled/running anywhere?
What is the hardware revision?  cat /proc/cpuinfo
uname -a
pigs pigpv
ps -aux | grep -i pigpiod
How did you install pigpio?

... and show the script you are running to generate the waveform.

Here are requested details regarding the Pi4 :

OS is running headless : it is raspbian lite without X server running from an M2 SSD ( on argon one M2 case )
Additional software are mainly : argonone script ( https://download.argon40.com/argon1.sh ) and docker
Audio is normally disable from /boot/config.txt file ( dtparam=audio=off ) ( I attach the file in case )
Harware revision is indicated as 1.5 :

`#cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
processor       : 1
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
processor       : 2
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
processor       : 3
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
Hardware        : BCM2835
Revision        : d03115
Serial          :<PHONE_NUMBER>b26661
Model           : Raspberry Pi 4 Model B Rev 1.5
#cat /sys/firmware/devicetree/base/model
Raspberry Pi 4 Model B Rev 1.5`

uname -a return : Linux raspberrypi 6.1.54-v8+ #1682 SMP PREEMPT Wed Sep 20 15:19:57 BST 2023 aarch64 GNU/Linux
pigs pigpv return : 79
pigpiod is started via crontab with following command : pigpiod -l -m -n <IP_ADDRESS> ( This is because script in use is only supposed to produce output but I also tried without -m args )
#ps -aux | grep -i pigpiod root        5333  0.8  0.0 205720  1820 ?        SLsl 17:59   0:00 pigpiod -l -m -n <IP_ADDRESS> root        5342  0.0  0.0   6044   652 pts/0    S+   17:59   0:00 grep -i pigpiod
pigpio has been installed from apt and pip3
Regarding changes on OS config, I also disabled some services from following commands :
#systemctl disable hciuart
#systemctl disable bluealsa.service
#systemctl disable bluetooth.service

And the script used to generate wave is from following repo : https://github.com/Nickduino/Pi-Somfy
Code involved is inside operateShutters.py file : since same code works on pi 2 and script logs produce same output on pi4 and pi2 when using same input parameters, I suspected issue to be more on gpio part
Part of code that produce wave should be this ( I removed some lines to put only parts that are used to generate signal but complete script is available on the git ) : `
       pi.wave_add_new()
       pi.set_mode(self.TXGPIO, pigpio.OUTPUT)

       self.frame[0] = 0xA7;       # Encryption key. Doesn't matter much
       self.frame[1] = button << 4 # Which button did  you press? The 4 LSB will be the checksum
       self.frame[2] = code >> 8               # Rolling code (big endian)
       self.frame[3] = (code & 0xFF)           # Rolling code
       self.frame[4] = teleco >> 16            # Remote address
       self.frame[5] = ((teleco >>  8) & 0xFF) # Remote address
       self.frame[6] = (teleco & 0xFF)         # Remote address

       #This is where all the awesomeness is happening. You're telling the daemon what you wanna send
       wf=[]
       wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 9415)) # wake up pulse
       wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 89565)) # silence
       for i in range(2): # hardware synchronization
          wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 2560))
          wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 2560))
       wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 4550)) # software synchronization
       wf.append(pigpio.pulse(0, 1<<self.TXGPIO,  640))

       for i in range (0, 56): # manchester enconding of payload data
          if ((self.frame[int(i/8)] >> (7 - (i%8))) & 1):
             wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 640))
             wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 640))
          else:
             wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 640))
             wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 640))

       wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 30415)) # interframe gap

       for j in range(1,repetition): # repeating frames
                for i in range(7): # hardware synchronization
                      wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 2560))
                      wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 2560))
                wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 4550)) # software synchronization
                wf.append(pigpio.pulse(0, 1<<self.TXGPIO,  640))

                for i in range (0, 56): # manchester enconding of payload data
                      if ((self.frame[int(i/8)] >> (7 - (i%8))) & 1):
                         wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 640))
                         wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 640))
                      else:
                         wf.append(pigpio.pulse(1<<self.TXGPIO, 0, 640))
                         wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 640))

                wf.append(pigpio.pulse(0, 1<<self.TXGPIO, 30415)) # interframe gap

       pi.wave_add_generic(wf)
       wid = pi.wave_create()
       pi.wave_send_once(wid)
       while pi.wave_tx_busy():
          pass
       pi.wave_delete(wid)

       pi.stop()`

pigpiod is started via crontab with following command : pigpiod -l -m -n <IP_ADDRESS>

I have not invoked the daemon using crontab before.  Can you explain how that is done?
Agree with you on -m argument shouldn't matter but just for sanity can you run the ./x_pigpiod_if2 test suite.

Here are test results :
#./x_pigpio.py
Connected to pigpio daemon.
Testing pigpio Python module 1.78
Python 3.9.2 (default, Feb 28 2021, 17:03:44)  [GCC 10.2.1 20210110]
pigpio version 79.
Hardware revision 13644053.
Mode/PUD/read/write tests.

TEST  3.1  PASS (get servo pulsewidth: 500)
Traceback (most recent call last):
File "./x_pigpio.py", line 1028, in
if '3' in tests: t3()
File "./x_pigpio.py", line 209, in t3
CHECK(3, t, int((1E3*(on+off))/on), int(2E7/x), 1, "set servo pulsewidth")
ZeroDivisionError: float division by zero
Regarding launching deamon from crontab, I simply add this line to crontab ( I was starting deamon same way on PI 2 on did some test after launching it manually ) :
@reboot pigpiod -l -m -n <IP_ADDRESS>
After killing deamon and starting it again without -m option, I got those results :
killall pigpiod
pigpiod -l -n <IP_ADDRESS>
#./x_pigpio.py
Connected to pigpio daemon.
Testing pigpio Python module 1.78
Python 3.9.2 (default, Feb 28 2021, 17:03:44)  [GCC 10.2.1 20210110]
pigpio version 79.
Hardware revision 13644053.
Mode/PUD/read/write tests.

TEST  4.1  PASS (notify open/begin: 0)
TEST  4.2  PASS (notify pause: 0)
TEST  4.3  PASS (notify close: 0)
TEST  4.4  PASS (sequence numbers ok: 1)
TEST  4.5  PASS (gpio toggled ok: 1)
TEST  4.6  PASS (number of notifications: 80)
Waveforms & bit bang serial read/write tests.

TEST  6.1  PASS (gpio trigger count: 5)
TEST  6.2  PASS (gpio trigger pulse length: 150)
Watchdog tests.
TEST  7.1  PASS (set watchdog on count: 39)
TEST  7.2  PASS (set watchdog off count: 0)
Bank read/write tests.
TEST  8.1  PASS (read bank 1: 0)
TEST  8.2  PASS (read bank 1: 33554432)
TEST  8.3  PASS (clear bank 1: 0)
TEST  8.4  PASS (set bank 1: 1)
TEST  8.5  PASS (read bank 2: 0)
TEST  8.6  PASS (clear bank 2: 0)
TEST  8.7  PASS (clear bank 2: -42)
TEST  8.8  PASS (set bank 2: 0)
TEST  8.9  PASS (set bank 2: -42)
Script store/run/status/stop/delete tests.
TEST  9.1  PASS (store/run script: 100)
TEST  9.2  PASS (run script/script status: 201)
TEST  9.3  PASS (run/stop script/script status: 110)
TEST  9.4  PASS (delete script: 0)
Wavechains & filter tests.
TEST 13.1  PASS (callback, set mode, wave clear: 0)
TEST 13.2  PASS (pulse, wave add generic: 6)
TEST 13.3  PASS (clear glitch filter: 0)
TEST 13.4  PASS (clear noise filter: 0)
TEST 13.5  PASS (wave chain: 0)
TEST 13.6  PASS (wave chain, tally: 2688)
TEST 13.7  PASS (glitch filter, wave chain, tally: 1792)
TEST 13.8  PASS (glitch filter, wave chain, tally: 896)
TEST 13.9  PASS (glitch filter, wave chain, tally: 0)
TEST 13.10 PASS (noise filter, wave chain, tally: 1500)
TEST 13.11 PASS (noise filter, wave chain, tally: 750)
TEST 13.12 PASS (noise filter, wave chain, tally: 0)
TEST 13.13 PASS (wave delete: 0)
Regarding those tests, are waves timers compared to what is generated using same clock ?
I am not good enough to understand all your code but maybe if clock is not correct for generating and have same drift for reading, it may pass tests even if waves are incorrectly generated.
Please ignore if it is not relevant but is there any way to measure from PI2 with a wire between GPIO 25 of both devices so measure are taken from a device which have correct clock ?

Can you now observe your waveform via piscope on your Pi2?
Essentially, I want to understand if the problem is related to pigpiod coming up in a bad state from boot.    Then see if you can observer your waveforms as your original post.

I killed pigpiod and launched it manually with following commands :
#killall pigpiod
#pigpiod -l
Same result than previous tests ( see VCD file on GPIO 5 ). I am pretty sure I already tested multiple pigpiod arguments and also without crontab launch.
wave-signal.zip

Do you have a pigpiod service installed?  systemctl status pigpiod
If so, please show systemctl cat pigpiod

Same result than previous tests ... without crontab launch.

Does this mean you have tested starting from a clean boot without pigpiod invoked, then manually running your waveform?  This is the result I'm interested in.  I'm suspicious of some corruption occurring if pigpiod tries to initialize before the system is ready.  Please bear with my persistent questions.

I have service installed but not active :
systemctl status pigpiod
● pigpiod.service - Daemon required to control GPIO pins via pigpio
Loaded: loaded (/lib/systemd/system/pigpiod.service; disabled; vendor preset: enabled)
Active: inactive (dead)
systemctl cat pigpiod
/lib/systemd/system/pigpiod.service
[Unit]
Description=Daemon required to control GPIO pins via pigpio
[Service]
ExecStart=/usr/bin/pigpiod -l
ExecStop=/bin/systemctl kill pigpiod
Type=forking
[Install]
WantedBy=multi-user.target
I did several tests but yes, I also tried to run manually after reboot ( crontab entry is disabled )
I also tried without argonone script and then without argon one case ( only pi with official power attached ) and results looks similar ( see attached )
sample-without-case-and-script.zip
I also tried to disable 3D acceleration from /boot/config.txt by commenting following lines ( like below ) :
Enable DRM VC4 V3D driver
#dtoverlay=vc4-kms-v3d
#max_framebuffers=2
Still same kind of result each time

Thanks.  I'm at a loss to understand what is going on.  It seems like the pwm peripheral, which controls timing of waveforms as well as pwm output, is getting fouled up on your Pi4.  I'm not sure where to go next.  I'll think about it.

Can you paste a copy of your /boot/config.txt?

Here is my last /boot/config.txt
I also tried to force CPU frequency to maximum value by changing governor with those commands :
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
Still same results
config.txt

Please comment this section:
# Enable audio (loads snd_bcm2835)
#dtparam=audio=off

For sanity, we should insure we are on the same code base.  Do you mind installing the latest pigpio from the 'develop' branch?  This is the one I'm running.  It's possible some bug fixes were made that may have relevance, but it's been too long for me to remember.
wget https://github.com/joan2937/pigpio/archive/refs/heads/develop.zip
unzip develop.zip
cd pigpio-develop
make
sudo make install

I tried with dev version and line commented after a reboot ( default line was dtparam=audio=on )
Still same, here is the VCD output ( GPIO 5 )
sample-dev-version.zip

Also found some discussion related to the other issue on which you referenced this one here : https://forums.raspberrypi.com/viewtopic.php?t=331110
I don't know if following results can help :
vcgencmd measure_clock plld && vcgencmd measure_clock pwm
frequency(100)=3000000000
frequency(25)=9993164

I get:
vcgencmd measure_clock plld && vcgencmd measure_clock pwm
frequency(100)=3000000000
frequency(25)=107143064

This is the max value I saw for now :
vcgencmd measure_clock plld && vcgencmd measure_clock pwm
frequency(100)=3000000000
frequency(25)=10006348

I made following code to test :
#!/usr/bin/python3

import time
import pigpio

TXGPIO = 5 #Define which GPIO to use
iteration = 2
pi = pigpio.pi() # connect to Pi

#pi.wave_add_new()
pi.set_mode(TXGPIO, pigpio.OUTPUT)

#Init to 0
pi.write(TXGPIO, 0)

i = 0
print('Starting ' + str(iteration) + ' loop')
while i < iteration:
  print('Loop ' + str(i))
  print('Current time : ' + str(int(round(time.time() * 1000))))
  pi.write(TXGPIO, 1)
  time.sleep(1)
  print('Current time : ' + str(int(round(time.time() * 1000))))
  pi.write(TXGPIO, 0)
  time.sleep(1)
  print('-------------------------')
  i += 1

print('Done')
pi.write(TXGPIO, 0)

pi.stop()

Output is :
./gpio_test2.py
Starting 2 loop
Loop 0
Current time :<PHONE_NUMBER>876
Current time :<PHONE_NUMBER>877
-------------------------
Loop 1
Current time :<PHONE_NUMBER>878
Current time :<PHONE_NUMBER>880
-------------------------
Done

But measure from PI shows following :

Here is the VCD
test2.zip
Is PWM clock involved for this test ?

I'm not sure which clock source is used for pwm.  I think joan mentioned PLLD in the link you referenced above.  But, pigpio wave apis use the pwm to time the dma transfers that generate the output waveform.  In your test above, you are writing directly to the gpio so, no, pwm should not be involved.  Strange behavior.

Try adding a callback function that prints level and tick.
def cbf(gpio, level, tick):
   print(gpio, level, tick)

pi.callback(TXGPIO, pigpio.EITHER_EDGE, cbf)

Reading about the Argon One, it appears that raspi-gpio is installed which is likely to conflict with pigpio.  Also, the Argon forum reveals complaints about the lack of documentation on gpio usage.  My recommendation is to do a fresh install of RPi OS and pigpio and then re-run waveforms.

I tested after removing packages installed by argon script except i2c-tools ( raspi-gpio python3-rpi.gpio python3-smbus i2c-tools ) and rebooting.
It seems better ( I can now measure a 1s signal while running test script ) but I still have strange behaviours :

running test script ( with callback ) : one 1s signal is missing

./gpio_test2.py
Starting 2 loop
Loop 0
Current time :<PHONE_NUMBER>993
5 0 801707006
5 1 801707476
Current time :<PHONE_NUMBER>993
5 0 802707906
-------------------------
Loop 1
Current time :<PHONE_NUMBER>995
5 1 803709241
Current time :<PHONE_NUMBER>996
5 0 804710296
-------------------------
Done

test-without-argontool.zip

wave signal is still not correct :
wave-test-without-argontool.zip

Expected :

Result :

Regarding GPIO usage on argon case, there were a paper document with the case indicating that GPIO 4, 22 and 23 are used. From the internet I found following usage ( https://forum.argon40.com/t/argon-one-v2-used-pins/1576 ) :
BMC pin  Function
=================
2        SDA.1 (I2C data)
3        SCL.1 (I2C clock)
4        GPIO.7 Shutdown button
14       TXD transmit UART <== Function?
22       GPIO.3 IR LED pin
23       GPIO.4 IR receiver pin

Not sure if it can interfer as well but since I removed argon script and installed tools, I put following script in place to control FAN :
#!/bin/bash
# Script require package i2c-tools
# Fan control info : https://github.com/Argon40Tech/Argon-ONE-i2c-Codes
while true; do
    cpu_temp=$(($(cat /sys/class/thermal/thermal_zone0/temp) / 1000)) #Retrieve CPU temp in °C

    if [ $cpu_temp -gt 60 ]; then
        i2cset -y 1 0x1A 0x64
    elif [ $cpu_temp -gt 55 ]; then
        i2cset -y 1 0x1A 0x4B
    elif [ $cpu_temp -gt 50 ]; then
        i2cset -y 1 0x1A 0x3C
    elif [ $cpu_temp -gt 47 ]; then
        i2cset -y 1 0x1A 0x14
    else
        i2cset -y 1 0x1A 0x00
    fi

    sleep 60  # Wait for 60 seconds
done

I would tend to believe the callback and suspect the waveforms.   Do you have an independent tool such as an oscilloscope?  If not, can you run piscope from the Rpi4?

I made new test and all is now working perfectly : issue was most probably due to conflict with raspi-gpio or other tools.
The reason it was not working after test on fresh install was due to dead RF transceiver : since it was working from pi2 with transceiver inside pi4 case, it probably died after reassembling inside case following test with pi4 only.
I can now confirm that GPIO wave is working perfectly on the pi4 8GB ( version 1.5 ) inside argon case ( but without installing script ).
For those who are interested, it is mandatory to not install argon one script. I did not test other features ( button and ir as I don't use them ) but FAN control is still possible using pigpio :
Only add following line to /boot/config.txt :
dtparam=i2c_arm=on
And control FAN using python script like this :
#!/usr/bin/env python3

import sys
import pigpio

import time

def i2c_set_device(bus, address, register, value):
    pi = pigpio.pi()
    if not pi.connected:
        print("Unable to connect to Pigpio daemon.")
        return

    handle = pi.i2c_open(bus, address)
    if handle < 0:
        print("Unable to open I2C device.")
        pi.stop()
        return

    pi.i2c_write_byte_data(handle, register, value)
    pi.i2c_close(handle)
    pi.stop()

def get_cpu_temp():
    # Retrieve CPU temp in °C
    with open("/sys/class/thermal/thermal_zone0/temp", "r") as file:
        temp = int(file.read()) / 1000
    return temp

def set_fan_speed(temp):
    if temp > 60:
        i2c_set_device(1, 0x1A, 0x64, 0x00)
    elif temp > 55:
        i2c_set_device(1, 0x1A, 0x4B, 0x00)
    elif temp > 50:
        i2c_set_device(1, 0x1A, 0x3C, 0x00)
    elif temp > 47:
        i2c_set_device(1, 0x1A, 0x14, 0x00)
    else:
        i2c_set_device(1, 0x1A, 0x00, 0x00)

if __name__ == "__main__":
    while True:
        cpu_temp = get_cpu_temp()
        set_fan_speed(cpu_temp)
        time.sleep(60)  # Wait for 60 seconds

It is possible to run as a service starting after pigpio ( service has also to be enable ) :
[Unit]
Description=Control argonone fan depending on CPU temperature
After=pigpiod.service
Wants=pigpiod.service

[Service]
ExecStart=/$PATH_TO_SCRIPT$/$SCRIPT_NAME$.py
ExecStop=/bin/pkill -f "python3 /$PATH_TO_SCRIPT$/$SCRIPT_NAME$.py"

[Install]
WantedBy=multi-user.target

Thank you for your help


05/25/2015 All setup done. First commit. hub.fish file created.

05/26/2015
2 support functions: __fish_hub_needs_command, __fish_hub_using_command
Tab completions: help, fork [--no-remote]

Thanks for the update, @hasit! However, we won't pull this into github/hub yet because I'd like the script to mature more before merging into master branch here. For instance, I would like to see it wrapping the official git completion script in fish and expanding it to include hub commands and extra flags. Let's continue the work in your fork until then.


Propagates no_sandbox flag to binary reader.

Adds the no_sandbox option to ReadBinaryOptions and arranges for it to be populated identically to WriteCOptions in wasm2c.
Furthermore, propagates ReadBinaryOptions to BinaryReaderIR.
This will make it possible to customize reader actions for no-sandbox mode.

Why does the reader need to know about this option?


Switch search plugin

I found that there's another version of the docusearch local plugin - tried it out and lo and behold, it does all the things that seem to be broken on our own version.

Searches page titles by search string - I could never find the "Running server tests" page in the current docusaurus but now I can!
Shows the page title on the search list - helps contextualize when you have a number of identical hits.
Provides a "see all results" link to see all the hits instead of just the top few.

Generally it seems to return a lot more results and give a lot more context for the searcher.
Reviewer: please let me know if there's any other processes to remove one plugin/package with yarn and install another.
Check it out!

And that is giving me a different file than the one in this PR. How did you regenerate the yarn.lock?

I updated it - is this one better? Do you know why there's all that algolia stuff in there - we are not using algolia.

And that is giving me a different file than the one in this PR. How did you regenerate the yarn.lock?

I updated it - is this one better? Do you know why there's all that algolia stuff in there - we are not using algolia.

Yep, this looks good!
Running npm list -a (and doing an output redirect to a file since it's a lot of output), it looks like the algolia stuff is coming from "@docusaurus/preset-classic".
I'm guessing prior folks who added dependencies forgot to delete the yarn.lock and do yarn install.


Error on using dynamic loading

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/home/vscode/.local/lib/python3.11/site-packages/watchdog/observers/api.py", line 204, in run
    self.dispatch_events(self.event_queue)
  File "/home/vscode/.local/lib/python3.11/site-packages/watchdog/observers/api.py", line 380, in dispatch_events
    handler.dispatch(event)
  File "/home/vscode/.local/lib/python3.11/site-packages/watchdog/events.py", line 276, in dispatch
    {
  File "/home/vscode/.local/lib/python3.11/site-packages/edifice/runner.py", line 119, in on_modified
    if isinstance(observer, FSEventsObserver) and event.is_directory:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union

Looks like importing watchdog.observers.fsevents.FSEventsObserver caused ModuleNotFoundError somehow (I don't know why so I added it to issue of watchdog: https://github.com/gorakhargosh/watchdog/issues/1008) and messed up
try:
    from watchdog.observers.fsevents import FSEventsObserver
except ImportError:
    FSEventsObserver = None

in runner.py.
I monkey-patched the code like this -
        def on_modified(self, event):
            if FSEventsObserver is not None and isinstance(observer, FSEventsObserver) and event.is_directory:

(line 117-118) and it worked properly
Maybe something related to my installation (I install this on codespace with devcontainer and quite a lot of apt install mess) or installation of watchdog went something wrong.
(Oh and plus - this project doesn't requires watchdog in requirements.txt which made me install watchdog manually, would be nice if it is installed along with edifice when I just pip install pyedifice.)

dupe of https://github.com/pyedifice/pyedifice/issues/18. Sorry.


Consider adding functionality equivalent to Linux syscall BPF_PROG_TEST_RUN

It looks like folks are using BPF_PROG_TEST_RUN to measure the performance of eBPF on Linux. This seems like it might be useful concept to adopt as it would permit side-by-side comparisons of BPF performance on the two platforms.

@teknoraver Can you verify where the data comes from?  If it comes from the calling app or from the eBPF program, then how does the verifier know it is safe?

@teknoraver Can you verify where the data comes from? If it comes from the calling app or from the eBPF program, then how does the verifier know it is safe?

both data_in and data_out are buffers supplied by userspace.


removing leading space within the sqlite dsn

the extra space was causing the below error when using an absolute path like /home/my.app.sq3
Unable to connect to the specific sql server: SQLSTATE[HY000] [14] unable to open database file

Good catch, thanks!

you're welcome ☺


Update Release Notes for Service Bus preview 2

Where you have the "Service Bus Changelog" heading under the Changelog section, you could call this "New Features" instead.

@czubair does this change make sense? This will be mixing multiple releases into New Features.


chore(deps): update dependency winston to v3.3.4

This PR contains the following updates:

Package
Type
Update
Change

winston
dependencies
patch
3.3.3 -> 3.3.4

Release Notes

winstonjs/winston
v3.3.4
Compare Source
Compared to v3.3.3, this version fixes some issues and includes some updates to project infrastructure,
such as replacing Travis with Github CI and dependabot configuration.
There have also been several relatively minor improvements to documentation, and incorporation of some updated dependencies.
Dependency updates include a critical bug fix [#2008] in response to self-vandalism by the author of a dependency.

[#1964] Added documentation for how to use a new externally maintained Seq transport.
[#1712] Add default metadata when calling log with string level and message.
[#1824] Unbind event listeners on close
[#1961] Handle undefined rejections
[#1878] Correct boolean evaluation of empty-string value for eol option
[#1977] Improved consistency of object parameters for better test reliability

Configuration
📅 Schedule: At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box.

This PR has been generated by Renovate Bot.

Renovate Ignore Notification
As this PR has been closed unmerged, Renovate will now ignore this update (3.3.4). You will still receive a PR once a newer version is released, so if you wish to permanently ignore this dependency, please add it to the ignoreDeps array of your renovate config.
If this PR was closed by mistake or you changed your mind, you can simply rename this PR and you will soon get a fresh replacement PR opened.


fix link

@Rick-Anderson please review

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

aspnetcore/security/authentication/azure-active-directory/toc.md
:white_check_mark:Succeeded

Details

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

No need for a review on these simple typo's.


Remove cli

At the moment we don't have a clear use case for the CLI so should remove it, we can add back in in future.

@Rlamboll just to check before I do this, should I just remove silicone.cli, or do you use anything in there?

Nope, it's dead weight to me.


[docs] Removed odd capitalization guidance

We've received a few questions about sentence vs. title casing. Our examples were still confusing, so I removed some and clarified the surrounding copy.

Not sure what's going on with commits here. Closing and reopening with clean branch.


do not save payload because it can cause stackoverflow

@schfeca75

Missing jira issue from commit message

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/cloudbreak-pull-request/2765/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/cloudbreak-pull-request/2766/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/mock-integration-test/256/

Refer to this link for build results (access rights to CI server needed):
http://ci.sequenceiq.com//job/mock-integration-test/257/


PriceR behaves in trade_aggregations differently from other endpoints.

What version are you using?
0.24.1-e632f45c553b24f210ebe955779a4025fbeb250caccess
What did you do?
access https://horizon.stellar.org/trade_aggregations?base_asset_type=native&counter_asset_type=credit_alphanum4&counter_asset_code=XCN&counter_asset_issuer=GCNY5OXYSY4FKHOPT2SPOQZAOEIGXB5LBYW3HVU3OWSTQITS65M5RCNY&resolution=300000&order=desc

What did you expect to see?
In this endpoint, PriceR behaves differently from other endpoints.
In trade_aggregations:
"close_r": {
          "N": 25,
          "D": 166
        }

In others(expect):
"price_r": {
          "n": 25,
          "d": 166
        }

This is not a bug, but I think it is beneficial to maintain consistency across endpoints.
What did you see instead?
"close_r": {
          "N": 25,
          "D": 166
        }

Fixed in 2.9.0.


Use mutation testing to evaluate test quality

Mutation testing can uncover ineffective tests or uncovered conditions without significant manual effort by mutating source code before running the tests and seeing if they still pass. https://github.com/mull-project/mull is an LLVM-based framework for mutation testing, under the Apache 2.0 license.
Per Wikipedia:

Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways. Each mutated version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. This is called killing the mutant. Test suites are measured by the percentage of mutants that they kill. New tests can be designed to kill additional mutants. Mutants are based on well-defined mutation operators that either mimic typical programming errors (such as using the wrong operator or variable name) or force the creation of valuable tests (such as dividing each expression by zero). The purpose is to help the tester develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution. Mutation testing is a form of white-box testing.

This is a cool idea but I doubt we will have time to work on it for a while, and we are cleaning up the bug DB.


Battery save mode and NOAA scan mode overlaps

Coz not updating state in statusbar after it changes.

I'm not sure why NS is up there in the first place. It should be down where the power level for the channel is.

@Tunas1337 right place (in stock fw). But battery save mode will not enable when NS mode on...

Fixed.


Add option to disable counter suffixes

This addresses the issue here: https://github.com/pointfreeco/swift-snapshot-testing/issues/429
This allows users to disable the default behavior of snapshot creation, where a counter is added whenever a test doesn't specify a test name.
For tests that only validate one snapshot, adding a suffix to these tests seems excessive. For me, this would result in most test files looking like MyViewControllerSnapshotTests/testLayout.1.png and it would be nice to be able to disable this functionality when it's not desired to keep file names simpler: MyViewControllerSnapshotTests/testLayout.png

@stephencelis Just checking in on this to see if there's any action you'd need from me here for this PR to be accepted.
Ultimately, this would help devs with a workflow like my own where there's only one snapshot per test function, with no need for suffixing being added to file names.

@stephencelis +1 this would be nice. I thought this was an issue on our end at first.


Dependency on time 0.3.5 conflicts with requirements in ibc-rs dependency tree

Version(s) of tendermint-rs: v0.23.x branch pre 0.23.2
What went wrong?
Changes landed in #1036 break the build for ibc-rs because it requires rouille, which pins its dependency on time to =0.3.2: https://github.com/tomaka/rouille/issues/246
De-facto tendermint now requires 0.3.5 though (which erroneously is not what Cargo.toml specifies) due to the API it uses.
Steps to reproduce
Uncomment the patch section in the ibc-rs project's top-level Cargo.toml, change the branch to v0.23.x and try to build.
Definition of "done"
The quickest fix is to remove any use of time API post 0.3.2 and make sure nothing in the project requires a higher version.
Luckily, only some non-essential additions currently require that, which can be undone. I'd like to keep the full API In master, though. ibc-rs would have time to address the dependency problems before switching to that.

Still no cigar: I left a couple more checked_add/checked_sub uses in the code. Will fix in another PR.


Improve safety when URLs redirect

Highlights:

Each provider now checks whether any redirects occurred and if so, whether the redirect is "safe" (i.e., redirects to the same release). If the redirect is unsafe, the extraction process is aborted.

For providers which load the HTML of the provider page (e.g., Bandcamp, Apple Music, Amazon), we check whether the page is redirected and whether the unique ID of the page URL is the same before and after the redirect.
For providers which use an API (Discogs, Qobuz, etc), we check whether the ID returned in the API response is the same as the one we requested, just to be sure.

When a URL pointing directly to an image is redirecting, a warning is emitted and that redirection is indicated in the edit note.

Known issues:

When provider extraction is aborted, the script logs an error asking the user to double-check the redirection and use the redirected URL itself if it's still correct. However, that message is too long, leading to bad positioning of the DOM elements. Could be easily fixed with some CSS, but since we'll add an improved status banner GUI at some point in the future (#81), I decided against torturing myself with CSS today. Those redirections should be rare enough anyway, so it won't lead to issues too often.
The warning emitted when a direct image link is redirecting isn't going to be visible to the user for very long, since it'll be overwritten by a success message quite quickly. Again, this should get fixed when #81 is implemented.

Closes #79.
Depends on #72.
Bumping version for release.

Dropping the version bump again, we'll merge this after #84 so that we can test whether CD deploys correctly 😊


Edit reply bookmark titles

This PR started as an experiment with HTMX to allow Tanzawa to edit reply/bookmark titles. As all of the entry creation forms/views are inherited from similar base-classes for easy/speed of initial implementation, it's proving to be brittle to change.
Thus, this PR has expanded in scope to first refactor the existing entry creation forms/views to adopt the layered architecture so the changes will be easy and then add the new edit functionality.

This PR started as an experiment with HTMX to allow Tanzawa to edit reply/bookmark titles. As all of the entry creation forms/views are inherited from similar base-classes for easy/speed of initial...
I've re-worked all of the create/update views for each post type to use application functions and added functional tests. Editing the url/title for reply posts, works really well. HTMX is allowing me to create the simple interactions I want without Javascript.
The biggest change is reply data is no longer updated all at once with the rest of the post, but rather when you finish editing the title/url, similar to how editing a GitHub PR title works.
The last bit that I think needs some more work, unrelated to the rest main objective of his issue, is the first step when creating a reply where I request the URL to, and it uses Turbo to fetch the meta-data / return a page that can create a reply.
This flow may need a bit more yak shaving so it doesn't use Turbo at all. Will give it a fiddle to see if I can refactor and simplify the flow using HTMX.
(Originally published at: https://jamesvandyne.com/3b17bbb6-b228-46a5-bdf6-05c596d9faf7)


[Enhancement Request] FaceSwap Node: Cache faces and face_analyser model

Hi!
I just saw that the faces (source_face and  target_face) in function swap_face are not being cached.
Also in function get_face_single the face_analyser model is loaded everytime the node is executed.
I assume it would save some time if the faces and face_analyser model is cached after the FaceSwap node is executed.
(If you agree, please be aware of caching this data in the FaceSwap node wouldn't help because it's going to be resetted immediately if any input is changed - so I suggest you to store this cache data in the LoadFaceSwapModel node.)
Kind regards

Thanks!
Good points! I will look into it

@MoonMoon82 This was auto closed by my commit, but it's indeed much faster now! I think there is still a bit of optimization possible but it's much better already!
I made this to test it out 😅
Georges Lucas as Terminator:
The player seems to no like it, but you can click here to watch it

I also updated the example file to match this update

@melMass This is awesome fast!! Great work!!
Just a tiny issue: antelopev2.zip and buffalo_m.zip could not been downloaded automatically. I guess these file are taken down. I needed to search the web to download them from a google drive and extract them manually.


Emergency documentation on how to recover after a crash

Author Name: Carlos Martín (Carlos Martín)
Original Redmine Issue: 3407, https://dev.opennebula.org/issues/3407
Original Date: 2014-12-05

Requested during the OpenNebulaConf.
The request was to have documentation with clear steps on what to do after a server crash; how to recover the unknown VMs.

Original Redmine Comment
Author Name: Arnold Bechtoldt (Arnold Bechtoldt)
Original Date: 2014-12-05T15:56:06Z

+1


Dependency verification result hides real errors between warnings

Given a build with dependency verification enabled and a lot of unsigned artifacts (e.g. the gradle build itself).
When you introduce a real error (e.g. a missing sha256) by adding a new dependency or removing a component from the verification-metadata.xml, you get a very long list of errors and its up to impossible to figure out which one the real error is.
Example by commenting out a component in verification-metadata.xml:
* What went wrong:
Execution failed for task ':bootstrap:classpathManifest'.
> Dependency verification failed for configuration ':bootstrap:runtimeClasspath':
    - On artifact ansi-control-sequence-util-0.2.jar (net.rubygrapefruit:ansi-control-sequence-util:0.2) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact antlr-2.7.7.jar (antlr:antlr:2.7.7) in repository 'Gradle libs': artifact wasn't signed
    - On artifact asciidoctor-gradle-plugin-<IP_ADDRESS>.jar (org.asciidoctor:asciidoctor-gradle-plugin:<IP_ADDRESS>) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact asm-7.1.jar (org.ow2.asm:asm:7.1) in repository 'Gradle libs': artifact wasn't signed
    - On artifact asm-analysis-7.1.jar (org.ow2.asm:asm-analysis:7.1) in repository 'Gradle libs': artifact wasn't signed
    - On artifact asm-commons-7.1.jar (org.ow2.asm:asm-commons:7.1) in repository 'Gradle libs': artifact wasn't signed
    - On artifact asm-tree-7.1.jar (org.ow2.asm:asm-tree:7.1) in repository 'Gradle libs': artifact wasn't signed
    - On artifact asm-util-7.1.jar (org.ow2.asm:asm-util:7.1) in repository 'Gradle libs': artifact wasn't signed
    - On artifact common-0.74.jar (org.gradle.ci.health:common:0.74) in repository 'maven': artifact wasn't signed
    - On artifact ezmorph-1.0.6.jar (net.sf.ezmorph:ezmorph:1.0.6) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact github-client-0.74.jar (org.gradle.ci.health:github-client:0.74) in repository 'maven': artifact wasn't signed
    - On artifact gpars-1.2.1.jar (org.codehaus.gpars:gpars:1.2.1) in repository 'Gradle Central Plugin Repository': artifact was signed but all keys were ignored
    - On artifact gradle-build-tag-plugin-0.74.jar (org.gradle.ci.health:gradle-build-tag-plugin:0.74) in repository 'maven': artifact wasn't signed
    - On artifact gradle-enterprise-gradle-plugin-3.1.1.jar (com.gradle:gradle-enterprise-gradle-plugin:3.1.1) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact gradle-enterprise-gradle-plugin-3.1.jar (com.gradle:gradle-enterprise-gradle-plugin:3.1) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact gradle-guides-plugin-0.16.1.jar (org.gradle.guides:gradle-guides-plugin:0.16.1) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact grolifant-0.9.jar (org.ysb33r.gradle:grolifant:0.9) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact groovy-all-1.3-2.5.8.jar (org.gradle.groovy:groovy-all:1.3-2.5.8) in repository 'Gradle libs': artifact wasn't signed
    - On artifact http-builder-0.7.jar (org.codehaus.groovy.modules.http-builder:http-builder:0.7) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact javax.inject-1.jar (javax.inject:javax.inject:1) in repository 'Gradle libs': artifact wasn't signed
    - On artifact jcifs-1.3.17.jar (org.samba.jcifs:jcifs:1.3.17) in repository 'Gradle libs': artifact wasn't signed
    - On artifact jquery.min-1.11.0.js (jquery:jquery.min:1.11.0) in repository 'googleApisJs': artifact wasn't signed
    - On artifact json-lib-2.3-jdk15.jar (net.sf.json-lib:json-lib:2.3) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact jsr166y-1.7.0.jar (org.codehaus.jsr166-mirror:jsr166y:1.7.0) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact multiverse-core-0.7.0.jar (org.multiverse:multiverse-core:0.7.0) in repository 'Gradle Central Plugin Repository': artifact wasn't signed
    - On artifact tagging-0.74.jar (org.gradle.ci.health:tagging:0.74) in repository 'maven': artifact wasn't signed
    - On artifact teamcity-client-0.74.jar (org.gradle.ci.health:teamcity-client:0.74) in repository 'maven': artifact wasn't signed
    - On artifact xbean-reflect-3.7.jar (org.apache.xbean:xbean-reflect:3.7) in repository 'Gradle libs': Artifact was signed with key '223d3a74b068eca354dc385ce126833f9cf64915' (Richard Kenneth McGuire (CODE SIGNING KEY<EMAIL_ADDRESS>and passed verification but the key isn't in your trusted keys list.
    - On artifact xml-resolver-1.2.jar (xml-resolver:xml-resolver:1.2) in repository 'Gradle Central Plugin Repository': artifact wasn't signed

Already fixed on master


Test.py this file I can't run it.

Although I have already set the parameters.

Did you test directly using our model and code, or did you make your own modifications?

I ran the test.py file directly and did the operation of matching the dataset to test the ITS test set.Thanks your reply.

It's possible that I uploaded the model file before making any modifications to the code. In this case, the fifth attention block actually corresponds to the fourth attention block. Renaming the model by replacing 5 with 4; or alternatively, changing 4 to 5 in the model code would resolve this issue.

ok,ths.I try it


Idea: better low level networking code

We have some ideas that we want to try, which involve giving low-level networking for ADO.NET providers a similar makeover treatment to what did with Kestrel some time ago.
@davidfowl (and @anpete?) volunteered to do some experimentation with PostgreSQL, but this could apply to any database.

@davidfowl, @anepe could you please detail a bit more (and better) what you had in mind here?

@roji I'd like to get a good idea of the threading model and buffering strategy you chose for Npgsql. That'll help with my mental model (I'd like to do some experiments with pipelines).

@davidfowl are you actively working on this?

@davidfowl it's great to be having this conversation with you guys. Below are some notes on Npgsql works, please don't hesitate to ask more questions.
Synchronization

The only major point where synchronization is currently necessary is the connection pool, which is invoked when NpgsqlConnection.Open() and NpgsqlConnection.Close() are called and which is responsible for assigning a physical connection (what Npgsql calls a "connector") to the user's NpgsqlConnection.
Apart from the connection pool, Npgsql code contains almost no synchronization. Here's a list where it does, although I'm not sure how important this is to this discussion:
** A SemaphoreSlim is taken as long as any user action (e.g. query) is in progress. This is to produce meaningful "already in use" errors in case the user wrongfully uses the same connection concurrently, and also to synchronize with Npgsql's internal keepalive mechanism. Note that the SemaphoreSlim itself is protected by a regular lock, to prevent race conditions with disposing the semaphore.
** There's also a lock to make sure we wait until command cancellation completes before sending a new command (cancellation in PostgreSQL is a completely "asynchronous" affair that takes place on a different physical socket).

I/O and Buffering

Npgsql tries to buffer quite aggressively (IMHO). Each physical connection has a read and a write buffer (8k each by default), and almost all Npgsql code reads and writes binary data to and from these buffers, only flushing when they're full (or when the operation has been completely written/read). Take a look at https://github.com/npgsql/npgsql/blob/dev/src/Npgsql/NpgsqlReadBuffer.cs and https://github.com/npgsql/npgsql/blob/dev/src/Npgsql/NpgsqlWriteBuffer.cs, which contain methods such as {Read,Write}Int32().
Actual I/O only takes place when necessary. In other words, once my query has been sent to PostgreSQL, a single NetworkStream.Read{,Async}() call takes place for the entire read buffer size (8k). At this point it's probable that many result rows are read into the buffer, and subsequent calls to NpgsqlDataReader.Read() will be memory-only - until the read buffer is exhausted and another I/O call occurs.
The minimum unit for buffering is the row. As long as rows are relatively small, they get read in one I/O call and are later processed from memory. However, big rows (>8k) will cause the read buffer to grow, causing allocations. To avoid this, you can either increase the buffer size (connection string parameter), or use CommandBehavior.Sequential when executing the command - this will switch to a more advanced mode where we don't attempt to buffer the entire row in memory (or anything else for that matter), and it's the user's responsibility to read the row's columns in the order in which they arrive from the database (this is a standard ADO.NET API feature). This really only makes sense for scenarios with very big rows (e.g. binary data).
Importantly, Npgsql performs reading and writing concurrently, not waiting for the entire operation to write before we start to read the results. This is first of all necessary to avoid an important deadlock: when sending a multi-statement command, PostgreSQL will stop reading from its connection once it sees the protocol message for executing the first statement - and will start writing back its results. We therefore can't attempt to write all the statements' messages before transferring control back to the user and starting to read the first statement's results. This also has the nice side effect of optimizing I/O, since writing and reading happens in parallel (although this is only relevant for multi-statement queries with large parameter and result payloads).

Async and Sync I/O

When implementing async support for Npgsql, I made the decision to keep providing both a true sync and true async API. This was first and foremost because ADO.NET provides both options, and because in some scenarios truly sync database access may be more appropriate (simpler programming model, possibly more efficient implementation depending on O/S, etc.).
The implementation went through several iterations, ending up in a single codepath of async methods inside Npgsql, which all pass an async flag down the stack. At the very bottom, when actual I/O has to occur, we test on the flag and execute either Read() or ReadAsync(). This allows us to avoid duplicating code (i.e. same functionality written once as a regular method and once as an async method) without any significant overhead (I hope!).
Note that I did try to minimize calling async methods as much as possible to avoid the added overhead. ValueTask is used in all places where it's not 100% sure that I/O will occur (i.e. thanks to in-the memory buffering described above).

I hope this is a first fair description of Npgsql's internals, whipped up just before I head off for work :) Please feel free to ask more questions, I'll be closely monitoring this thread and the other performance-related ones.

@davidfowl are you actively working on this?

No, it's on my (now shorter) list of things to look at. I'm looking at using minimal code to build a simple postgres driver based on a protocol spec that @mikeharder shared a while back. Basically a single query with results to benchmark some of the patterns.
@roji Is something like this interesting for npgsql?

@davidfowl do you mean building a non-ADO.NET driver for PostgreSQL, possibly non-managed (i.e. based on libpq)? Or just an attempt to build things properly from the ground up to see where it leads?
Npgsql is obviously a fully managed ADO.NET driver, but of course anything you do in terms of PostgreSQL connectivity interests me - you're also welcome to ask me questions, I've had a lot of experience working with the wire protocol. Also, at some point in the email exchanges I mentioned the possibility of splitting Npgsql (or any other ADO.NET driver) into two parts: a low-level ultra-efficient API, on top of which we'd built an ADO.NET adapter. This would allow performance-hungry users to drop down to the low-level API as needed. The low-level API could simply be libpq, or it could be fully-managed implementation (possibly libq first and managed later). There's also the question of whether this low-level component would expose some sort of standardized new database API - a high-performance competitor to ADO.NET, which, among other things, would allow you to include database sockets in an epoll/kqueue programming model (currently impossible with ADO.NET).
All this is a bit theoretical, and this would obviously mean very big changes for Npgsql (it would definitely be Npgsql 4.0 :)). Let me know if this is the direction you were thinking of etc.

Or just an attempt to build things properly from the ground up to see where it leads?

This. I wasn't thinking about going as far as building an entire driver. The end goal would be to improve npgsql. The idea is to write a minimal low level API that does the working and protocol so we can focus on things like buffering, IO, async and threading models. Ideally we'd take a very simple query and write the simplest code possible then build up from there. I'd also like to see this example using the lower layer of npgsql (if that's possible).

So yeah, I'm absolutely interested... Feel free to reach out if any questions arise during implementation. Note that at the moment Npgsql doesn't really have a low-level layer internally which can be easily separated from an higher-level ADO.NET layer - that was more an idea for the future.

Great, once you push something I'll definitely take a look - it'll be interesting to compare with Npgsql.
On my side I'm working on bringing the dev branch back to working order (there are some pending issues) and then the plan is to implement command-level caching (https://github.com/npgsql/npgsql/issues/1701), which will provide quite a substantial boost.

Here are the result based on @anpete 's implementation of the protocol for the fortunes query.
Same environment as TE on cloud (Azure D3V2 machines). Database server scaled up to 8 cores as it was a limiting factor for the max perf. The new driver is compared to npgsql in async mode with full cached strategy (see https://github.com/aspnet/DataAccessPerformance/issues/3 for explanation).

This configuration we actually max out the client CPU, the client Network and the Database CPU, changing any of these would not change the overall result significantly.
What we can learn from that is that we reached pgbench baseline numbers with a fully managed .NET Core implementation.
We also tried this driver in the TE Fortunes benchmark. We are seeing 45K rps, compared to 16K with in raw ADO.NET mode with npgsql. On this infrastructure the current best result (undertow framework) is at 30K rps.
Next step is to decide what to do with that data, and more specifically if we want to go further with the idea of a different abstraction layer than ADO.NET, or even no abstractions at all but custom providers for each micro/full ORM.

@anpete and @sebastienros, it's really great to see Peregrine and its associated benchmarks. It indeed looks like a solid baseline of where we might end up with a managed implementation, and I think it shows that looking at a native solution (e.g. libpq wrapper) or a radically different I/O approach may not be worthwhile.
Here are some notes after reviewing Peregrine. Some of these will point out missing features, which are totally understandable in an early-stage attempt such as this, and the idea is mainly to get us thinking and to evaluate to what extent the high performance is a result of a lack of needed functionality.

First, don't forget that pgbench doesn't exactly do the same as what you're asking Npgsql and Peregrine to do. For one thing, I'm pretty sure the results aren't even parsed - no UTF8 decoding - which I think was a major part of the running time when @damageboy profiled Npgsql.
One big mystery is why Npgsql peaks out at 32 threads whereas Peregrine peaks out at 64 threads. @sebastienros, any chance you can provide more data on what is going on at 64 threads with Npgsql? CPU usage, network utilization? Is it a result of the GC hanging us up?
I like the idea of prepared statements being specified at the pool level, so that all connections returned from that pool have them, and they can be explicitly used by name later. However, it's difficult to see layers such as Dapper or EF leveraging this - those layers produce SQL and generally don't contain explicit APIs for knowing about or reusing server-side statements. So while this is appropriate for a lower-level access API, some automatic/implicit mechanism seems necessary, which would transparently prepare based on usage (any SQL used more than X times), similar to what Npgsql 3.2 introduced. This kind of mechanism would surely have some performance impact (e.g. hashing on the command's SQL to get its corresponding prepared statements).
While we're on the subject... @sebastienros, would it be possible for you to benchmark a special version I hacked up alongside the three above, which does command caching (https://github.com/npgsql/npgsql/issues/1701)? The code isn't fit for merging, but it should show what is possible in Npgsql. The version is available as version 3.3.0-command-caching from the Npgsql unstable feed.
The current implementation assumes that the entire messages to be written (and read) will always fit in the buffer. While this works for TechEmpower's very short statements, in real life messages are sometimes going to be larger than the buffer, or will simply be split down the middle. The wire protocol requires the length upfront, so Npgsql supports this by doing two passes: we first go over the message and calculate how many bytes will be necessary, then we start writing the message itself, possibly filling the buffer several times and flushing.
Note that the above "two-pass" system is also related to parameter validation: Npgsql makes sure parameters are OK before starting to actually send the command to the server (incorrect parameter can be un-encodable strings, out-of-range values and other values which are un-representable in PostgreSQL). Npgsql does this upfront because if a validation error occurs but part of the command has already been written, we cannot complete the operation and the protocol sync will be broken (so the connection is unusable).
For "ultra" high performance, Npgsql can implement a mode where validation is disabled (so a param validation error breaks the connection), and where if the message happens to fit in the buffer a single writing pass is done (it makes sense to optimize for this case). This will shave some time off, but I wonder how actually relevant it is for real-world scenarios (i.e. is the gain worth it). I've opened https://github.com/npgsql/npgsql/issues/1727 to look into this.
Peregrine shifts the entire burden of parsing the results to the user - no Describe message is sent, and the driver doesn't interpret any RowDescription messages. While this is OK for a baseline/proof-of-concept, this means that if a user accidentally tries to read a string as an int they'll get totally corrupt data etc.

@roji
As requested, here is and updated graph with your experimental version. This one is from Linux on the async mode. The improvements are impressive.

A note however is that it seems to have issues with the sync mode, with a lot of variation and nearing 0 sometimes.


[Question] ReplicatedMergeTree can't insert duplicate rows ?

Hi, ReplicatedMergeTree can't insert duplicate rows? what's wrong?
a minimal reproducible example
DROP TABLE IF EXISTS test.replicated_alter1;

CREATE TABLE test.replicated_alter1
(
    d Date,
    k UInt64,
    i32 Int32
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/alter', 'r1', d, k, 8192);

INSERT INTO test.replicated_alter1 VALUES ('2015-01-03', 10, 44);
INSERT INTO test.replicated_alter1 VALUES ('2015-01-03', 10, 44);

SELECT *
FROM test.replicated_alter1 ;

┌──────────d─┬──k─┬─i32─┐
│ 2015-01-03 │ 10 │  44 │
└────────────┴────┴─────┘

@alexey-milovidov, hi, can you help me, thank you.

It's similar to #827. how to solve it now. thanks

Maybe check insert_deduplicate setting

@zhang2014 :100:


Bug: Insights creation don't use selected repos (beta)

Describe the bug
When creating a new insight, The selected repositories are ignored and replaced with a bunch of random repositories
https://user-images.githubusercontent.com/62995161/228157539-03f55e54-c750-4cd1-a917-d5f7d61cc5a3.mov
Steps to reproduce

Goto insights page https://beta.insights.opensauced.pizza/hub/insights
Click on add insights page
Select repos and add the page title
Click on the create page button and you'll be redirected to the insights home page
Edit the same created page

Affected services
insights.opensauced.pizza
Platforms
No response
Browsers
No response
Environment
Development, Testing
Additional context
This bug can only be found in the beta environment
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Contributing Docs

[X] I agree to follow this project's Contribution Docs

Is the bug happening during the creation or editing of the page? It's important to understand if the database insert is accurate. Can look into specific insight pages if you give me some IDs 🍕

I look closer and found out that for some reason, repo.host_id is not a valid key in the repo object when creating and updating an insights page, and the backend did not guard against creating insights with id of undefined or null
 const response = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/user/insights`, {
      method: "POST",
      headers: {
        "Content-type": "application/json",
        Authorization: `Bearer ${sessionToken}`
      },
      body: JSON.stringify({
        name,
        repos: repos.map((repo) => ({ id: repo.host_id, fullName: repo.full_name })),
        // eslint-disable-next-line
        is_public: isPublic
      })
    });

That is a known issue, our api uses repo.id similar to github; gitsense was using host_id; however, because you can create insight page repos without having any repos in the database, for creating and editing those pages the public github api is required. Please refer to the unfinished work in the alpha branch for this, I have personally fixed adding insight pages/repos but not fixed editing.
@brandonroberts made some beta changes but i doubt any of the work is finished - you can however take it to beta ❤

you can however take it to beta ❤

yeah... Just tested using repo.id and it worked as espected 🍕


Unified Goldberg settings

Would be nice if the interface included unified Goldberg settings for custom options, such as default save location, username, LAN mode, etc.

Thanks for the suggestion! I believe this is a duplicate of #55
I'll take your suggestion into account though, as it seems you're talking about the settings in %appdata%\Goldberg SteamEmu Saves\settings\. I'll try to add an option to either use the default config, and allows to easily modify it, as well as an option to use custom settings (which will be in steam_settings in the game folder)

If I may elaborate. I was already working on a similar project before I realized that you were already working on it. But instead of the program acting like a crack patcher, it was acting as a centralised hub-launcher for cracked steam games. For example, you run the launcher and setup the default settings, for example your username, language, etc. then you add the game to the library, it cracks it, as your patcher does and copies the default settings. If you change the username, it overwrites the settings for all of the games in your library.
On a related note, it would be an absolute god-tier, if we integrated ZeroTier directly node into the launcher, as it's open source (https://github.com/zerotier/ZeroTierOne) and set the Broadcast directly to it:
Custom Broadcast ips: If you want to set custom ips (or domains) which the emulator will send broadcast packets to, make a list of them, one on each line in: Goldberg SteamEmu Saves\settings\custom_broadcasts.txt If the custom ips/domains are specific for one game only you can put the custom_broadcasts.txt in the steam_settings\ folder. An example is provided in steam_settings.EXAMPLE\custom_broadcasts.EXAMPLE.txt

Seems like an interesting project! Unfortunately it seems a lot of work to integrate and I have 0 knowledge about ZeroTier.
You can probably put a custom_broadcasts.txt file in %appdata%\Goldberg SteamEmu Saves\settings\ though, and it should apply to all games that are cracked with Goldberg, as if I understand correctly, Goldberg's emu will use all the cfg files in this location, and then all the ones in the game's location.
Overwriting settings & all should be needed because of that. You can simply edit the files in %appdata%\Goldberg SteamEmu Saves\settings\.
I can only motivate you to work on your project if you have the time :)


[entropy_src/dv] New regression: "live"

For convenience this commit adds a new regression called "live" which
combines the entropy_src_rng and entropy_src_fw_ov tests, which
together make up much of the unit test coverage for entropy_src.
Signed-off-by: Martin Lueker-Boden<EMAIL_ADDRESS>

LGTM. Not sure why "live" was chosen as the name, other than it might be the "real life" scenario testing case. Letting others approve.

That was the idea, by combining fw_ov and rng this covers the two broad use cases.  All other tests are mopping up exceptions.


Official 1.30 Release Docs

🛑 DO NOT MERGE 🛑 UNTIL KUBERNETES 1.30 IS RELEASED
Target Date: 2024-04-17
Release deadlines
From sig-release:

Description
Date

Docs deadline — Open placeholder PRs
Thursday 22nd February 2024

Docs deadline — PRs ready for review
Tuesday 12th March 2024

Docs Freeze — All PRs reviewed and ready to merge
Tuesday 26th March 2024

v1.29.0 released
Wednesday 17th April 2024

For More Information, please reach out to the SIG Release Docs team:
cc: @drewhagen
/hold

Hey y'all @kubernetes/sig-docs-en-owners @kubernetes/sig-docs-blog-owners
This is the PR to merge the integration branch into main for the Kubernetes website tomorrow, after 1.30 has been realized. Can you look this over to make sure nothing looks out of order?
Thanks!

Whoops, we missed asking for reviews on this PR.

I see that https://deploy-preview-44710--kubernetes-io-main-staging.netlify.app/releases/download/#binaries is blank; we should make sure that this doesn't stay blank post release.

This looks like a valid change, but: I don't see the updates for generated documentation. Are we happy we'll get those in?

Whoops, we missed asking for reviews on this PR.

I will modify the SIG Release Docs handbook so that we review this more in advance in the next cycle.
I'm following guidance under "Day before Release":
https://github.com/kubernetes/sig-release/blob/b51237ddb556bb6d24e43d7424e55cb9e66694d5/release-team/role-handbooks/docs/Release-Timeline.md#get-approvals-for-open-prs

Oh wait, my apologies, as it looks like I pinged the wrong groups.
@kubernetes/sig-docs-en-reviews Can you give this a review before we merge this in a few hours?

This looks like a valid change, but: I don't see the updates for generated documentation. Are we happy we'll get those in?

x-post of our plan: https://kubernetes.slack.com/archives/CA1MMM1HU/p1712071114613859
@tengqm have you generated the docs?

/approve

/remove hold

/remove-hold


No Windows support for running our unit tests easily

@FlyingGraysons

This isnt really top priority as most people will be running *nix systems. Haven't seen many bat files in most open source repos. Something to consider for the future though.


UserManager in LoginPage

Nice example!
One question about the loginPage. Why does it get the UserManager by import userManager from '../../utils/userManager';instead of getting it from the context?
Also curious if you have you considered having the login page just doing an action like REQUEST_LOGIN and doing the userManager.signinRedirect();part in a saga instead? This way the loginPage component does not have to know about userManager, and the side-effects are kept in the saga.

Thanks for your question! I could've used the context of the <OidcProvider /> as well - that's true. React's context API is subject to change however, so I use it sparingly. The import is guaranteed to always work.
Your idea of putting the signinRedirect call into a saga is also valid.  Both are equally valid.
The main reason for doing it this way is that I want to keep it simple and demonstrate the core features of redux-oidc. It is by no means a styleguide on how to build redux apps and shouldn't be interpreted this way.

Thank you for the fast reply :-). I am currently trying to figure out where to put things that concern user management but I get now the example is not a guideline for that, it just demonstrates the capabilities of the lib.


Draft: Refactoring of selection method of packages incl time and well selection

In GitLab by @jdelsman on Jan 18, 2021, 09:22
Moved selection method down to packages, giving each package a selection method.

Time selection: selects the active stress for a selected time. So if the t of the stress starts at say 2000-01-01, and you select slice(2000-01-05, 2000-02-01), this time is included (and its time renamed to 2000-01-05)
Well handling: the different nature of the Well package necessitates a different handling of the sel method. It now allows for selection on the underlying dataframe. Also allow for time selection. The active time is selected per separate well.

TODO:

[x] documentation in docstring of package, especially time foo
[ ] more user guide documentation...

In GitLab by @jdelsman on Jan 18, 2021, 12:05
marked this merge request as draft

In GitLab by @jdelsman on Jan 18, 2021, 14:22
added 2 commits
44def25f - docstrings86e81fba - black
Compare with previous version

In GitLab by @Huite on Jan 18, 2021, 17:35
added 1 commit
7933cb30 - Some docstring formatting changes:
Compare with previous version

In GitLab by @Huite on Jan 18, 2021, 20:58
Commented on imod/wq/pkgbase.py line 505
Are there packages in which time is a dimension, but the package is not a BoundaryCondition?
I'm wondering if this check is necessary, since .pop() will give the appropriate None value.
I think time_sel might be called time_indexer?
sel_dims maybe indexers?
Finally sel maybe selection?
Would this work? This cleans up the logic a little:
        indexers = {k: v for k, v in dimensions.items() if k in self.dims}
        time_indexer = indexer.pop("time", None)
        if len(indexers) == 0:
            return self
        else:
            # Note: .sel for Well is overloaded
            # explicitly call xr.Dataset's sel method
            selection = xr.Dataset.sel(
                self, indexers, method=method, tolerance=tolerance, drop=drop
            )
            if time_indexer:
                selection = selection._sel_time(time_indexer)
            return selection

P.S.
I just found out that the "Pythonic" way of checking an empty dict is:
if d:
    print("not empty")

But len(d) seems a lot clearer.

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 76
I think the dimensions are my faulty choice, but maybe we ought to use the opportunity to get them in line with the xarray nomenclature?
https://xarray.pydata.org/en/stable/generated/xarray.DataArray.sel.html
If we're just forwarding everything, could we use the same keyword arguments?

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 851
Assert statements are for debugging, these look like ValueErrors to me.
(Asserts will be ignored by running python -O, where I think the O stands for "optimized")

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 871
Is there a reason for putting this function in this scope?
It is only called once later on, with dims None.
I'm also having trouble telling what it does. Given the dropna, the goal is to get the extremities of the coordinate array once nodata values have been removed, and dtmp is the cellsize?
Is this function maybe useful?
https://gitlab.com/deltares/imod/imod-python/-/blob/master/imod/util.py#L303

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 900
ml might as well be model?

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 901
pkgname instead of pck?

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 902
variable instead of d?
I think a few assignments would improve the readability, e.g.:
da = model[pkgname][variable]
model[pkgname][variable] = da.where(is_edge)

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 905
I'd go with:
# Check if geospatial (IDF) data
if "x" in da.dims and "y" in da.dims:

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 915
This is a very expensive way of shifting the array: combine_first does an element-wise check on every value.
Maybe:
concentration_boundary = xr.concat(
    [
        concentration_boundary.shift(time=-1).isel(slice(None, -1)),
        concentration_boundary.isel(time=-1),
    ],
    dim="time",
)

(If I didn't mess up the direction of the shift)
isel also works directly, I think, but it'll duplicate the time coordinate, which has to be restored then:
time = concentration_boundary["time"]
index = np.arange(1, time.size + 1)
index[-1] -= 1
concentration_boundary = concentration_boundary.isel(time=index)
concentration_boundary["time"] = time

First one is more readable I think.

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/model.py line 914
Is already boolean, no need for == 1

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/pkgbase.py line 500
I don't think this is quite right: if it's a keyword in the docstring, it should be a keyword in the function args.
Forwarding the kwargs shouldn't be a problem at any rate as long as it's all consistent.
It also emphasizes that the dimensions aren't the right keyword, as I mentioned for model.py, methinks.

In GitLab by @Huite on Jan 18, 2021, 21:11
Commented on imod/wq/timeutil.py line 43
Nice

In GitLab by @Huite on Jan 18, 2021, 21:19
I haven't gotten to the Well changes yet, I'll have another look tomorrow.

In GitLab by @jdelsman on Jan 19, 2021, 10:09
Commented on imod/wq/pkgbase.py line 505
@time in non-BoundaryCondition packages: it is not uncommon to have a time coordinate dangling around in your dataset, even it is not a BC. I didn't want it to fail on that. So the check is for trying to access sel_time if not a BC. But this would be handled by the in dims check. So yeah, would work I think.
@the renaming in accordance with xarray: yeah, I would prefer that. The dimensions was indeed already there.

In GitLab by @jdelsman on Jan 19, 2021, 10:35
@ extent and binary dilation: as it is now: the extent da has the size of the original model. So binary dilation is no problem, if it crosses the model bounds, there are no boundary conditions. And erosion gives a slightly smaller model than requested.
@ dimensions: indeed, move to xarray nomenclature.
@ get_clip_na_slices: will look into it. Goal is indeed to get smaller da's...
@ combine_first: sounds good

In GitLab by @jdelsman on Jan 19, 2021, 11:47
Commented on imod/wq/model.py line 76
as above, definitely in favor.

In GitLab by @jdelsman on Jan 19, 2021, 13:48
Commented on imod/wq/pkgbase.py line 505
changed this line in version 5 of the diff

In GitLab by @jdelsman on Jan 19, 2021, 13:48
Commented on imod/wq/pkgbase.py line 500
changed this line in version 5 of the diff

In GitLab by @jdelsman on Jan 19, 2021, 13:48
Commented on imod/wq/pkgbase.py line 783
changed this line in version 5 of the diff

In GitLab by @jdelsman on Jan 19, 2021, 13:48
added 1 commit
b79bde93 - refactor pkgbase
Compare with previous version

In GitLab by @jdelsman on Jan 19, 2021, 14:56
added 2 commits
480f72c8 - refactored well selection444a0817 - refactored well selection
Compare with previous version

In GitLab by @jdelsman on Jan 19, 2021, 14:56
marked the checklist item documentation in docstring of package, especially time foo as completed

In GitLab by @jdelsman on Jan 19, 2021, 17:14
added 1 commit
f705d650 - bugfix time_indexer wel
Compare with previous version

In GitLab by @jdelsman on Jan 22, 2021, 17:50
Commented on imod/wq/model.py line 76
changed this line in version 10 of the diff

In GitLab by @jdelsman on Jan 22, 2021, 17:50
Commented on imod/wq/model.py line 851
changed this line in version 10 of the diff

In GitLab by @jdelsman on Jan 22, 2021, 17:50
Commented on imod/wq/model.py line 900
changed this line in version 10 of the diff

In GitLab by @jdelsman on Jan 22, 2021, 17:50
added 1 commit
0cbf67d0 - model review comments
Compare with previous version

In GitLab by @jdelsman on Jan 22, 2021, 18:00
added 1 commit
6852a06b - scoping of tests
Compare with previous version

In GitLab by @jdelsman on Jan 25, 2021, 09:29
added 1 commit
ae6b2cf7 - black...
Compare with previous version

In GitLab by @Huite on Feb 15, 2021, 22:18
added 5 commits
e80b51a7 - Remove now-irrelevant comment63aeff9e - Remove deprecated TODO8d6ea7ac - Merge branch 'sel_time' of https://gitlab.com/deltares/imod/imod-python into sel_time810d42ac - sparse dataset for selection rather than dataframe6f564b91 - Implement Well .sel via sparse Datasets
Compare with previous version

In GitLab by @Huite on Feb 16, 2021, 13:44
I've been doing some reworking of the Well selection, by utilizing sparse datasets, the regular _sel_time method will work. There's a lot of other stuff that doesn't quite work nicely with the sparse datasets, so I've opted for a temporary conversion from and to. Initializing and storing the data primarily as a sparse dataset is definitely not attractive currently.
While I got all the tests for the wel package working, I moved on to getting the tests working for the model as well.
Unfortunately, the sparse dataset doesn't index as nicely (slices work fine, but .sel(x=[1.0, 2.0, 3.0], y=[1.0, 2.0, 3.0] doesn't work). It can still be done in two steps, but the more serious limitation is that this approach only works if the coordinates of the wells agrees exactly with the x-y midpoints coordinates of like.
This breaks the zone selection for the well package. I think the only good solution is forwarding a (polygon) geometry to the well package .sel instead, and using geopandas to select the wells that actually fall inside.
Anyway, yet another special case for wells (which makes sense, it being the only point-based data)...

In GitLab by @jdelsman on Mar 30, 2021, 17:49
added 1 commit
2708364a - added sparse to dependencies
Compare with previous version

Closing because this branch has been stale for 2 years. Furthermore, new features will be implemented in the mf6 module, adding features to the wq  is thinning out the team's capacity.


Using std::deque<> as an allocation space for factories.

This allows to have pre-sized buffers and easily de-allocated objects.
Since malloc allocators already do something similar, this is not
creating huge saving (about 70 MiB RAM saving for Rsd, 5-10% CPU time)
This particular implementation will increase the compile time by
a lot as we need to pull in all the classes from uhdm/uhdm.h (as
we need to know the size in the factory), so it would need some
refinement (e.g. some PIMPL-style) to be usable.
So leaving it here as draft if someone wants to experiment with
it.

FYI @hs-apotell

This problem is actually harder than it seems on the surface. Deleting the objects within std::deque is trivial (call the destructor explicitly since the object was created using inplace new) but the memory still can't be claimed or reused safely. There is a likelihood that something or someone might still be holding a pointer to this deleted instance and try to use it. If reclaimed and reused, it will end up doing things to the new instance. Nightmare to debug these issues in large systems. In short, objects can be deleted but the memory can't be reclaimed, making the whole exercise pointless.
In the past, for situation like these, a handle based object allocation with block allocation scheme has worked well. No-one holds a reference to any object. Everybody only has a handle that should be dereferenced/resolved within scope and then discarded. There are multiple O(1) handle based system strategies that fit well for problem like these. These systems also easily lend themselves to garbage collection and memory packing strategies on top. This is, however, a big change to the way things are done today.
All that said, by the time Surelog is feature complete, all compilers should support enough features of std::pmr to not have to worry about these problems. The std::pmr module has a lot to offer and as long as used wisely, these memory related performance issues can be stories of the past (fingers crossed). Let's not try and reinvent the wheel, once again!

Well, if someone is keeping a reference or using the memory beyond the lifetime of factories, this is wrong code now and then.
Main issue is that Windows has a brain-dead std::deque implementation that allocates each element individually instead of using a buffer, so it won't help there and possibly even create overhead.
But indeed, there are other fish to fry first.

No, the lifetime implications are the same. It will work exactly as the current factory, as the lifetime are exactly the same as the original.
But it is not necessarily too beneficial at this point as it saves only a little memory and CPU.


[WIP]ImproveRefactor JOB

Abstract
refactor JOB design to support non-timer-scheduled task type jobs and provide a unified job registration interface
NOTICE
This will cause the master's metadata to be incompatible

run buildall

run buildall

run FE UT (Doris FE UT)

UT failed has nothing to do with this change

run buildall

I commented out the metadata write so there will in memory for now, since the PR is large I recommend completing the optimization of the metadata in the next PR

run buildall

run buildall


Close #207, let new projects store their test files in da Sources

Close #207. This will allow developers to update tests while editing in the da Playground instead of locally or in github.
Sorry for the extra comments added.

Yes, just sources. I forgot to change the issue. This will only affect newly created projects, of course. Past projects can either update or keep what they have.

I've just noticed, our workflow doesn't save the test file to the right location anymore. Should we just delete that workflow at this point since we have the new interview we can do it with? That might be best - less duplication.


Not overwrite conf region when awsconfig is empty.

When I used ls-hosts in AWS EC2 with IAM Role,
MissingRegion: could not find region configuration has occurred.
I had set region in .ls-hosts configuration, but there was no .aws/config.
When there was no .aws/config, I want to use .ls-hosts configuration.
.ls-hosts configでregionを設定していても.aws/configが存在しない場合にregionが空設定されるので、
.aws/configの結果が空でない場合のみ上書きするようにしました。

@ki38sato
I tried to remove aws configuration files and run ls-hosts, and saw the same error.
Thank you for your PR!
I'll merge this 👍

👍


plot distances for report

@doutib I  merged pull request #119 .  I tried to build the report and noticed that you hadn't included your plot in it.  Then when I started to add the plots to the report (see #163 ), I noticed that the plotting code takes too long to run.  Could save the data you want to plot?  Then you could just load and plot it.  Just make sure to include the code you use to generate the data.
When you add the data you want to plot, please clean up the plotting script.  For example, for the lines
plt.style.use('ggplot')
plt.style.use('seaborn-notebook')

won't the first one be executed and then undone when the second one is executed?

Yes I will do this, you are right it is a bit slow. For the plots the two styles can be used together, I think this is more aesthetic in this case.

@jarrodmillman should I save the plot instead?
If not, how should I save the data: can I use numpy.save in a python script somewhere?

@jarrodmillman Thanks for merging!
We can close this issue now.


🌐 Add Chinese translation for docs/tutorial/security/first-steps.md

as title

Thanks for the contribution @jaystone776! :cake:
And thanks for the reviews @notnotype and @zy7y! :coffee:


Gulp 4 Tasks Never Completes Using Gulp.Series or Gulp.Parallel

If you run gulp.series or parallel like this:
// Display this help
gulp.task('help', gulp.series(doc.help()));

This error is thrown:
[01:56:25] The following tasks did not complete: help, <anonymous>
[01:56:25] Did you forget to signal async completion?

As there isn't a way to invoke the done callback.  Is there a way this could be achieved:
// Display this help
gulp.task('help', gulp.series(function(done) {

    doc.help(); // and have this output to console using console.log(doc.help());

    done();
}));

Thank you for the report


Cancel order and it's permits only when the permit was marked to be ended immediately

Description
Cancel order and it's permits only when the permit was marked to be ended immediately.
Updates:

Change existing PermitEndType to be a new Django-model instead of Enum-class
Add PermitEndType to ParkingPermit-class
Update existing ORDER_CANCELLED-Talpa webhook to only cancel permits that have end_type=IMMEDIATELY
Add migrations
Update translations

Context
PV-796
How Has This Been Tested?
Through unit-tests.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4e664c6) 85.05% compared to head (90e234f) 85.11%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #487      +/-   ##
===========================================
+ Coverage    85.05%   85.11%   +0.05%
===========================================
  Files          166      167       +1
  Lines        10348    10385      +37
===========================================
+ Hits          8802     8839      +37
  Misses        1546     1546

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.