Problem with 2D parameter scan

I am trying to use PySCeS for a very simple chemical reaction, the goal is to define a model, perform a parameter scan and finally run simulations with the fitted parameters.
I followed the documentation for a 2D parameter scan, but unfortunately, I ran into a problem. There is probably a problem with my code, but at the moment I'm not exactly sure what the problem is in detail
My model is:
Modelname: simulatio_metaraminol
Description: Kinetic model to describe the synthesis of metaraminol from 3-OH-PAC

R1:
    x0 = s0
    k1*x0 - k2*s0
# parameters
k1 = 0.2
k2 = 0.1

# init values
x0 = 35.0
s0 = 0

The code for the parameter fit is:
import pysces

model = pysces.model('simulatio_metaraminol')

time_values = [0,1,2,3,4,5,6,7,8,23.6,24]
concentrations_s0 = [0, 6.27, 8.47, 10.77, 11.58,10.97, 11.75, 12.09, 12.10,14.55, 14.09]
concentrations_x0 = [35.16, 27.30,26.12, 28.45, 29.27, 25.90,24.29, 25.24,24.86,24.51,24.55]

scan = pysces.Scanner(model)

output = ['k1', 'k2']

p1 = ["x0", 35, 24.55, concentrations_x0]
p2 = ["s0", 0,14.09,concentrations_s0]

model.Scan2D(p1,p2, output, log=False)

At the moment I get as an error message:

Info: No reagents have been fixed

Calculating L matrix . . . . . . .  done.

MaxMode 0

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_32904\2699428414.py in <module>
     14 p2 = ["s0", 0,14.09,concentrations_s0]
     15
---> 16 model.Scan2D(p1,p2, output, log=False)

~\anaconda3\envs\pysces2\lib\site-packages\pysces\PyscesModel.py in Scan2D(self, p1, p2, output, log)
   9213         sc1.addScanParameter(*p2)
   9214         sc1.addUserOutput(output)
-> 9215         sc1.Run()
   9216
   9217         self.__scan2d_pars__ = [p1[0], p2[0], output]

~\anaconda3\envs\pysces2\lib\site-packages\pysces\PyscesScan.py in Run(self, ReRun)
    273         for gen in self.GenOrder:
    274             if self.GenDict[gen][4] == False:  # don't increase Tsteps for follower
--> 275                 Tsteps *= self.GenDict[gen][2]
    276         print(next(self.scanT.RUN))
    277         analysis_counter = 0

TypeError: can't multiply sequence by non-int of type 'list'

I'm unfortunately also a little bit confused with the Syntax of the 2D parameter scan:
    p1 is a list of [model parameter 1, start value, end value, points]

    p2 is a list of [model parameter 2, start value, end value, points]

    output the steady-state variable e.g. 'J_R1' or 'A_ss'

    log if True scan using log ranges for both axes

Are the start and end values the time or the measured values?
Are the points only the measured values or do they have to be submitted as for example data frame together with the time?

Hi Stephan,
Parameter scanning is only for repeated run of steady-state analyses, not for time simulations. What this method actually does, is changes the two parameters through the list of values and calculates the steady state for each of these, outputting the results.
The syntax of the parameter scan list is as follows:
p1 = ["k1", 1, 10, 10]
p2 = ["k2", 2, 20, 10]

This would vary the parameter "k1" from 1 to 10 in 10 points, i.e. [1, 2, 3, ..., 10]. If you select log, this would be on a log scale. And parameter "k2" from 2 to 20. The output argument is a list of steady-state variables to output (this would typically be fluxes or steady-state concentrations). In your example you are outputting the parameters k1 and k2, so that won't work.
What you want to do is different, i.e. run various simulations with different starting conditions. You would have to code this manually in a for loop, each time setting mod.x0_init and mod.s0_init (the initial values), setting mod.sim_time (list of time points you want to simulate over, and then calling mod.Simulate(userinit=1).
If you want to see how this is implemented for data fitting of time series, then you can look at the source code of the PySCeS ThinLayer to PyEnzyme: https://github.com/EnzymeML/PyEnzyme/blob/main/pyenzyme/thinlayers/TL_Pysces.py


make DESTDIR=/foo install is not honored.

Description of problem
When make install DESTDIR=/tmp/dest is used not all is installed in /tmp/dest
Expected result
Install all under /tmp/dest
Actual result
tree /tmp/dest/
/tmp/dest/
└── usr
    ├── bin
    │   └── kata-collect-data.sh
    └── share
        └── defaults
            └── kata-containers
                └── configuration.toml

Duplicate of https://github.com/kata-containers/runtime/issues/337
I don't have strong opinion on removing it or making it work.

Let me close this and discuss more in previous thread #337


Feature: manage smartcard authentication

Hi, I have added functionality to setup SmartCard authentication into your module.  At time of this PR, I have tested it on Scientific Linux 7.1 with each of our card reader devices.  We have deployed about 7 workstations using this feature branch, I'm hoping you'll find it suitable to be packaged into your forge module.
Thanks!

Is this of interest at all?  If not, I will withdraw the PR and maintain a local fork for our organization.  Thanks!

Hey @salderma sorry I've missed the first message. It looks great thanks for the contribution !


How can I add non-standard variables?

Not a real issue but where would I add non-standard variables to pandoc-citeproc or custom item types?
Currently, pandoc-citeproc seems to ignore variables it does not know.

You mean variables that you can pass to Pandoc, e.g. with a switch --variable=value? Did you either try a YAML frontmatter or add them to the command in the preferences?

That's right, pandoc-citeproc parses CSL files to a structure representing a Style, so things that can't be represented in that structure are just ignored.  There is currently no way aronud this limitation.

But I guess I could add things in reference.hs and then compile from source, right? Will this override the already installed version or will I end up with two versions? If I end up with two versions, is there a best practice to make sure the right one is used?

Or are there more places where I need to add a custom CSL variable?

I have now managed to add a custom type bookinbook. It works if I use the filter option with explicit paths. But the standard behaviour seems to be that pandoc uses its own version of pandoc-citeproc even though my version is started when called from the command line. How can I change that?

No, pandoc will just use the version in your path.  I'm not sure what is happening here, but maybe the path is set differently in the environment where you run pandoc?

I'll investigate. I'm running both from the same command line.

Another question: a custom item type w

Oh and I'm on Windows if that makes a difference.

It probably does. Try setting the path in the registry if you have only set it in the terminal.

I've set it via system preferences and in the terminal. Same results. What helps is renaming the original pandoc-citeproc binary. Are you sure pandoc isn't prioritising its own directory, and starting lookups in the path only if the filter isn't found there?

Sorry, you're right.
From the manual:
    In order of preference, pandoc will look for filters in

     1. a specified full or relative path (executable or
     non-executable)

     2. `$DATADIR/filters` (executable or non-executable)
     where `$DATADIR` is the user data directory (see
     `--data-dir`, above).

     3. `$PATH` (executable only)

So if you have an executable filter in the working directory, it will use that.

I'll test later. Is there a difference between system path and user path settings?


How do I get the size o a ciphertext in bytes

I need to know the size of a ciphertext variable in bytes, but using the following code is always returning 2.
Ciphertext foo
//Write encrypted stuff to foo
foo.size()

@sebashb this is a guess, but I believe there is a save_size method which maybe what you’re looking for.

Thank you very much for the quick reply. I just tested the save_size method but is returns always 131230. I need the actual size of the Ciphertext because I need to read a Ciphertext from a file.

The method depends on a couple of factors, if it has been seeded and the compression mode that is passed in.
I could be wrong here, as I haven’t tested seeded ciphers so I can’t speak to that case. I think if it is not seeded and if the compression is none, then I would assume the size of the cipher in bytes will always be consistent for a given set of encryption parameters because the values are saved according to the data type - i.e. a 64 bit word will be the same size regardless of the value it holds.
If your implementation uses deflate, then the size will differ almost every time you create a new cipher due to the way it will compress the data which may vary ever so slightly for each cipher.

@s0l0ist Thanks, Nick, for "stealing our job". :) The explanation of deflate is correct. The size of a Ciphertext object does depend on "seeded" and "compression mode". The true size in bytes of a Ciphertext is different from the amount of memory allocated to store those two big polynomials.
If you do want to obtain the size of those two big polynomials, here is a break down (I'm abusing :: to represent membership even for non-static members):

auto parms = SEALContext::get_context_data(Ciphertext::parms_id()) should give you the parameters used for this ciphertext,
each ciphertext contains Ciphertext::size() (which is 2) big polynomials,
each big polynomial has parms.coeff_modulus().size() (which is the number of primes used for this ciphertext) small polynomials,
each small polynomials has parms.poly_modulus_degree() (which is the polynomial ring degree) coefficients,
each coefficient is a uint64_t value that takes 8 bytes.
When you multiply them (2 -- 5) together, you have the amount of memory allocated.

Ciphertext::save_size() contains the size of other data members in a Ciphertext object. Ciphertext::save() and Ciphertext::load() should be used to write to and read from files. These two methods automatically handles various compression and seeding modes.

@WeiDaiWD totally didn’t mean to, just trying to help out. Great breakdown of the sizes - I’ve always been curious.

I was kidding. Much appreciated!

@WeiDaiWD  thank you very much. I know that save() and load() handles this automatically, the thing is I have a big file with multiple ciphertexts in it, so I have to first separate them in auxiliar files so that I can use the load method.
//Scheme : BFV
EncryptionParameters parms(scheme_type::BFV);
// 1st Parameter
size_t poly_modulus_degree = 4096;
parms.set_poly_modulus_degree(poly_modulus_degree);
// 2nd Parameter
parms.set_coeff_modulus(CoeffModulus::BFVDefault(poly_modulus_degree));
// 3rd Parameter
parms.set_plain_modulus(1024);
// Starting
auto context = SEALContext::Create(parms);

With the parameters below the ciphertext size I’m getting when doing save() is not always the same.
Given this how do I get the size (in reality the number of characters used) so that when splitting the big file I can know how many characters I need to read.
Thank you in advanced!

It depends on whether you have ZLIB enabled (and you use the default compr_mode_type::deflate) or not. Note that if using compr_mode_type::none then Ciphertext::save_size will return the EXACT right size (in bytes); this size is deterministic and can be computed from the encryption parameters, as was described in the above comments. If using compr_mode_type::deflate, then Ciphertext::save_size always returns an upper bound on the byte size and hence it cannot be used to figure out the size.
Answering your question though, if you look at native/src/seal/serialization.h, you'll see a description of the SEALHeader struct. If you look at your serialized ciphertext file, it should start with a SEALHeader struct describing the serialized object. The first two bytes contain the 16-bit integer 0xA15E, and the third byte is a 0x00. The fourth byte is either 0x00 or 0x01, depending on the compr_mode_type. The next four bytes are a 32-bit integer describing the byte-count of the entire object, starting from the beginning of the SEALHeader. Therefore, if you read the 32-bit integer from the bytes 5--8 you'll find the exact number of bytes you want to extract to separate the Ciphertext object.
Another note though: I'm not really understanding your motivation of doing this. SEAL will automatically parse the objects correctly, even if they are saved in the same file. Just attach an ifstream to the file and you should be able to read however many Ciphertext or any other objects in a for-loop, or however you want.


start transition of CI according to apache infra reqs

Description
This PR transitions from travis to gha. it is trying to follow https://docs.github.com/en/actions/migrating-to-github-actions/migrating-from-travis-ci-to-github-actions
An alternative approach would be to use: https://github.com/marketplace/actions/run-travis-yml (parallel PR?)

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] Platfor/build maintenance
[ ] Other: ...

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[x] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[x] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

so far only in the original branch on the PR repo. It can be replicated to your own.

matrix is working but hanging on install (build) with error:
Error: The operation was canceled

https://github.com/actions/runner-images/issues/1918 gives some clues but I am not sure how to
--memory-swap -1

after this I am sure the actual script runner will need some love as well

Thanks for starting this @DaanHoogland I saw on the PMCs/announce list they're killing Travis by '22 end.

@potiuk can you have a look at this?

closing in favour of #6896


Traffic-Portal build fails - openssl issues

Trying to build 2.1.x latest (calling ./pkg) on my centos7 VM, I get the below error during traffic-portal build (due to the recent RedHat openssl issues).
npm: relocation error: npm: symbol SSL_set_cert_cb, version libssl.so.10 not defined in file libssl.so.10 with link time reference
I overcame the issue by changing the image in the Dockerfile to centos:7.4.1708

@nir-sopher - this good now? can you reference this issue in the PR you created for it?

@mitchell852 Isn't Centos:7 is actually the same as Centos:7.4.1708? I guess one needs to make sure they pull down the latest image.
$ docker run --rm -it centos:7 cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)

It works well after the change.


Vertical Pod Autoscaler Operator

I cannot find the Vertical Pod Autoscaler (https://github.com/openshift/vertical-pod-autoscaler-operator) on OperatorHub in OKD or on https://operatorhub.io/
Why is it not showing?

where is it located in https://github.com/operator-framework/community-operators?

nvm i guess the issue here is that the docs of okd showed operators which are not availabe under community, and only with licensed openshift.. so yeah so it should be more titled make a community version of it

So can we close it?


WSL2 installation: Escaped asterisks while echoing ddev and docker sources as it list directory contents without it [skip ci]

The Issue

How This PR Solves The Issue
Manual Testing Instructions
Automated Testing Overview

Related Issue Link(s)
Release/Deployment Notes

Are you perhaps not using PowerShell v5?

Yes sorry I am using the latest:
PowerShell 7.3.3

For example, Line 50 of your script:
wsl -u root -e bash -c 'echo \"deb [signed-by=/etc/apt/trusted.gpg.d/ddev.gpg] https://apt.fury.io/drud/ * *\" > /etc/apt/sources.list.d/ddev.list'
Gives me this content on  /etc/apt/sources.list.d/ddev.list

As stated in the instructions, please run this with PowerShell 5.
I haven't been able to figure out a way to get it to work with both.

The provided PowerShell script can do most of the work for you, or you can handle these things manually. (This script works with the built-in PowerShell v5, but not with the newer v7.)

Maybe there's a way to detect the PS version and abort the script?

I just tried with version 5, and you are right, it works good.
But my code looks good for both  5 and 7.
Feel free to close it or merge. Its up to you

It has to more than look good. It has to be tested with both.
If you can test this with both PS5 and PS7 successfully, we can go forward. Like I say, I didn't succeed.
But you need to avoid the trap of an existing setup. Not all the same things fail if you have already run it successfully once. So please create a new WSL2 distro each time when testing.
This was already approached in

https://github.com/ddev/ddev/pull/4402

Yes you are right. I now tested on both.
I install WSL Ubuntu 22.04 and set it to default. Tried with PowerShell 7. Output:
`
PS C:\Users\oMikR> Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072;
PS C:\Users\oMikR> iex ((New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/alexbogias/ddev/5580e3315cde30b5eb0aed3494d702a344a68c5f/scripts/install_ddev_wsl2_docker_inside.ps1'))
Chocolatey v1.3.0
Upgrading the following packages:
ddev;gsudo;mkcert
By upgrading, you accept licenses for the packages.
ddev v1.21.5 is the latest version available based on your source(s).
gsudo v2.0.4 is the latest version available based on your source(s).
mkcert v1.4.4 is the latest version available based on your source(s).
Chocolatey upgraded 0/3 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
The local CA is already installed in the system trust store! 👍
SUCCESS: Specified value was saved.
Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [107 kB]
Fetched 226 kB in 1s (373 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
lsb-release is already the newest version (11.1.0ubuntu4).
lsb-release set to manually installed.
ca-certificates is already the newest version (20211016ubuntu<IP_ADDRESS>).
ca-certificates set to manually installed.
curl is already the newest version (7.81.0-1ubuntu1.8).
curl set to manually installed.
gnupg is already the newest version (2.2.27-3ubuntu2.1).
gnupg set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Get:1 https://download.docker.com/linux/ubuntu jammy InRelease [48.9 kB]
Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:3 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages [13.6 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease
Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [107 kB]
Get:7 https://apt.fury.io/drud * InRelease
Get:8 https://apt.fury.io/drud / amd64 Packages [4180 B]
Fetched 298 kB in 1s (276 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
adwaita-icon-theme at-spi2-core bc cpp cpp-11 dbus-user-session dconf-gsettings-backend dconf-service
...
x11-xserver-utils xdg-utils
0 upgraded, 190 newly installed, 0 to remove and 0 not upgraded.
Need to get 222 MB of archives.
After this operation, 823 MB of additional disk space will be used.
Get:1 https://download.docker.com/linux/ubuntu jammy/stable amd64 containerd.io amd64 1.6.18-1 [28.2 MB]
...
Get:170 http://archive.ubuntu.com/ubuntu jammy/main amd64 libxft2 amd64 2.3.4-1 [41.8 kB]
Setting up libgdk-pixbuf2.0-bin (2.42.8+dfsg-1ubuntu0.2) ...
Setting up x11-utils (7.7+5build2) ...
Setting up libwww-perl (6.61-1) ...
Setting up adwaita-icon-theme (41.0-1ubuntu1) ...
update-alternatives: using /usr/share/icons/Adwaita/cursor.theme to provide /usr/share/icons/default/index.theme (x-cursor-theme) in auto mode
Setting up liblwp-protocol-https-perl (6.10-1) ...
Setting up libxml-parser-perl:amd64 (2.46-3build1) ...
Setting up libxml-twig-perl (1:3.52-1) ...
Setting up libnet-dbus-perl (1.2.0-1build3) ...
Setting up humanity-icon-theme (0.6.16) ...
Setting up ubuntu-mono (20.10-0ubuntu2) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for install-info (6.8-4build1) ...
Processing triggers for libglib2.0-0:amd64 (2.72.4-0ubuntu1) ...
Setting up libgtk-3-0:amd64 (3.24.33-1ubuntu2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
Setting up libgtk-3-bin (3.24.33-1ubuntu2) ...
Setting up libvte-2.91-0:amd64 (0.68.0-1) ...
Setting up libvted-3-0:amd64 (3.10.0-1ubuntu1) ...
Setting up at-spi2-core (2.44.0-3) ...
Setting up libgtkd-3-0:amd64 (3.10.0-1ubuntu1) ...
Setting up tilix (1.9.4-2build1) ...
update-alternatives: using /usr/bin/tilix.wrapper to provide /usr/bin/x-terminal-emulator (x-terminal-emulator) in auto mode
Processing triggers for libgdk-pixbuf-2.0-0:amd64 (2.42.8+dfsg-1ubuntu0.2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
[sudo] password for alex:
CAROOT=/mnt/c/Users/oMikR/AppData/Local/mkcert
The local CA is now installed in the system trust store! ⚡

Starting Docker: docker                                                                                [ OK ]
Refreshing environment variables from registry for cmd.exe. Please wait...Finished..
Downloading https://github.com/docker/compose/releases/download/v2.16.0/docker-compose-linux-x86_64 ...
docker-compose 45.50 MiB / 45.50 MiB [=====================================================================] 100.00% 15s
Download complete.
ITEM             VALUE
DDEV version     v1.21.5
architecture     amd64
db               drud/ddev-dbserver-mariadb-10.4:v1.21.5
dba              phpmyadmin:5
ddev-ssh-agent   drud/ddev-ssh-agent:v1.21.5
docker           23.0.1
docker-compose   v2.16.0
docker-platform  unstablebox
mutagen          0.16.0
os               linux
router           drud/ddev-router:v1.21.5
web              drud/ddev-webserver:v1.21.5

PS C:\Users\oMikR> $PSVersionTable
Name                           Value

PSVersion                      7.3.3
PSEdition                      Core
GitCommitId                    7.3.3
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0
PS C:\Users\oMikR> wsl -u root -e bash -c 'cat /etc/apt/sources.list.d/docker.list'
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu focal stable
PS C:\Users\oMikR> wsl -u root -e bash -c 'cat /etc/apt/sources.list.d/ddev.list'
deb [signed-by=/etc/apt/trusted.gpg.d/ddev.gpg] https://apt.fury.io/drud/ * *
`
Then I installed WSL Ubuntu 20.04 and set it to default. Tried with PowerShell 5. Output:
`
PS C:\Users\oMikR> wsl --install Ubuntu-20.04
Installing: Ubuntu 20.04 LTS
Ubuntu 20.04 LTS has been installed.
Launching Ubuntu 20.04 LTS...
Installing, this may take a few minutes...
Please create a default UNIX user account. The username does not need to match your Windows username.
For more information visit: https://aka.ms/wslusers
Enter new UNIX username: alex
New password:
Retype new password:
passwd: password updated successfully
Installation successful!
To run a command as administrator (user "root"), use "sudo ".
See "man sudo_root" for details.
Welcome to Ubuntu 20.04.3 LTS (GNU/Linux <IP_ADDRESS>-microsoft-standard-WSL2 x86_64)

Documentation:  https://help.ubuntu.com
Management:     https://landscape.canonical.com
Support:        https://ubuntu.com/advantage

System information as of Fri Mar 10 11:41:35 EET 2023
System load:  0.22                Processes:                9
Usage of /:   0.1% of 1006.85GB   Users logged in:          0
Memory usage: 3%                  IPv4 address for docker0: <IP_ADDRESS>
Swap usage:   0%                  IPv4 address for eth0:    <IP_ADDRESS>
1 update can be applied immediately.
To see these additional updates run: apt list --upgradable
The list of available updates is more than a week old.
To check for new updates run: sudo apt update
This message is shown once a day. To disable it please create the
/home/alex/.hushlogin file.
alex@unstablebox:~$ exit
logout
PS C:\Users\oMikR> wsl --set-default Ubuntu-20.04
The operation completed successfully.
PS C:\Users\oMikR> Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072;
PS C:\Users\oMikR> iex ((New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/alexbogias/ddev/5580e3315cde30b5eb0aed3494d702a344a68c5f/scripts/install_ddev_wsl2_docker_inside.ps1'))
Chocolatey v1.3.0
Upgrading the following packages:
ddev;gsudo;mkcert
By upgrading, you accept licenses for the packages.
ddev v1.21.5 is the latest version available based on your source(s).
gsudo v2.0.4 is the latest version available based on your source(s).
mkcert v1.4.4 is the latest version available based on your source(s).
Chocolatey upgraded 0/3 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
The local CA is already installed in the system trust store! 👍
SUCCESS: Specified value was saved.
Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease
...
Get:42 http://archive.ubuntu.com/ubuntu focal-backports/multiverse amd64 c-n-f Metadata [116 B]
Fetched 26.0 MB in 16s (1622 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
lsb-release is already the newest version (11.1.0ubuntu2).
lsb-release set to manually installed.
The following additional packages will be installed:
dirmngr gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm gpgv libcurl4
Suggested packages:
pinentry-gnome3 tor parcimonie xloadimage scdaemon
The following packages will be upgraded:
ca-certificates curl dirmngr gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm
gpgv libcurl4
14 upgraded, 0 newly installed, 0 to remove and 176 not upgraded.
Need to get 3099 kB of archives.
After this operation, 7168 B disk space will be freed.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpg-wks-client amd64 2.2.19-3ubuntu2.2 [97.4 kB]
..
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libcurl4 amd64 7.68.0-1ubuntu2.16 [235 kB]
Fetched 3099 kB in 3s (1082 kB/s)
Preconfiguring packages ...
(Reading database ... 32289 files and directories currently installed.)
Preparing to unpack .../00-gpg-wks-client_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpg-wks-client (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../01-dirmngr_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking dirmngr (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../02-gpg-wks-server_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpg-wks-server (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../03-gnupg-utils_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gnupg-utils (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../04-gpg-agent_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpg-agent (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../05-gpg_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpg (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../06-gpgconf_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpgconf (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../07-gnupg-l10n_2.2.19-3ubuntu2.2_all.deb ...
Unpacking gnupg-l10n (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../08-gnupg_2.2.19-3ubuntu2.2_all.deb ...
Unpacking gnupg (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../09-gpgsm_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpgsm (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Preparing to unpack .../10-gpgv_2.2.19-3ubuntu2.2_amd64.deb ...
Unpacking gpgv (2.2.19-3ubuntu2.2) over (2.2.19-3ubuntu2.1) ...
Setting up gpgv (2.2.19-3ubuntu2.2) ...
(Reading database ... 32289 files and directories currently installed.)
Preparing to unpack .../ca-certificates_20211016ubuntu<IP_ADDRESS>_all.deb ...
Unpacking ca-certificates (20211016ubuntu<IP_ADDRESS>) over (20210119~20.04.2) ...
Preparing to unpack .../curl_7.68.0-1ubuntu2.16_amd64.deb ...
Unpacking curl (7.68.0-1ubuntu2.16) over (7.68.0-1ubuntu2.7) ...
Preparing to unpack .../libcurl4_7.68.0-1ubuntu2.16_amd64.deb ...
Unpacking libcurl4:amd64 (7.68.0-1ubuntu2.16) over (7.68.0-1ubuntu2.7) ...
Setting up ca-certificates (20211016ubuntu<IP_ADDRESS>) ...
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
7 added, 11 removed; done.
Setting up gnupg-l10n (2.2.19-3ubuntu2.2) ...
Setting up gpgconf (2.2.19-3ubuntu2.2) ...
Setting up libcurl4:amd64 (7.68.0-1ubuntu2.16) ...
Setting up curl (7.68.0-1ubuntu2.16) ...
Setting up gpg (2.2.19-3ubuntu2.2) ...
Setting up gnupg-utils (2.2.19-3ubuntu2.2) ...
Setting up gpg-agent (2.2.19-3ubuntu2.2) ...
Setting up gpgsm (2.2.19-3ubuntu2.2) ...
Setting up dirmngr (2.2.19-3ubuntu2.2) ...
Setting up gpg-wks-server (2.2.19-3ubuntu2.2) ...
Setting up gpg-wks-client (2.2.19-3ubuntu2.2) ...
Setting up gnupg (2.2.19-3ubuntu2.2) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for install-info (6.7.0.dfsg.2-5) ...
Processing triggers for ca-certificates (20211016ubuntu<IP_ADDRESS>) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Get:1 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB]
Get:2 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages [24.7 kB]
Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:6 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Get:7 https://apt.fury.io/drud * InRelease
Get:8 https://apt.fury.io/drud / amd64 Packages [4180 B]
Fetched 92.2 kB in 1s (106 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
176 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree
Reading state information... Done
wslu is already the newest version (2.3.6-0ubuntu2~20.04.0).
The following additional packages will be installed:
cpp cpp-9 docker-buildx-plugin docker-ce-rootless-extras docker-compose-plugin docker-scan-plugin gcc-9-base
...
perl-openssl-defaults pigz slirp4netns x11-xserver-utils xdg-utils
0 upgraded, 59 newly installed, 0 to remove and 176 not upgraded.
Need to get 137 MB of archives.
After this operation, 467 MB of additional disk space will be used.
Get:1 https://download.docker.com/linux/ubuntu focal/stable amd64 containerd.io amd64 1.6.18-1 [28.2 MB]
...
Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 libwww-robotrules-perl all 6.02-1 [12.6 kB]
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.
invoke-rc.d: could not determine current runlevel
Setting up libfile-desktopentry-perl (0.22-1) ...
Setting up libwww-robotrules-perl (6.02-1) ...
Setting up libhtml-parser-perl (3.72-5) ...
Setting up cpp-9 (9.4.0-1ubuntu1~20.04.1) ...
Setting up libio-socket-ssl-perl (2.067-1) ...
Setting up libhttp-message-perl (6.22-1) ...
Setting up libhtml-form-perl (6.07-1) ...
Setting up libfile-mimeinfo-perl (0.29-1) ...
Setting up libhttp-negotiate-perl (6.01-1) ...
Setting up libhttp-cookies-perl (6.08-1) ...
Setting up libhtml-tree-perl (5.07-2) ...
Setting up libhtml-format-perl (2.12-1) ...
Setting up libnet-smtp-ssl-perl (1.04-1) ...
Setting up libmailtools-perl (2.21-1) ...
Setting up libhttp-daemon-perl (6.06-1ubuntu0.1) ...
Setting up cpp (4:9.3.0-1ubuntu2) ...
Setting up x11-xserver-utils (7.7+8) ...
Setting up liblwp-protocol-https-perl (6.07-2ubuntu2) ...
Setting up libwww-perl (6.43-1) ...
Setting up libxml-parser-perl (2.46-1) ...
Setting up libxml-twig-perl (1:3.50-2) ...
Setting up libnet-dbus-perl (1.2.0-1) ...
Processing triggers for systemd (245.4-4ubuntu3.15) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Extracting templates from packages: 100%
[sudo] password for alex:
CAROOT=/mnt/c/Users/oMikR/AppData/Local/mkcert
The local CA is now installed in the system trust store! ⚡

Starting Docker: docker                                                                                [ OK ]
Refreshing environment variables from registry for cmd.exe. Please wait...Finished..
Downloading https://github.com/docker/compose/releases/download/v2.16.0/docker-compose-linux-x86_64 ...
docker-compose 45.50 MiB / 45.50 MiB [=====================================================================] 100.00% 15s
Download complete.
ITEM             VALUE
DDEV version     v1.21.5
architecture     amd64
db               drud/ddev-dbserver-mariadb-10.4:v1.21.5
dba              phpmyadmin:5
ddev-ssh-agent   drud/ddev-ssh-agent:v1.21.5
docker           23.0.1
docker-compose   v2.16.0
docker-platform  unstablebox
mutagen          0.16.0
os               linux
router           drud/ddev-router:v1.21.5
web              drud/ddev-webserver:v1.21.5

PS C:\Users\oMikR> $PSVersionTable
Name                           Value

PSVersion                      5.1.22621.963
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22621.963
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
PS C:\Users\oMikR> wsl -u root -e bash -c 'cat /etc/apt/sources.list.d/docker.list'
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu focal stable
PS C:\Users\oMikR> wsl -u root -e bash -c 'cat /etc/apt/sources.list.d/ddev.list'
deb [signed-by=/etc/apt/trusted.gpg.d/ddev.gpg] https://apt.fury.io/drud/ * *
`

I would love to have a solution that works both places. And of course I'd rather it be an installer instead of a script.
PS is very much an unfamiliar language, and nobody has pointed out why something as important as quoting would work differently between PS5 and PS7. But since 5 is the standard... we'll have to stick with it as preferred.
I'd love to have you come up with something that works both places.

Closing this for now, but if you can either improve it by checking the PS version and saying "Can't do with PS 7" or edit it so it works both places, it will be very welcome. Thanks for giving it a try!

I already tried it in both versions and works for me

You agreed that I was correct about the problem above in https://github.com/ddev/ddev/pull/4731#issuecomment-1463547015 :)
You do have to start with a fresh WSL2 distro to demonstrate the problem. Running your version with PS7 and then PS5 will work, but running your version with a fresh WSL2 and PS5 will not.
Again, if you can either improve it by checking the PS version and saying "Can't do with PS 7" or edit it so it works both places, it will be very welcome

Yes I said that you were correct that PS5 worked just fine with your code.
I then tried mine to both versions and I also included outputs with both and new distros and you can see that nothing failed.

Yeah looks different:
Mine:
wsl -u root -e bash -c 'echo deb [signed-by=/etc/apt/trusted.gpg.d/ddev.gpg] https://apt.fury.io/drud/ \* \* > /etc/apt/sources.list.d/ddev.list'
And scarselli:
wsl -u root -e bash -c 'echo "deb [signed-by=/etc/apt/trusted.gpg.d/ddev.gpg] https://apt.fury.io/drud/ * *" > /etc/apt/sources.list.d/ddev.list'

Ah, so your is not using the nested double-quotes at all. Maybe it could work. Thanks for looking closely. I don't like echoing random content without quotes but you've escaped the asterisks so it's more-or-less safe. I'll reopen and hope to test this in coming days. As you know (thanks) it's a bit of work :)

Thanks a lot for your time @rfay

Thanks for your effort on this, it's going to help a lot. You seem very confident of your testing, so please update the install doc with the PR to remove the constraints about PS5. I'm way behind on PRs will will get there. And this one takes some effort to test as you already know.

I updated the docker-desktop script and the docs and tested on PS5 and PS7, thanks for solving this!


fix cmd arugents bug

For now , there is bug that in  pilotage daemon command line,  when you run with arguments such as  pilotage daemon daemon -a <IP_ADDRESS>   , the corba returns unknown shorthand flag: 'a' in -a   .  this PR fix it.
Signed-off-by: Wang Qilin<EMAIL_ADDRESS>

LGTM


SAK-44705 - Resources: Copy URL to Clipboard - Dead link

Tested and the URL is copied now. I traced the click action and it wasn't finding the text area.

Thanks Miguel, just wanted to make sure I wasn't missing something else obvious! Maybe just some uncommitted code?


Add “smart replace” functionality to the fish shell widget

Implementing “smart replace” functionality to the fish shell widget
#648

v1 complete! ready for review # # # # #655 #648 #491 @denisidoro


free/available/total space calculation

Here in GUI https://github.com/whamcloud/integrated-manager-for-lustre/blob/31cf641cf649ef3dd2bd2bd75f98ee4ddb994708/iml-gui/crate/src/page/filesystem.rs#L434-L446
and in this PR https://github.com/whamcloud/integrated-manager-for-lustre/pull/1803
we calculate used space as the difference between total and free, it looks like the used space is always non-zero (hundreds of kilobytes in VBox) on a new filesystem (without files). I also suspect that eventually the used space will be greater than the available space which we show on charts and in command line.
So calculation is like this:
used = total - free and we show used / available
Should we instead do this:
used = available - free? (and still show used / available).
In other words, was this change incomplete https://github.com/whamcloud/integrated-manager-for-lustre/commit/756e3b3400c86102513e68ec050a4a62ab890ac6?
On the other side, existing calculation is in line with df -h, for example:
root@c1 ~]# df -h /mnt/*
Filesystem                            Size  Used Avail Use% Mounted on
<IP_ADDRESS>@tcp:<IP_ADDRESS>@tcp:/fs    47G  333M   44G   1% /mnt/fs
<IP_ADDRESS>@tcp:<IP_ADDRESS>@tcp:/fs2   47G  332M   44G   1% /mnt/fs2

I am totally wrong :)


Include/define/flag docs

Resolves #177

Thanks! Looks like the doc tests may be failing though?

@alexcrichton fixed!

Thanks!


Adding app logrotate when appengine role is not running.

Trusty: https://ocd.appscale.com:8080/job/Daily Build/1199/

https://ocd.appscale.com:8080/job/Daily Build/1200/

Latest build: https://ocd.appscale.com:8080/job/Daily Build/1203/


Allow naming conventions up to 40 characters long

While I've run into this wall previously, I was again prompted to take
action here by a recent PR [1] where a well-intentioned variable name
needed to be truncated just to pass quality checks.
This is one of those instances where the letter of the law (no long
names) conflicts with the spirt of the law (chose names that effectively
communicate purpose). As such, we should adapt.

If we find the current quality configuration to be burdensome to the
point of curtailing free expression, we should just update the config
:)

[1] https://github.com/edx/edx-platform/pull/10756

@stvstnfrd I've created #29 to improve the docs.
For this pull request, you've edited both files, which isn't quite right, you need to generate the pylintrc from edx-lint/files/pylintrc.  There's a hash at the end of the file that won't be right unless you do.

This was handled in #30.


Suggestion Responsive Helpers - Sizing

The sizing helpers are great, I have suggested them in the past and I am so glad you added them.
They need only one tweak, add responsiveness to them:

h-100 h-sm-75 h-md-50 w-100 w-sm-75 w-md-50

Example

Thanks, but we'll pass for now. I don't see a huge value in this for our project.


ripgrep-all: 0.9.6 -> 1.0.0-alpha.5

Fixes #250306
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 264362 run on aarch64-darwin 1

1 package built:

ripgrep-all

Tested the resulted binary against the files in nixpkgs-review/exampledir/demo, and it seems to work. Confirmed that the tests that were removed from installCheckPhase are part of the cargo test tests that run during the build now.

Yes, they are used internally. Afaik rga expects them to be in the same location the rga binary is

I can take a look later today and merge if everything seems OK :)


[OTHER] Fix 'No links found' page in the search

What would you like to share or ask?
Align the content in centre

For demo: visit this
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[ ] I am willing to work on this issue (optional)

Could you please assign this issue to me also tell me the file name where you created this component

Could you please assign this issue to me also tell me the file name where you created this component

Did you locally setup the linkshub or not?

Yeah i already setup

Yeah i already setup

So you can find by yourself the filename.

Okay i will try to find it and solve it

Is this what I had to do?

Is this what I had to do? If yes then I will push my code

@rupali-codes, check this out! It's similar to what you were looking for.

@ParamDhama assign this issue to you.


PR not put back to 'needs_reviewer'

In https://github.com/NixOS/nixpkgs/pull/98938#issuecomment-723469357 , the reviewer is inactive, and marvin (correctly) adds a comment that the PR will be set back from awaiting_reviewer to needs_reviewer, but it doesn't actually do so.

It is supposed to set a timeout_pending label when posting the warning. For some reason I forgot to add the label (but the code for actually timing out PRs with the label exists). Odd, I'm pretty sure I tested that behavior.
Thanks for the notification!

marvin should probably only consider reviewers with merge access

No, making reviews from reviewers without merge access more useful is arguably the main purpose of the bot. See the README of this repo for more info.

The nagging message probably could use an overhaul with some more concrete actionable tips for the reviewer, since they might not be familiar with marvin. The usage is already explained in marvin's initial comment, but having the concrete next steps summarized in the nag message would probably help.


guide

If you follow the guide, http: <IP_ADDRESS> should float, but it won't open ~
What should I change about Linux or private_constants.js?

put the actual IP address the Opentrade is operating in. Make sure the appropriate file is edited to reflect this.


[BUG] AttributeError: module 'cudf' has not attribute '_gdf' (in advanced Dask with cuDF and XGBoost)

This bug was found in:
/notebooks-extended/advanced/examples/Dask_with_cuDF_and_XGBoost.ipynb (NOTE: Problem also occurs in Dash_with_cuDF_and_XGBoost_from_Disk.ipynb)
When I try to run the second cell under the section "Use cuIO to Load GPU DataFrames using Dask," I get the following error:
AttributeError: module 'cudf' has no attribute '_gdf'
This error message is followed by a bunch of lines that seem to be causing the error. Consequently, I cannot execute most of the commands that follow that error message. If needed, I can paste the entire error message in this thread.
To be honest, I am not sure what the expected behavior is because there is no written explanation leading up to the cells that need to be executed. Any help would be appreciated. Thanks!

Environment location: Docker
Method of RAPIDS libraries install: Docker

docker pull & docker run commands used:

docker pull rapidsai/rapidsai:0.7-cuda10.0-runtime-ubuntu18.04-gcc7-py3.7
docker run --runtime=nvidia --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 \
rapidsai/rapidsai:0.7-cuda10.0-runtime-ubuntu18.04-gcc7-py3.7

@dlymarg thanks for finding this bug, and sorry about the delay.  While this is in the wrong repo, we need to update this notebook as it uses dask_delayed and not dask_cudf.  RMM is becoming its own standalone repo, and even some the quick fixes, it doesn't pickle well.  I'm not sure how  critical the bug is for you, but please let me know if this is a blocking you for our prioritization and stand by for a PR with the solution!
Thanks again!

@dlymarg talked with the notebook creator and he has a similar, yet updated, notebook in the repo.  We'll be archiving this notebook and a few others.  Thanks again!


[feat]:  Only return parquet metadata if intending to write

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 92.97%. Comparing base (8cb8994) to head (5aa1a26).
Report is 148 commits behind head on main.

Files with missing lines
Patch %
Lines

src/dask_awkward/lib/io/parquet.py
75.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #549      +/-   ##
==========================================
- Coverage   93.06%   92.97%   -0.10%
==========================================
  Files          23       22       -1
  Lines        3290     3390     +100
==========================================
+ Hits         3062     3152      +90
- Misses        228      238      +10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Thank you for adding this option @martindurant so quickly!
I'm redirecting this test to @ikrommyd as he has an analysis setup that can test this :+1:

@martindurant could we also get an option for the tree reduce?

@lgray : tree= option seems to work

I will try those out. Takes some time because I’m trying a large enough sample to see those issues and also because I’m monitoring the dashboard as it’s running.

@pfackeldey : added fire_and_forget flag to to_parquet. Give it a try? You must already have your dask client instantiated.

So we need the client to be already up when we will dak.to_parquet (when building the graph)?

Correct. This was the quick and easy way to do it.
On 8 Nov 2024 17:50, Iason Krommydas @.***> wrote:
@pfackeldeyhttps://github.com/pfackeldey : added fire_and_forget flag to to_parquet. Give it a try? You must already have your dask client instantiated.
So we need the client to be already up when we will dak.to_parquet (when building the graph)?
—
Reply to this email directly, view it on GitHubhttps://github.com/dask-contrib/dask-awkward/pull/549#issuecomment-2465867944, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABODEZCYBEHZ6EB3C6EV4C3Z7U537AVCNFSM6AAAAABQMSMIOSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVHA3DOOJUGQ.
You are receiving this because you were mentioned.Message ID: @.***>

So first report from trying out the new fire_and_forget and tree options:

The tree option seems to work very well. The workflow succeeded the first time without errors. The workers still had unmanaged memory(old) in the GB order but the to-parquet tasks weren't accumulating on them. They were being forgotten and weren't staying in memory. When the merging to the final write-parquet task after the tree reduction was about to happen, I saw no spike in memory of 1-2 workers that which would happen if they had to accumulate all the to-parquet tasks. Even better, because one worked had died for different reasons, not a lot of tasks hard to be redone with tree reduction. The computation reached the end, couldn't gather the results from this dead worker and only went back and re-did ~100 tasks (out of many thousands) and then the computation finished first try.

the fire_and_forget option seems to work (I see files being written on disk) but there is a problem. The dask.compute(to_compute) doesn't hold the interpreter. So the script proceeds and then reaches its end and when the python interpreter exits, it kills the client and the scheduler. I don't know if there is a way to prevent this or if I'm doing something wrong. However, until the client got killed, I was able to see tasks being computed in the dashboard and files being written to disk. There is some sense of tracking in the dashboard as well. I'm seeing 0/X to-parquet tasks completed as it doesn't track them with this option but X keeps getting smaller and smaller due to the remaining tasks number becoming smaller. So you do have a sense of how many writing tasks you have remaining.

Cool - this is useful information re: tree reduction. It would seem we should try to use it in as many remaining places as possible where we otherwise have N:1 input-to-output partitions (like collecting finalized arrays or similar things).
Histograms are already a tree reduction but those face different issues. However, used in a few places here it could bring us the robustness we appear to be missing?
This also brings up the issue - why the heck are distributed tasks taking up so much memory!? There's an additional class that represents a task in distributed which is surely eating up some space if tasks are hanging around.
I guess we should think carefully about lifecycles.

why the heck are distributed tasks taking up so much memory

Quite. I suggested that perhaps a worker plugin can figure out what's being allocated as tasks go through their lifecycles, perhaps on one-thread workers. Usual tools like object growth and reference cycle finders would be the first line of attack. I'm not certain that the worker plugin system (transition method) has enough granularity, but it's an easy place to start. https://distributed.dask.org/en/stable/plugins.html#worker-plugins

We need to be careful with fire_and_forget since it depends on whatever is executing tasks being interface-similar to distributed. We already have options that are not, some logic to check what's being used to execute the graph and error out if it isn't distributed is probably useful.

it depends on whatever is executing tasks being interface-similar to distributed.

At least we would fail early at get_client, but your point is valid. As implemented in this PR, it is only for trialing and getting the kind of information @ikrommyd supplied, of course.

I've just tried the fire_and_forget as well. I stopped the interpreter from going past the dask.compute() call by just adding a input("Press enter when the computation finishes") and monitored the dashboard. All went fine just like in the tree-reduction case. By the end, two workers had died and the tasks those workers had into memory we just redone by other workers that spawned in the end to do just that. I got exactly the same number of parquet files with tree-reduction and fire and forget and no memory problems (the unmanaged memory (old) of the workers was still in the GB scale for fire and forget as well).

I'm the weekly awkward meeting, we decided that tree reduction should become the only implementation for write-parquet (it amounts to the same layout in the case of few partitions). The fire-and-forget route will be removed from this PR and maybe can be resurrected in a separate one for those that want it. Aside from being distributed-specific, it comes with the problem of not knowing when your process is finished.

Would be nice to add the same feature here: https://github.com/scikit-hep/uproot5/blob/734700ef1f822338b03a7573df484909b317b2c2/src/uproot/writing/_dask_write.py

@ikrommyd - certainly, but that would be a separate PR of course. It does seem like the metadata in that case is simply discarded anyway.

Yeah it's hard to achieve the scale in CI to test the actual performance impact of this PR.


ZeissDataReader Shifts and Metadata

For about one month, I have been trying to reconstruct a dataset (cone-beam geometry) using the ZeissDataReader, and I found some bugs in terms of reading the actual data and reading the metadata.

First of all, using the dxchange instead of xrmreader, we lose some important information for the metadata. For my data,  if I use dxchange.read_txrm(filename_sinogram) function the keys of the metadata dictionary are

dict_keys(['facility', 'image_width', 'image_height', 'data_type', 'number_of_images', 'pixel_size', 'reference_filename',
'reference_data_type', 'thetas', 'x_positions', 'y_positions', 'z_positions', 'x-shifts', 'y-shifts', 'reference'])

If I use the xrmreader.read_metadata(filename_sinogram) function the keys of the metadata dictionary are

dict_keys(['facility', 'image_width', 'image_height', 'data_type', 'number_of_images', 'pixel_size', 'reference_exposure_time',
'reference_current', 'reference_voltage', 'reference_data_type', 'image_data_type', 'thetas', 'x-shifts', 'y-shifts', 'AMC-x-shifts',
'AMC-y-shifts', 'temperature-x-shifts', 'temperature-y-shifts', 'source-x-shifts', 'source-y-shifts', 'align-mode', 'center_shift',
'rotation_angle', 'source_isocenter_distance', 'detector_isocenter_distance', 'cone_angle', 'fan_angle', 'camera_offset', 'source_drift',
'current', 'voltage', 'power', 'exposure_time', 'binning', 'filter', 'scaling_min', 'scaling_max', 'reference’])

I do not know why, but the latter seems more complete.  Besides the x-shifts, y-shifts and the reference we have more important shifts such as  AMC, temperature and source. For more information, see here. There are also some other keys which I have no idea what they describe, such as align-mode, cone_angle, fan_angle, camera_offset, scaling_min, scaling_max.  If anyone is familiar with these keys, I would really appreciate some explanation.
Example
If we use the ZeissDataReader to read the WalnutData, everything is ok. Using the xrmreader.read_metadata(filename_sinogram) function, the AMC, temperature are None but the source-x-shifts and source-y-shifts are close to 0, meaning that we do not need to include this correction.
AMC-x-shifts =  None
AMC-y-shifts =  None
temperature-x-shifts =  None
temperature-y-shifts =  None
source-x-shifts =  [0.00010398 0.00020615 0.00030651 0.00040505 0.00050179 0.0005967
 0.00068981 0.0007811  0.00087058 0.00095825]
source-y-shifts =  [-1.95432222e-05 -3.88436601e-05 -5.79013140e-05 -7.67161837e-05
 -9.52884438e-05 -1.13617745e-04 -1.31704321e-04 -1.49548112e-04
 -1.67149119e-04 -1.84507342e-04]
align-mode =  None
center_shift =  None
rotation_angle =  None

However, in my data I have something like:
x-shifts =  [-0.02338324 -8.31449032  3.44475412  7.60942364  4.46493196  1.39798403
  6.30554199 -0.50505334 -2.10339952  9.3211956 ]
y-shifts =  [-0.06622199 -0.95438933  2.88818741  0.95634484 -2.7804389   0.52345252
 -1.7584523  -1.91345716  2.86733198 -0.94697839]
AMC-x-shifts =  [-0.03012854 -0.02821015 -0.02629176 -0.02437337 -0.02245498 -0.02053659
 -0.0186182  -0.01669981 -0.01478142 -0.01286303]
AMC-y-shifts =  [0.10338005 0.10249254 0.10160504 0.10071754 0.09983003 0.09894253
 0.09805502 0.09716751 0.09628001 0.09539251]
temperature-x-shifts =  [-0.07080591 -0.06542346 -0.07087189 -0.0660259  -0.07146409 -0.07147949
 -0.06607211 -0.07149497 -0.07149501 -0.06607226]
temperature-y-shifts =  [-0.11793955 -0.11193907 -0.11793955 -0.10978469 -0.11578517 -0.11578517
 -0.10978469 -0.11578517 -0.11578517 -0.10978469]
source-x-shifts =  [0.00125837 0.00251942 0.00378316 0.00504959 0.00631873 0.00759054
 0.00886504 0.01014224 0.01142212 0.0127047 ]
source-y-shifts =  [0.00138348 0.00277969 0.00418863 0.00561029 0.00704469 0.00849178
 0.00995162 0.01142418 0.01290946 0.01440747]
align-mode =  [3]
center_shift =  0.8091099262237549
rotation_angle =  0.0

meaning that somehow we need to correct our projections for all of these shifts.
Solution
I have tried a lot of things which I will not mention here, in order to understand what is the problem with the CIL reconstruction and why is so different from the recon.txrm which I guess is from a Zeiss software.  The problem is in the following lines
https://github.com/TomographicImaging/CIL/blob/be57697c96d3e66032bac5de5246d0d276e3ba1e/Wrappers/Python/cil/io/ZEISSDataReader.py#L273-L276
We use np.roll which works only with integer values. Meaning that with an int rounding we do not perform a good interpolation. Actually, all the values in the x-shifts and y-shifts vectors that are in $\in(-1. , 1.)$ will end up to be 0 after the rounding. Below are two histogram plots of the y-shifts restricted to  $\in(-1. , 1.)$ for the Walnut dataset with/without the int rounding

Why the Walnut recon looks correct using the np.roll and the int rounding???
Because the number of projections (108) where the y-shifts is $\in(-1. , 1.)$ is small compared to the total number of projections(800).
In my dataset the histogram plots are:

which means that the y-shifts for 300 projections out of 800 projections were approximated by 0.
Correct Implementation
is using the shift by scipy. This is how xrmreader implements the interpolation for all the shifts. Although, I am not sure if nearest mode and order 1 are good defaults.
def revert_shifts(projections, x_shifts, y_shifts):
    '''

    :param projections: projection data
    :param x_shifts: x shift parameter per projection
    :param y_shifts: y shift parameter per projection
    :return: shift corrected porjections
    '''
    num_projections, img_size_x, img_size_y = projections.shape
    shifts = np.zeros((num_projections, 2))
    shifts[:, 0] = x_shifts
    shifts[:, 1] = y_shifts

    # shift each projection
    for i in range(num_projections):
        # input is extended by repeating its last value to avoid zeros after cropping
        projections[i, :, :] = shift(projections[i, :, :], (shifts[i, 1], shifts[i, 0]), order=1, mode='nearest')

    return projections

In my case, I need also to correct for AMC, temperature, source shifts as well as to correct for the offset due to the center_shift. Below is a comparison, with all these different options a) using np.roll for x/y shifts, b) using scipy.shift for x/y shifts, c) using scipy.shift for all shifts & correcting the offset and d) the txrm.recon.

@antonyvam for future reference.

@epapoutsellis thanks for this detailed report.
It looks like there are 2 issues we should deal with:

extra fields that are not read in by the reader. I had a look at the xrmreader package and it seems it has 1 version only and the license is similar to MIT.
interpolating data for subpixel shifts. Our preferred path would be to handle this with a per-projection geometry setup, which is not currently available in CIL, but in development.

Maybe you can join the next developer's meeting or the user support.

@paskino Apologies but have meetings Tuesday and Friday. Will try to join on Friday. This is a recent package from Andreas Maier group.
What do you mean by per-projection geometry setup? I strongly suggest to use a parallel processing library to do all these shift corrections for all projections. Otherwise, it takes A LOT OF TIME. You know my favourite library for it.

I mean that we don't have to apply the shifts at all, just record them in the geometry of each projection. Then we pass this info to the engine which will do the projection.
Feel free to suggest another meeting time if those don't work for you.

@epapoutsellis can you list the additional parameters you require? These could be added to dxchange with a PR.
Thanks for sharing a ZEISS reference.

After discussion with @epapoutsellis @gfardell @effepivi we propose to tackle the reader issues in different ways:

remove np.roll in favour of scipy.shift with default to do same as what we do now (no backward incompatibility)
Proceed adding new metadata to dxchange, see above
The future reader will be able to return a per-projection geometry OR a shifted dataset with one single geometry


UserWarning: Incompatible model and inference data.

Hi,
Beginner here. I tried to reduce my model to as few variables as possible, and follow the Quickstart, but still getting this error. Not sure how to debug.
ppi = kpt.ProjectionPredictive(model, idata)

Repro is here: https://github.com/lucabol/divs/blob/master/Analysis.ipynb
Full error below
UserWarning                               Traceback (most recent call last)
Cell In[6], [line 3](vscode-notebook-cell:?execution_count=6&line=3)
      [1](vscode-notebook-cell:?execution_count=6&line=1) #idata.to_netcdf("kptidata1.nc")
      [2](vscode-notebook-cell:?execution_count=6&line=2) #idata = az.from_netcdf("kptidata1.nc")
----> [3](vscode-notebook-cell:?execution_count=6&line=3) ppi = kpt.ProjectionPredictive(model, idata)
      [4](vscode-notebook-cell:?execution_count=6&line=4) ppi.search()

File ~/miniconda3/envs/bap3/lib/python3.11/site-packages/kulprit/reference.py:62, in ProjectionPredictive.__init__(self, model, idata)
     [60](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/lucabol/dev/divs/~/miniconda3/envs/bap3/lib/python3.11/site-packages/kulprit/reference.py:60) # test compatibility between model and idata
     [61](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/lucabol/dev/divs/~/miniconda3/envs/bap3/lib/python3.11/site-packages/kulprit/reference.py:61) if not check_model_idata_compatability(model=model, idata=idata):
---> [62](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/lucabol/dev/divs/~/miniconda3/envs/bap3/lib/python3.11/site-packages/kulprit/reference.py:62)     raise UserWarning("Incompatible model and inference data.")
     [64](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/lucabol/dev/divs/~/miniconda3/envs/bap3/lib/python3.11/site-packages/kulprit/reference.py:64) # log reference model and inference data
     [65](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/lucabol/dev/divs/~/miniconda3/envs/bap3/lib/python3.11/site-packages/kulprit/reference.py:65) self.model = model

UserWarning: Incompatible model and inference data.

Hi @lucabol, thanks for reporting the issue and providing an example. I am on vacations and away from my laptop. I will be able to check this next week, when I come back.

No problem. Take your time. I think it is likely something crazy I did as I am new to all of this.

Having a look at your notebook cell count it looks like you ran ppi = kpt.ProjectionPredictive(model, idata) before running idata = model.fit(idata_kwargs={'log_likelihood': True}). What happens if you restart the notebook and run from top to bottom?

I am not sure why github puts those numbers beside the blocks. In VsCode the code blocks are numbered in increasing order (as you see it in the files).
In any case, I had rerun it block by block making sure I run model.fit first, and still see the same problem.

Closing as I am not able to reproduce the error with older or current versions. @lucabol If you are still interested in this please try the version from main, or wait for the next release and reopen if you still see this issue.  Thanks!


Fetch loop on repeated mounting and un-mounting

I've noticed a strange behaviour on repeated mounting and un-mounting.
What happens is that i have tabs component that mounts a specific components on every tab change,
so every time i switch between tabs, the current tab component is mounted again.
I'm using useQuery on each components to fetch the relevant information for the current tab.
When i'm doing the following actions the useQuery getting into fetch loop that never ends.

mount the current tab component
useQuery fetches the data
move to other tab ( the previous component now unmounted and a new component tab mounted)
get back to the first tab component ( now after the cacheTimeout has been expired )
useQuery now getting into fetching loop with infinite fetch requests.

See the video below:
https://streamable.com/a90vs
Hope i'm clear enough, let me know if you need more information in order to reproduce it.
Thank you so much!

Great idea -  i will prepare one

@tannerlinsley after many failed attempts to try and reproduce it in codesandbox , i've updated react and react-query package and the problem was gone..
it will remain a mystery ...


Support mobile, react native

The first thing i want to say thank you so much to you tradingview/lightweight-charts team for create an awesome open source for us.
I'm currently using this lib as well for my business.
I hope that you guys can start to create a new version for mobile, react native. Because i know that for now, don't have to much chart lib support for mobile as well, like the way you did with this lib.
Best regards

Any update here?

@dotku any update on what? lightweight-charts is web-based library which does work on mobile devices and handles touch devices properly

@timocov How do you use the createChart API with react native? There is no HTML element there.
Is this the only way to do? example


DATAMIN, DATAMAX and BUNIT in FITS header for each product

'DATAMIN'  > 'Minimum valid physical value'
'DATAMAX' > 'Maximum valid physical value'
'BUNIT' > 'Units of physical value, after application of BSCALE, BZERO'

are fits header keywords used by SOAR for user filtering...
as some products have multiple data-streams we have to decide for the "most interesting one"
so fare we only provide real values for count data

'DATAMIN'  =  max('counts')
'DATAMAX' = max('counts')
'BUNIT' = "counts"

ql-lightcurve
ql-background
ql-spectra
xray-scpd
xray-cpd
xray-rpd

open to decide: currently proposals

xray-vis
aspect-burst min/max(cha_diode0, cha_diode, chb_diode0, chb_diode1)
ql-variance
ql-flareflag min/max(loc_y, loc_z) unit ?
cal-energy
tmstatusflarelist
hk-mini
hk-maxi
aux-ephemeris: min/max(y_srf, z_stf) unit km

It would be really usefull to add explain, expmax for the shortest and longest exposure time too.

i added a "XPOMAX" keyword with the longest exposure time: self.data['timedel'].as_float().max().to_value('s')

To #393 or somewhere else?


Link to nuget-package

Hi there, will Leaflet get an update on the package?

Hi
Any plan on updating the nuget package ?

Nobody in the leaflet team has permission (nor time) to manage the nuget package. According to https://www.nuget.org/packages/leaflet , the burden lays on @resnyanskiy , @ernstmartinj and either @Harel or @Harelba.

I'm no longer involved in this.
/M
On Mon, Nov 21, 2016 at 2:23 PM Iván Sánchez Ortega <
<EMAIL_ADDRESS>wrote:

Nobody in the leaflet team has permission (nor time) to manage the nuget
package. According to https://www.nuget.org/packages/leaflet , the burden
lays on @resnyanskiy https://github.com/resnyanskiy , @ernstmartinj
https://github.com/ernstmartinj and either @Harel
https://github.com/Harel or @Harelba https://github.com/Harelba.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Leaflet/Leaflet/pull/1841#issuecomment-261937417, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA-P7h2xDZHuMHhlKeBQ-abfLZD8BZ9fks5rAZtjgaJpZM4Ay9OX
.

--
Martin Jacobsson
<EMAIL_ADDRESS>+46 70 584 00 94
www.fablernas.com


Push Code to repo

Automate tasks of running add, commit and push commands to directly push code instead of manual using cmd

Kindly assign me this issue under Gssoc'23

Hey @bhavika2502 , you can start working on this issue.


fix: canonicalize path for tmp test directories

closes #465.

I cannot always reproduce #465 always but with this change it seems like it does not happen anymore, curious about what it does in your systems.


rename sample playbooks, delete duplicate

Purpose

...

Does this introduce a breaking change?

[ ] Yes
[ ] No

Pull Request Type
What kind of change does this Pull Request introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

Install ansible

$ pip install ansible[azure]

[Optional] Install ansible role

$ ansible-galaxy install Azure.azure_preview_modules
$ pip install -r ~/.ansible/roles/Azure.azure_preview_modules/files/requirements-azure.txt

Get the sample playbook

git clone https://github.com/Azure-Samples/ansible-playbooks.git
cd ansible-playbooks
git checkout [branch-name]

Test the code

ansible-playbook <filename>.yml

What to Check
Verify that the playbook is successfully run.
Other Information

@kyliel This PR need you review, Could you help to check it？

@kyliel Could you help review this PR? Thanks!

@Fred-sun  I'll take care of this one. thanks.

Thanks @yungezz

@yungezz Could you help confirm this PR finish review and go to next step forward? Thanks!

@yungezz Could you help to update the PR status? Thanks1

@Fred-sun Please skip this PR since @yungezz will take care of this one

@yuwzho Ok, I will ,Thanks!

@yungezz

@yungezz

@yungezz Could you help resolve the conflicting files and push to merged? Thanks!

@yungezz

@yungezz Could you help resolve the conflicting files and push for merge? Thanks!

@yungezz Could you help resolve the conflicting files when you're available?  and push for review. Thanks!

@yungezz need_update

ping


Chai AssertionError for emit withArgs(nestedStruct)

Hi,
Env:

<EMAIL_ADDRESS><EMAIL_ADDRESS>
Solidity:
struct Task {
    address foo;
    uint256 next;
    address[] bar;
}

struct TaskReceipt {
    uint256 id;
    address userProxy;
    Task task;
}

I get this errror when trying to listen to the emission of TaskReceipt via
....
to.emit(contract, "LogTaskSubmitted").withArgs(taskReceiptStruct);

Error:
AssertionError: expected { Object (id, userProxy, ...) } to equal [ Array(3) ]
I assume this error is because of the way ethers v4 4.0.47 represents nested structs.
Because in this example I pass taskReceiptStruct to .withArgs as a JS obj. Something like:
const taskReceiptStruct = {
     id: 1,
     userProxy: "0xad..",
     task: {
         foo: "0xdff...",
         next: 2,
         bar: ["0xfa."]
     }
}

It would be great to get some help on this issue.
Thank you!

I don’t know Chai and that error doesn’t look like it is coming from ethers. Can you provide an example that just uses ethers?

As discussed in Gitter @ricmoo, I am pretty sure the issue is here: .withArgs(taskReceiptStruct);
I pass taskReceiptStruct as a nested JS obj. However, ethers v4 structs in parsed event logs are represented as Arrays. So that's where the errror comes from, I am pretty certain.

Hi,
So I have an update on this:
I now use a script to convert my nested JS Obj that should represent a nested Solidity struct into an ethers v4 complian nested Array that is supposed to match the Array you get from parsing the event log in which the nested solidity struct was emitted.
I have made some progress in my waffle test but I am stuck with this:
The nested Struct JS Obj which I converted into an Array in order to test, whether it is emitted in an event:
[
  BigNumber { _hex: '0x01' },
  '0x6CBb2766B408f9A603792b8B546169E0b6Ad47ae',
  [
    [ [Array], [], [Array], [BigNumber], false ],
    BigNumber { _hex: '0x01' },
    []
  ]
]

The actual array representation thereof that I fetch from the parsed event log:
[
  BigNumber { _hex: '0x01' },
  '0x6CBb2766B408f9A603792b8B546169E0b6Ad47ae',
  [
    [ [Array], [], [Array], [BigNumber], false ],
    BigNumber { _hex: '0x01' },
    []
  ]
]

Hence, this looks pretty similar to me.
I also checked if the 2 arrays are similar using the recursive function (converting all elements to String)
function nestedArraysEqual(a, b) {
  if (a === b) return true;
  if (a == null || b == null) return false;
  if (a.length != b.length) return false;

  for (var i = 0; i < a.length; ++i) {
    // console.log(
    //   `${a[i]} !== ${b[i]}  ? : ${a[i].toString() !== b[i].toString()}`
    // );
    if (Array.isArray(a[i])) return nestedArraysEqual(a[i], b[i]);
    if (a[i].toString() !== b[i].toString()) return false;
  }
  return true;
}

And this function tells me the 2 arrays are equals.
So I wanna test if the struct is emitted in an event:
.withArgs(taskReceiptAsArray);
But waffle gives me this error:

      at assertArgsArraysEqual (node_modules/@ethereum-waffle/chai/dist/cjs/matchers/emit.js:37:59)
      at tryAssertArgsArraysEqual (node_modules/@ethereum-waffle/chai/dist/cjs/matchers/emit.js:47:20)
      at /Users/luis/Desktop/gelato-V1/node_modules/@ethereum-waffle/chai/dist/cjs/matchers/emit.js:59:13
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)
      at Context.<anonymous> (test/gelato_core/core/task_cycles/cycle.test.js:215:5)

At this point I assume that the waffle chai emit matcher does not know how to compare nested ethers v4 style Array representations of nested Structs from parsed Event Logs.
@ricmoo Do you maybe have any idea why this comparison fails?

@gitpusha Yeah, it is probably the case that the matcher could do a better job comparing the arrays.

Hi @sz-piotr . Thanks for stepping in. Yes, indeed. However, apparently there is a significant difference in the way ethers-v4 and ethers-v5 represent nested solidity structs in the parsed event logs. I intend to switch over to ethers-v5 as soon as Waffle supports it too.
Do you think you will release an update fixing this for v4 still any time soon, or is it not a prio?

Hard to tell. Currently we have our eyes set straight on Waffle 3.0.0 with Ethers 5.0.0 support. If bugs for waffle v2 keep piling up we'll probably release another v2 version.

Hi @gitpusha,
That looks like something we should do.
Currently however we are snowed under projects and it is hard to find just couple hours to fix it.
We are looking for new people to join, so they can help, will let you know as soon as we have more time.
Marek

Hi all..
Same problem for me too even in 2021..
@gitpusha @marekkirejczyk  any interesting workaround you followed ? I'd appreciate if you could share it ..

Hey @novaknole - havent run into this in my code recently so sorry can't help with a quick workaround right now.

Still having the issue, we used the following workaround @ juicebox (ie pushing every value in a new array, effectively getting rid of the Result object from ethers): https://github.com/jbx-protocol/juice-contracts-v2/blob/405d67c16ed150313912d9a64dbf92f19cb7bbb1/test/jb_splits_store/set.test.js#L68
Unfortunately, it's not always possible (for instance when having the same events n-times within the same tx) :/

I also experienced this problem for an array nested in a struct:
      AssertionError: expected [] to equal []
      + expected - actual

      at assertArgsArraysEqual (node_modules/@ethereum-waffle/chai/dist/cjs/matchers/emit.js:48:57)
      at tryAssertArgsArraysEqual (node_modules/@ethereum-waffle/chai/dist/cjs/matchers/emit.js:65:20)
      at /Users/jakub/workspace/project/node_modules/@ethereum-waffle/chai/dist/cjs/matchers/emit.js:77:13
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at runNextTicks (internal/process/task_queues.js:64:3)
      at listOnTimeout (internal/timers.js:526:9)
      at processTimers (internal/timers.js:500:7)

Yep getting the same issue:
  Uncaught AssertionError: expected [] to equal []
  + expected - actual

having the same issue here! is there a proper way of solving this?

We have some more resources in our internal team and are able to look into it now.
I will keep you updated in this issue.
If anyone has a workaround for now, please do share.

Structs in the arguments are now handled in Waffle 4.
Feel free to open new issues if you encounter any problems with it in Waffle 4.

I ran into this problem testing an event which returns a struct array, and taking a look at the changes to fix this issue, it seems like the case of struct arrays will not be fixed. The code would short circuit here since the expectedArgs[index] is an array https://github.com/TrueFiEng/Waffle/commit/b9af4f0d7fc99cd8ca3950aa9c9de31bd933c0ee#diff-8213c5fcf48a8703531394a5bde4bd4afa84d2b46bdf292cf94fa46fa0bf8fe8R30 and this ends up doing a normal equals comparison https://github.com/TrueFiEng/Waffle/commit/b9af4f0d7fc99cd8ca3950aa9c9de31bd933c0ee#diff-8213c5fcf48a8703531394a5bde4bd4afa84d2b46bdf292cf94fa46fa0bf8fe8R37-R39.


Implement Robusta action for C# debugger and diagnostics

Reference:

https://docs.microsoft.com/en-us/dotnet/core/diagnostics/#net-core-diagnostic-global-tools
https://github.com/OmniSharp/omnisharp-vscode/wiki/Attaching-to-remote-processes
https://github.com/Samsung/netcoredbg

Badly in need :)

Hey @RomitMehta,
What tools or workarounds are you using today, if any?
Do you have any interest in trying to add the feature yourself? We can help you out if so

Hi @aantn,
Yes, as of now we use dotnet monitor to get the trace file and dump for diagnostics.
I do have a interest in developing it along with my colleague @jaya-rama

That would be great. How can we help you get started?

@RomitMehta and @jaya-rama would you like to do this as part of the Rawkode month-long hackdays?
https://github.com/RawkodeAcademy/hackdays/issues/6


DOC: fixed the link for fluiddyn's transonic vision in dev/roadmap.html.

Reference issue
Couldn't find a relevant issue.
What does this implement/fix?
This fixed a wrong link for fluiddyn's transonic vision in dev/roadmap.html.
Additional information
Extremely minor fix, no additional info.

Thanks all.


[SPARK-38898][PYTHON][K8S] Force to remove pip .cache in python dockerfile

What changes were proposed in this pull request?
Add force flag to remove .cache
Why are the changes needed?
There are some flaky error when build spark python docker images due to  rm: cannot remove '/root/.cache': No such file or directory.
Does this PR introduce any user-facing change?
No
How was this patch tested?
K8s IT passed

Merged to master.


docs: update documentation (new links, minor improvements)

Hi @paul-shannon,
I've made minor corrections in the documentation :). Mostly fixes for the github.io links. Can you please have a look?

Looks fine, Arek!

There's also some links in the top-level README.md that need updating.

There's also some links in the top-level README.md that need updating.

Thanks @jmarshall. The links will be updated in this PR.


Generate redirect pages with sphinx

This PR, along with ros2/rosindex#139, fixes ~the first part of~ https://github.com/ros-infrastructure/rosindex/issues/138.

Generate pages that redirect the old URLs to the valid ones, ~based on a dictionary defined in the conf.py file~ collected by means of a custom redirect-from directive.
This conf.py is now used instead of the one previously located in the rosindex root directory, ~so that it can be updated if a document is moved/deleted without having to create two separate PRs.~ to make use of the added functionality.
Removed the (now unnecessary) hard coded redirect pages.

@dirk-thomas would you be willing to take a look?

@dirk-thomas @chapulina may I get a final review on this one?

Thanks for the review! Going in now!


DatagramSocket.getLocalAddress() is not returning the local address on Mac OS

Please provide a brief summary of the bug
Tested in 1.8.0_272/25.272-b10 and 1.8.0_322/25.322-b06 on Mac OS, use DatagramSocket to open a remote connection and then query the address of the local side of the connection.  Seems to work well on Windows and Linux, but returns a zero address on Mac OS.
Please provide steps to reproduce where possible
Test code to reproduce: https://github.com/cowtowncoder/java-uuid-generator/blob/master/src/main/java/test/EgressDiagnostics.java
Expected Results
Local address of the connection is returned, as it is on Windows: https://github.com/cowtowncoder/java-uuid-generator/pull/56#issuecomment-1214415012
Actual Results
On Mac OS, the local address is reported as zeroes: https://github.com/cowtowncoder/java-uuid-generator/pull/56#issuecomment-1214489004
What Java Version are you using?
1.8.0_322-b06
What is your operating system and platform?
java.runtime.version: 1.8.0_322-b06
java.vm.name: OpenJDK 64-Bit Server VM
java.vm.vendor: Temurin
java.vm.version: 25.322-b06
os.arch: x86_64
os.name: Mac OS X
os.version: 10.15.7
How did you install Java?
No response
Did it work before?
No response
Did you test with other Java versions?
No response
Relevant log output
No response

@pgalbraith  Can you try with the latest 8u345 and report back?

@karianna I do not see any difference on my system, unfortunately (either with 8u332 or 8u345)


get animation packet

I also added delay, which makes this a lot more stable

doesn't seem to get it for several versions ?

not working


Generating entity with sea-orm-cli panics when foreign key was added on table on postgres

Description
Generating entity with sea-orm-cli panics when the foreign key is added to the table on postgres.
Steps to Reproduce

Run this query to create the tables

create table "user"
(
    user_id       serial
        primary key,
    username      text                     not null
);
create table chat_channel_user
(
    user_id      integer              not null
        constraint chat_channel_user_user_user_id_fk
            references "user",
    channel_id   integer              not null,
    hidden       boolean default true not null,
    primary key (user_id, channel_id)
);

run sea-orm-cli generate entity
see the panics.

Expected Behavior
Should not crash and generate a schema.
Actual Behavior
It panics. I've checked with (tables without foreign keys) and it ran successfully.
Reproduces How Often
I'm guessing that this problem happens every time when foreign key is added.
Versions

OS: Windows 11 (x64)
PostgreSQL Version: PostgreSQL 15.3, compiled by Visual C++ build 1914, 64-bit
sea-orm-cli: 0.12.3

Additional Information

The code where it crashes: https://github.com/SeaQL/sea-schema/blob/bdc6f5e2894838f5230a93108eed2c2b1467232c/src/postgres/parser/table_constraints.rs#L52

The logs that ran sea-orm-cli generate entity --verbose: log.txt

May relavant: https://github.com/SeaQL/sea-schema/issues/28
Here's the query result of
SELECT
  "table_constraints"."constraint_schema",
  "table_constraints"."constraint_name",
  "table_constraints"."table_schema",
  "table_constraints"."table_name",
  "table_constraints"."constraint_type",
  "table_constraints"."is_deferrable",
  "table_constraints"."initially_deferred",
  "check_constraints"."check_clause",
  "key_column_usage"."column_name",
  "key_column_usage"."ordinal_position",
  "key_column_usage"."position_in_unique_constraint",
  "referential_constraints_subquery"."unique_constraint_schema",
  "referential_constraints_subquery"."unique_constraint_name",
  "referential_constraints_subquery"."match_option",
  "referential_constraints_subquery"."update_rule",
  "referential_constraints_subquery"."delete_rule",
  "referential_constraints_subquery"."table_name",
  "referential_constraints_subquery"."column_name"
FROM
  "information_schema"."table_constraints"
  LEFT JOIN "information_schema"."check_constraints" ON "table_constraints"."constraint_name" = "check_constraints"."constraint_name"
  AND "table_constraints"."constraint_catalog" = "check_constraints"."constraint_catalog"
  AND "table_constraints"."constraint_schema" = "check_constraints"."constraint_schema"
  LEFT JOIN "information_schema"."key_column_usage" ON "table_constraints"."constraint_name" = "key_column_usage"."constraint_name"
  AND "table_constraints"."constraint_catalog" = "key_column_usage"."constraint_catalog"
  AND "table_constraints"."constraint_schema" = "key_column_usage"."constraint_schema"
  AND "table_constraints"."table_catalog" = "key_column_usage"."table_catalog"
  AND "table_constraints"."table_schema" = "key_column_usage"."table_schema"
  AND "table_constraints"."table_name" = "key_column_usage"."table_name"
  LEFT JOIN (
    SELECT
      DISTINCT "referential_constraints"."constraint_name",
      "referential_constraints"."unique_constraint_schema",
      "referential_constraints"."unique_constraint_name",
      "referential_constraints"."match_option",
      "referential_constraints"."update_rule",
      "referential_constraints"."delete_rule",
      "constraint_column_usage"."table_name",
      "constraint_column_usage"."column_name"
    FROM
      "information_schema"."referential_constraints"
      LEFT JOIN "information_schema"."constraint_column_usage" ON "referential_constraints"."constraint_name" = "constraint_column_usage"."constraint_name"
  ) AS "referential_constraints_subquery" ON "table_constraints"."constraint_name" = "referential_constraints_subquery"."constraint_name"
WHERE
  "table_constraints"."table_schema" = 'public'
  AND "table_constraints"."table_name" = 'chat_channel_user'
  AND "referential_constraints_subquery"."table_name" NOT IN (
    SELECT
      "pg_class"."relname"
    FROM
      "pg_inherits"
      JOIN "pg_class" ON "pg_inherits"."inhrelid" = "pg_class"."oid"
      AND "pg_class"."relkind" IN ('r', 't', 'v', 'm', 'f', 'p')
  )
ORDER BY
  "table_constraints"."constraint_name" ASC,
  "key_column_usage"."ordinal_position" ASC,
  "referential_constraints_subquery"."unique_constraint_name" ASC,
  "referential_constraints_subquery"."constraint_name" ASC

constraint_schema
constraint_name
table_schema
table_name
constraint_type
is_deferrable
initially_deferred
check_clause
column_name
ordinal_position
position_in_unique_constraint
unique_constraint_schema
unique_constraint_name
match_option
update_rule
delete_rule
table_name
column_name

public
2200_18205_1_not_null
public
chat_channel_user
CHECK
NO
NO
user_id IS NOT NULL
null
null
null
null
null
null
null
null
null
null

public
2200_18205_2_not_null
public
chat_channel_user
CHECK
NO
NO
channel_id IS NOT NULL
null
null
null
null
null
null
null
null
null
null

public
2200_18205_3_not_null
public
chat_channel_user
CHECK
NO
NO
hidden IS NOT NULL
null
null
null
null
null
null
null
null
null
null

public
chat_channel_user_pkey
public
chat_channel_user
PRIMARY KEY
NO
NO
null
user_id
1
null
null
null
null
null
null
null
null

public
chat_channel_user_pkey
public
chat_channel_user
PRIMARY KEY
NO
NO
null
channel_id
2
null
null
null
null
null
null
null
null

public
chat_channel_user_user_user_id_fk
public
chat_channel_user
FOREIGN KEY
NO
NO
null
user_id
1
1
public
user_pkey
NONE
NO ACTION
NO ACTION
user
user_id_i

However, the values are different than expected.
result.constraint_schema: "public"
result.constraint_name: "chat_channel_user_user_user_id_fk"
result.table_schema: "public"
result.table_name: "chat_channel_user"
result.constraint_type: "FOREIGN KEY"
result.is_deferrable: "NO"
result.initially_deferred: "NO"
result.check_clause: None
result.column_name: Some("user_id")
result.ordinal_position: Some(1)
result.position_in_unique_constraint: Some(1)
result.unique_constraint_schema: Some("public")
result.unique_constraint_name: Some("user_pkey")
result.match_option: Some("NONE")
result.update_rule: Some("NO ACTION")
result.delete_rule: Some("NO ACTION")
result.referential_key_table_name: None
result.referential_key_column_name: None

I've found that the user that connected does not have permission for something. I got fixed by adding permission to the user.
I'm not going to close this issue for better handling of this or so.

This seems to be a very rare situation, the column shouldn't return null.


message translated with LOCALES.lookup_complete() contains unexpected extra characters around place-ables

Hello,
i'm trying to write simple wrapper for my app to allow translation to a language changeable during runtime.
When I try to get the translation, the args are filled into the ftl string, and i get the translated result, but it contains extra "wrapper" characters around every place-able.
FTL file contents:
# $tokens (String) - Number of tokens on input.
# $values (String) - Number of values on input.
valid-count-mismatch =
    Number of tokens is not equal to number of values on input!
    Tokens: { $tokens }
    Values: { $values }

Code being executed:
type ArgsMap<'m> = HashMap<String, FluentValue<'m>>;

fn translate(text_id: &str, args: Option<&ArgsMap>) -> String {
    let lang_code = current_lang().to_lang_code();
    if let Some(li) = LANG_IDS.get(&lang_code) {
        println!("LanguageIdentifier:\n\t{:?}", li);
        println!("text_id:\n\t{:?}", text_id);
        println!("args:\n\t{:?}", args);
        let translated = &*LOCALES.lookup_complete(&li, text_id, args);
        println!("translated:\n\t{}", translated);
        println!("translated debug:\n\t{:?}", translated);
        return translated.to_string();
    }
    text_id.to_string()
}

Console output (please note extra characters around two place-ables on last line of printout):
LanguageIdentifier:
        LanguageIdentifier { language: Language(Some("en")), script: None, region: Some(Region("US")), variants: None }
text_id:
        "valid-count-mismatch"
args:
        Some({"values": String("1"), "tokens": String("3")})
translated:
        Number of tokens is not equal to number of values on input!
Tokens: 3
Values: 1
translated debug:
        "Number of tokens is not equal to number of values on input!\nTokens: \u{2068}3\u{2069}\nValues: \u{2068}1\u{2069}"

Windows console shows chars as sort graphical noise (console line translated:).
Github here does not show these above... It causes problem on windows, as GUI dialogs etc.
It is visible in debug dump on next line (translated debug:) - \u{2068} - \u{2069}.
Is this place-able wrapper intentionally done by the LOOKUP methods? If so, can it be disabled?
If not, what could be causing this behavior?...
I do NOT need any type specific extra  localization/internationalization features, so limiting myself to string is not a problem...
I'm not sure whether other argument types (thank String shown above) have same issue. I tried to work with generic argument map originally, that could allow numbers etc. (via Into), but failed to pass them around as parameters of methods due to being Rust beginner...
All files (FTL, rust .rs, ...) are UTF-8 encoded, LF line endings.
Thank you for tips/guidelines...

Thank you for your issue! Yes, I believe this is an intentional feature of fluent. Those are isolating marks that are used for allowing bidirectional text. You can disable this with FluentBundle::set_use_isolating. See the below example.
static_loader! {
    static LOCALES = {
        locales: "./tests/locales",
        fallback_language: "en-US",
        // Removes unicode isolating marks around arguments
        customise: |bundle| bundle.set_use_isolating(false),
    };
}

thank you for help and prompt response!
disadvantage of interconnected web of crates ~ sometimes beginners fail to identify the suspect correctly! :)

No worries, I was also confused when I first saw it. I'll add a note about it to the README.