CentOS 7 to AlmaLinux 8 Failed — Multiple Reasons?

Failed: making the conversation!. The reason: Command '['/usr/bin/leapp', 'preupgrade']' returned non-zero exit status 1.
The log is attached.
Server-Upgrade-Failed-log-20230427.txt.zip

Hello @gregarios
Thank you for contacting us. It seems that the problem is related to the conversion when old migrator repositories are enabled. To fix the problem, you could remove the repository. You could likely find it in the /etc/yum.repos.d/plesk-ext-panel-migrator.repo file.
I am currently investigating a better way to fix the problem within the tool itself.

To fix the problem, you could remove the repository. You could likely find it in the /etc/yum.repos.d/plesk-ext-panel-migrator.repo file.

Should I remove the plesk-migrator.repo file itself?
It contains:
`[plesk-migrator]
name=Plesk packages for migrator
baseurl=http://autoinstall.plesk.com/PMM_0.1.10/dist-rpm-CentOS-7-x86_64
enabled=1
gpgcheck=1
gpgkey=http://autoinstall.plesk.com/plesk.gpg
[plesk-migrator-tp]
name=Plesk 3d-party packages for migrator
baseurl=http://autoinstall.plesk.com/PMM_0.1.10/thirdparty-rpm-CentOS-7-x86_64
enabled=1
gpgcheck=0`

I removed the migration repo completely from Yum and restarted but I still gett he same error. Upon looking at the logs, it seems it is from a new repo your Alma update installs? See here:
2023-04-28 08:21:06,073 - INFO - No matches found for the following disable plugin patterns: subscription-manager
2023-04-28 08:21:06,073 - INFO - Repository extras is listed more than once in the configuration
2023-04-28 08:21:06,073 - INFO - Repository extras-source is listed more than once in the configuration
2023-04-28 08:21:06,073 - INFO - Errors during downloading metadata for repository 'alma-plesk-migrator-tp':
2023-04-28 08:21:06,073 - INFO - - Status code: 404 for http://autoinstall.plesk.com/PMM_0.1.11/thirdparty-rpm-RedHat-el8-x86_64/repodata/repomd.xml (IP: <IP_ADDRESS>)
2023-04-28 08:21:06,074 - INFO - Error: Failed to download metadata for repo 'alma-plesk-migrator-tp': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

Attached is a new complete log.
Server-Upgrade-Failed-log-20230428.txt.zip

Also, It seems the alma-plesk-migrator-tp repo, which I cannot find, requests the missing repo file too.
Attached is the leapp-preupgrade.log -- do a search for "Errors during downloading metadata for repository" in it to see more errors.
Server-Upgrade-Failed-leapp-preupgrade.log-20230428.txt.zip

I have incorporated some changes that should stop Leapp from utilizing the third-party repository during the conversion process. Related PR - #93.
It will not help if the thirdparty repository already leaked into leapp configs, but should prevent the problem on new conversion

Another fail from a fresh backup of my server. Before I started I made sure the Plesk Migrator repo files were all removed.
I got the same errors as before, then I went in looking for the migrator repo references you mentioned above but found none.
Here are the current contents of the two leapp files you mentioned:
` cat /etc/leapp/files/repomap.csv
CentOS 7 repoid,AlmaLinux 8 repoid in custom repo file,AlmaLinux 8 repo name in PES,CentOS 7 minor versions,AlmaLinux 8 minor versions,architecture,type(rpm/srpm/debuginfo),source product type (ga/beta,htb),target product type (ga/beta/htb)
Version: 1
Date: 20210726
x86_64
base,almalinux8-baseos,almalinux8-baseos,all,all,x86_64,rpm,ga,ga
base,almalinux8-appstream,almalinux8-appstream,all,all,x86_64,rpm,ga,ga
base,almalinux8-powertools,almalinux8-powertools,all,all,x86_64,rpm,ga,ga
base,almalinux8-ha,almalinux8-ha,all,all,x86_64,rpm,ga,ga
base,almalinux8-resilientstorage,almalinux8-resilientstorage,all,all,x86_64,rpm,ga,ga
updates,almalinux8-baseos,almalinux8-baseos,all,all,x86_64,rpm,ga,ga
updates,almalinux8-appstream,almalinux8-appstream,all,all,x86_64,rpm,ga,ga
updates,almalinux8-powertools,almalinux8-powertools,all,all,x86_64,rpm,ga,ga
updates,almalinux8-ha,almalinux8-ha,all,all,x86_64,rpm,ga,ga
updates,almalinux8-resilientstorage,almalinux8-resilientstorage,all,all,x86_64,rpm,ga,ga
extras,almalinux8-extras,almalinux8-extras,all,all,x86_64,rpm,ga,ga
epel,alma-epel,alma-epel,all,all,x86_64,rpm,ga,ga
epel-debuginfo,alma-epel-debuginfo,alma-epel-debuginfo,all,all,x86_64,rpm,ga,ga
epel-source,alma-epel-source,alma-epel-source,all,all,x86_64,rpm,ga,ga
mariadb,alma-mariadb,alma-mariadb,all,all,x86_64,rpm,ga,ga
`
`cat /etc/leapp/files/leapp_upgrade_repositories.repo
[almalinux8-baseos]
name=AlmaLinux 8 - BaseOS
mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/baseos
baseurl=https://repo.almalinux.org/almalinux/8/BaseOS/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux
[almalinux8-appstream]
name=AlmaLinux 8 - AppStream
mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/appstream
baseurl=https://repo.almalinux.org/almalinux/8/AppStream/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux
[almalinux8-powertools]
name=AlmaLinux 8 - PowerTools
mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/powertools
baseurl=https://repo.almalinux.org/almalinux/8/PowerTools/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux
[almalinux8-highavailability]
name=AlmaLinux 8 - HighAvailability
mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/ha
baseurl=https://repo.almalinux.org/almalinux/8/HighAvailability/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux
[almalinux8-resilientstorage]
name=AlmaLinux 8 - ResilientStorage
mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/resilientstorage
baseurl=https://repo.almalinux.org/almalinux/8/ResilientStorage/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux
[almalinux8-extras]
name=AlmaLinux 8 - Extras
mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/extras
baseurl=https://repo.almalinux.org/almalinux/8/extras/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-AlmaLinux
[alma-epel]
name=Alma Extra Packages for Enterprise Linux 8 - $basearch
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-8&arch=$basearch&infra=$infra&content=$contentdir
It is much more secure to use the metalink, but if you wish to use a local mirror
place its address here.
#baseurl=http://download.example/pub/epel/7/$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-8
[alma-epel-debuginfo]
name=Alma Extra Packages for Enterprise Linux 8 - $basearch - Debug
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-8&arch=$basearch&infra=$infra&content=$contentdir
It is much more secure to use the metalink, but if you wish to use a local mirror
place its address here.
#baseurl=http://download.example/pub/epel/7/$basearch/debug
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-8
gpgcheck=1
[alma-epel-source]
name=Alma Extra Packages for Enterprise Linux 8 - $basearch - Source
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-8&arch=$basearch&infra=$infra&content=$contentdir
It is much more secure to use the metalink, but if you wish to use a local mirror
place it's address here.
#baseurl=http://download.example/pub/epel/7/source/tree/
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-8
gpgcheck=1
[alma-mariadb]
name=Alma MariaDB
baseurl=https://mirror.rackspace.com/mariadb/yum/10.11/centos8-amd64
gpgkey = https://mirror.rackspace.com/mariadb/yum/RPM-GPG-KEY-MariaDB
gpgcheck=1
`

The issue is occurring due to the presence of the https://mirror.rackspace.com/mariadb/yum/10.11/ repository.  It seems that we are trying to map the currently used https://mirror.rackspace.com/mariadb/yum/10.11/centos7-amd64 to https://mirror.rackspace.com/mariadb/yum/10.11/centos8-amd64, but there is no such repository available. The correct mapping target should be https://mirror.rackspace.com/mariadb/yum/10.11/rhel8-amd64. I will verify the mapping on our end and update you with the results.

I've merged Rackspace mariadb repository support, related PR - #96.
To proceed with the conversion, remove the already added Rackspace repository mapping from the leapp_upgrade_repositories.repo.txt. Additionally, you could just change URL for the mapping, but note, that the repository adoption step in the finishing stage will be failed in this case.

No, released version don't contains changes I mentioned before.Let me just provide last build here.
centos2alma.zip

No, released version don't contains changes I mentioned before.Let me just provide last build here. centos2alma.zip

That made it a lot further! When it needed to reboot, it just disconnected me (I only have SSH access to the server on Google Cloud).  It would be nice if it gave a notice with a pause requiring a continue in order to get the message that it needed to reboot, as I just couldn't connect again indefinitely... after an hour of waiting for it to do something, I forced the server to shut down in the GCE Instance controls and it then booted back up and continued as seen in the log.
Nevertheless, it again failed but this time with a new problem seen at the end of the log attached:
centos2alma-20230503a.log.txt.zip
It seems AlmaLinux was installed, but failed to update the apps.

When it needed to reboot, it just disconnected me (I only have SSH access to the server on Google Cloud). It would be nice if it gave a notice with a pause requiring a continue in order to get the message that it needed to reboot

We will consider adding a 30-second pause before initiating the reboot to allow users to stop the process if they wish to avoid the reboot. However, the automatic reboot is necessary anyway, as we expect users to convert multiple servers and it would be inconvenient to switch between all of them to proceed with the conversion.
It is worth noting that after the preparation stage is completed, the server becomes inapplicable because all Plesk-related services are stopped and some conflicting packages are removed. Plesk can be restored using the ./centos2alma --revert command, but it is intended to be used mostly if the conversion cannot proceed.
To prevent reboot you could use hidden option --no-reboot. I could make it visible in the help message if you think it is necessary.

Nevertheless, it again failed but this time with a new problem seen at the end of the log attached:

It looks like the Plesk repositories are disabled during this step. So when the tool trying to install fail2ban it fails to find some dependencies.
Could you please provide a feedback archive from your host? You could create it by calling ./centos2alma --prepare-feedback.

Could you please provide a feedback archive from your host? You could create it by calling ./centos2alma --prepare-feedback.

Yes I'll run through the install again and get this for you.

We will consider adding a 30-second pause before initiating the reboot to allow users to stop the process if they wish to avoid the reboot. However, the automatic reboot is necessary anyway, as we expect users to convert multiple servers and it would be inconvenient to switch between all of them to proceed with the conversion.

I'm not looking to avoid the reboot, but rather just know that it is happening. Maybe while "paused" for a minute the status here could read "REBOOTING IN ONE MINUTE" so it can be known what's happening:

Also, my GCE instance did not seem to "reboot" when this happened, but rather just stopped. I had to manually turn it off and back on to get it back up. The only indicator that something wasn't right was that the CPU readings in the Google monitoring tools were showing no activity after it "rebooted" itself. I waited about an hour then forced it off and back on like a power button. Seems like there needs to be some kind of notification possibility for those of us without any screen capability to our server?

What the heck?! It worked this time? How can this be? I started with a clone of the previous try before it was attempted.
No errors this time. This is disturbing since I don't know what the problem was I can't be sure what the production server will do...

Any ideas?

my GCE instance did not seem to "reboot" when this happened, but rather just stopped. I had to manually turn it off and back on to get it back up.

Unfortunately, there is no way to monitor the conversion process when it is in the temporary OS. Since it is specialized for conversion, it does not have a network interface up, so there is no way to connect to it. This is why we advise having a console connection. It is the simplest way to monitor the conversion process inside the temporary OS. We have been looking for ways to make it possible to monitor the conversion process without a console connection, but so far, we have had no success.

Any ideas?

I assume the problem was caused by disabled Plesk repositories, hoping the feedback archive could confirm it. Let me try to find a way to reproduce the problem on my side. About why the problem didn't repeat - maybe it was fixed by plesk daily maintenance.

Seems the Plesk Settings for each hosted domain are defaulted.

Do you mean nginx/apache configurations, or something you configured inside the Plesk interface? Have you tried running plesk repair to see if it helps?

Do you mean nginx/apache configurations, or something you configured inside the Plesk interface? Have you tried running plesk repair to see if it helps?

Yes, the nginx/apache directives were default for the domain.
I ran plesk repair fully on the server and it did not fix it.
I will revert my test server back, take some screenshots of the domain settings, then convert it again and compare. I'll let you know the results and do another ./centos2alma --prepare-feedback

Ok I tried it again. Seems I didn't have any settings in there after all so it wasn't defaulted. Forget that problem.
Again, it installed AlmaLinux and I am able to log in to Plesk on it. However, the domain hosted is not accessible again. Everything in it's settings appears to be ok, but I get a "502 Bad gateway" error when trying to visit the site, and error messages in the log like this:
2023/05/05 10:01:19 [crit] 48354#0: *9 connect() to <IP_ADDRESS>:7081 failed (13: Permission denied) while connecting to upstream, client: <IP_ADDRESS>, server: askb.org, request: "GET / HTTP/2.0", upstream: "https://<IP_ADDRESS>:7081/", host: "askb.org"
I did run the feedback and gathered the centos2alma.log as well and here they are:
centos2alma_feedback_20230505.zip
centos2alma-20230505.log.zip
I ran plesk repair all -y -verbose and it completed with no errors showing and nothing to fix.
Here is the server info showing:
Product version: Plesk Obsidian <IP_ADDRESS>
OS version: AlmaLinux 8.7 x86_64
Build date: 2023/05/02 13:00
Revision: 1c44bda10dedda3c088c3da0271c725d3e2ec53b

Next problem: Spamassassin will not launch.
spamassassin-startup-messages.log.txt.zip
Doesn't appear to be an SELinux issue this time.
Ran plesk repair and it finds nothing.
Trying to turn on or off the mail filter settings in the UI causes Plesk to crash and go into the repair UI.

The main SELinux rule update that fixed a host of problems was:

Could you please verify if the httpd_can_network_connect was disabled prior to the conversion? As far as I am aware, leapp does not attempt to modify SELinux options directly; it only triggers relabeling during the initial boot after the conversion.

Next problem: Spamassassin will not launch.
...
running anything needing Perl produces this fatal error: /usr/bin/perl: symbol lookup error: /usr/local/lib64/perl5/auto/Cwd/Cwd.so: undefined symbol: Perl_xs_version_bootcheck

It is likely that a Perl-related package was not updated during the conversion. Could you please provide the package that provides the library file by running rpm -qf /usr/local/lib64/perl5/auto/Cwd/Cwd.so?

Could you please verify if the httpd_can_network_connect was disabled prior to the conversion? As far as I am aware, leapp does not attempt to modify SELinux options directly; it only triggers relabeling during the initial boot after the conversion.

Both before and after they are enabled. However, there are new entries in the updated version. Here are the output of  semanage boolean -l before and after the upgrade:
selinux_original_settings.txt.zip
selinux_updated_settings.txt.zip

It is likely that a Perl-related package was not updated during the conversion. Could you please provide the package that provides the library file by running rpm -qf /usr/local/lib64/perl5/auto/Cwd/Cwd.so?

I get this:
file /usr/local/lib64/perl5/auto/Cwd/Cwd.so is not owned by any package

Both before and after httpd_can_network_connect is enabled.

I have noticed that centos2alma sets selinux status to enforced even if it was disabled before the conversion. I've fixed this behaviour - PR #103.
Just to confirm, is SELinux in enforcing mode on your system during the conversion?

Just to confirm, is SELinux in enforcing mode on your system during the conversion?

It may have been, actually. In retrospect I should have turned it off.

Here is the output of perl -V on the machine pre-upgrade. Does this tell you anything?

Not really, it's actually the same perl as my own. I believe the problem in the library itself. Could you check if rpm -qf /usr/local/lib64/perl5/auto/Cwd/Cwd.so will show something adequate on CentOS 7 (before upgrade) by any chance?

Could you check if rpm -qf /usr/local/lib64/perl5/auto/Cwd/Cwd.so will show something adequate on CentOS 7 (before upgrade) by any chance?

It says:
file /usr/local/lib64/perl5/auto/Cwd/Cwd.so is not owned by any package

# rpm -qf /usr/local/lib64/perl5/auto/Cwd/Cwd.so
file /usr/local/lib64/perl5/auto/Cwd/Cwd.so is not owned by any package
# ll /usr/local/lib64/perl5/auto/Cwd/Cwd.so
-r-xr-xr-x. 1 root root 84K Feb 24  2020 /usr/local/lib64/perl5/auto/Cwd/Cwd.so
# ll /usr/local/lib64/perl5/auto/Cwd/
total 88K
-r-xr-xr-x. 1 root root 84K Feb 24  2020 Cwd.so
-rw-r--r--. 1 root root 988 Feb 24  2020 .packlist
# cat /usr/local/lib64/perl5/auto/Cwd/.packlist
/usr/local/lib64/perl5/Cwd.pm
/usr/local/lib64/perl5/File/Spec.pm
/usr/local/lib64/perl5/File/Spec/AmigaOS.pm
/usr/local/lib64/perl5/File/Spec/Cygwin.pm
/usr/local/lib64/perl5/File/Spec/Epoc.pm
/usr/local/lib64/perl5/File/Spec/Functions.pm
/usr/local/lib64/perl5/File/Spec/Mac.pm
/usr/local/lib64/perl5/File/Spec/OS2.pm
/usr/local/lib64/perl5/File/Spec/Unix.pm
/usr/local/lib64/perl5/File/Spec/VMS.pm
/usr/local/lib64/perl5/File/Spec/Win32.pm
/usr/local/lib64/perl5/auto/Cwd/Cwd.so
/usr/local/share/man/man3/Cwd.3pm
/usr/local/share/man/man3/File::Spec.3pm
/usr/local/share/man/man3/File::Spec::AmigaOS.3pm
/usr/local/share/man/man3/File::Spec::Cygwin.3pm
/usr/local/share/man/man3/File::Spec::Epoc.3pm
/usr/local/share/man/man3/File::Spec::Functions.3pm
/usr/local/share/man/man3/File::Spec::Mac.3pm
/usr/local/share/man/man3/File::Spec::OS2.3pm
/usr/local/share/man/man3/File::Spec::Unix.3pm
/usr/local/share/man/man3/File::Spec::VMS.3pm
/usr/local/share/man/man3/File::Spec::Win32.3pm

You can try executing the following command: cpan -r Cwd

I assume you mean before doing the AlmaLinux upgrade?

I assume you mean before doing the AlmaLinux upgrade?

I believe it doesn't really matter, and you can reinstall the modules after the conversion as well. However, I'm not entirely certain as I'm not familiar with the inner workings of cpan.

I believe it doesn't really matter, and you can reinstall the modules after the conversion as well. However, I'm not entirely certain as I'm not familiar with the inner workings of cpan.

Unfortunately, cpan is one of the things not working after the upgrade because of the Cwd error.

Unfortunately, cpan is one of the things not working after the upgrade because of the Cwd error.

Oh, in that case, the option to recompile the module seems to be unacceptable now. Good to know.
By the way, I will attempt to reproduce the problem on my end to confirm if the modules installed from cpan are causing the issue..

Likely the /usr/lib64/perl5/vendor_perl/auto/Cwd/Cwd.so was installed by the perl_PathTools package. Example from my instance:
# rpm -qf /usr/lib64/perl5/vendor_perl/auto/Cwd/Cwd.so
perl-PathTools-3.40-5.el7.x86_64

Maybe you even could fully remove the one that was installed by cpan.

Maybe you even could fully remove the one that was installed by cpan.

When you say "remove" do you mean, say:
mv /usr/local/lib64/perl5/auto/Cwd /home/gjpiper/storage/Cwd.bak
Is there a method? I've never removed anything from perl.

I have added the reinstallation of modules installed by CPAN and a checker to verify if the modules are in our mapping list (#104). I believe this should help with your problem.
However, if you have additional modules installed from CPAN, the conversion may fail during the Perl modules check. In such cases, you can use the --remove-unknown-perl-modules flag to force the conversion.

I will be trying the upgrade with a fresh clone of my server in the next few days. I will keep this thread "unclosed" to see if anything else unexpected happens on the next run. (if that works for you)
Thank you so much for your help! You really filled in the blanks in my knowledge to help me through this. So far so good.
Any chance there will be a new release of the script soon?

Any chance there will be a new release of the script soon?

Sure, we have plans to publish new release by 5th of June.

Sure, we have plans to publish new release by 5th of June.
If you would, can you send me a "nightly" again so I can try your latest beta update this round?

Sure. Here it is centos2alma.zip

Sure. Here it is centos2alma.zip

This is much improved, and in fact works quite well. I upgraded again and the server was still up and functioning, as were most of the Perl modules even from CPAN that were installed. I only see one problem with my install I can't seem to fix:
My original SpamAssasin installation (v3.4.6 running on Perl v5.16.3) is running and filtering mailboxes, but the RelayCountry MaxMind plugin is no longer working. Spamassassin just doesn't even recognize it. I reinstalled and am troubleshooting but it just won't recognize the plugin — as if it wasn't installed:
Mail::SpamAssassin::Plugin::RelayCountry
CPAN says it is installed and up to date.
The MaxMind GeoIP2 database is installed and working fine with Apache, Nginx, and even WordPress, but I can't get SpamAssasin to use it. Maybe it is the new version of Perl? Or something else.

If you happen to find a way to install the RelayCountry module on AlmaLinux, please feel free to share the installation instructions with us or provide a pull request with an action to install the module within the conversion tool.

I will indeed. Not sure why reinstalling is being difficult on my server. Think I'm missing a crucial piece of experience.

If you happen to find a way to install the RelayCountry module on AlmaLinux, please feel free to share the installation instructions with us or provide a pull request with an action to install the module within the conversion tool.

Ok so I worked for two days trying to figure out how to get RelayCountry to be recognized by SpamAssassin...
Only after I did this ridiculously simple idea did it work:
In order to keep all the updater software (yum, cpan) from trying to install SpamAssasin 4 every time I needed a module for SpamAssassin 3.4 I downloaded the Mail-SpamAssassin-3.4.6.tar.gz tarball directly from Apache, retrieved the RelayCountry.pm file out of it, and placed it here where it instantly began working:
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/
Also, after a few other Perl script problems trying to load using GeoIP2 I figured out that this Perl module location is not apparently being recognized by my Perl scripts, and that may be a big issue with all the Perl modules:
/usr/share/perl5/vendor_perl/
In this case I got it to work by by loading the Perl module directly in the script using this:
use lib '/usr/share/perl5/vendor_perl';
Maybe these are related somehow, and can be of help in your script.

I took the dive and upgraded my production server to AlmaLinux 8.8 and it all worked as expected!
Thank you for all your support in this!
I'm good to 2029!
If you get a chance to create an update script from AL8 to AL9 I'll be all for it. :-)

Hay @gregarios,
I'm delighted to hear that! =)
Thank you for your assistance. We have made significant improvements to the tool during the resolution of this issue.


[BUG] Improve log message when qual tool could not access an eventlog

Describe the bug
If a file , say in hdfs file system is not accessible by the tool, the verbose output shows the following but does not propagate the error message from the exception. This will apply to other exceptions as well since the catch is for Exception
Steps/Code to reproduce bug
Run qual tool on an eventlog that is not accessible/have permissions
spark_rapids qualification --eventlogs= hdfs:/nn:8020/my-app-eventlog

 WARN UnknownAppResult: File: hdfs:/nn:8020/my-app-eventlog, Message: AccessControlException: Got unexpected exception processing file: hdfs://nn:8020/my-app-eventlog

Expected behavior
Add more info to the verbose log line. If this seems excessive we can skip it but I found the o/p useful while debugging and also brings more attention to the warning.
Environment details (please complete the following information)

hdfs file path , qual 24.06.1

Additional context
Add any other context about the problem here.

#1187 and #1235 resolved this issue when tools would not show any output. Now, we show the number of apps that are provided, successfully processed and are top candidates Additionally the status csv was updated to capture the exact cause of error.
Output
Environment: latest dev
Console
    - Application status report: /Users/psarthi/Work/tools-run/qual_20240812213733_3EaE3EdF/rapids_4_spark_qualification_output/rapids_4_spark_qualification_output_status.csv

Qualification tool found no successful applications to process.

Report Summary:
----------------------  -
Total applications      1
Processed applications  0
Top candidates          0
----------------------  -

Status File
File: qual_2024xxx/rapids_4_spark_qualification_output/rapids_4_spark_qualification_output_status.csv
|-------------------------------|---------|-------|-------------------------------------------------------------|
| Event Log                     | Status  | AppID | Description                                                 |
|-------------------------------|---------|-------|-------------------------------------------------------------|
| hdfs:/nn:8020/my-app-eventlog | FAILURE | N/A   | Incomplete HDFS URI, no host: hdfs:/nn:8020/my-app-eventlog |
|-------------------------------|---------|-------|-------------------------------------------------------------|

Could you test on the latest dev branch and let us know if the issue still persists?

@parthosa my changes are against dev. I'm not sure my change corresponds to the status file you mentioned. When we print verbose and we are calling out the exception we should give info on how to triage it by simply including the stack.


Not allowed to load local resource<EMAIL_ADDRESS>

When using this npm plugin, an error occurs when executing ffmpeg.load with the following error message:
Not allowed to load local resource<EMAIL_ADDRESS>how can i fix it?

A similar issue is happening with @ffmpeg/core:

The file is definitely present.

Same error

node#fetch can't read file system path, I wonder why they didn't notice this


Convert to go module and rework protogen and imports

Hello,
My goal here was to convert the sdk to a proper go module. Once I started down this path, I ended up having to fix some things with the way the compiled protobufs were included and references. To summarize, I've done the following:

Modified the Dockerfile so it works on arm64 Mac/Linux (non-breaking change, repo weirdness).
Fixed a typo in gen.sh that caused old mocks were not being removed before generating new ones.
Changed all internal references to use full paths (both protobuf references and stuff inside the examples).
Add a cgo directive to the pem_loader to instruct it to use pkg-config to find openssl (this can be discussed...I don't think it will cause much problem at all on Linux, but is necessary to get it to use homebrew openssl on mac).
Regenerate the protobufs and mocks in protobuf and remove the copy at src/protobuf.
Finally, convert to go module by adding go.mod and go.sum.

I think the biggest 'disruption' here is the recompilation of the protobufs and the dropping of src/protobuf. Keep in mind that the stuff in src/protobuf (and even in protobuf in v0.1.3) did not actually work, as the references needed to be re-written as full paths (the user had to run go generate anyway!).
Now, the SDK can simply be downloaded/imported without calling go generate.
Please let me know if anyone has any comments or changes they would like.
-Bill

Can you confirm that the README.md file instructions for updating the auto-generated files is accurate? (inc. the new sum file)

Can you confirm that the README.md file instructions for updating the auto-generated files is accurate? (inc. the new sum file)

This is correct, but is not only required for developers. I'll do a quick revision to the README that breaks down what is required for usage vs. development.

Ok, so I fixed the linter errors and added README.md updates.


"Create groups for similar attributes" seems always on

I have a SVG with two (ungrouped) paths.
Using "Save as Inkskape SVG" produces:
    <g
       id="g3160">
      <path
         style="fill:none;fill-opacity:0.194118;stroke:#ff0000;stroke-width:0.264583;stroke-linejoin:round;stroke-miterlimit:7.70054;stroke-opacity:1"
         d="M 50,60 100,120 M 50,60 V 200 L 100,120"
         id="path3128" />
      <path
         d="M 130.0238,69.547617 150.43452,220.73808 91.470236,213.93451 177.6488,69.547617 Z"
         id="path3150"
         style="fill:none;fill-opacity:0.194118;stroke:#ff0000;stroke-width:0.264583;stroke-linejoin:round;stroke-miterlimit:7.70054;stroke-opacity:1" />
    </g>

Save as "optimised SVG" (with the "convert CSS attributes to XLM attributes" option checked and the "collapse groups" and "create groups for similar attributes" either unchecked or checked) produces:
 <g fill="none" stroke="#f00" stroke-linejoin="round" stroke-miterlimit="7.7005" stroke-width=".26458">
  <path d="m50 60 50 60m-50-60v140l50-80"/>
  <path d="m130.02 69.548 20.411 151.19-58.964-6.8036 86.179-144.39z"/>
 </g>

What I'd like to export is:
<g>
 <path d="m50 60 50 60m-50-60v140l50-80" fill="none" stroke="#f00" stroke-linejoin="round" stroke-miterlimit="7.7005" stroke-width=".26458"/>
 <path d="m130.02 69.548 20.411 151.19-58.964-6.8036 86.179-144.39z" fill="none" stroke="#f00" stroke-linejoin="round" stroke-miterlimit="7.7005" stroke-width=".26458"/>
</g>

Shouldn't this be the output with "create groups for similar attributes" unchecked?
Thanks

@sunjaxx :
"Create groups" means it creates a group, where no group was before
Example:
<rect fill="red"  width="1" height="2">
<rect fill="red"  width="2" height="1">

will create (if enabled)
<g fill="red">
 <rect width="1" height="2">
 <rect width="2" height="1">
</g>

You might want to read https://github.com/scour-project/scour/wiki/Documentation#--create-groups for a more detailed explanation.
possible Solution/Workaround
You might want to try --disable-style-to-xml (in Inkscape it is imho untick "Convert CSS attributes to XML attributes")

python3 -m scour.scour -i Input.svg -o Output.svg --disable-style-to-xml

Output
Output.svg

<?xml version="1.0" encoding="UTF-8"?>
<svg id="svg5" width="170.54" height="214.59" version="1.1" viewBox="49.868 59.915 127.9 160.95" xmlns="http://www.w3.org/2000/svg">
 <g id="g3160">
  <path id="path3128" d="m50 60 50 60m-50-60v140l50-80" style="fill:none;stroke-linejoin:round;stroke-miterlimit:7.7005;stroke-width:.26458;stroke:#f00"/>
  <path id="path3150" d="m130.02 69.548 20.411 151.19-58.964-6.8036 86.179-144.39z" style="fill:none;stroke-linejoin:round;stroke-miterlimit:7.7005;stroke-width:.26458;stroke:#f00"/>
 </g>
</svg>

I hope the suggested workaround was a viable option.
For Scour I believe this is out-of-scope as it's basically asking to tell scour not to optimize the SVG, when the result reported in the first post is actually a perfectly safe optimization.


Get it together.

Ratio (2)

Seriously? Grow up. You're already supporting a basically pay-to-win server, I doubt you aren't paid.


return HTTP 400 if hook rules were not satisfied

Hi,
we'd like to be able to easily check if a hook was successfully executed. If the hook rules were not satisfied, webhook should return a non-200 HTTP code. I think non-matching hook rules are likely a result of a "Bad Request", so this should be returned instead of 200.

First of all, thanks for using webhook and making a PR.
I agree this would be great addition to webhook.
However, since this PR would change the default behavior of webhook other users might rely on, I would love to see this feature implemented as a per-hook configurable and global setting similar to how response headers are currently implemented. Leaving the 200 Ok to be default global setting.
If you did that, I would be more than glad to merge your contribution!
Thanks :-)

Good point. I'll see if I can come up with a proper solution.

I rebased my changes on master. The HTTP code is now configurable per hook. If not set or set to an invalid HTTP code, the default is 200 OK.

I thought of this exact name as well but found it a little bit too long. But I agree that the longer name is more self-explanatory. Thanks :-)


js  partial syntax highlighting

Describe the issue
js  partial syntax highlighting
js
js  partial syntax highlighting
Are you using highlight or highlightAuto?
no
Sample Code to Reproduce
no error:
  /* svg minify */

  gulp.task('svgMin', function() {
    return gulp.src('source/image/*.svg')
      .pipe(image({svgo: ['--enable', 'cleanupIDs', '--disable', 'convertColors']}))
      .pipe(gulp.dest('source/image'));
  });

error:
  /* svg sprite end inject */

  gulp.task('svgSprite', function() {
    const svgs = gulp.src('source/image/*.svg', {base: 'source/image'})
      .pipe(rename({prefix: 'icon-'}))
      .pipe(svgstore({inlineSvg: true}));

    function fileContents(filePath, file) {
      return file.contents.toString();
    }

    return gulp.src('source/**/*.html')
      .pipe(inject(svgs, {transform: fileContents}))
      .pipe(gulp.dest('source'));
  });

https://drive.google.com/file/d/17vO81oRGZxXrVwKp_Sc0-b4opXsyfUWb/view?usp=sharing
Expected behavior
Problem with any styles
Additional context
problem star with:
    const svgs = gulp.src('source/image/*.svg', {base: 'source/image'})
      .pipe(rename({prefix: 'icon-'}))
      .pipe(svgstore({inlineSvg: true}));

Can you confirm this on the latest master?  I'm seeing no issues.

10.2.0 had a bug that would result in this... so if you're seeing this on StackOverflow or a site using an older version of the library, that's likely what is going on here.  Upgrading to the latest version should resolve.

I'm uploading a file from offic site.
I have little experience. I don’t understand how to compile from sources for github

Maybe we're talking about different things?  Looks fine to me.

@GurovDmitriy Your code is being detected as Bash.  Auto-detect is not 100% accurate.  For 100% accuracy you need to manually tell us what type of code you are highlighting.  You need to manually specify the language in of your code blocks:

...

when i insert
  ...

or
  ...
  - no highlighting. now removed the tips, the bash was highlighted. 1 minute updating the site..

In the first picture, the pipe of different blocks is highlighted, but the other does not

Your Highlight.js build does not include Javscript language support: https://www.typenote.io/js/highlight.js  Use the default build or create a new build that includes JS support.  JS can't be highlighted if you don't build support for it into your library.

I didn’t find it listed for individually assembling. I thought it was turned on by default. is she on the list ...?

I didn’t find it listed for individually assembling. I thought it was turned on by default. is she on the list ...?

Well now - it's missing, that's incredibly annoying. :-)

P.S. If you need a layout, tell me, I'm a beginner web developer. I work for thanks)))

@GurovDmitriy It's back in the list now if you build a new version you should get it.

thank you so much

I set everything up. updated. Working!


Show date and time

Is there any way or option to also show the date in there?

Took a look at firechat-ui.js. There's a prototype function of FirechatUI called formatTime. Overwriting this can provide the format I want.
https://github.com/FirebaseExtended/firechat/blob/715f2cfdd87892eac3b9035692d1dbcf98b38f71/src/js/firechat-ui.js#L1067


[Feature Request] No fiat api support

API URL : https://binance-docs.github.io/apidocs/spot/en/#fiat-endpoints
There is no references of /sapi/v1/fiat/ in the whole project.
I will work on it in a pull request if I find the time.

This has been added in the last release: https://github.com/binance/binance-connector-go/releases/tag/v0.7.0


Add initial code and tests for the SA1414 Analyzer for converting fie…

…lds that are marked internal as internal auto properties.
See #2780. I closed the other pull request for this as I think this would be a great thing to add to help people with fields they mark internal. However this excludes static, and const ones for now as I am not sure if I can actually suggest making those auto properties.

@sharwell uh I think I need help with figuring out the failed tests. I would debug them but I still cannot build due to the same issue I have from the other pull request.

going to debug the if now and find out why the 1 test fails.

@AraHaan Did you update the permissions as described in the issue I linked to?

yes, I did.

oh wow it is detecting as ProtectedOrInternal this does not seem right. I would think in the case of protected internal to be detected as ProtectedAndInternal. @sharwell

LOL seems like a GitHub bug now... why is it always me that finds the bugs. 🤔

@sharwell I think the build bugged out on the powershell script.


Acceptance scenario 080: expect unary constructors

:round_pushpin: Add new acceptance test scenario: expect unary constructors.Error:
× Main thread panicked.
├─▶ at crates/aiken-lang/src/gen_uplc.rs:1852:33
╰─▶ called `Option::unwrap()` on a `None` value

Ready to merge


Add Go CLI to OpenWhisk - Defect Fixes

Defect fixes

@mdeuser could you help review this PR? Once you give me a 👍 I will merge.

@dubeejw - A few questions/comments.

Should annotations be processed similar to the parameters?  I think both can take JSON
Regarding the parseParameters() and parseParametersArray() functions - can a single parseParameters() function handle the parsing such that the command handler is not entirely aware of the parameter value format?
Why is a '+' (plus) character replaced with a ' ' (space) character in the action name, package name, rule name, and trigger name after the name is URL escaped?  I see that the url.QueryUnescape() does the inverse of ur.QueryEscape(), but it converts '+' to ' ' (space).

@mdeuser,
url.QueryEspace replace spaces with plus characters. In comparison, the Python CLI never replaces spaces for the action, package, rule, and trigger names. Replacing the plus characters with spaces in needed in the Go CLI to achieve parity with the Python CLI. Also, if the spaces are not preserved, automated tests fail.
The spaces have to be added back because the the automation expects to see the space when it reads responses from the CLI. This is due to the fact that the Python CLI treats spaces in the same manner.

@dubeejw - Ah.  I think I see the problem.  The resource name (i.e. package name, action name, etc) is actually part of the URL path, not part of the URL query.  The URL escaping process is different depending on whether the string to escape will be part of the URL path or the URL query.  In a URL path, a space is escaped with %20; in a URL query, a space is escaped with a '+'.
http://stackoverflow.com/questions/1634271/url-encoding-the-space-character-or-20
The url.QueryEscape() function should not be used on these name strings when they are part of the URL path and not in a URL query

I opened issues for my comments to be addressed outside of this PR.  PR should work as-is, but issues should be considered.
LGTM

Rebase and merged 20178d6b88c18ad5abbcf373e101b41d04952388


Upgraded electron to v1 and remove crashReporter

Removed crashReporter (we don't want to setup a server here to keep the report)
https://github.com/sindresorhus/generator-electron/issues/19

Nothing mentioned about target: 'electron-renderer' here...


chore: Update Cloud Pak for Data to release 4.6.4

Is your feature request related to a problem? Please describe.
Cloud Pak for Data 4.6.4 was released, so we need to update this repository.
https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=overview-whats-new#whats-new__refresh-4
Describe the solution you'd like
Update the CRs and jobs to reflect the updated installation procedures.
Describe alternatives you've considered
None, the requirement is pretty narrow.
Additional context
CP4D installation procedure shifted towards a procedural approach in release 4.5.x. See #173 for reference and corresponding PR with the changes to understand the nature of that change.

:tada: This issue has been resolved in version 0.20.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


feat: Add support for cloud-connector to read kms encrypted logs in S3 from existing cloudtrail

This change allows using an existing KMS key (that may also be in another account) for cloud connector to read from an existing encrypted cloudtrail log bucket.
Checklist

[ ] If test/fixtures/*/main.tf files are modified. I have updated:

[ ] the snippets in the README.md file under root folder.
[ ] the snippets in the README.md file for the corresponding example.

[X] If examples folder are modified. I have updated:

[X] README.md file with pertinent changes.
[ ] test/fixtures/*/main.tf in case the snippet needs modifications.

[ ] If any architectural change has been made, I have updated the diagrams.

-->

@wideawakening Ive addressed all of your comments

@wideawakening Ive addressed all of your comments

thanks! :)
need to prioritize this task to schedule some time for it and give it a proper bump, bear with us :D
because some org GH-level secrets would be accessible to your user if we make you collaborator seems like it's not gonna be feasible for the moment.
anyhow, we gotta work on some clarifications

what are the required permissions for the existing cloudtrail_kms_arn? this KMS will be used by cloudtrail-s3 in write time, and by cloud-connector on read time
can the kms_key live in any account or it has to be in the same account as where the cloud-trail lives?
think we could benefit from creating a test to validate this usage, at least in the happy path (same account)


New package: Crux v0.1.0

Registering package: Crux
Repository: https://github.com/sisl/Crux.jl
Created by: @ancorso
Version: v0.1.0
Commit: 45a17ee16d947a89d15fa0cfac67bd862d6ae788
Reviewed by: @ancorso
Reference: https://github.com/sisl/Crux.jl/commit/45a17ee16d947a89d15fa0cfac67bd862d6ae788#commitcomment-137497169
Description: Julia library for deep reinforcement learning

I'm going to assume the name similarity to Flux is somewhat intended. So, the name seems fine to me, giving the context of existing packages in the ML space.

Yep, you are correct about the name.

@goerz It looks like everything except the name checks are now passing. I'd like to keep the name as is based on the reasoning you mentioned in your comment. Is it possible to merge?

I would merge it, but I don't have maintainer status on the Registry. You'll have to ask on Slack in the #pgk-registration channel to get the attention of a maintainer.


Also update the width/unicode data when replacing missing glyphs in non-embedded Type1 fonts (issue 18059)

Please note: This causes a little bit of movement in the issue2770 test-case, however this matches the rendering in both Adobe Reader and PDFium.

/botio test

/botio browsertest

@calixteman Gentle review ping?

/botio makeref


AIX builds failing due to /tmp/sh-np.* file leakage...?

Platform:
AIX
Architecture:
ppc
AIX builds are current failing with weird errors:
/tmp//sh-np.MnrMic: ambiguous redirect
Examining machine shows /tmp contains a huge number of these sh-np.* files...

This command seems to be able to delete them:
find . -ls | awk '{print $11}' | grep sh-np | xargs rm

This might help from our experience cleaning up slaves
https://github.com/eclipse/openj9/blob/master/buildenv/jenkins/jobs/infrastructure/Cleanup-Nodes.groovy

pruned /tmp on AIX machine test-osuosl-ppc64-aix-71-1
attempting rebuild on it here: https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-aix-ppc64-openj9/341/console

Also clearing up the second build- AIX box. Have put in some logic into the processCheck job that is run periodically to seek out and delete any such files going forward.


Add poster info to newznab module

i think we need to check for a poster first. I don't think every indexer returns one

I already tried with usenet-crawler, spotweb and 6box. All posts I tried include the poster info.

Ok, its a easy fix if there is a provider that dont supplies a poster


feat: Remove tmpdir from updater

Proposed Change

Remove tmpdir command line parameter
Refactor to have the installDir be the base of all paths, as opposed to the mix of relative paths and paths based on tmpDir.

Checklist

[ ] Changes are tested
[ ] CI has passed

@StefanKurek Ripped it out now, I think it should be good.


PP-539 Allowing self links in payment search

PP-539 Allowing self link for payments from search
* Including self link in payment search response. Only self link is
available in the '_links' object of the payment search response.
* Introducing separate objects to represent search results, links that
only have safe links and links that have both safe links and next_url
links.
* Every response other than payment search and cancel will have the
next_url in the links object
* Adding tests to assert we are returning everything we need to return
* Fixing typo in documentation

rebuild

I have spoke with with @imranbohoran about the solution and the problem stems from Swagger and Jackson that require the class structure to be present.
@imranbohoran also agrees that the Payment structure requires some refactoring and we are happy for that work to be done as part of the Payment Events PR.


Add a HDFS Non-HA DC/OS Service

As explained in this document HDFS - Minimal installation, the cost to run the HDFS service is prohibitive when you just want to provide a small HDFS cluster to few datascientists or just to test a simple program using HDFS. I needed a minimal installation without all the HA components (journal node, zkfc's and second namenode) because my cluster is small and I have other frameworks running on it.
I saw in this link that this contribution would be welcome so ... I wrote a HDFS Service based on the existing HDFS HA that requires less resources than this latter. I would like to contribute to the DC/OS Universe with this light version of HDFS.
All the files for this package (scheduler, bootstrap ...) are on S3. There's also a hadoop file which is the 2.8.1 version without all the documentation.
Here is the dcos-commons fork I did. I tried to write a README.md the most complete I could but maybe some informations are missing (that's my first PR to mesosphere universe). So don't hesitate to tell me if that's the case !

Any update?

@triclambert can you please take a look?


fix: proteger l'affichage d'une demande

close jira 148

:tada: This PR is included in version 0.10.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


update property name from peek to peekProperties in README.md

@roshal Thanks a ton for you contribution . Did you test your changes locally ? if not please

@Viijay-Kr aprove please


Logs: show cucumber (and other?) error messages

Not sure about the cucumber error stacktrace, but can we save a bit of pain by adding cucumber's own [output] ~scenario failure messages~ to the report. We're already doing a lot of work with the scenario object. I bet the message is in there somewhere...
[This would be useful to add to the reports so the user and we can see it later without being on GitHub.]

I think the log would be more appropriate.

Two maybe helpful or unhelpful links:

https://github.com/cucumber/cucumber-js/blob/main/docs/profiles.md
https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#Formats

Another one! https://github.com/cucumber/cucumber-js/blob/main/docs/formatters.md#message

FIxed by #544


Add response or response headers to RequestFailedException

You get something like this in Message. Would be nice to have the information as separate properties (Content and Headers) for easy parsing/identification of the issue.
Service request failed.
Status: 501 (Not Implemented)

Content:
123

Headers:
X-Powered-By: REDACTED
Vary: REDACTED
Access-Control-Allow-Credentials: REDACTED
Access-Control-Expose-Headers: REDACTED
ETag: W/"3-QL0AFWMIX8NRZTKeof9cXsvbvu8"
Date: Wed, 15 Apr 2020 21:42:30 GMT
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Content-Length: 3

Another example for 429 is the retry-after header

+1 it's not clear what the expected template for responses is, and if we can count on it always being in the format above. The SDK should be exposing these fields to callers to prevent ugly regex hacks that are prone to breaking.

+1 it's not clear what the expected template for responses is, and if we can count on it always being in the format above. The SDK should be exposing these fields to callers to prevent ugly regex hacks that are prone to breaking.

It's worth asking why this information is needed, though. We recommend against using it for branching in code because the data is guaranteed to be compatible.

429 was the big one, as services often include a retry-after header that must be parsed and presently is inaccessible.

429 was the big one, as services often include a retry-after header that must be parsed and presently is inaccessible.

This should already be handled by the RetryPolicy.


chore: added net6.0/IdentitySample and sample cleanup

:tada: This PR is included in version 6.7.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


NGINX.NGINXPLUS/stable/2022-08-01 - define SKU names as Enum

Can the name of nginx deployments SKU defined as Enum values, so it can be generated in SDK and easy to use?
related swagger: https://github.com/Azure/azure-rest-api-specs/blob/1d9c9c68844d2b5883621069d5bf0c958841782f/specification/nginx/resource-manager/NGINX.NGINXPLUS/stable/2022-08-01/swagger.json#L1059

ping @ruowan @JackTn any idea of a timeline for this one?


Updating Dev portal and Admin REST API postman collections

Purpose
Update the Postman collection of

QSG REST APIs
Dev portal REST APIs
Admin REST APIs

Tracked in https://github.com/wso2/api-manager/issues/1076

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@tgtshanika, please send these changes to the master branch as well


[Linear] Support Milestones in Create and View issues

Extension
https://www.raycast.com/linear/linear
Description
Linear recently added support for Milestones to further organize and group issues. I use this feature in a few projects. However, today I realized that you can't assign an issue a milestone in the Raycast Linear extension, nor view what milestone an issue belongs to when viewing an issue.
Who will benefit from this feature?
All Linear users will benefit that have milestones enabled and setup for a Linear project.
Anything else?
No response

Thanks for the feature request. Would you like to work on it? I can provide some guidance on how to contribute to this extension so that you can open a pull request. You can contact me here or in the Slack community if you need further help.


Unable to use keras.layers.Normalization on multiple GPUs

Enviroment
I am using tensorflow 2.7.0 with python 3.8.10, linux cluster with multiple gpus.
Context
I am trying to use a normalization layer as demonstrated in load model code below:
The code works normally as expected when removing the tf.keras.layers.Normalization layer or when I use 1 or 2 GPUs, although when i try to use 4 GPUs the error below occurs. Therefore i discard any problem related to input pipeline, tfrecord parsing, incorrect label format, etc... The error occurs at the end of the epoch.
I created this gist in order to reproduce the error but everything ran normally before I was able to test other tensorflow versions. I even installed the exact same python version (3.8.10) to verify whether it would be the case but it worked with no errors.
Back in the original enviroment I've tested limitting the number of GPU devices to 1 or 2 and the code ran normally. Otherwise when i try to run it in 4 GPUs the error occurs.
Code:
def load_and_configure_model(optimizer, loss, metrics, path):
  model = ResNet50V2(include_top=True, weights='imagenet')
  transfer_layer = model.get_layer('avg_pool')
  resnet_submodel = Model(inputs=model.input,outputs=transfer_layer.output)
  model_config = resnet_submodel.get_config()

  submodel = model_config['layers']
  submodel.remove(submodel[0]) # Remove the previous input layer

  input_layer = keras.Input(shape=(224, 224, 3), dtype='float32',name="input") # Create a new input layer
  normalization = tf.keras.layers.Normalization(mean=[118.662, 119.194, 96.877], variance=[2769.232, 2633.742, 2702.492], axis=-1, dtype='float32')(input_layer)
  rescaling = tf.keras.layers.experimental.preprocessing.Rescaling(1./127.5, offset=-1, dtype='float32')(normalization)

  new_model = Model(inputs=input_layer,outputs=rescaling) # Declare pre-processing model to be merged with the ResNet model.
  new_model_cfg = new_model.get_config()

  new_model_cfg['layers'].extend(submodel) # Merge two models.

  # Replace the previous input layer with the output from the preprocessing model
  # (Connect the preprocessing model to the resnet)
  output_name = new_model_cfg['layers'][2]['name'] # Get the output layer name (rescaling).

  new_model_cfg['layers'][3]['inbound_nodes'] = [[[output_name, 0, 0, {}]]] # Replace last inbound node name with the preprocessing model layer name.

  new_model = new_model.__class__.from_config(new_model_cfg, custom_objects={})  # change custom objects if necessary

  # Set back pre-trained weights on new model
  weights = [layer.get_weights() for layer in resnet_submodel.layers[1:]] # For each layer (after the input_layer) in the original resnet50:
  for layer, weight in zip(new_model.layers[3:], weights): # Set imagenet weights on each new model layer.
      layer.set_weights(weight)

  for layer in new_model.layers[:]:
    layer.trainable = False
  for layer in new_model.layers[:]:
    trainable = True
    layer.trainable = trainable # Train everything.

  transfer_layer = new_model.get_layer('avg_pool')
  species = Dense(1000, activation='softmax', dtype='float32',name='species')(transfer_layer.output) # Specify dtype for handling mixed precision specifications.

  model = keras.Model(
      inputs=[new_model.inputs],
      outputs=[species],
  )
  if not path == None :
    model.load_weights(path)
  model.compile(optimizer=optimizer, loss=loss, metrics=metrics)
  return model

I'm liking the error that you are running into here. There are a few things that you can try as mentioned here. Try all the mentioned solutions and let me know if you are still running into this error. Thanks!


Not working on MacOS Monterey

Hi,
Thanks for this amazing app, But unfortunately this is not working on macOS Monterey, I have enabled  Keycastr in Accessibility settings but I don't see keystrokes, nor start casting does anything.
System Info
MacBook Air (M1, 2020)
version 12.1 (21C52) MacOS Monterey
Let me know if you need any further info.
Thanks

There are two likely causes for the app seeming not to work: not receiving keystroke events from macOS due to security settings, or the window being offscreen. Sometimes switching from the Default to the Svelte visualizer can help you figure out which problem it is.
There isn't a compatibility problem with Monterey specifically, but the accessibility settings can be deceiving.
To troubleshoot, I would recommend that you:

quit the app
remove KeyCastr from any of the Privacy areas in the Security & Privacy preferences panel (typically the Accessibility or Input Monitoring areas)
start the app
when you see the "Keystroke Receiving" dialog from macOS, click "Open System Preferences" or open the Preferences app yourself
under Input Monitoring, enable KeyCastr by unlocking your preferences and clicking the box next to KeyCastr which should have been automatically added to the list
when macOS prompts you, restart KeyCastr. Or you can do it manually.

I hope it helps. Some updates to our README are long overdue.

Thanks for the prompt reply, Enabling Input monitoring fixed the issue.
We can close this issue.

The above steps still doesn't work for me. When I tried, I can see the Cmd/Alt/Option/Shift key blink on Svelte UI. But keystrokes doesn't appear.


fix:improved the UI of login with google button

ISSUE #155

Describe the changes you've made
Improved the UI for login with google button

Describe if there is any unusual behavior (Any Warning) of your code(Write NA if there isn't)
NA

Additional context (OPTIONAL)

Test plan (OPTIONAL)
A good test plan should give instructions that someone else can easily follow.

Checklist

[x] My code follows the code style of this project.
[x] The title of my pull request is a short description of the requested changes.

Preview

I have made the requested changes @Abhishek-kumar09 , Please review

I am holding it for time being, have to confirm with designer if it is OK

Thanks for your btw!

Thanks for your PR btw!


Is React specific SDK available?

Hi,
Can I use this JS SDK in React application or any alternate SDKs available ?

There is a tracer for the web (see here) that you can use. I'm not sure what you are refering to for a "react specific" sdk

@vmarchaud  Thanks I will check that. Seems the npm modules are bit specific for node environment, So I asked about the web front-end support.

I will close the issue, feel free to re-open if you have other questions


move transactions table

This PR moves the transactions table to the bridge page, removes the transactions page, makes some small changes to the table style (add background colour, add copy if there are no transactions, move subscan link to the bottom of the page), and reduced the number of items per page from 20 to 5.
It also removes a redundant component from the vault page, and redirects the manual issue execution links to the bridge page.
This is a small change ahead of the de-fi hub launch so no other changes to the table needed at this point (i.e. we're not updating it to use the new UI components).

Yes that makes sense—this hook (and the vaults page in general) could do
with a bit of attention, so let's pick that up once we've got the de-fi hub
out of the way. @Peter Slaný @.***> could you add a ticket to
the UI repo, and add it to the tech debt milestone?
On Fri, 9 Jun 2023 at 15:05, Peter Slaný @.***> wrote:

@.**** approved this pull request.
Changes look good :)
In src/utils/hooks/api/vaults/use-get-vault-transactions.tsx
https://github.com/interlay/interbtc-ui/pull/1278#discussion_r1224354878
:

@@ -16,6 +15,19 @@ import { getCurrencyEqualityCondition } from '@/utils/helpers/currencies';

import { useGetCurrencies } from '../use-get-currencies';
+type TransactionStatus = 'pending' | 'cancelled' | 'completed' | 'confirmed' | 'received' | 'retried';
This is out of scope of this PR, but maybe it would make sense to merge
this type with the ExtrinsicStatus type we use in the transactions hook,
what do you think? @danielsimao https://github.com/danielsimao
—
Reply to this email directly, view it on GitHub
https://github.com/interlay/interbtc-ui/pull/1278#pullrequestreview-1472184031,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTBEQXDCHBLTHBBAX2JKF3XKMUTVANCNFSM6AAAAAAZAO2PIU
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Interlay
Ltd is a private limited company registered in England and Wales
with company number 12286246. This email and any attachments may contain
confidential and/or privileged information.  If you have received this
email in error, please notify us and delete the email immediately.