All AppInstallcerCLIE2ETests fail with the same message: "Access to the registry key ... AppModelUnlock ... is denied."

Hello friends,
I am trying to get my local development environment set up, and while the code compiles and runs, all 67 of the end-to-end tests fail with the same message, similar to this one:
Test Name:	DisplayFeatures
Test FullName:	AppInstallerCLIE2ETests.AppInstallerCLIE2ETests.FeaturesCommand.DisplayFeatures
Test Source:	C:\Users\kindr\Source\Repos\winget-cli\src\AppInstallerCLIE2ETests\FeaturesCommand.cs : line 27
Test Outcome:	Failed
Test Duration:	0:00:00

Test Name:	DisplayFeatures
Test Outcome:	Failed
Result Message:	OneTimeSetUp: System.UnauthorizedAccessException : Access to the registry key 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock' is denied.

The line that's triggering the exception is: src\AppInstallerCLIE2ETests\SetUpFixture.cs(130).
I'm relatively new to Windows development, especially with Visual Studios, so I may be missing something obvious. I have followed all the steps in the README.md under "Building the client>Prerequisites" including enabling Developer Mode. I can also confirm that the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock exists (though I dared not tamper with the permissions.) I'm running the latest build of Windows in the stable channel:
Edition	Windows 10 Pro
Version	20H2
Installed on	2021-02-25
OS build	19042.867
Experience	Windows Feature Experience Pack 120.2212.551.0

Any help is appreciated!

These tests are built to run on the build servers, and so expect to set up everything themselves.  This requires Admin permissions to do, and in order to run them locally you will need to run VS as Admin.  There is a README in the root of the E2E tests directory as well that should help: https://github.com/microsoft/winget-cli/tree/master/src/AppInstallerCLIE2ETests
Alternately, you can be lazy like myself and not bother running them until they fail in the build.  They take much longer to run than the unit tests and often have build server specific issues anyway, as well as the difficult setup, so I have optimized for that path.  I may be a special case though, being so entrenched in the project.  I certainly understand the desire to ensure that things are running smoothly before you start changes on an unfamiliar project.

Thanks for the explaination and pointing me to the readme. When submitting a PR with a new feature, would you expect a new e2e tests or just new unit tests?

It depends on the feature, but for zip package handling I would expect new E2E test(s).  I do prefer as much testing as possible be done in the unit tests, but having some smoke testing in the E2E is good too.  I think it is especially needed to ensure that things are working correctly with regards to where files are placed and registry updates, because of how being inside a package works.


Anisha/backend flow

Description

Added transcript hover
Changed "Transcript" to "Current flow" for better clarity and updated any text that previously used transcript
Edited landing title to "Transcript to chatbot, made simple" bc lesbreal, it aint a couple of clicks XD

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

Tested transcript hover with a couple sessions

Checklist:

[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes

Yeah, just some leftover edits in this branch that I didn't push in time for the previous merge!


[FP16]For OPTForCausalLM train on stock pytorch, aten::eq cost time on pvc-1100 worse than A100 * ratio

🐛 Describe the bug
For more details, please refer to https://jira.devtools.intel.com/browse/PYTORCHDGQ-5161?filter=-2.
Versions
pytorch commit:03480213dea1f60f6d12e7348904d2f3ef7314d0
torch-xpu-ops commit:718bc42c667539977e5eadb11ea4dec602544bf2
driver : hotfix_agama-ci-devel-881.19
pti : l_intel-pti-dev_p_<IP_ADDRESS>_offline.sh
basekit : l_BaseKit_p_2<IP_ADDRESS>_offline.sh

Please refer https://github.com/intel/torch-xpu-ops/issues/884


Broken URL for 'Differences between Windows PowerShell 5.1 and PowerShell 7.x'

Under 'What if a module doesn't work with machine configuration?' the URL for 'Differences between Windows PowerShell 5.1 and PowerShell 7.x' is updated but still broken.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0adb1940-7b77-ac38-1c50-bcc048aeac25
Version Independent ID: 15f7d100-0647-91be-f725-1e9992681dac
Content: Azure Automation State Configuration to machine configuration migration planning
Content Source: articles/governance/machine-configuration/migrate-from-azure-automation.md
Service: machine-configuration
GitHub Login: @michaeltlombardi
Microsoft Alias: mlombardi

@Borgquite Thanks for your comment! We will review and provide an update as appropriate.

@Borgquite It's fixed now. Thanks again!


Too many UI Automation + keyboard tab navigation issues

Thanks for your feedback. We're having trouble understanding what you're requesting and need additional information to investigate this item. Please re-open when you've filled out the issue template to the best of your ability.


"extra.symfony.require" not enforced anymore

Hi!
I'm currently on SF 6.4.*, but noticed a lot of indirect SF 7 dependencies were installed. I have the following in my composer.json:
    "extra": {
        "symfony": {
            "allow-contrib": true,
            "require": "6.4.*",
            "endpoint": [
                "https://raw.githubusercontent.com/schranz-php-recipes/symfony-recipes-php/flex/main/index.json",
                "https://raw.githubusercontent.com/schranz-php-recipes/symfony-recipes-php-contrib/flex/main/index.json",
                "flex://defaults"
            ]
        }
    },

When I go back to the default settings, without custom endpoints:
    "extra": {
        "symfony": {
            "allow-contrib": true,
            "require": "6.4.*"
        }
    },

And I run php composer update "symfony/*", I get:
Loading composer repositories with package information
Restricting packages listed in "symfony/symfony" to "6.4.*"
Updating dependencies
Lock file operations: 0 installs, 20 updates, 0 removals
  - Downgrading symfony/cache (v7.0.0 => v6.4.0)
  - Downgrading symfony/clock (v7.0.0 => v6.4.0)
  - Downgrading symfony/doctrine-bridge (v7.0.0 => v6.4.0)
  - Downgrading symfony/error-handler (v7.0.0 => v6.4.0)
  - Downgrading symfony/filesystem (v7.0.0 => v6.4.0)
  - Downgrading symfony/finder (v7.0.0 => v6.4.0)
  - Downgrading symfony/intl (v7.0.0 => v6.4.0)
  - Downgrading symfony/messenger (v7.0.1 => v6.4.0)
  - Downgrading symfony/mime (v7.0.0 => v6.4.0)
  - Downgrading symfony/monolog-bridge (v7.0.0 => v6.4.0)
  - Downgrading symfony/options-resolver (v7.0.0 => v6.4.0)
  - Downgrading symfony/password-hasher (v7.0.0 => v6.4.0)
  - Downgrading symfony/psr-http-message-bridge (v7.0.0 => v6.4.0)
  - Downgrading symfony/security-core (v7.0.1 => v6.4.0)
  - Downgrading symfony/security-csrf (v7.0.1 => v6.4.0)
  - Downgrading symfony/security-http (v7.0.1 => v6.4.0)
  - Downgrading symfony/stopwatch (v7.0.0 => v6.4.0)
  - Downgrading symfony/string (v7.0.0 => v6.4.0)
  - Downgrading symfony/var-dumper (v7.0.0 => v6.4.0)
  - Downgrading symfony/var-exporter (v7.0.1 => v6.4.1)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 20 updates, 0 removals
  - Downgrading symfony/string (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/var-dumper (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/error-handler (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/doctrine-bridge (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/var-exporter (v7.0.1 => v6.4.1): Extracting archive
  - Downgrading symfony/filesystem (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/finder (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/cache (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/stopwatch (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/password-hasher (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/security-core (v7.0.1 => v6.4.0): Extracting archive
  - Downgrading symfony/security-http (v7.0.1 => v6.4.0): Extracting archive
  - Downgrading symfony/security-csrf (v7.0.1 => v6.4.0): Extracting archive
  - Downgrading symfony/clock (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/options-resolver (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/psr-http-message-bridge (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/messenger (v7.0.1 => v6.4.0): Extracting archive
  - Downgrading symfony/mime (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/monolog-bridge (v7.0.0 => v6.4.0): Extracting archive
  - Downgrading symfony/intl (v7.0.0 => v6.4.0): Extracting archive
Package php-http/message-factory is abandoned, you should avoid using it. Use psr/http-factory instead.
Generating autoload files
168 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
infection/extension-installer: No extensions found
phpstan/extension-installer: Extensions installed

What about running composer global require symfony/thanks && composer thanks now?
This will spread some 💖  by sending a ★  to the GitHub repositories of your fellow package maintainers.

Run composer recipes at any time to see the status of your Symfony recipes.

Executing script cache:clear [OK]
Executing script assets:install public [OK]

No security vulnerability advisories found.

When I add back the custom endspoints as stated in this repositories readme:
    "extra": {
        "symfony": {
            "allow-contrib": true,
            "require": "6.4.*",
            "endpoint": [
                "https://raw.githubusercontent.com/schranz-php-recipes/symfony-recipes-php/flex/main/index.json",
                "https://raw.githubusercontent.com/schranz-php-recipes/symfony-recipes-php-contrib/flex/main/index.json",
                "flex://defaults"
            ]
        }
    },

And I run php composer update "symfony/*", indirect dependencies are again updated to SF7:
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 20 updates, 0 removals
  - Upgrading symfony/cache (v6.4.0 => v7.0.0)
  - Upgrading symfony/clock (v6.4.0 => v7.0.0)
  - Upgrading symfony/doctrine-bridge (v6.4.0 => v7.0.0)
  - Upgrading symfony/error-handler (v6.4.0 => v7.0.0)
  - Upgrading symfony/filesystem (v6.4.0 => v7.0.0)
  - Upgrading symfony/finder (v6.4.0 => v7.0.0)
  - Upgrading symfony/intl (v6.4.0 => v7.0.0)
  - Upgrading symfony/messenger (v6.4.0 => v7.0.1)
  - Upgrading symfony/mime (v6.4.0 => v7.0.0)
  - Upgrading symfony/monolog-bridge (v6.4.0 => v7.0.0)
  - Upgrading symfony/options-resolver (v6.4.0 => v7.0.0)
  - Upgrading symfony/password-hasher (v6.4.0 => v7.0.0)
  - Upgrading symfony/psr-http-message-bridge (v6.4.0 => v7.0.0)
  - Upgrading symfony/security-core (v6.4.0 => v7.0.1)
  - Upgrading symfony/security-csrf (v6.4.0 => v7.0.1)
  - Upgrading symfony/security-http (v6.4.0 => v7.0.1)
  - Upgrading symfony/stopwatch (v6.4.0 => v7.0.0)
  - Upgrading symfony/string (v6.4.0 => v7.0.0)
  - Upgrading symfony/var-dumper (v6.4.0 => v7.0.0)
  - Upgrading symfony/var-exporter (v6.4.1 => v7.0.1)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 20 updates, 0 removals
  - Upgrading symfony/string (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/var-dumper (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/error-handler (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/doctrine-bridge (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/var-exporter (v6.4.1 => v7.0.1): Extracting archive
  - Upgrading symfony/filesystem (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/finder (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/cache (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/stopwatch (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/password-hasher (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/security-core (v6.4.0 => v7.0.1): Extracting archive
  - Upgrading symfony/security-http (v6.4.0 => v7.0.1): Extracting archive
  - Upgrading symfony/security-csrf (v6.4.0 => v7.0.1): Extracting archive
  - Upgrading symfony/clock (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/options-resolver (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/psr-http-message-bridge (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/messenger (v6.4.0 => v7.0.1): Extracting archive
  - Upgrading symfony/mime (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/monolog-bridge (v6.4.0 => v7.0.0): Extracting archive
  - Upgrading symfony/intl (v6.4.0 => v7.0.0): Extracting archive
Package php-http/message-factory is abandoned, you should avoid using it. Use psr/http-factory instead.
Generating autoload files
168 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
infection/extension-installer: No extensions found
phpstan/extension-installer: Extensions installed

What about running composer global require symfony/thanks && composer thanks now?
This will spread some 💖  by sending a ★  to the GitHub repositories of your fellow package maintainers.

Run composer recipes at any time to see the status of your Symfony recipes.

Executing script cache:clear [OK]
Executing script assets:install public [OK]

No security vulnerability advisories found.

When default and/or the contrib endpoints are specified, the dependencies are also downgraded to 6.4.*.
    "extra": {
        "symfony": {
            "allow-contrib": true,
            "require": "6.4.*",
            "endpoint": [
                "https://raw.githubusercontent.com/schranz-php-recipes/symfony-recipes-php-contrib/flex/main/index.json",
                "flex://defaults"
            ]
        }
    },

The recipe repositories do not provide any logic. The logic would be in symfony/flex project if false version are installed.
@nicolas-grekas do you have an idea why this appears?
@darthf1 does it only appear when custom recipes endpoints are configured?

The recipe repositories do not provide any logic. The logic would be in symfony/flex project if false version are installed.

Ah ok, makes absolute sense!

@darthf1 does it only appear when custom recipes endpoints are configured?

Yes. I updated the first post with more examples. Without custom recipes I'm staying on 6.4.x. With the non-contrib custom recipe dependencies are updated to 7.0.x

Does a second run of composer update changes anything or removing composer.lock, symfony.lock and vendor? Also make sure that you use the latest version of symfony/flex, maybe its an older bug which was already fixed.

Does a second run of composer update changes anything

No (I ran composer update "symfony/*" multiple times)

or removing composer.lock, symfony.lock and vendor?

No (I removed the files and folder you mentioned, and ran composer install)

Also make sure that you use the latest version of symfony/flex, maybe its an older bug which was already fixed.

Im on 2.4.2 which is the latest.

Flex relies on the versions.slits entries to decide which packages the restriction applies to. Since 7.0 is not in this repo, flex doesn't enforce the restriction.
This might be something to improve either on flex' side (it could account for the data in the stack of endpoints) or in this repo. I don't remember the exact rules that flex uses for merging endpoints so maybe there's already a way to rely on the default endpoint (by not listing this entry here?). The for-sure fix is to sync the entries with upstream (but it'd be nice to find a way that doesn't require updating this repo everytime a new Symfony release is made).
Let me know your findings.


Interpolation Search | C++

Title - Interpolation Search
what will change - Adding code for implementation of Interpolation search.
Type of Issue -
Please add/delete options that are not relevant.

[x] Adding New Code
[ ] Improving Code
[ ] Improving Documentation
[ ] Bug Fix

Programming Language
Please add/delete options that are not relevant.

[ ] Python
[x] C++
[ ] Java
[ ] C
[ ] Go
[ ] Other language

Self Check

Ask for issue assignment before making Pull Request.
Add your file in the proper folder
Clean Code and Documentation for better readability
Add Title and Description of the program in the file

:star2: Star it :fork_and_knife:Fork it :handshake: Contribute to it!
Happy Coding,

I want to add code for interpolation search algorithm. Can you please assign it to me?

Can i contribute-


Adding Travis-Ci Support For Arm64

Travis-CI has added support for ARM64. Added ARM64 job in Travis-CI.

Coverage remained the same at 76.24% when pulling 011e9f98c04918d24f752c88b6baeedf57856e84 on ossdev07:travis_ci_aarch64 into ac784a219688cb9cdd6da2af374640d2e5953259 on holoviz:master.

Only one Travis-CI job is failing and that is because it failed to download python3.7 from Travis archives. If we will re-run the railing test, it will pass.

@ossdev07 I have the same requests of you here as in https://github.com/pyviz-dev/pyct/pull/78: please explain why this is valuable and worth the increased build times.
The ARM64 jobs for param are e.g. 2.5 mins or 7.5 mins vs just 30 seconds for AMD64 jobs. Granted, there isn't a lot of activity right now, so it's not the end of the world. But note that on some holoviz projects it might be prohibitive. (By the way, what about trying ARM64 for datashader - that might be more interesting :P )

Also, param has a matrix of jobs; I'm sure not all jobs need an ARM64 version - maybe just pick one?

I've restarted that job.
I agree; I think we'd only want ARM tests to be enabled for one Python version (3.7, presumably), and we could add more in the unlikely case that an actual user files a bug for ARM64 on some other version of Python.

ping @ceball, this PR will increase the scope of testing by adding Travis-CI job for ARM64 architecture. As per your suggestion, I have modified the PR to enable ARM tests on only one Python version(3.7).

@ossdev07 Thanks for fixing - I think we can merge this now.
The new ARM64 job takes about 120 seconds, while the rest take about 30. If that becomes a problem in the future, we could change the ARM64 job to just be a daily cron job (param's unlikely to grow some new functionality that would need to be tested on ARM64 before being merged...).

Thanks!


Add step parameter to #list_item

Would be useful to add a step parameter to the list_item method for ordered lists. That would allow users to do the following:
def list_item text, list_type, step = nil
  case list_type
  when :ordered
    "
#{text}
"
  when :unordered
    "
#{text}
"
  end
end

Then one could parse the first data-step in the contents passed to the list method to ensure that lists start at the given index. Or better yet, the gem could pass a new start parameter to the list method.
Either way, ordered lists continuing from an arbitrary index would then be possible, if they aren't already:
def list contents, list_type, start = 1
  case list_type
  when :ordered
    "
#{contents}
"
  when :unordered
    # ...
  end
end

I've proposed PR #741 to support broken-up ordered lists without resetting the numbering.


Add CompressionLevel.SmallestSize and actual Gzip/Brotli compression levels

Since .Net 6 there has been an additional option: CompressionLevel.SmallestSize. And since .Net 7 CompressionLevel.Optimal was also changed for brotli  to not mean smallest size anymore but rather a balanced value.
This change is also documents what these abstracted CompressionLevel values means for underlying Gzip and Brotli compression libraries, given those are enabled by default. In my case I came to this page wanting to get an Idea of balancing CPU use vs size of the response, as there are plenty of pages online benchmarking zlib vs brotli at all possible levels and cpu types.

Internal previews

📄 File
🔗 Preview link

aspnetcore/performance/response-compression.md
Response compression in ASP.NET Core

Note: The mapping of compression levels probably should be on stream classes and linked to from here to explain what the CompressionLevel values are translated to.

https://learn.microsoft.com/en-us/dotnet/api/system.io.compression.brotlistream.-ctor?view=net-8.0#system-io-compression-brotlistream-ctor
https://learn.microsoft.com/en-us/dotnet/api/system.io.compression.gzipstream.-ctor?view=net-8.0#system-io-compression-gzipstream-ctor

@andrerom ... Thanks for submitting this. For the future per our contribution guidelines, we require an issue to be opened before a significant change to the docs is made on a PR.


No error when Gift Card source Payment Method is deleted

If the source Payment Method of a Gift Card is deleted, we get the following error:
Cannot read property 'id' of null

Likely after trying to fetch it with:
const sourcePaymentMethod = await models.PaymentMethod.findByPk(paymentMethod.SourcePaymentMethodId);

We should make sure to better handle that case.

Closing as duplicate of https://github.com/opencollective/opencollective/issues/3483


Add mock for $cordovaActionSheet

Hi @chadcampbell
Here is the mock for $cordovaActionSheet service.
Can I get feedback to know if I make something wrong with my mocks  (see also  #7 ) ?

@eweap Thank you for the contribution!


Added code for BST Implementation

Please accept my PR

Hey @shaneahmad !
Great work.
You can follow me and star this repository for further updates.
Have fun 🙌🏼


Remove/replace MySQL-python library by something else that supports python3

Impact of the new feature
WMCore in general (and perhaps other services depending on it)
Is your feature request related to a problem? Please describe.
Yes, this package MySQL-python does not support python3. Actually, its last commit dates back from 2014...
Describe the solution you'd like
Either completely deprecate this package from the COMP cmsdist repo:
https://github.com/cms-sw/cmsdist/blob/comp_gcc630/py2-mysqldb.spec
and update all the other specs depending on it. Or if this functionality is needed in WMCore (or other DMWM services), we need to find a library that supports python3 (and probably works still in python2), like PyMySQL:
https://pypi.org/project/PyMySQL/
Describe alternatives you've considered
There is no alternative.
Additional context
Issue initially created here: https://github.com/cms-sw/cmsdist/issues/6025

From a quick look in WMCore, it seems this is the only place still depending on this library:
https://github.com/dmwm/WMCore/blob/master/src/python/WMQuality/TestInit.py#L23
so it can be easily deprecated in WMCore.

Currently, MySQL-python source code is hosted at https://github.com/farcepest/MySQLdb1 . The python package name is MySQLdb. In addition to the the occurrence that you found, where we can drop the dependency by using a more generic exception, there are some occurrences of MySQLdb in src/python/WMCore/REST/Server.py, such as

L1250
L1519
L1535
L1825

It does not seem that this dependency is playing a big role. It seems to be used just to set some metadata in the form of some class members, or to select some errorcode depending on the DB backend. We can either make some changes to state clearly that we are not using MySQL-python / MySQLdb or we can leave this module untouched and avoid to insert any bug and simply drop the dependency.

Hmm, from the package dependencies:
https://github.com/dmwm/WMCore/blob/master/setup_dependencies.py#L8
it looks like such change in the REST module would affect:

reqmgr2
reqmgr2ms
workqueue
crabcache
crabserver
reqmon

I though DBS was also relying on it... So my concern is with CRABServer (and maybe CRABCache). In case you come up with a PR to remove this functionality from WMCore, we should ping Stefano to test things before we merge anything.

That is why I would actually only drop the dependency for now and leave all the code cleaning for the future.
I assume that "discarding the dependency" for us only means

changing https://github.com/cms-sw/cmsdist/blob/42f35692d6547ac859e017a05ebc221e6f611e77/wmagent.spec#L9
changing https://github.com/dmwm/WMCore/blob/4f58231e34ee256deb1a93d8ad6bf9af6cece61a/requirements.wmagent.txt#L4
changing https://github.com/dmwm/WMCore/blob/4f58231e34ee256deb1a93d8ad6bf9af6cece61a/requirements.txt#L5

Am I correct? Do you see any problem in just changing our dependencies and trying to run our agents without that dependency before contacting others?

I was going to remove py2-mysqldb from the Requires list for the t0.spec and wmagent.spec, but I think it's too risky to make this change just before I'm gone. I will keep this issue open and resume working on it in August.

@mapellidario I guess this issue is still opened because there was this pending item to remove the dependency from the spec files in cmsdist, right?

Yes, you are correct.

Sigh... I forgot testing it this week when a new WMAgent production release was made. Will try again in the coming days. Thanks Dario.

We continued our investigations to understand if a repository uses MySQL-python.
Purpose:

drop py2-mysqldb from the list of dependencies in WMCore packages spec files and WMCore clients spec files, such as crabserver.spec

References:

py2-mysqldb.spec - v1.2.4b4
pypi MySQL-python
github MySQLdb1 - v1.2.4b4

How can this package be imported
import MySQLdb
# or also
import _mysql_exceptions.py

WMCore does not contain such import. It should be safe to drop py2-mysqldb from WMCore pacakges spec files

We opened a similar issue in dmwm/CRABServer: https://github.com/dmwm/CRABServer/issues/6228

I have not debugged this problem yet, but deployment of 1.4.2.pre7 - which no longer has the dependency on py2-mysqldb - failed when installing the database, see:
*** Starting services ***
WMAgent with Rucio enabled: true
Starting Services...
starting couch...
CouchDB has not been initialised... running pre initialisation
Initialising CouchDB on <IP_ADDRESS>:5984...
Apache CouchDB has started, time to relax.
CouchDB has not been initialised... running post initialisation
Starting mysql...
MySQL has not been initialised... running pre initialisation
Installing the mysql database area...
./manage: line 336: mysql_install_db: command not found
starting mysqld_safe...
Checking MySQL Socket file exists...
ERROR: Timeout waiting for mysqld to start.
Done!

Dario, this is likely something that we have to discuss and see what our options are.

And another problem while stopping the agent
cmst1@vocms0262:/data/srv/wmagent/current $ $manage stop-agent
WMAgent with Rucio enabled: true
Shutting down WMAgent...
Checking default database connection... Unable to make connection to using
parameters provided in mysql://cmst1:Someslc7Senha@localhost/wmagent
No module named MySQLdb
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.2.pre7/sw/slc7_amd64_gcc630/cms/wmagent/1.4.2.pre7/bin/wmcoreD", line 351, in <module>
    connectionTest(config)
  File "/data/srv/wmagent/v1.4.2.pre7/sw/slc7_amd64_gcc630/cms/wmagent/1.4.2.pre7/bin/wmcoreD", line 191, in connectionTest
    raise ex
ImportError: No module named MySQLdb

Since we failed to get rid of this library, let me get it reassigned to you Dario ;-)

@mapellidario as we discussed in the past week, I'm setting this issue free such that others can pick it, if they want.

Adding some more command output here, just for logging purposes.
cmst1@vocms0260:/data/srv/wmagent $  /data/srv/wmagent/current/config/wmagentpy3/manage stop-services
Shutting down services...
stopping couch...
Apache CouchDB has been shutdown.
stopping mysql...
/data/srv/wmagent/current/config/wmagentpy3/manage: line 390: mysqladmin: command not found
Making sure the MySQL socket file is removed...
MySQL is shutdown.

Hi @amaltaro Regarding this  comment: https://github.com/dmwm/WMCore/issues/9805#issuecomment-705519358
Can you please, provide a pointer to the so pending issue in cmsdist. Because it seems to me the dependency from py2-mysqldb.spec has already been dropped in the wmagentpy3.spec which basically drops the dependency regarding the client, but strangely enough, the MariaDB server dependency is somehow referred through this spec file [1]. So the final result is - no MariaDB service installed on the py3 wmagent version, hence the lack of all the admin commands needed by the $manage script like mysql_install_db and mysqladmin, hence the errors above. I also wonder if we should just link the dependency of mariadb directly into the wmagentpy3.spec or we should swap the MySQL-python package with it's fork for py3 [2], which is supposed to provide full compatibility with the MySQLdb1 library provided by MySQL-python package.
Please let me know your opinion here. But there is one thing I am sure about - there is no MariDB installed on the py3 wmagent currently.
[1]
https://github.com/cms-sw/cmsdist/blob/9d61b122cbf0b69f8cc601408895637c55253e42/py2-mysqldb.spec#L6
[2]
https://pypi.org/project/mysqlclient/
info taken from this reply in stackoverflow:
https://stackoverflow.com/questions/4960048/how-can-i-connect-to-mysql-in-python-3-on-windows

Hi @amaltaro Regarding this comment: #9805 (comment)
Can you please, provide a pointer to the so pending issue in cmsdist.

So, I did test WMAgent without the py2-mysqldb dependency and the result is the failure reported here
https://github.com/dmwm/WMCore/issues/9805#issuecomment-720787167
meaning, we somehow still need it (not sure what for though).

Because it seems to me the dependency from py2-mysqldb.spec has already been dropped in the wmagentpy3.spec

Yes, I didn't even try to create the mysqldb package on python3 (thus py3-mysqldb.spec) because the developers say there is no support for python3. So there was no point in trying to build it and/or add it as a dependency for the python3 stack.

but strangely enough, the MariaDB server dependency is somehow referred through this spec file [2]

good catch! That means, our wmagentpy3.spec file has no dependency at all in mariadb. We could try adding this dependency and try to deploy wmagent again (both in py2 and py3 stacks).

I also wonder if we should just link the dependency of mariadb directly into the wmagentpy3.spec

Yes, this is the first attempt I would chase. If it does not work, then we need to build a py3-mysqlclient.spec file to ship the python3 supported mysqlclient, and if needed, add MariaDB in there as a "Requires".
Just to wrap up these comments, here is what I would suggest to change (ordered):

add mariadb dependency directly to the wmagentpy3.spec file and try to deploy it (it might be needed to do the same with wmagent.spec, just to have a solid comparison baseline)
create a py3-mysqlclient.spec spec to ship mysqlclient library in python3 stack (without MariaDB, to start with). Add this spec as a dependency to wmagentpy3.spec and test deployment/running.
last option, create the spec above but adding mariadb as a dependency inside this spec. Add it to the wmagentpy3.spec and test deployment/running.

What do you think? Have I missed anything? Thanks for this investigation, Todor.

Hi @amaltaro thanks. This plan is exactly what I was having in my mind too. So we are on the same page here. But just to be sure: should we try the first two bullets separately/consequently or both together from the very first attempt?

Just before I forget I need to log two additional steps which are a must in order to put the above plan in action.

We need to fix the fact that with installing the wmagentpy3 package the deployment area changes and we need a soft link of the type:

cmst1@vocms0260:/data/srv $ ln -s /data/srv/wmagent/current/config/wmagentpy3/  /data/srv/wmagent/current/config/wmagent
/data/srv/wmagent/current/config/wmagent -> /data/srv/wmagent/current/config/wmagentpy3/

Should be notified here too https://github.com/dmwm/WMCore/pull/10302

When installing from a personal repository in cmsrep (adding the option -r comp=comp.<username> to the deployment script) the deployment area is created as /data/srv/wmagent/current/sw.<username>, but all sort of *PATH variables in the environment point to /data/srv/wmagent/current/sw. An example is:

/data/srv/wmagent/current/apps/wmagent -> ../sw/slc7_amd64_gcc630/cms/wmagentpy3/1.4.7.patch1-comp

Which is crucial for the rest of the software to initiate and run. So we will have to fix the deployment script such that it also creates the following symlink:
/data/srv/wmagent/current/sw -> sw.<username>

Otherwise even if the MariaDB dependency is properly set in the rpm [1], we still experience the previous errors, only because the environment is not set properly.
[1]
cmst1@vocms0260:/data/srv $ rpm -q --requires -p cms+wmagentpy3+1.4.7.patch1-comp-1-1.slc7_amd64_gcc630.rpm  |grep mariadb
external+mariadb+10.1.21-comp3

FYI @amaltaro

Todor, on your first bullet above. Another option would be to update the env.sh script, as mentioned in #10302

Hi @amaltaro. Yes, indeed this is an option, but it would require a manual intervention every time we deploy with -py3, which is error prone. (I personally tend to forget those procedural peculiarities the minute I am done with the first deployment after the PR introducing the change). So I'd rather put the link and let the deployment script do the rest for me.

Following the first scenario from this comment : https://github.com/dmwm/WMCore/issues/9805#issuecomment-820374596 and adding the MariaDB dependency directly into the wmagentpy3.spec: https://github.com/cms-sw/cmsdist/pull/6817 ,  together with the proposed changes to the deployment script from https://github.com/dmwm/WMCore/pull/10454 I managed to properly deploy the WMAgent and the relevant mysql databases [1]. I am fighting with another error now [2]. But I think we already agreed the later is a problem related to a separate set of package dependency.
So @amaltaro if you think we may merge those two PRs (in WMCore and cmsdist repositories) and close the current issue it would be great.  The error mentioned in [2],  we can follow in it's dedicated GH issue.
[1]
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
To do so, start the server, then issue the following commands:

'/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/mariadb/10.1.21-comp3/bin/mysqladmin' -u root password 'new-password'
'/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/mariadb/10.1.21-comp3/bin/mysqladmin' -u root -h vocms0260.cern.ch password 'new-password'

Alternatively you can run:
'/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/mariadb/10.1.21-comp3/bin/mysql_secure_installation'

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the MariaDB Knowledgebase at http://mariadb.com/kb or the
MySQL manual for more instructions.

You can start the MariaDB daemon with:
cd '/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/mariadb/10.1.21-comp3' ; /data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/mariadb/10.1.21-comp3/bin/mysqld_safe --datadir='/data/srv/wmagent/v1.4.7.patch1-comp/install/mysql/database'

You can test the MariaDB daemon with mysql-test-run.pl
cd '/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/mariadb/10.1.21-comp3/mysql-test' ; perl mysql-test-run.pl

Please report any problems at http://mariadb.org/jira

The latest information about MariaDB is available at http://mariadb.org/.
You can find additional information about the MySQL part at:
http://dev.mysql.com
Consider joining MariaDB's strong and vibrant community:
https://mariadb.org/get-involved/

starting mysqld_safe...
Checking MySQL Socket file exists...
Socket file exists: /data/srv/wmagent/v1.4.7.patch1-comp/install/mysql/logs/mysql.sock
MySQL has not been initialised... running post initialisation
Installing the mysql schema...
Socket file exists, proceeding with schema install...
Installing WMAgent Database: wmagent
Checking Server connection...
Connection OK
Done!

[2]
*** Initializing the agent ***
Initialising Agent...
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.7.patch1-comp/bin/wmagent-mod-config", line 14, in <module>
    standard_library.install_aliases()
  File "/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/py3-future/0.18.2/lib/python3.8/site-packages/future/standard_library/__init__.py", line 453, in install_aliases
    __import__(newmodname)
ImportError: No module named reprlib
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.7.patch1-comp/bin/wmcore-db-init", line 19, in <module>
    from WMCore.Configuration import loadConfigurationFile
ImportError: No module named WMCore.Configuration
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.7.patch1-comp/bin/wmagent-couchapp-init", line 9, in <module>
    standard_library.install_aliases()
  File "/data/srv/wmagent/v1.4.7.patch1-comp/sw.tivanov/slc7_amd64_gcc630/external/py3-future/0.18.2/lib/python3.8/site-packages/future/standard_library/__init__.py", line 453, in install_aliases
    __import__(newmodname)
ImportError: No module named reprlib
Done!

*** Checking if couchdb migration is needed ***
Done!

Perfect! Thanks for spotting these issues and providing these fixes.
However, I think those were pre-requirements to get you to be able to debug the actual issue reported in this issue. There has been quite some discussion here, but I'm still not sure whether we need MySQL-python (or a python3 version of it) integrated into WMAgent or not.
As we have already discussed in other venues, when creating the wmagentpy3 spec file, py2-mysqldb.spec was not added because that library does not support python3. That said, it does not mean we don't need it, because our codebase might be trying to use it somewhere in the code.
So, the expected outcome of this issue will be either:
a) WMCore codebase no longer depends in MySQL-python and there is nothing else to be done in wmagentpy3.spec
b) WMCore codebase still depends on MySQL-python and we have to find a python3 base library replacement; then update the WMCore codebase with the new library.
Could you please investigate it?

Oook then. Let me take a deep breath and I am about to dive deeper now :)

Hi  @amaltaro . So far, from what I am capable of tracing as imports in WMCore, I can not spot any direct or indirect import of the library MySQLdb (which is the one provided by the MySQL-python package). And also @mapellidario managed to run the pythondep tool as explained  here [1], and confirmed my observation. All that said, I dare to state that we are in the position:

a) WMCore codebase no longer depends in MySQL-python and there is nothing else to be done in wmagentpy3.spec

[1]
https://cms-dmwm-test.docs.cern.ch/py2py3/tools/pydeps/

Wonderful! Thanks Todor and Dario.
Just a note, if we face issues related with this in the future, we might want to directly add the "mariadb" dependency in wmagent.spec and drop "py2-mysqldb" one. This way we can compare things against something that we know is solid and working.

Now that I managed to progress with the python3 environment, we seem to have missed one usage of this library (but I could be wrong as well...):
No module named 'MySQLdb'
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre1/bin/wmcore-db-init", line 158, in <module>
    connectionTest(cfgObject)
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre1/bin/wmcore-db-init", line 123, in connectionTest
    raise ex
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre1/bin/wmcore-db-init", line 115, in connectionTest
    wmInit.setDatabaseConnection(dbConfig = config.CoreDatabase.connectUrl,
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre1/lib/python3.8/site-packages/WMCore/WMInit.py", line 131, in setDatabaseConnection
    myThread.dbFactory = DBFactory(logging, dbConfig, options)
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre1/lib/python3.8/site-packages/WMCore/Database/DBFactory.py", line 84, in __init__
    create_engine(self.dburl,
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/external/py3-sqlalchemy/1.3.3/lib/python3.8/site-packages/sqlalchemy/engine/__init__.py", line 435, in create_engine
    return strategy.create(*args, **kwargs)
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/external/py3-sqlalchemy/1.3.3/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/data/srv/wmagent/v1.4.9.pre1/sw.amaltaro/slc7_amd64_gcc630/external/py3-sqlalchemy/1.3.3/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 118, in dbapi
    return __import__("MySQLdb")
ModuleNotFoundError: No module named 'MySQLdb'

debugging

@amaltaro
Where is this StackTrace from?

It looks like we have a couple of optional drivers to use for MySQL which are supported in python3:
https://docs.sqlalchemy.org/en/14/core/engines.html#mysql
I had a quick look and PyMySQL seems to be easier to integrate with our stack, given that it's purely written in python. However, I have also found a few places suggesting that it delivers a performance worse than mysqlclient, another driver supported in python3 and based in the mysql library (c?).
I guess I will try to build mysqlclient in our COMP repo first. If it works, we will have to change the connect string in WMCore.

@amaltaro , in python if you need speed it is accomplished by Python-C bindings, therefore "pure" python is always slow with respect to Python-C binding implementation. Therefore, based on your use-case you need to decide to go with pure python or c-based implementation. The latter will always require gcc/glibc stack and therefore will be dependent on it. All high-performance tasks/libraries in python, e.g. NumPy, are basically Python-C binding to underlying C libraries.

@amaltaro I am trying to create a pip build backage of mysqlclient and installing in a minute
I will post the result here.

@amaltaro  here [1] is the PR providing a properly build package mysqlclient for py3 but the error during the deployment still persists. I am checking what else is missing.
https://github.com/cms-sw/cmsdist/pull/6926

Hi again @amaltaro,
Actually I figured out why it was having troubles. I had to put some more dependencies here and there - like making py3-sqlachemy depending on py3-mysqlclien[1], which was basically the reason for the error we were facing (The one from Alan's stack trace). Now  my deployment passed through it  and hit another wall [2], but this one I believe is no longer related to MySQLdb for py3.
[1]
https://github.com/cms-sw/cmsdist/pull/6926/commits/af7fc4a9e85b1005d2e14cd8d3063c8a76e1f1ab
[2]
*** Populating resource-control ***
Adding only T1 and T2 sites to resource-control...
Executing wmagent-resource-control --add-T1s --plugin=SimpleCondorPlugin --pending-slots=50 --running-slots=50 --down ...
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/bin/wmagent-resource-control", line 16, in <module>
    from WMCore.ResourceControl.ResourceControl import ResourceControl
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/ResourceControl/ResourceControl.py", line 11, in <module>
    from WMCore.BossAir.BossAirAPI import BossAirAPI
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/BossAir/BossAirAPI.py", line 23, in <module>
    from WMCore.JobStateMachine.ChangeState import ChangeState
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/JobStateMachine/ChangeState.py", line 20, in <module>
    from WMCore.Services.Dashboard.DashboardReporter import DashboardReporter
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/Services/Dashboard/DashboardReporter.py", line 13, in <module>
    from WMCore.Services.Dashboard.DashboardAPI import DashboardAPI
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/Services/Dashboard/DashboardAPI.py", line 12, in <module>
    from WMCore.Services.Dashboard import apmon
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/Services/Dashboard/apmon.py", line 55, in <module>
    from types import LongType
ImportError: cannot import name 'LongType' from 'types' (/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/types.py)
Executing wmagent-resource-control --add-T2s --plugin=SimpleCondorPlugin --pending-slots=50 --running-slots=50 --down ...
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/bin/wmagent-resource-control", line 16, in <module>
    from WMCore.ResourceControl.ResourceControl import ResourceControl
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/ResourceControl/ResourceControl.py", line 11, in <module>
    from WMCore.BossAir.BossAirAPI import BossAirAPI
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/BossAir/BossAirAPI.py", line 23, in <module>
    from WMCore.JobStateMachine.ChangeState import ChangeState
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/JobStateMachine/ChangeState.py", line 20, in <module>
    from WMCore.Services.Dashboard.DashboardReporter import DashboardReporter
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/Services/Dashboard/DashboardReporter.py", line 13, in <module>
    from WMCore.Services.Dashboard.DashboardAPI import DashboardAPI
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/Services/Dashboard/DashboardAPI.py", line 12, in <module>
    from WMCore.Services.Dashboard import apmon
  File "/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/cms/wmagentpy3/1.4.9.pre2-comp2/lib/python3.8/site-packages/WMCore/Services/Dashboard/apmon.py", line 55, in <module>
    from types import LongType
ImportError: cannot import name 'LongType' from 'types' (/data/srv/wmagent/v1.4.9.pre2-comp2/sw.tivanov/slc7_amd64_gcc630/external/python3/3.8.2-comp/lib/python3.8/types.py)
Done!

Actually I think the dependency I've put in py3-sqlachemy is redundant. The py3-mysqlclient is already properly referred by wmagentpy3.spec. Maybe during the first deployment attempt I messed up the wmagentpy3 package versions compiled and uploaded in my repository. So I am removing this from the py3-sqlalchemy.spec file and we are ready to go now ... I think...


Upgrade machine-controller to v1.57.2

What this PR does / why we need it:
With updated machine-controller we can finally drop replace directive from the go.mod. And we'll be able to go install<EMAIL_ADDRESS>after the release.
What type of PR is this?
/kind bug
Special notes for your reviewer:
Does this PR introduce a user-facing change? Then add your Release Note here:

Upgrade machine-controller to v1.57.2

Documentation:

NONE

/retest


5.5.64-MariaDB Replication Issue

System information:

Operating system (distribution) and version
Host: Deepin GNU/Linux 15.11
Virtual Machine: CentOS Linux release 7.7.1908 (Core)
DBeaver version
<IP_ADDRESS>904071344

Connection specification:

Database name and version
5.5.64-MariaDB MariaDB Server

Describe your question:
I tried to simulate MySQL master-slave replication.
At first I was using the GUI on DBeaver to create、insert.......
show master status;
show slave status\G

I found the pos of bin-log did change.But there's nothing happened on the Slave.
I almost gave up my experiment and eventually tried using the command line approach to create tables and insert data in the virtual machine MySQL, and found that it worked.
I tried to do the same thing in another application (Navicat), it worked too.
Then I went back to DBeaver's SQL editor, ran the SQL script, and the slave changed.
I checked the log and found no difference.

In addition, I found a similar question in #995
What should i do?

I get the same result in the MySQL Workbench.....

I just turn on the option of "log-bin" on the slave instance.
And then i found it fix my problem!
......I checked show master status; on the slave instance, the position didn't changed..
.......It still work when i #(turn off) it..
So ....i fix my problem>....sry...


Entire name of events with very long titles are displayed on the home page, which distorts the layout of the events

Tested on multiple devices:
Galaxy S8, Galaxy Note Edge, iPhone 7+, iPhone 6, iPhone X (BrowserStack)
The entire name of events with very long titles get displayed on the home page when the event is added to the Featured events list. This disrupts the layout of the featured events and they become misaligned, leaving large sections of whitespace:

Would love to get UI to chime in here. We can probably truncate after a certain size. Would love to know how many lines should we allow? One? Two? More?

Two lines has been decided 😀


Upgrade cvat from v 0.2.0 to v 0.4.2

Hello,
We have a lot of work done on our instance of cvat v 0.2.0 and we are trying to update to version 0.4.2. The installation process went fine but when we try to copy the database from the old version to the new version the front end stops working. It seems as though as the schema of the database is different. Also, as far as I verified the images are not contained in the database. They are on a virtual machine called cvat under a folder called data. I will also have to copy all the images to the new version. Maybe the directory structure where the images are placed also changed from the old version to the new, right? Is there some way to overcome this situation and to be able to update keeping all the work that we have already done?
Regards,
Luciano

@LucianoTarouco1
Database schemes is actually different.
But I don't understand why you need to copy old database to new instance manually.
CVAT update don't erase old data.

Hi, thanks for your reply.
Please let me know how should I proceed to do the update. Shall I follow the steps on the installation guide? Updating from the repository to the same folder and then doing "docker-compose build" etc?
Will this process update the database schema and maintain the data?
Regards,
Luciano

@LucianoTarouco1
Something like that:
git clone https://github.com/opencv/cvat.git
cd cvat
git checkout master
# the same build and up commands like before

Done.

@LucianoTarouco1
If you have downloaded previous repository, then from it's root dir:
git checkout master
git pull origin master
# the same build and up commands like before

@LucianoTarouco1
I'll close this issue. Don't be shy to reopen it if you still have questions. Or you can ask them on gitter.

Hi,
I am also trying to upgrade. I have pulled the new version from git and ran docker-compose build.
After the upgrade, the dashboard looks fine but when opening a task I get a blank screen.
I understand that the schema has changed. Is there a way to update the old data so it fits the new schema?
Thanks


Fix build issue on macOS (aarch64, gcc) and use debugtrap wherever possible

This PR makes debugbreak prefer __builtin_debugtrap() wherever it's available (clang).
The code snippet I've used is @jwakely's work, I've added a signoff to my commit message to mark the original author.
The patch fixes a build issue too on macOS (Apple silicon) when a real GCC compiler is used:

warning: implicit declaration of function '__builtin_debugtrap';

Depends on #22
Fixes #24

I'm testing this change on the following architectures (with clang + gcc):

[x] x86-64

gcc/clang: works

[x] ARM mode, 32-bit (Raspberry Pi 1 armv6l)

gcc/clang: works (stepping in GDB does NOT work)

[x] Thumb mode, 32-bit (Cirrus CI)

gcc/clang: works (stepping in GDB does NOT work)

[x] AArch64, ARMv8 (Raspberry Pi 4)

gcc/clang: works (stepping in GDB does NOT work)

[x] Apple silicon (macOS M1 machine):

gcc: works (stepping in LLDB does NOT work)
clang: works

Unfortunately, I won't be able to test the change on riscv and powerpc.

I'll see if I can test it on powerpc64.

I've completed my tests on the platforms that were available to me.
It seems nothing got worse with this change, just slightly better, for example, on macOS.
@scottt What do you think? Is this something you would consider integrating into the master branch?

on POWER8 running CentOS 7.9:
$ test/fib ; test/trap ; test/break ; test/break-c++
Trace/breakpoint trap (core dumped)
Trace/breakpoint trap (core dumped)
Trace/breakpoint trap (core dumped)
Trace/breakpoint trap (core dumped)

@jwakely Thank you.

We had to start using a fork of debugbreak due to this build issue on macOS.

@scottt Do you have any comments on this?


Why is main async? It doesn't return a task. Sloppy work.

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 17d27a0d-c072-8c9f-7e31-3de60bee9ae8
Version Independent ID: e0c09794-e712-19ff-cb9f-12648d60d4f6
Content: Asynchronous programming in C#
Content Source: docs/csharp/programming-guide/concepts/async/index.md
Product: dotnet-csharp
Technology: csharp-async
GitHub Login: @BillWagner
Microsoft Alias: wiwagn

Can you clarify which example you're talking about and expand on what's wrong with it?
In this article, I only see static void Main and static async Task Main, which are both correct.

The Task-returning method using async and await not needing a return statement is what I find confusing. I will need to read up on that some more. I suppose some note or link in the docs addressing this potential confusion might be useful.


chore: try to get release version back on track

Trying to get releases back on track by jumping to 0.8.0
Release-As: 0.8.0

i think the "release-as: 0.8.0" has to go in the title so it ends up in the commit message


Installation

Tried to install panamax on Mac Sierra.
Result was:
Error: Calling Formula.sha1 is disabled!
Use Formula.sha256 instead.
<localuserpath>/Library/Caches/Homebrew/Formula/panamax.rb:7:in `<class:Panamax>'

NB: why on earth should a management software which try to use SHA1 ?

It is calling SHA1 from Brew.  They do this to validate the integrity of the download.  Unfortunately, Brew no longer supports SHA1 and their recommendation is to use SHA256.  You could try downloading the installer, edit the file and remove the line that starts with "SHA1" (there are two).  Then do 'brew install ~/panamax.rb'.  When you do this, Brew will suggest a SHA256 hash that you can then insert in file if you so desire.  However, even after you plug in the correct SHA256 values into the panamax.rb file, you will find that there are other errors that come up if you are running Sierra...


JSX DOM render extremely slow

What version of Hono are you using?
4.5.2
What runtime/platform is your app running on?
Bun
What steps can reproduce the bug?
import { render } from "hono/jsx/dom";

const root = document.getElementById("root");

render(
	<>

	</>
, root);

What is the expected behavior?
Rendering something of that complexity should be immediate
What do you see instead?
It takes about 12 seconds to render the code above. Twice as much if I add one more

Additional information
The time it takes for the render function to complete increases exponentially with the amount of nodes.
From what I gathered it seems to get stuck in the findInsertBefore function

Hi @luaVolk, Thanks for the report.
I am now investigating!


Complete the last argument if not matching a hint

Instead of stopping the completion when there is a value for an
argument, try to verify if the value is matching one of the valid
option. If it's not matching, try to suggest all options starting by the
entered value.
Hopefully it should address https://github.com/alecthomas/kingpin/issues/159

It is a really good idea. I just would like to include tests.

I've added tests and fixed some edge cases.
@alecthomas Is there anything preventing this to be merged?

Is there any update on this?
I'm using MiLK's fork because this is an important feature for me.

I honestly have no idea why I didn't merge this, sorry :(


How to sync sessions between different TC servers ?

[ ] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description
I start two TC servers, <IP_ADDRESS>:8091 and <IP_ADDRESS>:8092 in 'file' store mode .

The business service send a begin transcation request to 8091.
The business logic throws a Exception,  then TM send a rollback request to 8092.
The TC server return rollback success, though it can not find the session and the transcation has not be rollbacked.

Ⅱ. Describe what happened
None
Ⅲ. Describe what you expected to happen
The TC server return rollback success, and  the session is rollbacked.
How to sync sessions between different TC servers in 'file' store mode?

tow TC is High Availability,please use db mode

The session is no need to synchronized between TCs，they should share data with database.


SCSI device scan fails on some kernels

While testing custom kerenls, I often noticed seeing the following error in pod status:
write /sys/class/scsi_host/host1/scan invalid argument

This is caused by:
https://github.com/kata-containers/agent/blob/c1d24f92e9e2fb454ffc70fe8583e49ca80ed893/device.go#L321-L348
I wonder if this is provides some critical functionality in Kubernetes context, or a configuration parameter could be added to turn it off without a major compromise? From the original PR https://github.com/kata-containers/agent/pull/163 it sounds as if this was not a critical feature as such, but I'm not sure if it has become one since then.
I guess aside from adding a toggle for this, there could be a way to fix this that e.g. by turning this error into a warning or being less opportunistic on how the SCSI scan is performed?

@jodh-intel @devimc  I have enabled debug logs, but not been able to tell as what comes out in the journal is rather very verbose and mostly unreadable as most messages get doubly quoted etc. Is there a tool that people use to parse it?
To be clear, my pod doesn't have any volumes, it would help to know if the this arises from the fact that kata passes default secret as a SCSI device to the VM, I have no idea how to tell whether this is the case or it's something else.

I've double checked the kernel config that I used, and there are indeed a few difference, albeit SCSI is generally enabled.
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_MQ_DEFAULT is not set
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set

And kata's config has the following:
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_MQ_DEFAULT=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set

@errordeveloper - https://github.com/kata-containers/tests/tree/master/cmd/log-parser might be able to help you as it can convert the logs to various formats.

@errordeveloper any update on this (and/or the related #768). Is this still an issue/blocker?
@devimc - any thoughts on those config differences for SCSI above?

It is still an issue, I am using a fork from my PR. I think this could be really to do with many disks being passed from the host, isn't that what it is attempting to do?

@errordeveloper A bit later here. But looking at the kernel configs you posted, those should be sufficient.
Can you add debug logs to check if the path actually exists in your case /sys/class/scsi_host/host1/scan ?
I would also like to understand what storage driver you are using and the pod yaml you are using.

I am not working on this project right now, but the PoC can be found in https://github.com/errordeveloper/kube-test-cluster-operator and if someone wants to dig into this, I am happy to guide them.

We still have this functionality in the new agent here:
https://github.com/kata-containers/kata-containers/blob/2.0-dev/src/agent/src/device.rs#L167
/cc @egernst, @sboeuf in case they have thoughts on this.

Since we don't see this error on the kernel we provide we are closing this issue.
Feel free to open this issue if needed.

If this is causing a problem for usesr with custom kernels, the agent could be modified (PRs welcome! ;) to check for the existence of /sys/bus/scsi/ maybe?


All strat runners are lost after restart

I'm submitting a ...
[x ] bug report
Action taken (what you did)
Restart Gekko
Expected result (what you hoped would happen)
Having my strat runners back after restart
Actual result (unexpected outcome)
All strat runners are lost
Is there a possibility to resume my strat runners after restart ?

I'm wondering the same. Or is there a plugin we can use?

I solved the problem with some changes in gekkoManager.js i will share the code once i believe it's stable.

I solved the problem with some changes in gekkoManager.js i will share the code once i believe it's stable.

Please do, im dealing with this aswell and its quite annoying especially having to warmup

@elaarabi ?

@elaarabi ?

Hello GregoryGHarding, sorry for being late, i share here my file, it was not properly programmed or commented, but im sure this will help u a lot figuring out how to reload the strat runners after a crash or a pm2 reload for example.
gekkoManager.zip


How to compile or cross compile into Windows binary programs on the Windows platform

Is cargo build from the root directory not working?

Thanks for the follow up @SamuelMarks, we should get this documented in the README as well.


Fix to server models unit tests so they run in EAGER mode

This PR is introducing little fixes so server/models/test_causal_lm.py  can work in EAGER mode.
To run mentioned unit tests in eager mode:
ENABLE_HPU_GRAPH=false PT_HPU_LAZY_MODE=0 HUGGING_FACE_HUB_TOKEN=<your HF token> HF_HUB_ENABLE_HF_TRANSFER=1 pytest -s -vv  server/tests/models/test_causal_lm.py

Changes merged in #157


Clarify stance on AI generated content

You should probably put something in the rules about whether AI generated code is allowed

Added with latest commit 👍


Don't show entry delete confirmation if the entry has no data

If entry has duration of under 15 seconds and no description and no project  don't ask for confirmation if user deletes it. No data is lost and this makes cleaning up a lot faster.

Maybe it's a personal preference, but I'd find it useful if this worked for any entry with <15 seconds.
Sometimes I click "continue" on a wrong entry, and then you have to go through the whole: stop task>right click entry>click delete>confirm.
I mean, how big can a loss of 15 seconds task can be?

@Trost123 this kind of makes sense. I'll reopen this case and we'll take another look.


Unread notice doesn't shrink completely when composeBox expands full screen.

On iOS:

This is also reproducible on Android.
Steps to reproduce:

Open any chat which have unread messages.
Press enter many times in compose box message input, so that compose box expands.
Unread notice doesn't goes off completely.

This is not completely fixed.
Even after 4ef9a13, in topic narrow it is reproducible


ref: Remove extraneous codecs from arroyo

As of version 0.1.0, sentry-kafka-schemas now contains this code.
Arroyo still needs to have some concept of encoding because it
implements commit log topics... Not entirely sure what to do there. I
guess we'll just keep a minimal codecs impl around?

I'd like to eventually move the commit log codec to sentry-kafka-schemas too. It's bugged me for a while that it was in Arroyo as it's extremely specific to how Sentry uses that topic. Wdyt about changing the format to json while we're at it rather than relying on information spread between the header, key and value that has to be decoded in a very specific way?


Ardrone velocity

I thank you for this great code. However, when I fly my Ardrone with this code, its velocity is not constant. What should I do for constant velocity flight?

The code here: https://github.com/andrew/ar-drone-gps/blob/master/server.js#L115 should make it move at a constant speed, although the frequent changes in direction could be slowing it down, it's hard to tell without more information.

I'm no longer supporting this library as I don't use it any longer so I'm going to close this issue.


Big Commerce orders are not getting downloaded

Expected behavior
Orders placed in Big Commerce (new tastefullysimple.com orders) should be downloaded to TS databases and appear in the big commerce table

TSN, myHQ, Prowess.
Actual behavior
I placed a few orders, #806, and #808 and they are not being downloaded. The last order appearing in the Big C. dbo.order table is #804 from 11:28am this morning.

Jared checked for me and Big commerce jobs is running on HQINTRA10.
I'm also not able to view the event log, it just keeps 'executing' and not giving me results:
SELECT TOP 500 DATEADD(HOUR,-5,EventDate) AS 'Central Time',*
FROM [TSApplicationLog].[dbo].[EventLog] WITH (nolock)
WHERE logger = 'ScheduledJobs.Jobs.Integration.BigCommerce.Import.DownloadOrdersForFulfillmentJob'
ORDER BY EventDate DESC
Steps to reproduce behavior
Place an order in Big C and wait. LOL. :) View the table:

SELECT TOP 500 DATEADD(HOUR,-5,EventDate) AS 'Central Time',*
FROM [TSApplicationLog].[dbo].[EventLog] WITH (nolock)
WHERE logger = 'BigCommerceJobs.Jobs.Download.DownloadOrdersForFulfillmentJob'
AND EventXID > 229485223
ORDER BY EventDate DESC

Please check again. I posted the updated event log query above.

7-29 SH - I see the orders are being downloaded now. My order from yesterday that I attached to a party is not showing in myhq so I will log a new bug for this.


[WIP] JExtractSwiftPlugin

🚧 early work in progress 🚧
This is first steps to make an jextract-swift plugin.
If we're able to pull this off as a swiftpm plugin we would basically have a way to "make me a jar that exposes this swift module, that would be super simple to consume from Java.
Need to figure out how to actually make this work -- it can't be a build plugin, but preBuild also isn't quite enough.
We may have to make it a command plugin, since we need compilation OUTPUT of the module to then act on the swift interfaces.

Before weekend snapshot:

we now generate swift thunks (!)
first thunk is to "give me the Any.Type of this type" rather than using swift interfaces

these are done via public func swiftjava_getType_JExtractPluginSampleLib_MyCoolSwiftClass() -> Any /* Any.Type */ generated thunks

jextract is now a prebuild plugin so we can get seamless build integration

Basically this means you "just" write some swift sources, add the extract plugin to the given target. Add a small task in gradle to trigger swiftpm and this way gradle/java immediately see the wrappers. Upon changing swift -> gradle triggers swift build again and it's all very seamless 🥳


Tags on Redhat container catalog appear incorrect

It looks like tagging on the Redhat container catalog is not working right.  I've pulled the image tagged '1.5.0' expecting to get Kong 1.5.0 but the image contains Kong 2.0.0.  Tag '1.4.2' has the expected Kong version, but the later ones here all appear to be Kong 2.0.0 based on what shows in the 'package list' in the Redhat catalog website.

looks like redhat muddled our version control sometime around 2.0.0.
We've since moved away from using their automated build service and instead build / push from our own infrastructure which includes tests to catch this and explains why all the tags after 2.0.0 are fine.
I've deleted the offending tags, build and pushed fixed tags with -0 since it seems redhat repository won't let me overwrite existing tags


improve support for Easy Digital Downloads

When WP-Stateless is used alongside Easy Digital Downloads, the "redirect" download method is always used regardless of the setting in EDD. Ideally, WP-Stateless should respect the download method setting and support the "Forced" download method option.

Done.


Confusing message in scope.log related to missing uv_fileno

Overview:
For the processes which don't use uv_fileno API we should omit the following error information in the log file:
Scope: java(pid:11): ERROR: Could not get function address of uv_fileno.
The message can be confusing for the end user.
Reproduction steps:

Start container with Java 10

make java10-shell

Start ldscope of the Java Server

cd /opt/java_http
ldscope java SimpleHttpServer 2> /dev/null &

Observe scope.log file

cat /opt/test-runner/logs/scope.log

See that after the init command message there is "Error information" for missing uv_fileno :
Scope: java(pid:11): {"current":{"metric":{"enable":"true","transport":{"type":"udp","host":"localhost","port":"8125","tls":{"enable":"false","validateserver":"true","cacertpath":""}},"format":{"type":"statsd","statsdprefix":"","statsdmaxlen":512,"verbosity":4}},"libscope":{"log":{"level":"info","transport":{"type":"file","path":"/opt/test-runner/logs/scope.log","buffering":"line"}},"configevent":"true","summaryperiod":1,"commanddir":"/tmp"},"event":{"enable":"true","transport":{"type":"file","path":"/opt/test-runner/logs/events.log","buffering":"line"},"format":{"type":"ndjson","maxeventpersec":10000,"enhancefs":"true"},"watch":[{"type":"http","name":".*","field":".*","value":".*","headers":[]},{"type":"net","name":".*","field":".*","value":".*"},{"type":"fs","name":".*","field":".*","value":".*"},{"type":"dns","name":".*","field":".*","value":".*"}]},"payload":{"enable":"false","dir":"/tmp"},"tags":{},"protocol":[]}}
Scope: java(pid:11): ERROR: Could not get function address of uv_fileno.

Details:
The confusing message is related to commit: https://github.com/criblio/appscope/commit/f8d0826a03848c5e8f546c5e759be57169aa66ed and it comes from load_func:
https://github.com/criblio/appscope/blob/2f3ced0045904bd9626dde8f4a3f23e2f6ec41c9/src/wrap.c#L1089
My only additional comment is possibly the simillar logic just like with the should_we_patch
can be used here: to determine if we are looking for the uv_fileno symbol so we can avoid false-positive error message.

QA Instructions:
$ scope run -- ps -ef
$ scope events
Verify that you have events from process ps
$ scope logs
Verify that a log message related to uv_fileno is not emitted. Example: "No logs generated"

Verified on scope:1.0.0-tc2. Message No logs generated is displayed.


How to make this package compatible with Laravel Nova 4?

nova-resizable:2 Uncaught TypeError: Cannot read properties of undefined (reading 'components')
at nova-resizable:2:4548

Hey, @kylescousin 👋🏻
This package does not yet support nova 4, pull requests are welcome on this matter.
You'd need to follow Nova's official documentation about upgrading
https://nova.laravel.com/docs/4.0/upgrade.html

How much work would that be? Not sure where to even start to be honest.
Can you point me in the right direction?


Feature ✨: Add scroll to top button

What would you like to share?
Hey @punyakrit  ,
I have noticed a small issue that I believe could enhance the user experience. Currently, there is no scroll to top button present on the website, which makes navigation a bit cumbersome, especially when users need to return to the top of a lengthy page.
Though it may seem minor, the absence of this feature can impact the overall usability and performance of the website.
Screenshot(Mandatory):

@punyakrit  ,Could you please assign me this issue under gssoc'24 ?
Are you contributing under any open-source program ?
GSSOC'24
Additional information
No response
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

none of the page are lengthy for now so don't require it


[fix] Use org_name argument in all cases

Without this, providing a custom organization name does not work.

coverage: 97.77%. remained the same when pulling 9b9b6469468635280d853509e10180c554d37f2b on fix-test-utils into 8403e5a242b2a4605868592729b18c2b520f2ec8 on master.


7th generation instances are not visible on the costs tab

Instances that belong to the AWS 7th generation, e.g. m7i, are not visible on the cost card.

If you think this could be connected to "spot". No, in another cluster where I have 6th and 7th gen, I only see 6th gen, no matter spot or on-demand:

@gkalwig, thanks for the report. We've updated the cloud pricing data.
Coroot automatically updates it once a day from GitHub, but you can force the update by restarting Coroot

Thanks, after restart looks good.
You need to update that data manually? There is no pipeline to cyclic refresh it from AWS?

It's updated every day now


Need Loader please

Ir shows no rows found for 2 3 seconds and after that renders the data even if data is already there. Please help
https://nimb.ws/HqQS0C

Fixed in version 1.5.0


Best way to style the field please espacially the label

Hello,
Do you recommend an approach to change the style of the field, for example to change the label text to bold and put it to the left of the field?

thanks, a lot. if you don't mind where can i know all the names of the CSS classes of the field other than the ones from the theme?

Thank a lot


Question : addNote capability

Can you force addNote to set to auto "Remind me"

There currently isn't an option to do this through that function. I will add one for the next release.

The addNote function has been updated so that the remind me can be set for a passed in list of user IDs. Check out the update documentation for more! https://simplecalendar.info/functions/SimpleCalendar.api.addNote.html


http.nonProxyHosts system property ignored

Hi,
It seems the plugin ignores the properties http.nonProxyHosts and https.nonProxyHosts.
For example, I get a HttpHostConnectException if I try to download a file from Google and I pass non-existent proxy settings, while ignoring *.google.com at the same time:
gradle downloadFile -Dhttp.proxyHost=localhost -Dhttp.proxyPort=999 -Dhttp.nonProxyHosts=*.google.com -Dhttps.nonProxyHosts=*.google.com

Thanks!

Yes, I can confirm this. Thanks for reporting.
I'll try to implement this feature when I've got some time, but it might take a couple of days/weeks. Pull requests are very appreciated :-)

Yeah, I'm already starting to fiddle around with the code. I'm not a very big expert on HttpClient, but hopefully I'll have something to send you for review soon :blush:

I'm closing this one in favour of #63.

@michel-kraemer I think we should keep this open as #63 seems to be closed :(

@timotei What do you mean? There was a pull request #68 which has been merged. So both issues can be closed now. If you have a problem related to this feature please open a new issue.

Sorry. I meant #72. This was the PR that was merged at the end.

@michel-kraemer Oh. OK. I haven't seen anywhere the link to #72, that's why I was confused.
Thanks for the clarification!


readFileSync, writeFileSync breaking change since rootDir for the wor…

file path in test files while reading/writing has to be changed to the workspace dir in order for the test file to run tests for the workspace.
when running tests for the workspace using,
npm run test -ws
FileNotFound error for the read/write files in the test files. like,
ENOENT: no such file or directory, open 'src/rz/angular/effects/interface/index.ts'
For example:
src/rz/angular/effects/directive/user.directive.ts
has to be changed into
effects/directive/user.directive.ts .
since rootDir for the workspace file is rz/angular not src/rz/angular

I am thinking on this now


updates dependencies

Codecov Report

Merging #11 into master will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff          @@
##           master   #11   +/-   ##
====================================
  Coverage       0%    0%
====================================
  Files          12    12
  Lines         241   241
  Branches        6     6
====================================
  Misses        241   241

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39e98fd...6a22043. Read the comment docs.


backing as array

Based on the docs the backing field could be an array or a string.

closing as repo will be archived.


Upgradeable contracts

Demonstrates upgrading ink! contracts with a set_code

I've taken a quick look and I get the general idea. I definitely support teaching the concept of upgradeable contracts. I actually didn't know pallet contracts allowed that. I'm excited that students will first encounter this at the smart contract level so they are better prepared to understand runtime upgrades which are really the same thing, but sometimes feel harder to understand or more "mind bending".
Overall, I'm fine with it being merged, but all this ink content still feels out of place to me in this repo that is mostly about a PoW node.
I understand that you want to test this stuff against the node, so, again I'm fine to have it all here especially for the upcoming wave. But ultimately I wonder if the ink content should have a separate loosely-coupled home.
My plan for the contract writing workshop (at least for now) is to take some of the contract ideas covered here and reduce them to starter code for students to implement on their own.

Exactly, the idea is to have it tested against the node, and then create a submodule in the contracts repository.
None of it is production quality code, far from, and there is plenty of improvements and extensions possible - happy to list them


Sending Response entities as exceptions.

Updated Exception Handling mechanism, which involves sending ResponseEntities eg:

{
"timestamp":<PHONE_NUMBER>.752647600,
"status": 417,
"message": "Invalid details supplied for the project ProjectDto(id=0, projectId=STAGING_PROJECT, createdAt=2023-04-06T13:31:32.897Z, updatedAt=2024-04-06T13:31:32.897Z)",
"path": "uri=/projects",
"cause": "'id' must not be supplied when creating a project as it is autogenerated."
}

Right now which is:

{
"timestamp":<PHONE_NUMBER>402,
"status": 417,
"error": "Expectation Failed",
"path": "/projects"
}

@yatharthranjan, @mpgxvii
Please review this and let me know if you have any suggestions for improvement.
Thanks

@mpgxvii how will the aRMT app be affected by this? Will we need to update the error catching code in the app?

@mpgxvii how will the aRMT app be affected by this? Will we need to update the error catching code in the app?

Yes currently for all appserver errors, we just show a generic error message "Unable to connect to the appserver". I'll make the changes on the app side to show the appropriate error message.

Thanks for the review!!
Also updated handling of unhandled exceptions too.

Hi, one more thing I noticed--for the exceptions already registered (e.g., HttpRequestMethodNotSupportedException), it is responding with the default entity-body.
Here, two entities can be returned, one with a cause and the other without a cause. Is it fine, or should I create a single class for ErrorDetails, leaving the cause field empty when there is no throwable?


Added krzysztofjankowski.com

https://gtmetrix.com/reports/krzysztofjankowski.com/WtvatMQh/
This PR is:

[*] Adding a new domain
[ ] Updating existing domain size
[ ] Changing domain name
[ ] Removing existing domain from list
[ ] Website code changes (512kb.club site)
[ ] Other not listed

Do not tick a checkbox if you haven’t performed its action. Honesty is indispensable for a smooth review process.

[*] I used the uncompressed size of the site
[*] I have included a link to the GTMetrix report
[*] The domain is in the correct alphabetical order
[*] This site is not a ultra lightweight site
[*] The following information is filled identical to the data file

- domain: krzysztofjankowski.com
  url: https://krzysztofjankowski.com/
  size: 131
  last_checked: 2022-01-30

GTMetrix Report:

I optimized the fonts and got a huge size reduction. I updated the entry.

Love the design of your site. Great work!


Cycler

Between Julia 0.4 and 0.5 boot time has gone up from ~5s to ~40s. This PR resolves this in several components.
ipc.coffee factors out our mechanism for remote procedure calls, which was previously tied to the Julia client. This is simply so that we can create an RPC communication channel with other kinds of server (and is used below).
cycler.coffee takes advantage of opportunities for concurrency between the user and the computer. If the user is working with an already-booted instance of Julia, we can use that time to boot up the next instance preemptively. This reduces boot time to sub-second. It gets even better; if the process finishes booting but still isn't needed, we can spend some time warming it up (by simulating the execution of common commands) and make it even more responsive.
server.coffee makes this even more complicated advanced by doing the above in an entirely separate process. This means startup time is saved even when opening fresh atom windows. It should also enable a much faster feedback loop when writing tests, which currently takes several minutes due to the need to boot multiple clients.
This has been working well for me and makes things feel much snappier and nicer, but given the extra surface area for issues we should probably roll it out gradually. Feedback and testing are of course appreciated.

I really like this change, even though the code got slightly more complex ;)
Apart from some spurious issue I had with interrupts not working (which I couldn't reproduce since), I only would change the cache size to something like 1 (maybe 2) -- I don't think it's hugely common to kill a julia process more than once every ~15s (or apparently ~40s on OSX), right?

Just a bit :)
It was around 40s for me on Windows as well. I often find needing a couple of clients in a row because I need to attempts to fix everything, but that might not be the common case. We can start with 1 and leave it as an option.


Time Question

Can I use all  the cron shortcuts, like @hourly?  If not, how would I write the Job_Time parameter if I want something to run hourly?

Jobber has not full cron library and is limited to some simple expressions. If you want to run something hourly you will have to fix seconds and minutes, e.g.: "0 0 * * * *"
Jobber's documentation on cron strings can be found here: http://dshearer.github.io/jobber/#time-strings


Add a way to convert TextureRegion to Texture or more advanced SpriteBatch calls

I've just swiched from assets manager to texture atlas.
Here is my code before:
Graphics.batch.draw(this.tiles, x * Block.SIZE,
			y * Block.SIZE, Block.SIZE, Block.SIZE,
			Block.SIZE * (binary), variant * Block.SIZE, Block.SIZE,
			Block.SIZE, false, false);

But now, because I can't convert TextureRegion (with is produced by TextureAtlas), I've switched to TextureRegions.
But the thing is, that I can't render it with the same code! There is not SpriteBatch call,  for drawing TextureRegion with custom src!
Please, add a way to convert TextureRegion to Texture or more advanced SpriteBatch calls !
Thanks.

@Tom-Ski, why did you close the issue?

You don't switch from asset manager to texture atlas, these are two very different things.
You don't convert a TextureRegion to a Texture, a TextureRegion references a certain area of a Texture.
Please use the forum or the irc channel for assistance, or state explicitly what it is you think is missing.
Please see https://github.com/libgdx/libgdx/wiki/Issue-Tracker


feat: section bg and drop-shadow alignment

Description
This PR aligns bg and drop shadow on the sections between landing and /partner pages

Review Error for AshishYelekar @ 2023-08-17 21:34:40 UTC
User failed mfa authentication, public email is not set on your github profile. see go/mfa-help


deny from added

How do not earn 403, and directly added IP's to deny from?

I think this is beyond scope of this project. Firewall like CSF can be instructed to permanently or temporarily block IPs with too many 403. Also there are more complex scripts that can block IP if it create set number of 403s, such script is ZBBlock (though it is very aggressive in blocking), another such script more advised now is https://github.com/Maikuolan/CIDRAM/ (from its Docs: ""infraction_limit" - Maximum number of infractions an IP is allowed to incur before it is banned by IP tracking. Default = 10.")


[MM-31224] fix reloading servers and other tab issues

Summary
Change server load, so it reuses browserviews, preventing white pages flashes
Fixed some errors from the renderer that were turning the tabs white
what's not fixed:

remove saving... label, when changing any settings the label remains. Created ticket MM-31554
Sometimes it doesn't update which tab is in focus.

Issue link
MM-31224

LGTM but I couldn't figure out the error in CI

seems like the deps are failing, i'll have to take a look into that, not sure if it is temporary or permanent

LGTM but I couldn't figure out the error in CI

seems like the deps are failing, i'll have to take a look into that, not sure if it is temporary or permanent


TST: Ensure dot results are placed in unique files.

[x] Tests added / passed
[x] Passes black dask / flake8 dask

Fixes #5932.

@QuLogic did this resolve the fedora issue you were seeing ?

Yes, the last 3 builds are passing now.

cool!   merging in -- adding tmpdir makes sense


External or Internal ip address ?

Hello !
Thanks for this plugin ! I love it ! Easy to use.
I receive this address ip : <IP_ADDRESS> when i do this :
String ipAddress = await GetIp.ipAddress;
    print(ipAddress);

It's an internal address ip ? Is it possible to get the external address ip ?
Thanks

Hey! I just checked it. It provides the external address ip

Thanks !! I Appreciate :)
Have a great day

I have the same problem.

i am getting internal IP also,  how did you solved it?
@pascalf22 @javib51

i am getting internal IP also, how did you solved it?
@pascalf22 @javib51

Are you testing it from the phone or the emulator? I tested using the phone and it was working fine

Hi,
Teste on emulator and also from device.
Both of them are getting 192.xxx

Did anybody find a way to get the public IP address?

Hey everyone.
The package returns the ip address your device is aware of. If you are connected to wifi you get the internal ip address.
If you are connected to mobile data you get the external.
You can do a simple GET request to https://api.ipify.org/ to get the public/external ip address of your device. To perfom the request use following code:
var external = await HttpRequest.getString("https://api.ipify.org");


package osbs to fedora

[x] rpm package osbs -- resolved in #15
[x] add it to fedora

resolved by @jpopelka


Error of NexusRepository with the name provided already exists

Hello, I have been facing a error constantly while trying to use this plugin.
A problem occurred evaluating root project 'api'.
> Cannot add a NexusRepository with name 'kromaticsnapshots' as a NexusRepository with that name already exists.

I already modified the name multiple times and it continues to throw that problem.
Configuration:
nexusPublishing {
        repositories {
            kromaticsnapshots {
                snapshotRepositoryUrl = uri('https://repo.julianstaudt.de/repository/kromatic-snapshots/')
                username = System.getenv("KROMATIC_REPO_CI_USER") as String
                password = System.getenv("KROMATIC_REPO_CI_PASSWORD") as String
        }
    }
}

Publishing:
publishing {
        publications {
            bukkit(MavenPublication) { publication ->
                artifactId "kromatic-bukkit-api-$version"
                project.shadow.component(publication)
            }
        }
}

Are you sure you apply the GNPP plugin only in the root project?

The plugin and nexus repository was applied to the root project, the publishing has been applied to the sub modules (A total of 3)
It is a regular project using JDK 11 and Gradle 7.0

I'm somehow puzzled.
Is it the only occurrence of kromaticsnapshots in your build configuration?

That's correct. It's my first time using this plugin, I have tried also to change the repository name to something else then "kromaticsnapshots" but no luck.
The publications also have different names in each project, so that can't be the error too

I don't know what is the problem caused by. If that project is open source, please give me a link to it. If not, please remove all the code and dependencies to upload somewhere a minimal working example project that reproduces the problem. Then, I will be able to debug it to get know why it happens.
Btw, you might also check with Gradle 6.8.3, but we have a basic regression tests with Gradle 7.0, so it shouldn't be a problem as well.

Unfortunately the project is private but I was able to remove all the code and dependencies.
Here is the repository: https://github.com/mluizaa00/debug-nexus-plugin
Please let me know what you found debugging it, thank you for the help!

Thanks. And everything is clear now. That error was caused by the facts that the nexusPublishing {} block was placed inside the allprojects {} block. That triggers a try to create the aforementioned kromaticsnapshots multiple times. However, as distinct from publishing itself, the staging repository where the artifacts are created (and later released) is shared for all the modules and has to be defined once (in the root project for now). To solve it, simply move nexusPublishing {} outside.
It gave it a try locally and after the previous change it will fails with:

Type 'InitializeNexusStagingRepository' property 'repository.nexusUrl' doesn't have a configured value.

That value is required. Even more, if you don't plan to release the project (just publish snapshots) our plugin is not needed. Otherwise, please set also nexusUrl, which should not be used for the snapshot versions.
Two more remarks. As the root project does not contain any code allprojects {} could be replaced with subprojects {}. Only group and version should be kept in the root project (as our plugin looks there for those values).
Please let me know, if it works for you.

Thank you! That worked, but now my CI at Gitlab is having a error related to credentials:
A problem was found with the configuration of task ':api-bukkit:publishBukkitPublicationToKromaticRepository' (type 'PublishToMavenRepository').
  - Type 'PublishToMavenRepository' property 'credentials.username' doesn't have a configured value.

    Reason: This property isn't marked as optional and no value has been configured.

    Possible solutions:
      1. Assign a value to 'credentials.username'.
      2. Mark property 'credentials.username' as optional.

The credentials username is currently a System environment variable already set in the machine that is executing the Gitlab, so that value isn't null
Also just to let you know, the CI executed the same thing but in Maven (We transferred everything from Maven to Gradle) so again it isn't possible the credentials doesn't exists

I have the same error, but with:
set -x KROMATIC_REPO_CI_USER x
./gradlew publish

it works (well, it fails with 401 Unauthorized, which is expect). Therefore, I would check if the values are in fact set in GitLab (maybe you run it as a merge request or from a not-protected branch)?

Thank you very much! It finally worked because of your assist.
I will be using this plugin for sure in the future
As help is no longer needed, this issue will be closed

same issue

Thank you! That worked, but now my CI at Gitlab is having a error related to credentials:
A problem was found with the configuration of task ':api-bukkit:publishBukkitPublicationToKromaticRepository' (type 'PublishToMavenRepository').
  - Type 'PublishToMavenRepository' property 'credentials.username' doesn't have a configured value.

    Reason: This property isn't marked as optional and no value has been configured.

    Possible solutions:
      1. Assign a value to 'credentials.username'.
      2. Mark property 'credentials.username' as optional.

The credentials username is currently a System environment variable already set in the machine that is executing the Gitlab, so that value isn't null
Also just to let you know, the CI executed the same thing but in Maven (We transferred everything from Maven to Gradle) so again it isn't possible the credentials doesn't exists

same issue, any solution?


Implement hash truncation for DSA signatures.

The behavior is required by FIPS 186-4 and 4 and RFC 4880.
Fixes https://github.com/google/end-to-end/issues/345.

LGTMed by Thai


Prism misvalidate boolean query params

Context
It fails request against our Prism mock server during Functional test suite running.
Current Behavior
Considering the following OpenAPI spec:
paths:
  /api/endpoint:
    post:
      // …
      parameters:
        - name: fooBoolParam
          in: query
          required: false
          schema:
            type: boolean
            default: true

Calling a URL like /api/endpoint?fooBoolParam=1 leads to a Prism failure
[5:34:16 PM] › [HTTP SERVER] post /api/endpoint ℹ  info      Request received
[5:34:16 PM] ›     [NEGOTIATOR] ℹ  info      Request contains an accept header: application/json
[5:34:16 PM] ›     [VALIDATOR] ⚠  warning   Request did not pass the validation rules
[5:34:16 PM] ›     [VALIDATOR] ✖  error     Request query parameter fooboolparam must be boolean
[5:34:16 PM] ›     [NEGOTIATOR] ✔  success   Found response 400. I'll try with it.
[5:34:16 PM] ›     [NEGOTIATOR] ✔  success   The response 400 has an example. I'll keep going with this one
[5:34:16 PM] ›     [NEGOTIATOR] ℹ  info      Adding "Deprecation" header since operation is deprecated
[5:34:16 PM] ›     [NEGOTIATOR] ✔  success   Responding with the requested status code 400
[5:34:16 PM] ›     [NEGOTIATOR] ℹ  info      > Responding with "400"
[5:34:16 PM] ›     [VALIDATOR] ✖  error     Violation: request.query.fooboolparam Request query parameter fooboolparam must be boolean

Expected Behavior
Prism should validate boolean like format in query string like 0 or 1.
Possible Workaround/Solution
Nothing, I had to change the type of my parameter in the OpenAPI spec to mixed to bypass this bug.
Steps to Reproduce

Run Prism with the small extract of OpenAPI spec previously provided
Call the URL
Enjoy the error

Environment

Version used: 4.10.3 (tested also in 5.5.3)

@welcoMattic according to OpenAPI spec the evaluation of the boolean type is left up to Json Schema since it doesn't have a corresponding format:

Types that are not accompanied by a format property follow the type definition in the JSON Schema.

And JSON Schema does not support 1s and 0s as acceptable boolean values:

The boolean type matches only two special values: true and false. Note that values that evaluate to true or false, such as 1 and 0, are not accepted by the schema.


Abstracting sgRNA generation functions, added additional checks for hdf files.

Since the two functions that generate sgrnas share similar codes, I'm making an effort to try and see what can be abstracted and not repeated.
Also, I ran into an issue while testing. It seems like gen_sgRNAs.py only uses the gens file to check for homozygosity, and the targ files are used to generate guides. But the script didn't inforce that the files needed to be from the same samples. So I added those checks.
I'm going to keep working on this for the rest of the week.

Added additional files for handing Cas objects, which will hold the name, PAM, and primeness info. It can also return PAMs as human-readable text or as a regex expression.


Polytopes refactor

FieldApplications are now lazily constructed, and the trackball models are part of them.  In addition, the legacy commands available from the main menu are now restored, and available via the FieldApplications and SymmetryPerspectives.
There are also some bug fixes, to H4 polytopes and LinearMapTool.

All looks good to me. So far, it's working well with my new parameterized fields branch. I'd say go ahead and commit it.


Overriding a runnable via ID

This idea came from @pietrovismara. An ID is currently only used for debug, but it also allows an alias for overriding.
This would be useful for mocking during tests, but also for ecosystem integration where you can override a runnable that is loaded by another library.

Interestingly enough, I was debating changing the underlying dag to have nodes identified by an ID instead of the function itself.  That change would allow ID based overriding.