[#2557] improvement(core): Reuse Isolated class loader for catalog alter operation

What changes were proposed in this pull request?
Reuse isolated classloader instances for catalog alter operations.
Why are the changes needed?
If we repeat the alter catalogs, there will be lots of isolated classloaders, it's no need to create a new isolated classloader instance for altering operations, so we'd better reused the old one.
Fix: #2557
Does this PR introduce any user-facing change?
N/A
How was this patch tested?
Test locally and UTs

@FANNG1 @jerryshao
Could you help to take a look?

what tests did you do locally to make sure it works?

what tests did you do locally to make sure it works?

Repeatedly create/alter/drop the catalogs with multiple threads and check the following things.

All requests are normal.
Check the number of classloaders as followings

what tests did you do locally to make sure it works?

Repeatedly create/alter/drop the catalogs with multiple threads and check the following things.

All requests are normal.
Check the number of classloaders as followings

the former leaked resources are not increased any more?

what tests did you do locally to make sure it works?

Repeatedly create/alter/drop the catalogs with multiple threads and check the following things.

All requests are normal.
Check the number of classloaders as followings

the former leaked resources are not increased any more?

The previous resource leak was that each class loader had one daemon thread and couldn't be closed. Reducing the number of class loaders will resolve it by the way.

LGTM

Do you still need to fix jdbc-mysql threading issue after this PR?
Also, is there any potential UGI issue? @qqqttt123 please also take a look.

Do you still need to fix jdbc-mysql threading issue after this PR?
Also, is there any potential UGI issue? @qqqttt123 please also take a look.

I have verified that Full GC like System.gc() can assure that the classloader can be removed as expected, but it seems that call System.gc() explicitly is not very elegant and advised in Java.

I have verified that Full GC like System.gc() can assure that the classloader can be removed as expected, but it seems that calling System.gc() explicitly is not very elegant and advised in Java.

Do you mean that previously, it can also be GCed, so it actually doesn't have leakage problem?

I have verified that Full GC like System.gc() can assure that the classloader can be removed as expected, but it seems that calling System.gc() explicitly is not very elegant and advised in Java.

Do you mean that previously it can also be GCed, so it actually doesn't have leakage problem?

Not exactly, Full GC can resolve cases where the classloader has not loaded any classes. As long as the class loader loads a catalog-specific class (such as loading a table or something similar), GC will not function as expected, so we still need to fix it.

This PR can be delayed by #2707. From current tests, it can resolve the classloader and resource leaks.

Can we refactor the code to only reuse the catalog for "alter" scenario?

Can we refactor the code to only reuse the catalog for "alter" scenario?

Sure, this has started and hasn't finished yet.

@jerryshao
This PR is ready for review.

The current change seems a little broken, can we reuse the CatalogWrapper and just update the CatalogEntity? Or, is there a simple way to handle this?

The current change seems a little broken, can we reuse the CatalogWrapper and just update the CatalogEntity? Or, is there a simple way to handle this, like updateCatalogWrapper?

The issue lies in the following code logic:
      // We do not change the name of the catalog, things are relatively easy to handle.
      if (updatedCatalog.nameIdentifier().equals(ident)) {
        catalogWrapper.setCatalog(updatedCatalog);
        return catalogWrapper;
      } else {

     // Name changed, we need first validate the cache, as there is a close handler in the caffeine cache, it would close the
     // wrapper and classloader automatically if we do not do anything
        catalogWrapper.setShouldCloseClassLoader(false);
        catalogCache.invalidate(ident);

        // At the same time, we need to handle the case: if the following process fails halfway, we need to close the classloader manually as we have not closed it the cache close handler.
        CatalogWrapper updateCatalogWrapper =
            catalogCache.get(
                updatedCatalog.nameIdentifier(),
                id -> createCatalogWrapper(updatedCatalog, catalogWrapper.classLoader));
        return updateCatalogWrapper.catalog;
      }

Do you have any good suggestions?

The current change seems a little broken, can we reuse the CatalogWrapper and just update the CatalogEntity? Or, is there a simple way to handle this, like updateCatalogWrapper?

The issue lies in the following code logic:
      // We do not change the name of the catalog, things are relatively easy to handle.
      if (updatedCatalog.nameIdentifier().equals(ident)) {
        catalogWrapper.setCatalog(updatedCatalog);
        return catalogWrapper;
      } else {

     // Name changed, we need first validate the cache, as there is a close handler in the caffeine cache, it would close the
     // wrapper and classloader automatically if we do not do anything
        catalogWrapper.setShouldCloseClassLoader(false);
        catalogCache.invalidate(ident);

        // At the same time, we need to handle the case: if the following process fails halfway, we need to close the classloader manually as we have not closed it the cache close handler.
        CatalogWrapper updateCatalogWrapper =
            catalogCache.get(
                updatedCatalog.nameIdentifier(),
                id -> createCatalogWrapper(updatedCatalog, catalogWrapper.classLoader));
        return updateCatalogWrapper.catalog;
      }

Do you have any good suggestions? @jerryshao

Let me think about it.

@jerryshao
Code updated.

After rethinking, I'm hesitatant to do this classloader reuse:

The classloader reuse for alteration makes the code a little broken, and I cannot figure out a clear way to solve this.
Classloader reuse for alteration may have some side effects, for example, if we modify some properties that require rebooting and refreshing the classloader, with this we cannot support it.
Furthermore, sharing classloader between catalogs is dangerous, because some static variables that were created based on logic A will be used by another catalog that are based on logic B. Since we reused the classloader, so we will never reinitiate it and make the logic wrong.

So overall, I'm hesitant to make this change, what do you think? @yuqi1129

I also believe the current implementation is not so elegant to be merged, Think twice, as we have almost solved the GC problem with the class loader. I think we can postpone it and reopen it when we have a better choice.  After all, it's not a comprehensive solution for class loader reuse.


Consolidate SelectList (tentative name) and SelectField (and possibly MultiSelect/Menu/Field)

SelectList vs SelectField

TextField

Doesn't show chevron
Enabled autoFocus (at least for QuickSearch)

Removed Menu

Resolved via https://github.com/techniq/svelte-ux/pull/127


rewrite-options: don't turn on CoreFilters just because of query params

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1190

(test is in the mod_pagespeed repo)

LGTM

I believe all comments are addressed now?


MK80 Refinements

About the PR
Increases the MK80s baseFirerate from 2.0 > 2.5
Decreases the burst fireDelay from 0.1665 > 0.1225
Increases the shotsPerBurst from 3 > 5
Decreases maxScatterModifier from 10 > 7
Adds a "structural" damage component of 20
Adds a unique (explosive) soundCollection for the purposes of displaying the properties of the squash-head ammunition.
All in all - increases the gun's firerate slightly when used alongside another weapon, and increases the effectiveness of the MK80 far more when used alone with full burst capabilities. It also hopefully makes it sound more satisfying in active combat.
Why / Balance
There's a discrepancy between the VP78 in CM13 and its equivalent in RMC. Colloquially the VP78 is understood as the best sidearm comparative to the other options, or the sidearm most capable of matching the damage output of a primary weapon. In RMC, however, the weapon is woefully underused and generally overlooked considering it's a restricted weapon that shares no ammunition type with anything else. This means that it is both incredibly hard to come by without investment, and requires constant financing in order to be used on the field. The only other gun available that shares this problem, the SU-16, makes up for its relative faults through full IFF capability. Non-restricted sidearms like the M77 and M44 have strong enough damage output (and AP) to compete with the MK80 outright despite not being restricted in terms of ammo, and are more modular than the MK80 as well.
So why is the weapon considered far better in CM13? The most likely answer is that the VP78 is best used with burst fire to take advantage of its high base AP and damage output per bullet. Due to the way Byond works, burst damage output is generally far higher, primarily due to its accuracy as a single entity, whereas in RMC burst fire usually results in higher DPS in exchange for a total loss of accuracy. It doesn't help that for most marines the best way to make use of the increased burst-fire DPS the VP78 has to be used solo - pitting its viability against the primary, non-restricted weapons. All in all, a resounding "why buy?".
The changes to the fire-rate and burst-fire-rate, audio rework, alongside the (unique) addition of structural damage will likely help make the gun stand out as both a primary and secondary option - bring it closer to parity with CM13 - and hopefully make it a worthy addition for specialists alongside the SU-16.
Technical details
In mk80_pistol.yml:
shotsPerBurst: 3 >> 5

baseFireRate: 2.0 >> 2.5

fireDelay: 0.1665 >> 0.1225
maxScatterModifier: 10 >> 7

Added:
Damage:
    Types:
        Structural: 20

In Audio\Effects:
Added:
RMC_MK80-Xenohit1.ogg
RMC_MK80-Xenohit2.ogg
RMC_MK80-Xenohit3.ogg
In gun_impacts.yml
Added:

type: soundCollection
id: ExplosiveBulletImpact
files:

"/Audio/Effects/explosion_small3.ogg"
"/Audio/Effects/explosion_small1.ogg"
"/Audio/Effects/RMC_MK80-Xenohit1.ogg"
"/Audio/Effects/RMC_MK80-Xenohit3.ogg"
"/Audio/Weapons/Guns/Hits/bullet_hit.ogg"
"/Audio/Weapons/Guns/Hits/bullet_hit.ogg"
"/Audio/Weapons/Guns/Hits/bullet_hit.ogg"
"/Audio/Weapons/Guns/Hits/bullet_hit.ogg"
"/Audio/Weapons/Guns/Hits/bullet_hit.ogg"

Media

https://github.com/user-attachments/assets/b6866b1a-e7aa-443b-8b4d-a4e68225f303
Requirements

[X] I have read and am following the Pull Request and Changelog Guidelines.
[X] I have added media to this PR or it does not require an ingame showcase.

Breaking changes
N/A
This is my first pull-request for RMC-14. If there's anything wrong with this, please let me know.
Changelog

:cl:

tweak: Increased the fire-rate and burst fire-rate of the MK80
added: Added unique hit-sounds for the MK80
added: Added structural damage component for the MK80, it will now deal (slightly) increased damage towards walls, hive walls and specific emplacements.

P.S. I am not responsible for the submodule changes - I didn't stage the file changes, and the code is not mine. Unsure why it shows up under my PR - outdated repo?


Document CHAINERX_CUDNN_USE_CUPY

Fixes #7573

I do think CHAINERX_CUDNN_USE_CUPY is not the "authentic" method that anyone can use. Do you agree with that?

Well if this is true then I wouldn't document CHAINERX_CUDNN_USE_CUPY since it's an internal feature.
I would think it's a feature that I can use as usual flags when a feature is documented.
Or it should be documented to a place where only well trained ChainerX developer read(for example like: https://github.com/chainer/chainer/blob/339684a8e6acec102c0c6b885fdc42a76f2009a3/chainerx_cc/README.md )
(About relation between CUDNN_ROOT_DIR:)
To avoid pitfalls of the relation with it, I feel it should be documented.
It may cause some troublesome path error.

Well if this is true then I wouldn't document CHAINERX_CUDNN_USE_CUPY since it's an internal feature.

In my understanding, it's not an internal feature either. It's not an "authentic", but useful short-hand feature that a portion of ordinary users can benefit from.
But assuming we are going to provide ChainerX binary distribution soon, we might as well make it an internal feature as you suggest.

To avoid pitfalls of the relation with it, I feel it should be documented.

Current change includes a mention about it.

Writing this in CUDA support it's more optimal to use CHAINERX_CUDNN_USE_CUPY since it installs cupy too.
Thanks @hvy for pointing this out!

Fixed as suggested.

Fixed installation examples.
Please take a look.

Jenkins test this please

Jenkins CI test (for commit 71cfcd0d09db00035f18fc5a707612c57763cb4a, target branch master) failed with status FAILURE.

Document generation isn't failing in jenkins so seems to be fine

pfn CI, test this please!

Jenkins CI test (for commit 71cfcd0d09db00035f18fc5a707612c57763cb4a, target branch master) succeeded!


Always out of memory

@yunjey As the training continues, the memory it needs increased ,and stopped when   it report the runtimeError said that out of memory.
I don't know why the memory increased all the time as training proceed, and I wonder if i modify the code to use muitl-gpu , whether  i will also  receive the same error or not?
btw, my gpu memory is 12G
thanks~

thanks, i have solved this problem by update my code according to your advice ~


All packages with sources hosted by GitHub currently do not download due to changes in GitHub's archives

Host Environment

OS: Ubuntu 22.04 as used by GitHub actions
Compiler: GCC, clang, icpx

To Reproduce
I run this inside my GitHub action:
vcpkg install alpaka fmt tinyobjloader boost-mp11 boost-atomic boost-smart-ptr boost-functional boost-container boost-iostreams[core] catch2 xsimd
Failure logs
Excerpted from: https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764
Computing installation plan...
The following packages will be built and installed:
    alpaka[core]:x64-linux -> 0.9.0#1
  * boost-align[core]:x64-linux -> 1.81.0
  * boost-array[core]:x64-linux -> 1.81.0
  * boost-assert[core]:x64-linux -> 1.81.0
    boost-atomic[core]:x64-linux -> 1.81.0
  * boost-bind[core]:x64-linux -> 1.81.0
  * boost-build[core]:x64-linux -> 1.81.0
  * boost-concept-check[core]:x64-linux -> 1.81.0
  * boost-config[core]:x64-linux -> 1.81.0
    boost-container[core]:x64-linux -> 1.81.0
  * boost-container-hash[core]:x64-linux -> 1.81.0
  * boost-conversion[core]:x64-linux -> 1.81.0
  * boost-core[core]:x64-linux -> 1.81.0
  * boost-describe[core]:x64-linux -> 1.81.0
  * boost-detail[core]:x64-linux -> 1.81.0
  * boost-dynamic-bitset[core]:x64-linux -> 1.81.0
  * boost-function[core]:x64-linux -> 1.81.0
  * boost-function-types[core]:x64-linux -> 1.81.0
    boost-functional[core]:x64-linux -> 1.81.0
  * boost-fusion[core]:x64-linux -> 1.81.0
  * boost-integer[core]:x64-linux -> 1.81.0
  * boost-intrusive[core]:x64-linux -> 1.81.0
  * boost-io[core]:x64-linux -> 1.81.0
    boost-iostreams[core]:x64-linux -> 1.81.0
  * boost-iterator[core]:x64-linux -> 1.81.0
  * boost-modular-build-helper[core]:x64-linux -> 1.81.0
  * boost-move[core]:x64-linux -> 1.81.0
    boost-mp11[core]:x64-linux -> 1.81.0
  * boost-mpl[core]:x64-linux -> 1.81.0
  * boost-numeric-conversion[core]:x64-linux -> 1.81.0
  * boost-optional[core]:x64-linux -> 1.81.0
  * boost-predef[core]:x64-linux -> 1.81.0
  * boost-preprocessor[core]:x64-linux -> 1.81.0
  * boost-random[core]:x64-linux -> 1.81.0
  * boost-range[core]:x64-linux -> 1.81.0
  * boost-regex[core]:x64-linux -> 1.81.0
    boost-smart-ptr[core]:x64-linux -> 1.81.0
-- Performing post-build validation
Stored binary cache: "/home/runner/.cache/vcpkg/archives/14/14a7e106a0e9aee1d43b39d00c62a19db02aa62430357d6a214483e0610b9e70.zip"
Elapsed time to handle boost-config:x64-linux: 417.8 ms
Installing 4/56 boost-assert:x64-linux...
Building boost-assert[core]:x64-linux...
-- Downloading https://github.com/boostorg/assert/archive/boost-1.81.0.tar.gz -> boostorg-assert-boost-1.81.0.tar.gz...
[DEBUG] To include the environment variables in debug output, pass --debug-env
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] Detected CI environment: GitHub_Actions
[DEBUG] 1000: popen( curl --fail -L https://github.com/boostorg/assert/archive/boost-1.81.0.tar.gz --create-dirs --output /usr/local/share/vcpkg/downloads/boostorg-assert-boost-1.81.0.tar.gz.1750.part 2>&1)
[DEBUG] 1000: cmd_execute_and_stream_data() returned 0 after   206285 us
error: Failed to download from mirror set
error: File does not have the expected hash:
url: https://github.com/boostorg/assert/archive/boost-1.81.0.tar.gz
File: /usr/local/share/vcpkg/downloads/boostorg-assert-boost-1.81.0.tar.gz.1750.part
Expected hash: c443b61a9ad29d4c9707aaaa997300a959fdef2d360aec9ee221abc[12](https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764#step:8:13)cd8446b3bc969e6c282ba778bee7d6fb54cc02655f15e01eacf1ad87262534dc4cc32bf
Actual hash: cb031c35d3f4342e862b46f309874bc28773cc95657841b772ff7b52dd8495c92eb496273e80ee50155b418e0[13](https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764#step:8:14)a3cc62b352bdd6d90274f49206c89089c1aa6
[DEBUG] /mnt/vss/_work/1/s/src/vcpkg/base/downloads.cpp(956):
[DEBUG] Time in subprocesses: 206285 us
[DEBUG] Time in parsing JSON: 3 us
[DEBUG] Time in JSON reader: 0 us
[DEBUG] Time in filesystem: 100 us
[DEBUG] Time in loading ports: 0 us
[DEBUG] Exiting after 207.4 ms (207047 us)

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:32 (message):

      Failed to download file with error: 1
      If you use a proxy, please check your proxy setting. Possible causes are:

      1. You are actually using an HTTP proxy, but setting HTTPS_PROXY variable
         to `https://address:port`. This is not correct, because `https://` prefix
         claims the proxy is an HTTPS proxy, while your proxy (v2ray, shadowsocksr
         , etc..) is an HTTP proxy. Try setting `http://address:port` to both
         HTTP_PROXY and HTTPS_PROXY instead.

      2. You are using Fiddler. Currently a bug (https://github.com/microsoft/vcpkg/issues/17752)
         will set HTTPS_PROXY to `https://fiddler_address:port` which lead to problem 1 above.
         Workaround is open Windows 10 Settings App, and search for Proxy Configuration page,
         Change `http=address:port;https=address:port` to `address`, and fill the port number.

      3. Your proxy's remote server is out of service.

      In future vcpkg releases, if you are using Windows, you no longer need to set
      HTTP(S)_PROXY environment variables. Vcpkg will simply apply Windows IE Proxy
      Settings set by your proxy software. See (https://github.com/microsoft/vcpkg-tool/pull/49)
      and (https://github.com/microsoft/vcpkg-tool/pull/77)

      Otherwise, please submit an issue at https://github.com/Microsoft/vcpkg/issues

Call Stack (most recent call first):
  scripts/cmake/vcpkg_download_distfile.cmake:273 (z_vcpkg_download_distfile_show_proxy_and_fail)
  scripts/cmake/vcpkg_from_github.cmake:106 (vcpkg_download_distfile)
  ports/boost-assert/portfile.cmake:3 (vcpkg_from_github)
  scripts/ports.cmake:[14](https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764#step:8:15)7 (include)

error: building boost-assert:x64-linux failed with: BUILD_FAILED
error: Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
    https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+boost-assert
You can submit a new issue at:
    https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[boost-assert]+Build+error
Include '[boost-assert] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above.
    vcpkg-tool version: [20](https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764#step:8:21)[22](https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764#step:8:23)-12-14-7ae0d8527fb488fde10a89c2813802dc9b03b6f9
    vcpkg-scripts version: e69dd972b 20[23](https://github.com/alpaka-group/llama/actions/runs/4046805344/jobs/6961596764#step:8:24)-01-20 (10 days ago)

Please use the prefilled template from /usr/local/share/vcpkg/installed/vcpkg/issue_body.md when reporting your issue.
Error: Process completed with exit code 1.

Unfortunately, my GitHub action does not expose the /usr/local/share/vcpkg/installed/vcpkg/issue_body.md, so I cannot attach it :/
Additional context
This just started failing about an hour ago. Earlier GitHub actions were completed successfully.

I have a similar problem in my local computer with a different package, also just started today while everything was working 24 hours ago:
OS: Windows 10
vcpkg-tool version: 2023-01-24-8a88d63f241d391772fbde69af9cab96c3c64c75
vcpkg-scripts version: 91dd61bd4 2023-01-27 (3 days ago)
Just ran git pull and vcpkg update. Running vcpkg install glfw3 reports:
Downloading https://github.com/glfw/glfw/archive/7482de6071d21db77a7236155da44c172a7f6c9e.tar.gz
error: Failed to download from mirror set
error: File does not have the expected hash:
url: https://github.com/glfw/glfw/archive/7482de6071d21db77a7236155da44c172a7f6c9e.tar.gz
File: C:\dev\vcpkg\downloads\glfw-glfw-7482de6071d21db77a7236155da44c172a7f6c9e.tar.gz.20140.part
Expected hash: ec45b620338cf36a8dbdf7aaf54d7c3a49a1be4ae1a1ef95f1531094fec670870713969bbc23476769d374c7a71d93f6540ab64c46fb5f66f4402bb2d15c7d87
Actual hash: f2541cc801adb9a4d6ea6932c234953240d902212bd0a2d3a6f8d95c1dbff3adbd8e8e757ae991a544d979000c351c209d5e4054f71427d1a0490dbbfe0763fb

If I download the URL https://github.com/glfw/glfw/archive/7482de6071d21db77a7236155da44c172a7f6c9e.tar.gz with Chrome independently of vcpkg, the sha512sum of that file matches the "Actual hash" reported above.
Same thing also with vcpkg install openvr, but vcpkg install nlohmann-json seems to have worked.

Same issue with several boost packages - windows and ubuntu tested so far
Same commit was compiling fine on Friday (using Github Actions)

I've just opened a support ticket with GitHub. I suspect this is an issue on their end.

I've just opened a support ticket with GitHub. I suspect this is an issue on their end.

They changed something about their gzipper

I've just opened a support ticket with GitHub. I suspect this is an issue on their end.

Do you have a link for this? They have been a bit vocal in the past that "if you expect the hash to not change you need to get the bit from a download/artifact rather than from a source archive" so they may say "too bad" to us.

I can't link to it as it's some sort of "personal ticket" or something associated with my account. Will let you know what they say, though.
Surprising that they are okay with that happening. I have been using the same hash for the same URL for a year and a half or so and not had any issues. Also, since the archive is listed as a release artifact, you'd think the same rules would apply there...

our production builds are broken due to this issue

Same issue is happening here. Multiple packages are failing. Is it possible to forcibly change the hash keys as a temp fix?

Same issue is happening here. Multiple packages are failing. Is it possible to forcibly change the hash keys as a temp fix?

Technically, you can always just hard-edit the portfiles themselves to have the proper matching hashes.

Note that the /archive/ API endpoint has always been git archive under the hood. I don't believe that git archive has ever had reproducible guarantees about its behavior. Proper release entities with source archives should always be preferred where possible (please prod projects to provide them as well). Though given how hard it is to even get tags sometimes…

https://github.com/bazel-contrib/SIG-rules-authors/issues/11#issuecomment-1409373032 GitHub folks commenting here

https://github.blog/changelog/2023-01-30-git-archive-checksums-may-change/ GitHub confirmed the changes

Our pipeline (Microsoft internal repo) failed...

Our pipeline (Microsoft internal repo) failed...

If an MS internal pipeline is broken by this you need to do what it says in https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/secure-supply-chain/project-artemis/vcpkg#asset-caching -- but now that things are broken it's probably too late to fix

Our pipeline (Microsoft internal repo) failed...

If an MS internal pipeline is broken by this you need to do what it says in https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/secure-supply-chain/project-artemis/vcpkg#asset-caching -- but now that things are broken it's probably too late to fix

Thanks for pointing out this... Would Github or vcpkg fix this issue?

Thanks for pointing out this... Would Github or vcpkg fix this issue?

We don't know yet. I'm preparing the change to manually update all ~1500 ports affected by this but it's probably going to take a couple days, since it would be bad to merge that only for it to be toasted by GitHub reverting the change or something if they do.

I'm preparing the change to manually update all ~1500 ports affected by this

Is it possible to use this to file issues to request stable artifacts of sources be made to avoid this no matter what github does in the future? A script that uses the GitHub API to edit releases might be handy to have for it so that it is easy for maintainers to apply it.

Would it be possible to use a hashing method that only hashes the contents of the archive, not the archive itself.
Something akin to decompressing the archive, turning into a tar files and hashing that.
That way vcpkg should be resilient to any future changes from any source, assuming they don't actually change the contents of the archives.

Is it possible to use this to file issues to request stable artifacts of sources be made to avoid this no matter what github does in the future? A script that uses the GitHub API to edit releases might be handy to have for it so that it is easy for maintainers to apply it.

It is impractical for vcpkg to host such an archive 'automatically' because that would implicate us in "distribution" rules of whatever license applies to that archive.

Would it be possible to use a hashing method that only hashes the contents of the archive, not the archive itself?

That would (1) still leave the entire world currently broken, and (2) breaks our existing asset caching protocol, where upstream caching servers are assumed to be able to validate the hashes on their own.

I see the changes are going to be reverted: https://github.com/bazel-contrib/SIG-rules-authors/issues/11#issuecomment-1409438954

Make it opt in, gradually move packages over to the new system. Eventually remove the old system.
I'm unfamiliar with asset caching, how would this break it? From skimming the docs it seems the SHA512 hash is used as a filename for the blob, so could it just use the invariant hash instead? (As far as I can tell the docs says its experimental and still in flux, so changing it might not be impossible.)

Granted, I'm just spitballing, and I probably wouldn't be able to implement the changes needed, but relying on the hashes of an archive seems like peak Hyrum's law to me.

Make it opt in, gradually move packages over to the new system. Eventually remove the old system.

That's no help because this isn't about the contract between ports and vcpkg, this is about the contract between vcpkg and our customers.

I'm unfamiliar with asset caching, how would this break it? From skimming the docs it seems the SHA512 hash is used as a filename for the blob, so could it just use the invariant hash instead?

Asset cache operators get to independently download the bits and check the hash themselves. Several customers using that believe that it is important for being able to provide audit trails for all sources used in producing a product, for example.

relying on the hashes of an archive seems like peak Hyrum's law to me.

I disagree. They list that path on release pages like any other release asset and as a result encouraged treating that like release assets. Release assets need to have auditable SHAs. vcpkg is far Far FAR from alone here, conan, spack, homebrew, bazel, etc. were also broken.
Of course, whether the assumption is reasonable or not, we are going to have to respond to this incident by making product changes to be more resilient in the future to this kind of change. Such changes aren't, however, a short term solution.

Update on this issue - GitHub announced they are reverting this change for now: https://github.blog/changelog/2023-01-30-git-archive-checksums-may-change/
This should get us back into a working state with all the ports. We will monitor the situation and determine any further changes that need to be made to vcpkg to ensure stability going forward.

So when shall this be in effect?

I cannot find anywhere else on the internet that anyone else appears to be having this issue.
We are trying to build AliceVisio from source; this issue crops up both with something they've spliced into CMake to download from there, and using vcpkg for dependencies too.

So when shall this be in effect?

We are waiting to see just as much as everyone else.

It is impractical for vcpkg to host such an archive 'automatically' because that would implicate us in "distribution" rules of whatever license applies to that archive.

I wasn't saying for vcpkg to rehost. I was asking if vcpkg (in coordination with other distros) could push for releases to contain actual archives (even if just re-uploads of the /archive/ API at the time) instead of relying on the automatic /archive/ links.

I wasn't saying for vcpkg to rehost. I was asking if vcpkg (ideally in coordination with other distros) could push for releases to contain actual archives (even if just re-uploads of the /archive/ API at the time) instead of relying on the automatic /archive/ links.

That only works if we are pointing to an actual release. In this case GitHub appears to be trying to revert the change so it should be automatically fixed.

My CI seems to work again and vcpkg seems to download all dependencies correctly. My original issue is thus resolved. However, since the title has changed to raise attention to a more general problem, I leave it up to one of the maintainers here to close the issue. Thanks for keeping everyone up-to-date!

So that people don't have to search:
Git community discussion: https://github.com/orgs/community/discussions/45830#discussioncomment-4828473
Bazel discussion: https://github.com/bazel-contrib/SIG-rules-authors/issues/11
Git mailing list to make git archive reproducible<EMAIL_ADDRESS>


WIP: BC-6 # bm bmp pull downloads config and code

uses previously-designed approach for splitting code out of JSON files into separate files: https://github.com/blinkmobile/json-as-files.js

just replaces files at the destination, no protection or warning so far

Should be rebased and merged after BC-5 (#2) goes through.

git rebased, etc.

@simonmarklar okay, I've stopped those functions from swallowing errors. I've done this in a slightly repetitive fashion, but I've deduped these functions in BC-7 (#5), so I didn't want to keep that future git rebase simple. :)

cool, LGTM :+1:


Update to manifestival 0.5.0

This is mostly to make Apply more robust to better handle strategic
patch creation involving certain merge keys: manifestival/manifestival#37
It also makes the None predicate a little less verbose

/lgtm


Update SpringMvcContract.java

When using property placeholders in the @RequestMapping annotation the ones placed in the headers attribute do not get replaced. This appears to be because the other attributes, such as the url, use the resolve() function to perform the substitution whereas the header values are parsed as-is. This change adds a call to resolve for each side of the equals in the header string so that the properties are substituted correctly.

Have you signed the Contributor License Agreement?

Hi Spencer,
Sorry, forgot to do that earlier so have just done it now.
Damian.

Merged with a test via 711f095ebc2093bce4e8f3f25ab0e71c4dcf544c


Interpolate the content of text file

With this PR, the content of text_file is interpolated the same way text is.
The change in out is quite trivial, and we've added two automated tests to cover the new expected behavior.
This new behavior may produce unexpected results when the content of the text_file includes some environment variables that are to be printed verbatim and not expanded.
We think that including variables like $BUILD_PIPELINE_NAME in the ù text_file` and expecting these to be printed verbatim in a Slack message is a very uncommon requirement. Thus, we expect the impact to be very limited.

PR-testing pipeline job is broken but the build and tests are passing ✅


[Source request] - Cure Gecko

Cure Gecko's Booru Search is a service that provides a metasearch of the most popular boorus.
It is basically a database of direct links to the images hosted on boorus.
I have been using it for a very long time and found it very convenient for casual browsing, I still often use it as an alternative to grabber.
It allows querying more than two tags at a time from danbooru and also matches your query to the words that are contained in tags as opposed to full tags, which is less precise, but I personally find it way more convenient.
It has an api, described here, that can provide results in html, json and rss formats. It also used to be able to return plaintext and html list of links, but as of now this part of an api appears to be broken.

It indeed looks like a good web-based Grabber-like website. But I'm not sure of what it would add compared to simply using Grabber on the sources individually.
You do mention a few interesting advantages:

It allows querying more than two tags at a time from danbooru
[It] matches your query to the words that are contained in tags as opposed to full tags, which is less precise

Which raises a few questions:

Does that mean that it keeps a local dump of the whole Danbooru "database"? Because if they did as Grabber does, that is to query Danbooru for every request, that would be impossible. This leads to a question of "Grabber as a web service", which allows for similar caching.
Same question, since Danbooru's API does not allow it, but keeping a local dump would.

Adding the source would be quite quick, especially given it supports JSON and RSS. The question is whether we should add such "aggregating" sources, if the added value is small.
Does it have other advantages than the two you listed? For web-based, I also see that it has a very clean and "no-bullshit" interface, but that doesn't really matter for a Grabber source.

I always thought that they just keep a database, scraping the post data from the source sites and then the images themselves are downloaded by the user's browser, but that does not appear to be the case now that I've looked into it.
It looks like there is an image server.
As for advantages, there aren't really any other than what I've already mentioned. It feels really quick as far as both search and navigation go, but that probably has to do with the interface.
If it really has it's own image server,  that could help with sankaku's throttling, but it appears to only use it when serving the webinterface, returning the source site links instead when answering the api calls, so that would probably be a bottleneck of it's own.

So, it looks like the author has a local image database, and the images that are on there get pulled from the database, with the ones that are not being pulled from the original source.


How to contact David Morrissey

Hi, I know it's the wrong place for this but how to contact you @davemorrissey ?
I have an App to put into your "hall of fame".
Other users needs to do that also maybe.

Just comment here with a link and I'll add it!

The application is on Android: Humack


Content auto update FR

Hello once again!
I'm creating an angularJS app, and I have a tooltips on dunamic scope blocks. So when angular updates scope with element (toolpits) it just adds (recreate) "title" attribute in the element.
When this happens, i have an element with class "tooltipstered" and with new "title" attribute. I don want to use any methods to update content from my app, because it is unnecessary logic in the app.
I suggest you to add auto update of content, when title has appeared in the element.
I have done it in "_showNow" method, like this:
if(self.$elProxy.attr('title') !== undefined) {

	this.Content = self.$elProxy.attr('title');

	self.$elProxy.removeAttr('title');

}

This code will really help angular users )

Hello and thanks for contributing.
I'm not comfortable with "magic" updates though. One could argue just as well that the tooltip should update even if it's open when the title attribute is changed, and that's not going to happen.
If you want to check the title attribute and update the content at each opening, you should use the functionBefore callback to do so. That should be easy and non-intrusive enough :)

Thank you! Actually I forgot about this callback, will do it :)

You're welcome !


Added a very buggy theme change options and will work on accent color once theme is working

@qw123wh I'm stuck on adding the theme options, it very buggy but it does work. you do need to leave and reopen the setting many times before it works though. I was if you could help me with this? if not I will leave it for a bit and work on other things.

added double zero to the timer
added vibration to the clock


Fix rating calculation and upload

I'd like to be able to compare the performance of noaa-apt and gnu-radio recipes against each other. To do that, image quality assessment is necessary. I looked at the analog rating protocol, enabled it in the config, but it didn't help. I've checked the code and pushed a fix in 31de664466cc1f0e606e1a8a5be3e24969e910ea. Before that fix, the rating was always None.
This now uncovered two problems:

[ ] The station calculates score of 1.0 even for poor images, e.g. the station calculated 1.0 for observation#3881.
[ ] Something was broken in the score upload to the server. Station logs say this:

receiver.py:88: Getting rating for product /home/pi/observations/2021-06-11-1904-noaa-18/product.png (rating algorithm is analog)
receiver.py:90: Product /home/pi/observations/2021-06-11-1904-noaa-18/product.png got rating 1.0

yet the server shows no rating. Need to debug it.

I've looked around a bit and I think there's a couple things that could be improved:

The notes field is currently unusable. Hardcoded to `` in one place and to ... in another. This should read a content of some file, e.g. ~svarog/notes.txt. The idea behind this is that the station maintainer could put there some info easily, e.g. running on airspy now, disabled LNA etc. This kind of things would be easy to keep up to date and there would be a written trace of the changes done over time.
The log file is not uploaded. It could be another product.
I strongly feel that we need to introduce some structured data here that would convey the parameters, such as frequency, Doppler correction, name of the recipe, rating method etc. The syntax should be extensible, so adding more parameters (e.g. all those knobs in gnu radio) could be transparent and wouldn't require server changes. If done correctly, this could easily replace the single value rating field.

I strongly feel that we need to introduce some structured data here that would convey the parameters, such as frequency, Doppler correction, name of the recipe, rating method etc. The syntax should be extensible, so adding more parameters (e.g. all those knobs in gnu radio) could be transparent and wouldn't require server changes. If done correctly, this could easily replace the single value rating field.

It may be useful to assign any key-value dictionary for observations. Data in the dictionary may contain any attributes, variety for each antenna, station, satellite, etc.
Postgres has some possibilities to handle this case. The first is JSON data type. It may contain any data and Postgres has few functions to query the JSON content. But JSON data type isn't intended for this (in my opinion). It is very good for store JSON as a single blob, but functions to operate on it are just an addition.
Probably better approach will be to use "hstore" official Postgres extensions to support key-value columns. It should be a little more efficient to query them for example for filtering.
We should analyze if key-value data should be assigned to observation (as "notes") or to a specific product.

I was thinking about JSON, but can do some experiments with hstore. I know there are queries possible that look into JSON keys.
For the time being I think it's ok to have the meta-data (or key-value or whatever you want to call it) to be associated with the whole observation. This will be useful as it is. If we need to get it per product some time later, we can always add additional meta-data for products. This may be useful later, e.g. when we'll start receiving SSTV images from ISS and it would transmit multiple images during one pass. Each image could have its own meta-data (width, height, quality, etc.).

I've updated the station code and it calculated the rating properly!
This indeed fixes the problem.
The score is unreasonably high (41%), but there's a good explanation. Gnu Radio recipe always produces an image of constant height, with the remainder part being black. The rating algorithm treats black parts are noiseless, and thus good.
That it, however, something I can fix in the recipe. The algorithm now works fine. As such, closing this.


Uncaught Exception: Attempting to call a function in a renderer window that has been closed or released. Function provid

An uncaught exception was reported.
Info:
OS: win32
Arch: ia32
GPMDP Version: 3.0.1
Time: Tue Mar 15 2016 09:51:39 GMT-0600 (Hora estándar central (México))
Error:
Attempting to call a function in a renderer window that has been closed or released. Function provid
Error: Attempting to call a function in a renderer window that has been closed or released. Function provided here: Object.<anonymous> (D:\Users\praxis03\AppData\Local\GPMDP_3\app-3.0.1\resources\app\build\inject\GPMInject\interface\mouseButtonNavigation.js:8:8.
    at BrowserWindow.callIntoRenderer (D:\Users\praxis03\AppData\Local\GPMDP_3\app-3.0.1\resources\atom.asar\browser\lib\rpc-server.js:201:19)
    at emitTwo (events.js:92:20)
    at BrowserWindow.emit (events.js:172:7)
    at EventEmitter.<anonymous> (D:\Users\praxis03\AppData\Local\GPMDP_3\app-3.0.1\resources\app\build\main\features\core\controlBar.js:10:45)
    at emitTwo (events.js:87:13)
    at EventEmitter.emit (events.js:172:7)
    at EventEmitter.<anonymous> (D:\Users\praxis03\AppData\Local\GPMDP_3\app-3.0.1\resources\atom.asar\browser\api\lib\web-contents.js:134:25)
    at emitTwo (events.js:87:13)
    at EventEmitter.emit (events.js:172:7)

This issue was created automatically inside the uncaughtException handler

Closing this automatically reported issue as a duplicate of #470


add(starters): Vite React TypeScript Tailwind 3.x

Name
Link

Vite + React + TypeScript + Tailwind 3.x starter
GitHub

GitHub Template for Vite, React + Tailwind 3.x + TypeScript.

[x] My item is in the right category
[x] My item is logically grouped below similar items
[x] My item's name and description respects the conventions of the list
[x] My item is awesome
[x] My item is in line with the Tailwind brand usage guidelines
[x] I have read and followed the contribution guidelines

Thanks!

Thanks for merging!


Cart::total() exception

Cart::total();

return an exception
Type error: Argument 2 passed to Gloudemans\Shoppingcart\Cart::Gloudemans\Shoppingcart\{closure}() must be an instance of Gloudemans\Shoppingcart\CartItem, integer given

CAn't recreate that ..

It happens after some manipulation on the cart, using ::add and ::update, then the method total() is raising this exception.

Ah alright. I'm currently developing a big store using cart Ill check if I come across but no problems atm.

I have the issue every time now.
I just add a product then try to get the cart content and have this other issue similar to the previous one:
$product = ProductModel::where("active",1)->find($item['id']);
$data = Cart::add($product, $item['qty'], ['options' => $options ]);

and getting cart total
$data = [];
$data = Cart::content();
$data['total'] = floatval(Cart::total(2));

type error: Argument 2 passed to Gloudemans\Shoppingcart\Cart::Gloudemans\Shoppingcart\{closure}() must be an instance of Gloudemans\Shoppingcart\CartItem, float given

/Users/christophevidal/Sites/oc-vodka/plugins/vodka/toolbox/vendor/gloudemans/shoppingcart/src/Cart.php
                    line 241
```

the context of this code is an API, so I added the middleware 'web' in my routes.php.

OK I found that the origin of the issue is the presence of this [tax] => 1.68 in the content of the cart;
I am not sure if this is supposed to be there or not, but it is the source of the exception.
and yes I using the tax function.


cinched, fleaworts (logical-conflict)

This pull request was generated by the 'mq' tool

/trunk merge


[webpack 4] Build failing due to removed UglifyJsPlugin

Environment

Windows 10
npm 6.0
node 10.1.0
webpack 4.5.0
webpack-cli 2.0.14
serviceworker-webpack-plugin 0.2.3

Steps to reproduce

Install webpack 4
Include the default config of the serviceworker plugin as provided by the documentation
Start a webpack build

Actual result
The build fails with the following error:
Error: webpack.optimize.UglifyJsPlugin has been removed, please use config.optimization.minimize instead.
Expected result
The build does not fail due to removed plugins.
Potential solution
Since webpack 4 uses the mode option in conjunction with optimization.minimize, I would suggest to either minify the service-worker assets based on either the mode being set to production or make the minification configurable (minify: true).

Nvm... I did not look into the resolved issues.


Kick support

Checklist

[x] I'm reporting a new site support request
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that none of provided URLs violate any copyrights
[x] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

Single video: https://kick.com/video/82a3c11d-7a17-4747-aecb-2e61413eb11f

Description
No login credentials needed, the website is basically a twitch clone

In the page is JS variable const Zippy whose value defines the site:
{
  'url': 'https://kick.com',
  'port': null,
  'defaults': {
  },
  /* --- snipped lots of APIs --- */
    'api.search': {
      'uri': 'api/search',
      'methods': [
        'GET',
        'HEAD'
      ]
    },
  /* --- snipped lots of APIs --- */
    'video.show': {
      'uri': 'api/v1/video/{video}',
      'methods': [
        'GET',
        'HEAD'
      ],
      'bindings': {
        'video': 'uuid'
      }
    },
    'video.delete': {
      'uri': 'api/v1/video/{video}',
      'methods': [
        'DELETE'
      ],
      'bindings': {
        'video': 'uuid'
      }
    },
  /* --- snipped lots of APIs --- */
  }
}

So let's look at https://kick.com/api/v1/video/82a3c11d-7a17-4747-aecb-2e61413eb11f:
{
  'id': 28440,
  'live_stream_id': 31814,
  'slug': null,
  'thumb': null,
  's3': null,
  'trading_platform_id': null,
  'created_at': '2022-12-17T15:02:05.000000Z',
  'updated_at': '2022-12-21T11:26:30.000000Z',
  'uuid': '82a3c11d-7a17-4747-aecb-2e61413eb11f',
  'views': 742,
  'source': 'https://stream.kick.com/ivs/v1/196233775518/xJf2jIQZk382/2022/12/17/12/30/4uV5fWgp8QfV/media/hls/master.m3u8',
  'livestream': {
    'id': 31814,
    'slug': 'b17d6-weekly-stake-stream',
    'channel_id': 22,
    'created_at': '2022-12-17 12:33:14',
    'session_title': 'Weekly Stake Stream',
    'is_live': false,
    'source': 'obs',
    'twitch_channel': null,
    'duration': 9007000,
    'thumbnail': 'https://kick-prod-videos.s3.us-west-2.amazonaws.com/thumbnails/livestream/31814/thumb145/video_thumbnail/thumb145.jpg?X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAS3MDRZGPPQDUDQXQ%2F20221221%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221221T112631Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Signature=876d88673a61419800f0df4951ee3b121e351e80ed6dad2b7cee3d99b81fdfdd',
    'channel': {
      'id': 22,
      'user_id': 22,
      'slug': 'eddie',
      'playback_url': 'https://fa723fc1b171.us-west-2.playback.live-video.net/api/video/v1/us-west-2.196233775518.channel.xJf2jIQZk382.m3u8',
      'name_updated_at': null,
      'followersCount': 11447,
      'user': {
        'profilepic': 'https://d2egosedh0nm8l.cloudfront.net/images/user/22/profile_image/358e5d87-be7f-4b39-aed3-aa3d9df8203a',
        'bio': 'Eddie from Stake. Streaming every Saturday at 12:30pm GMT with Steve & Mikey. Whole heap of giveaways to enter and bonuses to be won. Happily answering any questions people might have about Stake!',
        'twitter': 'StakeEddie',
        'facebook': null,
        'instagram': null,
        'youtube': null,
        'discord': 'PrimeEdd',
        'tiktok': null,
        'username': 'Eddie'
      },
      'verified': {
        'id': 26,
        'channel_id': 22,
        'created_at': '2022-12-12T16:48:07.000000Z',
        'updated_at': '2022-12-12T16:48:07.000000Z'
      }
    },
    'categories': [
      {
        'id': 28,
        'category_id': 4,
        'name': 'Slots & Casino',
        'slug': 'slots',
        'tags': [
          'Gambling'
        ],
        'description': null,
        'category': {
          'id': 4,
          'name': 'Gambling',
          'slug': 'gambling',
          'icon': '🎰'
        }
      }
    ]
  }
}

This is all that's needed for basic video extraction with metadata.

hey is this issue still available, if yes can i work on this?

Please review the linked PR if you wish.

Single video link in OP has been removed, try this one https://kick.com/video/7237f151-70e4-439c-8306-2a7b0b0cc366

Video link above has also been removed, tried using this one: https://kick.com/video/e466d965-b67d-471d-bed5-e3d4f19a4a9d but unfortunately it returned HTTP 403: forbidden error. Could be a bug?

I tried using the Kick implementation from  #31576 and it was working at first, but then started getting 403's as well. Probably still due to the cloudflare fingerprinting I'm guessing.
I have a branch with the code from #31576 for VOD's and have a clip extractor too, not sure if we are still wanting to add functionality if the 403's are happening due to the cloudflare issue?


Identifiers validation failed for dataset Israel's sea turtle monitoring program (aggregated per 1-degree cell)

Identifier validation failed for the dataset Israel's sea turtle monitoring program (aggregated per 1-degree cell):

Crawler attempt: 117
Publishing organization: OBIS-SEAMAP
Publishing country: UNITED_STATES
Installation: OBIS-SEAMAP IPT
Number of occurrences indexed for this dataset at the time of the issue: 141
Cause: GBIF ID problems exceed 50% threshold: 100% duplicates; 141 total records; 141 absent records
Pipelines execution steps: http://api.gbif.org/v1/pipelines/history/execution/3182483/step

Publisher email
Hello,
I am contacting you from the GBIF Secretariat about a dataset published by the Israel's sea turtle monitoring program (aggregated per 1-degree cell) : https://doi.org/10.15468/9fb4vp.
We noticed that the occurrenceIDs were changed. We have temporarily paused the ingestions of this dataset.
As you might already know, when an occurrence record has a new occurrenceID for a given dataset, our system considers it to be a new occurrence.
This means that it will be given a new gbifid and a new occurrence URL (like this one: https://www.gbif.org/occurrence/1252968762) and the old gbifid and URL will be deprecated.
In this case, this means that the occurrence URLs would be deprecated when ingesting the newest versions of these datasets.
We would like to check with you if those changes were intentional. Do you know if this is the case?
Please let us know, thanks! We are happy to resume the dataset ingestion.
Note that some users rely on those occurrence URLs and gbifids (like https://bionomia.net for example). In an attempt to improve
the stability of the occurrence URLs and gbifids, we have implemented a warning system to detect these type of changes in datasets (see this news item).
If the data publisher can provide us with a list of old and new occurrenceIDs per record, we can avoid the identifier and URL changes. Could that be an option?
Please let us know if you have any question. Thanks!
All the best,

You can skip/fix identifier validation using the registry UI.

The data was aggregated, we were told in the past to not check those as the IDs will change every update.
I added the tag and ingested the dataset.


SendEmail.py using hard-coded email/password

We should not be using hard-coded email/passwords anywhere. We should be updating the code so that it picks up the email/password from environment variables.

Fixed this in commit https://github.com/shahrk/ItemStockTracker/commit/29ffb8710287a0f8fba1780009d21dfe2bafec69


App Service Deploy fails for some builds with EEXIST: file already exists

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name:
AzureRmWebAppDeployment (4.163.3)
Environment

Server - Azure Pipelines or TFS on-premises?

If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

Agent - Hosted or Private:

If using Hosted agent, provide agent queue name:

Issue Description
We sometimes experience random builds to be "indeployable" - giving the error message below, EEXIST: file already exists, mkdir ...
The error is persistant for the same build, i.e. we have to make a code change to make a "deployable" build.  Redeploying does not help, nor a new release with the same artifact, nor a new CD build with the same sources.
Also tried at the same time to run an older release, just to see if it was something to do with the agent state at thet time, but that worked fine.
Task logs
tasklog_6_r.zip
Error logs
2020-02-26T15:09:44.5195784Z ==============================================================================
2020-02-26T15:09:44.5196259Z Task         : Azure App Service deploy
2020-02-26T15:09:44.5196778Z Description  : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2020-02-26T15:09:44.5197223Z Version      : 4.163.3
2020-02-26T15:09:44.5197511Z Author       : Microsoft Corporation
2020-02-26T15:09:44.5197980Z Help         : https://aka.ms/azureappservicetroubleshooting
2020-02-26T15:09:44.5198290Z ==============================================================================
2020-02-26T15:09:45.9612565Z Got service connection details for Azure App Service:'redacted'
2020-02-26T15:09:47.2882829Z ##[error]Error: EEXIST: file already exists, mkdir 'D:\a_temp\temp_web_package_39831687094834645\Content\D_C'

@snaits Sorry for delayed response. Are you still facing this issue?

Closing this issue due to long inactivity. Please feel free to reopen if issue still persists.


[Feature] Jump list support

It would be nice if this launcher could support jump lists - pin the launcher in the Windows task bar, right-click on it, and you can run your last games with your last patches without even running the launcher!
It would also be nice to be able to pin a game+config in the jump list, but I have no idea how the UI for that would be.
Also, I think it's possible to share the content of the jump list with thcrap_launcher. Thcrap_launcher doesn't support the jump list for now, but that's something we should think about.

Should be done 7dcd844bb406760efa47a615417b13c1b08c4773


types: fix syn:prop regex to support globs

this enables glob props from data models. currently, there's an error thrown.

Coverage decreased (-0.3%) to 86.704% when pulling e0553225d4cac7b78eff23ca0f9471975f57d5e4 on williballenthin:patch-9 into 3a4136db971d77c5c3ed9423ee064ae1ecc59966 on vivisect:master.

Coverage decreased (-0.3%) to 86.704% when pulling e0553225d4cac7b78eff23ca0f9471975f57d5e4 on williballenthin:patch-9 into 3a4136db971d77c5c3ed9423ee064ae1ecc59966 on vivisect:master.


Add a missing psa_crypto test suite test name

Dropped here: https://github.com/ARMmbed/mbedtls/commit/71b0567c87e07ba7376b991f46d94bcac8afbfa8#diff-cc82e7d511f22a0a29b24a8ad0c62d6c62b5f8c84ad67e55ea71d7fc035101aeL1421

Thanks for spotting this!
Specifically it was dropped in f494bcd31347456c58362efa606832de14e8f461 (git log -S 'PSA symmetric encrypt: AES-ECB, 0 bytes, good'), and this was backported in 2.x, so please make a 2.x backport.
Why this didn't lead to a test failure: the depends_on: line became the test description, so the test cases didn't have any dependencies. We currently don't do any build with PSA crypto active but without both AES and ECB support. So this test case only ever ran with PSA crypto active (otherwise the whole test suite is skipped) those always had AES and ECB available so the test was always running when it should and not when it shouldn't.


Tweak docker-compose.yml file to redirect HTTP connections to HTTPS

Fixes #18

Easier to test on the server so merging for now.


Replacing SAI with DAI in the existing pairs and adding new SAI/DAI p…

…air. Also adding new main net market contract address

Applying #377


Stack overflow error with RGBA colors in Scale.color_discrete_manual()

I'm attempting to make some plots that have elements with some level of transparency, but when I try to use RGBA colors, I get a stack overflow error.
The following works fine:
mycolors = [RGB(0.8,0,0), RGB(0,0.8,0), RGB(0,0,0.8)]

plot(x=[1,2,3], y=[2,6,4], Geom.point(), color=x, Scale.color_discrete_manual(mycolors...))

But this:
mycolors = [RGBA(0.8,0,0,0.5), RGBA(0,0.8,0,0.5), RGBA(0,0,0.8,0.5)]

plot(x=[1,2,3], y=[2,6,4], Geom.point(), color=x, Scale.color_discrete_manual(mycolors...))

Throws the error: StackOverflowError:
Ultimately, I want to do this in a color gradient, but it looks like that's not allowed:
colorx = RGBA(0.8,0,0,0.5)
colory = RGBA(0,0.8,0,0.5)
plot(x=[1,2,3], y=[2,6,4], Geom.point(), color=x,
    Scale.color_continuous(colormap=Scale.lab_gradient(colorx, colory)))

error:
MethodError: `lab_gradient` has no method matching lab_gradient(::ColorTypes.RGBA{Float64}, ::ColorTypes.RGBA{Float64})

Based on #224, it looks like there's some other grammar to support this, but I can't find examples of how it's supposed to work.

It doesn't work.  Looking at the current Gadfly code for scales (https://github.com/dcjones/Gadfly.jl/blob/master/src/scale.jl) there are some statements that any color scheme you provide is converted to RGB{Float32}. Note that Theme ((https://github.com/dcjones/Gadfly.jl/blob/master/src/theme.jl)) does provide ways of making some colors transparent, but these need to be implemented in a particular geometry for them to work.
One work around is to use Guide.annotation (http://dcjones.github.io/Gadfly.jl/guide_annotation.html) , but this doesn't produce a color key.
The other way is to make your own geometry:
using Colors, Compose, DataFrames

immutable PointGeom2 <: Gadfly.GeometryElement
     tag::Symbol

    function PointGeom2(; tag::Symbol=Gadfly.Geom.empty_tag)
         new(tag)
     end
end

function Gadfly.element_aesthetics(::PointGeom2)
    [:x, :y, :size, :color, :shape]
end

function Gadfly.render(geom::PointGeom2, theme::Gadfly.Theme, aes::Gadfly.Aesthetics)

     default_aes = Gadfly.Aesthetics()
    default_aes.color = DataArrays.PooledDataArray(Colors.RGBA{Float32}[theme.default_color])
     default_aes.size = Measure[theme.default_point_size]
    aes = Gadfly.inherit(aes, default_aes)
    # aes.color arrives as a vector of RGB colors ->
    # so change them to back to RGBA colors
    alpha_color = map(theme.lowlight_color, aes.color)
    ctx = context()
    compose!(ctx, circle(aes.x, aes.y, aes.size, geom.tag), fill(alpha_color))
    return compose!(context(order=4), svgclass("geometry"), ctx)
end

point2 = PointGeom2

D = DataFrame(x=1:5, y=randn(5), dcol=["a","b","b","a","c"], ccol=[1,2,2,1,3])
dcolscale = Scale.color_discrete_manual(RGB(0.8,0,0), RGB(0,0.8,0), RGB(0,0,0.8))
ccolscale = Scale.color_continuous(colormap=Scale.lab_gradient(colorant"green",colorant"red"))
# In the next line, opacity = 0.5
theme1 = Theme(default_point_size=10pt, lowlight_color=c -> RGBA{Float32}(c.r, c.g, c.b, 0.5))

p1 = plot(D,x=:x,y=:y, color=:dcol, point2, theme1, dcolscale)
p2 = plot(D,x=:x,y=:y, color=:ccol, point2, theme1, ccolscale)
hstack(p1,p2)

If you are using multiple geometries in a single plot, than put the Theme(lowlight_color=) in the separate layers.

Wow, thanks for writing that out. I'll give this a shot. I'll presume on your generosity - but feel free to ignore these questions, this code is already more than helpful.
If I understand correctly, this creates a new geometry for point-type graphs only, and you're using point2 in the definition of p1 and p2 in place of Geom.point right? So now when it goes to render, it knows to grab your new aesthetics since the new type ::PointGeom2 ends up being passed to render and element_aethetics? So if I wanted to use alpha colors in a bar chart, I'd need to make a BarGeom2 and define a separate element_aesthetics and render for that type?
Final question - can you think of a compelling reason not to fork Gadfly and just change the render functions for the various geometries to accept alpha colors?

I think you've largely got it (yes point2 is an instance of the type PointGeom2 just as Geom.point is as an instance of the type Gadfly.Geom.PointGeometry). As I mentioned above opacity maybe already implemented for some geometries. It is controlled through Theme(), so you need to read through the Theme code (https://github.com/dcjones/Gadfly.jl/blob/master/src/theme.jl).
I do have a fork: https://github.com/Mattriks/Gadfly.jl, with a new geometry:
https://github.com/Mattriks/Gadfly.jl/blob/Geom_segment/doc/geom_segment.md
I think the issue with color opacity/transparency is that it is does not seem to be implemented for color keys, e.g. in my code above try changing the opacity to 0.1, the opacity of the color keys does not change. So currently opacity can be implemented for geometries, but there is only 'one scale for all' (or two if you want to be technical: discrete and continuous).  Thus I think there are some larger design questions here about how opacity should be implemented. So for the moment, I'd rather provide people with simple workaround type solutions.

Interesting - that makes sense, thanks! I can certainly make this work for now.
Github etiquette questions: Do you think I should leave the issue open or close it out?


Added dbscan sycl example; reduced size of dbscan dataset

@amgrigoriev Why my previous PR is not used? https://github.com/IntelPython/daal4py/pull/202
We talked about it and I sent the link yesterday.
In any case, I think that comparing of results of execution (with different devices too) in example is helpful. We can expand it like here https://github.com/IntelPython/daal4py/blob/feature/sycl-dbscan/examples/sycl/dbscan_batch.py#L78.
Examples are part of our tests, so it will check that sycl version works and provides appropriate results.

I have an issue with execution time even on this small dataset
from daal4py.oneapi import sycl_context
import daal4py as d4p
import numpy as np

X = np.array([[1, 2], [2, 2], [2, 3],
              [8, 7], [8, 8], [25, 80]])
with sycl_context('gpu'):
    clustering = d4p.dbscan(epsilon=3, minObservations=2,
                            resultsToCompute="computeCoreIndices",
                            memorySavingMode=1).compute(X)
    print("clustering.labels_")
    print(clustering.labels_)
    print("clustering")
    print(clustering)

Looks like it hangs somewhere inside.


browser.screenshot throwing a Ferrum::DeadBrowserError on Raspberry pi

I encounter some issue with Ferrum on the raspberry pi 3 (raspbian).
Things are overall working fine it seems, but when I try to get a screenshot a Browser is dead (Ferrum::DeadBrowserError) is thrown.
 I tried to increase the timeouts to 100 but it's simply waiting longer and fail.
Any idea what could be the problem? Save to pdf feature leads to the same error.
Thanks

I just noticed something. The screenshot feature is not the problem. It looks like the whole thing is not working properly, I can't control the browser at all. If I do 2 calls to goto in a row, only the first one works, then I get the "Browser is dead" error.

I checked the available RAM and that doesn't seem to be a memory issue.
As mentionned it's not only happening for screenshots but any interaction. I can't do more than one goto() for instance.
https://github.com/route/ferrum/blob/1ab9c566c29dc417d95a92a50579ecad1224de8f/lib/ferrum/browser/client.rb#L41
This is where it fails.
I checked and noticed that data is nil and @ws.messages.closed? returns true too. And @pendings equals {}.
Any idea why this situation happens?

You don't need chrome driver BTW, only chrome. I could also try to use the latest Chrome if possible.
As for why this is happening, for some reason when you issue subsequent Page.navigate we send it to chrome and start waiting for response if nothing returned then data is nil and we check if for some reason message queue is closed which can happen only when socket is closed in case of error or when chrome dies. Try to set timeout to a higher value, it's 5 by default, try to use latest chrome, and check if chrome process really dead while setting timeout to a higher value. That's all clues so far :)

I'm gonna close issue for now, let me know if anything from the things I had mentioned above helped.


fix input icon loading

Please makes sure these boxes are checked before submitting your PR, thank you!

[ ] Make sure you follow Element's Contributing Guide.
[ ] Rebase before creating a PR to keep commit history clear.
[ ] Add some descriptions and refer relative issues for you PR.

#861


Unable to bold link text in wysiwyg

Version
1.4.6
Test Environment
Chrome 76, windows 10
Current Behavior
Cannot bold link text in wysiwyg if that is the only thing you highlight
Expected Behavior
If you highlight the link and hit bold, the line will be bold

Almost seems like this is was done on purpose?
https://github.com/nhn/tui.editor/blob/master/src/js/wysiwygCommands/bold.js#L48

@gregroyal
It's not intended, it looks like a bug. Because it's a spec that works fine in Markdown. I'll check the issue!


make_texture doc2test

Description
Takes the make_texture section of the imagebufalgo doc and turns it into a unit test in both python and c++.
Tests
This adds a make_texture section to the docs-examples-cpp test and the docs-examples-python test.
Checklist:

[x] I have read the contribution guidelines.
[x] I have updated the documentation, if applicable.
[x]  I have ensured that the change is tested somewhere in the testsuite
(adding new test cases if necessary).
[x] If I added or modified a C++ API call, I have also amended the
corresponding Python bindings (and if altering ImageBufAlgo functions, also
exposed the new functionality as oiiotool options).
[x] My code follows the prevailing code style of this project. If I haven't
already run clang-format before submitting, I definitely will look at the CI
test that runs clang-format and fix anything that it highlights as being
nonconforming.

I think you still don't have a DCO sign-off on one of the commits, can you fix that, please?

Rolled everything into a single signed commit. Thanks!


0234 - Palindrome Linked List

Description of the Problem
Given a singly linked list, determine if it is a palindrome.
Code
class Solution(object):
def reverse(self, head):
    p = head.next
    while p and p.next:
        tmp = p.next
        p.next = p.next.next
        tmp.next = head.next
        head.next = tmp

def isPalindrome(self, head):
    """
    :type head: ListNode
    :rtype: bool
    """

    #get middle pointer
    p1 = ListNode(0)
    p1.next = head
    p2 = p1
    while p2 and p2.next:
        p1 = p1.next
        p2 = p2.next.next

    # reverse second half of list
    self.reverse(p1)

    # check palindrome
    p1 = p1.next
    p2 = head
    while p1:
        if p1.val != p2.val:
            return False
        p1 = p1.next
        p2 = p2.next
    return True

Link To The LeetCode Problem
LeetCode

@vJechsmayr  Please assign me this problem

@flick-23 Waiting for your PR 😸

@flick-23 Waiting for y


Snappy decompression performance

Ran a few tests and found that the snappy decompress function is a bit of a bottleneck.
Using python-snappy instead helped me a lot:
....
import snappy
...

def decompress(data: typing.BinaryIO) -> bytes:
    return snappy.uncompress(data.read())

Hi there, thanks for that. I think last time I checked that module required the snappy C library installed (and was a pain to use on Windows). Do you know if that's still the case?

I am only using ubuntu....so I dont know..sorry.
/Mark
On Mon, Jan 9, 2023, 17:14 CCL Solutions Group @.***>
wrote:

Hi there, thanks for that. I think last time I checked that module
required the snappy C library installed (and was a pain to use on Windows).
Do you know if that's still the case?
—
Reply to this email directly, view it on GitHub
https://github.com/cclgroupltd/ccl_chrome_indexeddb/issues/15#issuecomment-1375889973,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAUZW2WP5LVDS2KOZGZXHYTWRQ2PNANCNFSM6AAAAAATRGVZ7Q
.
You are receiving this because you authored the thread.Message ID:
@.***>


add grating equation schema

Adds a schema for a grating equation model. Two models in astropy will use it: WavelengthFromGratingEquation and AnglesFromGratingEquation.

Moved to gwcs.


Connecting to localhost QUIC server

I am trying to connect to the example server running on localhost with the example client. I replaced the existing certificates with the ones I generated. But it says "certificate is valid for localhost, not quic.clemente.io". How can I solve this?
$ go run example/client/main.go https://localhost:6121
GET https://localhost:6121
Starting new connection to localhost ([::]:56484 -> <IP_ADDRESS>:6121), connectionID 79b6bee48fbfff47, version 512
Certificate validation failed: x509: certificate is valid for localhost, not quic.clemente.io
Closing session with error: ProofInvalid
Connection 79b6bee48fbfff47 closed.
panic: Get https://localhost:6121: ProofInvalid

This is not QUIC specific. Please refer to the countless TLS tutorial on the internet.

edit example/client/main.go
replace
hclient := &http.Client{ 	Transport: &h2quic.RoundTripper{QuicConfig: quicConfig}, }
with
`
hclient := &http.Client{
Transport: &h2quic.RoundTripper{QuicConfig: quicConfig, TLSClientConfig: &tls.Config{InsecureSkipVerify: true}},
}
`


[INLONG-4726][SortStandalone] Support ClickHouse sink.

Prepare a Pull Request

Title: [INLONG-4726][SortStandalone] Support ClickHouse sink

Fixes #4726

Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve?
Modifications
Describe the modifications you've done.
Verifying this change
(Please pick either of the following options)

[ ] This change is a trivial rework/code cleanup without any test coverage.

[ ] This change is already covered by existing tests, such as:
(please describe tests)

[ ] This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

@luchunliang please add license for the new dependency

the license added in https://github.com/apache/inlong/pull/4747


After first installer the install drive doesn't show.

Hi friend, I have tried few time but its seems consistent issue. After booting from USB and installer load some files to SSD and then it reboots and that SSD disappear. I tried use HFS partition then it shows the drive and after 2nd boot installer continues, but then after 2nd reboot it again disappear. It seems the USB only able to show the HFS drives but not APFS. Since it convert back to APFS during installation, I think may be that's why it's not showing. I third both Mojave and Catalina both have the issue. But rest of others versions, its good.
Can you please guide what I should do? Thanks

I have also tried vBoxHFS as well but same issue.

I dont have issue with Catalina latest Program update config.plist with this
This way you dont have anny APFS restriction

Ah I see now it my installer its 0 0

Awesome I change it to -1 and it worked.  Thanks

Please see the code I think this part is missing because I didn't change the plist after creating the installer.
But Thanks again !

Please see the code I think this part is missing because I didn't change the plist after creating the installer. But Thanks again !

No its there See commit

Yes you are right but somehow it's not working on Mojave and Catalina as I'm using the latest release of yours. And for proof I'm also attaching a video post installation as it didn't add the -1 in the plist. Somehow it's working on later OS versions but not on Mojave and Catalina. I've to reinstall the OS because after reboot the MacOS drive did't work and even the usb with -1 couldn't pickup the drive.
Please see https://drive.google.com/file/d/1-urCdwoAMwDyOZWNQDtg47R3VPyClY0Y/view?usp=share_link


Adding circle ci checks

For circle CI to create the correct .zip file you need to setup a read/write deployment key.
https://circleci.com/docs/1.0/adding-read-write-deployment-key/


add pyprind progress bar for voucher table

Coverage increased (+0.06%) to 93.04% when pulling d6e1f4c82d70cc1809988454cf270f3267ddcfd5 on importer into 02f6fd297c9bd63eb54a6e8ff0e54f6ff97df723 on master.


Provide methods for converting starlark.Value to/from interface{}

Similarly to how google.golang.org/protobuf/types/known/structpb does the conversion:

to interface{}: https://pkg.go.dev/google.golang.org/protobuf/types/known/structpb#Value.AsInterface
from interface{}: https://pkg.go.dev/google.golang.org/protobuf/types/known/structpb#NewValue

For user-defined Value's an interface can be provided, similar to Golang's JSON Marshaler and Unmarshaler interfaces:
type ToInterface interface {
	ToInterface() (interface{}, error)
}

type FromInterface interface {
	FromInterface(interface{}) error
}

The error allows handling the case when an attempt to call ToInterface() is made on an outer Value which contains user-defined Value inside without ToInterface implemented.
These methods would simplify passing arbitrary YAML/JSON-compatible data structures as ordinary dict's/list's/etc to Starlark functions from Golang (e.g. Cirrus CLI uses Starlark to implement build/task hooks, and this conversion is currently done manually).

I am wary of defining a standard function for converting Starlark values to Go values, or the reverse, because there are so many different ways to do it. For example, which Go integer type should a Starlark int value be mapped to? The smallest one capable of representing the value? int? *big.Int?  There are many sensible answers to such questions, and that suggests that no particular one belongs in the standard package.  (A secondary concern is that such conversions invite very inefficient deep copying.)
Fortunately, there is no need for these functions to live in this repository. You can define your own conversion functions that do exactly what you want. If many users want the same function, it can be shared in a utility package. If the passage of time reveals a particularly good implementation, then let's talk about making it standard.
(By the way, see https://github.com/google/starlark-go/commit/ac23acb182e187e6835e4c0cc42205fe37a05624 for a standard Starlark JSON package that encoding and decoding operations for all the data types of JSON.)

Thanks for the quick reply 😀
Just wanted to vent the idea before actually making a PR and I think you have a point. In order to provide a one-size-fits-all implementation a bunch of options would probably need to be introduced that would control the conversion behavior.
Will close this for now.


always says "[       scan] [   INFO] Completed  0.00% of scan." but it is actually progressing

see screenshot. however, it IS scanning & progressing

The error is in here:
            if ScanMetrics.NUM_STEPS:
                ScanMetrics.CURRENT_SCAN_PERCENT = float(ScanMetrics.STEPS_COMPLETED) / ScanMetrics.NUM_STEPS * 100
            else:
                ScanMetrics.CURRENT_SCAN_PERCENT = 0

NUM_STEPS is 0 somehow for you. Restarting the server will probably fix this.

sometimes restarting fixes it, sometimes it doesn't. usually after restart, the first scan has this problem. any consecutive scan is correct.

Closing this because it's a really minor issue.


how to abort an in-flight request with the abort() method

This isn't supported because it's not part of the  A+ spec. If you want to add the ability to abort a request feel free to submit a PR.


Incorrect handling of url-encoded username/password in proxy url for socket mode

Reproducible in:
The Slack SDK version
slack-sdk==3.13.0
Python runtime version
3.6.13
OS info
ProductName:	macOS
ProductVersion:	11.6.3
BuildVersion:	20G415
Darwin Kernel Version 20.6.0: Wed Jan 12 22:22:45 PST 2022; root:xnu-7195.141.19~2/RELEASE_ARM64_T8101

Steps to reproduce:

Pass an authenticated proxy url containing url-encoded characters (e.g<EMAIL_ADDRESS>to a synchronous slack_sdk.socket_mode.SocketModeClient, either via HTTPS_PROXY env variable or proxy argument.
Connect to the Slack API.

Expected result:
The connection through the proxy is successful.
Actual result:
The connection fails with the following error:
Traceback (most recent call last):
  File "[redacted]/lib/python3.6/site-packages/slack_sdk/socket_mode/builtin/connection.py", line 116, in connect
    trace_enabled=self.trace_enabled,
  File "[redacted]/lib/python3.6/site-packages/slack_sdk/socket_mode/builtin/internals.py", line 81, in _establish_new_socket_connection
    f"Failed to connect to the proxy (proxy: {proxy}, connect status code: {status})"
Exception: Failed to connect to the proxy (proxy: http://[redacted]:[redacted with url encoded chars]@[redacted url]:80, connect status code: 407)

The documentation for urllib.parse.urlparse mentions that % escapes are not expanded, and username and password should not be url-encoded when they are passed as part of a Basic authentication.
As far as I understand the issue is simply that the username and password should be url-decoded before being base64'd in the following code: https://github.com/slackapi/python-slack-sdk/blob/77e906a2bd0ff3d45c65635476c625c0ed5be3a8/slack_sdk/socket_mode/builtin/internals.py#L60-L64
(As a side note, printing the full proxy info including the password in the exception is not so great, but that's another issue.)

HI @Lordshinjo, thanks a lot for taking the time to report this issue! We'll fix this issue in the next release. If you're interested in fixing this issue, your pull request contributions are always welcomed 👋


ClusterLoader - Adding templating

Replacing object names + index template.

/assign @shyamjvs

Couple of high-level comments:

Don't reimplement logic for template evaluation - try to reuse the golang template library.
Let's follow the approach we discussed offline to support templating and linking across the yamls. If at some places you have a trade-off between 'implementation complexity' and 'extra requirements imposed on users' - I'd suggest choosing the latter for now. That'll keep things simple and we can think about bettering usability later.

Thanks. Lgtm modulo some comments on variable naming.
Also notice how crisp and simple this PR is now, after using golang template library :)

/lgtm


[FEAUTURE REQUEST] Custom Categories API

I'm working on a Client Mod for my Minecraft Server and it integrates with various mods too if they are installed.
One of those mods was Mod Manager, I had a mixin that added a custom category for the (optional) mods the client and the server support for Mod Manager 1.0.1 to help the playerbase discover and install new mods from other developers since the philosophy of the client is not to replace other mods but to integrate them in order to give credit to the original mod creator and idea and allow for modularity.
Now in ModManager 1.1 this is not possible anymore because Mixins are not supported in Kotlin and I haven't found a good way to add custom origins or categories from another mod in the code.
Thanks in advance!

Hi ModManager has an API for custom providers since 1.0.0 it allows you to register a custom provider.

A basic provider would look like this:

package xyz.deathsgun.modmanager;

import net.minecraft.text.TranslatableText;
import org.jetbrains.annotations.NotNull;
import xyz.deathsgun.modmanager.api.http.CategoriesResult;
import xyz.deathsgun.modmanager.api.http.ModResult;
import xyz.deathsgun.modmanager.api.http.ModsResult;
import xyz.deathsgun.modmanager.api.http.VersionResult;
import xyz.deathsgun.modmanager.api.mod.Category;
import xyz.deathsgun.modmanager.api.provider.IModProvider;
import xyz.deathsgun.modmanager.api.provider.Sorting;

import java.util.ArrayList;

public class TestProvider implements IModProvider {

    @NotNull
    @Override
    public CategoriesResult getCategories() {
        ArrayList<Category> categories = new ArrayList<>(); // Add your categories here
        return new CategoriesResult.Success(categories);
    }

    @NotNull
    @Override
    public ModsResult getMods(@NotNull Sorting sorting, int page, int limit) {
        return new ModsResult.Success(new ArrayList<>());
    }

    @NotNull
    @Override
    public ModsResult getMods(@NotNull Category category, @NotNull Sorting sorting, int page, int limit) {
        return new ModsResult.Success(new ArrayList<>());
    }

    @NotNull
    @Override
    public ModsResult search(@NotNull String query, @NotNull Sorting sorting, int page, int limit) {
        return new ModsResult.Success(new ArrayList<>());
    }

    @NotNull
    @Override
    public ModResult getMod(@NotNull String id) {
        return new ModResult.Error(new TranslatableText(""), null);
    }

    @NotNull
    @Override
    public String getName() {
        return "My custom mods";
    }

    @NotNull
    @Override
    public VersionResult getVersionsForMod(@NotNull String id) {
        return new VersionResult.Success(new ArrayList<>());
    }
}

And you register it like this:

package xyz.deathsgun.modmanager;

import net.fabricmc.api.ClientModInitializer;

public class TestMod implements ClientModInitializer {
    @Override
    public void onInitializeClient() {
        ModManager.modManager.getProvider().put("myProvider", new TestProvider());
    }
}

If need something else let me know.

Hi and thanks for replying to me so quickly!
The integration is working, I had to work around some jank because I don't want to make the mod a required dependency, but it works!
I wanted to just include my category inside the Modrinth provider, but this works too, I'll work on categorizing the various mods now.
Thanks for the support!


Missing CLANG_LIBS for plugin_print_functions.so

$(BUILDDIR)/plugin_print_funcnames.so: $(SRC_CLANG_DIR)/plugin_print_funcnames.cpp
$(CXX) $(PLUGIN_CXXFLAGS) $(CXXFLAGS) $(LLVM_CXXFLAGS) $(CLANG_INCLUDES) $^ \
$(PLUGIN_LDFLAGS) $(LLVM_LDFLAGS) -o $@
threw a linker error, fixed by changing to
$(BUILDDIR)/plugin_print_funcnames.so: $(SRC_CLANG_DIR)/plugin_print_funcnames.cpp
$(CXX) $(PLUGIN_CXXFLAGS) $(CXXFLAGS) $(LLVM_CXXFLAGS) $(CLANG_INCLUDES) $^ \
$(CLANG_LIBS) $(PLUGIN_LDFLAGS) $(LLVM_LDFLAGS) -o $@

This is curious. What platform are you on, and what compiler is that with?
In general on Linux you shouldn't need to link libs into the plugin, as it will find those symbols in the Clang binary that loads it.
Consider this: why link MBs of Clang stuff into each and every small plugin, the symbols are there in Clang anyway, and the plugin does not exist in a vacuum - it's loaded by Clang.


Syntax is incorrect - nltest /dsgetdc: /keylist /kdc

The syntax of the NLTEST command is incorrect.  It should be nltest /dsgetdc: /keylist /kdc instead of nltest /dsgetdc: /keylist /kdc.

@mbiasMSFT
It would be great if you could add a link to the documentation you are following for these steps? This would help us redirect the issue to the appropriate team. Thanks!!

@mbiasMSFT
Since we have not heard back from you, we will now proceed to close this thread.If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.
#please-close


update dependencies

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@2fa3964). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master      #39   +/-   ##
=========================================
  Coverage          ?   33.83%
=========================================
  Files             ?       32
  Lines             ?      334
  Branches          ?        0
=========================================
  Hits              ?      113
  Misses            ?      221
  Partials          ?        0

Impacted Files
Coverage Δ

src/ApiBundle/Transformer/YamlTransformer.php
0% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2fa3964...24e0cdc. Read the comment docs.


Implement Exciting-Mixing, Scalar and Diagonal Broyden approximations

https://docs.scipy.org/doc/scipy/reference/optimize.root-excitingmixing.html
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.diagbroyden.html#scipy.optimize.diagbroyden
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linearmixing.html#scipy.optimize.linearmixing

LinearMixing is just J = -1 / alpha, and J is never updated, I don't think there is any point of having that, for small problems our Broyden operates in nanoseconds via StaticArrays and for larger problems this won't work anyways


Memory allocation error when using quickmongo

When using the quickmongo module, I am encountering a memory allocation error that results in an unexpected termination of my application. The problem occurs during the execution of certain operations and is independent of the amount of available memory on the device. Below is a detailed description of the issue:
Initialize the connection to MongoDB using quickmongo.
Execute a specific operation, such as a query or data insertion.
The error occurs, and the application terminates with the following error message:

Operating System: Debian
Node.js Version: v18.12.1
quickmongo Version: v18.12.1

It seems like you're encountering a memory allocation error in your application when using the quickmongo module. The error indicates that the JavaScript heap memory is running out.
The error message "FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory" indicates that the JavaScript heap has run out of memory due to an invalid table size.
This type of error usually occurs when the application is trying to allocate more memory than is available. It can be caused by a large amount of data being handled or by inefficient operations in the code.
Here are some suggestions to resolve this issue:

Check memory consumption: Analyze the amount of memory your application is consuming. Check if there are any parts of the code that are using a lot of memory unnecessarily or if there are any memory leaks.
Optimize operations: Check if there are any operations in the code that are handling large amounts of data at once. Consider dividing these operations into smaller parts or implementing some form of pagination to reduce memory load.
Free up resources: Make sure you are properly releasing resources after use, such as closing connections to MongoDB or freeing unnecessary objects from memory.
Split operations into smaller batches: If you're inserting or querying large amounts of data, divide these operations into smaller batches. This will help reduce memory pressure.5
Increase memory limit: If possible, you can try increasing the JavaScript heap memory limit. This can be done through the --max-old-space-size parameter when running Node.js. For example, node --max-old-space-size=4096 your_app.js to set the memory limit to 4 GB.

LoL, I did the basic search for the error. Responses generated by ChatGPT will not solve this problem.

LoL, I did the basic search for the error. Responses generated by ChatGPT will not solve this problem.

Hey, I believe the amount of data you request exceeds the amount Node sets for the JavaScript heap. Which unfortunately isn't something we can do to fix that.
If you are not requesting a lot of data then that could be a bug.
There is a way to make Node's heap limit bigger since as you stated, it doesn't take the full ram.
That would be like a workaround if you ask me. If you have a lot of data it should be probably separated in more manageable blocks instead of a whole big block of data.
Without knowing how much/what data you are loading, I cannot do much more than that.
I would also point out that quickmongo is not really maintained anymore as you can see from the last commit date

Well, I am carrying 4 objects that contain short strings, it doesn't make sense to be overloading, and besides, when I access what I supposedly stored, it returns an unimaginable amount of null contents in an array in the database.

That is the first time I see something like that, the last release hasn't been changed for quite a while. The nulls are probably something that happened in the way you used quickmongo that wasn't normal or added them without intending to do so. That is probably why it gets quite big and goes over the memory limit.
Without code and knowing what you load or an example of it. I cannot do much as I stated earlier. It is also impossible for us to change the memory limit as you can only do it when you run Node

Yes, but regardless, I can't wrap my head around this overload for creating a bot on Discord. I'll provide you with the connection and how I'm using it:
-> BanCommand.js https://sourceb.in/hQLZIFOxeQ
-> BansCommand.js https://sourceb.in/N44xC2b028
-> KickCommand.js https://sourceb.in/a1dvtGCZpQ
-> UnBanCommand.js https://sourceb.in/tWkzieWSnG
Structure and Connection -> https://sourceb.in/xvf4DgFHxp
Command Structure -> https://sourceb.in/Iv1lYcrdTh
That's basically it. I neither use nor send any more data than what was mentioned above. If you can find what is supposedly sending thousands of null data, I would be grateful!

I am unsure what could be the cause of all of these nulls you have. It could come from something you had before but changed like it could also come from another program or anything that used the mongodb database you are using with quickmongo.
Unfortunately, I am not in charge of quickmongo but I do reply to people to help solve issues since nobody else really does it. That problem though is more technical than a simple problem of just mis using quickmongo.
I don't have a lot of knowledge of how it works internally as I am not the last developer who worked on this (for a quick story this repo was given to plexidev from another developer but nobody has actually tried taking the responsibility for it)
With that being said, I am not sure if I will be able to provide you with a fix for your issue as I personally don't have time for working on quickmongo


Need a Logo

some AI art to get the ideas flowing lol

I kinda like the idea of lightning bolt on a note

I like it to do; I could do a stylized 3d Render in Blender and make it animated. I am going to have a stylized 3d logo for my blog and my company site site


[ENHANCE] Improve File Change Detector

Improve the file change detector logic. Rn, every time a file is changed, it processes all the files again. I'll create a way to re-process only the files affected by the change;

Enhance implemented here: https://github.com/wpdas/alem/pull/22.
Check out release here: https://github.com/wpdas/alem/releases/tag/v1.1.0


test_logsparse.test_multiprocess fails frequently

This test often fails(timeout) on github actions, but I can never reproduce it on my local machine. This could be some racing issue or only happens when the host is slow. I don't know if it's a real failure yet, it would be nice if someone can take a look into it and figure it out.

Changed the test not to use pool. It's test_multiprocess's responsibility.


Make sure small methods and constructors in package:vector_math are inlined in AOT

package:vector_math has a lot of small methods and constructors, which are not always inlined in AOT-compiled code.
This prevents other optimizations (e.g. allocation sinking) from happening.
For example, on ParticleSystemUpdate benchmark the following methods are not inlined:
  Vector2 clone() => Vector2.copy(this);

  double normalize() {
    final l = length;
    if (l == 0.0) {
      return 0.0;
    }
    final d = 1.0 / l;
    _v2storage[0] *= d;
    _v2storage[1] *= d;
    return l;
  }

@mraleph @mkustermann

This normalize method doesn't look very small - most likely if this is inlined into multiple places it will create increase in code size. So if one would like to inline it for performance reasons I would suggest changing package:vector_math to use @pragma('vm:prever-inline').


CI/CD improvements

weird. i accidentally activated the .org version and am now getting redundant travis tests


Dev/plugins

tl;dr:
It's okay to merge this even with the CI/CD error. We will not merge wit-parsing back into dev/plugins until the ci/cd error is fixed.

This PR exposed another issue with the build environment.
error[E0463]: can't find crate for `core`
2024-02-21T16:28:18.3446507Z   |
2024-02-21T16:28:18.3447450Z   = note: the `wasm32-unknown-unknown` target may not be installed
2024-02-21T16:28:18.3448853Z   = help: consider downloading the target with `rustup target add wasm32-unknown-unknown`

The wasm32-unknown-unknown target needs to be installed in order to build components that don't need wasi interfaces.
I've added a PR to create yet another builder image, but it's going to take a little while for this change to make it's way to dev/plugins.


Add copyTargetWidth / copyTargetHeight options

This PR implements two new options that allow to force tethered elements to take the same size than the target. It's useful in a large number of use case, such as the one demonstrated by the Select library (https://github.com/HubSpot/select/issues/13).

👍

Ping ?

I feel like this is beyond the scope of what Tether should do. Tether is for positioning, not styling. If Tether provided a callback at this stage, it would be easy for the user to add this functionality.

I don't quite agree. Positionning is directly related with an object size, as evidenced by the constraint feature. Furthermore, I think there is only few use cases requiring to manually update the tethered element - and because how basic copying the container width/height is, I believe it should be done manually.
WDYT?

ping ? 😃

I've rebased my changes and commited a version increase commit.

Ping @adamschwartz @TrevorBurnham?

@arcanis instead of just copy target width and height, what about being able to specify a percentage of the width and height?

That would be a possibility. However, I don't work on the project who used tether anymore, so I won't be able to test any further change. You can open another PR if you wish to implement a more complex solution.

Ah that's a bummer. Yea it seems like a useful feature and a good PR. Oh well. Thanks for your help and effort.

Definitely useful, in scope with this module and very simple ! And I need this, too :)

I'm going to go ahead and close this PR to clean my list. Maintainers don't seem willing to merge this anytime soon, despite the clear interest.

Sad, this was such a trivial PR. As far as im concerned I'll stick to the fork.

Is this still impossible in upstream? :sob:

I feel like this is beyond the scope of what Tether should do. Tether is for positioning, not styling.

Once the tethered element is moved outside its original position, there's no elegant way to assign a width to the element equal to its target. It is plain impossible to do with CSS, and it's extremely frustrating to do with JS due to having to wait for animations, transitions, etc.
Therefore, Tether.js is the only player that can make it right. This PR is simple and efficient.
Please @TrevorBurnham @arcanis, you must advance this. :bow:

Agree with @lolmaus
I just installed the version @arcanis created, and it worked flawlessly. I recommend install his fork if you need this functionality:
npm install aracanis/tether#patch-1

You may want to look at https://popper.js.org, it's similar, with a plugin API.