need a simple python script to run WhisperSpeech locally to compare to bark

I really want to test this out and compare it to Bark, but it's implementation seems convoluted to someone like me who's never used Google Colab notebooks.  I didn't see any straight Python scripts on the repository that I could tweak to get it working....I'm running an 4090 and am very familiar with OpenAI's whisper, ctranslate2's implementation (of which WhisperX is an offshoot), Transformers libraries, etc.  I DO NOT use llama.cpp but suppose I could if need be.
Basically, is there or can they be provided....straight python scripts?

You can pretty much take the necessary lines from the notebook itself:
from whisperspeech.pipeline import Pipeline

pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model', torch_compile=True)
pipe.generate_to_file("output.wav", "Hello from WhisperSpeech.")

you need to install WhisperSpeech first - pip install -Uqq WhisperSpeech.

Thanks for the quick response...here's my entire script so far:
import torch
from whisperspeech.pipeline import Pipeline

pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model')

# Generating speech and saving it to a file
pipe.generate_to_file('output_audio.wav', """
This is the first demo of Whisper Speech, a fully open source text-to-speech model trained by Collabora and Lion on the Juwels supercomputer.
""")

print("Audio file generated: output_audio.wav")

However, I keep getting an error saying that torch.save can't get the correct "backend", which apparently originates from "torchaudio?"

The traceback indicates that the error originates from the line beginning with line reading torchaudio.save(fname, audio.cpu(), 24000), which is a portion of the "decode" method within a2wav.py:
    def decode_to_file(self, fname, atoks):
        audio = self.decode(atoks)
        torchaudio.save(fname, audio.cpu(), 24000)
        if self.is_notebook():
            from IPython.display import display, HTML, Audio
            display(HTML(f'Listen to {fname}'))

That's quite strange. It seems torchaudio does not know how to write a WAV file?
There seems to be a pretty long discussion (3+ years) about Windows torchaudio support here: https://github.com/pytorch/audio/issues/425
The docs say that soundfile should work on Windows and support WAV files. I tried forcing the soundfile backend on Linux but I cannot reproduce the issue.
Maybe you could add print(torchaudio.__version__) to your script to rule out version-related differences?

Wow, great that you got it to work. We'd love to integrate this solution into WhisperSpeech itself so it works out of the box on Windows.
Before we do that you could use pipe.generate to avoid having to modify pipeline.py. It will return the audio samples without trying to save a file.

Sure I can do that.  BTW, what are all the different models I can try?  For example,
collabora/whisperspeech:s2a-q4-tiny-en+pl.model

What other English ones are there?  I'm unfamiliar with your guys' naming conventions...

Here's the completed script.  Was necessary to move to cpu as explained in the comments:
from pydub import AudioSegment
import numpy as np
from whisperspeech.pipeline import Pipeline

pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-small-en+pl.model')
# pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model')
# pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-base-en+pl.model')

audio_tensor = pipe.generate("""
I want to test this and compare it to Bark models and see how they compare in terms of quality of audio as well as processing speed!
""")

# generate puts uses CUDA if available; therefore, it's necessary to move to CPU before converting to NumPy array
audio_np = (audio_tensor.cpu().numpy() * 32767).astype(np.int16)

if len(audio_np.shape) == 1:
    audio_np = np.expand_dims(audio_np, axis=0)
else:
    audio_np = audio_np.T

print("Array shape:", audio_np.shape)
print("Array dtype:", audio_np.dtype)

try:
    audio_segment = AudioSegment(
        audio_np.tobytes(),
        frame_rate=24000,
        sample_width=2,
        channels=1
    )
    audio_segment.export('output_audio.wav', format='wav')
    print("Audio file generated: output_audio.wav")
except Exception as e:
    print(f"Error writing audio file: {e}")

Sure I can do that. BTW, what are all the different models I can try? For example,
collabora/whisperspeech:s2a-q4-tiny-en+pl.model

What other English ones are there? I'm unfamiliar with your guys' naming conventions...

These names come from the file names in this repo: https://huggingface.co/collabora/whisperspeech/tree/main
You can try out the other en+pl models listed there. The sizes of T2S and S2A can be different (we have t2s_ref and s2a_ref arguments).

Anyone knows what is the script to one-shot voice cloning? This one from colab is also not working "pipe.generate("""
This is the first demo of Whisper Speech, a fully open source text-to-speech model trained by Collabora and Lion on the Juwels supercomputer.
""", lang='en', speaker='https://upload.wikimedia.org/wikipedia/commons/7/75/Winston_Churchill_-_Be_Ye_Men_of_Valour.ogg')"..

What is the error you see?

@Joosheen I've seen that loading the sample directly from a URL did not seem to work on Windows.
Could you try downloading the file, putting it in the same folder as the script and modifying the command to use speaker="Winston_Churchill_-_Be_Ye_Men_of_Valour.ogg".

What is the error you see?

OSError: [WinError 1314] Klient nie ma wymaganych uprawnień: 'C:\Users\AJusi\.cache\huggingface\hub\models--speechbrain--spkrec-ecapa-voxceleb\snapshots\5c0be3875fda05e81f3c004ed8c7c06be308de1e\hyperparams.yaml' -> '~\.cache\speechbrain\hyperparams.yaml'

@Joosheen I've seen that loading the sample directly from a URL did not seem to work on Windows.
Could you try downloading the file, putting it in the same folder as the script and modifying the command to use speaker="Winston_Churchill_-_Be_Ye_Men_of_Valour.ogg".

ok, I saved the file in the same folder as script and made command: "from whisperspeech.pipeline import Pipeline
pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model')
pipe.generate("""
This is the first demo of Whisper Speech, a fully open source text-to-speech model trained by Collabora and Lion on the Juwels supercomputer.
""", lang='en', cps=10, speaker="Winston_Churchill_-_Be_Ye_Men_of_Valour.ogg"
and the the same error:
"OSError: [WinError 1314] Klient nie ma wymaganych uprawnień: 'C:\Users\AJusi\.cache\huggingface\hub\models--speechbrain--spkrec-ecapa-voxceleb\snapshots\5c0be3875fda05e81f3c004ed8c7c06be308de1e\hyperparams.yaml' -> '~\.cache\speechbrain\hyperparams.yaml'ml"
I'm a newbie so i don't know if i'm not doing something really wrong..

I couldn't get it to work either...I ran into some kind of error regarding couldn't load a .ogg file though, don't have the error anymore.  I gave up after I created my basic script to test whisperspeech...But for what it's worth I can confirm that the code snippet telling to just add the "speaker" parameter didn't work for some reason.

@Joosheen I've seen that loading the sample directly from a URL did not seem to work on Windows.
Could you try downloading the file, putting it in the same folder as the script and modifying the command to use speaker="Winston_Churchill_-_Be_Ye_Men_of_Valour.ogg".

ok, I saved the file in the same folder as script and made command: "from whisperspeech.pipeline import Pipeline
pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model')
pipe.generate(""" This is the first demo of Whisper Speech, a fully open source text-to-speech model trained by Collabora and Lion on the Juwels supercomputer. """, lang='en', cps=10, speaker="Winston_Churchill_-_Be_Ye_Men_of_Valour.ogg"
and the the same error: "OSError: [WinError 1314] Klient nie ma wymaganych uprawnień: 'C:\Users\AJusi.cache\huggingface\hub\models--speechbrain--spkrec-ecapa-voxceleb\snapshots\5c0be3875fda05e81f3c004ed8c7c06be308de1e\hyperparams.yaml' -> '~.cache\speechbrain\hyperparams.yaml'ml"
I'm a newbie so i don't know if i'm not doing something really wrong..

There's some more discussion here https://github.com/collabora/WhisperSpeech/issues/72  This issue is was more geared towards getting a simple working script...
I'd recommend that the DEVS close this issue and we continue our discussion at the other issue for ease of reference??

Closing since it's been addressed.  Thanks!


9779 dataset sharing queries

Closes #9779

Qualifies table name with user schema on import (for organizations)
Fixes [SQL] tag display on dataset view and builder (to take into account the schema)

This does not rewrite queries for existing datasets, although it will mark them as "SQL applied" and clearing the SQL will qualify them. I think that's enough, but we can consider rewriting existing queries in the backend either via rake, or when sharing a dataset.
The change only affects organizations, lone users are unchanged.

Please CR @xavijam @gfiorav

Re @gfiorav review (any way to reply to CR comments?)
That's a good point. Yes, initializing the analysis takes the query from the canonical visualization layer. However, when a new layer to a map (not creating a map from a layer), the query is reset, and right now it's created without qualifying it if you own the layer.
I don't think that's a problem for maps where qualifying the query is not always necessary (as you can't share them with write permissions). Anyway, I feel changing it makes everything more consistent. Also, I tested this and I need to make some adjustments to the UI to handle this case correctly.

Fixed the case when adding a layer after the map is created and some little fun thing along the way. The code is ugly, can be improved, but I want to talk about the user_name and table_name options (which of those we are going to use and how) to see if we can simplify.

Arg, as expected, this keeps on giving. Adding a default qualified query confuses the old dataset editor (it goes to read-only mode).

Deployed in staging cloud and everything seems to work fine, except for issues #9933 and #9927 which I think are best left for another PR.


Old version of Multer is causing a high severity vulnerability

https://github.com/APIDevTools/swagger-express-middleware/blob/c0268efb41f726b2bb57d3b67684812c362a1eaa/package.json#L64
https://www.npmjs.com/advisories/1469

https://github.com/APIDevTools/swagger-express-middleware/blob/c0268efb41f726b2bb57d3b67684812c362a1eaa/lib/request-parser.js#L36
Looks like this is also fixed now as of 1.0.5 https://github.com/expressjs/multer/issues/212

closing this issue since it's duplicate of https://github.com/APIDevTools/swagger-express-middleware/issues/98


Clenup JS test. Resolve external $ref during tests.

Main improvements:

cleanup package.json. Mark it as private package to forbid publishing. swagger-validator is totally [different] https://www.npmjs.com/package/swagger-validator) package on npm.
Update all dependencies related to tests. Biggest update is z-schema.
Remove tv4 validation. It was failing for long period and was marked as expected. Another point is I'm working on switching unittests from Java to JS as disscussed here, and it easy to do on z-schema but is very hard on tv4.

@webron Can you please review it?

@webron I force Travis run on this PR and it pass.
Please review it so I can continue with switching unit tests from Java to JS.

Hi, all tests for OAI V3 should be against the OpenApi.next branch.

@fehguy Hi, thank you for reply.

all tests for OAI V3 should be against the OpenApi.next branch.

It's not intended for OAI V3. It's fixing and improving test for Swagger v2.0.
I'm working with 200+ Swagger 2.0 spec so I'm really interested in improving JSON Schema which validate them. With PRs like #413 but it's blocked by current tests.
Can you please review it?
P.S. same code can be extended to validate OAI V3 in future. So this PR important for both versions.

Code is very simple and straightforward and takes only 43 lines after cleanup.

@fehguy @webron  Can you please take a look?

@fehguy Can you please review it?
Especially since you add the same broken test suite into OpenAPI.next as part of #597.

@fehguy Thank you.
I will start working on switching unit tests from Java to JS as discussed here.

@IvanGoncharov we should probably decide first that that's the current goal. That discussion was prior to the OAI.

@webron I understand.
My argument will be that OpenAPI.next should keep JSON Schema in sync with spec changes.
That mean it will benefit from having fully working and maintainable test suite.
As for my personal goals, I want to push few schema fixes to 2.0 or next what will better suit them.
Most important fix is #598.


Remove as-pect from massa-as-sdk and replace it with envy

Currently as-pect is still inside the massa-as-sdk repo and is creating issues because of the asterster.imports.js to create and run as-pect tests. We need to remove it and replace it with envy and transfer all the tests into envy format.

After discussin this with the AS community, we will go back to as-pect which is now working properly.

@longgamma-massa just saw this ticket in QA. Maybe it relates to you.


Resilience

This issue is for anything related to resilience in Legion.

[ ] We want to have a version on which we can base a paper by mid-October.

[ ] We should plan on merging resilience into master in the spring.

@karthikmurthy @lightsighter @alexaiken What's the plan here?

This is not going to make the 20.06 milestone so I'm optimistically moving it to 20.09. In practice though it will take as long as it takes.


Adding FAQ

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/TOC.md
:white_check_mark:Succeeded
View

docs/faq.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/TOC.md
:white_check_mark:Succeeded
View

docs/faq.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/TOC.md
:white_check_mark:Succeeded
View

docs/faq.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add Apex Engine

SteamDB app page links to a few games using this
https://steamdb.info/app/6880/
https://steamdb.info/app/234140/
https://steamdb.info/app/518790/
https://steamdb.info/app/517630/
Brief explanation of the change
Add Apex Engine used in Avalanche Studio's games

Did they completely rebrand Avalanche Engine into Apex Engine?
Earlier games were Avalanche Engine, with Mad Max and Just Cause 3 being Avalanche Engine 3.0.

it's what their wikipedia page
"bases them on their proprietary Apex game engine (formerly known as Avalanche Engine)."
and their website  says


feat: persist caption settings in localStorage on track mode change (code100x#1493)

…code100x#1493)
PR Fixes:

1 Added functionality to persist subtitle/caption settings to localStorage
2 Monitored track mode changes and saved the current mode ('showing' or 'hidden')
3 On player initialization, retrieved and applied the saved caption setting from localStorage
4 Added cleanup for event listeners to prevent memory leaks

Resolves #1493
Checklist before requesting a review

[✓] I have performed a self-review of my code
[✓] I assure there is no similar/duplicate pull request regarding same issue

hey @devsargam Please check now

Thank you sir


Is it possible to integrate the provider with kubelogin exec authentication?

Is it currently possible to integrate workload identity / managed identity based kubelogin into the flux provider as of now? E.g. for the kubectl and helm providers I can authenticate in this manner
provider "kubectl" {
  host                   = data.azurerm_kubernetes_cluster.current.kube_config[0].host
  apply_retry_count      = 5
  cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.current.kube_config[0].cluster_ca_certificate)

  # Using kubelogin to get an AAD token for the cluster. Pre-req that kubelogin is installed on jumpbox.
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "kubelogin"
    args = [
      "get-token",
      "--login",
      "workloadidentity",
      "--server-id",
      data.azuread_service_principal.aks_aad_server.application_id,
      "--client-id",
      data.azuread_client_config.current.client_id,
      "--tenant-id",
      data.azurerm_subscription.current.tenant_id,
      "--authority-host",
      "https://login.microsoftonline.com/",
      "--federated-token-file",
      "/home/azbuildadmin/.token"
    ]
  }
}

where  the content in /home/azbuildadmin/.token is an Azure AD JWT retrieved via Github Actions via identity federation
I tried config such as the following, but I assume the token needs to be the exchanged token rather than the AAD JWT, as I get permissions hinting towards permission denied  Error: Could not get NetworkPolicy flux-system/allow-webhooks failed to get API group resources:
provider "flux" {
  git = {
    url = "https://github.com/myorg/myrepo"
  }
  kubernetes = {
    token = "/home/azbuildadmin/.token"
    cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.current.kube_config[0].cluster_ca_certificate)
  }
}

Duplicate of: https://github.com/fluxcd/terraform-provider-flux/issues/406


Add support for executable rule inputs

Instead of rule inputs being fixed values, we should allow them to be arbitrarily executable pieces of code. This will allow definition of custom logic instead of static values.
A few things to consider here.

Performance may be impacted if the logic is intensive.
What data be passed to these inputs and how.
Does this mean a new rule input type (e.g. 'EVAL') and how does that play with the data type construct.

This is also a significant deviation from the current simple approach of Rulette towards rule inputs, viz that of treating them as stored "values". With this, we shift more towards Drools style rule "execution", although we may be able to support both modes.

The executable rules can have some side-effect or are just pure function returning a value?

No side Effects. They will comply with the Rule input contract and use "some" data. What should that data be though, is the question


python3Packages.radio_beam: update to the latest upstream and fix depends

Description of changes
Update python3Packages.radio_beam to the latest upstream, and update/fix dependencies on the way.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 193223 run on x86_64-darwin 1

8 packages failed to build:

python310Packages.aplpy
python310Packages.asdf
python310Packages.pyregion
python310Packages.spectral-cube
python39Packages.aplpy
python39Packages.asdf
python39Packages.pyregion
python39Packages.spectral-cube

12 packages built:

python310Packages.astroquery
python310Packages.pytest-astropy
python310Packages.pytest-astropy-header
python310Packages.pyvo
python310Packages.radio_beam
python310Packages.reproject
python39Packages.astroquery
python39Packages.pytest-astropy
python39Packages.pytest-astropy-header
python39Packages.pyvo
python39Packages.radio_beam
python39Packages.reproject


Thanks a lot

Hi,
just wanted to say thanks. This is super handy as the esp32 ist cheap and the library is easy to use. Gonna build a pedal with multiple switches to change scenes and tracks within ableton live when I practice bass guitar :)

Hi !
thanks a lot, i appreciate! i am happy it can be useful to you !


Support internationalization

Hello,
in some countries we use comma instead of dot as separator.
This is not implemented, could you provide a solution, maybe passing the separator as an argument during init?
Thank you for the great job.

https://github.com/openalloc/SwiftNumberPad/pull/6
Please check


Need better error reporting in inquiry flow / Spike in "Problem sending inquiry" emails

Reports came in early Feb 1, that a partner "registered interest through her personal email and was able to log an inquiry with some artists but not all". @joeyAghion noted that there has been a recent spike in "Problem sending inquiry emails" (these emails get sent when the inquiry flow errors out at any point).
Trying to replicate an error with the same artwork on staging (with both new and old accounts) did not result in an error, which limits our ability to troubleshoot this issue. First step here is to revisit how we deal with client-side error reporting.

Is it originating from here? if so we can get more out of the context
https://github.com/artsy/force/blob/master/analytics/inquiry_questionnaire.js#L137

Did done


There is no transformer_helper in isegm.model.modeling

Hello, I find that there maybe something wrong in your code

Please check out the v1.0 branch: https://github.com/uncbiag/SimpleClick/tree/v1.0/isegm/model/modeling. I am currently refactoring the code in the main branch. Sorry for the confusion.

Close this issue. Free free to reopen it if you still have questions.


Failure to build parallel tests

Environment Information

What platform are you using? (please provide specific distribution/version in summary)

[x] Linux

32 and/or 64 bit?

[x] 64-bit

What build system are you using?

[x] autotools (configure)

Summary of Issue
When running make check with --enable-parallel-tests added to ./configure, I’m having a failure with:
/usr/bin/ld: tst_h_par.o: undefined reference to symbol 'MPI_Wtime'
/usr/lib/openmpi/libmpi.so.12: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

Steps to reproduce the behavior
./configure --prefix=/usr --enable-shared --enable-pnetcdf --enable-parallel-tests
make
make check

EDIT: Sorry, message was sent before being finished.

(Note that I’ve fixed https://github.com/Unidata/netcdf-c/issues/343 / https://github.com/Unidata/netcdf-c/issues/377 by patching <IP_ADDRESS> using https://github.com/Unidata/netcdf-c/commit/a83702834938b23cc2e843589aa223e2024a7e6f)

Excellent; it should be fixed in the 4.5.0 release candidate as well; I'll double check before closing this out.

Corrected, closing this out.

@WardF: Not sure I’ve made myself clear. I faced #343/#377 and fixed those through https://github.com/Unidata/netcdf-c/commit/a83702834938b23cc2e843589aa223e2024a7e6f. But the issue I’m reporting in my initial post here is what I’m facing next, and is not fixed on my side.
If there is a fix in master, please tell. ;)

No problem @ArchangeGabriel .  As I read your post, you are encountering this with <IP_ADDRESS>.  This issue has been fixed in master since it was reported, and is also in the recently released 4.5.0 release candidate 1.  My phrasing was unclear as to whether it was future or present tense.
If you are encountering this in master or 4.5.0-release candidate 1 let me know as it should be fixed; indeed, I can no longer recreate it on my end!

Can you link me to the commit that fixed it? I failed to found any.

(I’ve not tried building with master or the RC yet, will do soon though)

It's the link you provided above, a83702834938b23cc2e843589aa223e2024a7e6f, was merged into master on November 29, 2016, capturing the fix.  A hash for the merge of @opoplawski's pull request is here:  1e39435adb08177fb85b35fd136ef49156329ce3

So I have not made myself clear, sorry about that… Yes, <IP_ADDRESS> is affected by an issue that is fixed through https://github.com/Unidata/netcdf-c/commit/a83702834938b23cc2e843589aa223e2024a7e6f.
I’ve mentioned this initially to explain that I know about it but fixed that locally (by applying this commit on top of <IP_ADDRESS>) before building, but then faced a new failure, which is the issue I’m reporting here.
I’ll try to build the RC tomorrow, but I expect the same issue now.

Oh ok, I just saw your edit to the top level; my mistake and apologies, I do understand this is a new issue now. Let me take a look.

Adding a bit more lines:
depbase=`echo tst_h_par.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
gcc -DHAVE_CONFIG_H -I. -I..  -I../include -I../oc2  -D_FORTIFY_SOURCE=2  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -MT tst_h_par.o -MD -MP -MF $depbase.Tpo -c -o tst_h_par.o tst_h_par.c &&\
mv -f $depbase.Tpo $depbase.Po
/bin/sh ../libtool  --tag=CC   --mode=link gcc  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong  -Wl,-O1,--sort-common,--as-needed,-z,relro -o tst_h_par tst_h_par.o  -lhdf5_hl -lhdf5 -ldl -lm -lz -lcurl
libtool: link: gcc -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z -Wl,relro -o tst_h_par tst_h_par.o  -lhdf5_hl -lhdf5 -ldl -lm -lz -lcurl

(Apparently this could be an issue of linking order or missing library linking. Like MPI?

Let me take a look and see if I can recreate it; once I've got it recreated I should be able to untangle the knot of linking/etc.  I suspect you are right (now that I understand this is a new issue, sorry for taking so long to figure that out!) and neither the RC nor the master fix this.  I will follow up after I have a moment to take a look.

Hi, @ArchangeGabriel
The  messages from your make command indicate gcc is used. To enable parallel I/O feature, you need to specify the MPI compiler at configure command line, such as
CC=mpicc ./configure --enable-parallel --prefix=${H5DIR}

See NetCDF user guide below for full description of command-line options
http://www.unidata.ucar.edu/software/netcdf/docs/getting_and_building_netcdf.html#build_parallel

@wkliao You’re right, thanks. I had an export CC=mpicc but after the ./configure line… I should have been warned by those lines (discovered now after looking back at logs):
checking for ncmpi_create in -lpnetcdf... no
checking Is libpnetcdf version 1.6.0 or later?... yes
configure: WARNING: Cannot link to pnetcdf library, --enable-pnetcdf disabled.

PNetCDF Support:        no

Sorry for the noise, everything is working now.


EthStorage Devs Meeting #84 Agenda

Meeting Info

Jun 11, 2024, 11:00 UTC+8
Duration: 60 minutes
Meeting link
Anyone can comment on the agenda where they would like this to be discussed.

Agenda

Protocol/App/Research progress
BD/Marketing progress

Public testnet campaign

1st campaign result announced, added 3 address
2nd campaign: application: 596 => 4542 (Need to find out filtering criteria); submission: 263 => 463; passed: 229

Rollbridge: add a nginx https server
Upgrade the storage contract on Sepolia
Deploy ethstorage on L2 - WIP

web3:// website implementation.
Vitalikblog uploads blobs to the sepolia network.

Investigate L2 blob tx DDOS risk
Add some test for PRs submitted to optimism

https://github.com/ethstorage/es-node/issues/210

https://github.com/ethstorage/es-node/pull/291

https://github.com/ethstorage/es-node/issues/205
merge or not?

https://github.com/ethstorage/es-node/pull/181
https://github.com/ethstorage/es-node/pull/281

trust setup

Coordinator Guide

Fix issue: (https://github.com/ethstorage/es-node/pull/288, https://github.com/ethstorage/es-node/issues/262)

Chain ID application: Add chain id 100011/110011 for quarkchain l2 (mainnet/testnet)  ethereum-lists/chains#5138

Investigation: https://github.com/ethstorage/es-node/issues/176, risc zero trusted setup, p0tion)

[x] Build & deployment:

yarn firebase login --no-localhost
AWS: remove .terraform folder under backend/aws/ to reduce deploy size
Node 16 out date
Firebase 9.20 has bug

[x] Authentication:

signInToFirebase error FirebaseError: Firebase: Error (auth/account-exists-with-different-credential)
Function v1 and v2 using different roles

[ ] Process:

verification failure
Upload pot larger than 21 would upload fail as its size is 4.5G

Chain ID application: Add chain id 100011/110011 for quarkchain l2 (mainnet/testnet)  ethereum-lists/chains#5138

Public testnet: twitter filter updating

Research on EVM execution layer works of lambdaclass, megaeth lab, risechain
Research on EIPs of EOF https://evmobjectformat.org/
Review code of NethermindEth Intra-block cache and pre-warm PR.

Continue OP's zk-fraud proof work
Research on Binius/based-rollup

Support Nary=4

taiko grants
fuel mou signed


Problem with oneseq_dxsm_128_64 and uniform int distribution

/usr/include/c++/10/bits/uniform_int_dist.h:249:4: error: ‘typedef long unsigned int pcg_detail::engine<long unsigned int, __int128 unsigned, pcg_detail::dxsm_mixin<long unsigned int, __int128 unsigned>, false, pcg_detail::oneseq_stream<__int128 unsigned>, pcg_detail::default_multiplier<__int128 unsigned> >::result_type’ is inaccessible within this context

Problematic commit: ffd522e7188bef30a00c74dc7eb9de5faff90092

My own abstraction is broken. Sorry


Check that image is up-to-date in ServicePodWorker

Fixes #2171

If I understood #2171 correctly, that would be a service with something like image: foo:latest, and then kontena service deploy should actually do a docker pull foo:latest and re-create the service container if an updated image has been pushed for the latest tag.
I don't see that happening here? It seems like the ServicePodWorker would need to ensure_image to actually pull the latest image for that tag before checking if the container image is out of date.
In terms of the smallest possible change, that might imply dropping the service_container_outdated? check and just unconditionally calling ensure_running to invoke the existing Kontena::ServicePods::Creator logic whenever the deploy_rev changes... Longer-term, I think that the log should be moved into some kind of ImageManager that deals with them....
BTW also fix #2154 while you're at it?

@SpComb good point, added image pull before comparing.

Refactored comparison logic to ServicePodWorker and wrote some additional tests. Also noticed that comparing timestamps is not very reliable because image might actually change so that timestamp is still "older" than what container has => changed image comparison to check id's.


v5.3.0  flow control Causes QPS fluctuation

Bug Report

What version of TiKV are you using?

TiKV
Release Version:   5.3.0-alpha
Edition:           Community
Git Commit Hash:   e16bf9c0f86d26fd41335f9714adbdd387862ee7
Git Commit Branch: heads/refs/tags/v5.3.0
UTC Build Time:    2021-11-11 10:05:28
Rust Version:      rustc 1.56.0-nightly (2faabf579 2021-07-27)
Enable Features:   jemalloc mem-profiling portable sse protobuf-codec test-engines-rocksdb cloud-aws cloud-gcp
Profile:           dist_release

What operating system and CPU are you using?

NVME Disk
centos 7.5
model name      : Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz * 2

Steps to reproduce

What did you expect?
1， The server is busy message is prompted in tikv-detail > ERROR rows
2，QPS smooth operation
What did happened?
tikv in 15:45 add parameters rocksdb.rate-bytes-per-sec: "110MB"

/assign Connor1996

/severity major

@buggithubs Before the config setting, what's the RocksDB total write flow (write flush + compaction flush)? I think it's likely much more than 110MB/s. I guess "110MB" is too small.

Change this to enhancement as it's hard to make it steady as the compaction flow is restricted to a low level.


Getting 'no scheduler for type' in deploying a container from the REST API

Hi
I am new to shipyard and I am trying to run the a deploy command that has worked via the CLI now through the REST API. Below is an example with the service key and IP address mocked. When I run the command I get the following:
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Date: Thu, 09 Jul 2015 18:14:24 GMT
Content-Length: 23
Proxy-Connection: keep-alive
Keep-Alive: 60
Via: HTTP/1.1 proxy1732
no scheduler for type
CLI command
shipyard run --name dockerregistry.tools.nikecloud.com/gatling –-vol --label test1 /opt/gatling/user-files:/opt/gatling/user-files –-arg /opt/gatling/user-files/heartbeat.sh
REST API call
curl -D- 'http://<IP_ADDRESS>:8080/api/containers' -X POST -H 'X-Service-Key:<PHONE_NUMBER>' -H 'Content-Type: application/json' --data '{ "name": "dockerregistry.tools.nikecloud.com/gatling","labels": ["test1"],"args": ["/opt/gatling/user-files/heartbeat.sh"], "links": { "/opt/gatling/user-files":"/opt/gatling/user-files" }}'
Am I using one of the keys improperly in the JSON body?

This should work.  It may be a bug in the request for the cpu / mem
setting.  I'll take a look.
On Thu, Jul 9, 2015 at 3:36 PM, rydhar<EMAIL_ADDRESS>wrote:

Hi
I am new to shipyard and I am trying to run the a deploy command that has
worked via the CLI now through the REST API. Below is an example with the
service key and IP address mocked. When I run the command I get the
following:
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Date: Thu, 09 Jul 2015 18:14:24 GMT
Content-Length: 23
Proxy-Connection: keep-alive
Keep-Alive: 60
Via: HTTP/1.1 proxy1732
no scheduler for type
CLI command
shipyard run --name dockerregistry.tools.nikecloud.com/gatling –-vol
--label test1 /opt/gatling/user-files:/opt/gatling/user-files –-arg
/opt/gatling/user-files/heartbeat.sh
REST API call
curl -D- 'http://<IP_ADDRESS>:8080/api/containers' -X POST -H
'X-Service-Key:<PHONE_NUMBER>' -H 'Content-Type: application/json' --data '{
"name": "dockerregistry.tools.nikecloud.com/gatling","labels":
["test1"],"args": ["/opt/gatling/user-files/heartbeat.sh"], "links": {
"/opt/gatling/user-files":"/opt/gatling/user-files" }}'
Am I using one of the keys improperly in the JSON body?
—
Reply to this email directly or view it on GitHub
https://github.com/shipyard/shipyard/issues/571.

I think I was able to specify "type":"service" in the JSON body. I realized that I was misusing the links I think because I am getting the following error:
curl -D- 'http://<IP_ADDRESS>:8080/api/containers' -X POST -H 'X-Service-Key:<PHONE_NUMBER>' -H 'Content-Type: application/json' --data '{ "name": "dockerregistry.tools.nikecloud.com/gatling", "type": "service", "labels": ["test1"], "args": ["/opt/gatling/user-files/heartbeat.sh"],"links": { "/opt/gatling/user-files": "/opt/gatling/user-files" } }'
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Date: Mon, 13 Jul 2015 23:56:40 GMT
Content-Length: 80
Proxy-Connection: keep-alive
Keep-Alive: 60
Via: HTTP/1.1 proxy1732
500 Internal Server Error: Could not get container for /opt/gatling/user-files
Per the CLI docs : "--link: link another container"
I actually want to use : "--vol: container volumes (/host/path:/container/path or /container/path)"
There is no example in your REST API docs for this. Is it simply replacing "links" with "vol"?

Figured it out: "volumes": ["/opt/gatling/user-files:/opt/gatling/user-files"]

Thanks for the feedback and update.  Starting in v3 the API is the direct
Docker Remote API so it gets better.
On Mon, Jul 13, 2015 at 8:25 PM, rydhar<EMAIL_ADDRESS>wrote:

Closed #571 https://github.com/shipyard/shipyard/issues/571.
—
Reply to this email directly or view it on GitHub
https://github.com/shipyard/shipyard/issues/571#event-354913325.


Allow request attributes to be incorporated into person attributes

https://issues.jasig.org/browse/PERSONDIR-92
See issue for motivation.
NOTE:  This work is built on top of PERSONDIR-88 and PR https://github.com/Jasig/person-directory/pull/8

Need to release this so merging it in.  I still welcome any comments.


"rootDir" and "include" properties in tsconfig.json conflict with eachother

TypeScript Version:  3.5.1

Search Terms: rootDir, include, exclude, tsconfig.json
Code
{
  "compilerOptions": {
    "target": "ES2018",
    "module": "commonjs",
    "declaration": true,
    "sourceMap": true,
    "outDir": "./build",
    "rootDir": "./src",

    "strict": true,

    "noUnusedLocals": true,

    "esModuleInterop": true,

    "experimentalDecorators": true,
    "emitDecoratorMetadata": true,
  },
  "include": [
    "vendor"
  ]
}

Expected behavior: Compiler compiles contents of src folder and vendor folder into build folder.
Actual behavior: Compiler ONLY compiles vendor folder into build folder while also complaining with the following error:
File 'c:/Users/KG/code/project/vendor/vend.ts' is not under 'rootDir' 'c:/Users/KG/code/obi/src'. 'rootDir' is expected to contain all source files.

I tried a few other things and got really weird behavior. If all source files need to be in rootDir then why does the compiler compile all my project's source files (including unit tests) that are outside of rootDir?
I also noticed that if I use include, the compiler only ever compiles whatever is in include and not in addition to rootDir. Not only is this not documented anywhere, the tsconfig.json docs are outright wrong:  https://www.typescriptlang.org/docs/handbook/tsconfig-json.html
With the current behavior, the example tsconfig.json is nonsensical, since using include already excludes everything outside of it, what's the exclude then good for?

i also don't complire 'rootDir'

i also don't complire 'rootDir'


YARD module - fixes docstring when doc'd

comment is for PrivateConstantHandler
prev code
#COMMENT
module YARD
  module Handlers
    module Ruby
      class PrivateConstantHandler

changed code code
module YARD
  module Handlers
    module Ruby
      #COMMENT
      class PrivateConstantHandler

Good catch, thanks!

It wasn't the comment, it was the 'Defined in:'
You know the feeling, the 'I haven't been anywhere near that code' feeling...


Restriction of .NET functions to a single namespace prevents usage of some C# features

The restriction that a function's assembly can only have a single namespace prevents the usage of some C# features that embeds attributes in the resulting assembly:

readonly structs
(I will update this list as more features that embed code into the resulting assembly are found.)

My proposed solution is to ignore types that have attributes named EmbeddedAttribute and/or CompilerGeneratedAttribute in the namespace counting code:
https://github.com/kubeless/runtimes/blob/31b42a99b0827170ce36833a4dcce95e09bfdda3/stable/dotnetcore/src/Kubeless.Core/Invokers/CompiledFunctionInvoker.cs#L32-L37

Hi @GGG-KILLER,
The #58 probably solves that. Once this is merged, can you please check if that is the case?

Hi @allantargino, unfortunately I no longer work for the company that was using this project. I will however attempt to test this in a VM

Ok @allantargino, today I tried to setup the new 3.1 runtime on master but it proved to be nontrivial so I'm unable to test.
The following should be enough to test if that fixed it or not though:
using Kubeless.Functions;

public class module
{
    private readonly struct Box<T>
    {
        public T Boxed { get; }

        public Box(T boxed)
        {
            this.Boxed = boxed;
        }
    }

    public object handler(Event k8Event, Context k8Context)
    {
        return new Box<object>(k8Event.Data);
    }
}

@GGG-KILLER  I'm not sure I understand the actual problem about single namespace restriction (I don't see multiple namespaces in the test function?). But since I have dotnetcore3.1 runtime from https://github.com/kubeless/runtimes/pull/58 in my kubeless deployment, I tested your function (with a slight modification required by the runtime) and it works.

This is the exact function definition that I used for testing:
using System;
using System.Threading.Tasks;
using Kubeless.Functions;

public class module
{
    private readonly struct Box<T>
    {
        public T Boxed { get; }

        public Box(T boxed)
        {
            this.Boxed = boxed;
        }
    }

    public async Task<object> handler(Event k8Event, Context k8Context)
    {
        return new Box<object>(k8Event.Data);
    }
}

Do you think this test is enough to say that the pull request solved this issue?

I'm not sure I understand the actual problem about single namespace restriction (I don't see multiple namespaces in the test function?)

Actually, when you compile that code, the readonly struct is marked with an IsReadonlyAttribute, which in turn, is generated during compilation and stored on the System.Runtime.CompilerServices namespace inside your assembly (and that IsReadonlyAttribute in turn is marked with the EmbeddedAttribute which is stored on the Microsoft.CodeAnalysis namespace):

But yes, since your test passed, it's enough to say the pull request has solved this issue.


some changes in css file

I made some changes in the CSS file and make a beautiful website,
I  hope so you like it

please check my changes  and merged in your website


Add "Go to Home Page" in services context menu

Description
Add a new function to go to the home page in the added services, this function is linked with #870
Checklist:

[x] My pull request is properly named
[x] The changes respect the code style of the project ($ npm run lint)
[x] I tested/previewed my changes locally

@raicerk thank you for your contribution! 🙏
@all-contributors please add gautamsi for code

you're welcome, I just have one question!  Add gausami as a collaborator? Not me?

@kytwb you're welcome, I just have one question!  Add gausami as a collaborator?  I do not?

As a contributor, this adds you in the thank you list in the README of the project.
We can still revert 05b6020 if you prefer.

The bot pull request failed by the way. I will take another look at it as soon as possible.
Please clarify if you do not wish to be included in the contributors' list in the README, I don't think I clearly understood you. We also send out an invite to the GitHub organization to developers who contributed code, so feel free to accept it if you want to display the organization badge on your profile.
Thank you again for your contribution! 😄

@all-contributors please add gautamsi for code

@kytwb I guess @raicerk should have been added and not me

Oh, haha, that is why the bot pull request did not add anything! 😅
Off to a slow comeback guys, bear with me! 😄 💪
@all-contributors please add raicerk for code

@all-contributors please add raicerk for code

thank you @kytwb, my english is very bad, I wanted to say what he said @gautamsi 😅 thank you guys :smile:


Add Separate templates for different service types

Description:

In the current completion and suggestion implementation, we suggest http service by default. Instead of having one template for services it would be ideal to suggest possible service template types.
Same as the service, when suggesting resources, we need to analyze the enclosing service type and suggest resource templates accordingly.

Affected Versions:

v0.962.0

Created https://github.com/ballerina-platform/ballerina-lang/issues/10218 to track resource level suggestions


jvthumb is not updating

Expected Behavior
Sorted Actress name and pic would be saved to the list for future use
Current Behavior
The Actress info is not saving. Believe this is due to R18 becomes unavailable
Steps to Reproduce (for bugs)
Your Environment

Module version used: 2.5.10
Operating System and PowerShell version: Windows 10, powershell 7.3.3

Raised in the discord chat, hope this is getting on to the list of issues to be fixed

Should be usable now. Check the latest version.

It is starting to get new entries, but I don't believe the entries are correct however. Example below
"Wakuimito","","Wakuimito","和久井美兎","wakui_mito.jpg",
"Ashinahonoka","","Ashinahonoka","芦名ほのか","asina_honoka.jpg",
"Takahatachihana","","Takahatachihana","高畑ちはな","takahata_tihana.jpg",
"Hafu Yuria","Hafu","Yuria","葉風ゆりあ","hakaze_yuria.jpg",
"Kashiwagi Konatsu","Kashiwagi","Konatsu","柏木こなつ","kasiwagi_konatu.jpg",
"Kitayamayū","","Kitayamayū","北山憂","kitayama_yuu.jpg",
"Otsu Alice","Otsu","Alice","乙アリス","otu_arisu.jpg",
"Aratārina","","Aratārina","新ありな","arata_arina.jpg",
"Minase Akari","Minase","Akari","皆瀬あかり","minase_akari.jpg",
"Iorihinano","","Iorihinano","伊織ひなの","iori_hinano.jpg",
"Yuzuriha Karen","Yuzuriha","Karen","楪カレン","yuzuriha_karen.jpg",
"Mai Tsubasa","Mai","Tsubasa","つばさ舞","tubasa_mai.jpg",
"Unpai","","Unpai","うんぱい","unpai.jpg",
"Amamiya Hanan","Amamiya","Hanan","天宮花南","amamiya_kanan.jpg",
"Kamikirei","","Kamikirei","神木麗","kamiki_rei.jpg",

What version and scraper are you currently using?

Sorry, How do I check? I only updated the GUI to 2.5.13Sent from my iPhoneOn 7 May 2023, at 06:10, seeyabye @.***> wrote:
What version and scraper are you currently using?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

If you can share your settings, it would be better.
From the look for the jvThumbs.csv you shared, it looks fine except for names where the last name and first name are merged. This is highly dependent on the source you are scraping from. I will need to know which scraper you are using before I further debug it.

Just let me know what you need. I will share any files. I will provide them as soon as possible as I am traveling these days.Sent from my iPhoneOn 7 May 2023, at 08:12, seeyabye @.***> wrote:
If you can share your settings, it would be better.
From the look for the jvThumbs.csv you shared, it looks fine except for names where the last name and first name are merged. This is highly dependent on the source you are scraping from. I will need to know which scraper you are using before I further debug it.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Attached my settings file, is that what you need?
￼

On 7 May 2023, at 8:15 AM, Henry Wong @.***> wrote:
Just let me know what you need. I will share any files. I will provide them as soon as possible as I am traveling these days.
Sent from my iPhone

On 7 May 2023, at 08:12, seeyabye @.***> wrote:

If you can share your settings, it would be better.
From the look for the jvThumbs.csv you shared, it looks fine except for names where the last name and first name are merged. This is highly dependent on the source you are scraping from. I will need to know which scraper you are using before I further debug it.
—
Reply to this email directly, view it on GitHub https://github.com/jvlflame/Javinizer/issues/358#issuecomment-1537249884, or unsubscribe https://github.com/notifications/unsubscribe-auth/AW6FYUHS6OZB3PKTXFZ4ZNLXE3SIBANCNFSM6AAAAAAVII4F74.
You are receiving this because you authored the thread.

{
"throttlelimit": 3,
"location.input": "X:\",
"location.output": "X:\",
"location.thumbcsv": "",
"location.genrecsv": "",
"location.uncensorcsv": "",
"location.historycsv": "",
"location.tagcsv": "",
"location.log": "",
"scraper.movie.aventertainment": false,
"scraper.movie.aventertainmentja": true,
"scraper.movie.r18dev": true,
"scraper.movie.dmm": false,
"scraper.movie.dmmja": true,
"scraper.movie.jav321ja": true,
"scraper.movie.javbus": false,
"scraper.movie.javbusja": true,
"scraper.movie.javbuszh": false,
"scraper.movie.javdb": false,
"scraper.movie.javdbzh": false,
"scraper.movie.javlibrary": false,
"scraper.movie.javlibraryja": true,
"scraper.movie.javlibraryzh": false,
"scraper.movie.mgstageja": true,
"scraper.movie.tokyohot": false,
"scraper.movie.tokyohotja": true,
"scraper.movie.tokyohotzh": false,
"scraper.option.dmm.scrapeactress": true,
"scraper.option.idpreference": "id",
"scraper.option.addmaleactors": false,
"match.minimumfilesize": 0,
"match.includedfileextension": [
".asf",
".avi",
".flv",
".m4v",
".mkv",
".mp4",
".mov",
".rmvb",
".wmv"
],
"match.excludedfilestring": [
"^.-trailer",
"^.*-5\."

],
"sort.metadata.requiredfield": [
"id",
"coverurl",
"genre",
"maker",
"releaseDate",
"title"
],
"sort.metadata.priority.actress": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.alternatetitle": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.coverurl": [
"DmmJa",
"JavlibraryJa",
"JavbusJa",
"Jav321Ja",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.description": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.director": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.genre": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.id": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.contentid": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.label": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.maker": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.releasedate": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.rating": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.runtime": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.series": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.screenshoturl": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.title": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"sort.metadata.priority.trailerurl": [
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
"emby.url": "http://<IP_ADDRESS>:8096",
"emby.apikey": "",
"javlibrary.baseurl": "http://www.javlibrary.com",
"javlibrary.browser.useragent": "",
"javlibrary.cookie.cf_chl_2": "",
"javlibrary.cookie.cf_chl_prog": "",
"javlibrary.cookie.cf_clearance": "",
"javlibrary.cookie.session": "",
"javlibrary.cookie.userid": "",
"javdb.cookie.session": "",
"proxy.enabled": false,
"proxy.host": "",
"proxy.username": "",
"proxy.password": "",
"web.favorites.genre": [],
"web.favorites.tag": [],
"web.sort.recurse": false,
"web.sort.strict": false,
"web.sort.interactive": true,
"web.sort.update": false,
"web.sort.force": false,
"web.sort.preview": false,
"web.sort.recursedepth": 10,
"web.sort.confirm": false,
"web.sort.manualsearch.aventertainment": false,
"web.sort.manualsearch.aventertainmentja": true,
"web.sort.manualsearch.dmm": false,
"web.sort.manualsearch.dmmja": true,
"web.sort.manualsearch.jav321ja": true,
"web.sort.manualsearch.javbus": false,
"web.sort.manualsearch.javbusja": true,
"web.sort.manualsearch.javbuszh": false,
"web.sort.manualsearch.javdb": false,
"web.sort.manualsearch.javdbzh": false,
"web.sort.manualsearch.javlibrary": false,
"web.sort.manualsearch.javlibraryja": true,
"web.sort.manualsearch.javlibraryzh": false,
"web.sort.manualsearch.mgstageja": true,
"web.sort.manualsearch.r18dev": true,
"web.sort.manualsearch.tokyohot": true,
"web.sort.manualsearch.tokyohotja": true,
"web.sort.manualsearch.tokyohotzh": true,
"web.theme": "dark",
"admin.log": true,
"admin.log.level": "",
"admin.updates.check": true
}

It's not scraping from R18Dev at the moment.
You need to include R18Dev in your sort.metadata.priority section
Seems like getting a thumbnail from DmmJa doesn't work right now, will need to take a look at that. R18Dev should be working though.
...
"sort.metadata.priority.actress": [
"R18Dev",
"DmmJa",
"Jav321Ja",
"JavbusJa",
"JavlibraryJa",
"MGStageJa",
"TokyoHotJa",
"AVEntertainmentJa"
],
...

Added R18Dev and tried. Still the same.
Name error, missing thumbnail path
I have also updated to 2.5.15

Could you share what video code are you using it with?
Also, what do you get when you use the following in powershell?
Get-R18DevData -Url <url_link_to_one_of_r18dev_video_page>

Stars-734  I see the path for thumbnail using powershell but name is still wrong. I am using GUI though previously.
Get-R18DevData -Url https://r18.dev/videos/vod/movies/detail/-/id=1stars734r/
Source        : r18dev
Url           : https://r18.dev/videos/vod/movies/detail/-/id=1stars734r/
ContentId     : 1stars734r
Id            : STARS734
Title         : A Subordinate With Too Much Style Is Turned Into A Meat
Slave By Blackmailing Him. The Lingerie Training Of The
Power Harassment Boss In The Company! Rei Kamiki
Description   :
ReleaseDate   : 2023-04-13
ReleaseYear   : 2023
Runtime       : 140
Director      : Zaku Arai
Maker         : SOD Create
Label         : SOD star
Series        :
Actress       : @{LastName=; FirstName=Kamikirei; JapaneseName=神木麗;
ThumbUrl=https://pics.dmm.co.jp/mono/actjpgs/kamiki_rei.jpg}
Genre         : {Embarrassment, Shame, Office Lady, Big Tits…}
CoverUrl      : https://pics.dmm.co.jp/mono/movie/1stars734r/1stars734rpl.jp
g
ScreenshotUrl :
TrailerUrl    : https://cc3001.dmm.co.jp/litevideo/freepv/1/1st/1stars734/1s
tars734_mhb_w.mp4

The thumbnail path looks correct from the output. Are you doing a fresh sort? It should output the right thumbnail url to jvThumbs.csv
As for the name, it looks correct from r18.dev's data. In fact on r18, there was no information on which is the FirstName and which is the LastName. It's just Romaji=Kamikirei

Have you tried the latest version? 2.5.17?
The GUI just calls the CLI version of Javinizer for most of the operation, so the results should be the same
I ran with the mostly default settings, and I had no issue getting jvThumbs.csv to update with the correct thumbnail.

Unless I am misunderstanding something, but at this point I am still confused what is the issue.

After updating to 2.5.17 and deleting the existing entry in jvthumb. its getting the thumbs path.
I guess the name is r18 bug instead. In terms of manual amendment, even after updating in GUI for the name, it still won't store in jvthumb. I assume the save happened at point of sorting and not at the point of saving the video. Either way, thanks for sorting the thumb path out. Appreciate all your help on this.


fix anchor target label

If I am not wrong, it would be a bug when RPN is using (background_label = 0), leading to all samples are positive.
Not sure that RPN in model zoo is benchmarked with this bug or not.

Hi @thangvubk ,
The background_label does not affect the training because the filled part is weighted zero when calculating the loss. But this indeed could affect the performance in some situation. Thanks for fixing this bug.

Hi @ZwwWayne. I don't think that the loss for filled are filtered out by label-weights since the fill value also overwrite negative labels (which have label-weights of 1). You can double-check that.

See here fill of unmap only affects the out-of-range parts but does not affect the assigned parts. And from the performance, the results are reasonable.

Ah. I got it. Thank you :D


fix: bug in jump

off by one

#89  👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rphmeier and the rest of your teammates on  Graphite


test-branch/execute-job/quoted-authorization/91c124fb-0a7d-434c-92fa-913e36ded944

PR for testing 738355af5f79e51448c1bcd61bd99ba4b1d0e03e

/canonical/self-hosted-runners/run-workflows 738355af5f79e51448c1bcd61bd99ba4b1d0e03e


faiss: Fix missing phase hooks, set make -j correctly

Description of changes
This package lacks runHooks in its phases (), and also has make invocation without correct -j, making the build take much longer than necessary.
See https://nixos.org/manual/nixpkgs/stable/#sec-stdenv-phases

When overriding a phase, for example installPhase, it is important to start with runHook preInstall and end it with runHook postInstall

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

you can use buildFlags to specify the build targets and then eliminate the buildPhase and then use postBuild to do the python only wheel build. likewise, in installPhase remove the call to make and move the contents to postInstall sans runHook.

@paparodeo I will leave that type of cleanup to somebody else, I'm only fixing what's wrong here due to being out of time for this weekend.

you can use buildFlags to specify the build targets and then eliminate the buildPhase and then use postBuild to do the python only wheel build. likewise, in installPhase remove the call to make and move the contents to postInstall sans runHook.

@paparodeo I will leave that type of cleanup to somebody else, I'm only fixing what's wrong here due to being out of time for this weekend.

luckily i already did the work and provided a diff -- you just need to apply it.

you can use buildFlags to specify the build targets and then eliminate the buildPhase and then use postBuild to do the python only wheel build. likewise, in installPhase remove the call to make and move the contents to postInstall sans runHook.

@paparodeo I will leave that type of cleanup to somebody else, I'm only fixing what's wrong here due to being out of time for this weekend.

ok. superseded by https://github.com/NixOS/nixpkgs/pull/320419

@nh2 thanks; from the looks of it, we can and should merge @paparodeo's version directly, but I do want to comment that I don't think there's anything wrong with incremental improvements and with people only having limited time, and your PR is appreciated

@SomeoneSerge Yes all good, I completely agree that doing it as #320419 does it is an even better way.


When next release?

hello is there a schedule for the next release?
since the old uploader will get deprecated soon, I relay on latest changes not published jet
(v0.1.15)

@drazisil-codecov do you have any time estimation about next tagging?

As soon as I can get https://github.com/codecov/uploader/pull/577 approved. :)

oh big THANKS for the clear answer :) :+1:

Well, technically, it will be when https://github.com/codecov/uploader/pull/582 gets merged, but that's generally the easiest part. Short-staffed in the engineering group due to Covid wandering around the world hunting down our team members.
Appreciate the patience.

This was resolved with #587


Add BullMQ + Redis and prototype notifications queue

This is an initial attempt to add BullMQ and Redis (i.e., ioredis) to our code.
I've done the most basic thing possible vs. making this so it will scale.  At the moment we have:

app/queues/* which holds all of our queues.
app/queues/notifications.server.ts which is our first queue, dealing with sending user notifications asynchronously.  We'll add more as we go, but this can provide a template.
The Queue and Worker are both co-located in the same file, and in the same process.  Down the road, we can break the worker into its own process or container(s).  But I didn't want to do this at first, since it will complicate things significantly.

The way this queue would be used is like this.  Imagine we want to send a message to a user.  The code would be:
import notificationsQueue from '~/queues/notifications.server';

...

// Send a new user a welcome message. Create a unique job id for this job
const jobId = `${user.email}-welcome`;
// Add the job to the queue, passing the data it will need. We have to wait for
// the job to get put into Redis, but not for the email to be sent.
await notificationsQueue.add(jobId, {
  emailAddress: user.email,
  subject: "Welcome to my.custom.domain!",
  message: "We're thrilled to welcome you to my.custom.domain...",
});

The job can now be handled by the Worker, which will attempt to send it at some point in the near future.  If it fails, it will retry the job 3 more times, waiting progressively longer each time.  We can configure all of this.
I don't have anywhere in the code to hook into this yet, so it's not easy to test.

I added graceful shutdown for the workers and redis.
This won't build yet, I need to get the nodemailer stuff landed first, and then I'll iterate on this some more.
Thanks for the first review @dadolhay, appreciated!

I've updated this to use the work @TD just landed for notifications.  I also added a /dev page we can use to create test UI and added a button to send a notification.
Once you click the button, it adds a job to the notifications queue.  This will get picked up by the worker and an email sent.  You can check it in MailHog via http://localhost:8025:

To make this work, I also had to fix our CSP policy in helmetjs.  It breaks hot-reloads in the UI due to unsafe inline script injection.  We don't want that in production, but in development it's nice so the app responds to code changes in real-time.
What I'm doing here on the /dev page could be copied by other people, who need to try out UI-triggered actions (e.g., @Genne23v with DNS, @dadolhay with Certs).

Added a fix for the node EventEmitter max listeners warning you get when using Redis/BullMQ:
(node:35293) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 17 error listeners added to [Commander]. Use emitter.setMaxListeners() to increase limit

By default, node.js warns if an event emitter has more than 10 listeners; however, the way Redis works, you need way more than that all the time.  The recommended fix for this by the BullMQ maintainer is to bump the default value up.  In Telescope, we've used 32 as our default and not had any issues.


Adding date in the folder name?

Just wondering if there's a way to create a folder e.g. yyyy-mm-dd-[FTName | kebabcase]

Right now that's not possible but I really like the idea. Will get around to implementing that in the future


Api: Adding usePathParams/usePathParam helpers

There was an existing usePath but it returned an array of path segments (split on /). If you define your route like this:
{
  'POST /user/{id}': 'function/user/update.handler',
}

Then id will be present in event.pathParameters. In this case you could also get the id with:
const path = usePath();
const id = path[1];

But it's much nicer to use the 'named' path param by getting it from event.pathParameters. Especially if you are trying to re-use code where the id might not always be in the same position in the path.

Good addition! Right now we're focused on our sst2 branch so if you make the PR there we can add it in

Closing this because #2299 replaces it (for sst2)


Float typed array now supports int assignment

fix to #66789
Bugsquad edit: Fix #66789

Needs rebase, provided the conflicts don't make this approach impossible.
CC @vonagam


Allow overriding the default language by passing a query parameter

See the discussion in #453

After this has been dormant for way too long I took another stab at this in #830 , let's hope this makes it to the finish line.


[WIP] feat(v2): no-JS collapsible DocSidebar

Motivation
#3030
PoC no-JS collapsible sidebar using hidden checkboxes. code is pretty sloppy at the moment but it is working. would probably need to cleaned up significantly.
some functionality may still be missing but it should look mostly similar to the existing one.
live demo at https://duanwilliam.github.io/docusaurus-purecss-sidebar/
WIP
Have you read the Contributing Guidelines on pull requests?
yes
Test Plan
Disable JS in browser and check

Thanks for your POC @duanwilliam
I'm not going to follow-up on this because I think we should have "lazy sidebars" instead, and reduce the HTML markup rendered on the server to avoid rendering collapsed content.
This is important for perf on sites rendering very large sidebars.
I think it's not useful anymore to allow uncollapsing categories of the sidebar with CSS/HTML only.
Instead, we'll solve the no-JS category uncollapsing by making the category a link targeting a page (category index page, or first doc of that category...), and the target page will have the category uncollapsed by default.


0.12.36: pep517 build fails

Describe the bug
Look slike pep617 based build fails
To Reproduce
Just execute python3 -sBm build -w --no-isolation in source build tree.
Expected behavior
pep517 based build should work.
Screenshots

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
<string>:50: DeprecatedWarning: install_npm is deprecated as of 0.8 and will be removed in 1.0. Use `npm_builder` and `wrap_installers`
<string>:55: DeprecatedWarning: create_cmdclass is deprecated as of 0.8 and will be removed in 1.0. "
Use `wrap_installers` to handle prebuild steps in cmdclass.
Use `get_data_files` to handle data files.
Use `include_package_data=True` and `MANIFEST.in` for package data.

/usr/lib/python3.8/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running egg_info
creating bqplot.egg-info
writing bqplot.egg-info/PKG-INFO
writing dependency_links to bqplot.egg-info/dependency_links.txt
writing requirements to bqplot.egg-info/requires.txt
writing top-level names to bqplot.egg-info/top_level.txt
writing manifest file 'bqplot.egg-info/SOURCES.txt'
reading manifest file 'bqplot.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'tests'
adding license file 'LICENSE'
writing manifest file 'bqplot.egg-info/SOURCES.txt'
* Building wheel...
<string>:50: DeprecatedWarning: install_npm is deprecated as of 0.8 and will be removed in 1.0. Use `npm_builder` and `wrap_installers`
<string>:55: DeprecatedWarning: create_cmdclass is deprecated as of 0.8 and will be removed in 1.0. "
Use `wrap_installers` to handle prebuild steps in cmdclass.
Use `get_data_files` to handle data files.
Use `include_package_data=True` and `MANIFEST.in` for package data.

/usr/lib/python3.8/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running bdist_wheel
running jsdeps
yarn not found, ignoring yarn.lock file
`npm` unavailable.  If you're running this command using sudo, make sure `npm` is available to sudo
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 351, in <module>
    main()
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 333, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 249, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 413, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 398, in _build_with_temp_dir
    self.run_setup()
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 62, in <module>
  File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/jupyter_packaging/setupbase.py", line 726, in run
    [self.run_command(cmd) for cmd in cmds]
  File "/usr/lib/python3.8/site-packages/jupyter_packaging/setupbase.py", line 726, in <listcomp>
    [self.run_command(cmd) for cmd in cmds]
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/jupyter_packaging/setupbase.py", line 421, in run
    self.command.run()
  File "/usr/lib/python3.8/site-packages/jupyter_packaging/setupbase.py", line 345, in run
    c.run()
  File "/usr/lib/python3.8/site-packages/jupyter_packaging/setupbase.py", line 442, in run
    raise ValueError(("missing files: %s" % missing))
ValueError: missing files: ['share/jupyter/nbextensions/bqplot/index.js', 'share/jupyter/labextensions/bqplot/package.json']

ERROR Backend subprocess exited when trying to invoke build_wheel

Additional context
Here is list of installed modules in build env

Package                       Version
----------------------------- -----------------
alabaster                     0.7.13
anyio                         3.6.2
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
asttokens                     2.2.1
attrs                         22.2.0
Babel                         2.12.1
backcall                      0.2.0
beautifulsoup4                4.11.2
bleach                        6.0.0
build                         0.9.0
cffi                          1.15.1
charset-normalizer            3.0.1
comm                          0.1.2
coverage                      7.2.1
debugpy                       1.6.6
decorator                     5.1.1
defusedxml                    0.7.1
deprecation                   2.1.0
distro                        1.8.0
docutils                      0.19
entrypoints                   0.4
exceptiongroup                1.0.0
executing                     1.2.0
fastjsonschema                2.16.1
gpg                           1.18.0-unknown
html5lib                      1.1
idna                          3.4
imagesize                     1.4.1
importlib-metadata            6.0.0
importlib-resources           5.12.0
iniconfig                     2.0.0
ipykernel                     6.20.2
ipython                       8.6.0
ipython-genutils              0.2.0
ipywidgets                    8.0.3
jedi                          0.18.2
Jinja2                        3.1.2
json5                         0.9.12
jsonschema                    4.17.3
jupyter_client                7.4.9
jupyter_core                  5.2.0
jupyter_packaging             0.12.3
jupyter-server                1.23.3
jupyter-sphinx                0.4.0
jupyterlab                    3.5.1
jupyterlab-pygments           0.1.2
jupyterlab_server             2.18.0
jupyterlab-widgets            3.0.2
libcomps                      0.1.19
MarkupSafe                    2.1.2
matplotlib-inline             0.1.6
mistune                       2.0.5
nbclassic                     0.4.8
nbclient                      0.7.2
nbconvert                     7.2.9
nbformat                      5.7.3
nbval                         0.9.6
nest-asyncio                  1.5.6
notebook                      6.5.2
notebook_shim                 0.2.2
numpy                         1.24.2
packaging                     23.0
pandas                        1.5.2
pandocfilters                 1.5.0
parso                         0.8.3
pep517                        0.13.0
pexpect                       4.8.0
pickleshare                   0.7.5
pip                           22.3.1
pkgutil_resolve_name          1.3.10
platformdirs                  2.6.0
pluggy                        1.0.0
ply                           3.11
prometheus-client             0.16.0
prompt-toolkit                3.0.36
psutil                        5.9.2
ptyprocess                    0.7.0
pure-eval                     0.2.2
pycparser                     2.21
Pygments                      2.14.0
pyrsistent                    0.19.3
pytest                        7.2.2
python-dateutil               2.8.2
pytz                          2022.4
pyzmq                         24.0.1
requests                      2.28.2
rpm                           4.17.0
SciPy                         1.8.1
Send2Trash                    1.8.0
setuptools                    65.6.3
six                           1.16.0
sniffio                       1.2.0
snowballstemmer               2.2.0
soupsieve                     2.4
Sphinx                        6.1.3
sphinx-rtd-theme              1.1.1
sphinx-thebe                  0.2.0
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2.dev20230202
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib.jquery          3.0.0
sphinxcontrib-jsmath          1.0.1.dev20230128
sphinxcontrib-qthelp          1.0.3.dev20230128
sphinxcontrib-serializinghtml 1.1.5
stack-data                    0.6.2
terminado                     0.17.1
tinycss2                      1.2.1
tomli                         2.0.1
tomlkit                       0.11.6
tornado                       6.2
traitlets                     5.8.1
traittypes                    0.2.1
urllib3                       1.26.12
wcwidth                       0.2.6
webencodings                  0.5.1
websocket-client              1.5.1
wheel                         0.38.4
widgetsnbextension            4.0.5
zipp                          3.15.0

You need yarn in the build environment

Testeda and it works now.
Thank ypu 👍