Function groups with namespaces - not implemented

When importing a function group which is in a namespace, importing in another system is not working. The function modules are not imported correctly. (Source code and includes are imported correctly but can't be activated)

It seems that using namespaces is still a todo in the function group logic

@Gekke-Jakke can this issue be closed?

@larshp Yes it can! Everything is working fine now!


Too many GET requests on opening the Authorize API modal window in API Authorization tab in application edit view

Describe the issue:
$subject

How to reproduce:

Expected behavior:

Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: [e.g., IS 5.10.0, IS 5.9.0]
OS: [e.g., Windows, Linux, Mac]
Database: [e.g., MySQL, H2]
Userstore: [e.g., LDAP, JDBC]

Optional Fields
Related issues:

Suggested labels:

This is already tracked with https://github.com/wso2/product-is/issues/19204, hence closing.


Issue with GitHub/LinkedIn login on the Portal

When myself and @austin0 try to login using Github/LinkedIn it requests our email address. This is some kind of a bug that is not major but should be looked into further. This has been tested with a number of different users and it only has happened to us till this date.
Example outcome of signing in with LinkedIn bellow:

Edit: This does not happen every single time on GitHub.

Marika, Charlotte, and Gilbert encountered the same issue with Github account
The worst thing is if the email address was already registered in SN, no way to create the account. I guess users need to login  SN by the email and then manually link the social account. This experience is a resistant of using the cloud-hosted guides.

Marika, Charlotte, and Gilbert encountered the same issue with Github account
The worst thing is if the email address was already registered in SN, no way to create the account. I guess users need to login  SN by the email and then manually link the social account. This experience is a resistant of using the cloud-hosted guides.

Linked in login is functionally working but still has cosmetic issues that are proving difficult to solve.
When logging in via a SSO provider the login page reformats itself to remove said providers, this causes our styling to break.


Add external timestamp to general settings

The SettingsExternalTimestamp struct contains Enabled, Type, Frequency. The Timestamp type options are SMPTE, IRIG and CameraTime.

/// Signal Edge <----
public enum TimestampType


Support for Octave

Supporting Octave would allow running unit tests (incl. coverage) in CI...

implemented and merged by 49d139fd3884f2400c0e6d850a579e5c01849a09


Notification Area support

Implement the Notification Area (system tray) into Program Manager. Determine the most appropriate way to display it.

Duplicate of #3


specimen: anatomical_site

This corresponds to sampled_tissue in the GA4GH Biosample.
Add it using the cda_biosample_factory class.

It looks like NCIt has UBERON terms mapped already?  So we can just use anatomical_site directly with no conversion?
https://ncit.nci.nih.gov/ncitbrowser/ConceptReport.jsp?dictionary=NCI_Thesaurus&version=23.07e&code=C198975&ns=ncit


Implement SystemParam for WorldId

Objective
Fixes #7736
Solution
Implement the SystemParam trait for WorldId
Changelog

WorldId can now be taken as a system parameter and will return the id of the world the system is running in

bors r+


updates for latest rust changes. (derive, ToCStr)

Thanks for the PR :heart: I think this has already been handled by #36.


Is MySQL an option?

Am I reading this right?  Is this for SQLite only, or can this script also be run from MySQL?
Hopefully it's both.  It looks pretty nice, but my server only has MySQL.  Thanks!

Hi,
it is supposed to be used with sqlite so you wouldn't be needing to install mysql and work with just php running.
You can change it to work with mysql, but don't you have the module for SQLite or just don't want to use it ?

SQLite is not available on this server, so I have to use MySQL or MariaDB.

You were correct — it was there.  I guess I was working off old information.
I got it installed and it looks pretty nice so far.  I like the small footprint.  Thanks!

Glad everything worked out! :)


Add support for Proxy Protocol

This PR adds support for Proxy Protocol in Purpur.
More info in Paper PR, which is still open: https://github.com/PaperMC/Paper/pull/7710

Possible problem, if paper merged this then we have to maintain a config migration for god knows how long on this.

Right, is it better to add config to Paper? Not to Purpur?

I'll be honest I'm not certain, would prefer advice from others

IMO low-level changes such as this are more fit in Paper

I agree

As discussed, this feature is a better fit for Paper. If your upstream PR is not accepted, this PR can be revisited!


Remove TabBarController as it's added method is not used.

Remove AlphaWallet > UI > TabBarController and replace all instances where it is used with UITabBarController.

I think this issue could be closed, there is no TabBarController on master

@Mac3n thanks!


Minimum and recommended hardware and software requirements

In the qlever script #562 i am trying to gather some intelligence on the environment along what i understood are the minimum and recommmended hardware and sofware requirements. I'd love to know the "official" requirements so that the documentation might state these and the script might do checks on the feasibility of an environment.
So far I understand that a tested environment is something like:

OS: Ubuntu 20.04 LTS
RAM: 128 GB (recommended) - X GB (minimum)
Disk: 4 TB (recommended maybe rotating ) - X TB (mimimum)
ulimit -Sn set to 2^20=1024*1024=1048576 recommended - minimum depends on size of data to be indexed - 1 file as per  configured number of triples per file

For the qlever script (which might need renaming to e.g. qlever-demo) the following software is needed:
needed software (Ubuntu)
docker → /usr/bin/docker ✅
top → /usr/bin/top ✅
df → /usr/bin/df ✅
jq → /usr/bin/jq ✅
lsb_release → /usr/bin/lsb_release ✅
free → /usr/bin/free ✅
needed software (MacOS)
docker → /usr/local/bin/docker ✅
top → /usr/bin/top ✅
df → /bin/df ✅
jq → /opt/local/bin/jq ✅
sw_vers → /usr/bin/sw_vers ✅
i intend to use jq for modifying the json config files via command line and included it to the list

I want to add that the hardware requirements highly depend on the dataset. Your recommendations mirror our experiences with indexing and running the full Wikidata. If your dataset is smaller, of course you can use less RAM and disk drive.
Concerning the OS: QLever should run natively on Ubuntu 18.04 and newer (we have Dockerfiles building on 18.04 LTS, 20.04 LTS and 21.10, so you should be able to manually perform the steps that these Dockerfiles do, to run QLever outside of Docker.
When running QLever inside Docker, also many different operating systems should work. I think there might be some limitations on a minimal version of docker and the linux kernel that is used by docker (which is the native kernel on a linux host OS, on Windows and MacOs it depends on the docker implementation, I am no expert there).


refactor: separate pattern factory from patterns

Here's one that might be a little more controversial: I anticipate that all the PatternX::from_node() methods will always remain tied to TreeSitter, because they construct patterns from concrete Node types. I could maybe simplify them a little by using NodeWithSource, but I'm afraid it will be infeasible to make them abstract enough to use impl AstNode later on.
So instead, I propose to split the factory functions from the patterns and create a pattern_factory module for them, which is what this PR does. I've only created factory functions for Accessor and Step so far, and I've also moved the compiler (which directly uses TreeSitter of course) into the pattern_factory directory. I've moved auto_wrap.rs too, since it's only used by compiler.rs and step.rs.
This creates a bit of a circular dependency between the pattern module and the pattern_factory module, but that should be temporary. If we agree on this approach, I'll continue moving the functions out of the other patterns until pattern doesn't depend on pattern_factory anymore. And of course, the end goal is still for pattern to not depend on TreeSitter at all anymore, so eventually it can be moved wholesale into the future grit-core-patterns crate.
PS.: This PR appears large, but it's really almost all updated import statements and a little bit of moving around of code.

Summary by CodeRabbit

New Features

Introduced a new module pattern_factory to enhance pattern handling capabilities.
Added constants for index values and default file names to improve standardization across patterns.

Refactor

Streamlined import statements across various pattern-related files for improved clarity and maintainability.
Restructured the Accessor struct and renamed Key to AccessorKey for better code readability.
Consolidated imports under super and pattern_factory to maintain consistency in the codebase.
Simplified and refactored the implementation of the Step struct, removing unnecessary pattern checks.

Style

Reorganized imports and adjusted module structures across numerous files to align with coding standards and enhance code organization.

@arendjr Please request review when this is ready (or mark as draft).

I don’t think GitHub lets me request reviews by handle on this repo (though I can request re-review when someone’s left one already), but it’s ready for review, @morgante.


acpaquette

The Readme file is currently lengthy and complex, making it challenging for newcomers to comprehend how to contribute effectively to the project.
To enhance accessibility and clarity, I propose relocating the Contributing section to the beginning of the Readme file and streamlining its content where feasible.
I kindly request assistance from anyone willing to dedicate time to address this issue and improve the project's accessibility for all contributors. Thank you!

Hi all, I am a developer from the USGS and am interested in contributing to the software as it seems like a good fit for managing bibliographies in research papers.

@caiton1 Could you provide some insight into how the contributors section could either be rearranged or updated to further increase clarity? Is there any section in particular people are getting lost in?

Closed by #102


Support for logging to Graylog

I have started using WeedFS recently and really love it. It solved the problem I am trying to solve for quite some time with extremely easy setup.
We use graylog for all kinds of application and server logging. One thing I'd like to see in WeedFS is support for graylog.
Unfortunately my skills in Go are next to none. Good news is there is already a Go client available for Graylog if someone wants to take a stab at it:
https://github.com/Graylog2/go-gelf
thanks.

This request is very specific. I guess this could be done by some generic log redirecting tool later.

Graylog takes any number of types of logging, including syslog. Someone could go through and use a go syslog package to add a flag in the command line for "log to syslog". If i recall, this one worked OK: https://github.com/mcuadros/go-syslog, and https://github.com/nathanaelle/syslog5424 also seems like it would work.

There has been not enough interest in this.


Expense Splitter extension added

Description
Added an expense splitter which will take input of who paid how much money and calculate and tell who owes who how much.
Fixes:  #2594

Type of change

[X] New feature (non-breaking change which adds functionality)

Checklist:

[X] I have made this from my own
[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings

ATTACH SCREEN-SHOTS / DEPLOYMENT LINK

I'll create a new PR with the changes since i am getting merge conflicts in this branch.


faster-bootstrap/scripts/service_control.sh - should implement stop

the rhel_provisioner.sh calls service_control firewalld stop, but stop isn't recognized. It should be implemented.

Thanks! We'll investigate and fix this.

Fixed with fbf11f225a87e5bccd5f88eecc6064aa2eb5c0e3


Support with Meteor

Hello, there is no support for meteor since Jan 2015. (Last commit jsdom). Is It possibile to do some thing ?

I am currently using it in Meteor 1.5. It works, but I get a server warning in startup, that an npm module canvas is missing

I removed all NodeJS deps from Konva core. Probably the issue is fixed.
Add more info if you still have issues.


Clean up dependencies

Removes unnecessary babel packages, adds needed devDependencies, and gitignores node_modules 😨 .
Related to https://github.com/folio-org/ui-plugin-find-instance/pull/4 and https://github.com/folio-org/ui-plugin-find-user/pull/36.

thanks @cherewaty


Resource Icons clusters of 4

Upon installing this the current version of this mod (and trying the previous release) I've found that instead of resource icons appearing as they should, each icon appears as a cluster of 4 in a square. The top left is always the correct icon, however the other 3 are completely different resource icons. Any help in rectifying this would be appreciated. No other mods installed, and reverting to vanilla fixes.

There is a fix below in the 'Closed' section that I missed.


bug in crm_config_cmd

the name of the "vm_options" used for crm_config_cmd was not coherent:

the vm-mgr cli uses "add_crm_config_cmd"
the vm-manager livrary code uses "add_crm_config_cmd"
the ansible module uses "crm_config_cmd"

That makes this option work on the cli, but not with the ansible module.
This commit makes it coherent to that everybody uses "crm_config_cmd":

in the vm-mgr cli we copy "add_crm_config_cmd" to "crm_config_cmd"
in the vm-manager library code we use "crm_config_cmd" instead of "add_crm_config_cmd"

the name of the "vm_options" used for crm_config_cmd was not coherent:

the vm-mgr cli uses "add_crm_config_cmd"
the vm-manager livrary code uses "add_crm_config_cmd"
the ansible module uses "crm_config_cmd"

That makes this option work on the cli, but not with the ansible module.
This commit makes it coherent to that everybody uses "crm_config_cmd":

in the vm-mgr cli we copy "add_crm_config_cmd" to "crm_config_cmd"
in the vm-manager library code we use "crm_config_cmd" instead of "add_crm_config_cmd"


[Bug]: Stat Speed Mechanic

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
While attempting to create a passive skill that increases movement speed by 1.5x times I've encountered a problem using the stat mechanic. Modifying movements speed by a multiplier of 2 has the desired effect. However when using decimal values such as 1.5 the movement speed doesn't change at all. When I tried changing the operation from multiply-percentage to add-number it will cause an error to be thrown in console. Here is the pastebin of the error when using the add-number operation. https://pastebin.com/E0G3N2Rs
Expected Behavior
No response
Steps To Reproduce

Create a skill that modifies movement speed using the Stat mechanic. Use multiply percentage and set the amount to a decimal like 1.5.
Run the skill.

Environment
- Server Software: paper-1.18.1-76.jar(happened with 1.17 as well)
- ProSkillAPI Version: ProSkillAPI-<IP_ADDRESS>-ALPHA
- ProMCCore Version: ProMCCore-<IP_ADDRESS>-ALPHA
- Plugins: BetonQuest, Chunky, Citizens, CurseOfVanishing, Essentials, LibsDisguises, MythicMobs, ProMCCore, ProSkillAPI, ProtocolLib, Skript

latest.log
No response
Anything else?
No response

When I tried changing the operation from multiply-percentage to add-number it will cause an error to be thrown in console. Here is the pastebin of the error when using the add-number operation. https://pastebin.com/E0G3N2Rs

Regarding the add mechanic, walk speed is a default of 0.2. If you're adding more to the value to increase it beyond the bounds of [-1, 1] it will throw an error. Just keep that in mind.


feat: impl BorshSerialize/BorshDeserialize/BorshSchema for char

resolves #247

FYI, Unicode character is 21-bit number. This could be optimised to encode as 3-byte little endian number.  (Another alternative is to use UTF-8 which would be more in line with how strings are serialised).

@mina86 Thanks for pointing that out. I have also raised a concern (https://github.com/near/borsh-rs/issues/247#issuecomment-1772172152) that maybe we should not implement it in borsh unless we strictly define what char is, and given that I have never needed to serialize char nor have I heard a request to do that, I would suggest we don't spend time here.


structogram: create concept for rendering

at some point, I have to decide how to handle height predictions

finally done after so many attempts


Example 4.3 Gambler's Problem Cannot Reproduce Book's Plotting

https://github.com/superxingzheng/test/blob/master/reinforcement_learning/GamblersProblem/main.cpp
This is my implementation. Not only the tied best actions can cause different plotting. If we add the +1 reward gained from the transition to the Goal ( capital 100) state, the value of capital 99 state will approach 2.0 not 1.0.

https://github.com/ShangtongZhang/reinforcement-learning-an-introduction/issues/42

You are right. Thanks! After this fix, my code does the same as yours. With 20 value iterations, the best action plotting looks similar to the book's.


Option answer advice

Show an advice text if an answer with advice attribute is selected.

Implemented in 28292dc and 53abf49.


[Swift-6.0-nightly] Add missing python libs for AL2

The Swift REPL requires that the python libraries are available or it will fail to launch. Adding the missing dependency on AmazonLinux 2.

CC @shahmishal
CC @parkera

@swift-ci please test


Port Open CL POW to Rust

Summary
As mentioned on stream, the Open CL POW generation is currently not available within the Rust node. The code should be available already
What problem would be solved by this feature?
POW generation takes a lot of time and effort - currently on the CPU if I understand correctly. So Open CL should help with this and speed up the process.
Are there any previous requests for this feature?
Not for RsNano
Do you have a suggested solution?
Porting the existing implementation and making it available to the Rust Node.
If this feature is approved, would you be willing to submit a pull request with the solution?
I would
Possible solution
@simpago you mentioned it only in passing during your Stream - maybe you can provide a few starting points and extend the issue with the concrete requirements
Supporting files
No response

This is the repo with the distributed work server.


One test from boost libraries failed to run after boost submodule beast updated from 949fa9 to 1a80dd

Environment:
VS 2017 + Windows Server 2016
Issue description:
We build and run test for boost and we found one test failed to run after boost submodule beast updated from 949fa9 to 1a80dd boostorg/beast@936e885 Could you please take a look?
Reproduce steps:

git clone -c core.autocrlf=true --recursive https://github.com/boostorg/boost.git D:\Boost\src
open a VS 2017 x64 command prompt and browse to D:\Boost\src
.\bootstrap
.\b2 headers variant=release --build-dir=..\out\x64rel address-model=64
.\b2 variant=release --build-dir=..\out\x64rel address-model=64
.\b2 -j4 variant=release --build-dir=..\out\x64rel libs\beast\test

ErrorMessage:
compile-c-c++ ..\out\x64rel\boost\bin.v2\libs\beast\msvc-14.1\release\debug-symbols-on\link-static\threading-multi\test\asio.obj
asio.cpp
D:\Boost\src\boost/asio/ssl/detail/openssl_types.hpp(20): fatal error C1083: Cannot open include file: 'openssl/conf.h': No such file or directory
log_x64_test_124.log

OpenSSL is now required to build the tests, I'll add a note to the README and build instructions.

Thanks for your reply! Can you provide detailed steps to build the tests with Openssl?

use vcpkg install openssl and then re-run your steps. You might need this in your user-config.jam:
import os ;
using msvc ;
local OPENSSL_ROOT = [ os.environ OPENSSL_ROOT ] ;
project
  : requirements
    <include>$(OPENSSL_ROOT)/include
    <variant>debug:<library-path>$(OPENSSL_ROOT)/debug/lib
    <variant>release:<library-path>$(OPENSSL_ROOT)/lib
  ;

Thanks for you information! We've builded the tests pass with Openssl on latest version. So close it.


Can OPT or Blenderbot be used for commercial?

❓ Questions and Help
Before asking:

search the issues.
search the docs.

Can OPT or Blenderbot be used for commercial?
Can OPT 30B be used for commercial?
What is your question?
Code

What have you tried?
What's your environment?

metaseq Version (e.g., 1.0 or master):
PyTorch Version (e.g., 1.0)
OS (e.g., Linux):
How you installed metaseq (pip, source):
Build command you used (if compiling from source):
Python version:
CUDA/cuDNN version:
GPU models and configuration:
Any other relevant information:

Hey Philip, commercial use of OPT (and by extension BlenderBot) is restricted.
The license can be found here: https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/MODEL_LICENSE.md
Feel free to re-open if you have any follow-up questions.


AVRO-1363 Fix the C# parsing of a union schema

Fix the C# parsing of a union schema that contains  duplicate names but in different namespaces. The full name is now used when parsing the schema. This is identical to my previous pull request on this topic, but I've rebased to apache/master and squashed all the commits to a single commit.

@rdblue  Sorry it took so long to get this done. Hopefully there are no more issues merging this into apache/master.


Update prompt appears with HTML for new version when behind firewall requiring authentication

This just happens because I am behind a firewall requiring authentication to access external websites. Mapbox Studio shows the encoded HTML of the authentication page instead of the new version string.

Should note this also happens if I am using the latest version, presumably because there is a comparison of the two version strings and they always differ because of the HTML. Maybe a regex check on the latest version string and don't show the prompt if it fails.


Close console with Ctrl-D

I can't figure out how to configure this. Is it even possible?

Switch to vi-mode, it's in there by default.

In emacs mode, it is also the default, but you can bind it yourself with:
Set-PSReadlineKeyHandler -Key Ctrl+d -Function DeleteCharOrExit

@srdubya How do I change to Vi-mode. It complains no matter what I try to give it.

@srdubya Do you by any chance have a good ps1-file?

Here's what I use.  Format-Bash is a module I created to sort items column wise.  PS may already provide a way of doing this, but since I didn't/don't know about it I wrote my own.  I'd be happy to share it if you're interested.
function prompt {
Write-Host( [Environment]::NewLine + $(get-location) )
return "> "
}
if( $host.Name -eq 'ConsoleHost' ) {
Import-Module PSReadline
<##>
Set-PSReadlineOption -EditMode Vi
Set-PSReadlineOption -DingTone 2400
Set-PSReadlineOption -DingDuration 30
Set-PSReadlineOption -ViModeIndicator Prompt
Set-PSReadlineOption -ExtraPromptLineCount 1
Import-Module Format-Bash
<##>
}
Function lsf($dir) {
Get-ChildItem $dir | Format-Bash
}
Function lsa {
Get-ChildItem -Hidden | Format-Bash -All
}
Function ll {
Get-ChildItem -verbose
}
Function which($name) {
Get-Command $name | Select-Object -ExpandProperty Definition
}
Function whence($name) {
Get-Command -Syntax $name
}
$VIMPATH    = "C:\Program Files (x86)\Vim\vim74\vim.exe"
Set-Alias vi $VIMPATH
Function Edit-Profile
{
vi $profile
}
Function Edit-Vimrc {
vi $home_vimrc
}
$env:VISUAL = "vi"
$env:PATH = $env:Path + ";C:\Program Files\Git\bin"

Well, my main issue with this is that Set-PSReadlineOption -EditMode Vi is not accepted. I get:
Set-PSReadlineOption : Cannot bind parameter 'EditMode'. Cannot convert value "Vi" to type "Microsoft.PowerShell.EditMo
de". Error: "Unable to match the identifier name Vi to a valid enumerator name. Specify one of the following enumerator
 names and try again:
Windows, Emacs"
At C:\Users\jimmy\Documents\WindowsPowerShell\profile.ps1:9 char:32
+ Set-PSReadLineOption -EditMode Vi
+                                ~~
    + CategoryInfo          : InvalidArgument: (:) [Set-PSReadlineOption], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.SetPSReadlineOption

I solved it now. It seems that the Vi-mode is not in the default Windows 10 version of PSReadline. This makes it necessary to to a
PS> Install-Module -Name PSReadline

contrary to what the documentation says.
I also put
Import-Module PSReadLine -MinimumVersion 1.2

in my ps1, since it seems to prefer version 1.1.


Connect VideoDataOutputDelegate

Hi, how to connect VideoDataOutputDelegate?
It isnt work...
I trying:
fileprivate lazy var captureController: CaptureViewController = {
let controller = CaptureViewController(inputs: [.video], outputs: [.stillImage, .metadata([AVMetadataObjectTypeQRCode])])
controller.captureDelegate = self
controller.captureManager.dataOutputDelegate = self
    return controller
}()

Please help

Hey! Let me check up on this. I just realized I didn't expose that delegate directly through CaptureViewController.

Hey @ronysteel, so the problem is that .videoData is missing from your CaptureViewController's outputs. Add that in your initializer and you should be good to go!
Soon, I'm going to go ahead and change the implementation to fix this bug though! I'll leave the issue open until I ship the changes.

@gonzalonunez , I added it, bu isn't work.

open var outputs = [CaptureSessionOutput.videoData, CaptureSessionOutput.stillImage] {
didSet {
didChangeInputsOrOutputs()
}
}

after that I added /// captureManager.dataOutputDelegate = self in my controller,

// MARK: - VideoDataOutputDelegate
extension CameraViewController: VideoDataOutputDelegate {

func captureManagerDidOutput(_ sampleBuffer: CMSampleBuffer) {
    print(1)
}

}

... Nothing

Hey! So, I'm confirmed that the following works:
  private lazy var captureVC: CaptureViewController = {
    let vc = CaptureViewController(inputs: [.video], outputs: [.stillImage, .videoData, .metadata([AVMetadataObjectTypeQRCode])])
    vc.captureDelegate = self
    vc.captureManager.dataOutputDelegate = self
    vc.captureManager.metadataOutputDelegate = self
    return vc
  }()

  //MARK: VideoDataOutputDelegate

  func captureManagerDidOutput(_ sampleBuffer: CMSampleBuffer) {
    print("\(Date()) received sample buffer...")
  }

Does this work for you? Let me know 🙂

Thank you)) nice work)

No prob! Just FYI, among other things, I'll be changing how the input/outputs work (by adding associated values) to make sure that delegates are provided when necessary at the time you decide to add a particular input or output.
Thanks for you patience!


handle the offline case

Fixes https://github.com/lvwerra/trl/issues/409
Properly not call whoami() method that requires internet connection and raise a proper warning if that's the case.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


fix for multi gpu training

Fixes for multi-GPU training with train_network.py.
I haven't fully tested it yet, so I'm submitting it as a draft.
Open it when everything is tested.
It would be helpful if you could let me know if you have noticed anything.

train_network.py のマルチGPUトレーニングの修正です。
まだ完全にテストしていないのでドラフトです。正常に動作したら開きます。
何かお気づきの点ありましたら教えていただけると助かります。

I have 2 4090s, so I can test this to train a LoRA. Have I to add a parameter to the training, or do I have to select multi-gpu in accelerate config?

@Panchvzluck
Very helpful.
Run accelerate config and answer:
Which type of machine are you using? -> multi-GPU
How many GPU(s) should be used for distributed training? ->
Other than that, the default settings are fine.
The problem that cannot be tested is that the model output during training is partially corrupted.
(If you run networks/check_lora_weights.py, some will contain nan)
https://github.com/kohya-ss/sd-scripts/pull/247/files#diff-62cf7de156b588b9acd7af26941d7bb189368221946c8b5e63f69df5cda56f39R581-R584
I think this fix fixes it, but I haven't been able to test it.

When trying to train, I get this issue (I'm on Windows 10)
Traceback (most recent call last):
  File "G:\acc\sd-scripts\train_network.py", line 642, in <module>
    train(args)
  File "G:\acc\sd-scripts\train_network.py", line 108, in train
    accelerator, unwrap_model = train_util.prepare_accelerator(args)
  File "G:\acc\sd-scripts\library\train_util.py", line 1965, in prepare_accelerator
    accelerator = Accelerator(gradient_accumulation_steps=args.gradient_accumulation_steps, mixed_precision=args.mixed_precision,
  File "G:\acc\sd-scripts\venv\lib\site-packages\accelerate\accelerator.py", line 308, in __init__
    self.state = AcceleratorState(
  File "G:\acc\sd-scripts\venv\lib\site-packages\accelerate\state.py", line 150, in __init__
    torch.distributed.init_process_group(backend="nccl", **kwargs)
  File "G:\acc\sd-scripts\venv\lib\site-packages\torch\distributed\distributed_c10d.py", line 602, in init_process_group
    default_pg = _new_process_group_helper(
  File "G:\acc\sd-scripts\venv\lib\site-packages\torch\distributed\distributed_c10d.py", line 727, in _new_process_group_helper
    raise RuntimeError("Distributed package doesn't have NCCL " "built in")
RuntimeError: Distributed package doesn't have NCCL built in
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 1436) of binary: G:\acc\sd-scripts\venv\Scripts\python.exe

My accelerate configs were:

This Machine
multi-GPU
How many different machines will you use (use more than 1 for multi-node training)?: 1
Do you wish to optimize your script with torch dynamo?[yes/NO]: no
Do you want to use DeepSpeed? [yes/NO]: no
Do you want to use FullyShardedDataParallel? [yes/NO]: no
Do you want to use Megatron-LM ? [yes/NO]: no
How many GPU(s) should be used for distributed training?: 2
What GPU(s) (by id) should be used for training on this machine as a comma-seperated list? [all]:0,1 (tried with all and had the same issue)
fp16

After looking into it, it seems to be an error that occurs in the windows environment.
Can you add the code below to the top of train_network.py and try it?
import os
os.environ["PL_TORCH_DISTRIBUTED_BACKEND"] = "gloo"

@ddPn08 At the end, managed to fix it by replacing all "nccl" string values into "gloo" in sd-scripts\venv\Lib\site-packages\accelerate\state.py
I tried
import os
os.environ["PL_TORCH_DISTRIBUTED_BACKEND"] = "gloo"

before and sadly that doesn't fix it.
Then, started to train a LoRA, and for some reason it seems to be duplicated? It is that a expected behaviour?
Also, my settings were
$learning_rate = 0.0005
$lr_warmup_ratio = 0.05
$train_batch_size = 5
$num_epochs = 2
$save_every_n_epochs=1
$scheduler="cosine_with_restarts"
$network_dim=128
$text_encoder_lr=1e-5
$unet_lr=0.00025
$mts = [int]($total / $train_batch_size * $num_epochs)
$lr_warmup_steps = [math]::Round($mts * $lr_warmup_ratio)

accelerate launch --num_cpu_threads_per_process 16 train_network.py `
    --keep_tokens=1 `
    --cache_latents `
    --enable_bucket `
    --max_data_loader_n_workers=1 `
    --persistent_data_loader_workers `
    --shuffle_caption `
    --pretrained_model_name_or_path=$ckpt `
    --train_data_dir=$image_dir `
    --reg_data_dir=$reg_dir `
    --resolution=768,768 `
    --output_dir=$output `
    --train_batch_size=$train_batch_size `
    --lr_scheduler=$scheduler `
    --learning_rate=$learning_rate `
    --lr_warmup_steps=$lr_warmup_steps `
    --max_train_steps=$mts `
    --use_8bit_adam `
    --xformers `
    --mixed_precision=bf16 `
    --save_every_n_epochs=$save_every_n_epochs `
    --seed=31337 `
    --save_precision=bf16 `
    --logging_dir="" `
    --caption_extension=.txt `
    --save_model_as=safetensors `
    --network_module=networks.lora `
    --text_encoder_lr=$text_encoder_lr `
    --unet_lr=$unet_lr `
    --network_dim=$network_dim `
    --network_alpha=$network_dim `
    --clip_skip=2 `

But then it does 4 epochs, andl the output look like this
Measuring folders:
        monagenshin: 20 repeats * 132 images = 2640
Total images with repeats: 2640
Max training steps 2640 / 5 * 2 = 1056
NOTE: Redirects are currently not supported in Windows or MacOs.
[W ..\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [kubernetes.docker.internal]:29500 (system error: 10049 - La direcci≤n solicitada no es vßlida en este contexto.).
[W ..\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [kubernetes.docker.internal]:29500 (system error: 10049 - La direcci≤n solicitada no es vßlida en este contexto.).
prepare tokenizerprepare tokenizer

Use DreamBooth method.
Use DreamBooth method.
prepare images.
found directory D:\Mona training\20_monagenshin contains 132 image files
prepare images.
found directory D:\Mona training\20_monagenshin contains 132 image files
2640 train images with repeating.2640 train images with repeating.

0 reg images.
0 reg images.no regularization images / 正則化画像が見つかりませんでした

no regularization images / 正則化画像が見つかりませんでした
[Dataset 0]
  batch_size: 5
  resolution: (768, 768)
  enable_bucket: True
  min_bucket_reso: 256
  max_bucket_reso: 1024
  bucket_reso_steps: 64
  bucket_no_upscale: False

  [Subset 0 of Dataset 0]
    image_dir: "D:\Mona training\20_monagenshin"
    image_count: 132
    num_repeats: 20
    shuffle_caption: True
    keep_tokens: 1
    caption_dropout_rate: 0.0
    caption_dropout_every_n_epoches: 0
    caption_tag_dropout_rate: 0.0
    color_aug: False
    flip_aug: False
    face_crop_aug_range: None
    random_crop: False
    is_reg: False
    class_tokens: monagenshin
    caption_extension: .txt

[Dataset 0]
  batch_size: 5
  resolution: (768, 768)
  enable_bucket: True
  min_bucket_reso: 256
  max_bucket_reso: 1024
  bucket_reso_steps: 64
  bucket_no_upscale: False

  [Subset 0 of Dataset 0]
    image_dir: "D:\Mona training\20_monagenshin"
    image_count: 132
    num_repeats: 20
    shuffle_caption: True
    keep_tokens: 1
    caption_dropout_rate: 0.0
    caption_dropout_every_n_epoches: 0
    caption_tag_dropout_rate: 0.0
    color_aug: False
    flip_aug: False
    face_crop_aug_range: None
    random_crop: False
    is_reg: False
    class_tokens: monagenshin
    caption_extension: .txt

[Dataset 0][Dataset 0]

loading image sizes.loading image sizes.

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 132/132 [00:02<00:00, 46.34it/s]

make buckets
make buckets
number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む）
bucket 0: resolution (512, 1024), count: 140
number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む）
bucket 1: resolution (576, 960), count: 600bucket 0: resolution (512, 1024), count: 140

bucket 1: resolution (576, 960), count: 600bucket 2: resolution (576, 1024), count: 180

bucket 3: resolution (640, 896), count: 1460bucket 2: resolution (576, 1024), count: 180

bucket 4: resolution (704, 832), count: 100bucket 3: resolution (640, 896), count: 1460

bucket 4: resolution (704, 832), count: 100bucket 5: resolution (768, 768), count: 40

bucket 5: resolution (768, 768), count: 40
bucket 6: resolution (832, 704), count: 20bucket 6: resolution (832, 704), count: 20

bucket 7: resolution (896, 640), count: 80
bucket 7: resolution (896, 640), count: 80bucket 8: resolution (1024, 512), count: 20

bucket 8: resolution (1024, 512), count: 20
mean ar error (without repeats): 0.024412930218512426
mean ar error (without repeats): 0.024412930218512426
prepare accelerator
prepare accelerator
[W ..\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [kubernetes.docker.internal]:29500 (system error: 10049 - La direcci≤n solicitada no es vßlida en este contexto.).
[W ..\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [kubernetes.docker.internal]:29500 (system error: 10049 - La direcci≤n solicitada no es vßlida en este contexto.).
[W ..\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [kubernetes.docker.internal]:29500 (system error: 10049 - La direcci≤n solicitada no es vßlida en este contexto.).
[W ..\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [kubernetes.docker.internal]:29500 (system error: 10049 - La direcci≤n solicitada no es vßlida en este contexto.).
Using accelerator 0.15.0 or above.Using accelerator 0.15.0 or above.

load StableDiffusion checkpoint
load StableDiffusion checkpoint
loading u-net:loading u-net:  <All keys matched successfully><All keys matched successfully>

loading vae: loading vae:<All keys matched successfully>
<All keys matched successfully>
Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel: ['vision_model.encoder.layers.15.mlp.fc2.weight', 'vision_model.encoder.layers.14.self_attn.out_proj.bias', 'vision_model.encoder.layers.20.self_attn.k_proj.weight', 'vision_model.encoder.layers.17.self_attn.v_proj.bias', 'vision_model.encoder.layers.15.self_attn.k_proj.weight', 'vision_model.encoder.layers.15.self_attn.v_proj.bias', 'vision_model.encoder.layers.8.mlp.fc2.weight', 'vision_model.encoder.layers.18.mlp.fc1.weight', 'vision_model.encoder.layers.22.mlp.fc2.bias', 'vision_model.encoder.layers.15.self_attn.k_proj.bias', 'vision_model.encoder.layers.1.mlp.fc1.bias', 'vision_model.encoder.layers.5.self_attn.v_proj.weight', 'vision_model.encoder.layers.5.mlp.fc1.bias', 'vision_model.encoder.layers.3.mlp.fc1.weight', 'vision_model.encoder.layers.14.self_attn.k_proj.weight', 'vision_model.encoder.layers.11.self_attn.v_proj.weight', 'vision_model.encoder.layers.4.self_attn.v_proj.bias', 'vision_model.encoder.layers.17.mlp.fc2.bias', 'vision_model.encoder.layers.0.self_attn.v_proj.bias', 'vision_model.encoder.layers.2.layer_norm2.bias', 'vision_model.encoder.layers.14.layer_norm2.bias', 'vision_model.encoder.layers.23.mlp.fc2.weight', 'vision_model.encoder.layers.13.mlp.fc1.weight', 'vision_model.encoder.layers.14.self_attn.v_proj.bias', 'vision_model.encoder.layers.10.mlp.fc1.bias', 'vision_model.encoder.layers.11.self_attn.out_proj.weight', 'vision_model.encoder.layers.13.mlp.fc2.weight', 'vision_model.encoder.layers.8.layer_norm1.weight', 'vision_model.encoder.layers.18.mlp.fc2.weight', 'vision_model.encoder.layers.8.mlp.fc1.weight', 'vision_model.encoder.layers.1.layer_norm1.bias', 'vision_model.encoder.layers.0.layer_norm2.weight', 'vision_model.encoder.layers.8.layer_norm1.bias', 'vision_model.encoder.layers.5.layer_norm1.bias', 'vision_model.encoder.layers.10.layer_norm1.weight', 'vision_model.encoder.layers.10.layer_norm2.weight', 'vision_model.encoder.layers.9.layer_norm1.bias', 'vision_model.encoder.layers.15.mlp.fc2.bias', 'vision_model.encoder.layers.19.self_attn.q_proj.bias', 'vision_model.encoder.layers.20.layer_norm2.bias', 'vision_model.encoder.layers.0.self_attn.q_proj.weight', 'vision_model.encoder.layers.16.layer_norm1.weight', 'vision_model.encoder.layers.2.mlp.fc2.weight', 'vision_model.encoder.layers.10.layer_norm1.bias', 'vision_model.encoder.layers.23.self_attn.q_proj.weight', 'vision_model.encoder.layers.13.mlp.fc2.bias', 'vision_model.encoder.layers.16.self_attn.v_proj.weight', 'vision_model.encoder.layers.0.self_attn.q_proj.bias', 'vision_model.encoder.layers.10.self_attn.q_proj.bias', 'vision_model.encoder.layers.7.mlp.fc2.weight', 'vision_model.encoder.layers.10.self_attn.q_proj.weight', 'vision_model.encoder.layers.13.self_attn.out_proj.weight', 'vision_model.encoder.layers.2.self_attn.out_proj.bias', 'vision_model.encoder.layers.4.layer_norm1.weight', 'vision_model.encoder.layers.16.mlp.fc1.bias', 'vision_model.encoder.layers.19.self_attn.k_proj.bias', 'vision_model.encoder.layers.22.layer_norm2.bias', 'vision_model.encoder.layers.17.self_attn.q_proj.weight', 'vision_model.encoder.layers.5.layer_norm2.bias', 'vision_model.encoder.layers.21.mlp.fc1.weight', 'vision_model.encoder.layers.6.layer_norm2.bias', 'vision_model.encoder.layers.4.layer_norm2.bias', 'vision_model.encoder.layers.18.self_attn.out_proj.weight', 'vision_model.encoder.layers.9.self_attn.v_proj.bias', 'vision_model.encoder.layers.9.mlp.fc1.bias', 'vision_model.encoder.layers.12.self_attn.k_proj.bias', 'vision_model.encoder.layers.6.mlp.fc2.bias', 'vision_model.encoder.layers.7.self_attn.q_proj.bias', 'vision_model.encoder.layers.22.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.mlp.fc2.weight', 'vision_model.encoder.layers.12.self_attn.out_proj.bias', 'vision_model.encoder.layers.15.layer_norm2.weight', 'vision_model.encoder.layers.16.layer_norm1.bias', 'vision_model.encoder.layers.14.layer_norm1.bias', 'vision_model.encoder.layers.0.layer_norm1.bias', 'vision_model.encoder.layers.10.mlp.fc2.bias', 'vision_model.encoder.layers.8.mlp.fc2.bias', 'vision_model.encoder.layers.22.self_attn.v_proj.bias', 'vision_model.encoder.layers.12.mlp.fc1.bias', 'vision_model.encoder.layers.4.self_attn.out_proj.weight', 'vision_model.encoder.layers.16.mlp.fc2.bias', 'vision_model.encoder.layers.4.layer_norm1.bias', 'vision_model.encoder.layers.12.layer_norm2.weight', 'vision_model.encoder.layers.0.mlp.fc2.weight', 'vision_model.encoder.layers.23.mlp.fc2.bias', 'vision_model.encoder.layers.13.self_attn.out_proj.bias', 'vision_model.encoder.layers.0.self_attn.v_proj.weight', 'vision_model.encoder.layers.4.self_attn.k_proj.weight', 'vision_model.encoder.layers.22.self_attn.q_proj.bias', 'vision_model.encoder.layers.22.layer_norm1.weight', 'vision_model.encoder.layers.23.layer_norm1.bias', 'vision_model.encoder.layers.2.self_attn.k_proj.bias', 'vision_model.encoder.layers.2.layer_norm1.weight', 'vision_model.encoder.layers.5.self_attn.q_proj.bias', 'vision_model.encoder.layers.2.mlp.fc1.weight', 'vision_model.encoder.layers.7.layer_norm1.bias', 'vision_model.encoder.layers.14.mlp.fc2.weight', 'vision_model.encoder.layers.10.mlp.fc2.weight', 'vision_model.encoder.layers.23.self_attn.out_proj.bias', 'vision_model.encoder.layers.14.self_attn.k_proj.bias', 'vision_model.encoder.layers.6.mlp.fc2.weight', 'vision_model.encoder.layers.21.self_attn.q_proj.bias', 'vision_model.encoder.layers.12.mlp.fc2.weight', 'vision_model.encoder.layers.8.layer_norm2.weight', 'vision_model.encoder.layers.10.self_attn.k_proj.bias', 'vision_model.encoder.layers.18.layer_norm1.bias', 'vision_model.encoder.layers.2.self_attn.q_proj.weight', 'vision_model.encoder.layers.12.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.mlp.fc1.bias', 'vision_model.encoder.layers.15.self_attn.q_proj.weight', 'vision_model.encoder.layers.8.self_attn.k_proj.bias', 'vision_model.encoder.layers.19.self_attn.out_proj.weight', 'vision_model.encoder.layers.13.layer_norm2.bias', 'vision_model.encoder.layers.7.mlp.fc2.bias', 'vision_model.encoder.layers.4.self_attn.k_proj.bias', 'vision_model.encoder.layers.22.mlp.fc1.bias', 'vision_model.encoder.layers.5.self_attn.out_proj.bias', 'vision_model.encoder.layers.11.self_attn.k_proj.weight', 'vision_model.encoder.layers.7.mlp.fc1.weight', 'vision_model.encoder.layers.16.self_attn.v_proj.bias', 'vision_model.encoder.layers.3.layer_norm2.weight', 'vision_model.encoder.layers.3.self_attn.q_proj.weight', 'vision_model.encoder.layers.0.mlp.fc1.bias', 'vision_model.encoder.layers.19.mlp.fc1.weight', 'vision_model.encoder.layers.1.layer_norm2.bias', 'vision_model.encoder.layers.15.self_attn.out_proj.weight', 'vision_model.encoder.layers.13.self_attn.k_proj.bias', 'vision_model.encoder.layers.9.self_attn.q_proj.weight', 'vision_model.encoder.layers.1.mlp.fc2.bias', 'vision_model.encoder.layers.12.self_attn.v_proj.weight', 'vision_model.encoder.layers.20.self_attn.k_proj.bias', 'vision_model.encoder.layers.7.self_attn.q_proj.weight', 'vision_model.encoder.layers.3.self_attn.k_proj.weight', 'vision_model.encoder.layers.14.mlp.fc2.bias', 'vision_model.encoder.layers.7.layer_norm1.weight', 'vision_model.encoder.layers.1.self_attn.k_proj.bias',
'vision_model.encoder.layers.14.mlp.fc1.weight', 'vision_model.encoder.layers.22.self_attn.k_proj.bias', 'vision_model.encoder.layers.7.layer_norm2.weight', 'vision_model.encoder.layers.6.self_attn.v_proj.bias', 'vision_model.encoder.layers.20.mlp.fc2.weight', 'vision_model.encoder.layers.11.self_attn.v_proj.bias', 'vision_model.encoder.layers.5.layer_norm2.weight', 'vision_model.encoder.layers.19.self_attn.v_proj.weight', 'vision_model.encoder.layers.4.self_attn.q_proj.weight', 'vision_model.encoder.layers.12.self_attn.v_proj.bias', 'vision_model.encoder.layers.18.self_attn.q_proj.weight', 'vision_model.encoder.layers.18.self_attn.v_proj.weight', 'vision_model.encoder.layers.23.self_attn.v_proj.weight', 'vision_model.encoder.layers.23.mlp.fc1.weight', 'vision_model.encoder.layers.21.self_attn.v_proj.bias', 'vision_model.encoder.layers.23.layer_norm1.weight', 'vision_model.encoder.layers.18.self_attn.k_proj.bias', 'vision_model.encoder.layers.1.self_attn.out_proj.bias', 'vision_model.encoder.layers.1.self_attn.k_proj.weight', 'vision_model.encoder.layers.17.self_attn.k_proj.weight', 'vision_model.encoder.layers.0.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.v_proj.weight', 'vision_model.encoder.layers.1.self_attn.v_proj.weight', 'vision_model.encoder.layers.15.self_attn.out_proj.bias', 'vision_model.encoder.layers.16.mlp.fc1.weight', 'vision_model.encoder.layers.16.self_attn.out_proj.weight', 'vision_model.encoder.layers.11.self_attn.out_proj.bias', 'vision_model.encoder.layers.23.layer_norm2.bias', 'vision_model.encoder.layers.1.layer_norm1.weight', 'vision_model.encoder.layers.15.self_attn.q_proj.bias', 'vision_model.encoder.layers.17.self_attn.q_proj.bias', 'vision_model.encoder.layers.0.mlp.fc2.bias', 'vision_model.encoder.layers.3.self_attn.k_proj.bias', 'vision_model.encoder.layers.16.self_attn.q_proj.bias', 'vision_model.post_layernorm.weight', 'vision_model.encoder.layers.1.mlp.fc1.weight', 'vision_model.encoder.layers.15.layer_norm1.bias', 'vision_model.encoder.layers.21.layer_norm2.weight', 'vision_model.encoder.layers.6.self_attn.out_proj.weight', 'vision_model.embeddings.position_ids', 'vision_model.encoder.layers.18.self_attn.k_proj.weight', 'vision_model.encoder.layers.22.self_attn.q_proj.weight', 'vision_model.encoder.layers.22.layer_norm1.bias', 'vision_model.encoder.layers.23.mlp.fc1.bias', 'vision_model.encoder.layers.8.self_attn.v_proj.bias', 'vision_model.encoder.layers.6.self_attn.q_proj.bias', 'vision_model.encoder.layers.21.mlp.fc2.bias', 'vision_model.encoder.layers.4.mlp.fc2.bias', 'visual_projection.weight', 'vision_model.encoder.layers.11.self_attn.q_proj.weight', 'vision_model.encoder.layers.2.self_attn.v_proj.bias', 'vision_model.encoder.layers.2.self_attn.out_proj.weight', 'vision_model.embeddings.class_embedding', 'vision_model.encoder.layers.5.self_attn.v_proj.bias', 'vision_model.encoder.layers.2.self_attn.q_proj.bias', 'vision_model.encoder.layers.20.layer_norm1.weight', 'vision_model.encoder.layers.11.self_attn.q_proj.bias', 'vision_model.encoder.layers.13.self_attn.v_proj.weight', 'vision_model.encoder.layers.10.self_attn.v_proj.weight', 'vision_model.encoder.layers.18.self_attn.v_proj.bias', 'vision_model.encoder.layers.20.mlp.fc2.bias', 'vision_model.encoder.layers.9.self_attn.q_proj.bias', 'vision_model.encoder.layers.0.self_attn.k_proj.weight', 'vision_model.encoder.layers.6.layer_norm1.weight', 'vision_model.encoder.layers.6.layer_nSome weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel: ['vision_model.encoder.layers.12.mlp.fc1.bias', 'visual_projection.weight', 'vision_model.encoder.layers.20.self_attn.v_proj.weight', 'vision_model.encoder.layers.8.self_attn.q_proj.weight', 'vision_model.encoder.layers.2.layer_norm1.weight', 'vision_model.encoder.layers.21.self_attn.q_proj.bias', 'vision_model.encoder.layers.12.mlp.fc2.weight', 'vision_model.encoder.layers.20.self_attn.k_proj.weight', 'vision_model.encoder.layers.3.mlp.fc1.weight', 'vision_model.encoder.layers.0.self_attn.q_proj.bias', 'vision_model.encoder.layers.17.layer_norm2.weight', 'vision_model.encoder.layers.11.self_attn.q_proj.bias', 'vision_model.encoder.layers.21.layer_norm2.bias', 'vision_model.encoder.layers.12.self_attn.v_proj.bias', 'vision_model.embeddings.class_embedding', 'vision_model.encoder.layers.8.self_attn.k_proj.weight', 'vision_model.encoder.layers.22.self_attn.k_proj.weight', 'vision_model.encoder.layers.16.layer_norm1.bias', 'vision_model.encoder.layers.3.mlp.fc1.bias', 'vision_model.encoder.layers.5.self_attn.k_proj.bias', 'vision_model.encoder.layers.0.self_attn.v_proj.bias', 'vision_model.post_layernorm.bias', 'vision_model.encoder.layers.18.self_attn.out_proj.bias', 'vision_model.encoder.layers.11.self_attn.v_proj.weight', 'vision_model.encoder.layers.13.layer_norm1.weight', 'vision_model.embeddings.position_ids', 'vision_model.encoder.layers.9.mlp.fc2.weight', 'vision_model.encoder.layers.21.mlp.fc1.bias', 'vision_model.encoder.layers.3.self_attn.k_proj.bias', 'vision_model.encoder.layers.3.self_attn.v_proj.bias', 'vision_model.encoder.layers.6.self_attn.v_proj.weight', 'vision_model.encoder.layers.4.mlp.fc1.bias', 'vision_model.encoder.layers.9.self_attn.k_proj.weight', 'vision_model.encoder.layers.6.self_attn.k_proj.bias', 'vision_model.encoder.layers.3.layer_norm1.weight', 'vision_model.encoder.layers.8.self_attn.v_proj.bias', 'vision_model.encoder.layers.5.layer_norm2.weight', 'vision_model.encoder.layers.18.self_attn.q_proj.bias', 'vision_model.encoder.layers.10.mlp.fc2.bias', 'vision_model.encoder.layers.6.mlp.fc1.weight', 'vision_model.encoder.layers.16.self_attn.out_proj.bias', 'vision_model.encoder.layers.6.layer_norm2.bias', 'vision_model.encoder.layers.23.layer_norm1.bias', 'vision_model.encoder.layers.15.self_attn.q_proj.bias', 'vision_model.encoder.layers.2.mlp.fc1.bias', 'vision_model.encoder.layers.7.layer_norm1.weight', 'vision_model.encoder.layers.16.mlp.fc1.bias', 'vision_model.encoder.layers.6.layer_norm2.weight', 'vision_model.encoder.layers.14.self_attn.out_proj.bias', 'vision_model.post_layernorm.weight', 'vision_model.encoder.layers.13.self_attn.v_proj.weight', 'vision_model.encoder.layers.14.self_attn.v_proj.bias', 'vision_model.encoder.layers.14.self_attn.out_proj.weight', 'vision_model.encoder.layers.12.layer_norm2.weight', 'vision_model.encoder.layers.21.layer_norm2.weight', 'vision_model.encoder.layers.4.self_attn.out_proj.bias', 'vision_model.encoder.layers.16.mlp.fc2.bias', 'vision_model.encoder.layers.13.self_attn.out_proj.weight', 'vision_model.encoder.layers.15.self_attn.k_proj.bias', 'vision_model.encoder.layers.20.self_attn.q_proj.weight', 'vision_model.encoder.layers.15.self_attn.out_proj.weight', 'vision_model.encoder.layers.18.self_attn.q_proj.weight', 'vision_model.encoder.layers.3.self_attn.out_proj.bias', 'vision_model.encoder.layers.4.layer_norm1.weight', 'vision_model.encoder.layers.8.mlp.fc1.weight', 'vision_model.encoder.layers.14.layer_norm1.weight', 'vision_model.encoder.layers.19.self_attn.v_proj.bias', 'vision_model.encoder.layers.22.layer_norm1.weight', 'vision_model.encoder.layers.20.mlp.fc1.weight', 'vision_model.encoder.layers.1.layer_norm2.weight', 'vision_model.encoder.layers.15.mlp.fc2.bias', 'vision_model.encoder.layers.20.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.self_attn.v_proj.bias', 'vision_model.encoder.layers.22.mlp.fc1.bias', 'vision_model.encoder.layers.22.layer_norm2.bias', 'vision_model.encoder.layers.11.self_attn.k_proj.weight', 'vision_model.encoder.layers.0.layer_norm1.bias', 'vision_model.encoder.layers.22.mlp.fc2.weight', 'vision_model.encoder.layers.2.layer_norm2.weight', 'vision_model.encoder.layers.5.self_attn.out_proj.bias', 'vision_model.encoder.layers.14.layer_norm1.bias', 'vision_model.encoder.layers.0.mlp.fc2.weight', 'vision_model.encoder.layers.16.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.out_proj.weight', 'vision_model.encoder.layers.3.layer_norm2.weight', 'vision_model.encoder.layers.1.layer_norm2.bias', 'vision_model.encoder.layers.13.layer_norm2.bias', 'vision_model.encoder.layers.19.mlp.fc2.weight', 'vision_model.encoder.layers.19.self_attn.out_proj.weight', 'vision_model.encoder.layers.19.layer_norm2.bias', 'vision_model.encoder.layers.2.layer_norm2.bias', 'vision_model.encoder.layers.15.layer_norm1.bias', 'vision_model.encoder.layers.23.self_attn.q_proj.bias', 'vision_model.encoder.layers.3.mlp.fc2.weight', 'vision_model.encoder.layers.3.layer_norm2.bias', 'vision_model.encoder.layers.12.self_attn.out_proj.weight', 'vision_model.encoder.layers.0.self_attn.out_proj.weight', 'vision_model.encoder.layers.1.mlp.fc1.weight', 'vision_model.encoder.layers.0.self_attn.q_proj.weight', 'vision_model.encoder.layers.14.self_attn.q_proj.bias', 'vision_model.encoder.layers.8.layer_norm2.bias', 'vision_model.encoder.layers.8.mlp.fc1.bias', 'vision_model.encoder.layers.12.layer_norm1.bias', 'vision_model.encoder.layers.3.self_attn.out_proj.weight', 'vision_model.encoder.layers.16.self_attn.k_proj.bias', 'vision_model.encoder.layers.18.self_attn.v_proj.weight', 'vision_model.encoder.layers.2.self_attn.v_proj.weight', 'vision_model.encoder.layers.16.self_attn.out_proj.weight', 'vision_model.encoder.layers.19.self_attn.k_proj.bias', 'vision_model.encoder.layers.10.self_attn.k_proj.weight', 'vision_model.encoder.layers.19.mlp.fc2.bias', 'vision_model.encoder.layers.17.layer_norm1.weight', 'vision_model.encoder.layers.9.mlp.fc2.bias', 'vision_model.encoder.layers.23.self_attn.out_proj.bias', 'vision_model.encoder.layers.2.mlp.fc1.weight', 'vision_model.encoder.layers.17.self_attn.out_proj.bias', 'vision_model.encoder.layers.14.self_attn.v_proj.weight', 'vision_model.encoder.layers.19.layer_norm1.bias', 'vision_model.encoder.layers.22.layer_norm2.weight', 'vision_model.encoder.layers.2.self_attn.q_proj.bias', 'vision_model.encoder.layers.5.self_attn.v_proj.bias', 'vision_model.encoder.layers.5.self_attn.v_proj.weight', 'vision_model.encoder.layers.3.self_attn.q_proj.bias', 'vision_model.encoder.layers.6.layer_norm1.bias', 'vision_model.encoder.layers.17.self_attn.out_proj.weight', 'vision_model.encoder.layers.8.mlp.fc2.bias', 'vision_model.encoder.layers.3.layer_norm1.bias', 'vision_model.encoder.layers.20.layer_norm1.weight', 'vision_model.encoder.layers.1.mlp.fc2.bias', 'vision_model.encoder.layers.2.self_attn.q_proj.weight', 'vision_model.encoder.layers.5.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.self_attn.q_proj.weight', 'vision_model.encoder.layers.10.mlp.fc1.weight', 'vision_model.encoder.layers.7.mlp.fc2.weight', 'vision_model.embeddings.position_embedding.weight', 'text_projection.weight', 'vision_model.encoder.layers.15.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.v_proj.weight', 'vision_model.encoder.layers.10.self_attn.out_proj.weight', 'vision_model.encoder.layers.7.self_attn.q_proj.weight', 'vision_model.encoder.layers.2.mlp.fc2.bias', 'vision_model.encoder.layers.5.self_attn.q_proj.weight', 'vision_model.encoder.layers.14.self_attn.k_proj.bias', 'vision_model.encoder.layers.1.self_attn.v_proj.bias', 'vision_model.encoder.layers.23.self_attn.v_proj.weight', 'vision_model.encoder.layers.6.layer_norm1.weight', 'vision_model.encoder.layers.18.mlp.fc1.weight', 'vision_model.encoder.layers.22.self_attn.v_proj.weight', 'vision_model.encoder.layers.9.self_attn.v_proj.bias', 'vision_model.encoder.layers.7.self_attn.v_proj.bias', 'vision_model.encoder.layers.19.self_attn.out_proj.bias', 'vision_model.encoder.layers.6.mlp.fc2.bias', 'vision_model.encoder.layers.18.layer_norm1.weight', 'vision_model.encoder.layers.21.self_attn.k_proj.bias', 'vision_model.encoder.layers.19.self_attn.k_proj.weight', 'vision_model.encoder.layers.11.self_attn.v_proj.bias', 'vision_model.encoder.layers.4.layer_norm2.bias', 'vision_model.encoder.layers.10.self_attn.k_proj.bias', 'vision_model.encoder.layers.10.self_attn.v_proj.weight', 'vision_model.encoder.layers.22.layer_norm1.bias', 'vision_model.encoder.layers.6.mlp.fc1.bias', 'vision_model.encoder.layers.7.self_attn.k_proj.bias', 'vision_model.encoder.layers.11.self_attn.k_proj.bias', 'vision_model.encoder.layers.23.mlp.fc1.weight', 'vision_model.encoder.layers.17.mlp.fc2.weight', 'vision_model.encoder.layers.0.layer_norm1.weight', 'vision_model.encoder.layers.11.self_attn.q_proj.weight', 'vision_model.encoder.layers.17.mlp.fc2.bias', 'vision_model.encoder.layers.4.self_attn.out_proj.weight', 'vision_model.encoder.layers.20.mlp.fc2.weight', 'vision_model.encoder.layers.23.self_attn.v_proj.bias', 'vision_model.encoder.layers.1.mlp.fc2.weight', 'vision_model.encoder.layers.9.mlp.fc1.weight', 'vision_model.encoder.layers.1.self_attn.k_proj.weight', 'vision_model.encoder.layers.18.self_attn.v_proj.bias', 'vision_model.encoder.layers.15.layer_norm1.weight', 'vision_model.encoder.layers.6.self_attn.k_proj.weight', 'vision_model.encoder.layers.9.layer_norm2.bias', 'vision_model.encoder.layers.16.self_attn.v_proj.bias', 'vision_model.encoder.layers.9.self_attn.k_proj.bias', 'vision_model.encoder.layers.22.self_attn.q_proj.weight', 'vision_model.encoder.layers.17.self_attn.k_proj.weight', 'vision_model.encoder.layers.5.mlp.fc1.bias', 'vision_model.encoder.layers.14.mlp.fc2.bias', 'vision_model.encoder.layers.2.layer_norm1.bias', 'vision_model.encoder.layers.9.self_attn.out_proj.bias', 'vision_model.encoder.layers.7.mlp.fc1.weight', 'vision_model.encoder.layers.23.self_attn.out_proj.weight', 'vision_model.encoder.layers.7.layer_norm2.weight', 'vision_model.encoder.layers.13.self_attn.out_proj.bias', 'vision_model.encoder.layers.15.mlp.fc2.weight', 'vision_model.encoder.layers.18.layer_norm2.weight', 'vision_model.encoder.layers.15.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.layer_norm1.bias', 'vision_model.encoder.layers.18.self_attn.k_proj.bias', 'vision_model.encoder.layers.4.self_attn.k_proj.weight', 'vision_model.encoder.layers.4.layer_norm1.bias', 'vision_model.encoder.layers.3.self_attn.q_proj.weight', 'vision_model.encoder.layers.13.layer_norm2.weight', 'vision_model.encoder.layers.22.self_attn.v_proj.bias', 'vision_model.encoder.loading text encoder:loading text encoder:  <All keys matched successfully><All keys matched successfully>

Replace CrossAttention.forward to use xformers
Replace CrossAttention.forward to use xformers
[Dataset 0]
caching latents.
  0%|                                                                                | 0/132 [00:00<?, ?it/s[
Dataset 0]
caching latents.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 132/132 [00:24<00:00,  5.43it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 132/132 [00:24<00:00,  5.43it/s]
import network module: networks.lora
import network module: networks.lora
create LoRA for Text Encoder: 72 modules.
create LoRA for Text Encoder: 72 modules.
create LoRA for U-Net: 192 modules.
enable LoRA for text encoder
enable LoRA for U-Net
prepare optimizer, data loader etc.

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
CUDA SETUP: Loading binary H:\sd-scripts\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll...
create LoRA for U-Net: 192 modules.
enable LoRA for text encoder
enable LoRA for U-Net
prepare optimizer, data loader etc.

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
CUDA SETUP: Loading binary H:\sd-scripts\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll...
use 8-bit AdamW optimizer | {}use 8-bit AdamW optimizer | {}

running training / 学習開始
  num train images * repeats / 学習画像の数×繰り返し回数: 2640
  num reg images / 正則化画像の数: 0
  num batches per epoch / 1epochのバッチ数: 264
  num epochs / epoch数: 4
  batch size per device / バッチサイズ: 5
  gradient accumulation steps / 勾配を合計するステップ数 = 1
  total optimization steps / 学習ステップ数: 1056
steps:   0%|                                                                                | 0/1056 [00:00<?, ?it/s]epoch 1/4
steps:   6%|████████████                                                                                | 64/1056 [01:14<19:19,  1.17s/it, loss=0.0997]

Both cards seem to use ~20GB of VRAM each, and so then it would be effectively "faster" to run a training instance per GPU for different LoRAs than use both cards for 1 LoRA.
Now, I checked the 4 epochs with networks/check_lora_weights.py and got no nans, so that's fixed.

So the output is not corrupted!

thank you! Some duplication of output is normal.

I got it working so I opened a PR.
Regarding the log duplication, it looks like I'll have to change the whole code, so I might open another PR.

動作するようになったのでPRを開きました。
ログの複製に関してはコード全体を変更しないといけなさそうなのでまた別のPRを開くかもしれません。

I am using 4 GPUs on Windows 10. Although the training process runs smoothly, I encounter an error when attempting to use the safetensors file generated by the Checkpoint to try txt2image.
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000002C2817C10C0>]: Exception Traceback (most recent call last):   File "C:\Users\Administrator\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate     extra_network.activate(p, extra_network_args)   File "C:\Users\Administrator\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate     lora.load_loras(names, multipliers)   File "C:\Users\Administrator\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 151, in load_loras     lora = load_lora(name, lora_on_disk.filename)   File "C:\Users\Administrator\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 83, in load_lora     sd = sd_models.read_state_dict(filename)   File "C:\Users\Administrator\stable-diffusion-webui\modules\sd_models.py", line 217, in read_state_dict     pl_sd = safetensors.torch.load_file(checkpoint_file, device=device)   File "C:\Users\Administrator\stable-diffusion-webui\venv\lib\site-packages\safetensors\torch.py", line 98, in load_file     with safe_open(filename, framework="pt", device=device) as f: Exception: Error while deserializing header: InvalidHeaderDeserialization

@HayashiItsuki
Is it an error when using a model trained using my fork?

Sorry for the delay, I had to redo it 3 times, but all were successful and I was able to use the model without any problems. (I think the reason the first one didn't succeed may be due to my environment)

@itskey-rsa I still can't figure out how to run training on multiple video cards? Where to prescribe parameters and which ones?

@ddPn08 I still can't figure out how to run training on multiple video cards? Where to prescribe parameters and which ones?

Sorry for the old dump, but since SDXL came, multiGPU training has to be a bit viable now.
I have 2x4090, and using --multi_gpu seems to work, but I'm not sure if I should set the epoch to half the size to what a single GPU training will do? Assuming batch size 1,
For example, if I set epoch amount to 6 and 5000 total steps, it will effectively do 6 epochs with 5000 steps.
If I set epoch amount to 6, total steps to 5000 and multiGPU (2 GPUs), I get 12 epochs but the same 5000 total steps.
It is training 2 times per step (1 per GPU), per step itself?

20:29:46-877134 INFO     accelerate launch --gpu_ids="0,1" --multi_gpu --num_cpu_threads_per_process=4
"C:\AI\kohya_ss/sd-scripts/sdxl_train_network.py"  --bucket_no_upscale --bucket_reso_steps=64
--cache_latents --enable_bucket --min_bucket_reso=512 --max_bucket_reso=2048
--gradient_checkpointing --learning_rate="0.0002" --logging_dir="C:/AI/train/Restaurant/log"
--network_weights="C:/AI/sd-webui/models/Lora/rest1024_xllora_v4_1.safetensors"
--lr_scheduler="cosine_with_restarts" --lr_scheduler_num_cycles="1"
--max_data_loader_n_workers="0" --max_grad_norm="1" --resolution="1024,1024"
--max_train_steps="13320" --min_snr_gamma=5 --mixed_precision="fp16" --network_alpha="64"
--network_dim=128 --network_module=networks.lora --no_half_vae --noise_offset=0.0372
--optimizer_type="AdamW8bit" --output_dir="C:/AI/train/Restaurant/model"
--output_name="rest1024_xllora_v5"
--pretrained_model_name_or_path="C:/AI/sd-webui/models/Stable-diffusion/realvisxlV40_v40Bakedva
e.safetensors" --save_every_n_epochs="1" --save_model_as=safetensors --save_precision="fp16"
--text_encoder_lr=2e-05 --train_batch_size="1" --train_data_dir="C:/AI/train/Restaurant/image"
--unet_lr=0.0002 --vae="C:/AI/sd-webui/models/VAE/sdxl_vae.safetensors" --xformers
--sample_sampler=dpm_2 --sample_prompts="C:/AI/train/Restaurant/model\sample\prompt.txt"
--sample_every_n_epochs=1 --sample_every_n_steps=1000
[2024-03-27 20:29:54,278] torch.distributed.elastic.multiprocessing.redirects: [WARNING] NOTE: Redirects are currently not supported in Windows or MacOs.
[W socket.cpp:663] [c10d] The client socket has failed to connect to [api2.enscape3d.com]:29500 (system error: 10049 -  在其上下文中，该请求的地址无效。).
Traceback (most recent call last):
File "C:\AI\kohya_ss\sd-scripts\sdxl_train_network.py", line 184, in
trainer.train(args)
File "C:\AI\kohya_ss\sd-scripts\train_network.py", line 228, in train
accelerator = train_util.prepare_accelerator(args)
File "C:\AI\kohya_ss\sd-scripts\library\train_util.py", line 4031, in prepare_accelerator
INFO     0 reg images.                                                            train_util.py:1511
WARNING  no regularization images / 正則化画像が見つかりませんでした              train_util.py:1516
accelerator = Accelerator(
File "C:\AI\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 371, in init
self.state = AcceleratorState(
File "C:\AI\kohya_ss\venv\lib\site-packages\accelerate\state.py", line 758, in init
PartialState(cpu, **kwargs)
File "C:\AI\kohya_ss\venv\lib\site-packages\accelerate\state.py", line 217, in init
torch.distributed.init_process_group(backend=self.backend, **kwargs)
File "C:\AI\kohya_ss\venv\lib\site-packages\torch\distributed\c10d_logger.py", line 74, in wrapper
func_return = func(*args, **kwargs)
File "C:\AI\kohya_ss\venv\lib\site-packages\torch\distributed\distributed_c10d.py", line 1148, in init_process_group
default_pg, _ = _new_process_group_helper(
File "C:\AI\kohya_ss\venv\lib\site-packages\torch\distributed\distributed_c10d.py", line 1268, in _new_process_group_helper
raise RuntimeError("Distributed package doesn't have NCCL built in")
RuntimeError: Distributed package doesn't have NCCL built in
How to fix it?

multi gpu broken kaggle training anyone has any guess how to fix?
https://github.com/kohya-ss/sd-scripts/issues/1272


Deprecate the assertion-message rule

It's now moot with the introduction of the assertion-arguments rule.
What's the best-practise for deprecating rules with ESLint?

https://github.com/eslint/eslint/issues/5845

Just saw this: https://github.com/eslint/eslint/commit/49d3c1b20ee50bc661bb4c5ee74e0bc1bf2663d9, but ESLint doesn't actually use it nor pass it on to the formatter. I suggest we set this property and prefix (DEPRECATED)  to the rule messages. We should document this process.
Can be done for the next major version.


 I'll fix tonight.  Thanks for the report.

Hm tried the update.  But looks like there's another issue I wasn't aware of.  So still working on this.

Hm tried the update. But looks like there's another issue I wasn't aware of:

All addresses show "Wallet address not found" on the website, which likely means wallet_exists function in report_algo.py always returns false.

So I'll likely need a bit more time to figure this out.

Was just about to report the follow-up ;) But yeah, something's still off.

Hmm. I was going to use this and saw this issue and thought "hey I know some python I'll see if I can help out and maybe make a PR!"
Checked out the python algorand SDK and the "account_exists" function isn't documented xD
Checked the specific version from requirements.txt too, in case you were wondering. I am cry.

Checked out the python algorand SDK and the "account_exists" function isn't documented xD
edit: to be clear, "wallet_exists" calls indexer.account_exists and I was assuming the indexer is the one from the python algo sdk

We are not using the algo sdk for this, just the REST API, see src/staketaxcsv/algo/api/indexer.py

All addresses show "Wallet address not found" on the website, which likely means wallet_exists function in report_algo.py always returns false.

Seems to work locally, is this possibly staketax's IP being throttled or even banned by the node provider?

That is actually pretty good news because I really need to use it. I'll try it locally tonight. And if its a throttled/banned IP issue I guess there's no point in me looking into helping to fix it anyway. Crappy.

just had the wallet address not found issue.   Came here and saw this.   Running this locally works fine?

verified it works great locally.

I'll see if I can make a GUI app for this when I get some free time so people have an easier way to run this locally. We've had issues with node throttling before, especially during tax season.

Apologies for the delay in addressing this (wasn't feeling the greatest last couple of weeks).  Thanks for the work and patience everyone, especially @jumperavocado .
There was a deployment/environment bug that always existed but didn't come to fruition until I tried changing a setting ( specifically STAKETAX_ALGO_INDEXER_NODE).  Fix is deployed to stake.tax.


Restructured to work with the latest frame

Upgrades to the latest frame (<IP_ADDRESS>)
Moved CSS to my-app.css
Changed URL to myuw to point to prod in a new window

:+1:

:+1:


CreateDataAdapter and DuckDbDataAdapter

There no DuckDbDataAdapter in the lib, can implement it?

I have a database agnostic application that overrides System.Data.Common classes. It appears that several are missing like CommandBuilder and especially DataAdapter. Are there any plans to support these classes?
Thanks!

No plans currently.

I think it is not too hard. Using the SQLite library as a go-by (not copying) I confirmed several things and wrote DuckDBCommandBuilder and DuckDBDataAdapter. The latter appears to be trivial, but CommandBuilder is about 200 lines of verbose VB code (can be easily converted to C#). I am stuck because the DuckDBDataReader class has an overridden method called GetSchemaTable which does not really give the information that DBCommandBuilder needs to create insert, etc., commands. Specifically, the field lengths and key field info is not placed in the schema table. I think you'd have to parse the select command to get the table name and then run the Describe command to extract the information needed (or use some other API call).
I'm providing this information (and my work so far) in case you decide to try to finish the work yourself. I don't feel confident enough with the database to fix GetSchemaTable myself. It is great that you have lots of the standard ADO.Net classes working; just adding a couple more like CommandBuilder and DataAdapter would provide much better functionality. Most of the hard work is already done in the System.Data.Common classes (as you know).
One other thing: in DuckDB when you set a query parameter name, all other providers I use (SQLite, SQL Server, Oracle, MySQL, etc.) are forgiving if you include the SQL prefix when defining the parameters (e.g., you can pass $PARM1 as the parameter name) and it works OK, however DuckDB requires that the name exclude the prefix (e.g., PARM1). It was easy to fix my code, but you might want to consider this.
DuckDB.Net.Data.zip

Thanks for investigation @lloydchriswilson. Can you send a PR for the parameter prefix issue?


The ball position is out of control when collision happened

I give a ball an impulse as following code:
var effect = new CANNON.Vec3(0, ForceY, ForceZ);
var offset = new CANNON.Vec3(this.sphereBody.position.x, this.sphereBody.position.y, this.sphereBody.position.z);
this.sphereBody.applyImpulse(effect, offset);
When the ball collsion with some physical wall, the position changed to very strange value, please see the picture:

can somebody give me some hint, what happened on earth.
Thanks advance!

The picture show the ball position. You can see at last line, the ball position is changed to strange value suddenly within 0.03s.

It collides with Chuck Norris


Adapt to new sciencemesh service

Will merge this when we update the instructions on https://developer.sciencemesh.io/docs/technical-documentation/iop/iop-nextcloud-owncloud10-integrations/ to the Feb-2023 version of Reva

https://developer.sciencemesh.io/docs/technical-documentation/iop/iop-nextcloud-owncloud10-integrations/#list-of-known-issues-to-be-aware-of-including-blessed-versions-of-components updated now


Update Contributing.md, and add license headers

@vishh @aronchick @jlewi PTAL
Using guidelines from https://github.com/google/new-project

LGTM. Thanks for cleaning up @foxish


Enforce client grant_types and response_types

Fixes #95 #96
Breaking changes
Existing client registrations that did not follow the OIDC specification will now behave according to specification. Clients must now be explicitly registered with the sets of response_type and grant_type values that they will use.
For example, a client that intends to use the implicit flow must have token id_token or id_token token defined on its response_types claim and implicit on its grant_types claim in order to work.
Existing clients without a defined response_types claim will act as though their only supported response_type values are code. Clients using other response types must have their registration data updated using nv update client [id] [json] or otherwise.
Summary

New: Validates and enforces response_types and grant_types
New: Allows supported response types to be configured
New: Added tests
Fix: Response type sets are validated in any order, e.g. code id_token is equivalent to id_token code
Fix: Whitespace around response types and response modes no longer breaks validation
Fix: Running the server in development without NODE_ENV set no longer validates client metadata with production logic

Reviewed.


Create rails recipe-app

Create rails recipe-app

Create recipe-app with rails new
Set up database.yml
Add.github/workflows/linters.yml to our app
Add rubocop.yml to our app
Add stylelint and update .gitignore
Run rubocop and fix linters issues

Hi @MMGGYY66
Looks pretty good for me ;)

Thanks bro


Missing commit progress indicator in SCM panel

Issue Type: Bug
When creating a new commit, there used to be a progress indicator just below the commit message input that indicated that the commit is still in progress of being made. Now this indicator is missing.
We have pre-commit hooks that perform code transformations, which can take considerable time. The indicator was helpful to determine when the commit was actually finished. Now there is no immediate visual feedback when creating the commit, Which can lead to the commit being issued twice, which just leads to further issues.
 If so,  maybe related to #135215

@gjsjohnmurray Not that I'm aware of. I can only add that I noticed this on 2 workstations so far. So the behavior is at least consistent between my machines :)

Maybe try dragging it elsewhere, then using the option on its context menu to reset it to the default location (see details in #135215). Does this help?
Re consistency between your 2 workstations, do you use Settings Sync?

After the dragging exercise from earlier, I now also have this going on:

Back to normal after reloading window.


Fix http security issue with meltano.com

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/993
Originally created by @bencodezen on 2019-09-16 20:59:41

Currently http://www.meltano.com does not automatically redirect to https://www.meltano.com.
The suspected cause is that the SCP command in the CI pipeline is ignoring the .htaccess file in docs/src/.vuepress/public, so we need to make sure it doesn't get ignored.

View 5 previous comments from the original issue on GitLab


Focus wysiwyg tiptap on

Description
Add  focus on editor on content-wrapper click

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: Timothée Blanchard:x: TimBlanchardTimothée Blanchard seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Feature Request: Retrieve All Note Paths via Socket

Description
It would be very useful to implement a feature that retrieves the paths of all notes within the Obsidian vault and transmits them through the existing Unix socket connection. This would allow Python scripts to access the indexed notes directly from Obsidian’s cache, avoiding the need to manually scan the vault file system in Python. This feature can improve efficiency for plugin users who wish to manipulate or automate tasks involving multiple notes.
Proposed Solution

Add a new method to fetch the paths of all notes in the vault using this.app.vault.getFiles().
Implement a socket action (get_all_note_paths) to transmit these note paths to Python scripts.
Ensure that the response includes a JSON object with the list of note paths.

Use Case
This feature would be helpful for users needing a quick and efficient way to interact with multiple notes in Python without having to perform filesystem operations, which can slow down processes and add unnecessary complexity.

I’ll be implementing a simple but very useful feature soon that will allow Python scripts to retrieve all note paths in the vault via the Unix socket. This will avoid the need to manually scan the vault’s filesystem in Python, leveraging Obsidian’s built-in metadata cache instead. Expect this feature to be available shortly—it's straightforward and will improve efficiency for multi-note operations.
Stay tuned for updates!


Reverting to codecov v3

Seems like https://github.com/facebook/akd/pull/412 somehow is causing CI issues where v4 cannot be found. And I see on codecov that v4 is still in pre-release, so let's keep it at v3 for now...

Codecov Report
Patch coverage: 96.77% and project coverage change: +0.02% :tada:

Comparison is base (8453217) 87.74% compared to head (e4ced24) 87.76%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #413      +/-   ##
==========================================
+ Coverage   87.74%   87.76%   +0.02%
==========================================
  Files          39       39
  Lines        8771     8789      +18
==========================================
+ Hits         7696     7714      +18
  Misses       1075     1075

Files Changed
Coverage Δ

akd_core/src/ecvrf/ecvrf_impl.rs
91.92% <ø> (ø)

akd_core/src/verify/mod.rs
0.00% <ø> (ø)

akd/src/append_only_zks.rs
95.11% <92.30%> (+0.01%)
:arrow_up:

akd_core/src/types/node_label/mod.rs
94.39% <100.00%> (ø)

akd_core/src/verify/base.rs
91.62% <100.00%> (+0.56%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


split money isn't working on groups

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

go to '+' -> new group -> add members -> next
on groups page, go to '+' -> split bill
Type in money  -> next ,
on split money page , notice the split XXX amount button at the bottom is disabled

Expected Result:
split XXX amount button at the bottom is disabled
Actual Result:
split XXX amount button at the bottom of split amount should not be  disabled
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.34-1
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/43996225/029e0521-687a-47c3-8aec-a261c519f436
https://github.com/Expensify/App/assets/43996225/3f96546f-b46d-4582-9440-d22ac782b1eb
https://github.com/Expensify/App/assets/43996225/967deb8b-5002-494a-b4fc-eef260d86564
Expensify/Expensify Issue URL:
Issue reported by: @ashimsharma10
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1687783287041229
View all open jobs on GitHub

@aldo will this be fixed in https://github.com/Expensify/App/issues/21268? Can we close this GH?

Waiting for @aldo-expensify to confirm so  that we can decide if we close this.

Will DM Aldo, he probably missed my comments above.

I agree, this looks like a duplicate of https://github.com/Expensify/App/issues/21268... closing.


Read-only queries on the admin ad-hoc query endpoint

Add user with read-only access to all collections
https://docs.arangodb.com/3.4/Manual/Programs/WebInterface/Users.html

Created user admin_readonly

Closed by #41


Documentation translated in Filipino/Tagalog

Summary
Here is the translation of the documentation in Filipino/Tagalog (Philippines)
Related Issues
It's part of the issue #738
Checklist

[x] This pull request has a meaningful title and description.
[x] I have read and followed the Contributing guidelines.
[x] My code follows the conventions and coding style of this project.

Additional information
I tried testing this but even after updating these files, the 'Filipino/Tagalog' option isn't showing up in the choices:

docs/theme.config.tsx
docs/next.config.js

I'm not quite sure what seems to be the issue but I've finished translating the docs, so I created a PR and also to fix this.

Hi @redKath, thanks for contributing to the documentation of the Potter DB: Docs.

I tried testing this but even after updating these files, the 'Filipino/Tagalog' option isn't showing up in the choices:

docs/theme.config.tsx
docs/next.config.js
I'm not quite sure what seems to be the issue but I've finished translating the docs, so I created a PR and also to fix this.

When visiting the preview page I can select Filipino/Tagalog option, I can reproduce the issue you have, what exactly do you mean?

Also could you please pull the latest changes from the master branch and resolve the conflicts? I basically had to change from
next.config.mjs to next.config.js. Afterwards I will start reviewing your PR and give some feedback. Thanks!

@redKath please resolved the merge conflicts and then I can start / continue reviewing your pr.

any updates @redKath?


Enable Image Streaming by default

We use modules in https://github.com/terraform-google-modules/terraform-google-kubernetes-engine for creating GKE clusters, which does not expose a config for image streaming yet.
We should either propose a fix to allow clusters to enable image streaming with that module or remvoe our depdendency to that module.

Looks like our RAG solution actually enables image streaming already which is great :)
For other deployments we can still enable it at the node pool level by setting enable_gcfs=true

Might also explain flakes when pulling Ray images: https://github.com/GoogleCloudPlatform/ai-on-gke/issues/520


pollpopup removes a slash from code

Google returns a slash as part of the code: '4/...' and pollpopup removes the slash making it invalid, satellizer.js line 684.
Note the version on satellizer.herokuapp.com is different with
search.replace(//$/, '');
not
search.replace(/[/$]/, '');

Hey @olegabu, this changed was added 11 days ago as part of https://github.com/sahat/satellizer/pull/394. If I understand correctly it was a suggested fix for IE11. Is there any way we can satisfy both issues without hard coding browser version checks? @ghoken

Fixed on master branch. I will be releasing 0.11 shortly.


Fixed minor issue with Countries.php

I was getting an error noting incompatibility with barrelstrength\sproutforms\base\FormField. Plugin now works as expected.

Hi @vieko This was fixed on v1.0.2
Thank you!


Suggestion: Sniper missions

Forgive me if i'm intrusive. This is another mission suggestion:
The elimination of priority targets as enemy officers / colonies to lower enemy alerts. This would be a good simulation of the fact that the player cuts enemy lines of command making them unable to organize well for some time. I hope you like it.

Nice idea, already planned. :wink:
https://github.com/Wyqer/kp_liberation/issues/7


chore: Add missing/forgotten changesets

@alco wondering if we should add an optional check that shows yellow if the PR doesn't have a changeset.

@alco wondering if we should add an optional check that shows yellow if the PR doesn't have a changeset.

Could be done - https://github.com/changesets/changesets#integrating-with-ci

Cherry-picked each commit individually into main.


Is this a MIT license project?

It is obvious you are passionate about Blazor. Could you offer your work to Blazor community with MIT licenses?

https://github.com/raphadesa/BlazorInputMask
https://github.com/raphadesa/BlazorModal
https://github.com/raphadesa/BlazorAuth
https://github.com/raphadesa/FileManager

Hello, sure, all my git content is under MIT license... Regards, Raphaël

@raphadesa I urge you to state clearly and in a standard way they are MIT in all your projects. This helps MORE people do recommend your projects to others as they are MIT licenses

Hi George, thanks a lot for letting me know, as I wasn't aware of this. I'll update all my projects to MIT as soon as I'm back from vacation ;)
Regards,
Raphael

Hi, it's done ! Thanks a lot for letting me know, and good coding fun !
Regards,
Raphael


Update to latest dependency versions

Attention: targets the update-dependencies-for-ol-and-react branch!
Updates all dependencies to their latest versions, noteable changes:

update to ol 10.
update to eslint 9 (including several lint fixes).

In addition:

fix invalid devEngines block in the package.json
enable typechecks in the *.spec files

Please review @terrestris/devs.

Thanks for the reviews!


Change in CocoaPods 0.38 causing setup exception

CocoaPods 0.38 throws an exception with this project on pod install. Looks like this is due to a change in the post-install hook. Setting CocoaPods to 0.37.2 in the Gemfile is a simple workaround for anyone else running in to this issue.
$ pod install
Analyzing dependencies

CocoaPods 0.38.0.beta.1 is available.
To update use: `sudo gem install cocoapods --pre`
[!] This is a test version we'd love you to try.

For more information see http://blog.cocoapods.org
and the CHANGELOG for this version http://git.io/BaH8pQ.

Pre-downloading: `Specta` from `https://github.com/specta/specta.git`, tag `v0.3.0.beta1`
Downloading dependencies
Installing Expecta (0.3.1)
Installing HockeySDK (3.6.2)
Installing Mixpanel (2.7.2)
Installing OCMock (3.1.2)
Installing OHHTTPStubs (3.1.10)
Installing ReactiveCocoa (2.4.7)
Installing Specta (0.3.0.beta1)
Generating Pods project
2015-06-28 14:13:38.395 ruby[10572:1662232] [MT] DVTAssertions: ASSERTION FAILURE in /Library/Caches/com.apple.xbs/Sources/IDEFrameworks/IDEFrameworks-8123.33/IDEFoundation/Initialization/IDEInitialization.m:590
Details:  Assertion failed: _initializationCompletedSuccessfully
Function: BOOL IDEIsInitializedForUserInteraction()
Thread:   <NSThread: 0x10508f720>{number = 1, name = main}
Hints: None
Backtrace:
  0  0x0000000108a19047 -[DVTAssertionHandler handleFailureInFunction:fileName:lineNumber:assertionSignature:messageFormat:arguments:] (in DVTFoundation)
  1  0x0000000108a187d4 _DVTAssertionHandler (in DVTFoundation)
  2  0x0000000108a18a40 _DVTAssertionFailureHandler (in DVTFoundation)
  3  0x0000000108a189a2 _DVTAssertionFailureHandler (in DVTFoundation)
  4  0x000000010e4491a9 IDEIsInitializedForUserInteraction (in IDEFoundation)
  5  0x000000011235fa5d +[PBXProject projectWithFile:errorHandler:readOnly:] (in DevToolsCore)
  6  0x00000001123615e2 +[PBXProject projectWithFile:errorHandler:] (in DevToolsCore)
  7  0x00007fff9c770f14 ffi_call_unix64 (in libffi.dylib)
Abort trap: 6

Thanks for the workaround!


Fix 185 & 186

The fix for #185 removes a whole portion of code that looked at Code Mirror url nodes. Those are exclusively outgoing links and we shouldn't look at them at all.
For #186 I did a quick fix that does not apply any style to folders in the Bookmarks pane. Actually though, the current solution for the bookmarks pane does not work at all; I'll create an issue for someone to do a proper fix.

Hey @HEmile, can we GTM?

@HEmile I need these fixes 😭
Can you please merge this?

Apologies for ignoring this. Can you describe your changes a bit more?

Poke @HEmile

Please check my comment here https://github.com/mdelobelle/obsidian_supercharged_links/pull/193#discussion_r1389319393

Oh, sorry. I didn't see that comment. (And still can't really find it except in the hover-preview) I'll find the comment and address it 😅

I literally can't find the comment. In multiple browsers it just leads me to this discussion which does not contain that comment for me. Can you copy-paste it here again for me, @HEmile? Sorry about that.

Yeah sorry, I cannot merge this yet. Some users prefer the  format for internal links. I can only merge your changes as long as there is no regression in the behaviour there. They shouldn't be ignored.

Poke @HEmile

Hey @HEmile, did you look at this already and I'm again not seeing your review or did you not have the time yet?

It's been another month. @HEmile any feedback?

I tested it, seems like it works so I'll merge! Thanks


Using wrong files for 'London St Pancras' on Thameslink Core

On the class 700 in real life London St Pancras on the Thameslink core uses the recording 'London St Pancras International' and uses the CRS code 'SPL'  instead of what you are using at the moment which is the high level platforms saying just 'london st Pancras' which uses CRS code 'STP'

The GTR and Southeastern CIS's announce St Pancras Low Level/High Speed as "London St Pancras International" in real life.
The EMR CIS announces St Pan High Level as "London St Pancras" in real life.
The system is simply attempting to model different systems as well as possible.
https://github.com/davwheat/rail-announcements/blob/bd7ac660fd46e7083fb41baa8905a51187e743ff/src/announcement-data/systems/stations/AmeyPhil.tsx#L3443-L3450

Fair enough

Shouldn't there be an option to choose whether you want the GTR 'London St Pancras International' or the EMR 'London St Pancras'.

Quite honestly, that's a big chunk of work to implement with how it's all set up at the moment, with rather little benefit.

Just wondering, is the GTR 'London St Pancras International' audio file on this website? I would like to use this myself

Can't you just add another button for it

He said it's a big chunk of work for little benefit, which is fair enough.

Does it really take a lot of work to add 1 file?

It's not about adding the file. All the files available are there, and all the files are already used as I mentioned earlier, depending on whether it's High Speed, Midland or Thameslink platforms.
There is no London St Pancras Intl file in the audio, though, to answer your question, hence the use of St Pancras Intl instead.

And I can't just "add another button for it", which is the main issue.

I found the file for the TL St Pancras Intl.

Because that's how many there are?

Whats the inlfection?

start/middle/end of a sentence

oh ok

I was referring to onboard the 700/707s. The files

does exist as STA_SPL and STB_SPL


Check Crawlera Error Header when checking bans

Checking only the status code is not enough to know if the response coming from crawlera is a ban or not. I have added extra checks to prevent false positives.

I like this, to be honest I thought that this was already in place. Since there are quite a lot of 503's that are not bans (https://doc.scrapinghub.com/crawlera.html#errors)
Thanks, could you please fix the broken tests so I can merge it and schedule a release.

@pablohoffman scrapy-crawlera doesn't do retries. It checks for bans for stats and for killing the spider should too many bans occur.
We have these 503's
client_conn_closed | 503 | Connection closed by client
no_proxies | 503 | No available proxies
banned | 503 | Proxy has been banned
serverbusy | 503 | Server busy: too many outstanding requests

Only the banned one is actually a ban. So I completely agree with changes proposed by @VMRuiz. Regardless of anything all 503's should be retried like it says on the Best Practices.

This is a fix for #39


[HTML5] Preloader does not reflect progress of a packed asset library

When using:
<library name="default" type="zip" />
to wrap individual assets like images into a single default.pak file, the preloader
(indirectly) does not take the size of the file into account. So if we have a packed library of e.g.
1MB and no other assets, the preloader bar jumps directly from 0 to 100% as soon as the file
finished loading.
This issue has been mentioned years ago in this thread: [Preloader Progress Doesn’t Update When Using “default” Library](https://community.openfl.org/t/preloader-progress-doesnt-update-when-using-default-library/10688/)
The findings by singmajesty did not get rid of the issue though. The problem is that
PackedAssetLibrary.hx does not monitor the loading of default.pak itself.
This pull request will take care of it.
By the way - HTML5 is surely not the only affected target but it's where I've noticed it.

Looks like this tackles the same problem as https://github.com/openfl/lime/pull/1542, right?
I missed the request there to rebase on the latest development branch, but if desired I can still do that. If this PR is gone with on the other hand, feel free to close that one. :)


Fix doc styles prop on MarketClusterer

relating to issue https://github.com/googlemaps/v3-utility-library/issues/582

@googlebot I signed it!


Reset Method Should Be Public

Hi
I'm using your ngx-image-viewer component. It works great except for one small issue. The reset method is private. I am managing a carousel of images and displaying the selected image with the ngx-image-viewer. If the user zooms in or out and then clicks on another image in the carousel, the new image is rendered, but at the zoom level set for the previous image. This is not what I need. If i use the component provided next/previous buttons to navigate the list of images, the component calls reset prior to rendering each new image. This is the behavior i need when i specify the new image index to display. I could get this behavior in my code if the reset() method were made public.
Anthony

This makes sense. I'll create the change, and publish it this evening

Thanks so much!!
Anthony

From: jpilfold<EMAIL_ADDRESS>Sent: Friday, September 14, 2018 12:35 AM
To: jpilfold/ngx-image-viewer
Cc: Anthony Arrigo; Author
Subject: Re: [jpilfold/ngx-image-viewer] Reset Method Should Be Public (#20)
This makes sense. I'll create the change, and publish it this evening
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/jpilfold/ngx-image-viewer/issues/20#issuecomment-421246301, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ApQASy3X--cVr4yDcm4PdhTq-13p_zghks5ua040gaJpZM4WoUQF.

Okay, I've just published a new version (1.0.11) to npm with this change in there, hope it helps. Let me know if there are any issues.